[FIXED] perform download via download button when request_url in browser-inspect does not work

Issue

I have almost the question already posted and answered here:

Perform Download via download button in Python

I also followed the instructions, in the answer of the above link.

In my case I want to download the data from the following page:

https://www.smard.de/home/downloadcenter/download-marktdaten#!?downloadAttributes=%7B%22selectedCategory%22:1,%22selectedSubCategory%22:1,%22selectedRegion%22:%22DE%22,%22from%22:1658872800000,%22to%22:1659563999999,%22selectedFileType%22:%22CSV%22%7D

The download-button is the one called "Datei herunterladen" in the lower right corner.

I went into the inspect mode downloaded the file and got the following output in the inspection

But the resulting

Request URL: 'https://www.smard.de/nip-download-manager/nip/download/market-data' 

does not help getting the csv-file. Opened in the browser I get: 'The requested URL was rejected.' On the other hand, it does not even contain the parameters anymore, so it can't be the right download url.

May anyone help to automate this download?

edit Now I also tried

url = 'https://www.smard.de/nip-download-manager/nip/download/market-data'
json_body = {'format': "CSV",
'language': "de",
'moduleIds': [1001224, 1004066, 1004067, 1004068, 1001223, 1004069, 1004071, 1004070, 1001226, 1001228, 1001227,1001225],
'region': "DE",
'timestamp_from': 1659304800000,
'timestamp_to': 1659391199999,
'type': "discrete"}

x = requests.post(url, json = json_body)
x.content

> b'Exception when parsing incoming request to JSON object.'

So how do I get the csv-file based on this method?

Solution

1. The json_body of the request-payload was not correct.
2. And I found an easy way to get the correct one with less effort and less probability to make it wrong.

• In the inspect mode after the file is downloaded:
• Click on the "Raw"-Button in the upper right corner of the "request"-tab, than you can get the correct payload per copy and paste.

The final solution is:

url = 'https://www.smard.de/nip-download-manager/nip/download/market-data'
payload = {"request_form":[{"format":"CSV","moduleIds":[1001224,1004066,1004067,1004068,1001223,1004069,1004071,1004070,1001226,1001228,1001227,1001225],"region":"DE","timestamp_from":1658872800000,"timestamp_to":1659563999999,"type":"discrete","language":"de"}]}
x = requests.post(url, json = payload)
with open(f'energy_ger.csv', 'wb') as f:
    f.write(x.content)

Answered By - Thomas R