[FIXED] Python requests post file upload fails while Curl succeeds

Issue

I'm trying to port a working curl solution to plain python.

I'm uploading a file (pdf in this case) to a documented API endpoint: https://developers.lexoffice.io/docs/#vouchers-endpoint-upload-a-file-to-a-voucher

The example from the documentation in curl works flawlessly:

retStream = os.popen('curl https://api.lexoffice.io/v1/vouchers/' + cr.json()['bookkeepingVoucherId'] + '/files -X POST'
            + ' -H "Authorization: Bearer ' + Settings.LEXOFFICE_CONFIG['api-key'] + '" -H "Content-Type: multipart/form-data"'
            + ' -H "Accept: application/json" -F "file=@' + zfo.filename + '"')

Now the same in Python, it fails and gets an unspecific 500:

   def _it__(self, apiKey):
        self.__apiKey = apiKey
        self.__headers = {
            'Authorization': 'Bearer ' + self.__apiKey,
            'Content-Type': 'application/json',
            'Accept': 'application/json'
        }
[...]
    def uploadFileToAVoucher(self, voucherId, filename):
        heads = self.__headers.copy()
        heads['Content-Type'] = 'multipart/form-data'
        print(filename)
        files = {
            'file': (filename, open(filename, 'rb'), 'application/pdf')
        }
        return requests.post(LexofficeApi.API_URL + '/v1/vouchers/' + voucherId + '/files',
                            files=files, headers=heads)

The API-Key is identical (and other API calls work perfectly, the basic structure must be correct). I even dumped the curl outgoing http body with --trace-ascii and compared it to the python request. The exact same sequence.

I have no clue what Python does differently. Any ideas?

curl trace:

=> Send header, 315 bytes (0x13b)
0000: POST /v1/vouchers/<removed>/files HTT
0040: P/2
0045: Host: api.lexoffice.io
005d: user-agent: curl/7.68.0
0076: authorization: Bearer <removed>
00b2: accept: application/json
00cc: content-length: 35015
00e3: content-type: multipart/form-data; boundary=--------------------
0123: ----a52203785a14b388
0139: 
<= Recv SSL data, 5 bytes (0x5)
0000: ....@
== Info: Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
=> Send SSL data, 5 bytes (0x5)
0000: ....!
=> Send SSL data, 5 bytes (0x5)
0000: ...@.
=> Send SSL data, 5 bytes (0x5)
0000: ....!
=> Send SSL data, 5 bytes (0x5)
0000: ...@.
=> Send SSL data, 5 bytes (0x5)
0000: ....!
=> Send SSL data, 5 bytes (0x5)
0000: .....
=> Send data, 35015 bytes (0x88c7)
0000: --------------------------a52203785a14b388
002c: Content-Disposition: form-data; name="file"; filename="<rem
006c: moved>"
0090: Content-Type: application/pdf
00af: 
00b1: %PDF-1.4.%.....1 0 obj.<</Creator (Chromium)./Producer (Skia/PDF
00f1:  m93)./CreationDate (D:20211004001328+00'00')./ModDate (D:202110
0131: 04001328+00'00')>>.endobj.3 0 obj.<</ca 1./BM /Normal>>.endobj.5
0171:  0 obj.<</ca .9333./BM /Normal>>.endobj.8 0 obj.<</Filter /Flate
01b1: Decode./Length 6736>> stream.x..]]...q.._q..T.7E.0.k{}.....:H..E
01f1: .4n.....t..3..sL...^{..45_.g8.R..tX.y....'..l..d..........+3...
[...]

python request body:

b'--ae550894d8422832a377fdd24f109898\r\nContent-Disposition: form-data; name="file"; filename="<removed>"\r\nContent-Type: application/pdf\r\n\r\n%PDF-1.4\n%\xd3\xeb\xe9\xe1\n1 0 obj\n<</Creator (Chromium)\n/Producer (Skia/PDF m93)\n/CreationDate (D:20211004002626+00\'00\')\n/ModDate (D:20211004002626+00\'00\')>>\nendobj\n3 0 obj\n<</ca 1\n/BM /Normal>>\nendobj\n5 0 obj\n<</ca .9333\n/BM /Normal>>\nendobj\n8 0 obj\n<</Filter /FlateDecode\n/Length 6736>> stream\nx\x9c\xed]]\xc

Looks very much the same to me.

Solution

requests.post doesn't set the Content-Length, it's always 0. The LexOffice-API seems to be picky about this.

A workaround with pycurl works. Here's a generic version of the question: python requests.post header content-length always 0

Answered By - MPW