[FIXED] A script built upon the requests module produces gibberish content after issuing a get request with parameters

Issue

I've created a script in Python to get the readable response content from the map of a webpage using the requests module. However, When I run the script, all I get is unintelligible stuff.

import requests
from pprint import pprint

link = 'https://services5.arcgis.com/wjH30HL7rqC55zVw/arcgis/rest/services/Development_Variance_Permits/FeatureServer/0/query'

params = {
    'f': 'pbf',
    'cacheHint': 'true',
    'maxRecordCountFactor': 4,
    'resultOffset': 0,
    'resultRecordCount': 8000,
    'where': '1=1',
    'orderByFields': 'OBJECTID_1',
    'outFields': 'OBJECTID_1',
    'outSR': 102100,
    'spatialRel': 'esriSpatialRelIntersects',
}

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36',
    'referer': 'https://csrd.maps.arcgis.com/apps/instant/sidebar/index.html?appid=685e92e3dd764236847fc52d4c295fb1',
    'accept': '*/*',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9'
}

res = requests.get(link,params=params,headers=headers)
print(res.status_code)
print(res.content)

The content the script produces is like (truncated):

b'\x12\xe9N\n\xe6N\n\nOBJECTID_1\x12\x0e\n\nOBJECTID_1\x10\x01B\x07\x08\xd4\x9d\x06\x10\x91\x1eb(\x12\x12\t-C\x1c\xeb\xe26\x1a?\x11-C\x1c\xeb\xe26\x1a?\x1a\x12\t\x00\x00\x00@\x04\x1cs\xc1\x11\x00\x00\x00\xc0\x14\xd7|\xc1j\x18\n\nOBJECTID_1\x10\x06\x1a\x08OBJECTIDz\x14\n\x02(\x01\x12\x0e\x1a\x0c\xd0\x91\xbb\x94\xf6\x03\xe7\xcd\xd7\x96\xb9\x15z\x14\n\x02(\x02\x12\x0e\x1a\x0c\xd0\x91\xbb\x94\xf6\x03\xe7\xcd\xd7\x96\xb9\x15z\x14\n\x02(\x03\x12\x0e\x1a\x0c\xd0\x91\xbb\x94\xf6\x03\xe7\xcd\xd7\x96\xb9\x15z\x14\n\x02(\x04\x12\x0e\x1a\x0c\xd0\x91\xbb\x94\xf6\x03\xe7\xcd\xd7\x96\xb9\x15z\x14\n\x02

What can I do to make the content understandable?

Solution

You have specified output format in "f" parameter as pbf, so you get binary data. Specify it as Json to get Json result back (more about the format here - official doc)

from pprint import pprint

import requests

link = "https://services5.arcgis.com/wjH30HL7rqC55zVw/arcgis/rest/services/Development_Variance_Permits/FeatureServer/0/query"

params = {
    "f": "json",  # <-- use Json here to get result in Json format
    "cacheHint": "true",
    "maxRecordCountFactor": 4,
    "resultOffset": 0,
    "resultRecordCount": 8000,
    "where": "1=1",
    "orderByFields": "OBJECTID_1",
    "outFields": "OBJECTID_1",
    "outSR": 102100,
    "spatialRel": "esriSpatialRelIntersects",
}

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36",
    "referer": "https://csrd.maps.arcgis.com/apps/instant/sidebar/index.html?appid=685e92e3dd764236847fc52d4c295fb1",
}

res = requests.get(link, params=params, headers=headers)
print(res.status_code)
print(res.text)

Prints:

200
{"objectIdFieldName":"OBJECTID_1","uniqueIdField":{"name":"OBJECTID_1", ...

Answered By - Andrej Kesely