[FIXED] Gather data with request on python

Issue

import requests
from pprint import pprint

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36',
}

params = (
    ('LeagueID', '00'),
    ('Season', '2017-18'),
    ('TeamID', '1610612757'),
)

data = requests.get('https://www.nba.com/stats/players/cut/',
                    headers=headers, params=params).json()
pprint(data)
columns=data['resultSets'][0]['headers']
for result_set in data['resultSets']:
    print("Result set", result_set['name'])
    for item in result_set['rowSet']:
        pprint(dict(zip(result_set['headers'], item)))

I try to run this code, but I systematically have this error:

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

My goal is to get the data from the table on the link basically. I succeeded with the API, but there aren't all the information I need for my analysis. This is why I prefer to do it by "Scraping" the nba.com site.

Solution

you're missing some of the headers parameters. Also, I don't think the team id query works right. You also need to use url = 'https://stats.nba.com/stats/synergyplaytypes'

import requests
from pprint import pprint

headers = {
'Accept': 'application/json, text/plain, */*',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'en-US,en;q=0.9',
'Connection': 'keep-alive',
'Host': 'stats.nba.com',
'Origin': 'https://www.nba.com',
'Referer': 'https://www.nba.com/',
'sec-ch-ua': '"Google Chrome";v="87", "\"Not;A\\Brand";v="99", "Chromium";v="87"',
'sec-ch-ua-mobile': '?1',
'Sec-Fetch-Dest': 'empty',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Site': 'same-site',
'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Mobile Safari/537.36',
'x-nba-stats-origin': 'stats',
'x-nba-stats-token': 'true'
}

params = {'LeagueID': '00',
'PerMode': 'PerGame',
'PlayType': 'Cut',
'PlayerOrTeam': 'P',
'SeasonType': 'Regular Season',
'SeasonYear': '2020-21',
'TypeGrouping': 'offensive'}

url = 'https://stats.nba.com/stats/synergyplaytypes'
data = requests.get(url,
                    headers=headers, params=params).json()
pprint(data)
columns=data['resultSets'][0]['headers']
for result_set in data['resultSets']:
    print("Result set", result_set['name'])
    for item in result_set['rowSet']:
        pprint(dict(zip(result_set['headers'], item)))

Output:

...
{'EFG_PCT': 0.5,
 'FGA': 1.4,
 'FGM': 0.7,
 'FGMX': 0.7,
 'FG_PCT': 0.5,
 'FT_POSS_PCT': 0.1,
 'GP': 7,
 'PERCENTILE': 0.244,
 'PLAYER_ID': 1630214,
 'PLAYER_NAME': 'Xavier Tillman',
 'PLAY_TYPE': 'Cut',
 'PLUSONE_POSS_PCT': 0.1,
 'POSS': 1.4,
 'POSS_PCT': 0.192,
 'PPP': 1.1,
 'PTS': 1.6,
 'SCORE_POSS_PCT': 0.5,
 'SEASON_ID': '22020',
 'SF_POSS_PCT': 0.1,
 'TEAM_ABBREVIATION': 'MEM',
 'TEAM_ID': 1610612763,
 'TEAM_NAME': 'Memphis Grizzlies',
 'TOV_POSS_PCT': 0.0,
 'TYPE_GROUPING': 'Offensive'}
{'EFG_PCT': 0.444,
 'FGA': 1.0,
 'FGM': 0.6,
 'FGMX': 0.6,
 'FG_PCT': 0.444,
 'FT_POSS_PCT': 0.273,
 'GP': 9,
 'PERCENTILE': 0.044,
 'PLAYER_ID': 203939,
 'PLAYER_NAME': 'Dwight Powell',
 'PLAY_TYPE': 'Cut',
 'PLUSONE_POSS_PCT': 0.091,
 'POSS': 1.2,
 'POSS_PCT': 0.234,
 'PPP': 0.909,
 'PTS': 1.1,
 'SCORE_POSS_PCT': 0.455,
 'SEASON_ID': '22020',
 'SF_POSS_PCT': 0.273,
 'TEAM_ABBREVIATION': 'DAL',
 'TEAM_ID': 1610612742,
 'TEAM_NAME': 'Dallas Mavericks',
 'TOV_POSS_PCT': 0.0,
 'TYPE_GROUPING': 'Offensive'}
{'EFG_PCT': 0.333,
 'FGA': 1.3,
 'FGM': 0.9,
 'FGMX': 0.9,
 'FG_PCT': 0.333,
 'FT_POSS_PCT': 0.091,
 'GP': 7,
 'PERCENTILE': 0.0,
 'PLAYER_ID': 1629060,
 'PLAYER_NAME': 'Rui Hachimura',
 'PLAY_TYPE': 'Cut',
 'PLUSONE_POSS_PCT': 0.0,
 'POSS': 1.6,
 'POSS_PCT': 0.126,
 'PPP': 0.636,
 'PTS': 1.0,
 'SCORE_POSS_PCT': 0.364,
 'SEASON_ID': '22020',
 'SF_POSS_PCT': 0.091,
 'TEAM_ABBREVIATION': 'WAS',
 'TEAM_ID': 1610612764,
 'TEAM_NAME': 'Washington Wizards',
 'TOV_POSS_PCT': 0.091,
 'TYPE_GROUPING': 'Offensive'}

You can find all that info by opeing your Dev Tools and Inspect. You want to go to Network -> XHR, then search through there to see which request gets the data (you may need to reload the page after you open Dev Tools)

Then change it to Headers and you'll see the url:

and scroll down for the other other headers and params used. You likely don't need ALL of them, but there seems to be a few that are definitely needed. you can play around and see which one or ones are needed for the nba api.

Answered By - chitown88