[FIXED] I can't spot some of the elements in the site's source code

Issue

I was trying to scrape this website to get the player data.

https://mystics.wnba.com/roster/

I viewed the code using 'Inspect' but the main table isn't in the source code. For example, this is the code for the first player's name:

<div class="content-table__player-name">
                            <a ng-href="https://www.wnba.com/player/ariel-atkins/" target="_self" href="https://www.wnba.com/player/ariel-atkins/">Ariel Atkins</a>
                        </div>

I can't find this piece of code (or any code for the player data) in the page source. I searched for most of the table's divs in the source code but I couldn't find any of them.

Solution

As the tag contains scrapy. So, here is a solution using scrapy.

import scrapy
import json

class Test(scrapy.Spider):
    name = 'test'
    start_urls = ['https://data.wnba.com/data/5s/v2015/json/mobile_teams/wnba/2021/teams/mystics_roster.json']

    def parse(self, response):
        data = json.loads(response.body)
        data = data.get('t').get('pl')

        for player in data:
            print(player.get('fn'),player.get('ln'))

Answered By - Shivam