[FIXED] Extract Data from Div class used multiple times

Issue

I am trying to get the company location from this website:https://slashdot.org/software/p/monday.com/

I am able to get close with the following code, but I am unable to navigate there.

Code:

url = 'https://slashdot.org/software/p/monday.com/'
profile = requests.get(url)
soup = bs(profile.content, 'lxml')
location = soup.select_one('div:nth-of-type(4).field-row').text

I feel like this is getting me in the area, but I've been unable to navigate over to "United States." Can someone show me what I am doing wrong?

Desired Out:

United States

Thanks!

Solution

To get the desired data you can use soup-contains() method and put them into a dict to get both the key value pairs

import pandas as pd
from bs4 import BeautifulSoup 
import requests

url= 'https://slashdot.org/software/p/monday.com/'
req = requests.get(url)

soup = BeautifulSoup(req.text,'lxml')


d = {soup.select_one('.field-row div:-soup-contains("Headquarters")').text.replace(':',''):soup.select_one('.field-row div:-soup-contains("Headquarters") + div').text}
print(d)

Output:

{'Headquarters': 'United States'}

Answered By - F.Hoque