[FIXED] Extract <td> Elements using BS4

Issue

I am trying to go through a website and extract some information using Chromedriver. The problem that I have when I use BeautifulSoup is that I can't find a way to extract table inside a class.

The way I am trying to extract the information looks like this:

results = soup.find_all("div", class_="widget widgetLarge fpPerfglissanteclassique")

Is there a way to change this line so that it will only return the Information in <td>...</td> that can be found inside the class?!

Thanks for your answers in advance!

Solution

Your results variable contains another BeautifulSoup object (ResultSet) which you can iterate though and call find and find_all on the individual result items.

Like this:

from bs4 import BeautifulSoup

html = """
<div class="widget widgetLarge fpPerfglissanteclassique">
    <td>item 1</td>
    <td>item 2</td>
    <td>item 3</td>
</div>
<div class="widget widgetLarge fpPerfglissanteclassique">
    <td>item 4</td>
    <td>item 5</td>
    <td>item 6</td>
</div>
"""

soup = BeautifulSoup(html, "html.parser")
results = soup.find_all("div", class_="widget widgetLarge fpPerfglissanteclassique")

for result in results:
    table_results = result.find_all("td")
    print(table_results)

Result:

[<td>item 1</td>, <td>item 2</td>, <td>item 3</td>]
[<td>item 4</td>, <td>item 5</td>, <td>item 6</td>]

Answered By - ptts