[FIXED] How to get text of td next to its previous sibling th with BeautifulSoup?

Issue

Using BeautifulSoup/Python, I like to get the td text, Med from the following screenshot, or the ETF Risk from this website.

I tried the following code, but it didn't work. What should be corrected?

Risk = soup.find('th', string="ETF Risk").find_sibling("td").text

Solution

You could also use use css selectors and pseudo class like :-soup-contains() to search by text values.

But why not using HTML structure here - There are id attributes available:

soup.select_one('#tooltip_etf_risk').parent.find_next_sibling('td').text

Example

from bs4 import BeautifulSoup
html = '''
<tr>
    <th class="alpha">ETF Risk 
        <div class="tooltip-wrapper"><button class="tt-trigger" aria-controls="tooltip_etf_risk" aria-expanded="false" tt-init="true"><span class="info-tooltip" aria-hidden="true"></span><span class="sr-only">More Info</span></button></div>
        <div class="tooltiptext hide" id="tooltip_etf_risk">
            <p>Zacks proprietary quantitative models divide each set of ETFs following a similar investment strategy (style box/industry/asset class) into three risk categories- High, Medium, and Low. The aim of our models is to select the best ETFs within each risk category, so that investors can pick an ETF that matches their particular risk preference in order to better achieve their investment goals.
            </p>
        </div>
    </th>
    <td><span class="rank_chip rankrect_3 rankchip_medium">Med</span></td>
</tr>
'''
soup = BeautifulSoup(html)

soup.select_one('#tooltip_etf_risk').parent.find_next_sibling('td').text

Answered By - HedgeHog