[FIXED] Extract string from tag with BeautifulSoup

Issue

I am trying to extract from below table. I cut it after the second <td>, with six more to follow. All eight strings need to be extracted, e.g. in the example below I would want the values 61.5, 56.43, etc.

The code below only gives me the first value, 61.5. How can I get the remaining values?

 soup.find("div", {"class":"value"}).text


<td class="flow">
    <div class="heading" style="min-height: 63px;">Dornum</div>
    <div class="data"><div class="value">61.5</div> MSm<sup>3</sup>/d</div>
</td>
<td class="flow">
    <div class="heading" style="min-height: 63px;">Emden EMS</div>
    <div class="data"><div class="value">56.43</div> MSm<sup>3</sup>/d</div>
</td>

Solution

Use soup.find_all() to obtain a list of matching elements, then grab the text attribute for each element:

from bs4 import BeautifulSoup

html = '''<td class="flow">
    <div class="heading" style="min-height: 63px;">Dornum</div>
    <div class="data"><div class="value">61.5</div> MSm<sup>3</sup>/d</div>
</td>
<td class="flow">
    <div class="heading" style="min-height: 63px;">Emden EMS</div>
    <div class="data"><div class="value">56.43</div> MSm<sup>3</sup>/d</div>
</td>'''

soup = BeautifulSoup(html)

data = [element.text for element in soup.find_all("div", "value")]

>>> data
[u'61.5', u'56.43']

Or, if you want them as floats:

data = [float(element.text) for element in soup.find_all("div", "value")]
>>> data
[61.5, 56.43]

Answered By - mhawke