Issue
Suppose I have a list of keywords, i.e. ["apple", "dog", "cat"], and some HTML. I'd like to return all the elements that contain one of these keywords as its immediate descendant. How would I go about doing this?
I've tried using soup.find_all(text=keywords), but that yields nothing.
from bs4 import BeautifulSoup
source = """
<html>
<p>I like apples</p>
<p>I don't want to match this</p>
<div>Dogs are cool. I don't match this either.</div>
<div>I have a cat.</div>
</html>
"""
soup = BeautifulSoup(source, "html.parser")
keywords = ["apple", "dog", "cat"]
Solution
BeautifulSoup supports regex for text searching, so you can utilize that with the IGNORECASE flag (since your keyword is dog, and your element contains Dogs)
import re
from bs4 import BeautifulSoup
source = """
<html>
<p>I like apples</p>
<p>I don't want to match this</p>
<div>Dogs are cool. I don't match this either.</div>
<div>I have a cat.</div>
</html>
"""
soup = BeautifulSoup(source, "html.parser")
keywords = ["apple", "dog", "cat"]
print(soup.find_all(text=re.compile("|".join(keywords), flags=re.IGNORECASE)))
>>>> ['I like apples', "Dogs are cool. I don't match this either.", 'I have a cat.']
As a note, you say "immediate descendant" and the element with dogs has "I don't match this either". Due to how your HTML is formatted, it will pick this up. If this line was something like <div>Dogs are cool. <div>I don't match this either.</div></div>, then the output would be
['I like apples', 'Dogs are cool. ', 'I have a cat.']
Answered By - Wondercricket
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.