[FIXED] BeautifulSoup: how to scrape meta tag description content

Issue

i am trying to scrape the contents of a website's meta description.

example:

<meta name="description" content="This is the home page meta description.">

the output that i'm looking for is: "This is the home page meta description."

my code is:

raw_html = simple_get(companyUrl)
html = BeautifulSoup(raw_html, 'html.parser')
x = html.select('meta', {'name' : 'description'})  ## this line errors out

can someone point me in the right direction?

(also - is it my imagination, or are BeautifulSoup tutorials/documentation not up to the level of other languages/applications?)

Solution

You have to use a css selector like so:

x = html.select('meta[name="description"]')
print(x[0].attrs["content"])

Read more about css selectors here:

Answered By - Martin Gergov