[FIXED] Use beautifulsoup to get a youtube video‘s information

Issue

I am new to python scriping, I am trying to use beautifulsoup to get a youtube video's information like title, description, number of view and number of likes. How should I be able to do so without using Youtube API but use beautifulsoup. Right now I click inspect and click on title of a youtube video,

<h1 class="title style-scope ytd-video-primary-info-renderer">
   <yt-formatted-string force-default-style="" class="style-scope ytd-video-primary-info-renderer">
      Tucker: Kamala Harris may end up running the country</yt-formatted-string></h1>

And my code is

# I also try use class as title style-scope ytd-video-primary-info-renderer 
for i in soup.findAll('h1',{'class':'style-scope ytd-video-primary-info-renderer'}):
    videoName.append(i)

But it did not get the video's title. Also this does not work on get video description and veido likes and views. Can some one help me to find how should I search those using beautifulsoup? Thanks a lot!!!

Solution

YouTube uses JavaScript, but requests doesn't support it. so we can use a library like Requests-HTML to scrape the page.

Install it using pip install requests-html.

For example, this script will get the title of the video:

from requests_html import HTMLSession
from bs4 import BeautifulSoup


video_url = "ENTER LINK"
# Initialize an HTML Session
session = HTMLSession()
# Get the html content
response = session.get(video_url)
# Execute JavaScript
response.html.render(sleep=3)

soup = BeautifulSoup(response.html.html, "lxml")

print("title:", soup.select_one('#container > h1').text)

Answered By - MendelG