[FIXED] How to web scrape youtube transcripts with Beautifulsoup4 and Python 3

Issue

Here is my current code. I am not sure what I am doing wrong. Maybe I am not digging deep enough in the html and giving Beautifulsoup the right tags? At the moment, my code is returning me blanks.

from bs4 import BeautifulSoup
from urllib.request import urlopen
html = urlopen("https://www.youtube.com/watch?v=5_zrHZdhaBU")
soup = BeautifulSoup(html,'html.parser')
nameList = soup.findAll("div", {"id": "cp-2"})
for name in nameList:
    print(name.get_text())

Here is the code that I inspected. I'm trying to get Python to return back to me "but it was untucked"

<div id="cp-2" class="caption-line" data-time="7.54"><div class="caption-line-time">0:07</div><div class="caption-line-text">but it was untucked.</div></div>

***Edit

The code can be found by clicking on "more" next to the share button. Then you click on transcripts and you will see all the text there.

Solution

Oh yes, it's loaded via Ajax: open the page, then open Network tab, sort requests by start time (latest requests first), click CC button on Youtube.

You get api/timedtext request, the response is an XML. Here it the full url to the transcript:

https://www.youtube.com/api/timedtext?signature=1A03D323CBD455E9993B7AC447CA64764FA6FE75.59F4BD2D45A32E89FBF54B418EE2F763283A1007&asr_langs=fr%2Cja%2Cnl%2Ces%2Cru%2Cko%2Cit%2Cde%2Cpt%2Cen&key=yttt1&caps=asr&v=5_zrHZdhaBU&hl=en_US&expire=1480702409&sparams=asr_langs%2Ccaps%2Cv%2Cexpire&lang=en&fmt=srv3

I have no idea how this URL is generated, though. This requires invesigation of complex YouTube scripts, etc.

EDIT: This answer helped me. You can omit most of these parameters and just use this URL:

https://www.youtube.com/api/timedtext?&v=5_zrHZdhaBU&lang=en

Or this in general:

https://www.youtube.com/api/timedtext?&v={video_id}&lang={language_code}

Answered By - Andrey Moiseev