[FIXED] Using Selenium+python to extract HTML code from a list of links

Issue

I'm trying to get several page sources from a list of links. My idea is to use a webdriver to open a link, save the page source in a variable and then go back to continue with the next link. But for some reason python tells me that an element is not attached to the page document. Is there any solution for this? Thanks in advance!

browserFut = webdriver.Chrome(PATH)
browserFut.get(link)

page_sources = []
links = browserFut.find_element_by_xpath('//div[@class="dml-page-loader dml-page-loader--pb"]').find_elements_by_tag_name('a')
for link in links:
    link.click()
    page_sources += [browserFut.page_source]
    browserFut.back()
    time.sleep(1)

Solution

In second loop, elements becomes stale so you have to define them again.

j = 0 
links = browserFut.find_element_by_xpath('//div[@class="dml-page-loader dml-page-loader--pb"]').find_elements_by_tag_name('a')
for link in range(len(links)):
    elements = browserFut.find_element_by_xpath('//div[@class="dml-page-loader dml-page-loader--pb"]').find_elements_by_tag_name('a')
    elements[j].click()
    page_sources += [browserFut.page_source]
    j = j +1 
    browserFut.back()
    time.sleep(5)

Answered By - cruisepandey