[FIXED] Selenium find_element by CSS selector giving unexpected behavior

Issue

I'm trying to detect the presence(or lack thereof) of a node on a dynamic webpage and getting very weird results with driver.find_element located via a CSS_selector.

First I'm looking if the full page is loaded via this func:

def is_fs_displayed():
    is_fullscreen_loaded = driver.find_element(By.ID, 'liveresults-sports-immersive__league-fullpage').is_displayed()
    print("is_fs?", is_fullscreen_loaded)
    return is_fullscreen_loaded

This is called with wait_until(lambda d: is_fs_displayed()) and works as expected: selenium polls the webpage until it either detects the element or times out as per the wait config.

Here's where the problem begins, when I look for the 2nd element, the program outputs results that I can't explain.

def is_ld_displayed():
    print("Test")
    is_results_loaded = not driver.find_element(By.CSS_SELECTOR, '#liveresults-sports-immersive__updatable-league-matches.yf.yl').is_displayed()
    print("is_ld?", is_results_loaded)
    return is_results_loaded

And wait_until(lambda d: is_ld_displayed()) gives result along the lines of:

Test
Test
is_ld? False
Test
Test
Test
Test
Test
...

and then it times out.

The element I'm trying to catch with the CSS selector is an element that the .yl class is added dynamically when the page is loading.

I'm at a complete loss, why does the function fail to print out "is_ld?" the first time, prints it out the 2nd time, and then fails to print it out all the other times.

And more importantly, why is the function unable to poll for the element, display False when it's not found and display True when it is found, like in is_fs_displayed?

I also tried using Expected Conditions as a param for wait_until, but that didn't work immediately and because I couldn't figure out how to make it print its result I abandoned it for now.

Any help greatly appreciated, thanks.

Solution

The issue was two-fold: 1)wait.until hides the element doesn't exist exception and 2) I flipped the Boolean indicating the presence of the element.

1. When I attempted to get driver.find_element(By.CSS_SELECTOR, '#liveresults-sports-immersive__updatable-league-matches.yf.yl'), if the element wasn't present the function would throw a NoSuchElementException, however wait.until would hide it. I've mistakenly believed that instead the function would resolve to False, as it did with is_fs_displayed() function, but looking at it in hindsight the element that that function was looking for must have been present but not visible, as opposed to the element in is_ld_displayed().

2. Because wait.until will only break out of its polling loop when its argument resolves to True, in my case it would always run until timeout. The iteration would exit because of the exception when element wasn't present as per my point 1), and when it was present the iteration would return False.

I have since moved to a better way of detecting the completion of dynamic page update, but I hope this might be useful to someone also confused by this behavior.

Answered By - ALT