[FIXED] Selenium/Python - how to expand the tree completely?

Issue

I try to completely expand a tree to see all children and to be able to scrap the tree in the end. But my code just opens the first level. Even if just repeat the task und rerun driver.find_elements then it does not open all levels of the tree and I get an error:

Traceback (most recent call last):
  File "/Users/hannes/Desktop/pubmed-scrapper/seleniummeshtree.py", line 18, in <module>
    plus.click()
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
    return self._parent.execute(command, params)
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 428, in execute
    self.error_handler.check_response(response)
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
  (Session info: chrome=106.0.5249.61)
Stacktrace:
0   chromedriver                        0x0000000104935a90 chromedriver + 3889808
1   chromedriver                        0x00000001048c4b54 chromedriver + 3427156
2   chromedriver                        0x00000001045b6108 chromedriver + 221448
3   chromedriver                        0x00000001045e83f0 chromedriver + 426992
4   chromedriver                        0x00000001045de0cc chromedriver + 385228
5   chromedriver                        0x00000001045dda10 chromedriver + 383504
6   chromedriver                        0x0000000104610944 chromedriver + 592196
7   chromedriver                        0x00000001045dc54c chromedriver + 378188
8   chromedriver                        0x00000001049092f8 chromedriver + 3707640
9   chromedriver                        0x000000010490cea8 chromedriver + 3722920
10  chromedriver                        0x00000001049133d8 chromedriver + 3748824
11  chromedriver                        0x000000010490d9a8 chromedriver + 3725736
12  chromedriver                        0x00000001048e8de8 chromedriver + 3575272
13  chromedriver                        0x0000000104927d1c chromedriver + 3833116
14  chromedriver                        0x0000000104927e84 chromedriver + 3833476
15  chromedriver                        0x000000010493c210 chromedriver + 3916304
16  libsystem_pthread.dylib             0x000000019842826c _pthread_start + 148
17  libsystem_pthread.dylib             0x000000019842308c thread_start + 8

Below you can find my code. What am I doing wrong?

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

# load selenium safari webdriver
driver = webdriver.Chrome()
driver.get("https://meshb.nlm.nih.gov/treeView")

# click on the plus-buttons to expand the tree
pluses = driver.find_elements(By.XPATH, "//i[@onclick='openTree(this)']")
for plus in pluses:
    plus.click()
    time.sleep(1)

# click on the plus-buttons to expand the tree
pluses = driver.find_elements(By.XPATH, "//i[@onclick='openTree(this)']")
for plus in pluses:
    plus.click()
    time.sleep(1)

driver.close()

Solution

You can resolve this issue with the approach:
Perform singe click while there is still not expanded element presented.
You will need to distinguish between expanded and still not expanded + elements - your locator does not check this.
The following code works for me:
It will run long time since the tree is big and deep.

import time

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = 'https://meshb.nlm.nih.gov/treeView'
driver.get(url)
time.sleep(2)
while driver.find_elements(By.XPATH, "//i[@onclick='openTree(this)' and(not(contains(@style,'none')))]"):
  driver.find_element(By.XPATH, "//i[@onclick='openTree(this)' and(not(contains(@style,'none')))]").click()
  time.sleep(0.5)

Answered By - Prophet