[FIXED] PyTorch: running Neural Network in identical venv-configuration - one working fine, one keeps throwing 'list index out of range' error

Issue

I set up an virtual environment via 'venv' to build and train a neural network using PyTorch and JupyterLab. However, when working on my computer 'PC 1' everything works fine, but running the same code with identical Python settings on my second computer ('PC 2') keeps constantly an error ('list index out of range') thrown while training. Any suggestions what might cause this behavior? I'm running out of ideas ...

To be more specific:

• On both computers Python 3.7.9 is installed via the Microsoft Store.
• To be clear: both computers access the same *.ipynb and the same data / datasets, which are synced via a cloud service.

I tried:

• I synced the created venv (via a cloud), activated venv and runned the *.ipynb via jupyter-lab (on 'PC 2') -> error
• I got my venv-configuration from the working 'PC 1' via pip freeze > requirements.txt and set up a fresh venv on 'PC 2' using the requirements.txt-> error

Nevertheless I try, the thrown error is always the same.

This is the error thrown: ('julab' is my venv)

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_9608\2329219183.py in <module>
     10     val_dl = test_loader,
     11     epochs=num_epochs,
---> 12     device='cpu')

~\AppData\Local\Temp\ipykernel_9608\132402798.py in train(model, optimizer, loss_fn, train_dl, val_dl, epochs, device)
     27         num_train_examples = 0
     28 
---> 29         for batch in train_dl:
     30 
     31             optimizer.zero_grad()

d:\<CLOUD>\<SUBFOLDER>\julab\lib\site-packages\torch\utils\data\dataloader.py in __next__(self)
    650                 # TODO(https://github.com/pytorch/pytorch/issues/76750)
    651                 self._reset()  # type: ignore[call-arg]
--> 652             data = self._next_data()
    653             self._num_yielded += 1
    654             if self._dataset_kind == _DatasetKind.Iterable and \

d:\<CLOUD>\<SUBFOLDER>\julab\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
    690     def _next_data(self):
    691         index = self._next_index()  # may raise StopIteration
--> 692         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    693         if self._pin_memory:
    694             data = _utils.pin_memory.pin_memory(data, self._pin_memory_device)

d:\<CLOUD>\<SUBFOLDER>\julab\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly_batched_index)
     47     def fetch(self, possibly_batched_index):
     48         if self.auto_collation:
---> 49             data = [self.dataset[idx] for idx in possibly_batched_index]
     50         else:
     51             data = self.dataset[possibly_batched_index]

d:\<CLOUD>\<SUBFOLDER>\julab\lib\site-packages\torch\utils\data\_utils\fetch.py in <listcomp>(.0)
     47     def fetch(self, possibly_batched_index):
     48         if self.auto_collation:
---> 49             data = [self.dataset[idx] for idx in possibly_batched_index]
     50         else:
     51             data = self.dataset[possibly_batched_index]

d:\<CLOUD>\<SUBFOLDER>\julab\lib\site-packages\torch\utils\data\dataset.py in __getitem__(self, idx)
    288         if isinstance(idx, list):
    289             return self.dataset[[self.indices[i] for i in idx]]
--> 290         return self.dataset[self.indices[idx]]
    291 
    292     def __len__(self):

~\AppData\Local\Temp\ipykernel_9608\2122586536.py in __getitem__(self, index)
     32 
     33     def __getitem__(self, index):
---> 34         image_name = os.path.join(self.image_dir, self.image_files[index])
     35         image = PIL.Image.open(image_name)
     36         label = self.data[index]

IndexError: list index out of range

And as this might help, this is my requirements.txt:

anyio==3.6.1
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
attrs==21.4.0
Babel==2.10.3
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
certifi==2022.6.15
cffi==1.15.1
charset-normalizer==2.1.0
colorama==0.4.5
cycler==0.11.0
debugpy==1.6.0
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.5.1
entrypoints==0.4
fastjsonschema==2.15.3
fonttools==4.33.3
idna==3.3
importlib-metadata==4.12.0
importlib-resources==5.8.0
ipykernel==6.15.0
ipython==7.34.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
joblib==1.1.0
json5==0.9.8
jsonschema==4.6.1
jupyter-client==7.3.4
jupyter-core==4.10.0
jupyter-server==1.18.0
jupyterlab==3.4.3
jupyterlab-pygments==0.2.2
jupyterlab-server==2.14.0
kiwisolver==1.4.3
MarkupSafe==2.1.1
matplotlib==3.5.2
matplotlib-inline==0.1.3
mistune==0.8.4
nbclassic==0.4.0
nbclient==0.6.6
nbconvert==6.5.0
nbformat==5.4.0
nest-asyncio==1.5.5
notebook-shim==0.1.0
numpy==1.21.6
packaging==21.3
pandas==1.3.5
pandocfilters==1.5.0
parso==0.8.3
pickleshare==0.7.5
Pillow==9.2.0
prometheus-client==0.14.1
prompt-toolkit==3.0.30
psutil==5.9.1
pycparser==2.21
Pygments==2.12.0
pyparsing==3.0.9
pyrsistent==0.18.1
python-dateutil==2.8.2
pytz==2022.1
pywin32==304
pywinpty==2.0.5
pyzmq==23.2.0
requests==2.28.1
scikit-learn==1.0.2
scipy==1.7.3
Send2Trash==1.8.0
six==1.16.0
sklearn==0.0
sniffio==1.2.0
soupsieve==2.3.2.post1
terminado==0.15.0
threadpoolctl==3.1.0
tinycss2==1.1.1
torch==1.12.0
torchsummary==1.5.1
torchvision==0.13.0
tornado==6.1
traitlets==5.3.0
typing_extensions==4.3.0
urllib3==1.26.9
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.3.3
zipp==3.8.0

Has anyone an idea what's the problem and how to solve it? Many thanks in advance!

Solution

[solved] see comments above...

Answered By - femonk