[FIXED] How to defend against accidentally shadowing the standard library?

Issue

I'm new to Python, and wrote a utility script that generates some numbers I need. I named the file numbers.py. This script depends on the PyGame library. I found that just importing that library caused errors, because PyGame tries to import numbers from the Python standard library, and the default lookup rules have my own file shadowing the standard library.

The linters I have tried (PyLint, Ruff) do not seem to catch this rookie mistake of mine, but I'm having trouble understanding how best to avoid it. As best as I can tell, the advice to "just rename your files" to not collide with the standard library requires either:

1. memorizing the 307 standard library names (that don't have periods; 1027 names in total), or
2. being able to recognize via errors from other libraries that maybe you accidentally shadowed a library, so you can rename

I'm hoping there's a better solution, like #include <foo> vs #include "foo" that would differentiate between the standard library and a local file, or a lint configuration that detects when a file I create shadows the standard library.

How can I avoid this problem as I go forward?

phrogz:~/proj/ai$ ls
numbers.py

phrogz:~/proj/ai$ cat numbers.py 
import pygame

phrogz:~/proj/ai$ python numbers.py
Traceback (most recent call last):
  File "/home/phrogz/proj/ai/numbers.py", line 1, in <module>
    import pygame
  File "/home/phrogz/.local/lib/python3.11/site-packages/pygame/__init__.py", line 264, in <module>
    import pygame.surfarray
  File "/home/phrogz/.local/lib/python3.11/site-packages/pygame/surfarray.py", line 47, in <module>
    import numpy
  File "/home/phrogz/.local/lib/python3.11/site-packages/numpy/__init__.py", line 130, in <module>
    from numpy.__config__ import show as show_config
  File "/home/phrogz/.local/lib/python3.11/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy.core._multiarray_umath import (
  File "/home/phrogz/.local/lib/python3.11/site-packages/numpy/core/__init__.py", line 72, in <module>
    from . import numerictypes as nt
  File "/home/phrogz/.local/lib/python3.11/site-packages/numpy/core/numerictypes.py", line 595, in <module>
    _register_types()
  File "/home/phrogz/.local/lib/python3.11/site-packages/numpy/core/numerictypes.py", line 590, in _register_types
    numbers.Integral.register(integer)
    ^^^^^^^^^^^^^^^^
AttributeError: module 'numbers' has no attribute 'Integral'

Solution

The issue happens because when running

python myscript.py

The directory of myscript.py gets prepended to the sys.path. The prepend means that your own module is found by import statements before any library modules.

Since this is a "feature" (ability to monkeypatch stdlib), it is unlikely to change any time soon, but Python 3.11+ does provide ways to opt-out of that potentially unsafe behavior with command-line options and environment variables:

-P

Don’t prepend a potentially unsafe path to sys.path:

• python -m module command line: Don’t prepend the current working directory.
• python script.py command line: Don’t prepend the script’s directory. If it’s a symbolic link, resolve symbolic links.
• python -c code and python (REPL) command lines: Don’t prepend an empty string, which means the current working directory.

See also the PYTHONSAFEPATH environment variable.

This ensures only stdlib and installed modules are picked up by import, and avoids unintentionally or maliciously shadowing modules with those in a local (and typically user-writable) directory.

ref: https://github.com/python/cpython/pull/31542

Answered By - wim