[FIXED] Python docker container not building

Issue

I have a repo which runs python on a docker container. Recently I have a purchased a new mac, and on this new workstation, this container is not building successfully when I try to open it with remote-container plug in from VS code.

FROM python:3.8.3-buster
WORKDIR /work

RUN apt-get update

# Install python requirements
COPY mts mts
COPY setup.py setup.py
RUN pip install -e .
RUN pip install pytest
RUN pip install pytest-html
RUN pip install pytest-profiling

This is how the setup.py file looks like:

from setuptools import setup, find_packages
import os

def package_files(directory):
    paths = []
    for (path, directories, filenames) in os.walk(directory):
        for filename in filenames:
            paths.append(os.path.join('..', path, filename))
    return paths


requirements = [
    "numpy==1.17.4",
    "scipy==1.3.3",
    "quadprog==0.1.7",
    "pydantic==1.2",
    "openpyxl==3.0.3"
]

test_requirements = ['pytest'] # [str(tr.req) for tr in parsed_test_requirements]

packages = find_packages(exclude=["tests.*", "tests"])

setup(
    name='mts',
    version='1.0',
    packages=packages,
    package_data={'mts': package_files('mts/data/')},
    include_package_data=True,
    install_requires=requirements,
    tests_require=test_requirements,
    entry_points="""
        [console_scripts]
        mts=mts.__main__:main
    """,
    test_suite='tests',
)

The issue seems to be something related with scipy, cython and lapack

#10 2.478   Downloading scipy-1.3.3.tar.gz (23.8 MB)
#10 5.424   Installing build dependencies: started
#10 72.54   Installing build dependencies: still running...
#10 76.12   Installing build dependencies: finished with status 'done'
#10 76.12   Getting requirements to build wheel: started
#10 76.37   Getting requirements to build wheel: finished with status 'done'
#10 76.37     Preparing wheel metadata: started
#10 76.55     Preparing wheel metadata: finished with status 'error'
#10 76.55     ERROR: Command errored out with exit status 1:
#10 76.55      command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp0depddtk
#10 76.55          cwd: /tmp/pip-install-77juwsbt/scipy
#10 76.55     Complete output (37 lines):
#10 76.55     setup.py:386: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-dh_achih'), proceeding with generating Cython sources and expanding templates
#10 76.55       warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
#10 76.55     Running from scipy source directory.
#10 76.55     /tmp/pip-build-env-3o62xagc/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
#10 76.55         Lapack (http://www.netlib.org/lapack/) libraries not found.
#10 76.55         Directories to search for the libraries can be specified in the
#10 76.55         numpy/distutils/site.cfg file (section [lapack]) or by setting
#10 76.55         the LAPACK environment variable.
#10 76.55       if getattr(self, '_calc_info_{}'.format(lapack))():
#10 76.55     /tmp/pip-build-env-3o62xagc/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
#10 76.55         Lapack (http://www.netlib.org/lapack/) sources not found.
#10 76.55         Directories to search for the sources can be specified in the
#10 76.55         numpy/distutils/site.cfg file (section [lapack_src]) or by setting
#10 76.55         the LAPACK_SRC environment variable.
#10 76.55       if getattr(self, '_calc_info_{}'.format(lapack))():
#10 76.55     Traceback (most recent call last):
#10 76.55       File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
#10 76.55         main()
#10 76.55       File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
#10 76.55         json_out['return_val'] = hook(**hook_input['kwargs'])
#10 76.55       File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
#10 76.55         return hook(metadata_directory, config_settings)
#10 76.55       File "/tmp/pip-build-env-3o62xagc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 174, in prepare_metadata_for_build_wheel
#10 76.55         self.run_setup()
#10 76.55       File "/tmp/pip-build-env-3o62xagc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 267, in run_setup
#10 76.55         super(_BuildMetaLegacyBackend,
#10 76.55       File "/tmp/pip-build-env-3o62xagc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 158, in run_setup
#10 76.55         exec(compile(code, __file__, 'exec'), locals())
#10 76.55       File "setup.py", line 505, in <module>
#10 76.55         setup_package()
#10 76.55       File "setup.py", line 501, in setup_package
#10 76.55         setup(**metadata)
#10 76.55       File "/tmp/pip-build-env-3o62xagc/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
#10 76.55         config = configuration()
#10 76.55       File "setup.py", line 403, in configuration
#10 76.55         raise NotFoundError(msg)
#10 76.55     numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
#10 76.55     ----------------------------------------
#10 76.55 ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp0depddtk Check the logs for full command output.
#10 76.56 WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
#10 76.56 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
------
failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c pip install -e .]: exit code: 1
[79166 ms] Command failed: docker-compose --project-name mts -f /Users/nithin/Documents/projects/mts/docker-compose.yml -f /Users/nithin/Documents/projects/mts/.devcontainer/docker-compose.yml build

How should I modify the dockerfile so that the build completes?

Solution

Reconsider your third-party dependency version-selection.

Docker Base Image

FROM python:3.8.3-buster

is Python 3.8

Dependency: Scipy

scipy 1.3.3 (released 2019!!!) doesn't support Python 3.8

Programming Language
C
Python
Python :: 3
Python :: 3.5
Python :: 3.6
Python :: 3.7

Recommendation

Update your dependencies. At least scipy, but you should check the others too.

Remark

Updating scipy should make this problem easily go away:

• Not only can Python 3.8 be an officially supported target
• But also: You will get binary-wheels and there isn't even compilation (C, C++, Fortran) involved as it will pick up a ManyLinux-based wheel (probably bringing openblas too -> BLAS/LAPACK)

Answered By - sascha