[FIXED] Unable to import numpy 1.19.1 in AWS Lambda No module named 'numpy.core._multiarray_umath'

Issue

I am unable to import numpy 1.19.1 in AWS Lambda with python3.8 on AWS Lambda

I am using the following dependencies:

• pandas 1.1.0
• pyarrow 1.0.0
• numpy 1.19.1
• psycopg2 2.8.5

Because I work on a windows environment, I created an EC2 Linux instance installed python3.8 and downloaded all required libraries, then I added them into the project, but the moment I try to import pandas I get the following:

[ERROR] ImportError: Unable to import required dependencies:
numpy: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
  1. Check that you expected to use Python3.8 from "/var/lang/bin/python3.8",
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy version "1.18.2" you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.

Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.

Original error was: No module named 'numpy.core._multiarray_umath'

Traceback (most recent call last):
  File "/var/task/src/py38-lib-test.py", line 28, in py38test
    import pandas
  File "/tmp/lib/pandas/__init__.py", line 16, in <module>
    raise ImportError(END RequestId: 07762380-1fc4)

Lastly, I noticed AWS Lambda provides a layer with numpy and sci-kit, I tried removing my numpy version but kept the rest and added the layer to the function, but the same error occurs.

Thanks in advance your comments.

Solution

Is your ec2 instance an amazon linux2 machine? You could also try building and run a docker image for amazon linux 2 and get the python libs compatible to the environment you need in your Lambda, by volume mounting to your host.

Something similat to docker lambda:

https://github.com/lambci/docker-lambda/tree/master/python3.8

Answered By - Markonick