[FIXED] the potential inadvisability of moving JS files to site-packages/notebook/static

Issue

I have a setup.py configuration that works, but it feels extremely hackish. It is for a widget (via ipywidgets.DOMWidget inheritance) whose JS code is multipartite and broken in all CDNs —so I cannot simply require(['www.some_cdn.com/foo']) in the JS injected into the browser. As a result post-installation I get setup.py to copy the required files into site-packages/notebook/static, allowing the JS require(['/static/jsfolder/some.entry.point.js']) to work (code below).

I assume there must be a standard way of doing this, especially since this hack only works with source distributions not wheel, but I inexplicably seem to be unable to find the "official" way. Where is this documentation that eludes me?

import os
import importlib.util
from distutils.dir_util import copy_tree, remove_tree
from distutils import log

def post_install():
    """
    Moves the jsfolder folder contents to site-packages/notebook/static
    """
    log.info('copying files over to site-packages/notebook/static')
    init_of_notebook = importlib.util.find_spec('notebook').origin
    copy_tree('jsfolder', os.path.join(os.path.dirname(init_of_notebook), 'static', 'jsfolder'))

# are we in a github or a source package?
if os.path.exists(os.path.join('.git')) and not os.path.exists('PKG-INFO'):
    # we are in a git repo, so the usual fluff for pypi
    with open('README.md') as f:
        readme = f.read(). # etc.
    # any pre-packaging tinkering would probably happen here
    # ...

setup(...,
      include_package_data=True,  # MANIFEST.in has `recursive-include * *.png *.js *.css *.gif *.txt` or `graft jsfolder/**` etc.
      )
post_install()

Explanatory footnote for curious passerby

This Q is quite technical, so just in case a future user stumbles across something said in passing I thought I'd expand upon it —just don't copy the hack!

In a Jupyter notebook you can not be served anything below the root specified in the URL (luckily), except the special route /static which serve files in site-packages/notebook/static with the expected MIME type except for files in static/contents which are text/html. site-packages is a folder where pip installed packages generally go.

The include_package_data=True argument in setup copies specified files to the source distribution that are in the root of the repo if there's a MANIFEST.in file, without requiring there to be an __init__.py in every folder unlike the cleaner but more laborious package_data argument.

There are several posts here for executing code pre-installation and post-install involving inheriting from setuptools.command.install import install. As of 2022, python setup.py install is deprecated in favour of pip install . and many of them do not work.

Also any log.info is better than print because it will show with pip install foo.tar.gz -v.

Solution

The official-ish solution appears to be https://github.com/jupyter/jupyter-packaging This is used for example in https://github.com/jupyter-widgets/widget-cookiecutter

from jupyter_packaging import wrap_installers, npm_builder
from types import FunctionType
from typing import Dict
builder:FunctionType = npm_builder()
cmdclass:Dict[str,type] = wrap_installers(pre_develop=builder, pre_dist=builder)
setup(cmdclass=cmdclass, ...))

Unlike my hack they install the js via the nodejs package manager. Namely, the npm_builder is a factory, which does...

import inspect
print(inspect.getsource(npm_builder))

the following:

def npm_builder(
    path=None, build_dir=None, source_dir=None, build_cmd="build", force=False, npm=None
):
...
npm_cmd = npm
...
def builder():
    ...
    npm_cmd = npm
    ...
    run(npm_cmd + ["install"], cwd=node_package)
            if build_cmd:
                run(npm_cmd + ["run", build_cmd], cwd=node_package)

Answered By - Matteo Ferla