How to reference static assets from the sphinx json builder?

[tombreit]

Dear Carlton Gibson,
Thank's a lot for this great idea and implementation - I just started integrating this thing in an existing project. I managed to render the sphinx content with my own template and have it served by Django.

But I can't figure out how to reference static assets from my sphinx generated /docs/_build/json/_static directory.

To - for example - "recreate" the (javascript-based) search functionality from the html builder, I need to reference/link some files from this directory in my custom (DTL) template - but how?

As a workaround, I extended the STATICFILES_DIRS to include this directory

# djangoproject/settings.py

STATICFILES_DIRS = [
    PROJECT_DIR / "static",
    BASE_DIR / "docs/_build/json/_static",
]

...and then use the standard static template tag:

# templates/sphinx_view/base.html

        <script src="{% static '_static/searchtools.js' %}"></script>

But that doesn't feel right (too tight coupling of the django project and the docs, "pollution" of the staticfiles directory after running collectstatic etc.).

Theming is not part of django-sphinx-view, and I'm happy bringing my own CSS and whatever for this "documentation site/view", but how to use the already existing files the json builder produces?

[carltongibson]

Hi @tombreit — good question. I should doc this up... 🤔

In general, I think you do want to add the _static folder to your STATICFILES_DIRS — then you can serve them in development, and collectstatic puts them in the right place for production. (The alternative is have have a view serve the static files from the build dir, or configure the web server to look there directly, in addition to STATIC_ROOT.)

If you add an image to a doc Sphinx will put the file in the _static folder, and you'll need to server it, otherwise it won't appear in your documentation. (Once you've got an answer for that, the Sphinx theme added files like searchtools.js require similar.)

Make sense?

[tombreit]

Ok, than I was on the right track. But even having docs/_build/json/_static/ in STATICFILES_DIRS does not let me reference the (top level) generated searchindex.json which is a generated content file by sphinx, not a static asset.

OT: And it seems Sphinx missed putting a referenced sourcemap file in _static which results in a ValueError for collectstatic:

Post-processing 'underscore-1.13.1.js' failed!
[...]
ValueError: The file 'underscore-umd.js.map' could not be found with <django.contrib.staticfiles.storage.ManifestStaticFilesStorage object at 0x7f5068406850>.

[carltongibson]

Ah, OK... the post-processing needing the source map complicates things slightly. 🤔

The basic principle is to have the static files served by the web server. ...

[tombreit]

This sourcemap thing could be handled by subclassing ManifestStaticFilesStorage. And this missing sourcemap file should be considered a bug (?) in the Sphinx distribution: if they ship underscore.js and that version links to underscore-umd.js.map and they do not ship that I would consider that "incomplete".

A minified JavaScript file shouldn't contain source map references if map is not provided by the vendor.
(Stolen from https://code.djangoproject.com/ticket/33353#comment:2)

But even with a patched ManifestStaticFilesStorage I do not know how to reference the file /docs/_build/json/searchindex.json in my template or how the build process could "move" that file to a static directory.

[carltongibson]

OK, I need to have a think about it more, to work out this issue with the source map.

• You could drop the Manifest storage for a step-gap.
• Maybe collecting the Sphinx files separately, or adding the folder to the web server config separately would work too.

I do not know how to reference the file /docs/_build/json/searchindex.json

Depends how you set it up, but if in static files then using the static tag is a good way.

[tombreit]

As the sphinx-generated file _build/json/searchindex.json is not part of sphinx _static/ directory, I'll serve it via a custom view (and modify it a bit so that it matches the Javascript file that get's generated by the html builder). With this setup I can enforce authorization for that view/file. Perhaps I should subclass DocumentationView and integrate the serving of that file...:

# urls.py
urlpatterns = [
    path('docs/searchindex.js', sphinx_searchindex_view, name='sphinx_searchindex_view'),
]

# views.py
def sphinx_searchindex_view(request):
    searchindex_json_path = settings.BASE_DIR / "docs/_build/json/searchindex.json"
    if searchindex_json_path.is_file():
        with open(searchindex_json_path, 'r') as fh:
            searchindex_js = f"Search.setIndex({fh.read()});"
            response = HttpResponse(searchindex_js, content_type="text/javascript")
            response['Content-Disposition'] = 'inline; filename="searchindex.js"'
            return response
    raise Http404

# template.html
        <script src="{% url 'sphinx_searchindex_view' %}" defer></script>

The "sphinx javascript search" now (partially) works on my django-sphinx-view-powered site. But as some Javascript errors remain I tend to use the sphinx html builder, served via the webserver - or just drop the idea to have the Sphinx javascript search on this django-sphinx-view-powered site.