Skip to content

Modernize the mu-python-template - #39

Open
nvdk wants to merge 23 commits into
masterfrom
feature/semantic-ai-fixes
Open

nvdk wants to merge 23 commits into
masterfrom
feature/semantic-ai-fixes

Conversation

@nvdk

@nvdk nvdk commented Oct 26, 2026

Copy link
Copy Markdown
Member

Server & base image

  • Replace the deprecated tiangolo/meinheld-gunicorn:python3.8 base with python:3.13-slim, running Gunicorn with a custom Uvicorn worker class (gunicorn_worker.py) that serves the Flask WSGI app. Behaviour is unchanged: 1 worker (WEB_CONCURRENCY), one request at a time, other Gunicorn settings available via GUNICORN_CMD_ARGS.
  • New DEV_SERVER=gunicorn development mode: production server with live reload, e.g. to test multi-worker/thread behaviour. Flask dev server (debugger) stays the default.
  • build.sh hook: services can install system packages before pip requirements (base image is slim now); gcc/g++ ship for packages without wheels. CI now builds linux/arm64 next to amd64.

Helpers

  • Thread safety: query() and update() execute on a per-call copy of the shared SPARQLWrapper instances, making the helpers safe under threads and multiple workers. Configuration set on the shared instances (timeouts, custom headers) still applies to every query.
  • mu-authorization support: new sudo and scope keyword arguments. sudo=True bypasses the requester's access rights but requires the container-level opt-in ALLOW_MU_AUTH_SUDO; DEFAULT_MU_AUTH_SCOPE provides a fallback scope.
  • Automatic request headers: MU session/call/auth headers are forwarded from the current request automatically; SPARQL helpers are now safe to use outside request contexts (startup code, background threads).
  • New BaseHTTPException: raise a JSONAPI error response from anywhere in a handler; registered error handler turns it into a response (also available as a builtin like app).
  • before_first_request polyfill for services written against older templates (deprecated).
  • LOG_DIR configurable; wait_for_triplestore removed (documented workaround in README).

Compatibility

Python 3.13, Flask ≥ 3.1, SPARQLWrapper ≥ 2.0, rdflib ≥ 7.1. See README "Upgrading from earlier versions"

Tested

Verified against a real service (catalog-harvester): builds, boots, and behaves identically to v2.0.0-beta.3.

madnificent and others added 17 commits October 1, 2026 01:01
This is a first step.  It's not fully tested.  It doesn't keep the full API
space of the previous parent's internals but it does keep the features we know
are used in the wild.

Needs testing.
…d expand minimally with some much needed compilers

(cherry picked from commit 5fba2e5)
(cherry picked from commit 3063183)
(cherry picked from commit da5fa38)
… parts of the helper function, so we don't need to be backward compatible with them

(cherry picked from commit 81c7dbb)
…port for providing scope, wait_for_triplestore no longer does a sudo query to not break non-sudo services

(cherry picked from commit 8533639)
…th an example implementation services could use while waiting for the real deal

(cherry picked from commit d02fbb6)
Raising a BaseHTTPException results in a JSONAPI error response. It is
available without import and takes the same arguments as in the FastAPI
based semantic-ai/mu-python-template. A sudo query without permission
raises it with status 403.

Co-authored-by: Joachim van der Herten <joachim@ml2grow.com>
Services may configure helpers.sparqlQuery and helpers.sparqlUpdate (e.g. a
timeout or custom headers such as mu-auth-sudo), which earlier versions of
the template applied to every query. Copying those per query keeps that
working while staying thread-safe. As every query starts from a copy, the
sudo and scope headers no longer need to be cleared.
- validate_resource_type compared strings by identity, rejecting valid
  resource types
- validate_json_api_content_type crashed on requests without Content-Type
- update_modified used the time the service started as default date
The Uvicorn worker sizes its WSGI thread pool with gunicorn's threads
setting (default 1, e.g. GUNICORN_CMD_ARGS="--threads 4"), so a worker
handles one request at a time like with meinheld. It also stops when
gunicorn's reloader requests so, which Uvicorn didn't check. The
uvicorn-worker package replaces the deprecated uvicorn.workers module.

DEV_SERVER=gunicorn runs the production server with live-reloading in
development mode. app.before_first_request, removed in Flask 2.3, is
emulated with a deprecation warning.
Co-authored-by: Joachim van der Herten <joachim@ml2grow.com>
Co-authored-by: karel kremer <karel.kremer@redpencil.io>
@nvdk
nvdk requested a review from madnificent October 26, 2026 15:41
@nvdk

nvdk commented Oct 26, 2026

Copy link
Copy Markdown
Member Author

I will also remove the logs being written to a file, since that's not in-line with other templates. If you want to keep logs this can be done through docker.

@nvdk nvdk self-assigned this Oct 27, 2026
Niels V added 6 commits October 27, 2026 08:31
Other templates only write to stdout and logs are available through stdout. Removing this for consistency.
Log files require management and it's easiest if this is done in one place (through docker logs)
to keep .git and CI config out of the image
In line with the js template. Allowed groups resolved by
mu-authorization are stored per request and forwarded to the client.
uv is a fast, pip-compatible installer. The pip-style interface is used
with --system: no virtual environments (the container provides the
isolation)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants