Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Builds the docs/ Antora component as a site of its own (docs/antora-playbook.yml) and attaches it
# as the docs-site artifact. docs.arcadedb.com does not publish the component yet
# (ArcadeData/arcadedb-docs#505), so this artifact is the rendered preview, and this job is the
# check a docs-only pull request gets: the test workflows ignore docs/** and *.adoc.
name: Docs

on:
push:
branches: [main]
paths:
- "docs/**"
- ".github/workflows/docs.yml"
pull_request:
paths:
- "docs/**"
- ".github/workflows/docs.yml"
workflow_dispatch:

permissions:
contents: read

jobs:
build:
name: Build docs site
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: "24"

- name: Build with Antora
# Same version as docs.arcadedb.com (@antora/cli and @antora/site-generator ^3.2.1 in
# ArcadeData/arcadedb-docs). The playbook fails the build on any warning.
run: npx --yes -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml

- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: docs-site
path: docs/build/site
retention-days: 14
if-no-files-found: error
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ python -m bandit -c pyproject.toml -r examples --severity-level medium --confide

## CI

Each package has one test and one release workflow: `test-jvm.yml` and `release-jvm.yml` for `arcadedb-jvm`, `test-native.yml` and `release-native.yml` for `arcadedb` (the release workflows call the test workflows with `workflow_call`; the PyPI trusted publishers are keyed on the release file names and the environments `pypi-jvm` and `pypi`, so never rename them without changing PyPI first). `test-python-examples.yml` runs `examples/`.
Each package has one test and one release workflow: `test-jvm.yml` and `release-jvm.yml` for `arcadedb-jvm`, `test-native.yml` and `release-native.yml` for `arcadedb` (the release workflows call the test workflows with `workflow_call`; the PyPI trusted publishers are keyed on the release file names and the environments `pypi-jvm` and `pypi`, so never rename them without changing PyPI first). `test-python-examples.yml` runs `examples/`. `docs.yml` builds the Antora component in `docs/` as a site of its own (`docs/antora-playbook.yml`, Antora 3.2.1, a warning fails it) and attaches it as the `docs-site` artifact; it is the only check of a docs-only pull request. Link into the main manual with `{arcadedb-docs-url}/<page>`, never an `arcadedb::` xref (the component on its own cannot resolve it).

`.github/workflows/test-jvm.yml` builds ONE wheel per platform (linux/amd64, linux/arm64, darwin/arm64, windows/amd64), tagged `py3-none-<platform>` by `setup.py` (`bdist_wheel.get_tag`; `scripts/verify_wheel_tag.py` checks it), and tests that wheel on Python 3.11–3.14; also runs nightly against `arcadedata/arcadedb:latest`. `requires-python` is `>=3.11,<3.15` because the tag no longer keeps pip off 3.15 and JPype has no 3.15 wheel; drop the cap with it. The org shares a runner cap with upstream, so the unit tests run 10 legs on every event (linux/amd64 on all four Pythons, the other three platforms on 3.11 and 3.14, the shape of test-native.yml), and the examples run all four platforms on 3.14 on a pull request (4 legs, 28 checks in all) and 16 legs on push to main, the nightly run, dispatch and the release (40 checks). `scripts/select_matrix.py` holds both lists (`PULL_REQUEST_LEGS`, keyed `jvm` and `examples`), the first job of each workflow runs it, and `tests/test_select_matrix.py` pins the full matrix; docs page `development/ci.adoc`. Keep the job names `Test arcadedb (<platform>, Python <version>)` unchanged. Both workflows accept an `arcadedb-tag` input to pin the image. `test-python-examples.yml` runs `examples/`. A dependency-floor audit resolves extras with `--resolution lowest-direct` for each Python classifier and runs pip-audit, so raise lower bounds when a floor has an advisory.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Releases from this repository start with 26.11.1. The 26.10.x wheels were built
## Documentation

- Here: [`docs/`](https://github.com/ArcadeData/arcadedb-python/tree/main/docs/modules/ROOT/pages) is an Antora component (AsciiDoc) with Installation, [Which package?](https://github.com/ArcadeData/arcadedb-python/blob/main/docs/modules/ROOT/pages/which-package.adoc), Quickstart, Databases and queries, and Concurrency, processes and crashes. It covers only what is specific to the Python bindings.
- The component is not published on docs.arcadedb.com yet; adding it as a content source is open in [ArcadeData/arcadedb-docs#505](https://github.com/ArcadeData/arcadedb-docs/pull/505).
- The component is not published on docs.arcadedb.com yet; adding it as a content source is open in [ArcadeData/arcadedb-docs#505](https://github.com/ArcadeData/arcadedb-docs/pull/505). Until then, the `Docs` workflow builds it as a site of its own on every change under `docs/` and attaches it as the `docs-site` artifact. To build it locally, run `npx -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml` and open `docs/build/site/index.html`.
- The older guide, API reference and example walkthroughs are at [docs.humem.ai/arcadedb](https://docs.humem.ai/arcadedb/latest/). They describe the previous packages and use the import name `arcadedb_embedded`; read it as `arcadedb`. They stay until those pages are ported here ([#19](https://github.com/ArcadeData/arcadedb-python/issues/19)). <!-- allow-old-name -->
- ArcadeDB itself (SQL, openCypher, schema, server): [docs.arcadedb.com](https://docs.arcadedb.com). It is not repeated here.

Expand Down
41 changes: 41 additions & 0 deletions docs/antora-playbook.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Builds this component as a site of its own, until docs.arcadedb.com publishes it
# (ArcadeData/arcadedb-docs#505). The workflow docs.yml runs it on every change under docs/.
# Local preview, from the repository root:
# npx -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml
# then open docs/build/site/index.html.
# The UI bundle and the asciidoc attributes are the ones in the antora-playbook.yml of
# ArcadeData/arcadedb-docs, so a page renders here as it does on docs.arcadedb.com.
site:
title: ArcadeDB Python
start_page: arcadedb-python::index.adoc

content:
sources:
# ./ makes the path relative to this file (a bare .. is relative to the working directory).
- url: ./..
branches: HEAD
start_path: docs

ui:
bundle:
url: https://gitlab.com/antora/antora-ui-default/-/jobs/artifacts/HEAD/raw/build/ui-bundle.zip?job=bundle-stable
snapshot: true

asciidoc:
attributes:
page-pagination: ''
experimental: ''
idprefix: ''
idseparator: '-'
page-toclevels: 3
# Reports a misspelled {attribute} (it renders as written either way); with the
# failure level below, it fails the build.
attribute-missing: warn

runtime:
log:
# Stricter than docs.arcadedb.com (error): a warning in these pages fails the build.
failure_level: warn

output:
dir: ./build/site
3 changes: 3 additions & 0 deletions docs/antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,6 @@ asciidoc:
native-pypi-url: https://pypi.org/project/arcadedb/
jvm-pypi-url: https://pypi.org/project/arcadedb-jvm/
which-package-url: https://github.com/ArcadeData/arcadedb-python/blob/main/docs/modules/ROOT/pages/which-package.adoc
# The main ArcadeDB manual. Pages link into it as {arcadedb-docs-url}/<page>, not with an
# arcadedb:: xref, so the component also builds on its own (antora-playbook.yml here).
arcadedb-docs-url: https://docs.arcadedb.com/arcadedb
2 changes: 2 additions & 0 deletions docs/modules/ROOT/pages/development/ci.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Each package has one test workflow and one release workflow:

The release workflows call the test workflows (`workflow_call`) and then publish; see xref:development/release.adoc[Release]. Two workflows test the `arcadedb-jvm` package: `test-jvm.yml` builds a wheel and runs the unit tests, and `test-python-examples.yml` runs the scripts in `examples/`. Both build the wheel once for each platform, on a native runner of that platform (`Build arcadedb-jvm wheel (<platform>)` and `Build wheel for examples (<platform>)`), and install that one file into each Python they test. `test-native.yml` builds and tests the `arcadedb` (native) package; see <<_native_build,Native build>>.

The workflow `docs.yml` builds these docs (the Antora component in `docs/`) as a site of its own with `docs/antora-playbook.yml`, on every pull request and push to `main` that changes `docs/`, and attaches the site as the `docs-site` artifact. Download it from the run to read a page as it will render. The test workflows ignore `docs/**` and `*.adoc`, so for a docs-only pull request this job is the only check. A warning fails it: a broken `xref:`, a misspelled `+{attribute}+`, an AsciiDoc syntax problem. Pages link into the main ArcadeDB manual as `+{arcadedb-docs-url}/<page>+`, not with an `arcadedb::` xref, which the component on its own cannot resolve. To build the site locally, run `npx -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml` from the repository root and open `docs/build/site/index.html`.

The package supports four platforms and four Python versions (3.11 to 3.14). That is 16 combinations. The organisation shares one runner limit with ArcadeData/arcadedb, so each macOS or Windows job delays the upstream jobs too, and not every combination runs.

== One wheel for each platform
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/how-to/bulk-loads.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ This section is about a `graph_batch()` load that stops before `close()` returns

* The vertices and edge records that a returned call committed are there. A load that is killed loses the work after the last commit and nothing before it.
* The edges are not traversable. The engine writes the edge records first and links them to their vertices (the head pointers and the incoming lists) in `close()`. Until `close()` has run, `outE()`, `inE()`, `out()` and `MATCH` find none of them, but `SELECT count(*) FROM E` counts them. In the example: 70,000 edge records, 0 reachable.
* `CHECK DATABASE` reports each of these edges as an `ORPHAN RECORD` and counts them in `unreachableEdgeRecords`. It also uses the words "was not connected from", "is not referenced back by" and "missing from that vertex's OUT/IN list" for the same state. Read the statement in the xref:arcadedb::reference/sql/sql-database-admin.adoc#sql-check-database[`CHECK DATABASE` reference]; this page does not repeat it.
* `CHECK DATABASE` reports each of these edges as an `ORPHAN RECORD` and counts them in `unreachableEdgeRecords`. It also uses the words "was not connected from", "is not referenced back by" and "missing from that vertex's OUT/IN list" for the same state. Read the statement in the {arcadedb-docs-url}/reference/sql/sql-database-admin#sql-check-database[`CHECK DATABASE` reference]; this page does not repeat it.

=== Recovery that works

Expand Down
6 changes: 3 additions & 3 deletions docs/modules/ROOT/pages/how-to/concurrency.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Concurrency, processes and crashes
:description: What the embedded bindings do with threads, asyncio, several processes, signals and crashes, as measured, and how to write an update that two writers cannot lose.

This page covers what is specific to the Python bindings: which calls work from several threads, from asyncio or from several processes, what a signal or a crash does to the process and the data, and how to write an update that concurrent writers cannot lose. What the engine guarantees at each isolation level is in the main ArcadeDB docs: see xref:arcadedb::concepts/transactions.adoc#isolation[Isolation].
This page covers what is specific to the Python bindings: which calls work from several threads, from asyncio or from several processes, what a signal or a crash does to the process and the data, and how to write an update that concurrent writers cannot lose. What the engine guarantees at each isolation level is in the main ArcadeDB docs: see {arcadedb-docs-url}/concepts/transactions#isolation[Isolation].

== How to read the numbers

Expand Down Expand Up @@ -52,7 +52,7 @@ All 7 runs ended with no error, no bad checksum and the same number of rows befo

== Several writers

Two transactions that change the same record conflict at commit. The one that loses raises `ArcadeDBError`, and the name `ConcurrentModificationException` is in the message or in the `__cause__` chain. The engine treats it as retryable: see xref:arcadedb::concepts/transactions.adoc#isolation[Isolation] for when it is raised. `db.run_in_transaction(fn, retries=...)` runs `fn` in a transaction again after such a conflict.
Two transactions that change the same record conflict at commit. The one that loses raises `ArcadeDBError`, and the name `ConcurrentModificationException` is in the message or in the `__cause__` chain. The engine treats it as retryable: see {arcadedb-docs-url}/concepts/transactions#isolation[Isolation] for when it is raised. `db.run_in_transaction(fn, retries=...)` runs `fn` in a transaction again after such a conflict.

The probe: 8 threads, one shared `Database`, 4 counter records. Each operation is one transaction that changes one counter, chosen by a fixed pseudo-random sequence, and inserts one row with a unique key into a ledger type. Each thread makes 1,500 operations (500 on Python 3.14t), so 12,000 in total (4,000 on Python 3.14t). The probe counts a lost update as an operation whose transaction committed but whose increment is not in the final counters. Each cell below is the number of lost updates out of the number of operations, from one run.

Expand Down Expand Up @@ -203,7 +203,7 @@ Based on that:

== Signals and crashes

A write that has returned from `commit()` survived every kill in the probe. The probe cannot say anything about a power loss or an operating system crash, which a process kill does not simulate. For how the engine flushes its write-ahead log, see xref:arcadedb::concepts/transactions.adoc#wal-flush-durability[WAL flush and durability].
A write that has returned from `commit()` survived every kill in the probe. The probe cannot say anything about a power loss or an operating system crash, which a process kill does not simulate. For how the engine flushes its write-ahead log, see {arcadedb-docs-url}/concepts/transactions#wal-flush-durability[WAL flush and durability].

The kill probe: a child process writes in a loop, in transactions of 5 or 200 rows, and prints the number of rows after every commit that returned. The parent sends `SIGKILL` (kill -9) at a random time between 0.2 and 2.5 s after the first commit, then reopens the same database. The reopen must recover, `CHECK DATABASE` must report nothing, every acknowledged row must be present with a matching checksum, no transaction may be half applied, and the database must accept a write. The same database was used for all runs of a build.

Expand Down
6 changes: 3 additions & 3 deletions docs/modules/ROOT/pages/how-to/databases-and-queries.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -255,13 +255,13 @@ Page a large type by `@rid` instead of `SKIP`: each page is `WHERE @rid > ? LIMI

=== SQLScript

`db.command("sqlscript", script)` runs several statements in one call, and the result set holds the last executed statement, which is why the script above reports the `ALTER TYPE` statement. The script syntax is in the xref:arcadedb::reference/sql/sql-script.adoc[SQLScript reference].
`db.command("sqlscript", script)` runs several statements in one call, and the result set holds the last executed statement, which is why the script above reports the `ALTER TYPE` statement. The script syntax is in the {arcadedb-docs-url}/reference/sql/sql-script[SQLScript reference].

=== openCypher

The same two calls run openCypher: pass `"opencypher"` as the language. The script creates an edge with a `CREATE` pattern and reads it back with `MATCH`. The language is documented in the xref:arcadedb::reference/cypher/chapter.adoc[openCypher reference].
The same two calls run openCypher: pass `"opencypher"` as the language. The script creates an edge with a `CREATE` pattern and reads it back with `MATCH`. The language is documented in the {arcadedb-docs-url}/reference/cypher/chapter[openCypher reference].

NOTE: ArcadeDB SQL has no JOINs, so SQLAlchemy and other ORMs, and tools that generate `JOIN` queries, do not work with these two calls. Write the statements yourself, following links as in xref:arcadedb::reference/sql/sql-introduction.adoc#no-joins[No JOINs], or use openCypher for relationships.
NOTE: ArcadeDB SQL has no JOINs, so SQLAlchemy and other ORMs, and tools that generate `JOIN` queries, do not work with these two calls. Write the statements yourself, following links as in {arcadedb-docs-url}/reference/sql/sql-introduction#no-joins[No JOINs], or use openCypher for relationships.

== Next steps

Expand Down
4 changes: 2 additions & 2 deletions docs/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,9 @@ ArcadeDB reaches Python in two ways. Pick by where the engine should run.
| A running server (the official Docker image `arcadedata/arcadedb`), plus `pip install arcadedb-driver`
|===

Choose the embedded bindings for a single script, a notebook, or an application with one Python process that owns its data. Choose the xref:arcadedb::how-to/connectivity/drivers/python-http.adoc[HTTP driver] when more than one process, another language, or a remote machine must reach the same database, or when the server must outlive your Python process.
Choose the embedded bindings for a single script, a notebook, or an application with one Python process that owns its data. Choose the {arcadedb-docs-url}/how-to/connectivity/drivers/python-http[HTTP driver] when more than one process, another language, or a remote machine must reach the same database, or when the server must outlive your Python process.

The two meet in one case. The bindings bundle an optional HTTP server. It is off until you call `create_server()`. The process that owns the database keeps direct access, and other processes connect over HTTP. That server lives and dies with your Python process. For a database that must outlive any one client, or for HA, replication and TLS, run the regular xref:arcadedb::index.adoc[ArcadeDB server] instead.
The two meet in one case. The bindings bundle an optional HTTP server. It is off until you call `create_server()`. The process that owns the database keeps direct access, and other processes connect over HTTP. That server lives and dies with your Python process. For a database that must outlive any one client, or for HA, replication and TLS, run the regular {arcadedb-docs-url}/[ArcadeDB server] instead.

[[jvm-vs-native]]
== Native build or JVM build
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/installation.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ export ARCADEDB_NATIVE_ISOLATE_ARGS="-XX:MaximumHeapSizePercent=70" # another p
export ARCADEDB_NATIVE_ISOLATE_ARGS="-XX:MaxRAM=4g" # a memory size for the percentage; GraalVM's 80% applies to it
----

When one of these three is present, the package adds no default. Do not use `-XX:MaxRAMPercentage`. The native image does not accept it, and the isolate then fails to start with `graal_create_isolate failed (rc=22)`. The same variable also takes the engine's `-D` settings: see xref:how-to/logs.adoc[Where the logs go]. For what a native image build supports, see xref:arcadedb::how-to/operations/native-image.adoc[Native image] in the main ArcadeDB docs.
When one of these three is present, the package adds no default. Do not use `-XX:MaxRAMPercentage`. The native image does not accept it, and the isolate then fails to start with `graal_create_isolate failed (rc=22)`. The same variable also takes the engine's `-D` settings: see xref:how-to/logs.adoc[Where the logs go]. For what a native image build supports, see {arcadedb-docs-url}/how-to/operations/native-image[Native image] in the main ArcadeDB docs.

[[native-limits]]
=== Limits of the native build
Expand Down
Loading
Loading