diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..65aec2a --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,46 @@ +# Builds the docs/ Antora component as a site of its own (docs/antora-playbook.yml) and attaches it +# as the docs-site artifact. docs.arcadedb.com does not publish the component yet +# (ArcadeData/arcadedb-docs#505), so this artifact is the rendered preview, and this job is the +# check a docs-only pull request gets: the test workflows ignore docs/** and *.adoc. +name: Docs + +on: + push: + branches: [main] + paths: + - "docs/**" + - ".github/workflows/docs.yml" + pull_request: + paths: + - "docs/**" + - ".github/workflows/docs.yml" + workflow_dispatch: + +permissions: + contents: read + +jobs: + build: + name: Build docs site + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: "24" + + - name: Build with Antora + # Same version as docs.arcadedb.com (@antora/cli and @antora/site-generator ^3.2.1 in + # ArcadeData/arcadedb-docs). The playbook fails the build on any warning. + run: npx --yes -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml + + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: docs-site + path: docs/build/site + retention-days: 14 + if-no-files-found: error diff --git a/CLAUDE.md b/CLAUDE.md index bcaf961..b45f712 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -69,7 +69,7 @@ python -m bandit -c pyproject.toml -r examples --severity-level medium --confide ## CI -Each package has one test and one release workflow: `test-jvm.yml` and `release-jvm.yml` for `arcadedb-jvm`, `test-native.yml` and `release-native.yml` for `arcadedb` (the release workflows call the test workflows with `workflow_call`; the PyPI trusted publishers are keyed on the release file names and the environments `pypi-jvm` and `pypi`, so never rename them without changing PyPI first). `test-python-examples.yml` runs `examples/`. +Each package has one test and one release workflow: `test-jvm.yml` and `release-jvm.yml` for `arcadedb-jvm`, `test-native.yml` and `release-native.yml` for `arcadedb` (the release workflows call the test workflows with `workflow_call`; the PyPI trusted publishers are keyed on the release file names and the environments `pypi-jvm` and `pypi`, so never rename them without changing PyPI first). `test-python-examples.yml` runs `examples/`. `docs.yml` builds the Antora component in `docs/` as a site of its own (`docs/antora-playbook.yml`, Antora 3.2.1, a warning fails it) and attaches it as the `docs-site` artifact; it is the only check of a docs-only pull request. Link into the main manual with `{arcadedb-docs-url}/`, never an `arcadedb::` xref (the component on its own cannot resolve it). `.github/workflows/test-jvm.yml` builds ONE wheel per platform (linux/amd64, linux/arm64, darwin/arm64, windows/amd64), tagged `py3-none-` by `setup.py` (`bdist_wheel.get_tag`; `scripts/verify_wheel_tag.py` checks it), and tests that wheel on Python 3.11–3.14; also runs nightly against `arcadedata/arcadedb:latest`. `requires-python` is `>=3.11,<3.15` because the tag no longer keeps pip off 3.15 and JPype has no 3.15 wheel; drop the cap with it. The org shares a runner cap with upstream, so the unit tests run 10 legs on every event (linux/amd64 on all four Pythons, the other three platforms on 3.11 and 3.14, the shape of test-native.yml), and the examples run all four platforms on 3.14 on a pull request (4 legs, 28 checks in all) and 16 legs on push to main, the nightly run, dispatch and the release (40 checks). `scripts/select_matrix.py` holds both lists (`PULL_REQUEST_LEGS`, keyed `jvm` and `examples`), the first job of each workflow runs it, and `tests/test_select_matrix.py` pins the full matrix; docs page `development/ci.adoc`. Keep the job names `Test arcadedb (, Python )` unchanged. Both workflows accept an `arcadedb-tag` input to pin the image. `test-python-examples.yml` runs `examples/`. A dependency-floor audit resolves extras with `--resolution lowest-direct` for each Python classifier and runs pip-audit, so raise lower bounds when a floor has an advisory. diff --git a/README.md b/README.md index 5924465..1ea4eb4 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ Releases from this repository start with 26.11.1. The 26.10.x wheels were built ## Documentation - Here: [`docs/`](https://github.com/ArcadeData/arcadedb-python/tree/main/docs/modules/ROOT/pages) is an Antora component (AsciiDoc) with Installation, [Which package?](https://github.com/ArcadeData/arcadedb-python/blob/main/docs/modules/ROOT/pages/which-package.adoc), Quickstart, Databases and queries, and Concurrency, processes and crashes. It covers only what is specific to the Python bindings. -- The component is not published on docs.arcadedb.com yet; adding it as a content source is open in [ArcadeData/arcadedb-docs#505](https://github.com/ArcadeData/arcadedb-docs/pull/505). +- The component is not published on docs.arcadedb.com yet; adding it as a content source is open in [ArcadeData/arcadedb-docs#505](https://github.com/ArcadeData/arcadedb-docs/pull/505). Until then, the `Docs` workflow builds it as a site of its own on every change under `docs/` and attaches it as the `docs-site` artifact. To build it locally, run `npx -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml` and open `docs/build/site/index.html`. - The older guide, API reference and example walkthroughs are at [docs.humem.ai/arcadedb](https://docs.humem.ai/arcadedb/latest/). They describe the previous packages and use the import name `arcadedb_embedded`; read it as `arcadedb`. They stay until those pages are ported here ([#19](https://github.com/ArcadeData/arcadedb-python/issues/19)). - ArcadeDB itself (SQL, openCypher, schema, server): [docs.arcadedb.com](https://docs.arcadedb.com). It is not repeated here. diff --git a/docs/antora-playbook.yml b/docs/antora-playbook.yml new file mode 100644 index 0000000..7307fbc --- /dev/null +++ b/docs/antora-playbook.yml @@ -0,0 +1,41 @@ +# Builds this component as a site of its own, until docs.arcadedb.com publishes it +# (ArcadeData/arcadedb-docs#505). The workflow docs.yml runs it on every change under docs/. +# Local preview, from the repository root: +# npx -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml +# then open docs/build/site/index.html. +# The UI bundle and the asciidoc attributes are the ones in the antora-playbook.yml of +# ArcadeData/arcadedb-docs, so a page renders here as it does on docs.arcadedb.com. +site: + title: ArcadeDB Python + start_page: arcadedb-python::index.adoc + +content: + sources: + # ./ makes the path relative to this file (a bare .. is relative to the working directory). + - url: ./.. + branches: HEAD + start_path: docs + +ui: + bundle: + url: https://gitlab.com/antora/antora-ui-default/-/jobs/artifacts/HEAD/raw/build/ui-bundle.zip?job=bundle-stable + snapshot: true + +asciidoc: + attributes: + page-pagination: '' + experimental: '' + idprefix: '' + idseparator: '-' + page-toclevels: 3 + # Reports a misspelled {attribute} (it renders as written either way); with the + # failure level below, it fails the build. + attribute-missing: warn + +runtime: + log: + # Stricter than docs.arcadedb.com (error): a warning in these pages fails the build. + failure_level: warn + +output: + dir: ./build/site diff --git a/docs/antora.yml b/docs/antora.yml index 810da11..ed8bcc9 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -24,3 +24,6 @@ asciidoc: native-pypi-url: https://pypi.org/project/arcadedb/ jvm-pypi-url: https://pypi.org/project/arcadedb-jvm/ which-package-url: https://github.com/ArcadeData/arcadedb-python/blob/main/docs/modules/ROOT/pages/which-package.adoc + # The main ArcadeDB manual. Pages link into it as {arcadedb-docs-url}/, not with an + # arcadedb:: xref, so the component also builds on its own (antora-playbook.yml here). + arcadedb-docs-url: https://docs.arcadedb.com/arcadedb diff --git a/docs/modules/ROOT/pages/development/ci.adoc b/docs/modules/ROOT/pages/development/ci.adoc index f53b179..ba5c001 100644 --- a/docs/modules/ROOT/pages/development/ci.adoc +++ b/docs/modules/ROOT/pages/development/ci.adoc @@ -18,6 +18,8 @@ Each package has one test workflow and one release workflow: The release workflows call the test workflows (`workflow_call`) and then publish; see xref:development/release.adoc[Release]. Two workflows test the `arcadedb-jvm` package: `test-jvm.yml` builds a wheel and runs the unit tests, and `test-python-examples.yml` runs the scripts in `examples/`. Both build the wheel once for each platform, on a native runner of that platform (`Build arcadedb-jvm wheel ()` and `Build wheel for examples ()`), and install that one file into each Python they test. `test-native.yml` builds and tests the `arcadedb` (native) package; see <<_native_build,Native build>>. +The workflow `docs.yml` builds these docs (the Antora component in `docs/`) as a site of its own with `docs/antora-playbook.yml`, on every pull request and push to `main` that changes `docs/`, and attaches the site as the `docs-site` artifact. Download it from the run to read a page as it will render. The test workflows ignore `docs/**` and `*.adoc`, so for a docs-only pull request this job is the only check. A warning fails it: a broken `xref:`, a misspelled `+{attribute}+`, an AsciiDoc syntax problem. Pages link into the main ArcadeDB manual as `+{arcadedb-docs-url}/+`, not with an `arcadedb::` xref, which the component on its own cannot resolve. To build the site locally, run `npx -p @antora/cli@3.2.1 -p @antora/site-generator@3.2.1 antora docs/antora-playbook.yml` from the repository root and open `docs/build/site/index.html`. + The package supports four platforms and four Python versions (3.11 to 3.14). That is 16 combinations. The organisation shares one runner limit with ArcadeData/arcadedb, so each macOS or Windows job delays the upstream jobs too, and not every combination runs. == One wheel for each platform diff --git a/docs/modules/ROOT/pages/how-to/bulk-loads.adoc b/docs/modules/ROOT/pages/how-to/bulk-loads.adoc index 74b6058..12e665e 100644 --- a/docs/modules/ROOT/pages/how-to/bulk-loads.adoc +++ b/docs/modules/ROOT/pages/how-to/bulk-loads.adoc @@ -110,7 +110,7 @@ This section is about a `graph_batch()` load that stops before `close()` returns * The vertices and edge records that a returned call committed are there. A load that is killed loses the work after the last commit and nothing before it. * The edges are not traversable. The engine writes the edge records first and links them to their vertices (the head pointers and the incoming lists) in `close()`. Until `close()` has run, `outE()`, `inE()`, `out()` and `MATCH` find none of them, but `SELECT count(*) FROM E` counts them. In the example: 70,000 edge records, 0 reachable. -* `CHECK DATABASE` reports each of these edges as an `ORPHAN RECORD` and counts them in `unreachableEdgeRecords`. It also uses the words "was not connected from", "is not referenced back by" and "missing from that vertex's OUT/IN list" for the same state. Read the statement in the xref:arcadedb::reference/sql/sql-database-admin.adoc#sql-check-database[`CHECK DATABASE` reference]; this page does not repeat it. +* `CHECK DATABASE` reports each of these edges as an `ORPHAN RECORD` and counts them in `unreachableEdgeRecords`. It also uses the words "was not connected from", "is not referenced back by" and "missing from that vertex's OUT/IN list" for the same state. Read the statement in the {arcadedb-docs-url}/reference/sql/sql-database-admin#sql-check-database[`CHECK DATABASE` reference]; this page does not repeat it. === Recovery that works diff --git a/docs/modules/ROOT/pages/how-to/concurrency.adoc b/docs/modules/ROOT/pages/how-to/concurrency.adoc index baccb37..33f909d 100644 --- a/docs/modules/ROOT/pages/how-to/concurrency.adoc +++ b/docs/modules/ROOT/pages/how-to/concurrency.adoc @@ -1,7 +1,7 @@ = Concurrency, processes and crashes :description: What the embedded bindings do with threads, asyncio, several processes, signals and crashes, as measured, and how to write an update that two writers cannot lose. -This page covers what is specific to the Python bindings: which calls work from several threads, from asyncio or from several processes, what a signal or a crash does to the process and the data, and how to write an update that concurrent writers cannot lose. What the engine guarantees at each isolation level is in the main ArcadeDB docs: see xref:arcadedb::concepts/transactions.adoc#isolation[Isolation]. +This page covers what is specific to the Python bindings: which calls work from several threads, from asyncio or from several processes, what a signal or a crash does to the process and the data, and how to write an update that concurrent writers cannot lose. What the engine guarantees at each isolation level is in the main ArcadeDB docs: see {arcadedb-docs-url}/concepts/transactions#isolation[Isolation]. == How to read the numbers @@ -52,7 +52,7 @@ All 7 runs ended with no error, no bad checksum and the same number of rows befo == Several writers -Two transactions that change the same record conflict at commit. The one that loses raises `ArcadeDBError`, and the name `ConcurrentModificationException` is in the message or in the `__cause__` chain. The engine treats it as retryable: see xref:arcadedb::concepts/transactions.adoc#isolation[Isolation] for when it is raised. `db.run_in_transaction(fn, retries=...)` runs `fn` in a transaction again after such a conflict. +Two transactions that change the same record conflict at commit. The one that loses raises `ArcadeDBError`, and the name `ConcurrentModificationException` is in the message or in the `__cause__` chain. The engine treats it as retryable: see {arcadedb-docs-url}/concepts/transactions#isolation[Isolation] for when it is raised. `db.run_in_transaction(fn, retries=...)` runs `fn` in a transaction again after such a conflict. The probe: 8 threads, one shared `Database`, 4 counter records. Each operation is one transaction that changes one counter, chosen by a fixed pseudo-random sequence, and inserts one row with a unique key into a ledger type. Each thread makes 1,500 operations (500 on Python 3.14t), so 12,000 in total (4,000 on Python 3.14t). The probe counts a lost update as an operation whose transaction committed but whose increment is not in the final counters. Each cell below is the number of lost updates out of the number of operations, from one run. @@ -203,7 +203,7 @@ Based on that: == Signals and crashes -A write that has returned from `commit()` survived every kill in the probe. The probe cannot say anything about a power loss or an operating system crash, which a process kill does not simulate. For how the engine flushes its write-ahead log, see xref:arcadedb::concepts/transactions.adoc#wal-flush-durability[WAL flush and durability]. +A write that has returned from `commit()` survived every kill in the probe. The probe cannot say anything about a power loss or an operating system crash, which a process kill does not simulate. For how the engine flushes its write-ahead log, see {arcadedb-docs-url}/concepts/transactions#wal-flush-durability[WAL flush and durability]. The kill probe: a child process writes in a loop, in transactions of 5 or 200 rows, and prints the number of rows after every commit that returned. The parent sends `SIGKILL` (kill -9) at a random time between 0.2 and 2.5 s after the first commit, then reopens the same database. The reopen must recover, `CHECK DATABASE` must report nothing, every acknowledged row must be present with a matching checksum, no transaction may be half applied, and the database must accept a write. The same database was used for all runs of a build. diff --git a/docs/modules/ROOT/pages/how-to/databases-and-queries.adoc b/docs/modules/ROOT/pages/how-to/databases-and-queries.adoc index c6a9bed..5424946 100644 --- a/docs/modules/ROOT/pages/how-to/databases-and-queries.adoc +++ b/docs/modules/ROOT/pages/how-to/databases-and-queries.adoc @@ -255,13 +255,13 @@ Page a large type by `@rid` instead of `SKIP`: each page is `WHERE @rid > ? LIMI === SQLScript -`db.command("sqlscript", script)` runs several statements in one call, and the result set holds the last executed statement, which is why the script above reports the `ALTER TYPE` statement. The script syntax is in the xref:arcadedb::reference/sql/sql-script.adoc[SQLScript reference]. +`db.command("sqlscript", script)` runs several statements in one call, and the result set holds the last executed statement, which is why the script above reports the `ALTER TYPE` statement. The script syntax is in the {arcadedb-docs-url}/reference/sql/sql-script[SQLScript reference]. === openCypher -The same two calls run openCypher: pass `"opencypher"` as the language. The script creates an edge with a `CREATE` pattern and reads it back with `MATCH`. The language is documented in the xref:arcadedb::reference/cypher/chapter.adoc[openCypher reference]. +The same two calls run openCypher: pass `"opencypher"` as the language. The script creates an edge with a `CREATE` pattern and reads it back with `MATCH`. The language is documented in the {arcadedb-docs-url}/reference/cypher/chapter[openCypher reference]. -NOTE: ArcadeDB SQL has no JOINs, so SQLAlchemy and other ORMs, and tools that generate `JOIN` queries, do not work with these two calls. Write the statements yourself, following links as in xref:arcadedb::reference/sql/sql-introduction.adoc#no-joins[No JOINs], or use openCypher for relationships. +NOTE: ArcadeDB SQL has no JOINs, so SQLAlchemy and other ORMs, and tools that generate `JOIN` queries, do not work with these two calls. Write the statements yourself, following links as in {arcadedb-docs-url}/reference/sql/sql-introduction#no-joins[No JOINs], or use openCypher for relationships. == Next steps diff --git a/docs/modules/ROOT/pages/index.adoc b/docs/modules/ROOT/pages/index.adoc index 4b00320..0a3cb70 100644 --- a/docs/modules/ROOT/pages/index.adoc +++ b/docs/modules/ROOT/pages/index.adoc @@ -73,9 +73,9 @@ ArcadeDB reaches Python in two ways. Pick by where the engine should run. | A running server (the official Docker image `arcadedata/arcadedb`), plus `pip install arcadedb-driver` |=== -Choose the embedded bindings for a single script, a notebook, or an application with one Python process that owns its data. Choose the xref:arcadedb::how-to/connectivity/drivers/python-http.adoc[HTTP driver] when more than one process, another language, or a remote machine must reach the same database, or when the server must outlive your Python process. +Choose the embedded bindings for a single script, a notebook, or an application with one Python process that owns its data. Choose the {arcadedb-docs-url}/how-to/connectivity/drivers/python-http[HTTP driver] when more than one process, another language, or a remote machine must reach the same database, or when the server must outlive your Python process. -The two meet in one case. The bindings bundle an optional HTTP server. It is off until you call `create_server()`. The process that owns the database keeps direct access, and other processes connect over HTTP. That server lives and dies with your Python process. For a database that must outlive any one client, or for HA, replication and TLS, run the regular xref:arcadedb::index.adoc[ArcadeDB server] instead. +The two meet in one case. The bindings bundle an optional HTTP server. It is off until you call `create_server()`. The process that owns the database keeps direct access, and other processes connect over HTTP. That server lives and dies with your Python process. For a database that must outlive any one client, or for HA, replication and TLS, run the regular {arcadedb-docs-url}/[ArcadeDB server] instead. [[jvm-vs-native]] == Native build or JVM build diff --git a/docs/modules/ROOT/pages/installation.adoc b/docs/modules/ROOT/pages/installation.adoc index 134ef7a..9a2e86a 100644 --- a/docs/modules/ROOT/pages/installation.adoc +++ b/docs/modules/ROOT/pages/installation.adoc @@ -177,7 +177,7 @@ export ARCADEDB_NATIVE_ISOLATE_ARGS="-XX:MaximumHeapSizePercent=70" # another p export ARCADEDB_NATIVE_ISOLATE_ARGS="-XX:MaxRAM=4g" # a memory size for the percentage; GraalVM's 80% applies to it ---- -When one of these three is present, the package adds no default. Do not use `-XX:MaxRAMPercentage`. The native image does not accept it, and the isolate then fails to start with `graal_create_isolate failed (rc=22)`. The same variable also takes the engine's `-D` settings: see xref:how-to/logs.adoc[Where the logs go]. For what a native image build supports, see xref:arcadedb::how-to/operations/native-image.adoc[Native image] in the main ArcadeDB docs. +When one of these three is present, the package adds no default. Do not use `-XX:MaxRAMPercentage`. The native image does not accept it, and the isolate then fails to start with `graal_create_isolate failed (rc=22)`. The same variable also takes the engine's `-D` settings: see xref:how-to/logs.adoc[Where the logs go]. For what a native image build supports, see {arcadedb-docs-url}/how-to/operations/native-image[Native image] in the main ArcadeDB docs. [[native-limits]] === Limits of the native build diff --git a/docs/modules/ROOT/pages/quickstart.adoc b/docs/modules/ROOT/pages/quickstart.adoc index ca6ea42..bd9325c 100644 --- a/docs/modules/ROOT/pages/quickstart.adoc +++ b/docs/modules/ROOT/pages/quickstart.adoc @@ -25,7 +25,7 @@ The scripts on this page ran against the JVM build, version 26.10.2.post1 (forme == Embedded mode -The database runs in your Python process. You use SQL for the schema and for CRUD, through two calls. ArcadeDB SQL is its own dialect, and it has no JOINs: records link to each other and a query follows the links (see xref:arcadedb::reference/sql/sql-introduction.adoc#no-joins[No JOINs]). +The database runs in your Python process. You use SQL for the schema and for CRUD, through two calls. ArcadeDB SQL is its own dialect, and it has no JOINs: records link to each other and a query follows the links (see {arcadedb-docs-url}/reference/sql/sql-introduction#no-joins[No JOINs]). * `db.command(language, statement, *params)` runs a statement that can change data or schema. * `db.query(language, statement, *params)` runs a read. diff --git a/docs/modules/ROOT/pages/which-package.adoc b/docs/modules/ROOT/pages/which-package.adoc index a43f417..d1f2a38 100644 --- a/docs/modules/ROOT/pages/which-package.adoc +++ b/docs/modules/ROOT/pages/which-package.adoc @@ -149,6 +149,6 @@ Releases of the new layout start with 26.11.1. The 26.10.x wheels were built in None of these contains the engine. They connect to a running ArcadeDB server. -* `{driver-pkg-name}`: the official HTTP client. See xref:arcadedb::how-to/connectivity/drivers/python-http.adoc[Python -- HTTP driver]. -* `{grpc-driver-pkg-name}`: the official gRPC client. See xref:arcadedb::how-to/connectivity/drivers/python-grpc.adoc[Python -- gRPC driver]. +* `{driver-pkg-name}`: the official HTTP client. See {arcadedb-docs-url}/how-to/connectivity/drivers/python-http[Python -- HTTP driver]. +* `{grpc-driver-pkg-name}`: the official gRPC client. See {arcadedb-docs-url}/how-to/connectivity/drivers/python-grpc[Python -- gRPC driver]. * `{community-driver-pkg-name}`: a community HTTP driver by Steve Reiner and others ({community-repo-url}[stevereiner/arcadedb-python]). It is not related to this repository, whose name is also `arcadedb-python`. Check which one a tutorial installs before you copy its code.