diff --git a/PYPI_README.md b/PYPI_README.md index a204306..2b695c4 100644 --- a/PYPI_README.md +++ b/PYPI_README.md @@ -112,9 +112,10 @@ may still download; `offline=True` prohibits those requests. Legacy tensor-forma `.paw` files are unsupported. Local-file inputs are supported by `function`, not `prepare_program` or `is_offline_ready`. -Advanced adapter-free inference is available with -`paw.function(None, interpreter="gpt2")`; see the Python API reference for -its intentionally strict semantics. +To run a base model without a compiled program, use +`paw.function(None, interpreter="gpt2")`. See the +[Python API reference](https://programasweights.readthedocs.io/en/latest/api-reference/python-sdk/#advanced-adapter-free-base-interpreter) +for supported models and options. ## Local image functions @@ -124,26 +125,20 @@ Install the optional image dependencies: pip install "programasweights[vision]" --extra-index-url https://pypi.programasweights.com/simple/ ``` -Prompt Qwen3.5-0.8B without an adapter: +Prompt Qwen3.5-0.8B locally with text and images (no adapter applied): ```python import programasweights as paw -with paw.function(None, interpreter="Qwen/Qwen3.5-0.8B") as describe: - print(describe("Describe this image.", paw.Image("photo.png"), max_tokens=128)) +describe = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B") +print(describe("Describe this image.", paw.Image("photo.png"))) ``` -Or load a compatible image `.paw` bundle with `paw.function("./locator.paw")`. -Image functions accept ordered positional strings and `paw.Image` objects; -generation options are keyword-only. `paw.Image` snapshots a local path, -encoded bytes, or a Pillow image. The SDK keeps native image dimensions and -composites transparency over white; the model backend constructs image patches. - -The first load downloads the exact model and projector. Use `offline=True` -after caching both files. Matching model/projector hashes and runtime settings -share one local runtime; calls reset model state and select the correct LoRA. -Use context managers or `close()` to release it. Image inference is local only; -this SDK change does not add a hosted image compiler or remote image inference. +To use a compiled image program, load its `.paw` file with +`paw.function("./locator.paw")`. + +See the [image API reference](https://programasweights.readthedocs.io/en/latest/api-reference/python-sdk/#local-textimage-calls) +for multiple images, input formats, and result metadata. ## Browser SDK diff --git a/README.md b/README.md index a77f090..cde090b 100644 --- a/README.md +++ b/README.md @@ -112,10 +112,10 @@ may still download; `offline=True` prohibits those requests. Legacy tensor-forma `.paw` files are unsupported. Local-file inputs are supported by `function`, not `prepare_program` or `is_offline_ready`. -Advanced adapter-free inference is available with -`paw.function(None, interpreter="gpt2")`; see the +To run a base model without a compiled program, use +`paw.function(None, interpreter="gpt2")`. See the [Python API reference](docs/api-reference/python-sdk.md#advanced-adapter-free-base-interpreter) -for its intentionally strict semantics. +for supported models and options. ## Local image functions @@ -125,26 +125,20 @@ Install the optional image dependencies: pip install "programasweights[vision]" --extra-index-url https://pypi.programasweights.com/simple/ ``` -Prompt Qwen3.5-0.8B without an adapter: +Prompt Qwen3.5-0.8B locally with text and images (no adapter applied): ```python import programasweights as paw -with paw.function(None, interpreter="Qwen/Qwen3.5-0.8B") as describe: - print(describe("Describe this image.", paw.Image("photo.png"), max_tokens=128)) +describe = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B") +print(describe("Describe this image.", paw.Image("photo.png"))) ``` -Or load a compatible image `.paw` bundle with `paw.function("./locator.paw")`. -Image functions accept ordered positional strings and `paw.Image` objects; -generation options are keyword-only. `paw.Image` snapshots a local path, -encoded bytes, or a Pillow image. The SDK keeps native image dimensions and -composites transparency over white; the model backend constructs image patches. - -The first load downloads the exact model and projector. Use `offline=True` -after caching both files. Matching model/projector hashes and runtime settings -share one local runtime; calls reset model state and select the correct LoRA. -Use context managers or `close()` to release it. Image inference is local only; -this SDK change does not add a hosted image compiler or remote image inference. +To use a compiled image program, load its `.paw` file with +`paw.function("./locator.paw")`. + +See the [image API reference](docs/api-reference/python-sdk.md#local-textimage-calls) +for multiple images, input formats, and result metadata. ## Browser SDK diff --git a/docs/api-reference/python-sdk.md b/docs/api-reference/python-sdk.md index 6438117..55296c9 100644 --- a/docs/api-reference/python-sdk.md +++ b/docs/api-reference/python-sdk.md @@ -61,17 +61,6 @@ This advanced hook is not built-in regex or JSON-schema validation. Each process **Context limits:** Spec + input + output share a ~2048 token window. Inputs that exceed it will error. `max_tokens` defaults to `None`: generation runs until EOS or the context limit. -Compiled mode is strict: the adapter, prompt template, matching metadata, -runtime manifest, and runtime-compatible base-model file must all validate. Version 0.4.5 -introduced runtime manifest version 1 with `adapter_format="gguf_lora"`. -Image bundles use version 2 and additionally pin the projector, ordered-content -prompt contract, and preprocessing settings. -Built-in models are checked against pinned size/SHA-256 metadata and GGUF -magic. Historical manifests for those known runtime IDs are normalized to the -same canonical integrity metadata, so missing server-side checksum fields -cannot weaken validation. Missing or failed adapters raise an error; the SDK -never silently falls back to an unadapted base model. - ### Remote inference ```python @@ -139,17 +128,10 @@ base = paw.function(None, interpreter="gpt2") output = base("raw prompt text") ``` -This mode is intentionally explicit: - -- `paw.function()` still requires the `program_id` argument. -- `program_id=None` without `interpreter` raises `ValueError`. -- `program_id=""` raises `ValueError` and explains that base mode requires explicit `None`. -- A non-empty program reference together with `interpreter` raises `ValueError`. -- No PAW API, slug lookup, program download, adapter load, or disk prefix cache is used. -- Online mode may download the selected base GGUF and, for image interpreters, its projector from the built-in runtime manifest. Offline mode never downloads. -- Every invocation resets model state. Text interpreters render and tokenize the complete text prompt; the image interpreter processes ordered text/image content. +Model files download on first use. Pass `offline=True` to use cached files only. +Each call is independent. -The built-in text prompt contracts are versioned with each runtime manifest and contain exactly one `{INPUT_PLACEHOLDER}`: +The text interpreters format prompts as follows: ```text # Qwen/Qwen3-0.6B @@ -165,35 +147,36 @@ The built-in text prompt contracts are versioned with each runtime manifest and {INPUT_PLACEHOLDER} ``` -The Qwen bytes are the exact raw-user rendering of -`apply_chat_template(add_generation_prompt=True, enable_thinking=False)`. -Zero-token prompts and prompts that consume the full context window raise -`ValueError`. +The Qwen3 interpreter uses its chat template with thinking disabled. GPT-2 uses +the input as a raw prompt. ### Local text/image calls -Install `programasweights[vision]` to use Qwen3.5 image inference. The public -call accepts ordered strings and `paw.Image` objects: +Install `programasweights[vision]` to prompt Qwen3.5-0.8B locally +(no adapter applied). Pass text and `paw.Image` objects in the order +the model should read them: ```python -with paw.function(None, interpreter="Qwen/Qwen3.5-0.8B") as compare: - answer = compare( - "Before:", paw.Image("before.png"), - "After:", paw.Image("after.png"), - "What changed?", - max_tokens=128, - ) +compare = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B") +answer = compare( + "Before:", paw.Image("before.png"), + "After:", paw.Image("after.png"), + "What changed?", +) +print(answer) ``` -The adapter-free function sends one user message containing these parts, with -no injected system message. The pinned model chat template adds the assistant -prefix with thinking disabled. It also accepts text-only calls. Each call is -independent; no conversation, image, or recurrent-state cache carries across calls. +Use the same inputs with a compiled image program: + +```python +fn = paw.function("./locator.paw") +answer = fn("Find the red cup.", paw.Image("scene.png")) +``` -A compatible image bundle loads through `paw.function("./locator.paw")` and -uses the same call interface. Its `prompt_template.txt` provides literal system -instructions; `{INPUT_PLACEHOLDER}` has no substitution meaning in that file. -Bundle loading still requires a valid adapter and matching runtime metadata. +`paw.Image(source)` accepts a local file path, encoded image bytes, or a Pillow +image. For an image URL, download the file first. Plain strings are text inputs. + +Image functions return a string. Generation options are passed by keyword: ```python output: str = fn( @@ -206,57 +189,37 @@ output: str = fn( ) ``` -Generation options are keyword-only for image functions. `response_format` -passes a llama.cpp-compatible JSON response-format specification to the backend; -always validate returned data, especially when `max_tokens` truncates the output. -`max_tokens=0` returns an empty string without generation. Empty input lists and -parts other than strings or `paw.Image` are rejected; unpack an existing list -with `fn(*parts)`. +`max_tokens`, `temperature`, and `logits_processor` work as described above for +text functions. Use `response_format={"type": "json_object"}` to request JSON +output. Pass `return_info=True` to receive a `paw.FunctionResult` instead of a string: ```python result = fn("Find the target:", paw.Image("scene.png"), return_info=True) print(result.text) -print(result.finish_reason) # Backend reason, e.g. "stop" or "length"; otherwise None. -print(result.usage) # Backend token counts, or None when unavailable. -print(result.elapsed_seconds) # Full call duration through native cleanup. +print(result.finish_reason) # e.g. "stop" or "length"; None when unavailable. +print(result.usage) # Token counts, or None when unavailable. +print(result.elapsed_seconds) # Total call time in seconds. ``` -The result is immutable and belongs to that call. `usage`, when present, is a -copied, read-only mapping; use `dict(result.usage)` when logging it as JSON. -Counts have the backend's semantics; they are not an estimate of image patches -or billable tokens. `elapsed_seconds` uses a monotonic clock and includes input -preparation, waiting for function/runtime locks, adapter selection, inference, -and native cleanup. It excludes loading the function and constructing input -`paw.Image` objects before the call. - -`return_info` is a keyword-only boolean and defaults to `False`, preserving -string returns. With `max_tokens=0`, the opt-in result has empty text and `None` -for both backend fields, because no generation ran. Truncated text is still -returned with the backend's `"length"` reason; errors still raise normally. -This option applies to the local Qwen3.5 image runtime, including text-only calls -to that interpreter and compiled image programs. Existing text runtimes and -remote inference do not accept this option. - -`paw.Image(source)` accepts a regular local path, encoded image bytes, or a -Pillow image and snapshots the supplied content. It does not fetch URLs. Images -are decoded locally and converted to RGB PNG without SDK resizing, EXIF rotation, -or ICC conversion. Transparency is composited over white. The backend uses the -manifest's image-token limits to construct patches; high-bit-depth and floating -point inputs require explicit conversion by the caller. - -Live functions share a model/projector only when the exact asset hashes, -preprocessing, context size, GPU settings, and verbosity match. Shared calls -serialize and switch adapters; closing one function leaves other users of that -runtime usable. Close the last function to release native resources. Native -runtimes must not be inherited through `fork`; use a spawned process instead. - -The built-in image interpreter pins a Q8_0 model and BF16 projector by repository -revision, size, and SHA-256. Existing bundles retain their own exact asset -identities; naming the same upstream model does not make different GGUF files -interchangeable. Offline mode requires both verified files. Remote image -inference and hosted image compilation are not introduced by this interface. +The result and its `usage` mapping are read-only. Use `dict(result.usage)` +for JSON serialization when token counts are available. `elapsed_seconds` +measures the full call, excluding model loading and construction of +`paw.Image` inputs. + +`return_info` is available for local Qwen3.5 functions, including text-only calls +and compiled image programs. A `finish_reason` of `"length"` means generation +reached a token limit. + +#### Offline use and closing functions + +Model files download on first use. Once cached, pass `offline=True` to +`paw.function` to load without network access. + +Call `fn.close()` when finished, or use `with paw.function(...) as fn:` to close +automatically. For multiprocessing, start workers with `spawn` and load the +function inside each worker. ## Preparing programs for offline use diff --git a/programasweights/__init__.py b/programasweights/__init__.py index f4820a7..7599c41 100644 --- a/programasweights/__init__.py +++ b/programasweights/__init__.py @@ -453,8 +453,8 @@ def function( interpreter: Advanced adapter-free mode. This is only valid when ``program_id`` is explicitly ``None``. Supported values are ``"Qwen/Qwen3-0.6B"``, ``"gpt2"``, and ``"Qwen/Qwen3.5-0.8B"``. - Qwen3.5 accepts ordered text/image parts as one user message, - without a system prompt or an adapter. + Qwen3.5 processes text and ``paw.Image`` arguments in the order + provided. Returns: A callable returning an output string. Text programs accept one input diff --git a/programasweights/_inputs.py b/programasweights/_inputs.py index 3e6a98e..0a5ba09 100644 --- a/programasweights/_inputs.py +++ b/programasweights/_inputs.py @@ -34,19 +34,10 @@ def _copy_pixels(source): @dataclass(frozen=True, init=False, repr=False, eq=False) class Image: - """Snapshot an image supplied as a local path, encoded bytes, or PIL image. + """An image to pass to a local image function. - Paths are read immediately and encoded bytes are retained unchanged. Pillow - images are copied in their current mode and at their current frame, including - palette and metadata. Later changes to the caller's file or image do not - affect this value. URLs, file handles, and mutable byte buffers are not input - types; a plain string outside Image remains text. - - Encoded data is not decoded or validated here. No resizing, color conversion, - or model preprocessing is performed. Copying a lazily opened Pillow image - may decode its current frame. Paths and encoded bytes do not require Pillow. - - This input type does not itself enable image inference in text functions. + Accepts a file path, encoded image bytes, or a Pillow image. For example: + ``fn("Describe this image.", paw.Image("photo.png"))``. """ _source: Union[bytes, _PILImage] diff --git a/programasweights/_result.py b/programasweights/_result.py index a6cb614..5f719f7 100644 --- a/programasweights/_result.py +++ b/programasweights/_result.py @@ -7,11 +7,10 @@ @dataclass(frozen=True) class FunctionResult: - """Text and optional backend metadata from one successful image-runtime call. + """Text, token counts, and timing returned with ``return_info=True``. - ``usage`` is a copied, read-only mapping of backend token counts, or None - when unavailable. ``elapsed_seconds`` covers the call through native cleanup, - including input preparation and lock waits, but not function loading. + ``usage`` is a read-only mapping of token counts, or None when unavailable. + ``elapsed_seconds`` is the total call duration, excluding model loading. """ text: str diff --git a/programasweights/_runtime_vision.py b/programasweights/_runtime_vision.py index 3faa01d..75a0313 100644 --- a/programasweights/_runtime_vision.py +++ b/programasweights/_runtime_vision.py @@ -246,12 +246,11 @@ def close(self): class VisionFunction: - """A compiled image program or base interpreter, with ordered parts. + """A local function accepting text and Image arguments. - Generation options are keyword-only. Calls return text (including partial - text when max_tokens is exhausted); callers validate structured outputs. - Opt into a per-call FunctionResult with return_info=True. - Close functions, or use a context manager, to release shared model memory. + Calls return a string, or a FunctionResult with ``return_info=True``. + Pass generation options by keyword. Use ``close()`` or a ``with`` block + to close the function when finished. """ def __init__(self, program_dir, n_ctx=2048, n_gpu_layers=-1, verbose=False,