Support complete prompt templates for local text and image functions

.github/workflows/test.yml

@@ -93,6 +93,7 @@ jobs:
             tests/test_compile_timeouts.py \
             tests/test_local_program.py \
             tests/test_base_interpreter.py \
+            tests/test_numbered_templates.py \
             tests/test_cli_auth.py \
             tests/test_remote_inference.py \
             tests/test_desktop_sdk.py \
@@ -102,6 +103,7 @@ jobs:
             tests/test_vision_metadata.py \
             tests/test_vision_assets.py \
             tests/test_vision_runtime.py \
+            tests/test_vision_prompt.py \
             tests/test_vision_result.py \
             tests/test_sdk.py::TestInstallAndImport
 

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## Unreleased
+
+- Add `prompt_template=` for local adapter-free text and image interpreters,
+  using shared numbered positional placeholders with repeatable, ordered slots.
+- Execute complete image prompt templates directly, without an added chat
+  wrapper. Older experimental image bundles must be re-exported with the
+  `rendered_text` / `{INPUT_N}` contract and a complete prompt template.
+- Preserve existing compiled text templates, call conventions and prefix caches.
+
 ## 0.4.12 (2026-10-07)
 
 - Add local Qwen3.5-0.8B functions with ordered text and `paw.Image` inputs,

PYPI_README.md

@@ -125,17 +125,29 @@ Install the optional image dependencies:
 pip install "programasweights[vision]" --extra-index-url https://pypi.programasweights.com/simple/
 ```
 
-Prompt Qwen3.5-0.8B locally with text and images (no adapter applied):
+The unreleased numbered-template API lets you specify the complete prompt for
+text-only or image-capable base models. For Qwen3.5-0.8B with no adapter:
 
 ```python
 import programasweights as paw
 
-describe = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B")
+prompt = (
+    "<|im_start|>user\n{INPUT_0}\n{INPUT_1}<|im_end|>\n"
+    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
+)
+describe = paw.function(
+    None, interpreter="Qwen/Qwen3.5-0.8B", prompt_template=prompt,
+)
 print(describe("Describe this image.", paw.Image("photo.png")))
 ```
 
-To use a compiled image program, load its `.paw` file with
-`paw.function("./locator.paw")`.
+`{INPUT_0}`, `{INPUT_1}`, etc. insert positional arguments into the prompt.
+Placeholders can repeat or appear in any order. The template supplies the
+model's role markers and assistant prefix. For text-only models such as
+`Qwen/Qwen3-0.6B`, pass string inputs.
+
+Load a compiled image program with `paw.function("./locator.paw")`, then pass
+the arguments required by its prompt template.
 
 See the [image API reference](https://programasweights.readthedocs.io/en/latest/api-reference/python-sdk/#local-textimage-calls)
 for multiple images, input formats, and result metadata.

README.md

@@ -125,17 +125,29 @@ Install the optional image dependencies:
 pip install "programasweights[vision]" --extra-index-url https://pypi.programasweights.com/simple/
 ```
 
-Prompt Qwen3.5-0.8B locally with text and images (no adapter applied):
+The unreleased numbered-template API lets you specify the complete prompt for
+text-only or image-capable base models. For Qwen3.5-0.8B with no adapter:
 
 ```python
 import programasweights as paw
 
-describe = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B")
+prompt = (
+    "<|im_start|>user\n{INPUT_0}\n{INPUT_1}<|im_end|>\n"
+    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
+)
+describe = paw.function(
+    None, interpreter="Qwen/Qwen3.5-0.8B", prompt_template=prompt,
+)
 print(describe("Describe this image.", paw.Image("photo.png")))
 ```
 
-To use a compiled image program, load its `.paw` file with
-`paw.function("./locator.paw")`.
+`{INPUT_0}`, `{INPUT_1}`, etc. insert positional arguments into the prompt.
+Placeholders can repeat or appear in any order. The template supplies the
+model's role markers and assistant prefix. For text-only models such as
+`Qwen/Qwen3-0.6B`, pass string inputs.
+
+Load a compiled image program with `paw.function("./locator.paw")`, then pass
+the arguments required by its prompt template.
 
 See the [image API reference](docs/api-reference/python-sdk.md#local-textimage-calls)
 for multiple images, input formats, and result metadata.

docs/api-reference/python-sdk.md

@@ -23,9 +23,9 @@ fn = paw.function(
     n_gpu_layers=None,
     verbose=False,
     offline=False,
-    *,
     remote=False,
     interpreter=None,
+    prompt_template=None,
 )
 ```
 
@@ -43,8 +43,10 @@ bundle directly. Required runtime metadata and base models are cached for reuse.
 | `offline` | Use only local files/cache and make zero network calls; fail if required validated assets are missing. `PAW_OFFLINE=1` has the same effect. |
 | `remote` | Run hosted inference without downloading model assets (default `False`). Accepts a `Program` object, ID, or slug. Cannot be combined with offline mode, local file paths, `interpreter`, or non-default local runtime options. |
 | `interpreter` | Advanced adapter-free mode only. Must be passed by keyword and only when `program_id` is explicitly `None`. Supported values are `Qwen/Qwen3-0.6B`, `gpt2`, and `Qwen/Qwen3.5-0.8B`. |
+| `prompt_template` | Unreleased. Optional complete numbered template for a local base interpreter. Pass by keyword with `program_id=None`. Works with text-only and image-capable models. Cannot override a compiled bundle or be used with `remote=True`. |
 
-For local text-only programs, the returned callable accepts:
+For existing local text-only programs and text base interpreters without an
+explicit `prompt_template`, the returned callable accepts:
 
 ```python
 output: str = fn(input_text, max_tokens=None, temperature=0.0, logits_processor=None)
@@ -148,35 +150,131 @@ The text interpreters format prompts as follows:
 ```
 
 The Qwen3 interpreter uses its chat template with thinking disabled. GPT-2 uses
-the input as a raw prompt.
+the input as a raw prompt. These defaults are unchanged when `prompt_template`
+is omitted.
+
+#### Complete numbered templates (unreleased)
+
+For a local base model, pass `prompt_template=` when loading the function to
+specify the complete prompt. This option works for both text-only and
+image-capable interpreters:
+
+```python
+prompt = (
+    "<|im_start|>system\n{INPUT_0}<|im_end|>\n"
+    "<|im_start|>user\n{INPUT_1}<|im_end|>\n"
+    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
+)
+answer = paw.function(
+    None, interpreter="Qwen/Qwen3-0.6B", prompt_template=prompt,
+)
+output = answer(
+    "Answer using only the facts in the question.",
+    "Alice owns three cats. How many cats does Alice own?",
+)
+```
+
+The template controls role delimiters, examples, whitespace, thinking markers
+and the assistant prefix. The SDK adds no chat wrapper. Use the format expected
+by the selected model. Text is tokenized with special tokens recognized and
+without an automatically added BOS prefix. Image-capable models insert the
+required vision tokens and embeddings at image slots.
+
+Numbered templates share these rules across text and image models:
+
+- `{INPUT_0}`, `{INPUT_1}`, etc. refer to positional arguments. Used indices must
+  start at zero without gaps; `{INPUT_1}/{INPUT_0}/{INPUT_1}` is valid.
+- Pass exactly the number of distinct indices. Repeating a slot repeats its
+  content at that position. A nonempty constant template may have no slots and
+  be called with `fn()`.
+- Input strings are inserted literally and are not parsed again for slots.
+  Only canonical numbered placeholders are interpreted. Other brace content is
+  literal; this is not Python `str.format`. Doubled braces do not escape a slot:
+  `{{INPUT_0}}` surrounds the inserted value with literal braces.
+- Text-only models accept strings. Image-capable models accept strings or
+  `paw.Image` in any slot, including calls containing only strings.
+- Generation options are keyword-only for numbered-template calls:
+  `fn("first", "second", max_tokens=32)`.
+
+For an already rendered text prompt, use `prompt_template="{INPUT_0}"`.
+Qwen3.5's default base template is this identity template: it accepts one
+argument unchanged and supplies no conversation wrapper. GPT-2 and Qwen3-0.6B
+keep the default text templates shown above when the option is omitted.
+
+`prompt_template=` is available only with `program_id=None` and local inference.
+Compiled programs use their bundled `prompt_template.txt`; passing an override
+with a compiled program or `remote=True` raises an error.
+
+Existing compiled text programs still require exactly one
+`{INPUT_PLACEHOLDER}`. Their literal text, tokenization boundaries and prefix
+cache behavior are unchanged. Existing text calls such as `fn(input_text="x")`
+and `fn("x", 32)` remain valid; explicit numbered templates use the positional
+inputs and keyword generation options described above.
 
 ### Local text/image calls
 
-Install `programasweights[vision]` to prompt Qwen3.5-0.8B locally
-(no adapter applied). Pass text and `paw.Image` objects in the order
-the model should read them:
+Install `programasweights[vision]` to use Qwen3.5-0.8B. With the unreleased
+numbered-template API, order images and text through the complete template:
 
 ```python
-compare = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B")
+prompt = (
+    "<|im_start|>user\n"
+    "Before: {INPUT_1}\nAfter: {INPUT_2}\n{INPUT_0}<|im_end|>\n"
+    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
+)
+compare = paw.function(
+    None, interpreter="Qwen/Qwen3.5-0.8B", prompt_template=prompt,
+)
 answer = compare(
-    "Before:", paw.Image("before.png"),
-    "After:", paw.Image("after.png"),
-    "What changed?",
+    "What changed?", paw.Image("before.png"), paw.Image("after.png"),
 )
 print(answer)
 ```
 
-Use the same inputs with a compiled image program:
+Here the question is argument zero, but the model receives both images before
+that question. Repeating `{INPUT_1}` would place the first image at each
+occurrence. There is no separate public image marker to insert manually.
+
+For a compiled image program, `prompt_template.txt` contains the complete
+prompt, for example:
+
+```text
+<|im_start|>system
+Locate the requested object.<|im_end|>
+<|im_start|>user
+{INPUT_0}
+{INPUT_1}<|im_end|>
+<|im_start|>assistant
+<think>
+
+</think>
+
+```
+
+With that template, the call is:
 
 ```python
 fn = paw.function("./locator.paw")
 answer = fn("Find the red cup.", paw.Image("scene.png"))
 ```
 
+The image runtime manifest declares the template contract as:
+
+```json
+"prompt_template": {
+  "format": "rendered_text",
+  "placeholder": "{INPUT_N}"
+}
+```
+
+This is the template field inside the full runtime manifest, not a standalone
+manifest. Keep the required model, projector, preprocessing and adapter metadata.
+
 `paw.Image(source)` accepts a local file path, encoded image bytes, or a Pillow
 image. For an image URL, download the file first. Plain strings are text inputs.
 
-Image functions return a string. Generation options are passed by keyword:
+Image functions return a string. Their positional argument count and order
+come from the template. Generation options are passed by keyword:
 
 ```python
 output: str = fn(

programasweights/__init__.py

@@ -426,6 +426,7 @@ def function(
     *,
     remote: bool = False,
     interpreter: str | None = None,
+    prompt_template: str | None = None,
 ):
     """Load a compiled program, or explicitly load a bare base interpreter.
 
@@ -453,13 +454,22 @@ def function(
         interpreter: Advanced adapter-free mode. This is only valid when
             ``program_id`` is explicitly ``None``. Supported values are
             ``"Qwen/Qwen3-0.6B"``, ``"gpt2"``, and ``"Qwen/Qwen3.5-0.8B"``.
-            Qwen3.5 processes text and ``paw.Image`` arguments in the order
-            provided.
+            Qwen3.5 places text and ``paw.Image`` arguments according to
+            its complete numbered prompt template.
+        prompt_template: Complete numbered template for a local adapter-free
+            interpreter, including text-only models. {INPUT_0}, {INPUT_1}, ...
+            bind positional arguments; generation options are keyword-only
+            when this is supplied. Text-only models require string arguments;
+            image-capable models also accept paw.Image. No chat wrapper is
+            added. Omitting this option preserves the interpreter's default
+            template and call convention. Compiled functions use their bundled
+            prompt_template.txt and do not accept this override.
 
     Returns:
         A callable returning an output string. Text programs accept one input
-        string. Image programs accept ordered positional strings and paw.Image
-        objects, with generation options passed by keyword.
+        string by default. Explicit numbered templates accept positional
+        strings; image programs also accept paw.Image objects. Numbered
+        template calls take generation options by keyword.
 
     Example:
         >>> fn = paw.function("email-triage")
@@ -475,11 +485,26 @@ def function(
         >>> base = paw.function(None, interpreter="gpt2")
 
         >>> vision = paw.function(None, interpreter="Qwen/Qwen3.5-0.8B")
-        >>> vision("Describe this image.", paw.Image("photo.png"))
+        >>> vision(paw.Image("photo.png"))  # default template is {INPUT_0}
+
+        >>> prompted = paw.function(
+        ...     None, interpreter="gpt2",
+        ...     prompt_template="Context: {INPUT_0}. Question: {INPUT_1}. Answer:",
+        ... )
+        >>> prompted("Context goes here.", "What happened?", max_tokens=32)
     """
     import os
     from . import cache
 
+    if prompt_template is not None:
+        if remote or program_id is not None:
+            raise ValueError(
+                "prompt_template requires local adapter-free inference; "
+                "compiled functions use their bundled template."
+            )
+        if not isinstance(prompt_template, str):
+            raise TypeError("prompt_template must be a string.")
+
     offline = _offline_requested(offline)
     if remote:
         _validate_remote_options(
@@ -514,12 +539,16 @@ def function(
             from .runtime_llamacpp import PawFunction
 
             function_type = PawFunction
+        template_options = (
+            {} if prompt_template is None else {"prompt_template": prompt_template}
+        )
         return function_type.from_base(
             interpreter,
             n_ctx=n_ctx,
             n_gpu_layers=n_gpu_layers,
             verbose=verbose,
             offline=offline,
+            **template_options,
         )
 
     from ._program_reference import local_program_path

programasweights/_image_content.py

@@ -105,3 +105,32 @@ def _to_chat_content(*parts: Union[str, Image]) -> List[Dict[str, Any]]:
                 "image_url": {"url": _image_url(part, index)},
             })
     return content
+
+
+def _to_prompt(*parts: Union[str, Image], image_marker: str) -> tuple:
+    """Convert bound parts into exact prompt text and ordered image URLs."""
+    if not isinstance(image_marker, str) or not image_marker:
+        raise ValueError("The backend image marker must be a nonempty string.")
+
+    text = []
+    images = []
+    for index, part in enumerate(_normalize_parts(*parts), start=1):
+        if isinstance(part, str):
+            if image_marker in part:
+                raise ValueError(
+                    "The backend image marker is reserved; use paw.Image."
+                )
+            text.append(part)
+        else:
+            text.append(image_marker)
+            images.append((index, part))
+
+    prompt = "".join(text)
+    # Also catch a reserved marker formed across adjacent text parts.
+    if prompt.count(image_marker) != len(images):
+        raise ValueError(
+            "The backend image marker is reserved; use paw.Image."
+        )
+
+    urls = [_image_url(image, index) for index, image in images]
+    return prompt, urls

programasweights/_inputs.py

@@ -36,7 +36,9 @@ def _copy_pixels(source):
 class Image:
     """An image to pass to a local image function.
 
-    Accepts a file path, encoded image bytes, or a Pillow image. For example:
+    Accepts a file path, encoded image bytes, or a Pillow image. Pass it as a
+    positional argument to the function's numbered prompt template. For example,
+    when the template uses {INPUT_0} for the question and {INPUT_1} for an image:
     ``fn("Describe this image.", paw.Image("photo.png"))``.
     """