Skip to content

Fix wrong answers in the task model and heuristics - #165

Merged
guilyx merged 5 commits into
mainfrom
feat/zen-gates-g3lolk
Nov 3, 2026
Merged

guilyx merged 5 commits into
mainfrom
feat/zen-gates-g3lolk

Conversation

@guilyx

@guilyx guilyx commented Nov 3, 2026

Copy link
Copy Markdown
Member

What and why

The full code review found a set of bugs where jupyddl returns a wrong answer without complaint: a plan that shouldn't exist, an "optimal" cost that isn't, a heuristic that overestimates or crashes. validate_plan didn't catch any of them, because it replays plans through the same Task.apply that has the bug.

Bug Before After
Derived facts never retracted off deletes (on) but (light) stays true; (not (light)) is true from the start, so bfs "solves" an unsolvable task Axioms are recomputed from scratch every step and negated derived facts are maintained. Negation inside a rule needs stratification, so it is refused by name
Atom both added and deleted move(a,a) leaves (at a) and (not (at a)) true, so a 2-step plan "beats" the real 3-step optimum. Same for (and (not p) (when q p)) For complement facts, the delete wins
Actions with no cost effect under a (total-cost) metric Charged 1, so pegsol and sokoban optimised the wrong objective Cost 0, as in IPC planners. With no metric, every action still costs 1
(increase (total-cost) 10) inside when Charged even when the condition is false Refused
Negative action cost Accepted, then dijkstra spins until the budget runs out Refused
Fluent never defined, e.g. (distance a c) Read as 0, so a road that doesn't exist is free Conditions and effects that read it fail, as PDDL 2.1 specifies
blind Skipped zero-cost actions and overestimated, so astar+blind returned cost 3 where 2 is optimal Takes the min over all costs, including 0
h1/h2/hm Crash on every numeric or timed task Fixed
LM-cut with an empty goal Crash, which also crashed the default solve() Returns 0
lmcut/h2 admissible flag Hard-coded True, even with conditional effects Cleared per task
solve_task(task, "astar") Used hff (not admissible): cost 6 where 5 is optimal Optimal planners default to lmcut. -H none meant hff; it now means blind
make_heuristic(LMCutHeuristic, task) Returned the class unbuilt, so search crashed Classes and partials are built for the task
Same run, different PYTHONHASHSEED gripper astar/lmcut expanded 1725, 1758 or 1760 nodes and returned different plans Fact ids and id sets are built in sorted order, and LM-cut breaks ties explicitly. Benchmarks now reproduce

The requirements registry, the README PDDL table and AGENTS.md now describe the new semantics.

How it was checked

  • New tests/test_semantics.py (33 cases): tiny hand-written tasks whose right answer is obvious by inspection. 16 of them fail on the previous code (I ran them against main to confirm) and all pass here. The seed test runs the same search under three PYTHONHASHSEEDs.
  • Full suite: 460 passed, 1 skipped. flake8 and black are clean, and web/dist is rebuilt.
  • The reviewer's original repro scripts now give the expected answer.

Guarantees changed

  • blind is now actually admissible on tasks with free actions.
  • lmcut and h2 report admissible=False on tasks with conditional effects; they were already inadmissible there.
  • No planner moves in or out of OPTIMAL_PLANNERS in this PR. A follow-up handles bfs/iddfs, which claim cost-optimality but only minimise plan length.

Checklist

  • flake8 jupyddl tests tools and pytest pass locally
  • python tools/build_web.py re-run; web/dist committed
  • Guarantees that changed are listed above
  • CHANGELOG.md updated (Unreleased → Fixed)

The history has a merge -s ours of the already-merged #164 tip so the push fast-forwards. The PR diff against main contains only this change. Squash-merge.


Generated by Claude Code

Write down where and why we mean to publish jupyddl as citable research
software (.docs/PUBLISHING.md): a Zenodo DOI per release, a JOSS paper,
ICAPS workshops for the research results, with a readiness checklist, the
paper outline, the AI-usage disclosure JOSS requires, and how contributors
can be considered for authorship. The README states the project's aim and
its Cite section now says which version the 2021 preprint describes.

Add the community files an outside contributor expects: a security policy,
a pull-request template, a "Propose an algorithm" issue template, and an
"Adding an algorithm" section in CONTRIBUTING.md.

Run tests and build on pull requests. They were `on: [push]`, which never
fires for a fork, so a contributor's PR showed no checks at all.

Fix AGENTS.md drift: the Python floor is 3.10 and Mergify is retired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RbGtRHF8XhkiPQCDVHtV2
A package can now add planners, heuristics and generators by declaring
`jupyddl.planners`, `jupyddl.heuristics` or `jupyddl.generators` entry
points. Once installed, they work from the CLI, the benchmark harness and
`jupyddl.solve` by name, without anyone importing the package.

Discovery is lazy and defensive (jupyddl/plugins.py): the CLI runs it
before building its parser, the library only on a registry miss, so
importing jupyddl never scans installed packages. A built-in name always
wins, a plugin that fails to import is skipped with a warning, and
JUPYDDL_NO_PLUGINS=1 turns it off. `jupyddl plugins` lists what was found.
The CLI now reads `requires_heuristic` from the registry at call time,
since INFORMED_PLANNERS is fixed at import and cannot know about plugins.

.docs/EXTENDING.md walks through a heuristic, a planner from a priority
function, a planner from scratch with the budget/observer contract, a
generator, testing the guarantees you claim, and shipping as a plugin.
tests/test_extending.py runs every Python block in it, so it cannot drift.

best_first now accepts a tuple priority for tie-breaking; observers get
its first element as `f`, so traces stay numeric.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RbGtRHF8XhkiPQCDVHtV2
# Conflicts:
#	.docs/PUBLISHING.md
#	AGENTS.md
#	CHANGELOG.md
Each of these returned a plan, a cost or a value without complaint, and
validate_plan accepted the plans because it replays them through the same
Task.apply. tests/test_semantics.py now pins them on tiny hand-written
tasks; 16 of its cases fail on the previous code.

- Derived facts were never retracted, and (not <derived>) held from the
  start. close() now recomputes axioms from scratch and maintains negated
  derived facts; negation inside a rule body (needs stratification) is
  refused. The relaxation gets the complement for free, so it never calls
  a solvable state a dead end.
- An atom both added and deleted left its complement true too; a `when`
  re-adding a deleted atom did the same. Complements: the delete wins.
- Under a (total-cost) metric an action without a cost effect is free, as
  in IPC planners (pegsol/sokoban optimised the wrong objective). Cost
  inside `when` was charged unconditionally: refused, as are negative costs.
- A never-defined numeric fluent read as 0; conditions reading one fail.
- blind skipped zero-cost actions and overestimated.
- h1/h2/hm crashed on numeric states; LM-cut crashed on an empty goal.
- lmcut and h2 clear `admissible` on tasks with conditional effects.
- Optimal planners default to lmcut, not hff; `-H none` means blind.
- make_heuristic builds a heuristic class or partial instead of passing it
  through unbuilt.
- Fact ids and id sets are built in sorted order, so LM-cut tie-breaks,
  expansions and plans no longer change with PYTHONHASHSEED.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RbGtRHF8XhkiPQCDVHtV2
@guilyx
guilyx marked this pull request as ready for review November 3, 2026 08:00
@guilyx
guilyx merged commit 861994e into main Nov 3, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants