Repository navigation
Conversation
philayres
force-pushed
the
handlebars-lazy-config-phase2-1481
branch
from
November 4, 2026 15:01
adb17f0 to
3405923
Compare
philayres
force-pushed
the
handlebars-lazy-config-phase2-1481
branch
6 times, most recently
from
November 11, 2026 13:31
4b8cdb4 to
5f1da02
Compare
…nsected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ing the master skeleton - refs consected#1481
…item partials - refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… - refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…istory_id fallback assertion - refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nsected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…efs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…- refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…endencies in template config freshness - refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ecific slot fetching - refs consected#1481 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…cs and embedded spec robustness - refs consected#1481 Includes the finding 4 and 6 code and spec changes that were picked up earlier in 66c6705. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
philayres
force-pushed
the
handlebars-lazy-config-phase2-1481
branch
from
November 14, 2026 17:00
fbb3280 to
93202c3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1481 (Phase 2; does not close #1481 or #1480).
Stacked on #1527 (Phase 1). The first commit (
d92f4b1a7) is Phase 1 and will drop out of this diff once #1527 merges. The remaining 14 commits are Phase 2 and are kept separate for review: the original change (5f1da02dc), refactors R1-R8 and review fixes, each described by its commit message. Squash on merge is fine. Phase 1 keepsgeneration_keystable across admin saves viaApplication.clear_cache_preserving_generation, and prewarm markers includeApplication.cache_epoch.What this does
A config-only edit to a dynamic model, activity log or external identifier no longer rebuilds the masters skeleton fragment. The client is told the resource's config version has moved on and fetches only that resource's
template_config.template_version_token: the latest history-row id, or a composite integer (>= 1e12) when it references config libraries withuse_current_version, so a library edit changes the token.def_versionis unchanged and still nil for live definitions.v<token>plus av0alias. The four hardcoded'v'slots are nowv0. Forms carry their own nonce-bearing inline state block. Shared base aliases (the ones list templates read) are promoted tov0only from the definition'sdefault_option_type_nameoption type, whichset_defaultsalways generates for dynamic models and external identifiers. This keeps the list chrome independent of YAML order, and activity logs, which have no shared base, are unaffected.template_configresponses are state-only (no re-emitted Handlebars tags). The external identifier endpoint now renders the shared set; it previously returned an empty body._fpa.js). A fetch happens only when thev<n>slot is absent, sends one representative record per type, option type and version, and patchesv<n>andv0.fpa_state_itemfalls back tov0for live tokens.master_templates_skeleton_cache_keyis a DB-derived set signature: active definitions, structural fingerprint (option types,references,embed), activity-logitem_type/rec_type, page-layout panels, access timestamps including global rows, and the page-layout/app-config history token. A config-only edit does not move it.partial_cache_keyis unchanged.vdef_version. Live tokens are refreshed from the DB on index, show, create and update, so a second worker does not report a stale token.template_configETags for all three controllers are keyed on the resource's own tokens.Needs a human decision
Rails.cache.clearis no longer called onDynamicModel,ExternalIdentifierand YAMLAdmin::ConfigLibrarysaves. The issue does not authorise this. It is required for the target behaviour: the clear deletedserver_cache_version, which rotated the compiled-template generation and wiped the skeleton on every definition save.ActivityLogandAppConfigurationalready behaved this way viaSelectorCache. Non-YAML libraries andAdmin::PageLayoutstill clear everything.To mitigate caches that depended on the clear, I added explicit shared-cache invalidation: definition-metadata keys (
db_column_comments,db_foreign_keys, per-table data dictionary), the selector-with-config-overrides cache, andviewable_tables. Please check this against #1289.Admin::MigrationGenerator#run_migrationdrops the metadata keys only once the migration has succeeded, so a failed or skipped migration leaves them alone and a concurrent reader cannot re-cache pre-migration catalog data.DynamicModel#set_comments_from_tableand theupdate_config_from_tablepath callcolumn_comments(force: true), so comments written back by a migration are never stale.data_diccaches under the basename (e.g.q2), but invalidation deleted theq2_datadickey and never matched. Invalidation now deletes both, and the cached rows expire after 5 minutes because dictionary rows change without any migration. ASELECT *plus array conversion measured 1-10 ms on the largest dev dictionary (1,313 rows).Behaviour changes to know about
spec/requests/pages/template_definition_edit_freshness_spec.rb(Master page template shows stale captions after a direct edit of a dynamic definition's options #1525/Fixed stale captions in the page template after a direct edit of a dynamic definition - fixes #1525 #1526) now asserts the new contract:/pages/<token>/templateis unchanged by a caption edit, and the new caption arrives viatemplate_config.rebuild=truenow evicts the skeleton under the new key.ExternalIdImplementer#versioned_definitionnow returns the current definition, and the external identifiertemplate_configrenders real output. This does not match the issue, which said the endpoint already rendered the set.Verification
Passing on this branch: pages requests, views/masters, helpers (343), initializer, prewarm, concerns, classification, config library, user access control, activity-log/admin dynamic-model/external-identifier controller specs, activity-log/dynamic-model/external-identifier request specs,
template_version_cache_poisoning_spec,dynamic_model_show_if_spec(browser, 4 examples), Jasmine (164 specs).A combined run of pages requests, views/masters, prewarm, concerns, classification, helpers and the dynamic model, external identifier and activity log request specs (629 examples) had two prewarm failures, both spec issues fixed on this branch:
candidates_spec.rbcollided with recently signed-in users left in the test DB, andmaster_templates_spec.rbread a skeleton signature memoized on the process-wideApplicationController.helpersby earlier pages specs (the shared reset helper now clears the new memos).spec/models/dynamic_model_spec.rbpasses (11 examples) on a freshly cleaned test database.spec/models/option_configs/dynamic_model_options_spec.rb('gets the correct version of extra options based on creation date of the instance') fails identically onup-develop.Issue verification items covered by dedicated specs: a brand-new definition (including a new
detailsmodel) appears on the next full page load without a restart (template_new_definition_discovery_spec.rb,dynamic_model_new_definition_discovery_spec.rb); data index responses return 304 for a matching ETag, and 200 with a new ETag after a config-only edit or a change to only a model-referenced record's definition (template_definition_edit_freshness_spec.rb).These fail the same way on the Phase 1 tip, so are not caused by this PR:
notes_field_plain_format_spec.rb,active_model_configurations_fail_open_spec.rb:27,def_config_triggers_spec.rb:89,dynamic_models_versions_spec.rb:40and:91, Jasmine "YAML top-level array first". Runningspec/models/dynamicas a whole directory also fails in file order (before_save_trigger_spec,model_reference_handler_spec, 32 of them pass when run alone); the same specs fail on the Phase 1 baseline.Review follow-ups (round 2)
A second independent review and a regression found these, all fixed with failing specs first:
show_ifof an embedded item (e.g. a dynamic model in an activity log form) was looked up under the parent's token and silently skipped in New and Edit mode.show_itemsnow resolves the version per item, embedded forms carry their owndata-vdef-versionand render their own state-only template, anddata_from_formcarries the version for edit mode. System specembedded_form_own_version_spec.rband a Jasmine spec cover it (parent and embedded on different versions; the spec passes on the pre-Phase-1 baseline).index_cache_keyincludes the current definition's live token so a tightenedshowable_ifis not bypassed by a cached index for records on an older version. A worker whose loaded definition is stale (another worker saved it) now reloads it (reload_if_stale) before reading the token, so a cache key never holds results computed from an older config.v0ordering. Library-derived composite tokens contain a hash and are not ordered. The token stays the slot key/ETag; a separatelive_order(definition history row id + latest config library history row id) is emitted with each live slot and compared forv0promotion.template_configtokens/ETag include the dependencies of every requested root, not only the first.MasterTemplates::SkeletonSignature(values for unchanged inputs are unchanged for DMs/EIs; activity-log skeleton keys change once).Rails.cache.clear. OneDynamic::DefinitionCacheVersiontoken replaces the per-class and shared tokens (viewable tables, access control resource names, selector overrides, reports); theinfo-block-gs-blockfragment is invalidated; selector-cache key tracking is guarded by a mutex.state_onlyis an explicit caller decision, the external identifier skeleton has its own item partial, v0 promotion and embedded-dependency logic are single_fpafunctions, template config freshness isDynamic::TemplateConfigFreshness, the edit form state is a helper, andEmbeddedItemHandleris no longer used by the external identifier controller.skipwas removed; ordering/leak problems between specs were fixed (config_library_versioned_spec, embedded form spec uses its ownaa_own_versionactivity log).Review follow-ups (round 1)
refresh_outdatedclears the class-levelall_versions_memo, so an older record resolves to its historical version after another worker edits the definition. Reads, population and invalidation are coordinated with a mutex and a generation counter, so a reader can no longer seenilmid-clear or store a stale list after a clear._common_search_results_template_setdedup keys include the definition class, id anddef_version, so same-named definitions of different types no longer suppress each other.template_configURL. The client adds&s=<salt>.<hash>: a random, non-secret per-Rails-session salt plus a hash of user, sign-in time, app type and roles. A browser-cached body cannot be reused across sessions or access changes.v0from forms. A form of a current definition (New or Edit) promotesv0through_fpa.promote_template_state_v0, which refuses to replace av0with a largerlive_order, so a form from a stale worker cannot replace a newerv0. Historical forms never touchv0.refresh_outdated/up_to_date?2 s timestamp tolerance can hold a stale definition).Review notes
DisableVDefis set, so definition edits still reach clients in development;def_versionstays nil and records still use the current definition.v0.updated_at, token).Not covered / known limits
v0(caption, add label, orientation) until a record fetch or skeleton rebuild, because the endpoint needs a record id. The plan's optional version-map endpoint would remove this.refresh_outdatedwith its existing short tolerance (Definition refresh ignores edits made within 2 seconds of the last loaded update #1533); definition changes are detected immediately viareload_if_stale.default, the list chrome now comes from the generated default, not the first declared option type.max_age: 30ontemplate_configis kept; the?v=query discriminator avoids serving a stale body to a changed token.Classification::ItemFlagNameis still not in any key (existing gap, Cache key audit: under-scoped and shared-cache/local-state mismatches across Rails.cache usages #1289).spec/suite. After the final fixes:spec/system(excludingnfs_storeandredcap) 645 examples with onlyplay_ipa_process_spec.rb:83andstudy_info_process_spec.rb:56failing (both also fail onup-develop); Jasmine 230 specs; the model, helper, request, view and controller groups for the changed areas pass. A few specs are order-dependent in very large combined runs and pass alone (message_template_spec.rb:233,item_flags_controller_spec.rb:130,switch_id_display_spec.rb:197/252,manage_users_spec.rb:169,admin_master_records_spec.rb:167,upgrade_to_2fa_spec.rb:156).template_configETags change once on deploy (definition resource tokens are now part of the key), as do activity-log skeleton keys.