Skip to content

EventFilterWindow suppresses mapper events when controller writes through shared InformerEventSource #3445

Closed

Description

@SamBarker

Description

When a controller patches a secondary resource's status through its own InformerEventSource (via eventFilteringUpdateAndCacheResource), the EventFilterWindow correctly identifies the resulting informer event as an "own update" and suppresses it. However, this suppression also prevents the SecondaryToPrimaryMapper from ever seeing the updated resource state, which can leave the controller in a dead state.

Scenario

A controller reconciles a primary resource and has an InformerEventSource for a secondary resource type. The mapper uses a generation/observedGeneration guard:

  • generation != observedGeneration → return empty set (secondary not ready, skip)
  • generation == observedGeneration → return primary ResourceID (secondary ready, trigger)

During reconciliation the controller patches the secondary's status.observedGeneration to match metadata.generation through its InformerEventSource. The EventFilterWindow suppresses the resulting event as an own update. The mapper never sees gen == obsGen, so the controller is never re-triggered.

Expected behaviour

The status-change event should reach the mapper so it can evaluate the new state and trigger reconciliation of the primary.

Actual behaviour

The EventFilterWindow suppresses the event because its resource version matches ownUpdateVersions. The mapper never fires. The controller enters a dead state for that secondary resource.

Root cause

ManagedInformerEventSource.eventFilteringUpdateAndCacheResource (line 96) opens a filter window, performs the update, records the response resource version as an "own" version, then closes the window. When the informer event arrives with that resource version, EventFilterWindow.eventForRangeAndClear (line 183) sees events.keySet().equals(ownResourceVersions) and returns Optional.empty() — the event is swallowed before it reaches propagateEvent and the mapper.

Impact

This affects any controller that:

  1. Writes to a secondary resource through its own InformerEventSource (directly or via a KubernetesDependentResource sharing the event source with useEventSourceWithName)
  2. Has a mapper that depends on seeing the result of that write

The pattern is common in controllers that manage secondary resources and use generation/observedGeneration guards to sequence reconciliation.

Reproducer

Branch: reproducer/status-event-filtering — see StatusEventFilteringIT in operator-framework/src/test/java/.../baseapi/statuseventfiltering/

Downstream issue

kroxylicious/kroxylicious#4241

Happy to look at a fix if the maintainers agree with the analysis.

Activity

  1. csviri commented on Jun 29, 2026

    @csviri
    Collaborator

    Hi @SamBarker ,

    just to make sure we are on same page, so the only issue here is that, the secondary update does not trigger the reconiliation?

    If yes, that is expected. If you want to still trigger the reconiliation use reSchedule as described here:
    |https://javaoperatorsdk.io/blog/2026/03/13/welcome-read-cache-after-write-consistency/#the-case-for-instant-reschedule

  2. SamBarker commented on Jun 30, 2026

    @SamBarker
    ContributorAuthor

    Thanks @csviri — our issue description over-simplified the actual problem we're seeing downstream, partly because we were misled by the test that was failing. You're right that the scenario as described (a controller suppressing its own write event) is expected behaviour, and reSchedule is the correct pattern for that.

    Our actual downstream issue in Kroxylicious is different: we have two independent controllers in the same operator, each managing different custom resource types but both patching status on their respective primaries concurrently. Controller A watches Controller B's primary as a secondary via InformerEventSource. When B patches its own status, A's mapper should see that event — and since A never writes to B's resource type, A's InformerEventSource never opens a filter window, so the event should pass through unfiltered.

    This works reliably on JOSDK 5.2.x but fails intermittently on 5.3.5. However, we've been unable to reproduce the failure in an isolated JOSDK integration test — the timing window appears too narrow. We also wrote unit tests against EventFilterWindow and EventFilterSupport to probe the cross-controller event sequences, and the algorithm handles them correctly in all orderings we tested.

    At this point we can't point to a specific bug in JOSDK's event filtering. It may be a different code path entirely (EventProcessor, informer delivery, etc.) or something specific to Kroxylicious's topology. We'll continue investigating on our side and if you have any thoughts on what else could cause cross-controller event loss between 5.2.x and 5.3.x we'd appreciate the pointer.

    I'm going to close this issue since the scenario described in it is working as designed. If we manage to isolate the actual root cause we'll open a new issue with accurate details. Sorry for the noise.

  3. csviri commented on Jun 30, 2026

    @csviri
    Collaborator

    Hi @SamBarker , thx for the clarification, not that with the latest release there is theoretically one special case that might cause very rarely an issue: #3448

    We will release this soon, and from that point, what you can do is just turn on debug logging for the test, based on those we could find the problem is one still remains.

    Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions