Skip to content

SSABasedGenericKubernetesResourceMatcher Incorrectly Detects Mismatch for initialDelaySeconds=0 in Deployment ReadinessProbe #2742

Closed

Description

@kamilchociej

Bug Report

What did you do?

  1. Define a Deployment CRUDKubernetesDependentResource in a Java Operator SDK controller with desired resource containing spec.template.spec.containers[0].readinessProbe.initialDelaySeconds=0.
  2. Trigger reconciliation.
  3. Observe that the controller repeatedly considers the resource as out-of-sync, even though there is no actual change needed.

The SSABasedGenericKubernetesResourceMatcher in Java Operator SDK incorrectly considers a Deployment resource as not matching the actual Kubernetes resource when spec.template.spec.containers[*].readinessProbe.initialDelaySeconds is set to 0.

Kubernetes omits fields with default values (0 for initialDelaySeconds) when storing resources, meaning the actual resource map retrieved from Kubernetes does not contain this field. However, the desired resource map includes initialDelaySeconds: 0, leading to an unnecessary update being triggered.

What did you expect to see?

If initialDelaySeconds: 0 is absent in the actual resource due to Kubernetes' defaulting behavior, the matcher should recognize this as equivalent to explicitly setting it to 0 in the desired resource.

What did you see instead? Under which circumstances?

The matcher detects a difference because the desired resource contains initialDelaySeconds: 0, while the actual resource lacks this field. This results in an unnecessary update being applied.

Environment

Kubernetes cluster type:

vanilla

$ Mention java-operator-sdk version from pom.xml file

4.9.7

$ java -version

21

$ kubectl version

v1.30.4

Possible Solution

Modify SSABasedGenericKubernetesResourceMatcher to handle cases where Kubernetes omits fields with default values, treating their absence as equivalent to explicitly setting the default value.

Additional context

This issue may affect other defaulted fields in Kubernetes objects, not just initialDelaySeconds. A general approach to handling default values might be necessary.

Activity

  1. csviri commented on Mar 27, 2025

    @csviri
    Collaborator

    Thx @kamilchociej , I think the current workaround is to fill the defaults in you current resources.

  2. csviri commented on Mar 27, 2025

    @csviri
    Collaborator

    Just to make sure I understand, this does not result in an infinite loop in your side, just an additional reconciliation right?

  3. kamilchociej commented on Mar 27, 2025

    @kamilchociej
    Author

    It is infinite reconciliation loop. The problem is that the desired resource contains the default (0) value while k8s does not contain the default value (0). I have no control over actual resource in io.javaoperatorsdk.operator.processing.dependent.kubernetes.SSABasedGenericKubernetesResourceMatcher.

  4. kamilchociej commented on Mar 27, 2025

    @kamilchociej
    Author

    Default value is removed from resource definition on k8s side while the property remains as managed field in controller managed fields entry.

  5. kamilchociej commented on Mar 27, 2025

    @kamilchociej
    Author

    Whole process looks like:

    1. Desired Deployment contains containing spec.template.spec.containers[0].readinessProbe.initialDelaySeconds=0.
    2. Controller applies desired state.
    3. K8s applies the change but returns Deployment without spec.template.spec.containers[0].readinessProbe.initialDelaySeconds
    4. Controller is notified about the Update
    5. Desired Deployment contains spec.template.spec.containers[0].readinessProbe.initialDelaySeconds=0
    6. Actual Deployment does not contain spec.template.spec.containers[0].readinessProbe.initialDelaySeconds
    7. SSABasedGenericKubernetesResourceMatcher returns false
    8. Deployment with spec.template.spec.containers[0].readinessProbe.initialDelaySeconds=0 is applied
    9. All previous steps staring from 4th happen again.
  6. kamilchociej commented on Mar 27, 2025

    @kamilchociej
    Author

    It looks like 0 for Integer is threaten as null by k8s but difference between setting 0 and null is that if you set 0 then field is marked as managed by operator controller.

  7. csviri commented on Mar 27, 2025

    @csviri
    Collaborator

    can you try to set previousAnnotationForDependentResourcesEventFiltering to false, if it helps with infinite loop?

  8. kamilchociej commented on Mar 27, 2025

    @kamilchociej
    Author

    Where can I set this?

  9. kamilchociej commented on Mar 27, 2025

    @kamilchociej
    Author

    Following logic applied on each desired Probe resolves the issue:

      private void sanitizeProbe(Probe probe) {
        if (probe != null) {
          if (Objects.equals(0, probe.getInitialDelaySeconds())) {
            probe.setInitialDelaySeconds(null);
          }
          if (Objects.equals(0, probe.getPeriodSeconds())) {
            probe.setPeriodSeconds(10);
          }
          if (Objects.equals(0, probe.getTimeoutSeconds())) {
            probe.setTimeoutSeconds(1);
          }
          if (Objects.equals(0, probe.getSuccessThreshold())) {
            probe.setSuccessThreshold(1);
          }
          if (Objects.equals(0, probe.getFailureThreshold())) {
            probe.setFailureThreshold(3);
          }
          if (Objects.equals(0L, probe.getTerminationGracePeriodSeconds())) {
            probe.setTerminationGracePeriodSeconds(null);
          }
        }
      }
    

    This is ugly workaround but at least it works for Probes.

  10. csviri commented on Mar 27, 2025

    @csviri
    Collaborator

    can you try to set previousAnnotationForDependentResourcesEventFiltering to false, if it helps with infinite loop?

    default boolean previousAnnotationForDependentResourcesEventFiltering() {

    within ConfigurationService, similar to this:

    Operator operator = new Operator(o -> o.withStopOnInformerErrorDuringStartup(false));

  11. csviri commented on Mar 27, 2025

    @csviri
    Collaborator

    That should fix the infinite loop.

  12. csviri commented on Mar 31, 2025

    @csviri
    Collaborator

    closing this as duplicate: #2553

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions