diff --git a/docs/_data/sidebar.yml b/docs/_data/sidebar.yml index fe27a8e6aa..607bccf6a2 100644 --- a/docs/_data/sidebar.yml +++ b/docs/_data/sidebar.yml @@ -11,6 +11,8 @@ url: /docs/features - title: Dependent Resource Feature url: /docs/dependent-resources + - title: Workflows + url: /docs/workflows - title: Patterns and Best Practices url: /docs/patterns-best-practices - title: FAQ diff --git a/docs/documentation/dependent-resources.md b/docs/documentation/dependent-resources.md index 665e439f01..5296ea245a 100644 --- a/docs/documentation/dependent-resources.md +++ b/docs/documentation/dependent-resources.md @@ -137,7 +137,7 @@ See the full source code [here](https://github.com/java-operator-sdk/java-operat ## Managed Dependent Resources -As mentioned previously, one goal of this implementation is to make it possible to semi-declaratively create and wire +As mentioned previously, one goal of this implementation is to make it possible to declaratively create and wire dependent resources. You can annotate your reconciler with `@Dependent` annotations that specify which `DependentResource` implementation it depends upon. JOSDK will take the appropriate steps to wire everything together and call your @@ -145,8 +145,7 @@ appropriate steps to wire everything together and call your most use cases where the logic associated with the primary resource is usually limited to status handling based on the state of the secondary resources and the resources are not dependent on each other. -Note that all dependents will be reconciled in order. If an exception happens in one or more reconciliations, the -followup resources will be reconciled. +See [Workflows](https://javaoperatorsdk.io/docs/dependent-resources) how/ in what order the resources are reconciled. This behavior and automated handling is referred to as "managed" because the `DependentResource` instances are managed by JOSDK. @@ -186,15 +185,16 @@ sample [here](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/s ## Standalone Dependent Resources -To use dependent resources in more complex workflows, when there are some resources needs to be created only in certain -conditions the standalone mode is available or the dependent resources are not independent of each other. -For example if calling an API needs to happen if a service is already up and running -(think configuring a running DB instance). +In case just some or sub-set of the resources are desired to be managed by dependent resources use standalone mode. In practice this means that the developer is responsible to initializing and managing and -calling `reconcile` method. However, this gives possibility for developers to fully customize the workflow for +calling `reconcile` method. However, this gives possibility for developers to fully customize the process for reconciliation. Use standalone dependent resources for cases when managed does not fit. -The sample is similar to one above it just performs additional checks, and conditionally creates an `Ingress`: +Note that [Workflows](https://javaoperatorsdk.io/docs/dependent-resources) support also standalone mode using +standalone resources. + +The sample is similar to one above it just performs additional checks, and conditionally creates an `Ingress`: +(Note that now this condition creation is also possible with Workflows) ```java diff --git a/docs/documentation/features.md b/docs/documentation/features.md index 7a604dc14e..91778ee275 100644 --- a/docs/documentation/features.md +++ b/docs/documentation/features.md @@ -324,6 +324,41 @@ intersections: will still happen, but won't reset the retry, will be still marked as the last attempt in the retry info. The point (1) still holds, but in case of an error, no retry will happen. +## Rate Limiting + +It is possible to rate limit reconciliation on a per-resource basis. The rate limit also takes +precedence over retry/re-schedule configurations: for example, even if a retry was scheduled for +the next second but this request would make the resource go over its rate limit, the next +reconciliation will be postponed according to the rate limiting rules. Note that the +reconciliation is never cancelled, it will just be executed as early as possible based on rate +limitations. + +Rate limiting is by default turned **off**, since correct configuration depends on the reconciler +implementation, in particular, on how long a typical reconciliation takes. +(The parallelism of reconciliation itself can be +limited [`ConfigurationService`](https://github.com/java-operator-sdk/java-operator-sdk/blob/ce4d996ee073ebef5715737995fc3d33f4751275/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java#L120-L120) +by configuring the `ExecutorService` appropriately.) + +A default rate limiter implementation is provided, see: +[`PeriodRateLimiter`](https://github.com/java-operator-sdk/java-operator-sdk/blob/ce4d996ee073ebef5715737995fc3d33f4751275/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiter.java#L14-L14) +. +Users can override it by implementing their own +[`RateLimiter`](https://github.com/java-operator-sdk/java-operator-sdk/blob/ce4d996ee073ebef5715737995fc3d33f4751275/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateLimiter.java) +. + +To configure the default rate limiter use `@ControllerConfiguration` annotation. The following +configuration limits +each resource to reconcile at most twice within a 3 second interval: + +`@ControllerConfiguration(rateLimit = @RateLimit(limitForPeriod = 2,refreshPeriod = 3,refreshPeriodTimeUnit = TimeUnit.SECONDS))` +. + +Thus, if a given resource was reconciled twice in one second, no further reconciliation for this +resource will happen before two seconds have elapsed. Note that, since rate is limited on a +per-resource basis, other resources can still be reconciled at the same time, as long, of course, +that they stay within their own rate limits. + + ## Handling Related Events with Event Sources See also this [blog post](https://csviri.medium.com/java-operator-sdk-introduction-to-event-sources-a1aab5af4b7b). diff --git a/docs/documentation/workflows.md b/docs/documentation/workflows.md new file mode 100644 index 0000000000..e5454b0425 --- /dev/null +++ b/docs/documentation/workflows.md @@ -0,0 +1,276 @@ +--- +title: Workflows +description: Reference Documentation for Workflows +layout: docs +permalink: /docs/workflows +--- + +## Overview + +Kubernetes (k8s) does not have notion of a resource "depends on" on another k8s resource, +in terms of in what order a set of resources should be reconciled. However, Kubernetes operators are used to manage also +external (non k8s) resources. Typically, when an operator manages a service, after the service is first deployed +some additional API calls are required to configure it. In this case the configuration step depends +on the service and related resources, in other words the configuration needs to be reconciled after the service is +up and running. + +The intention behind workflows is to make it easy to describe more complex, almost arbitrary scenarios in a declarative +way. While [dependent resources](https://javaoperatorsdk.io/docs/dependent-resources) describes a logic how a single +resources should be reconciled, workflows describes the process how a set of target resources should be reconciled. + +Workflows are defined as a set of [dependent resources](https://javaoperatorsdk.io/docs/dependent-resources) (DR) +and dependencies between them, along with some conditions that mainly helps define optional resources and +pre- and post-conditions to describe expected states of a resource at a certain point in the workflow. + +## Elements of Workflow + +- **Dependent resource** (DR) - are the resources which are managed in reconcile logic. +- **Depends-on relation** - if a DR `B` depends on another DR `A`, means that `B` will be reconciled after `A`. +- **Reconcile precondition** - is a condition that needs to be fulfilled before the DR is reconciled. This allows also + to define optional resources, that for example only created if a flag in a custom resource `.spec` has some + specific value. +- **Ready postcondition** - checks if a resource could be considered "ready", typically if pods of a deployment are up + and running. +- **Delete postcondition** - during the cleanup phase it can be used to check if the resources is successfully deleted, + so the next resource on which the target resources depends can be deleted as next step. + +## Defining Workflows + +Similarly to dependent resources, there are two ways to define workflows, in managed and standalone manner. + +### Managed + +Annotations can be used to declaratively define a workflow for the reconciler. In this case the workflow is executed +before the `reconcile` method is called. The result of the reconciliation is accessed through the `context` object. + +Following sample shows a hypothetical sample to showcase all the elements, where there are two resources a Deployment and +a ConfigMap, where the ConfigMap depends on the deployment. Deployment has a ready condition so, the config map is only +reconciled after the Deployment and only if it is ready (see ready-postcondition). The ConfigMap has attached reconcile +precondition, therefore it is only reconciled if that condition holds. In addition to that has a delete-postCondition, +thus only considered to be deleted if that condition holds. + +```java +@ControllerConfiguration(dependents = { + @Dependent(name = DEPLOYMENT_NAME, type = DeploymentDependentResource.class, + readyPostcondition = DeploymentReadyCondition.class), + @Dependent(type = ConfigMapDependentResource.class, + reconcilePrecondition = ConfigMapReconcileCondition.class, + deletePostcondition = ConfigMapDeletePostCondition.class, + dependsOn = DEPLOYMENT_NAME) +}) +public class SampleWorkflowReconciler implements Reconciler, + Cleaner { + + public static final String DEPLOYMENT_NAME = "deployment"; + + @Override + public UpdateControl reconcile( + WorkflowAllFeatureCustomResource resource, + Context context) { + + resource.getStatus() + .setReady( + context.managedDependentResourceContext() // accessing workflow reconciliation results + .getWorkflowReconcileResult().orElseThrow() + .allDependentResourcesReady()); + return UpdateControl.patchStatus(resource); + } + + @Override + public DeleteControl cleanup(WorkflowAllFeatureCustomResource resource, + Context context) { + // emitted code + + return DeleteControl.defaultDelete(); + } +} + +``` + +### Standalone + +In this mode workflow is built manually using [standalone dependent resources](https://javaoperatorsdk.io/docs/dependent-resources#standalone-dependent-resources) +. The workflow is created using a builder, that is explicitly called in the reconciler (from web page sample): + +```java +@ControllerConfiguration( + labelSelector = WebPageDependentsWorkflowReconciler.DEPENDENT_RESOURCE_LABEL_SELECTOR) +public class WebPageDependentsWorkflowReconciler + implements Reconciler, ErrorStatusHandler, EventSourceInitializer { + + public static final String DEPENDENT_RESOURCE_LABEL_SELECTOR = "!low-level"; + private static final Logger log = + LoggerFactory.getLogger(WebPageDependentsWorkflowReconciler.class); + + private KubernetesDependentResource configMapDR; + private KubernetesDependentResource deploymentDR; + private KubernetesDependentResource serviceDR; + private KubernetesDependentResource ingressDR; + + private Workflow workflow; + + public WebPageDependentsWorkflowReconciler(KubernetesClient kubernetesClient) { + initDependentResources(kubernetesClient); + workflow = new WorkflowBuilder() + .addDependent(configMapDR).build() + .addDependent(deploymentDR).build() + .addDependent(serviceDR).build() + .addDependent(ingressDR).withReconcileCondition(new IngressCondition()).build() + .build(); + } + + @Override + public Map prepareEventSources(EventSourceContext context) { + return EventSourceInitializer.nameEventSources( + configMapDR.initEventSource(context), + deploymentDR.initEventSource(context), + serviceDR.initEventSource(context), + ingressDR.initEventSource(context)); + } + + @Override + public UpdateControl reconcile(WebPage webPage, Context context) { + + var result = workflow.reconcile(webPage, context); + + webPage.setStatus(createStatus(result)); + return UpdateControl.patchStatus(webPage); + } + // emitted code +} + +``` + +## Workflow Execution + +This section describes how a workflow is executed in details, how is the ordering determined and how condition and +errors affect the behavior. The workflow execution as also its API denotes, can be divided to into two parts, +the reconciliation and cleanup. [Cleanup](https://javaoperatorsdk.io/docs/features#the-reconcile-and-cleanup) is +executed if a resource is marked for deletion. + + +## Common Principles + +- **As complete as possible execution** - when a workflow is reconciled, it tries to reconcile as many resources as + possible. Thus is an error happens or a ready condition is not met for a resources, all the other independent resources + will be still reconciled. This is the opposite to fail-fast approach. The assumption is that eventually in this way the + overall desired state is achieved faster than with a fail fast approach. +- **Concurrent reconciliation of independent resources** - the resources which are not dependent on each are processed + concurrently. The level of concurrency is customizable, could be set to one if required. By default, workflows use + the executor service from [ConfigurationService](https://github.com/java-operator-sdk/java-operator-sdk/blob/6f2a252952d3a91f6b0c3c38e5e6cc28f7c0f7b3/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java#L120-L120) + +## Reconciliation + +This section describes how a workflow is executed, first the rules are defined, then are explained on samples: + +### Rules + + 1. DR is reconciled if it does not depend on another DR, or ALL the DRs it depends on are ready. In case it + has a reconcile-precondition that condition must be met too. (So here ready means that it is successfully + reconciled - without any error - and if it has a ready condition that condition is met). + 2. If a reconcile-precondition of a DR is not met, it is deleted. If there are dependent resources which depends on it + are deleted too as first - this applies recursively. That means that DRs are always deleted in revers order compared + how are reconciled. + 3. Delete is called on a dependent resource if as described in point 2. it (possibly transitively) depends on a DR which + did not meet it's reconcile condition, and has no DRs depends on it, or if the DR-s which depends on it are already + successfully deleted (within actual execution). "Delete is called" means, that the dependent resource is checked + if it implements `Deleter` interface, if implements it but do not implement `GarbageCollected` interface, + the `Deleter.delete` method called. If a DR does not implement `Deleter` interface, it is considered as deleted + automatically. Successfully deleted means, that it is deleted and if a delete-postcondition is present it is met. + +### Samples + +Notation: The arrows depicts reconciliation ordering, or in depends-on relation in reverse direction: +`1 --> 2` mean `DR 2` depends-on `DR 1`. + +#### Reconcile Sample + +
+ +stateDiagram-v2 +1 --> 2 +1 --> 3 +2 --> 4 +3 --> 4 + +
+ +- At the workflow the reconciliation of the nodes would happen in the following way. DR with index `1` is reconciled. + After that DR `2` and `3` is reconciled concurrently, if both finished their reconciliation, node `4` is reconciled too. +- In case for example `2` would have a ready condition, that would be evaluated as "not met", `4` would not be reconciled. + However `1`,`2` and `3` would be reconciled. +- In case `1` would have a ready condition that is not met, neither `2`,`3` or `4` would be reconciled. +- If there would be an error during the reconciliation of `2`, `4` would not be reconciled, but `3` would be + (also `1` of course). + +#### Sample with Reconcile Precondition + +
+ +stateDiagram-v2 +1 --> 2 +1 --> 3 +3 --> 4 +3 --> 5 + +
+ +- Considering this sample for case `3` has reconcile-precondition, what is not met. In that case DR `1` and `2` would be + reconciled. However, DR `3`,`4`,`5` would be deleted in the following way. DR `4` and `5` would be deleted concurrently. + DR `3` would be deleted if `4` and `5` is deleted successfully, thus no error happened during deletion and all + delete-postconditions are met. + - If delete-postcondition for `5` would not be met `3` would not be deleted; `4` would be. + - Similarly, in there would be an error for `5`, `3` would not be deleted, `4` would be. + +## Cleanup + +Cleanup works identically as delete for resources in reconciliation in case reconcile-precondition is not met, just for +the whole workflow. + +The rule is relatively simple: + +Delete is called on a DR if there is no DR that depends on it, or if the DR-s which depends on it are +already deleted successfully (withing this execution of workflow). Successfully deleted means, that it is deleted and +if a delete-postcondition is present it is met. "Delete is called" means, that the dependent resource is checked if it +implements `Deleter` interface, if implements it but do not implement `GarbageCollected` interface, the `Deleter.delete` +method called. If a DR does not implement `Deleter` interface, it is considered as deleted automatically. + +### Sample + +
+ +stateDiagram-v2 +1 --> 2 +1 --> 3 +2 --> 4 +3 --> 4 + +
+ +- The DRs are deleted in the following order: `4` is deleted, after `2` and `3` are deleted concurrently, after both + succeeded `1` is deleted. +- If delete-postcondition would not be met for `2`, node `1` would not be deleted. DR `4` and `3` would be deleted. +- If `2` would be errored, DR `1` would not be deleted. DR `4` and `3` would be deleted. +- if `4` would be errored, no other DR would be deleted. + +## Error Handling + +As mentioned before if an error happens during a reconciliation, the reconciliation of other dependent resources will +still happen. There might a case that multiple DRs are errored, therefore workflows throws an +['AggregatedOperatorException'](https://github.com/java-operator-sdk/java-operator-sdk/blob/86e5121d56ed4ecb3644f2bc8327166f4f7add72/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/AggregatedOperatorException.java) +that will contain all the related exceptions. + +The exceptions can be handled by [`ErrorStatusHandler`](https://github.com/java-operator-sdk/java-operator-sdk/blob/86e5121d56ed4ecb3644f2bc8327166f4f7add72/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/AggregatedOperatorException.java) + +## Notes and Caveats + +- Delete is almost always called on every resource during the cleanup. However, it might be the case that the resources + was already deleted in a previous run, or not even created. This should not be a problem, since dependent resources + usually cache the state of the resource, so are already aware that the resource not exists, thus basically doing nothing + if delete is called on an already not existing resource. +- If a resource has owner references, it will be automatically deleted by Kubernetes garbage collector if + the owner resource is marked for deletion. This might not be desirable, to make sure that delete is handled by the + workflow don't use garbage collected kubernetes dependent resource, use for example [`CRUDNoGCKubernetesDependentResource`](https://github.com/java-operator-sdk/java-operator-sdk/blob/86e5121d56ed4ecb3644f2bc8327166f4f7add72/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDNoGCKubernetesDependentResource.java). +- After a workflow executed no state is persisted regarding the workflow execution. On every reconciliation + all the resources are reconciled again, in other words the whole workflow is evaluated again. + diff --git a/micrometer-support/pom.xml b/micrometer-support/pom.xml index 45b9302b03..502d681e64 100644 --- a/micrometer-support/pom.xml +++ b/micrometer-support/pom.xml @@ -5,7 +5,7 @@ java-operator-sdk io.javaoperatorsdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT 4.0.0 diff --git a/operator-framework-core/pom.xml b/operator-framework-core/pom.xml index f45b0805a7..25e8698bca 100644 --- a/operator-framework-core/pom.xml +++ b/operator-framework-core/pom.xml @@ -6,7 +6,7 @@ io.javaoperatorsdk java-operator-sdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT ../pom.xml diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/Operator.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/Operator.java index 7679267a78..a17e86d8cc 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/Operator.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/Operator.java @@ -68,7 +68,7 @@ public Operator(KubernetesClient kubernetesClient, ConfigurationService configur ConfigurationServiceProvider.set(configurationService); } - /** Adds a shutdown hook that automatically calls {@link #stop()} ()} when the app shuts down. */ + /** Adds a shutdown hook that automatically calls {@link #stop()} when the app shuts down. */ public void installShutdownHook() { Runtime.getRuntime().addShutdownHook(new Thread(this::stop)); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/AnnotationControllerConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/AnnotationControllerConfiguration.java index 2407189c1c..3e7fcfe393 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/AnnotationControllerConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/AnnotationControllerConfiguration.java @@ -1,42 +1,59 @@ package io.javaoperatorsdk.operator.api.config; +import java.lang.reflect.InvocationTargetException; import java.time.Duration; +import java.util.*; import java.util.Arrays; import java.util.Collections; import java.util.LinkedHashMap; import java.util.List; import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; import java.util.function.Function; +import java.util.function.Predicate; import java.util.stream.Collectors; +import java.util.stream.Stream; import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.OperatorException; import io.javaoperatorsdk.operator.ReconcilerUtils; import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; -import io.javaoperatorsdk.operator.api.reconciler.Constants; +import io.javaoperatorsdk.operator.api.config.eventsource.EventSourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.GenericEventSourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.InformerEventSourceSpec; +import io.javaoperatorsdk.operator.api.reconciler.*; import io.javaoperatorsdk.operator.api.reconciler.ControllerConfiguration; -import io.javaoperatorsdk.operator.api.reconciler.Reconciler; import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent; import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.VoidCondition; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfig; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; +import io.javaoperatorsdk.operator.processing.event.rate.PeriodRateLimiter; +import io.javaoperatorsdk.operator.processing.event.rate.RateLimiter; +import io.javaoperatorsdk.operator.processing.event.source.PrimaryToSecondaryMapper; +import io.javaoperatorsdk.operator.processing.event.source.SecondaryToPrimaryMapper; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilters; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidGenericFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnAddFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnDeleteFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnUpdateFilter; import static io.javaoperatorsdk.operator.api.reconciler.Constants.DEFAULT_NAMESPACES_SET; @SuppressWarnings("rawtypes") -public class AnnotationControllerConfiguration - implements io.javaoperatorsdk.operator.api.config.ControllerConfiguration { +public class AnnotationControllerConfiguration

+ implements io.javaoperatorsdk.operator.api.config.ControllerConfiguration

{ - protected final Reconciler reconciler; + protected final Reconciler

reconciler; private final ControllerConfiguration annotation; private List specs; - private Class resourceClass; + private Class

resourceClass; - public AnnotationControllerConfiguration(Reconciler reconciler) { + public AnnotationControllerConfiguration(Reconciler

reconciler) { this.reconciler = reconciler; this.annotation = reconciler.getClass().getAnnotation(ControllerConfiguration.class); if (annotation == null) { @@ -81,10 +98,10 @@ public Set getNamespaces() { @Override @SuppressWarnings("unchecked") - public Class getResourceClass() { + public Class

getResourceClass() { if (resourceClass == null) { resourceClass = - (Class) Utils.getFirstTypeArgumentFromSuperClassOrInterface(reconciler.getClass(), + (Class

) Utils.getFirstTypeArgumentFromSuperClassOrInterface(reconciler.getClass(), Reconciler.class); } return resourceClass; @@ -102,16 +119,16 @@ public String getAssociatedReconcilerClassName() { @SuppressWarnings("unchecked") @Override - public ResourceEventFilter getEventFilter() { - ResourceEventFilter answer = null; + public ResourceEventFilter

getEventFilter() { + ResourceEventFilter

answer = null; - Class>[] filterTypes = - (Class>[]) valueOrDefault(annotation, + Class>[] filterTypes = + (Class>[]) valueOrDefault(annotation, ControllerConfiguration::eventFilters, new Object[] {}); if (filterTypes.length > 0) { for (var filterType : filterTypes) { try { - ResourceEventFilter filter = filterType.getConstructor().newInstance(); + ResourceEventFilter

filter = filterType.getConstructor().newInstance(); if (answer == null) { answer = filter; @@ -141,14 +158,99 @@ public Optional reconciliationMaxInterval() { } } - public static T valueOrDefault( - ControllerConfiguration controllerConfiguration, - Function mapper, - T defaultValue) { - if (controllerConfiguration == null) { - return defaultValue; + @Override + public RateLimiter getRateLimiter() { + if (annotation.rateLimit() != null) { + return new PeriodRateLimiter(Duration.of(annotation.rateLimit().refreshPeriod(), + annotation.rateLimit().refreshPeriodTimeUnit().toChronoUnit()), + annotation.rateLimit().limitForPeriod()); } else { - return mapper.apply(controllerConfiguration); + return io.javaoperatorsdk.operator.api.config.ControllerConfiguration.super.getRateLimiter(); + } + } + + @Override + @SuppressWarnings("unchecked") + public Optional> onAddFilter() { + return (Optional>) createFilter(annotation.onAddFilter(), FilterType.onAdd, + annotation.getClass().getSimpleName()); + } + + private enum FilterType { + onAdd(VoidOnAddFilter.class), onUpdate(VoidOnUpdateFilter.class), onDelete( + VoidOnDeleteFilter.class), generic(VoidGenericFilter.class); + + final Class defaultValue; + + FilterType(Class defaultValue) { + this.defaultValue = defaultValue; + } + } + + private Optional createFilter(Class filter, FilterType filterType, String origin) { + if (filterType.defaultValue.equals(filter)) { + return Optional.empty(); + } else { + try { + var instance = (T) filter.getDeclaredConstructor().newInstance(); + return Optional.of(instance); + } catch (InstantiationException | IllegalAccessException | InvocationTargetException + | NoSuchMethodException e) { + throw new OperatorException( + "Couldn't create " + filterType + " filter from " + filter.getName() + " class in " + + origin + " for reconciler " + getName(), + e); + } + } + } + + @SuppressWarnings("unchecked") + @Override + public Optional> onUpdateFilter() { + return (Optional>) createFilter(annotation.onUpdateFilter(), + FilterType.onUpdate, annotation.getClass().getSimpleName()); + } + + @SuppressWarnings("unchecked") + @Override + public Optional> genericFilter() { + return (Optional>) createFilter(annotation.genericFilter(), + FilterType.generic, annotation.getClass().getSimpleName()); + } + + @Override + public List getEventSources() { + var informerSpecs = Arrays + .stream(annotation.eventSources().informers()).map(this::toInformerEventSourceSpec); + var otherEventSources = Arrays + .stream(annotation.eventSources().others()).map(this::toEventSourceSpec); + return Stream.concat(informerSpecs, otherEventSources).collect(Collectors.toList()); + } + + @SuppressWarnings("unchecked") + private EventSourceSpec toEventSourceSpec(EventSource eventSource) { + return new GenericEventSourceSpec(eventSource.name(), eventSource.type()); + } + + @SuppressWarnings("unchecked") + private InformerEventSourceSpec toInformerEventSourceSpec(Informer informer) { + try { + Set namespaces = new HashSet<>(); + Collections.addAll(namespaces, informer.namespaces()); + SecondaryToPrimaryMapper secondaryToPrimaryMapper = + informer.secondaryToPrimaryMapper().getConstructor().newInstance(); + PrimaryToSecondaryMapper primaryToSecondaryMapper = null; + // todo unit test + if (!informer.primaryToSecondaryMapper().equals(VoidPrimaryToSecondaryMapper.class)) { + primaryToSecondaryMapper = + informer.primaryToSecondaryMapper().getConstructor().newInstance(); + } + return new InformerEventSourceSpec(informer.name(), informer.resourceType(), + informer.labelSelector(), namespaces, + informer.followNamespaceChanges(), secondaryToPrimaryMapper, primaryToSecondaryMapper); + } catch (InstantiationException | IllegalAccessException | InvocationTargetException + | NoSuchMethodException e) { + throw new IllegalStateException(e); } } @@ -172,12 +274,17 @@ public List getDependentResources() { } final var name = getName(dependent, dependentType); - final var spec = specsMap.get(name); + var spec = specsMap.get(name); if (spec != null) { throw new IllegalArgumentException( "A DependentResource named: " + name + " already exists: " + spec); } - specsMap.put(name, new DependentResourceSpec(dependentType, config, name)); + spec = new DependentResourceSpec(dependentType, config, name, + Set.of(dependent.dependsOn()), + instantiateConditionIfNotVoid(dependent.readyPostcondition()), + instantiateConditionIfNotVoid(dependent.reconcilePrecondition()), + instantiateConditionIfNotVoid(dependent.deletePostcondition())); + specsMap.put(name, spec); } specs = specsMap.values().stream().collect(Collectors.toUnmodifiableList()); @@ -185,6 +292,20 @@ public List getDependentResources() { return specs; } + private Condition instantiateConditionIfNotVoid(Class condition) { + if (condition != VoidCondition.class) { + try { + return condition.getDeclaredConstructor().newInstance(); + } catch (InstantiationException + | IllegalAccessException + | InvocationTargetException + | NoSuchMethodException e) { + throw new OperatorException(e); + } + } + return null; + } + private String getName(Dependent dependent, Class dependentType) { var name = dependent.name(); if (name.isBlank()) { @@ -193,26 +314,60 @@ private String getName(Dependent dependent, Class d return name; } + @SuppressWarnings({"rawtypes", "unchecked"}) private Object createKubernetesResourceConfig(Class dependentType) { + Object config; - final var kubeDependent = dependentType.getAnnotation(KubernetesDependent.class); + final var kubeDependent = dependentType.getAnnotation( + io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent.class); var namespaces = getNamespaces(); var configuredNS = false; - if (kubeDependent != null && !Arrays.equals(KubernetesDependent.DEFAULT_NAMESPACES, - kubeDependent.namespaces())) { - namespaces = Set.of(kubeDependent.namespaces()); - configuredNS = true; - } - String labelSelector = null; + Predicate onAddFilter = null; + BiPredicate onUpdateFilter = null; + BiPredicate onDeleteFilter = null; + Predicate genericFilter = null; if (kubeDependent != null) { + if (!Arrays.equals(KubernetesDependent.DEFAULT_NAMESPACES, + kubeDependent.namespaces())) { + namespaces = Set.of(kubeDependent.namespaces()); + configuredNS = true; + } + final var fromAnnotation = kubeDependent.labelSelector(); labelSelector = Constants.NO_VALUE_SET.equals(fromAnnotation) ? null : fromAnnotation; + + final var kubeDependentName = KubernetesDependent.class.getSimpleName(); + onAddFilter = createFilter(kubeDependent.onAddFilter(), FilterType.onAdd, kubeDependentName) + .orElse(null); + onUpdateFilter = + createFilter(kubeDependent.onUpdateFilter(), FilterType.onUpdate, kubeDependentName) + .orElse(null); + onDeleteFilter = + createFilter(kubeDependent.onDeleteFilter(), FilterType.onDelete, kubeDependentName) + .orElse(null); + genericFilter = + createFilter(kubeDependent.genericFilter(), FilterType.generic, kubeDependentName) + .orElse(null); } config = - new KubernetesDependentResourceConfig(namespaces, labelSelector, configuredNS); + new KubernetesDependentResourceConfig(namespaces, labelSelector, configuredNS, onAddFilter, + onUpdateFilter, onDeleteFilter, genericFilter); + return config; } + + public static T valueOrDefault( + ControllerConfiguration controllerConfiguration, + Function mapper, + T defaultValue) { + if (controllerConfiguration == null) { + return defaultValue; + } else { + return mapper.apply(controllerConfiguration); + } + } + } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java index e2f3ac77c9..27f8d194d1 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java @@ -80,7 +80,7 @@ default boolean checkCRDAndValidateLocalModel() { return false; } - int DEFAULT_RECONCILIATION_THREADS_NUMBER = 5; + int DEFAULT_RECONCILIATION_THREADS_NUMBER = 10; /** * Retrieves the maximum number of threads the operator can spin out to dispatch reconciliation diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProvider.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProvider.java index 50419f6a5d..0a8503ae05 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProvider.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProvider.java @@ -7,10 +7,8 @@ * to the ConfigurationService is via the reconciliation context. */ public class ConfigurationServiceProvider { - static final ConfigurationService DEFAULT = - new BaseConfigurationService(Utils.loadFromProperties()); private static ConfigurationService instance; - private static ConfigurationService defaultConfigurationService = DEFAULT; + private static ConfigurationService defaultConfigurationService = createDefault(); private static boolean alreadyConfigured = false; private ConfigurationServiceProvider() {} @@ -64,8 +62,12 @@ synchronized static ConfigurationService getDefault() { } public synchronized static void reset() { - defaultConfigurationService = DEFAULT; + defaultConfigurationService = createDefault(); instance = null; alreadyConfigured = false; } + + static ConfigurationService createDefault() { + return new BaseConfigurationService(Utils.loadFromProperties()); + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfiguration.java index 33e5f0218f..83c9e5696e 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfiguration.java @@ -8,11 +8,18 @@ import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.ReconcilerUtils; import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.EventSourceSpec; +import io.javaoperatorsdk.operator.processing.event.rate.PeriodRateLimiter; +import io.javaoperatorsdk.operator.processing.event.rate.RateLimiter; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilters; +import io.javaoperatorsdk.operator.processing.retry.GenericRetry; +import io.javaoperatorsdk.operator.processing.retry.Retry; public interface ControllerConfiguration extends ResourceConfiguration { + RateLimiter DEFAULT_RATE_LIMITER = new PeriodRateLimiter(); + default String getName() { return ReconcilerUtils.getDefaultReconcilerName(getAssociatedReconcilerClassName()); } @@ -27,10 +34,24 @@ default boolean isGenerationAware() { String getAssociatedReconcilerClassName(); + default Retry getRetry() { + return GenericRetry.fromConfiguration(getRetryConfiguration()); // NOSONAR + } + + /** + * Use getRetry instead. + * + * @return configuration for retry. + */ + @Deprecated default RetryConfiguration getRetryConfiguration() { return RetryConfiguration.DEFAULT; } + default RateLimiter getRateLimiter() { + return DEFAULT_RATE_LIMITER; + } + /** * Allow controllers to filter events before they are passed to the * {@link io.javaoperatorsdk.operator.processing.event.EventHandler}. @@ -46,6 +67,10 @@ default ResourceEventFilter getEventFilter() { return ResourceEventFilters.passthrough(); } + default List getEventSources() { + return Collections.emptyList(); + } + @SuppressWarnings("rawtypes") default List getDependentResources() { return Collections.emptyList(); diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfigurationOverrider.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfigurationOverrider.java index d81a7fcb03..c51e9dd0c4 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfigurationOverrider.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ControllerConfigurationOverrider.java @@ -1,14 +1,23 @@ package io.javaoperatorsdk.operator.api.config; import java.time.Duration; -import java.util.*; +import java.util.HashSet; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Optional; +import java.util.Set; +import java.util.function.BiPredicate; import java.util.function.Predicate; import java.util.stream.Collectors; import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.EventSourceSpec; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfig; +import io.javaoperatorsdk.operator.processing.event.rate.RateLimiter; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter; +import io.javaoperatorsdk.operator.processing.retry.GenericRetry; +import io.javaoperatorsdk.operator.processing.retry.Retry; import static io.javaoperatorsdk.operator.api.reconciler.Constants.DEFAULT_NAMESPACES_SET; import static io.javaoperatorsdk.operator.api.reconciler.Constants.WATCH_CURRENT_NAMESPACE_SET; @@ -19,26 +28,36 @@ public class ControllerConfigurationOverrider { private String finalizer; private boolean generationAware; private Set namespaces; - private RetryConfiguration retry; + private Retry retry; private String labelSelector; private ResourceEventFilter customResourcePredicate; private final ControllerConfiguration original; private Duration reconciliationMaxInterval; private final LinkedHashMap namedDependentResourceSpecs; + private final List eventSourceSpecs; + private Predicate onAddFilter; + private BiPredicate onUpdateFilter; + private Predicate genericFilter; + private RateLimiter rateLimiter; private ControllerConfigurationOverrider(ControllerConfiguration original) { finalizer = original.getFinalizerName(); generationAware = original.isGenerationAware(); namespaces = new HashSet<>(original.getNamespaces()); - retry = original.getRetryConfiguration(); + retry = original.getRetry(); labelSelector = original.getLabelSelector(); customResourcePredicate = original.getEventFilter(); reconciliationMaxInterval = original.reconciliationMaxInterval().orElse(null); + this.eventSourceSpecs = original.getEventSources(); // make the original specs modifiable final var dependentResources = original.getDependentResources(); namedDependentResourceSpecs = new LinkedHashMap<>(dependentResources.size()); + this.onAddFilter = original.onAddFilter().orElse(null); + this.onUpdateFilter = original.onUpdateFilter().orElse(null); + this.genericFilter = original.genericFilter().orElse(null); dependentResources.forEach(drs -> namedDependentResourceSpecs.put(drs.getName(), drs)); this.original = original; + this.rateLimiter = original.getRateLimiter(); } public ControllerConfigurationOverrider withFinalizer(String finalizer) { @@ -90,11 +109,22 @@ public ControllerConfigurationOverrider watchingAllNamespaces() { return this; } - public ControllerConfigurationOverrider withRetry(RetryConfiguration retry) { + public ControllerConfigurationOverrider withRetry(Retry retry) { this.retry = retry; return this; } + @Deprecated + public ControllerConfigurationOverrider withRetry(RetryConfiguration retry) { + this.retry = GenericRetry.fromConfiguration(retry); + return this; + } + + public ControllerConfigurationOverrider withRateLimiter(RateLimiter rateLimiter) { + this.rateLimiter = rateLimiter; + return this; + } + public ControllerConfigurationOverrider withLabelSelector(String labelSelector) { this.labelSelector = labelSelector; return this; @@ -112,6 +142,21 @@ public ControllerConfigurationOverrider withReconciliationMaxInterval( return this; } + public ControllerConfigurationOverrider withOnAddFilter(Predicate onAddFilter) { + this.onAddFilter = onAddFilter; + return this; + } + + public ControllerConfigurationOverrider withOnUpdateFilter(BiPredicate onUpdateFilter) { + this.onUpdateFilter = onUpdateFilter; + return this; + } + + public ControllerConfigurationOverrider withGenericFilter(Predicate genericFilter) { + this.genericFilter = genericFilter; + return this; + } + public ControllerConfigurationOverrider replacingNamedDependentResourceConfig(String name, Object dependentResourceConfig) { @@ -140,11 +185,10 @@ public ControllerConfiguration build() { // if the spec has a config and it's a KubernetesDependentResourceConfig, update the // namespaces if needed, otherwise, just return the existing spec final Optional maybeConfig = spec.getDependentResourceConfiguration(); - final Class drClass = drsEntry.getValue().getDependentResourceClass(); return maybeConfig.filter(KubernetesDependentResourceConfig.class::isInstance) .map(KubernetesDependentResourceConfig.class::cast) .filter(Predicate.not(KubernetesDependentResourceConfig::wereNamespacesConfigured)) - .map(c -> updateSpec(drsEntry.getKey(), drClass, c)) + .map(c -> updateSpec(drsEntry.getKey(), spec, c)) .orElse(drsEntry.getValue()); }).collect(Collectors.toUnmodifiableList()); @@ -160,13 +204,20 @@ public ControllerConfiguration build() { customResourcePredicate, original.getResourceClass(), reconciliationMaxInterval, + onAddFilter, + onUpdateFilter, + genericFilter, + rateLimiter, + eventSourceSpecs, newDependentSpecs); } @SuppressWarnings({"rawtypes", "unchecked"}) - private DependentResourceSpec updateSpec(String name, Class drClass, + private DependentResourceSpec updateSpec(String name, DependentResourceSpec spec, KubernetesDependentResourceConfig c) { - return new DependentResourceSpec(drClass, c.setNamespaces(namespaces), name); + return new DependentResourceSpec(spec.getDependentResourceClass(), + c.setNamespaces(namespaces), name, spec.getDependsOn(), spec.getReadyCondition(), + spec.getReconcileCondition(), spec.getDeletePostCondition()); } public static ControllerConfigurationOverrider override( diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultControllerConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultControllerConfiguration.java index 38373f18f3..b5e6134644 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultControllerConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultControllerConfiguration.java @@ -5,10 +5,15 @@ import java.util.List; import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.EventSourceSpec; +import io.javaoperatorsdk.operator.processing.event.rate.RateLimiter; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter; +import io.javaoperatorsdk.operator.processing.retry.Retry; @SuppressWarnings("rawtypes") public class DefaultControllerConfiguration @@ -20,10 +25,12 @@ public class DefaultControllerConfiguration private final String crdName; private final String finalizer; private final boolean generationAware; - private final RetryConfiguration retryConfiguration; + private final Retry retry; private final ResourceEventFilter resourceEventFilter; + private final List eventSourceSpecs; private final List dependents; private final Duration reconciliationMaxInterval; + private final RateLimiter rateLimiter; // NOSONAR constructor is meant to provide all information public DefaultControllerConfiguration( @@ -33,25 +40,31 @@ public DefaultControllerConfiguration( String finalizer, boolean generationAware, Set namespaces, - RetryConfiguration retryConfiguration, + Retry retry, String labelSelector, ResourceEventFilter resourceEventFilter, Class resourceClass, Duration reconciliationMaxInterval, + Predicate onAddFilter, + BiPredicate onUpdateFilter, + Predicate genericFilter, + RateLimiter rateLimiter, + List eventSourceSpecs, List dependents) { - super(labelSelector, resourceClass, namespaces); + super(labelSelector, resourceClass, onAddFilter, onUpdateFilter, genericFilter, namespaces); this.associatedControllerClassName = associatedControllerClassName; this.name = name; this.crdName = crdName; this.finalizer = finalizer; this.generationAware = generationAware; + this.eventSourceSpecs = eventSourceSpecs; this.reconciliationMaxInterval = reconciliationMaxInterval; - this.retryConfiguration = - retryConfiguration == null - ? ControllerConfiguration.super.getRetryConfiguration() - : retryConfiguration; + this.retry = + retry == null + ? ControllerConfiguration.super.getRetry() + : retry; this.resourceEventFilter = resourceEventFilter; - + this.rateLimiter = rateLimiter; this.dependents = dependents != null ? dependents : Collections.emptyList(); } @@ -81,8 +94,8 @@ public String getAssociatedReconcilerClassName() { } @Override - public RetryConfiguration getRetryConfiguration() { - return retryConfiguration; + public Retry getRetry() { + return retry; } @Override @@ -99,4 +112,17 @@ public List getDependentResources() { public Optional reconciliationMaxInterval() { return Optional.ofNullable(reconciliationMaxInterval); } + + @Override + public List getEventSources() { + if (eventSourceSpecs == null) { + return Collections.emptyList(); + } + return eventSourceSpecs; + } + + @Override + public RateLimiter getRateLimiter() { + return rateLimiter; + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultResourceConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultResourceConfiguration.java index 407c352be0..4bb13fa06c 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultResourceConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/DefaultResourceConfiguration.java @@ -1,6 +1,9 @@ package io.javaoperatorsdk.operator.api.config; +import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import io.fabric8.kubernetes.api.model.HasMetadata; @@ -12,18 +15,26 @@ public class DefaultResourceConfiguration private final String labelSelector; private final Set namespaces; private final Class resourceClass; + private final Predicate onAddFilter; + private final BiPredicate onUpdateFilter; + private final Predicate genericFilter; public DefaultResourceConfiguration(String labelSelector, Class resourceClass, - String... namespaces) { - this(labelSelector, resourceClass, + Predicate onAddFilter, + BiPredicate onUpdateFilter, Predicate genericFilter, String... namespaces) { + this(labelSelector, resourceClass, onAddFilter, onUpdateFilter, genericFilter, namespaces == null || namespaces.length == 0 ? DEFAULT_NAMESPACES_SET : Set.of(namespaces)); } public DefaultResourceConfiguration(String labelSelector, Class resourceClass, - Set namespaces) { + Predicate onAddFilter, + BiPredicate onUpdateFilter, Predicate genericFilter, Set namespaces) { this.labelSelector = labelSelector; this.resourceClass = resourceClass; + this.onAddFilter = onAddFilter; + this.onUpdateFilter = onUpdateFilter; + this.genericFilter = genericFilter; this.namespaces = namespaces == null || namespaces.isEmpty() ? DEFAULT_NAMESPACES_SET : namespaces; @@ -48,4 +59,18 @@ public Set getNamespaces() { public Class getResourceClass() { return resourceClass; } + + @Override + public Optional> onAddFilter() { + return Optional.ofNullable(onAddFilter); + } + + @Override + public Optional> onUpdateFilter() { + return Optional.ofNullable(onUpdateFilter); + } + + public Optional> genericFilter() { + return Optional.ofNullable(genericFilter); + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ResourceConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ResourceConfiguration.java index 70d2b765a5..636ae99b49 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ResourceConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ResourceConfiguration.java @@ -1,7 +1,10 @@ package io.javaoperatorsdk.operator.api.config; import java.util.Collections; +import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.OperatorException; @@ -17,6 +20,18 @@ default String getResourceTypeName() { return ReconcilerUtils.getResourceTypeName(getResourceClass()); } + default Optional> onAddFilter() { + return Optional.empty(); + } + + default Optional> onUpdateFilter() { + return Optional.empty(); + } + + default Optional> genericFilter() { + return Optional.empty(); + } + /** * Retrieves the label selector that is used to filter which resources are actually watched by the * associated event source. See the official documentation on the diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/dependent/DependentResourceSpec.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/dependent/DependentResourceSpec.java index 40a7db53c9..2f22dd5a96 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/dependent/DependentResourceSpec.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/dependent/DependentResourceSpec.java @@ -1,9 +1,12 @@ package io.javaoperatorsdk.operator.api.config.dependent; +import java.util.Collections; import java.util.Objects; import java.util.Optional; +import java.util.Set; import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; public class DependentResourceSpec, C> { @@ -13,11 +16,30 @@ public class DependentResourceSpec, C> { private final String name; + private final Set dependsOn; + + private final Condition readyCondition; + + private final Condition reconcileCondition; + + private final Condition deletePostCondition; + public DependentResourceSpec(Class dependentResourceClass, C dependentResourceConfig, String name) { + this(dependentResourceClass, dependentResourceConfig, name, Collections.emptySet(), null, null, + null); + } + + public DependentResourceSpec(Class dependentResourceClass, C dependentResourceConfig, + String name, Set dependsOn, Condition readyCondition, + Condition reconcileCondition, Condition deletePostCondition) { this.dependentResourceClass = dependentResourceClass; this.dependentResourceConfig = dependentResourceConfig; this.name = name; + this.dependsOn = dependsOn; + this.readyCondition = readyCondition; + this.reconcileCondition = reconcileCondition; + this.deletePostCondition = deletePostCondition; } public Class getDependentResourceClass() { @@ -55,4 +77,23 @@ public boolean equals(Object o) { public int hashCode() { return Objects.hash(name); } + + public Set getDependsOn() { + return dependsOn; + } + + @SuppressWarnings("rawtypes") + public Condition getReadyCondition() { + return readyCondition; + } + + @SuppressWarnings("rawtypes") + public Condition getReconcileCondition() { + return reconcileCondition; + } + + @SuppressWarnings("rawtypes") + public Condition getDeletePostCondition() { + return deletePostCondition; + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/EventSourceSpec.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/EventSourceSpec.java new file mode 100644 index 0000000000..ff312c93b9 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/EventSourceSpec.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.api.config.eventsource; + +public abstract class EventSourceSpec { + + private final String name; + + public EventSourceSpec(String name) { + this.name = name; + } + + public String getName() { + return name; + } + + // todo add filters +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/GenericEventSourceSpec.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/GenericEventSourceSpec.java new file mode 100644 index 0000000000..e20e044af6 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/GenericEventSourceSpec.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.api.config.eventsource; + +@SuppressWarnings("rawtypes") +public class GenericEventSourceSpec extends EventSourceSpec { + + private final Class eventSourceClass; + + public GenericEventSourceSpec(String name, Class eventSourceClass) { + super(name); + this.eventSourceClass = eventSourceClass; + } + + public Class getEventSourceClass() { + return eventSourceClass; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/InformerEventSourceSpec.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/InformerEventSourceSpec.java new file mode 100644 index 0000000000..440c40d2d5 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/eventsource/InformerEventSourceSpec.java @@ -0,0 +1,60 @@ +package io.javaoperatorsdk.operator.api.config.eventsource; + +import java.util.Optional; +import java.util.Set; + +import io.javaoperatorsdk.operator.processing.event.source.PrimaryToSecondaryMapper; +import io.javaoperatorsdk.operator.processing.event.source.SecondaryToPrimaryMapper; + +@SuppressWarnings("rawtypes") +public class InformerEventSourceSpec extends EventSourceSpec { + + private final String labelSelector; + + private final Set namespaces; + + private final boolean followNamespaceChanges; + + private final SecondaryToPrimaryMapper secondaryToPrimaryMapper; + + private PrimaryToSecondaryMapper primaryToSecondaryMapper; + + private Class resourceType; + + public InformerEventSourceSpec(String name, Class resourceType, String labelSelector, + Set namespaces, + boolean followNamespaceChanges, SecondaryToPrimaryMapper secondaryToPrimaryMapper, + PrimaryToSecondaryMapper primaryToSecondaryMapper) { + super(name); + this.labelSelector = labelSelector; + this.namespaces = namespaces; + this.followNamespaceChanges = followNamespaceChanges; + this.secondaryToPrimaryMapper = secondaryToPrimaryMapper; + this.primaryToSecondaryMapper = primaryToSecondaryMapper; + this.resourceType = resourceType; + } + + public String getLabelSelector() { + return labelSelector; + } + + public Set getNamespaces() { + return namespaces; + } + + public boolean isFollowNamespaceChanges() { + return followNamespaceChanges; + } + + public SecondaryToPrimaryMapper getSecondaryToPrimaryMapper() { + return secondaryToPrimaryMapper; + } + + public Optional getPrimaryToSecondaryMapper() { + return Optional.ofNullable(primaryToSecondaryMapper); + } + + public Class getResourceType() { + return resourceType; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/InformerConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/InformerConfiguration.java index ddfae2919e..e9ee02c91b 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/InformerConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/InformerConfiguration.java @@ -1,7 +1,10 @@ package io.javaoperatorsdk.operator.api.config.informer; import java.util.Objects; +import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.config.DefaultResourceConfiguration; @@ -23,28 +26,41 @@ class DefaultInformerConfiguration extends private final PrimaryToSecondaryMapper primaryToSecondaryMapper; private final SecondaryToPrimaryMapper secondaryToPrimaryMapper; private final boolean followControllerNamespaceChanges; + private final BiPredicate onDeleteFilter; protected DefaultInformerConfiguration(String labelSelector, Class resourceClass, PrimaryToSecondaryMapper primaryToSecondaryMapper, SecondaryToPrimaryMapper secondaryToPrimaryMapper, - Set namespaces, boolean followControllerNamespaceChanges) { - super(labelSelector, resourceClass, namespaces); + Set namespaces, boolean followControllerNamespaceChanges, + Predicate onAddFilter, + BiPredicate onUpdateFilter, + BiPredicate onDeleteFilter, + Predicate genericFilter) { + super(labelSelector, resourceClass, onAddFilter, onUpdateFilter, genericFilter, namespaces); this.followControllerNamespaceChanges = followControllerNamespaceChanges; + this.primaryToSecondaryMapper = primaryToSecondaryMapper; this.secondaryToPrimaryMapper = Objects.requireNonNullElse(secondaryToPrimaryMapper, Mappers.fromOwnerReference()); + this.onDeleteFilter = onDeleteFilter; } + @Override public boolean followControllerNamespaceChanges() { return followControllerNamespaceChanges; } + @Override public SecondaryToPrimaryMapper getSecondaryToPrimaryMapper() { return secondaryToPrimaryMapper; } + public Optional> onDeleteFilter() { + return Optional.ofNullable(onDeleteFilter); + } + @Override public

PrimaryToSecondaryMapper

getPrimaryToSecondaryMapper() { return (PrimaryToSecondaryMapper

) primaryToSecondaryMapper; @@ -61,6 +77,14 @@ public

PrimaryToSecondaryMapper

getPrimaryToSecondary SecondaryToPrimaryMapper getSecondaryToPrimaryMapper(); + Optional> onAddFilter(); + + Optional> onUpdateFilter(); + + Optional> onDeleteFilter(); + + Optional> genericFilter(); +

PrimaryToSecondaryMapper

getPrimaryToSecondaryMapper(); @SuppressWarnings("unused") @@ -71,6 +95,10 @@ class InformerConfigurationBuilder { private Set namespaces; private String labelSelector; private final Class resourceClass; + private Predicate onAddFilter; + private BiPredicate onUpdateFilter; + private BiPredicate onDeleteFilter; + private Predicate genericFilter; private boolean inheritControllerNamespacesOnChange = false; private InformerConfigurationBuilder(Class resourceClass) { @@ -151,11 +179,33 @@ public InformerConfigurationBuilder withLabelSelector(String labelSelector) { return this; } + public InformerConfigurationBuilder withOnAddFilter(Predicate onAddFilter) { + this.onAddFilter = onAddFilter; + return this; + } + + public InformerConfigurationBuilder withOnUpdateFilter(BiPredicate onUpdateFilter) { + this.onUpdateFilter = onUpdateFilter; + return this; + } + + public InformerConfigurationBuilder withOnDeleteFilter( + BiPredicate onDeleteFilter) { + this.onDeleteFilter = onDeleteFilter; + return this; + } + + public InformerConfigurationBuilder withGenericFilter(Predicate genericFilter) { + this.genericFilter = genericFilter; + return this; + } + public InformerConfiguration build() { return new DefaultInformerConfiguration<>(labelSelector, resourceClass, primaryToSecondaryMapper, secondaryToPrimaryMapper, - namespaces, inheritControllerNamespacesOnChange); + namespaces, inheritControllerNamespacesOnChange, onAddFilter, onUpdateFilter, + onDeleteFilter, genericFilter); } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Cleaner.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Cleaner.java index e2e70a246d..2a53d5730b 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Cleaner.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Cleaner.java @@ -5,7 +5,7 @@ public interface Cleaner

{ /** - * Note that this method turns on automatic finalizer usage. + * This method turns on automatic finalizer usage. * * The implementation should delete the associated component(s). This method is called when an * object is marked for deletion. After it's executed the custom resource finalizer is diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Constants.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Constants.java index 68b88b4455..8d9c2ef200 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Constants.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Constants.java @@ -5,8 +5,9 @@ public final class Constants { + public static final Set DEFAULT_NAMESPACES_SET = - Collections.singleton(Constants.WATCH_ALL_NAMESPACES); + Collections.singleton(Constants.DEFAULT_NAMESPACE); public static final Set WATCH_CURRENT_NAMESPACE_SET = Collections.singleton(Constants.WATCH_CURRENT_NAMESPACE); @@ -18,6 +19,9 @@ public final class Constants { public static final String WATCH_ALL_NAMESPACES = "JOSDK_ALL_NAMESPACES"; public static final long NO_RECONCILIATION_MAX_INTERVAL = -1L; public static final String SAME_AS_CONTROLLER = "JOSDK_SAME_AS_CONTROLLER"; + public static final String DEFAULT_NAMESPACE = WATCH_ALL_NAMESPACES; + + public static final String GENERATED_EVENT_SOURCE_NAME = "JOSDK_GENERATED_NAME"; private Constants() {} } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Context.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Context.java index 845810c8a1..f6bc2d6a1d 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Context.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Context.java @@ -22,4 +22,5 @@ default Optional getSecondaryResource(Class expectedType) { ControllerConfiguration

getControllerConfiguration(); ManagedDependentResourceContext managedDependentResourceContext(); + } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ControllerConfiguration.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ControllerConfiguration.java index a5e1459cb7..b660d2d4ab 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ControllerConfiguration.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ControllerConfiguration.java @@ -4,9 +4,15 @@ import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; +import java.util.function.BiPredicate; +import java.util.function.Predicate; +import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidGenericFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnAddFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnUpdateFilter; @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.TYPE}) @@ -51,15 +57,31 @@ String labelSelector() default Constants.NO_VALUE_SET; /** - *

- * Resource event filters only applies on events of the main custom resource. Not on events from - * other event sources nor the periodic events. - *

+ * @deprecated Use onAddFilter, onUpdateFilter instead. + * + *

+ * Resource event filters only applies on events of the main custom resource. Not on + * events from other event sources nor the periodic events. + *

* * @return the list of event filters. */ + @Deprecated(forRemoval = true) Class[] eventFilters() default {}; + /** + * Filter of onAdd events of resources. + **/ + Class> onAddFilter() default VoidOnAddFilter.class; + + /** Filter of onUpdate events of resources. */ + Class> onUpdateFilter() default VoidOnUpdateFilter.class; + + /** + * Filter applied to all operations (add, update, delete). Used to ignore some resources. + **/ + Class> genericFilter() default VoidGenericFilter.class; + /** * Optional configuration of the maximal interval the SDK will wait for a reconciliation request * to happen before one will be automatically triggered. @@ -69,6 +91,11 @@ ReconciliationMaxInterval reconciliationMaxInterval() default @ReconciliationMaxInterval( interval = 10); + EventSources eventSources() default @EventSources; + + + RateLimit rateLimit() default @RateLimit; + /** * Optional list of {@link Dependent} configurations which associate a resource type to a * {@link io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource} implementation diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/DefaultContext.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/DefaultContext.java index 0b6d90065b..9891d358d4 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/DefaultContext.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/DefaultContext.java @@ -6,6 +6,7 @@ import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.config.ControllerConfiguration; +import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.DefaultManagedDependentResourceContext; import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.ManagedDependentResourceContext; import io.javaoperatorsdk.operator.processing.Controller; @@ -15,14 +16,14 @@ public class DefaultContext

implements Context

{ private final Controller

controller; private final P primaryResource; private final ControllerConfiguration

controllerConfiguration; - private final ManagedDependentResourceContext managedDependentResourceContext; + private final DefaultManagedDependentResourceContext defaultManagedDependentResourceContext; public DefaultContext(RetryInfo retryInfo, Controller

controller, P primaryResource) { this.retryInfo = retryInfo; this.controller = controller; this.primaryResource = primaryResource; this.controllerConfiguration = controller.getConfiguration(); - this.managedDependentResourceContext = new ManagedDependentResourceContext(); + this.defaultManagedDependentResourceContext = new DefaultManagedDependentResourceContext(); } @Override @@ -52,8 +53,9 @@ public ControllerConfiguration

getControllerConfiguration() { return controllerConfiguration; } + @Override public ManagedDependentResourceContext managedDependentResourceContext() { - return managedDependentResourceContext; + return defaultManagedDependentResourceContext; } public DefaultContext

setRetryInfo(RetryInfo retryInfo) { diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/EventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/EventSource.java new file mode 100644 index 0000000000..978a2b7ee4 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/EventSource.java @@ -0,0 +1,33 @@ +package io.javaoperatorsdk.operator.api.reconciler; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; +import java.util.function.BiPredicate; +import java.util.function.Predicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidGenericFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnAddFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnDeleteFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnUpdateFilter; + +import static io.javaoperatorsdk.operator.api.reconciler.Constants.GENERATED_EVENT_SOURCE_NAME; + +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.TYPE}) +public @interface EventSource { + + String name() default GENERATED_EVENT_SOURCE_NAME; + + Class type(); + + Class> onAddFilter() default VoidOnAddFilter.class; + + Class> onUpdateFilter() default VoidOnUpdateFilter.class; + + Class> onDeleteFilter() default VoidOnDeleteFilter.class; + + Class> genericFilter() default VoidGenericFilter.class; +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/EventSources.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/EventSources.java new file mode 100644 index 0000000000..0f8c336aeb --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/EventSources.java @@ -0,0 +1,23 @@ +package io.javaoperatorsdk.operator.api.reconciler; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.TYPE}) +public @interface EventSources { + + /** + * + * @return declarations of + * {@link io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource} + */ + Informer[] informers() default {}; + + /** + * @return non informer based (external resource) event sources + */ + EventSource[] others() default {}; +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Informer.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Informer.java new file mode 100644 index 0000000000..18fbeacf59 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Informer.java @@ -0,0 +1,46 @@ +package io.javaoperatorsdk.operator.api.reconciler; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; +import java.util.function.BiPredicate; +import java.util.function.Predicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.processing.event.source.PrimaryToSecondaryMapper; +import io.javaoperatorsdk.operator.processing.event.source.SecondaryToPrimaryMapper; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidGenericFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnAddFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnDeleteFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnUpdateFilter; +import io.javaoperatorsdk.operator.processing.event.source.informer.OwnerReferenceSecondaryToPrimaryMapper; + +import static io.javaoperatorsdk.operator.api.reconciler.Constants.GENERATED_EVENT_SOURCE_NAME; + +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.TYPE}) +public @interface Informer { + + String name() default GENERATED_EVENT_SOURCE_NAME; + + Class resourceType(); + + String labelSelector() default Constants.NO_VALUE_SET; + + String[] namespaces() default Constants.DEFAULT_NAMESPACE; + + boolean followNamespaceChanges() default false; + + Class secondaryToPrimaryMapper() default OwnerReferenceSecondaryToPrimaryMapper.class; + + Class primaryToSecondaryMapper() default VoidPrimaryToSecondaryMapper.class; + + Class> onAddFilter() default VoidOnAddFilter.class; + + Class> onUpdateFilter() default VoidOnUpdateFilter.class; + + Class> onDeleteFilter() default VoidOnDeleteFilter.class; + + Class> genericFilter() default VoidGenericFilter.class; +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/RateLimit.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/RateLimit.java new file mode 100644 index 0000000000..6c4feb2b2a --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/RateLimit.java @@ -0,0 +1,23 @@ +package io.javaoperatorsdk.operator.api.reconciler; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; +import java.util.concurrent.TimeUnit; + +import io.javaoperatorsdk.operator.processing.event.rate.PeriodRateLimiter; + +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.TYPE}) +public @interface RateLimit { + + int limitForPeriod() default PeriodRateLimiter.NO_LIMIT_PERIOD; + + int refreshPeriod() default PeriodRateLimiter.DEFAULT_REFRESH_PERIOD_SECONDS; + + /** + * @return time unit for max delay between reconciliations + */ + TimeUnit refreshPeriodTimeUnit() default TimeUnit.SECONDS; +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/UpdateControl.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/UpdateControl.java index 62f9bc0cd2..12f392901d 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/UpdateControl.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/UpdateControl.java @@ -3,15 +3,15 @@ import io.fabric8.kubernetes.api.model.HasMetadata; import io.fabric8.kubernetes.client.CustomResource; -public class UpdateControl extends BaseControl> { +public class UpdateControl

extends BaseControl> { - private final T resource; + private final P resource; private final boolean updateStatus; private final boolean updateResource; private final boolean patch; private UpdateControl( - T resource, boolean updateStatus, boolean updateResource, boolean patch) { + P resource, boolean updateStatus, boolean updateResource, boolean patch) { if ((updateResource || updateStatus) && resource == null) { throw new IllegalArgumentException("CustomResource cannot be null in case of update"); } @@ -92,7 +92,7 @@ public static UpdateControl noUpdate() { return new UpdateControl<>(null, false, false, false); } - public T getResource() { + public P getResource() { return resource; } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/VoidPrimaryToSecondaryMapper.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/VoidPrimaryToSecondaryMapper.java new file mode 100644 index 0000000000..6325d9038e --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/VoidPrimaryToSecondaryMapper.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.api.reconciler; + +import java.util.Set; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.processing.event.ResourceID; +import io.javaoperatorsdk.operator.processing.event.source.PrimaryToSecondaryMapper; + +public class VoidPrimaryToSecondaryMapper

+ implements PrimaryToSecondaryMapper

{ + @Override + public Set toSecondaryResourceIDs(P primary) { + throw new UnsupportedOperationException(); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/Dependent.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/Dependent.java index 25c7418445..0eba3fb598 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/Dependent.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/Dependent.java @@ -1,5 +1,7 @@ package io.javaoperatorsdk.operator.api.reconciler.dependent; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; + import static io.javaoperatorsdk.operator.api.reconciler.Constants.NO_VALUE_SET; public @interface Dependent { @@ -8,4 +10,15 @@ Class type(); String name() default NO_VALUE_SET; + + @SuppressWarnings("rawtypes") + Class readyPostcondition() default VoidCondition.class; + + @SuppressWarnings("rawtypes") + Class reconcilePrecondition() default VoidCondition.class; + + @SuppressWarnings("rawtypes") + Class deletePostcondition() default VoidCondition.class; + + String[] dependsOn() default {}; } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/GarbageCollected.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/GarbageCollected.java new file mode 100644 index 0000000000..1316c44873 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/GarbageCollected.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.api.reconciler.dependent; + +import io.fabric8.kubernetes.api.model.HasMetadata; + +/** + * Should be implemented by {@link DependentResource} implementations that are explicitly deleted + * during reconciliation but which should also benefit from Kubernetes' automated garbage collection + * during the cleanup phase. + *

+ * See this issue + * for more details. + */ +public interface GarbageCollected

extends Deleter

{ + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/VoidCondition.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/VoidCondition.java new file mode 100644 index 0000000000..7f4faf0ed1 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/VoidCondition.java @@ -0,0 +1,14 @@ +package io.javaoperatorsdk.operator.api.reconciler.dependent; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; + +/** Used as default value for Condition in annotations */ +@SuppressWarnings("rawtypes") +public class VoidCondition implements Condition { + @Override + public boolean isMet(DependentResource dependentResource, HasMetadata primary, Context context) { + throw new IllegalStateException("This is a placeholder class, should not be called"); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/DefaultManagedDependentResourceContext.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/DefaultManagedDependentResourceContext.java new file mode 100644 index 0000000000..5b1a21e5dd --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/DefaultManagedDependentResourceContext.java @@ -0,0 +1,61 @@ +package io.javaoperatorsdk.operator.api.reconciler.dependent.managed; + +import java.util.Optional; +import java.util.concurrent.ConcurrentHashMap; + +import io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowCleanupResult; +import io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowReconcileResult; + +@SuppressWarnings("rawtypes") +public class DefaultManagedDependentResourceContext implements ManagedDependentResourceContext { + + private WorkflowReconcileResult workflowReconcileResult; + private WorkflowCleanupResult workflowCleanupResult; + private final ConcurrentHashMap attributes = new ConcurrentHashMap(); + + @Override + public Optional get(Object key, Class expectedType) { + return Optional.ofNullable(attributes.get(key)) + .filter(expectedType::isInstance) + .map(expectedType::cast); + } + + @Override + @SuppressWarnings("unchecked") + public T put(Object key, T value) { + if (value == null) { + return (T) Optional.ofNullable(attributes.remove(key)); + } + return (T) Optional.ofNullable(attributes.put(key, value)); + } + + @Override + @SuppressWarnings("unused") + public T getMandatory(Object key, Class expectedType) { + return get(key, expectedType).orElseThrow(() -> new IllegalStateException( + "Mandatory attribute (key: " + key + ", type: " + expectedType.getName() + + ") is missing or not of the expected type")); + } + + public DefaultManagedDependentResourceContext setWorkflowExecutionResult( + WorkflowReconcileResult workflowReconcileResult) { + this.workflowReconcileResult = workflowReconcileResult; + return this; + } + + public DefaultManagedDependentResourceContext setWorkflowCleanupResult( + WorkflowCleanupResult workflowCleanupResult) { + this.workflowCleanupResult = workflowCleanupResult; + return this; + } + + @Override + public Optional getWorkflowReconcileResult() { + return Optional.ofNullable(workflowReconcileResult); + } + + @Override + public Optional getWorkflowCleanupResult() { + return Optional.ofNullable(workflowCleanupResult); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/ManagedDependentResourceContext.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/ManagedDependentResourceContext.java index 0d0b4c1412..42741c3a97 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/ManagedDependentResourceContext.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/managed/ManagedDependentResourceContext.java @@ -1,21 +1,16 @@ package io.javaoperatorsdk.operator.api.reconciler.dependent.managed; -import java.util.Map; import java.util.Optional; -import java.util.concurrent.ConcurrentHashMap; import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; -import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; +import io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowCleanupResult; +import io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowReconcileResult; /** * Contextual information related to {@link DependentResource} either to retrieve the actual * implementations to interact with them or to pass information between them and/or the reconciler */ -@SuppressWarnings("rawtypes") -public class ManagedDependentResourceContext { - - private final Map reconcileResults = new ConcurrentHashMap<>(); - private final ConcurrentHashMap attributes = new ConcurrentHashMap(); +public interface ManagedDependentResourceContext { /** * Retrieve a contextual object, if it exists and is of the specified expected type, associated @@ -28,11 +23,7 @@ public class ManagedDependentResourceContext { * @return an Optional containing the contextual object or {@link Optional#empty()} if no such * object exists or doesn't match the expected type */ - public Optional get(Object key, Class expectedType) { - return Optional.ofNullable(attributes.get(key)) - .filter(expectedType::isInstance) - .map(expectedType::cast); - } + Optional get(Object key, Class expectedType); /** * Associates the specified contextual value to the specified key. If the value is {@code null}, @@ -46,12 +37,7 @@ public Optional get(Object key, Class expectedType) { * {@link Optional#empty()} if none existed */ @SuppressWarnings("unchecked") - public Optional put(Object key, Object value) { - if (value == null) { - return Optional.ofNullable(attributes.remove(key)); - } - return Optional.ofNullable(attributes.put(key, value)); - } + T put(Object key, T value); /** * Retrieves the value associated with the key or fail with an exception if none exists. @@ -63,35 +49,9 @@ public Optional put(Object key, Object value) { * @see #get(Object, Class) */ @SuppressWarnings("unused") - public T getMandatory(Object key, Class expectedType) { - return get(key, expectedType).orElseThrow(() -> new IllegalStateException( - "Mandatory attribute (key: " + key + ", type: " + expectedType.getName() - + ") is missing or not of the expected type")); - } + T getMandatory(Object key, Class expectedType); - /** - * Retrieve the {@link ReconcileResult}, if it exists, associated with the - * {@link DependentResource} associated with the specified name - * - * @param name the name of the {@link DependentResource} for which we want to retrieve a - * {@link ReconcileResult} - * @return an Optional containing the reconcile result or {@link Optional#empty()} if no such - * result is available - */ - @SuppressWarnings({"rawtypes", "unused"}) - public Optional getReconcileResult(String name) { - return Optional.ofNullable(reconcileResults.get(name)); - } + Optional getWorkflowReconcileResult(); - /** - * Set the {@link ReconcileResult} for the specified {@link DependentResource} implementation. - * - * @param name the name of the {@link DependentResource} for which we want to set the - * {@link ReconcileResult} - * @param reconcileResult the reconcile result associated with the specified - * {@link DependentResource} - */ - public void setReconcileResult(String name, ReconcileResult reconcileResult) { - reconcileResults.put(name, reconcileResult); - } + Optional getWorkflowCleanupResult(); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/Controller.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/Controller.java index 50411ac4e8..7c9b4a4f47 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/Controller.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/Controller.java @@ -1,7 +1,8 @@ package io.javaoperatorsdk.operator.processing; -import java.util.*; -import java.util.stream.Collectors; +import java.lang.reflect.InvocationTargetException; +import java.util.Optional; +import java.util.Set; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -13,20 +14,36 @@ import io.fabric8.kubernetes.client.KubernetesClient; import io.fabric8.kubernetes.client.dsl.MixedOperation; import io.fabric8.kubernetes.client.dsl.Resource; -import io.javaoperatorsdk.operator.*; +import io.javaoperatorsdk.operator.CustomResourceUtils; +import io.javaoperatorsdk.operator.MissingCRDException; +import io.javaoperatorsdk.operator.OperatorException; +import io.javaoperatorsdk.operator.RegisteredController; import io.javaoperatorsdk.operator.api.config.ConfigurationServiceProvider; import io.javaoperatorsdk.operator.api.config.ControllerConfiguration; -import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.EventSourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.GenericEventSourceSpec; +import io.javaoperatorsdk.operator.api.config.eventsource.InformerEventSourceSpec; +import io.javaoperatorsdk.operator.api.config.informer.InformerConfiguration; import io.javaoperatorsdk.operator.api.monitoring.Metrics; import io.javaoperatorsdk.operator.api.monitoring.Metrics.ControllerExecution; -import io.javaoperatorsdk.operator.api.reconciler.*; -import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; -import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.Cleaner; +import io.javaoperatorsdk.operator.api.reconciler.Constants; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.ContextInitializer; +import io.javaoperatorsdk.operator.api.reconciler.DeleteControl; +import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext; +import io.javaoperatorsdk.operator.api.reconciler.EventSourceInitializer; +import io.javaoperatorsdk.operator.api.reconciler.Ignore; +import io.javaoperatorsdk.operator.api.reconciler.Reconciler; +import io.javaoperatorsdk.operator.api.reconciler.UpdateControl; import io.javaoperatorsdk.operator.api.reconciler.dependent.EventSourceProvider; -import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.DependentResourceConfigurator; -import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.KubernetesClientAware; -import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.ManagedDependentResourceException; +import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.DefaultManagedDependentResourceContext; +import io.javaoperatorsdk.operator.processing.dependent.workflow.ManagedWorkflow; +import io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowCleanupResult; import io.javaoperatorsdk.operator.processing.event.EventSourceManager; +import io.javaoperatorsdk.operator.processing.event.NamedEventSource; +import io.javaoperatorsdk.operator.processing.event.source.EventSource; +import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource; import static io.javaoperatorsdk.operator.api.reconciler.Constants.WATCH_CURRENT_NAMESPACE; @@ -41,12 +58,10 @@ public class Controller

private final ControllerConfiguration

configuration; private final KubernetesClient kubernetesClient; private final EventSourceManager

eventSourceManager; - private final LinkedHashMap dependents; private final boolean contextInitializer; - private final boolean hasDeleterDependents; private final boolean isCleaner; private final Metrics metrics; - + private final ManagedWorkflow

managedWorkflow; public Controller(Reconciler

reconciler, ControllerConfiguration

configuration, @@ -57,94 +72,10 @@ public Controller(Reconciler

reconciler, this.metrics = Optional.ofNullable(ConfigurationServiceProvider.instance().getMetrics()) .orElse(Metrics.NOOP); contextInitializer = reconciler instanceof ContextInitializer; - - eventSourceManager = new EventSourceManager<>(this); - - final var hasDeleterHolder = new boolean[] {false}; - final var specs = configuration.getDependentResources(); - final var size = specs.size(); - if (size == 0) { - dependents = new LinkedHashMap<>(); - } else { - final Map dependentsHolder = new LinkedHashMap<>(size); - specs.forEach(drs -> { - final var dependent = createAndConfigureFrom(drs, kubernetesClient); - // check if dependent implements Deleter to record that fact - if (!hasDeleterHolder[0] && dependent instanceof Deleter) { - hasDeleterHolder[0] = true; - } - dependentsHolder.put(drs.getName(), dependent); - }); - dependents = new LinkedHashMap<>(dependentsHolder); - } - - hasDeleterDependents = hasDeleterHolder[0]; isCleaner = reconciler instanceof Cleaner; - } - - @SuppressWarnings("rawtypes") - private DependentResource createAndConfigureFrom(DependentResourceSpec spec, - KubernetesClient client) { - final var dependentResource = - ConfigurationServiceProvider.instance().dependentResourceFactory().createFrom(spec); - - if (dependentResource instanceof KubernetesClientAware) { - ((KubernetesClientAware) dependentResource).setKubernetesClient(client); - } - - if (dependentResource instanceof DependentResourceConfigurator) { - final var configurator = (DependentResourceConfigurator) dependentResource; - spec.getDependentResourceConfiguration().ifPresent(configurator::configureWith); - } - return dependentResource; - } - - private void initContextIfNeeded(P resource, Context

context) { - if (contextInitializer) { - ((ContextInitializer

) reconciler).initContext(resource, context); - } - } - - @Override - public DeleteControl cleanup(P resource, Context

context) { - try { - return metrics - .timeControllerExecution( - new ControllerExecution<>() { - @Override - public String name() { - return "cleanup"; - } - - @Override - public String controllerName() { - return configuration.getName(); - } - - @Override - public String successTypeName(DeleteControl deleteControl) { - return deleteControl.isRemoveFinalizer() ? "delete" : "finalizerNotRemoved"; - } - - @Override - public DeleteControl execute() { - initContextIfNeeded(resource, context); - if (hasDeleterDependents) { - dependents.values().stream() - .filter(d -> d instanceof Deleter) - .map(Deleter.class::cast) - .forEach(deleter -> deleter.delete(resource, context)); - } - if (isCleaner) { - return ((Cleaner

) reconciler).cleanup(resource, context); - } else { - return DeleteControl.defaultDelete(); - } - } - }); - } catch (Exception e) { - throw new OperatorException(e); - } + managedWorkflow = + ManagedWorkflow.workflowFor(kubernetesClient, configuration.getDependentResources()); + eventSourceManager = new EventSourceManager<>(this); } @Override @@ -176,65 +107,132 @@ public String successTypeName(UpdateControl

result) { @Override public UpdateControl

execute() throws Exception { initContextIfNeeded(resource, context); - final var exceptions = new ArrayList(dependents.size()); - dependents.forEach((name, dependent) -> { - try { - final var reconcileResult = dependent.reconcile(resource, context); - context.managedDependentResourceContext().setReconcileResult(name, - reconcileResult); - log.info("Reconciled dependent '{}' -> {}", name, reconcileResult.getOperation()); - } catch (Exception e) { - final var message = e.getMessage(); - exceptions.add(new ManagedDependentResourceException( - name, "Error reconciling dependent '" + name + "': " + message, e)); - } - }); - - if (!exceptions.isEmpty()) { - throw new AggregatedOperatorException("One or more DependentResource(s) failed:\n" + - exceptions.stream() - .map(Controller.this::createExceptionInformation) - .collect(Collectors.joining("\n")), - exceptions); + if (!managedWorkflow.isEmptyWorkflow()) { + var res = managedWorkflow.reconcile(resource, context); + ((DefaultManagedDependentResourceContext) context.managedDependentResourceContext()) + .setWorkflowExecutionResult(res); + res.throwAggregateExceptionIfErrorsPresent(); } - return reconciler.reconcile(resource, context); } }); } - private String createExceptionInformation(Exception e) { - final var exceptionLocation = Optional.ofNullable(e.getCause()) - .map(Throwable::getStackTrace) - .filter(stackTrace -> stackTrace.length > 0) - .map(stackTrace -> { - int i = 0; - while (i < stackTrace.length) { - final var moduleName = stackTrace[i].getModuleName(); - if (!"java.base".equals(moduleName)) { - return " at: " + stackTrace[i].toString(); + @Override + public DeleteControl cleanup(P resource, Context

context) { + try { + return metrics.timeControllerExecution( + new ControllerExecution<>() { + @Override + public String name() { + return "cleanup"; } - i++; - } - return ""; - }); - return "\t\t- " + e.getMessage() + exceptionLocation.orElse(""); + + @Override + public String controllerName() { + return configuration.getName(); + } + + @Override + public String successTypeName(DeleteControl deleteControl) { + return deleteControl.isRemoveFinalizer() ? "delete" : "finalizerNotRemoved"; + } + + @Override + public DeleteControl execute() { + initContextIfNeeded(resource, context); + WorkflowCleanupResult workflowCleanupResult = null; + if (managedWorkflow.isCleaner()) { + workflowCleanupResult = managedWorkflow.cleanup(resource, context); + ((DefaultManagedDependentResourceContext) context.managedDependentResourceContext()) + .setWorkflowCleanupResult(workflowCleanupResult); + workflowCleanupResult.throwAggregateExceptionIfErrorsPresent(); + } + if (isCleaner) { + var cleanupResult = ((Cleaner

) reconciler).cleanup(resource, context); + if (!cleanupResult.isRemoveFinalizer()) { + return cleanupResult; + } else { + // this means there is no reschedule + return workflowCleanupResultToDefaultDelete(workflowCleanupResult); + } + } else { + return workflowCleanupResultToDefaultDelete(workflowCleanupResult); + } + } + }); + } catch (Exception e) { + throw new OperatorException(e); + } + } + + private DeleteControl workflowCleanupResultToDefaultDelete( + WorkflowCleanupResult workflowCleanupResult) { + if (workflowCleanupResult == null) { + return DeleteControl.defaultDelete(); + } else { + return workflowCleanupResult.allPostConditionsMet() ? DeleteControl.defaultDelete() + : DeleteControl.noFinalizerRemoval(); + } + } + + private void initContextIfNeeded(P resource, Context

context) { + if (contextInitializer) { + ((ContextInitializer

) reconciler).initContext(resource, context); + } } public void initAndRegisterEventSources(EventSourceContext

context) { - dependents.entrySet().stream() + configuration.getEventSources().stream().map(es -> initEventSource(es, context)) + .forEach(nes -> eventSourceManager.registerEventSource(nes.name(), nes.original())); + + // add manually defined event sources + if (reconciler instanceof EventSourceInitializer) { + final var provider = (EventSourceInitializer

) this.reconciler; + final var ownSources = provider.prepareEventSources(context); + ownSources.forEach(eventSourceManager::registerEventSource); + } + managedWorkflow + .getDependentResourcesByName().entrySet().stream() .filter(drEntry -> drEntry.getValue() instanceof EventSourceProvider) .forEach(drEntry -> { final var provider = (EventSourceProvider) drEntry.getValue(); final var source = provider.initEventSource(context); eventSourceManager.registerEventSource(drEntry.getKey(), source); }); + } - // add manually defined event sources - if (reconciler instanceof EventSourceInitializer) { - final var provider = (EventSourceInitializer

) this.reconciler; - final var ownSources = provider.prepareEventSources(context); - ownSources.forEach(eventSourceManager::registerEventSource); + private NamedEventSource initEventSource(EventSourceSpec esSpec, EventSourceContext

context) { + try { + if (esSpec instanceof InformerEventSourceSpec) { + InformerEventSourceSpec spec = (InformerEventSourceSpec) esSpec; + var config = + InformerConfiguration.from(spec.getResourceType(), context); + + config.withSecondaryToPrimaryMapper(spec.getSecondaryToPrimaryMapper()); + spec.getPrimaryToSecondaryMapper().ifPresent(config::withPrimaryToSecondaryMapper); + config.withLabelSelector(spec.getLabelSelector()); + config.withNamespaces(spec.getNamespaces()); + config.followNamespaceChanges(spec.isFollowNamespaceChanges()); + var es = new InformerEventSource(config.build(), context); + return new NamedEventSource(es, + finalEventSourceName(esSpec.getName(), es)); + } else { + GenericEventSourceSpec spec = (GenericEventSourceSpec) esSpec; + EventSource es = (EventSource) spec.getEventSourceClass().getConstructor().newInstance(); + return new NamedEventSource(es, finalEventSourceName(esSpec.getName(), es)); + } + } catch (InstantiationException | IllegalAccessException | InvocationTargetException + | NoSuchMethodException e) { + throw new IllegalStateException(e); + } + } + + private String finalEventSourceName(String name, EventSource es) { + if (Constants.GENERATED_EVENT_SOURCE_NAME.equals(name)) { + return EventSourceInitializer.generateNameFor(es); + } else { + return name; } } @@ -372,6 +370,6 @@ public void stop() { } public boolean useFinalizer() { - return isCleaner || hasDeleterDependents; + return isCleaner || managedWorkflow.isCleaner(); } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractDependentResource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractDependentResource.java index ac0fee99ee..5dbdba9358 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractDependentResource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractDependentResource.java @@ -6,7 +6,6 @@ import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.reconciler.Context; import io.javaoperatorsdk.operator.api.reconciler.Ignore; -import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; import io.javaoperatorsdk.operator.processing.event.ResourceID; @@ -18,7 +17,7 @@ public abstract class AbstractDependentResource protected final boolean creatable = this instanceof Creator; protected final boolean updatable = this instanceof Updater; - protected final boolean deletable = this instanceof Deleter; + protected Creator creator; protected Updater updater; diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractEventSourceHolderDependentResource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractEventSourceHolderDependentResource.java index 91d11309f8..2f7747cec6 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractEventSourceHolderDependentResource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/AbstractEventSourceHolderDependentResource.java @@ -1,5 +1,8 @@ package io.javaoperatorsdk.operator.processing.dependent; +import java.util.function.BiPredicate; +import java.util.function.Predicate; + import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext; import io.javaoperatorsdk.operator.api.reconciler.Ignore; @@ -16,13 +19,22 @@ public abstract class AbstractEventSourceHolderDependentResource onAddFilter; + protected BiPredicate onUpdateFilter; + protected BiPredicate onDeleteFilter; + protected Predicate genericFilter; + public EventSource initEventSource(EventSourceContext

context) { // some sub-classes (e.g. KubernetesDependentResource) can have their event source created // before this method is called in the managed case, so only create the event source if it - // hasn't already been set + // hasn't already been set. + // The filters are applied automatically only if event source is created automatically. So if an + // event source + // is shared between dependent resources this does not override the existing filters. if (eventSource == null) { eventSource = createEventSource(context); + applyFilters(); } isCacheFillerEventSource = eventSource instanceof RecentOperationCacheFiller; @@ -35,6 +47,13 @@ protected void setEventSource(T eventSource) { this.eventSource = eventSource; } + protected void applyFilters() { + this.eventSource.setOnAddFilter(onAddFilter); + this.eventSource.setOnUpdateFilter(onUpdateFilter); + this.eventSource.setOnDeleteFilter(onDeleteFilter); + this.eventSource.setGenericFilter(genericFilter); + } + protected T eventSource() { return eventSource; } @@ -55,4 +74,16 @@ protected void onUpdated(ResourceID primaryResourceId, R updated, R actual) { private RecentOperationCacheFiller recentOperationCacheFiller() { return (RecentOperationCacheFiller) eventSource; } + + public void setOnAddFilter(Predicate onAddFilter) { + this.onAddFilter = onAddFilter; + } + + public void setOnUpdateFilter(BiPredicate onUpdateFilter) { + this.onUpdateFilter = onUpdateFilter; + } + + public void setOnDeleteFilter(BiPredicate onDeleteFilter) { + this.onDeleteFilter = onDeleteFilter; + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDKubernetesDependentResource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDKubernetesDependentResource.java index 33e55104cd..9440898d2f 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDKubernetesDependentResource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDKubernetesDependentResource.java @@ -2,20 +2,23 @@ import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.reconciler.Ignore; -import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; import io.javaoperatorsdk.operator.processing.dependent.Creator; import io.javaoperatorsdk.operator.processing.dependent.Updater; /** - * Adaptor Class for standalone mode for resources that manages Create, Read, Update and Delete + * Adaptor class for standalone mode for resources that manage Create, Read and Update operations + * and that should be automatically garbage-collected by Kubernetes when the associated primary + * resource is destroyed. * - * @param Managed resource - * @param

Primary Resource + * @param the type of the managed dependent resource + * @param

the type of the associated primary resource */ @Ignore public abstract class CRUDKubernetesDependentResource extends - KubernetesDependentResource implements Creator, Updater, Deleter

{ + KubernetesDependentResource + implements Creator, Updater, GarbageCollected

{ public CRUDKubernetesDependentResource(Class resourceType) { super(resourceType); diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDNoGCKubernetesDependentResource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDNoGCKubernetesDependentResource.java new file mode 100644 index 0000000000..b9da7023b0 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUDNoGCKubernetesDependentResource.java @@ -0,0 +1,17 @@ +package io.javaoperatorsdk.operator.processing.dependent.kubernetes; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.Ignore; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.processing.dependent.Creator; +import io.javaoperatorsdk.operator.processing.dependent.Updater; + +@Ignore +public class CRUDNoGCKubernetesDependentResource + extends KubernetesDependentResource + implements Creator, Updater, Deleter

{ + + public CRUDNoGCKubernetesDependentResource(Class resourceType) { + super(resourceType); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUKubernetesDependentResource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUKubernetesDependentResource.java deleted file mode 100644 index f25aca8d86..0000000000 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/CRUKubernetesDependentResource.java +++ /dev/null @@ -1,23 +0,0 @@ -package io.javaoperatorsdk.operator.processing.dependent.kubernetes; - -import io.fabric8.kubernetes.api.model.HasMetadata; -import io.javaoperatorsdk.operator.api.reconciler.Ignore; -import io.javaoperatorsdk.operator.processing.dependent.Creator; -import io.javaoperatorsdk.operator.processing.dependent.Updater; - -/** - * Adaptor Class for standalone mode for resources that manages Create, Read and Update - * - * @param Managed resource - * @param

Primary Resource - */ -@Ignore -public abstract class CRUKubernetesDependentResource - extends - KubernetesDependentResource implements Creator, Updater { - - - public CRUKubernetesDependentResource(Class resourceType) { - super(resourceType); - } -} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependent.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependent.java index fc33975532..6ef9aba571 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependent.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependent.java @@ -4,8 +4,15 @@ import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; +import java.util.function.BiPredicate; +import java.util.function.Predicate; +import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.api.reconciler.Constants; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidGenericFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnAddFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnDeleteFilter; +import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnUpdateFilter; import static io.javaoperatorsdk.operator.api.reconciler.Constants.NO_VALUE_SET; @@ -32,4 +39,12 @@ * @return the label selector */ String labelSelector() default NO_VALUE_SET; + + Class> onAddFilter() default VoidOnAddFilter.class; + + Class> onUpdateFilter() default VoidOnUpdateFilter.class; + + Class> onDeleteFilter() default VoidOnDeleteFilter.class; + + Class> genericFilter() default VoidGenericFilter.class; } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResource.java index 5b574c9243..ff5cc308bf 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResource.java @@ -1,5 +1,6 @@ package io.javaoperatorsdk.operator.processing.dependent.kubernetes; +import java.util.HashMap; import java.util.Optional; import java.util.Set; @@ -11,11 +12,13 @@ import io.fabric8.kubernetes.client.KubernetesClient; import io.fabric8.kubernetes.client.dsl.NonNamespaceOperation; import io.fabric8.kubernetes.client.dsl.Resource; +import io.javaoperatorsdk.operator.OperatorException; import io.javaoperatorsdk.operator.api.config.informer.InformerConfiguration; import io.javaoperatorsdk.operator.api.reconciler.Constants; import io.javaoperatorsdk.operator.api.reconciler.Context; import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext; import io.javaoperatorsdk.operator.api.reconciler.Ignore; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.DependentResourceConfigurator; import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.KubernetesClientAware; import io.javaoperatorsdk.operator.processing.dependent.AbstractEventSourceHolderDependentResource; @@ -38,6 +41,7 @@ public abstract class KubernetesDependentResource matcher; private final ResourceUpdatePreProcessor processor; private final Class resourceType; + private final boolean garbageCollected = this instanceof GarbageCollected; private KubernetesDependentResourceConfig kubernetesDependentResourceConfig; @SuppressWarnings("unchecked") @@ -56,7 +60,6 @@ public void configureWith(KubernetesDependentResourceConfig config) { this.kubernetesDependentResourceConfig = config; } - @SuppressWarnings("unchecked") private void configureWith(String labelSelector, Set namespaces, boolean inheritNamespacesOnChange, EventSourceContext

context) { @@ -64,16 +67,27 @@ private void configureWith(String labelSelector, Set namespaces, namespaces = context.getControllerConfiguration().getNamespaces(); } - final SecondaryToPrimaryMapper primaryResourcesRetriever = - (this instanceof SecondaryToPrimaryMapper) ? (SecondaryToPrimaryMapper) this - : Mappers.fromOwnerReference(); var ic = InformerConfiguration.from(resourceType()) .withLabelSelector(labelSelector) - .withSecondaryToPrimaryMapper(primaryResourcesRetriever) + .withSecondaryToPrimaryMapper(getSecondaryToPrimaryMapper()) .withNamespaces(namespaces, inheritNamespacesOnChange) .build(); - configureWith(new InformerEventSource<>(ic, client)); + configureWith(new InformerEventSource<>(ic, context)); + } + + @SuppressWarnings("unchecked") + private SecondaryToPrimaryMapper getSecondaryToPrimaryMapper() { + if (this instanceof SecondaryToPrimaryMapper) { + return (SecondaryToPrimaryMapper) this; + } else if (garbageCollected) { + return Mappers.fromOwnerReference(); + } else if (useDefaultAnnotationsToIdentifyPrimary()) { + return Mappers.fromDefaultAnnotations(); + } else { + throw new OperatorException("Provide a SecondaryToPrimaryMapper to associate " + + "this resource with the primary resource. DependentResource: " + getClass().getName()); + } } /** @@ -123,10 +137,8 @@ public Result match(R actualResource, P primary, Context

context) { } public void delete(P primary, Context

context) { - if (!addOwnerReference()) { - var resource = getSecondaryResource(primary); - resource.ifPresent(r -> client.resource(r).delete()); - } + var resource = getSecondaryResource(primary); + resource.ifPresent(r -> client.resource(r).delete()); } @SuppressWarnings("unchecked") @@ -138,14 +150,23 @@ protected NonNamespaceOperation, Resource> prepa ResourceID.fromResource(desired)); if (addOwnerReference()) { desired.addOwnerReference(primary); + } else if (useDefaultAnnotationsToIdentifyPrimary()) { + addDefaultSecondaryToPrimaryMapperAnnotations(desired, primary); } Class targetClass = (Class) desired.getClass(); return client.resources(targetClass).inNamespace(desired.getMetadata().getNamespace()); } @Override + @SuppressWarnings("unchecked") protected InformerEventSource createEventSource(EventSourceContext

context) { if (kubernetesDependentResourceConfig != null) { + // sets the filters for the dependent resource, which are applied by parent class + onAddFilter = kubernetesDependentResourceConfig.onAddFilter(); + onUpdateFilter = kubernetesDependentResourceConfig.onUpdateFilter(); + onDeleteFilter = kubernetesDependentResourceConfig.onDeleteFilter(); + genericFilter = kubernetesDependentResourceConfig.genericFilter(); + configureWith(kubernetesDependentResourceConfig.labelSelector(), kubernetesDependentResourceConfig.namespaces(), !kubernetesDependentResourceConfig.wereNamespacesConfigured(), context); @@ -159,8 +180,26 @@ protected InformerEventSource createEventSource(EventSourceContext

cont return eventSource(); } + private boolean useDefaultAnnotationsToIdentifyPrimary() { + return !(this instanceof SecondaryToPrimaryMapper) && !garbageCollected && creatable; + } + + private void addDefaultSecondaryToPrimaryMapperAnnotations(R desired, P primary) { + var annotations = desired.getMetadata().getAnnotations(); + if (annotations == null) { + annotations = new HashMap<>(); + desired.getMetadata().setAnnotations(annotations); + } + annotations.put(Mappers.DEFAULT_ANNOTATION_FOR_NAME, primary.getMetadata().getName()); + var primaryNamespaces = primary.getMetadata().getNamespace(); + if (primaryNamespaces != null) { + annotations.put( + Mappers.DEFAULT_ANNOTATION_FOR_NAMESPACE, primary.getMetadata().getNamespace()); + } + } + protected boolean addOwnerReference() { - return creatable && !deletable; + return garbageCollected; } @Override diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResourceConfig.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResourceConfig.java index 8cc8e12164..8fd3bc2311 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResourceConfig.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/KubernetesDependentResourceConfig.java @@ -1,38 +1,55 @@ package io.javaoperatorsdk.operator.processing.dependent.kubernetes; import java.util.Set; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import io.javaoperatorsdk.operator.api.reconciler.Constants; import static io.javaoperatorsdk.operator.api.reconciler.Constants.NO_VALUE_SET; -public class KubernetesDependentResourceConfig { +public class KubernetesDependentResourceConfig { private Set namespaces = Constants.SAME_AS_CONTROLLER_NAMESPACES_SET; private String labelSelector = NO_VALUE_SET; - private boolean namespacesWereConfigured = false; + + private Predicate onAddFilter; + + private BiPredicate onUpdateFilter; + + private BiPredicate onDeleteFilter; + + private Predicate genericFilter; + public KubernetesDependentResourceConfig() {} + @SuppressWarnings("rawtypes") public KubernetesDependentResourceConfig(Set namespaces, String labelSelector, - boolean configuredNS) { + boolean configuredNS, Predicate onAddFilter, + BiPredicate onUpdateFilter, + BiPredicate onDeleteFilter, Predicate genericFilter) { this.namespaces = namespaces; this.labelSelector = labelSelector; - namespacesWereConfigured = configuredNS; + this.namespacesWereConfigured = configuredNS; + this.onAddFilter = onAddFilter; + this.onUpdateFilter = onUpdateFilter; + this.onDeleteFilter = onDeleteFilter; + this.genericFilter = genericFilter; } public KubernetesDependentResourceConfig(Set namespaces, String labelSelector) { - this(namespaces, labelSelector, true); + this(namespaces, labelSelector, true, null, null, null, null); } - public KubernetesDependentResourceConfig setNamespaces(Set namespaces) { + public KubernetesDependentResourceConfig setNamespaces(Set namespaces) { this.namespacesWereConfigured = true; this.namespaces = namespaces; return this; } - public KubernetesDependentResourceConfig setLabelSelector(String labelSelector) { + public KubernetesDependentResourceConfig setLabelSelector(String labelSelector) { this.labelSelector = labelSelector; return this; } @@ -48,4 +65,23 @@ public String labelSelector() { public boolean wereNamespacesConfigured() { return namespacesWereConfigured; } + + @SuppressWarnings("rawtypes") + public Predicate onAddFilter() { + return onAddFilter; + } + + @SuppressWarnings("rawtypes") + public BiPredicate onUpdateFilter() { + return onUpdateFilter; + } + + @SuppressWarnings("rawtypes") + public BiPredicate onDeleteFilter() { + return onDeleteFilter; + } + + public Predicate genericFilter() { + return genericFilter; + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Condition.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Condition.java new file mode 100644 index 0000000000..222433118e --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Condition.java @@ -0,0 +1,10 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +public interface Condition { + + boolean isMet(DependentResource dependentResource, P primary, Context

context); +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DefaultManagedWorkflow.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DefaultManagedWorkflow.java new file mode 100644 index 0000000000..e1162ebcca --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DefaultManagedWorkflow.java @@ -0,0 +1,65 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; + +@SuppressWarnings("rawtypes") +public class DefaultManagedWorkflow

implements ManagedWorkflow

{ + + private final Workflow

workflow; + private final boolean isCleaner; + private final boolean isEmptyWorkflow; + private final Map dependentResourcesByName; + + DefaultManagedWorkflow(KubernetesClient client, + List dependentResourceSpecs, + ManagedWorkflowSupport managedWorkflowSupport) { + managedWorkflowSupport.checkForNameDuplication(dependentResourceSpecs); + dependentResourcesByName = dependentResourceSpecs + .stream().collect(Collectors.toMap(DependentResourceSpec::getName, + spec -> managedWorkflowSupport.createAndConfigureFrom(spec, client))); + + isEmptyWorkflow = dependentResourceSpecs.isEmpty(); + workflow = + managedWorkflowSupport.createWorkflow(dependentResourceSpecs, dependentResourcesByName); + isCleaner = checkIfCleaner(); + } + + public WorkflowReconcileResult reconcile(P primary, Context

context) { + return workflow.reconcile(primary, context); + } + + public WorkflowCleanupResult cleanup(P primary, Context

context) { + return workflow.cleanup(primary, context); + } + + private boolean checkIfCleaner() { + for (var dr : workflow.getDependentResources()) { + if (dr instanceof Deleter && !(dr instanceof GarbageCollected)) { + return true; + } + } + return false; + } + + public boolean isCleaner() { + return isCleaner; + } + + public boolean isEmptyWorkflow() { + return isEmptyWorkflow; + } + + public Map getDependentResourcesByName() { + return dependentResourcesByName; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DependentResourceNode.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DependentResourceNode.java new file mode 100644 index 0000000000..d7af13bc43 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DependentResourceNode.java @@ -0,0 +1,88 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.LinkedList; +import java.util.List; +import java.util.Optional; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +@SuppressWarnings("rawtypes") +public class DependentResourceNode { + + private final DependentResource dependentResource; + private Condition reconcilePrecondition; + private Condition deletePostcondition; + private Condition readyPostcondition; + private final List dependsOn = new LinkedList<>(); + private final List parents = new LinkedList<>(); + + public DependentResourceNode(DependentResource dependentResource) { + this(dependentResource, null, null); + } + + public DependentResourceNode(DependentResource dependentResource, + Condition reconcilePrecondition) { + this(dependentResource, reconcilePrecondition, null); + } + + public DependentResourceNode(DependentResource dependentResource, + Condition reconcilePrecondition, Condition deletePostcondition) { + this.dependentResource = dependentResource; + this.reconcilePrecondition = reconcilePrecondition; + this.deletePostcondition = deletePostcondition; + } + + public DependentResource getDependentResource() { + return dependentResource; + } + + public Optional getReconcilePrecondition() { + return Optional.ofNullable(reconcilePrecondition); + } + + public Optional getDeletePostcondition() { + return Optional.ofNullable(deletePostcondition); + } + + public List getDependsOn() { + return dependsOn; + } + + @SuppressWarnings("unchecked") + public void addDependsOnRelation(DependentResourceNode node) { + node.parents.add(this); + dependsOn.add(node); + } + + @Override + public String toString() { + return "DependentResourceNode{" + + "dependentResource=" + dependentResource + + '}'; + } + + public DependentResourceNode setReconcilePrecondition( + Condition reconcilePrecondition) { + this.reconcilePrecondition = reconcilePrecondition; + return this; + } + + public DependentResourceNode setDeletePostcondition(Condition cleanupCondition) { + this.deletePostcondition = cleanupCondition; + return this; + } + + public Optional> getReadyPostcondition() { + return Optional.ofNullable(readyPostcondition); + } + + public DependentResourceNode setReadyPostcondition(Condition readyPostcondition) { + this.readyPostcondition = readyPostcondition; + return this; + } + + public List getParents() { + return parents; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflow.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflow.java new file mode 100644 index 0000000000..08e2c497b0 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflow.java @@ -0,0 +1,62 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.Collections; +import java.util.List; +import java.util.Map; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +@SuppressWarnings("rawtypes") +public interface ManagedWorkflow

{ + + ManagedWorkflow noOpWorkflow = new ManagedWorkflow() { + @Override + public WorkflowReconcileResult reconcile(HasMetadata primary, Context context) { + throw new IllegalStateException("Shouldn't be called"); + } + + @Override + public WorkflowCleanupResult cleanup(HasMetadata primary, Context context) { + throw new IllegalStateException("Shouldn't be called"); + } + + @Override + public boolean isCleaner() { + return false; + } + + @Override + public boolean isEmptyWorkflow() { + return true; + } + + @Override + public Map getDependentResourcesByName() { + return Collections.emptyMap(); + } + }; + + @SuppressWarnings("unchecked") + static ManagedWorkflow workflowFor(KubernetesClient client, + List dependentResourceSpecs) { + if (dependentResourceSpecs == null || dependentResourceSpecs.isEmpty()) { + return noOpWorkflow; + } + return new DefaultManagedWorkflow(client, dependentResourceSpecs, + ManagedWorkflowSupport.instance()); + } + + WorkflowReconcileResult reconcile(P primary, Context

context); + + WorkflowCleanupResult cleanup(P primary, Context

context); + + boolean isCleaner(); + + boolean isEmptyWorkflow(); + + Map getDependentResourcesByName(); +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowSupport.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowSupport.java new file mode 100644 index 0000000000..aad9475518 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowSupport.java @@ -0,0 +1,184 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.ArrayList; +import java.util.HashSet; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.OperatorException; +import io.javaoperatorsdk.operator.api.config.ConfigurationServiceProvider; +import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.DependentResourceConfigurator; +import io.javaoperatorsdk.operator.api.reconciler.dependent.managed.KubernetesClientAware; +import io.javaoperatorsdk.operator.processing.dependent.workflow.builder.WorkflowBuilder; + +@SuppressWarnings({"rawtypes", "unchecked"}) +class ManagedWorkflowSupport { + + private final static ManagedWorkflowSupport instance = new ManagedWorkflowSupport(); + + static ManagedWorkflowSupport instance() { + return instance; + } + + private ManagedWorkflowSupport() {} + + public void checkForNameDuplication(List dependentResourceSpecs) { + if (dependentResourceSpecs == null) { + return; + } + final var size = dependentResourceSpecs.size(); + if (size == 0) { + return; + } + + final var uniqueNames = new HashSet<>(size); + final var duplicatedNames = new HashSet<>(size); + dependentResourceSpecs.forEach(spec -> { + final var name = spec.getName(); + if (!uniqueNames.add(name)) { + duplicatedNames.add(name); + } + }); + if (!duplicatedNames.isEmpty()) { + throw new OperatorException("Duplicated dependent resource name(s): " + duplicatedNames); + } + } + + @SuppressWarnings("unchecked") + public

Workflow

createWorkflow( + List dependentResourceSpecs, + Map dependentResourceByName) { + var orderedResourceSpecs = orderAndDetectCycles(dependentResourceSpecs); + var workflowBuilder = new WorkflowBuilder

().withThrowExceptionFurther(false); + orderedResourceSpecs.forEach(spec -> { + final var dependentResource = dependentResourceByName.get(spec.getName()); + final var dependsOn = (Set) spec.getDependsOn() + .stream().map(dependentResourceByName::get).collect(Collectors.toSet()); + workflowBuilder + .addDependentResource(dependentResource) + .dependsOn(dependsOn) + .withDeletePostcondition(spec.getDeletePostCondition()) + .withReconcilePrecondition(spec.getReconcileCondition()) + .withReadyPostcondition(spec.getReadyCondition()); + }); + return workflowBuilder.build(); + } + + @SuppressWarnings({"rawtypes", "unchecked"}) + public DependentResource createAndConfigureFrom(DependentResourceSpec spec, + KubernetesClient client) { + final var dependentResource = + ConfigurationServiceProvider.instance().dependentResourceFactory().createFrom(spec); + + if (dependentResource instanceof KubernetesClientAware) { + ((KubernetesClientAware) dependentResource).setKubernetesClient(client); + } + + if (dependentResource instanceof DependentResourceConfigurator) { + final var configurator = (DependentResourceConfigurator) dependentResource; + spec.getDependentResourceConfiguration().ifPresent(configurator::configureWith); + } + return dependentResource; + } + + /** + * + * @param dependentResourceSpecs list of specs + * @return top-bottom ordered resources that can be added safely to workflow + * @throws OperatorException if there is a cycle in the dependencies + * + */ + public List orderAndDetectCycles( + List dependentResourceSpecs) { + + final var drInfosByName = createDRInfos(dependentResourceSpecs); + final var orderedSpecs = new ArrayList(dependentResourceSpecs.size()); + final var alreadyVisited = new HashSet(); + var toVisit = getTopDependentResources(dependentResourceSpecs); + + while (!toVisit.isEmpty()) { + final var toVisitNext = new HashSet(); + toVisit.forEach(dr -> { + final var name = dr.getName(); + var drInfo = drInfosByName.get(name); + if (drInfo != null) { + drInfo.waitingForCompletion.forEach(spec -> { + if (isReadyForVisit(spec, alreadyVisited, name)) { + toVisitNext.add(spec); + } + }); + orderedSpecs.add(dr); + } + alreadyVisited.add(name); + }); + + toVisit = toVisitNext; + } + + if (orderedSpecs.size() != dependentResourceSpecs.size()) { + // could provide improved message where the exact cycles are made visible + throw new OperatorException("Cycle(s) between dependent resources."); + } + return orderedSpecs; + } + + private static class DRInfo { + private final DependentResourceSpec spec; + private final List waitingForCompletion; + + private DRInfo(DependentResourceSpec spec) { + this.spec = spec; + this.waitingForCompletion = new LinkedList<>(); + } + + void add(DependentResourceSpec spec) { + waitingForCompletion.add(spec); + } + + String name() { + return spec.getName(); + } + } + + private boolean isReadyForVisit(DependentResourceSpec dr, Set alreadyVisited, + String alreadyPresentName) { + for (var name : dr.getDependsOn()) { + if (name.equals(alreadyPresentName)) + continue; + if (!alreadyVisited.contains(name)) { + return false; + } + } + return true; + } + + private Set getTopDependentResources( + List dependentResourceSpecs) { + return dependentResourceSpecs.stream().filter(r -> r.getDependsOn().isEmpty()) + .collect(Collectors.toSet()); + } + + private Map createDRInfos(List dependentResourceSpecs) { + // first create mappings + final var infos = dependentResourceSpecs.stream() + .map(DRInfo::new) + .collect(Collectors.toMap(DRInfo::name, Function.identity())); + + // then populate the reverse depends on information + dependentResourceSpecs.forEach(spec -> spec.getDependsOn().forEach(name -> { + final var drInfo = infos.get(name); + drInfo.add(spec); + })); + + return infos; + } + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Workflow.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Workflow.java new file mode 100644 index 0000000000..bdd4e3cd7c --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Workflow.java @@ -0,0 +1,109 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.HashSet; +import java.util.Set; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.stream.Collectors; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.config.ConfigurationServiceProvider; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +/** + * Dependents definition: so if B depends on A, the B is dependent of A. + * + * @param

primary resource + */ +@SuppressWarnings("rawtypes") +public class Workflow

{ + + public static final boolean THROW_EXCEPTION_AUTOMATICALLY_DEFAULT = true; + + private final Set dependentResourceNodes; + private final Set topLevelResources = new HashSet<>(); + private final Set bottomLevelResource = new HashSet<>(); + + private final boolean throwExceptionAutomatically; + // it's "global" executor service shared between multiple reconciliations running parallel + private ExecutorService executorService; + + public Workflow(Set dependentResourceNodes) { + this.executorService = ConfigurationServiceProvider.instance().getExecutorService(); + this.dependentResourceNodes = dependentResourceNodes; + this.throwExceptionAutomatically = THROW_EXCEPTION_AUTOMATICALLY_DEFAULT; + preprocessForReconcile(); + } + + public Workflow(Set dependentResourceNodes, + ExecutorService executorService, boolean throwExceptionAutomatically) { + this.executorService = executorService; + this.dependentResourceNodes = dependentResourceNodes; + this.throwExceptionAutomatically = throwExceptionAutomatically; + preprocessForReconcile(); + } + + public Workflow(Set dependentResourceNodes, int globalParallelism) { + this(dependentResourceNodes, Executors.newFixedThreadPool(globalParallelism), true); + } + + public WorkflowReconcileResult reconcile(P primary, Context

context) { + WorkflowReconcileExecutor

workflowReconcileExecutor = + new WorkflowReconcileExecutor<>(this, primary, context); + var result = workflowReconcileExecutor.reconcile(); + if (throwExceptionAutomatically) { + result.throwAggregateExceptionIfErrorsPresent(); + } + return result; + } + + public WorkflowCleanupResult cleanup(P primary, Context

context) { + WorkflowCleanupExecutor

workflowCleanupExecutor = + new WorkflowCleanupExecutor<>(this, primary, context); + var result = workflowCleanupExecutor.cleanup(); + if (throwExceptionAutomatically) { + result.throwAggregateExceptionIfErrorsPresent(); + } + return result; + } + + // add cycle detection? + private void preprocessForReconcile() { + bottomLevelResource.addAll(dependentResourceNodes); + for (DependentResourceNode node : dependentResourceNodes) { + if (node.getDependsOn().isEmpty()) { + topLevelResources.add(node); + } else { + for (DependentResourceNode dependsOn : node.getDependsOn()) { + bottomLevelResource.remove(dependsOn); + } + } + } + } + + public boolean isThrowExceptionAutomatically() { + return throwExceptionAutomatically; + } + + public void setExecutorService(ExecutorService executorService) { + this.executorService = executorService; + } + + Set getTopLevelDependentResources() { + return topLevelResources; + } + + Set getBottomLevelResource() { + return bottomLevelResource; + } + + ExecutorService getExecutorService() { + return executorService; + } + + public Set getDependentResources() { + return dependentResourceNodes.stream().map(DependentResourceNode::getDependentResource) + .collect(Collectors.toSet()); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutor.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutor.java new file mode 100644 index 0000000000..27ff266ad6 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutor.java @@ -0,0 +1,184 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.Future; +import java.util.stream.Collectors; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; + +@SuppressWarnings("rawtypes") +public class WorkflowCleanupExecutor

{ + + private static final Logger log = LoggerFactory.getLogger(WorkflowCleanupExecutor.class); + + private final Map> actualExecutions = + new ConcurrentHashMap<>(); + private final Map exceptionsDuringExecution = + new ConcurrentHashMap<>(); + private final Set alreadyVisited = ConcurrentHashMap.newKeySet(); + private final Set postDeleteConditionNotMet = + ConcurrentHashMap.newKeySet(); + private final Set deleteCalled = ConcurrentHashMap.newKeySet(); + + private final Workflow

workflow; + private final P primary; + private final Context

context; + + public WorkflowCleanupExecutor(Workflow

workflow, P primary, Context

context) { + this.workflow = workflow; + this.primary = primary; + this.context = context; + } + + public synchronized WorkflowCleanupResult cleanup() { + for (DependentResourceNode dependentResourceNode : workflow + .getBottomLevelResource()) { + handleCleanup(dependentResourceNode); + } + while (true) { + try { + this.wait(); + if (noMoreExecutionsScheduled()) { + break; + } else { + log.warn("Notified but still resources under execution. This should not happen."); + } + } catch (InterruptedException e) { + log.warn("Thread interrupted", e); + Thread.currentThread().interrupt(); + } + } + return createCleanupResult(); + } + + private synchronized boolean noMoreExecutionsScheduled() { + return actualExecutions.isEmpty(); + } + + private synchronized void handleCleanup(DependentResourceNode dependentResourceNode) { + log.debug("Submitting for cleanup: {}", dependentResourceNode); + + if (alreadyVisited(dependentResourceNode) + || isCleaningNow(dependentResourceNode) + || !allDependentsCleaned(dependentResourceNode) + || hasErroredDependent(dependentResourceNode)) { + log.debug("Skipping submit of: {}, ", dependentResourceNode); + return; + } + + Future nodeFuture = + workflow.getExecutorService().submit( + new NodeExecutor(dependentResourceNode)); + actualExecutions.put(dependentResourceNode, nodeFuture); + log.debug("Submitted for cleanup: {}", dependentResourceNode); + } + + private class NodeExecutor implements Runnable { + + private final DependentResourceNode dependentResourceNode; + + private NodeExecutor(DependentResourceNode dependentResourceNode) { + this.dependentResourceNode = dependentResourceNode; + } + + @Override + @SuppressWarnings("unchecked") + public void run() { + try { + var dependentResource = dependentResourceNode.getDependentResource(); + var deletePostCondition = dependentResourceNode.getDeletePostcondition(); + + if (dependentResource instanceof Deleter + && !(dependentResource instanceof GarbageCollected)) { + ((Deleter

) dependentResourceNode.getDependentResource()).delete(primary, context); + deleteCalled.add(dependentResourceNode); + } + alreadyVisited.add(dependentResourceNode); + boolean deletePostConditionMet = + deletePostCondition.map(c -> c.isMet(dependentResource, primary, context)).orElse(true); + if (deletePostConditionMet) { + handleDependentCleaned(dependentResourceNode); + } else { + postDeleteConditionNotMet.add(dependentResourceNode); + } + } catch (RuntimeException e) { + handleExceptionInExecutor(dependentResourceNode, e); + } finally { + handleNodeExecutionFinish(dependentResourceNode); + } + } + } + + private synchronized void handleDependentCleaned( + DependentResourceNode dependentResourceNode) { + var dependOns = dependentResourceNode.getDependsOn(); + if (dependOns != null) { + dependOns.forEach(d -> { + log.debug("Handle cleanup for dependent: {} of parent:{}", d, dependentResourceNode); + handleCleanup(d); + }); + } + } + + private synchronized void handleExceptionInExecutor( + DependentResourceNode dependentResourceNode, + RuntimeException e) { + exceptionsDuringExecution.put(dependentResourceNode, e); + } + + private synchronized void handleNodeExecutionFinish( + DependentResourceNode dependentResourceNode) { + log.debug("Finished execution for: {}", dependentResourceNode); + actualExecutions.remove(dependentResourceNode); + if (actualExecutions.isEmpty()) { + this.notifyAll(); + } + } + + private boolean isCleaningNow(DependentResourceNode dependentResourceNode) { + return actualExecutions.containsKey(dependentResourceNode); + } + + private boolean alreadyVisited( + DependentResourceNode dependentResourceNode) { + return alreadyVisited.contains(dependentResourceNode); + } + + private boolean allDependentsCleaned( + DependentResourceNode dependentResourceNode) { + var parents = dependentResourceNode.getParents(); + return parents.isEmpty() + || parents.stream() + .allMatch(d -> alreadyVisited(d) && !postDeleteConditionNotMet.contains(d)); + } + + private boolean hasErroredDependent( + DependentResourceNode dependentResourceNode) { + var parents = dependentResourceNode.getParents(); + return !parents.isEmpty() + && parents.stream().anyMatch(exceptionsDuringExecution::containsKey); + } + + private WorkflowCleanupResult createCleanupResult() { + var result = new WorkflowCleanupResult(); + result.setErroredDependents(exceptionsDuringExecution + .entrySet().stream() + .collect(Collectors.toMap(e -> e.getKey().getDependentResource(), Map.Entry::getValue))); + + result.setPostConditionNotMetDependents( + postDeleteConditionNotMet.stream().map(DependentResourceNode::getDependentResource) + .collect(Collectors.toList())); + result.setDeleteCalledOnDependents( + deleteCalled.stream().map(DependentResourceNode::getDependentResource) + .collect(Collectors.toList())); + return result; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupResult.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupResult.java new file mode 100644 index 0000000000..f52774b9ac --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupResult.java @@ -0,0 +1,64 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import io.javaoperatorsdk.operator.AggregatedOperatorException; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +@SuppressWarnings("rawtypes") +public class WorkflowCleanupResult { + + private List deleteCalledOnDependents = new ArrayList<>(); + private List postConditionNotMetDependents = new ArrayList<>(); + private Map erroredDependents = new HashMap<>(); + + public List getDeleteCalledOnDependents() { + return deleteCalledOnDependents; + } + + public WorkflowCleanupResult setDeleteCalledOnDependents( + List deletedDependents) { + this.deleteCalledOnDependents = deletedDependents; + return this; + } + + public List getPostConditionNotMetDependents() { + return postConditionNotMetDependents; + } + + public WorkflowCleanupResult setPostConditionNotMetDependents( + List postConditionNotMetDependents) { + this.postConditionNotMetDependents = postConditionNotMetDependents; + return this; + } + + public Map getErroredDependents() { + return erroredDependents; + } + + public WorkflowCleanupResult setErroredDependents( + Map erroredDependents) { + this.erroredDependents = erroredDependents; + return this; + } + + public boolean allPostConditionsMet() { + return postConditionNotMetDependents.isEmpty(); + } + + public boolean erroredDependentsExists() { + return !erroredDependents.isEmpty(); + } + + public void throwAggregateExceptionIfErrorsPresent() { + if (erroredDependentsExists()) { + throw new AggregatedOperatorException("Exception(s) during workflow execution.", + new ArrayList<>(erroredDependents.values())); + } + } + + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutor.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutor.java new file mode 100644 index 0000000000..0412e1c912 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutor.java @@ -0,0 +1,298 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.Future; +import java.util.stream.Collectors; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; +import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; +import io.javaoperatorsdk.operator.processing.event.ResourceID; + +@SuppressWarnings({"rawtypes", "unchecked"}) +public class WorkflowReconcileExecutor

{ + + private static final Logger log = LoggerFactory.getLogger(WorkflowReconcileExecutor.class); + + private final Workflow

workflow; + + /** Covers both deleted and reconciled */ + private final Set alreadyVisited = ConcurrentHashMap.newKeySet(); + private final Set notReady = ConcurrentHashMap.newKeySet(); + private final Map> actualExecutions = + new HashMap<>(); + private final Map exceptionsDuringExecution = + new ConcurrentHashMap<>(); + + private final Set markedForDelete = ConcurrentHashMap.newKeySet(); + private final Set deleteConditionNotMet = ConcurrentHashMap.newKeySet(); + // used to remember reconciled (not deleted or errored) dependents + private final Set reconciled = ConcurrentHashMap.newKeySet(); + private final Map reconcileResults = + new ConcurrentHashMap<>(); + + private final P primary; + private final Context

context; + + public WorkflowReconcileExecutor(Workflow

workflow, P primary, Context

context) { + this.primary = primary; + this.context = context; + this.workflow = workflow; + } + + public synchronized WorkflowReconcileResult reconcile() { + for (DependentResourceNode dependentResourceNode : workflow + .getTopLevelDependentResources()) { + handleReconcile(dependentResourceNode); + } + while (true) { + try { + this.wait(); + if (noMoreExecutionsScheduled()) { + break; + } else { + log.warn("Notified but still resources under execution. This should not happen."); + } + } catch (InterruptedException e) { + log.warn("Thread interrupted", e); + Thread.currentThread().interrupt(); + } + } + return createReconcileResult(); + } + + private synchronized void handleReconcile( + DependentResourceNode dependentResourceNode) { + log.debug("Submitting for reconcile: {}", dependentResourceNode); + + if (alreadyVisited(dependentResourceNode) + || isReconcilingNow(dependentResourceNode) + || !allParentsReconciledAndReady(dependentResourceNode) + || markedForDelete.contains(dependentResourceNode) + || hasErroredParent(dependentResourceNode)) { + log.debug("Skipping submit of: {}, ", dependentResourceNode); + return; + } + + boolean reconcileConditionMet = dependentResourceNode.getReconcilePrecondition().map( + rc -> rc.isMet(dependentResourceNode.getDependentResource(), primary, context)) + .orElse(true); + + if (!reconcileConditionMet) { + handleReconcileConditionNotMet(dependentResourceNode); + } else { + Future nodeFuture = workflow.getExecutorService() + .submit(new NodeReconcileExecutor(dependentResourceNode)); + actualExecutions.put(dependentResourceNode, nodeFuture); + log.debug("Submitted to reconcile: {}", dependentResourceNode); + } + } + + private void handleDelete(DependentResourceNode dependentResourceNode) { + log.debug("Submitting for delete: {}", dependentResourceNode); + + if (alreadyVisited(dependentResourceNode) + || isReconcilingNow(dependentResourceNode) + || !markedForDelete.contains(dependentResourceNode) + || !allDependentsDeletedAlready(dependentResourceNode)) { + log.debug("Skipping submit for delete of: {}, ", dependentResourceNode); + return; + } + + Future nodeFuture = workflow.getExecutorService() + .submit(new NodeDeleteExecutor(dependentResourceNode)); + actualExecutions.put(dependentResourceNode, nodeFuture); + log.debug("Submitted to delete: {}", dependentResourceNode); + } + + private boolean allDependentsDeletedAlready(DependentResourceNode dependentResourceNode) { + var dependents = dependentResourceNode.getParents(); + return dependents.stream().allMatch(d -> alreadyVisited.contains(d) && !notReady.contains(d) + && !exceptionsDuringExecution.containsKey(d) && !deleteConditionNotMet.contains(d)); + } + + + private synchronized void handleExceptionInExecutor(DependentResourceNode dependentResourceNode, + RuntimeException e) { + exceptionsDuringExecution.put(dependentResourceNode, e); + } + + private synchronized void handleNodeExecutionFinish(DependentResourceNode dependentResourceNode) { + log.debug("Finished execution for: {}", dependentResourceNode); + actualExecutions.remove(dependentResourceNode); + if (actualExecutions.isEmpty()) { + this.notifyAll(); + } + } + + // needs to be in one step + private synchronized void setAlreadyReconciledButNotReady( + DependentResourceNode dependentResourceNode) { + log.debug("Setting already reconciled but not ready for: {}", dependentResourceNode); + alreadyVisited.add(dependentResourceNode); + notReady.add(dependentResourceNode); + } + + private class NodeReconcileExecutor implements Runnable { + + private final DependentResourceNode dependentResourceNode; + + private NodeReconcileExecutor(DependentResourceNode dependentResourceNode) { + this.dependentResourceNode = dependentResourceNode; + } + + @Override + @SuppressWarnings("unchecked") + public void run() { + try { + DependentResource dependentResource = dependentResourceNode.getDependentResource(); + if (log.isDebugEnabled()) { + log.debug( + "Reconciling {} for primary: {}", + dependentResourceNode, + ResourceID.fromResource(primary)); + } + ReconcileResult reconcileResult = dependentResource.reconcile(primary, context); + reconcileResults.put(dependentResource, reconcileResult); + reconciled.add(dependentResourceNode); + boolean ready = dependentResourceNode.getReadyPostcondition() + .map(rc -> rc.isMet(dependentResource, primary, context)) + .orElse(true); + + if (ready) { + log.debug("Setting already reconciled for: {}", dependentResourceNode); + alreadyVisited.add(dependentResourceNode); + handleDependentsReconcile(dependentResourceNode); + } else { + setAlreadyReconciledButNotReady(dependentResourceNode); + } + } catch (RuntimeException e) { + handleExceptionInExecutor(dependentResourceNode, e); + } finally { + handleNodeExecutionFinish(dependentResourceNode); + } + } + } + + private class NodeDeleteExecutor implements Runnable { + + private final DependentResourceNode dependentResourceNode; + + private NodeDeleteExecutor(DependentResourceNode dependentResourceNode) { + this.dependentResourceNode = dependentResourceNode; + } + + @Override + @SuppressWarnings("unchecked") + public void run() { + try { + DependentResource dependentResource = dependentResourceNode.getDependentResource(); + var deletePostCondition = dependentResourceNode.getDeletePostcondition(); + + if (dependentResource instanceof Deleter + && !(dependentResource instanceof GarbageCollected)) { + ((Deleter

) dependentResourceNode.getDependentResource()).delete(primary, context); + } + alreadyVisited.add(dependentResourceNode); + boolean deletePostConditionMet = + deletePostCondition.map(c -> c.isMet(dependentResource, primary, context)).orElse(true); + if (deletePostConditionMet) { + handleDependentDeleted(dependentResourceNode); + } else { + deleteConditionNotMet.add(dependentResourceNode); + } + } catch (RuntimeException e) { + handleExceptionInExecutor(dependentResourceNode, e); + } finally { + handleNodeExecutionFinish(dependentResourceNode); + } + } + } + + private synchronized void handleDependentDeleted( + DependentResourceNode dependentResourceNode) { + dependentResourceNode.getDependsOn().forEach(dr -> { + log.debug("Handle deleted for: {} with dependent: {}", dr, dependentResourceNode); + handleDelete(dr); + }); + } + + private boolean isReconcilingNow(DependentResourceNode dependentResourceNode) { + return actualExecutions.containsKey(dependentResourceNode); + } + + private synchronized void handleDependentsReconcile( + DependentResourceNode dependentResourceNode) { + var dependents = dependentResourceNode.getParents(); + dependents.forEach(d -> { + log.debug("Handle reconcile for dependent: {} of parent:{}", d, dependentResourceNode); + handleReconcile(d); + }); + } + + private boolean noMoreExecutionsScheduled() { + return actualExecutions.isEmpty(); + } + + private boolean alreadyVisited( + DependentResourceNode dependentResourceNode) { + return alreadyVisited.contains(dependentResourceNode); + } + + + private void handleReconcileConditionNotMet(DependentResourceNode dependentResourceNode) { + Set bottomNodes = new HashSet<>(); + markDependentsForDelete(dependentResourceNode, bottomNodes); + bottomNodes.forEach(this::handleDelete); + } + + private void markDependentsForDelete(DependentResourceNode dependentResourceNode, + Set bottomNodes) { + markedForDelete.add(dependentResourceNode); + var dependents = dependentResourceNode.getParents(); + if (dependents.isEmpty()) { + bottomNodes.add(dependentResourceNode); + } else { + dependents.forEach(d -> markDependentsForDelete(d, bottomNodes)); + } + } + + private boolean allParentsReconciledAndReady( + DependentResourceNode dependentResourceNode) { + return dependentResourceNode.getDependsOn().isEmpty() + || dependentResourceNode.getDependsOn().stream() + .allMatch(d -> alreadyVisited(d) && !notReady.contains(d)); + } + + private boolean hasErroredParent( + DependentResourceNode dependentResourceNode) { + return !dependentResourceNode.getDependsOn().isEmpty() + && dependentResourceNode.getDependsOn().stream() + .anyMatch(exceptionsDuringExecution::containsKey); + } + + private WorkflowReconcileResult createReconcileResult() { + WorkflowReconcileResult workflowReconcileResult = new WorkflowReconcileResult(); + workflowReconcileResult.setErroredDependents(exceptionsDuringExecution + .entrySet().stream() + .collect(Collectors.toMap(e -> e.getKey().getDependentResource(), Map.Entry::getValue))); + workflowReconcileResult.setNotReadyDependents(notReady.stream() + .map(DependentResourceNode::getDependentResource) + .collect(Collectors.toList())); + workflowReconcileResult.setReconciledDependents(reconciled.stream() + .map(DependentResourceNode::getDependentResource).collect(Collectors.toList())); + workflowReconcileResult.setReconcileResults(reconcileResults); + return workflowReconcileResult; + } + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileResult.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileResult.java new file mode 100644 index 0000000000..3c75873920 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileResult.java @@ -0,0 +1,79 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +import io.javaoperatorsdk.operator.AggregatedOperatorException; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; + +@SuppressWarnings("rawtypes") +public class WorkflowReconcileResult { + + private List reconciledDependents; + private List notReadyDependents; + private Map erroredDependents; + private Map reconcileResults; + + public Map getErroredDependents() { + return erroredDependents; + } + + public WorkflowReconcileResult setErroredDependents( + Map erroredDependents) { + this.erroredDependents = erroredDependents; + return this; + } + + public List getReconciledDependents() { + return reconciledDependents; + } + + public WorkflowReconcileResult setReconciledDependents( + List reconciledDependents) { + this.reconciledDependents = reconciledDependents; + return this; + } + + public List getNotReadyDependents() { + return notReadyDependents; + } + + public WorkflowReconcileResult setNotReadyDependents( + List notReadyDependents) { + this.notReadyDependents = notReadyDependents; + return this; + } + + public Map getReconcileResults() { + return reconcileResults; + } + + public WorkflowReconcileResult setReconcileResults( + Map reconcileResults) { + this.reconcileResults = reconcileResults; + return this; + } + + public void throwAggregateExceptionIfErrorsPresent() { + if (!erroredDependents.isEmpty()) { + throw createFinalException(); + } + } + + private AggregatedOperatorException createFinalException() { + return new AggregatedOperatorException("Exception during workflow.", + new ArrayList<>(erroredDependents.values())); + } + + public boolean allDependentResourcesReady() { + return notReadyDependents.isEmpty(); + } + + public boolean erroredDependentsExists() { + return !erroredDependents.isEmpty(); + } + + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/builder/DependentBuilder.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/builder/DependentBuilder.java new file mode 100644 index 0000000000..70991dc91d --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/builder/DependentBuilder.java @@ -0,0 +1,57 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow.builder; + +import java.util.Arrays; +import java.util.HashSet; +import java.util.Set; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; +import io.javaoperatorsdk.operator.processing.dependent.workflow.DependentResourceNode; + +@SuppressWarnings("rawtypes") +public class DependentBuilder

{ + + private final WorkflowBuilder

workflowBuilder; + private final DependentResourceNode node; + + public DependentBuilder(WorkflowBuilder

workflowBuilder, DependentResourceNode node) { + this.workflowBuilder = workflowBuilder; + this.node = node; + } + + public DependentBuilder

dependsOn(Set dependentResources) { + for (var dependentResource : dependentResources) { + var dependsOn = workflowBuilder.getNodeByDependentResource(dependentResource); + node.addDependsOnRelation(dependsOn); + } + return this; + } + + public DependentBuilder

dependsOn(DependentResource... dependentResources) { + if (dependentResources != null) { + return dependsOn(new HashSet<>(Arrays.asList(dependentResources))); + } + return this; + } + + public DependentBuilder

withReconcilePrecondition(Condition reconcilePrecondition) { + node.setReconcilePrecondition(reconcilePrecondition); + return this; + } + + public DependentBuilder

withReadyPostcondition(Condition readyPostcondition) { + node.setReadyPostcondition(readyPostcondition); + return this; + } + + public DependentBuilder

withDeletePostcondition(Condition deletePostcondition) { + node.setDeletePostcondition(deletePostcondition); + return this; + } + + public WorkflowBuilder

build() { + return workflowBuilder; + } + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/builder/WorkflowBuilder.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/builder/WorkflowBuilder.java new file mode 100644 index 0000000000..14d1b96f76 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/builder/WorkflowBuilder.java @@ -0,0 +1,59 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow.builder; + +import java.util.HashSet; +import java.util.Set; +import java.util.concurrent.ExecutorService; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.config.ConfigurationServiceProvider; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.DependentResourceNode; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Workflow; + +import static io.javaoperatorsdk.operator.processing.dependent.workflow.Workflow.THROW_EXCEPTION_AUTOMATICALLY_DEFAULT; + +@SuppressWarnings({"rawtypes", "unchecked"}) +public class WorkflowBuilder

{ + + private final Set> dependentResourceNodes = new HashSet<>(); + private boolean throwExceptionAutomatically = THROW_EXCEPTION_AUTOMATICALLY_DEFAULT; + + public DependentBuilder

addDependentResource(DependentResource dependentResource) { + DependentResourceNode node = new DependentResourceNode<>(dependentResource); + dependentResourceNodes.add(node); + return new DependentBuilder<>(this, node); + } + + void addDependentResourceNode(DependentResourceNode node) { + dependentResourceNodes.add(node); + } + + DependentResourceNode getNodeByDependentResource(DependentResource dependentResource) { + return dependentResourceNodes.stream() + .filter(dr -> dr.getDependentResource() == dependentResource) + .findFirst() + .orElseThrow(); + } + + public boolean isThrowExceptionAutomatically() { + return throwExceptionAutomatically; + } + + public WorkflowBuilder

withThrowExceptionFurther(boolean throwExceptionFurther) { + this.throwExceptionAutomatically = throwExceptionFurther; + return this; + } + + public Workflow

build() { + return new Workflow(dependentResourceNodes, + ConfigurationServiceProvider.instance().getExecutorService(), throwExceptionAutomatically); + } + + public Workflow

build(int parallelism) { + return new Workflow(dependentResourceNodes, parallelism); + } + + public Workflow

build(ExecutorService executorService) { + return new Workflow(dependentResourceNodes, executorService, throwExceptionAutomatically); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventProcessor.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventProcessor.java index 6597131990..2580a1318e 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventProcessor.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventProcessor.java @@ -1,5 +1,6 @@ package io.javaoperatorsdk.operator.processing.event; +import java.time.Duration; import java.util.HashMap; import java.util.HashSet; import java.util.Map; @@ -7,7 +8,6 @@ import java.util.Set; import java.util.concurrent.ExecutorService; import java.util.concurrent.ScheduledThreadPoolExecutor; -import java.util.concurrent.locks.ReentrantLock; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -21,11 +21,11 @@ import io.javaoperatorsdk.operator.api.reconciler.RetryInfo; import io.javaoperatorsdk.operator.processing.LifecycleAware; import io.javaoperatorsdk.operator.processing.MDCUtils; +import io.javaoperatorsdk.operator.processing.event.rate.RateLimiter; import io.javaoperatorsdk.operator.processing.event.source.Cache; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceAction; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEvent; import io.javaoperatorsdk.operator.processing.event.source.timer.TimerEventSource; -import io.javaoperatorsdk.operator.processing.retry.GenericRetry; import io.javaoperatorsdk.operator.processing.retry.Retry; import io.javaoperatorsdk.operator.processing.retry.RetryExecution; @@ -34,19 +34,20 @@ class EventProcessor implements EventHandler, LifecycleAware { private static final Logger log = LoggerFactory.getLogger(EventProcessor.class); + private static final long MINIMAL_RATE_LIMIT_RESCHEDULE_DURATION = 50; + private volatile boolean running; private final Set underProcessing = new HashSet<>(); private final ReconciliationDispatcher reconciliationDispatcher; private final Retry retry; private final Map retryState = new HashMap<>(); private final ExecutorService executor; private final String controllerName; - private final ReentrantLock lock = new ReentrantLock(); private final Metrics metrics; - private volatile boolean running; private final Cache cache; private final EventSourceManager eventSourceManager; private final EventMarker eventMarker = new EventMarker(); + private final RateLimiter rateLimiter; EventProcessor(EventSourceManager eventSourceManager) { this( @@ -54,9 +55,9 @@ class EventProcessor implements EventHandler, LifecycleAw ExecutorServiceManager.instance().executorService(), eventSourceManager.getController().getConfiguration().getName(), new ReconciliationDispatcher<>(eventSourceManager.getController()), - GenericRetry.fromConfiguration( - eventSourceManager.getController().getConfiguration().getRetryConfiguration()), + eventSourceManager.getController().getConfiguration().getRetry(), ConfigurationServiceProvider.instance().getMetrics(), + eventSourceManager.getController().getConfiguration().getRateLimiter(), eventSourceManager); } @@ -65,6 +66,7 @@ class EventProcessor implements EventHandler, LifecycleAw EventSourceManager eventSourceManager, String relatedControllerName, Retry retry, + RateLimiter rateLimiter, Metrics metrics) { this( eventSourceManager.getControllerResourceEventSource(), @@ -73,6 +75,7 @@ class EventProcessor implements EventHandler, LifecycleAw reconciliationDispatcher, retry, metrics, + rateLimiter, eventSourceManager); } @@ -83,6 +86,7 @@ private EventProcessor( ReconciliationDispatcher reconciliationDispatcher, Retry retry, Metrics metrics, + RateLimiter rateLimiter, EventSourceManager eventSourceManager) { this.running = false; this.executor = @@ -96,11 +100,11 @@ private EventProcessor( this.cache = cache; this.metrics = metrics != null ? metrics : Metrics.NOOP; this.eventSourceManager = eventSourceManager; + this.rateLimiter = rateLimiter; } @Override - public void handleEvent(Event event) { - lock.lock(); + public synchronized void handleEvent(Event event) { try { log.debug("Received event: {}", event); @@ -115,7 +119,6 @@ public void handleEvent(Event event) { } handleMarkedEventForResource(resourceID); } finally { - lock.unlock(); MDCUtils.removeResourceIDInfo(); } } @@ -134,6 +137,11 @@ private void submitReconciliationExecution(ResourceID resourceID) { Optional latest = cache.get(resourceID); latest.ifPresent(MDCUtils::addResourceInfo); if (!controllerUnderExecution && latest.isPresent()) { + var rateLimiterPermission = rateLimiter.acquirePermission(resourceID); + if (rateLimiterPermission.isPresent()) { + handleRateLimitedSubmission(resourceID, rateLimiterPermission.get()); + return; + } setUnderExecutionProcessing(resourceID); final var retryInfo = retryInfo(resourceID); ExecutionScope executionScope = new ExecutionScope<>(latest.get(), retryInfo); @@ -199,61 +207,65 @@ private boolean isResourceMarkedForDeletion(ResourceEvent resourceEvent) { return resourceEvent.getResource().map(HasMetadata::isMarkedForDeletion).orElse(false); } + private void handleRateLimitedSubmission(ResourceID resourceID, Duration minimalDuration) { + var minimalDurationMillis = minimalDuration.toMillis(); + log.debug("Rate limited resource: {}, rescheduled in {} millis", resourceID, + minimalDurationMillis); + retryEventSource().scheduleOnce(resourceID, + Math.max(minimalDurationMillis, MINIMAL_RATE_LIMIT_RESCHEDULE_DURATION)); + } + private RetryInfo retryInfo(ResourceID resourceID) { return retryState.get(resourceID); } - void eventProcessingFinished( + synchronized void eventProcessingFinished( ExecutionScope executionScope, PostExecutionControl postExecutionControl) { - lock.lock(); - try { - if (!running) { - return; - } - ResourceID resourceID = executionScope.getResourceID(); - log.debug( - "Event processing finished. Scope: {}, PostExecutionControl: {}", - executionScope, - postExecutionControl); - unsetUnderExecution(resourceID); - - // If a delete event present at this phase, it was received during reconciliation. - // So we either removed the finalizer during reconciliation or we don't use finalizers. - // Either way we don't want to retry. - if (isRetryConfigured() - && postExecutionControl.exceptionDuringExecution() - && !eventMarker.deleteEventPresent(resourceID)) { - handleRetryOnException( - executionScope, postExecutionControl.getRuntimeException().orElseThrow()); - return; - } - cleanupOnSuccessfulExecution(executionScope); - metrics.finishedReconciliation(resourceID); - if (eventMarker.deleteEventPresent(resourceID)) { - cleanupForDeletedEvent(executionScope.getResourceID()); - } else if (postExecutionControl.isFinalizerRemoved()) { - eventMarker.markProcessedMarkForDeletion(resourceID); + if (!running) { + return; + } + ResourceID resourceID = executionScope.getResourceID(); + log.debug( + "Event processing finished. Scope: {}, PostExecutionControl: {}", + executionScope, + postExecutionControl); + unsetUnderExecution(resourceID); + + // If a delete event present at this phase, it was received during reconciliation. + // So we either removed the finalizer during reconciliation or we don't use finalizers. + // Either way we don't want to retry. + if (isRetryConfigured() + && postExecutionControl.exceptionDuringExecution() + && !eventMarker.deleteEventPresent(resourceID)) { + handleRetryOnException( + executionScope, postExecutionControl.getRuntimeException().orElseThrow()); + return; + } + cleanupOnSuccessfulExecution(executionScope); + metrics.finishedReconciliation(resourceID); + if (eventMarker.deleteEventPresent(resourceID)) { + cleanupForDeletedEvent(executionScope.getResourceID()); + } else if (postExecutionControl.isFinalizerRemoved()) { + eventMarker.markProcessedMarkForDeletion(resourceID); + } else { + postExecutionControl + .getUpdatedCustomResource() + .ifPresent( + r -> { + if (!postExecutionControl.updateIsStatusPatch()) { + eventSourceManager + .getControllerResourceEventSource() + .handleRecentResourceUpdate( + ResourceID.fromResource(r), r, executionScope.getResource()); + } + }); + if (eventMarker.eventPresent(resourceID)) { + submitReconciliationExecution(resourceID); } else { - postExecutionControl - .getUpdatedCustomResource() - .ifPresent( - r -> { - if (!postExecutionControl.updateIsStatusPatch()) { - eventSourceManager - .getControllerResourceEventSource() - .handleRecentResourceUpdate( - ResourceID.fromResource(r), r, executionScope.getResource()); - } - }); - if (eventMarker.eventPresent(resourceID)) { - submitReconciliationExecution(resourceID); - } else { - reScheduleExecutionIfInstructed(postExecutionControl, executionScope.getResource()); - } + reScheduleExecutionIfInstructed(postExecutionControl, executionScope.getResource()); } - } finally { - lock.unlock(); } + } private void reScheduleExecutionIfInstructed( @@ -261,11 +273,10 @@ private void reScheduleExecutionIfInstructed( postExecutionControl .getReScheduleDelay() .ifPresent(delay -> { - if (log.isDebugEnabled()) { - log.debug("ReScheduling event for resource: {} with delay: {}", - ResourceID.fromResource(customResource), delay); - } - retryEventSource().scheduleOnce(customResource, delay); + var resourceID = ResourceID.fromResource(customResource); + log.debug("ReScheduling event for resource: {} with delay: {}", + resourceID, delay); + retryEventSource().scheduleOnce(resourceID, delay); }); } @@ -299,7 +310,7 @@ private void handleRetryOnException( delay, resourceID); metrics.failedReconciliation(resourceID, exception); - retryEventSource().scheduleOnce(executionScope.getResource(), delay); + retryEventSource().scheduleOnce(resourceID, delay); }, () -> log.error("Exhausted retries for {}", executionScope)); } @@ -325,6 +336,7 @@ private RetryExecution getOrInitRetryExecution(ExecutionScope executionScope) private void cleanupForDeletedEvent(ResourceID resourceID) { log.debug("Cleaning up for delete event for: {}", resourceID); eventMarker.cleanup(resourceID); + rateLimiter.clear(resourceID); metrics.cleanupDoneFor(resourceID); } @@ -345,24 +357,14 @@ private boolean isRetryConfigured() { } @Override - public void stop() { - lock.lock(); - try { - this.running = false; - } finally { - lock.unlock(); - } + public synchronized void stop() { + this.running = false; } @Override public void start() throws OperatorException { - lock.lock(); - try { - this.running = true; - handleAlreadyMarkedEvents(); - } finally { - lock.unlock(); - } + this.running = true; + handleAlreadyMarkedEvents(); } private void handleAlreadyMarkedEvents() { diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceManager.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceManager.java index 29df2c6b8a..c0ee600ad7 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceManager.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceManager.java @@ -1,9 +1,6 @@ package io.javaoperatorsdk.operator.processing.event; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Objects; -import java.util.Set; +import java.util.*; import java.util.stream.Collectors; import org.slf4j.Logger; @@ -62,20 +59,22 @@ private void postProcessDefaultEventSources() { */ @Override public synchronized void start() { - for (var eventSource : eventSources) { - try { - logEventSourceEvent(eventSource, "Starting"); - eventSource.start(); - logEventSourceEvent(eventSource, "Started"); - } catch (MissingCRDException e) { - throw e; // leave untouched - } catch (Exception e) { - throw new OperatorException("Couldn't start source " + eventSource.name(), e); - } - } + startEventSource(eventSources.namedControllerResourceEventSource()); + eventSources.additionalNamedEventSources().parallel().forEach(this::startEventSource); eventProcessor.start(); } + @SuppressWarnings("rawtypes") + + + @Override + public synchronized void stop() { + stopEventSource(eventSources.namedControllerResourceEventSource()); + eventSources.additionalNamedEventSources().parallel().forEach(this::stopEventSource); + eventSources.clear(); + eventProcessor.stop(); + } + @SuppressWarnings("rawtypes") private void logEventSourceEvent(NamedEventSource eventSource, String event) { if (log.isDebugEnabled()) { @@ -89,19 +88,26 @@ private void logEventSourceEvent(NamedEventSource eventSource, String event) { } } - @Override - public synchronized void stop() { - for (var eventSource : eventSources) { - try { - logEventSourceEvent(eventSource, "Stopping"); - eventSource.stop(); - logEventSourceEvent(eventSource, "Stopped"); - } catch (Exception e) { - log.warn("Error closing {} -> {}", eventSource.name(), e); - } + private void startEventSource(NamedEventSource eventSource) { + try { + logEventSourceEvent(eventSource, "Starting"); + eventSource.start(); + logEventSourceEvent(eventSource, "Started"); + } catch (MissingCRDException e) { + throw e; // leave untouched + } catch (Exception e) { + throw new OperatorException("Couldn't start source " + eventSource.name(), e); + } + } + + private void stopEventSource(NamedEventSource eventSource) { + try { + logEventSourceEvent(eventSource, "Stopping"); + eventSource.stop(); + logEventSourceEvent(eventSource, "Stopped"); + } catch (Exception e) { + log.warn("Error closing {} -> {}", eventSource.name(), e); } - eventSources.clear(); - eventProcessor.stop(); } public final void registerEventSource(EventSource eventSource) throws OperatorException { @@ -127,7 +133,7 @@ public final synchronized void registerEventSource(String name, EventSource even @SuppressWarnings("unchecked") public void broadcastOnResourceEvent(ResourceAction action, R resource, R oldResource) { - for (var eventSource : eventSources) { + eventSources.additionalNamedEventSources().forEach(eventSource -> { if (eventSource instanceof ResourceEventAware) { var lifecycleAwareES = ((ResourceEventAware) eventSource); switch (action) { @@ -142,16 +148,19 @@ public void broadcastOnResourceEvent(ResourceAction action, R resource, R oldRes break; } } - } + }); } public void changeNamespaces(Set namespaces) { eventProcessor.stop(); + eventSources.controllerResourceEventSource() + .changeNamespaces(namespaces); eventSources - .eventSources() + .additionalEventSources() .filter(NamespaceChangeable.class::isInstance) .map(NamespaceChangeable.class::cast) .filter(NamespaceChangeable::allowsNamespaceChanges) + .parallel() .forEach(ies -> ies.changeNamespaces(namespaces)); eventProcessor.start(); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSources.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSources.java index aebe6caecb..17de7ff947 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSources.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSources.java @@ -14,7 +14,7 @@ import io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource; import io.javaoperatorsdk.operator.processing.event.source.timer.TimerEventSource; -class EventSources implements Iterable { +class EventSources { public static final String CONTROLLER_RESOURCE_EVENT_SOURCE_NAME = "ControllerResourceEventSource"; @@ -40,13 +40,22 @@ TimerEventSource retryEventSource() { return retryAndRescheduleTimerEventSource; } - @Override - public Iterator iterator() { + public Stream additionalNamedEventSources() { return Stream.concat(Stream.of( - new NamedEventSource(controllerResourceEventSource, CONTROLLER_RESOURCE_EVENT_SOURCE_NAME), new NamedEventSource(retryAndRescheduleTimerEventSource, RETRY_RESCHEDULE_TIMER_EVENT_SOURCE_NAME)), - flatMappedSources()).iterator(); + flatMappedSources()); + } + + Stream additionalEventSources() { + return Stream.concat( + Stream.of(retryEventSource()).filter(Objects::nonNull), + sources.values().stream().flatMap(c -> c.values().stream())); + } + + NamedEventSource namedControllerResourceEventSource() { + return new NamedEventSource(controllerResourceEventSource, + CONTROLLER_RESOURCE_EVENT_SOURCE_NAME); } Stream flatMappedSources() { @@ -54,12 +63,6 @@ Stream flatMappedSources() { .map(esEntry -> new NamedEventSource(esEntry.getValue(), esEntry.getKey()))); } - Stream eventSources() { - return Stream.concat( - Stream.of(controllerResourceEventSource(), retryEventSource()).filter(Objects::nonNull), - sources.values().stream().flatMap(c -> c.values().stream())); - } - public void clear() { sources.clear(); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/NamedEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/NamedEventSource.java index 92d1f40096..1ce166a7a1 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/NamedEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/NamedEventSource.java @@ -5,12 +5,15 @@ import io.javaoperatorsdk.operator.OperatorException; import io.javaoperatorsdk.operator.processing.event.source.EventSource; -class NamedEventSource implements EventSource { +/** + * For internal usage only. + */ +public class NamedEventSource implements EventSource { private final EventSource original; private final String name; - NamedEventSource(EventSource original, String name) { + public NamedEventSource(EventSource original, String name) { this.original = original; this.name = name; } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcher.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcher.java index a76fcff8e4..778468f269 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcher.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcher.java @@ -178,7 +178,7 @@ public int getAttemptCount() { @Override public boolean isLastAttempt() { - return controller.getConfiguration().getRetryConfiguration() == null; + return controller.getConfiguration().getRetry() == null; } }); ((DefaultContext) context).setRetryInfo(retryInfo); diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ResourceID.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ResourceID.java index 612cac8e41..7baeab6a4b 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ResourceID.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ResourceID.java @@ -61,9 +61,18 @@ public int hashCode() { @Override public String toString() { + return toString(name, namespace); + } + + public static String toString(HasMetadata resource) { + return toString(resource.getMetadata().getName(), resource.getMetadata().getNamespace()); + } + + private static String toString(String name, String namespace) { return "ResourceID{" + "name='" + name + '\'' + ", namespace='" + namespace + '\'' + '}'; } + } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiter.java new file mode 100644 index 0000000000..caa6075ee7 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiter.java @@ -0,0 +1,61 @@ +package io.javaoperatorsdk.operator.processing.event.rate; + +import java.time.Duration; +import java.time.LocalDateTime; +import java.util.HashMap; +import java.util.Map; +import java.util.Optional; + +import io.javaoperatorsdk.operator.processing.event.ResourceID; + +/** + * A Simple rate limiter that limits the number of permission for a time interval. + */ +public class PeriodRateLimiter implements RateLimiter { + + public static final int DEFAULT_REFRESH_PERIOD_SECONDS = 10; + public static final int DEFAULT_LIMIT_FOR_PERIOD = 3; + public static final Duration DEFAULT_REFRESH_PERIOD = + Duration.ofSeconds(DEFAULT_REFRESH_PERIOD_SECONDS); + + /** To turn off rate limiting set limit fod period to a non-positive number */ + public static final int NO_LIMIT_PERIOD = -1; + + private Duration refreshPeriod; + private int limitForPeriod; + + private Map limitData = new HashMap<>(); + + public PeriodRateLimiter() { + this(DEFAULT_REFRESH_PERIOD, DEFAULT_LIMIT_FOR_PERIOD); + } + + public PeriodRateLimiter(Duration refreshPeriod, int limitForPeriod) { + this.refreshPeriod = refreshPeriod; + this.limitForPeriod = limitForPeriod; + } + + @Override + public Optional acquirePermission(ResourceID resourceID) { + if (limitForPeriod <= 0) { + return Optional.empty(); + } + var actualState = limitData.computeIfAbsent(resourceID, r -> RateState.initialState()); + if (actualState.getCount() < limitForPeriod) { + actualState.increaseCount(); + return Optional.empty(); + } else if (actualState.getLastRefreshTime() + .isBefore(LocalDateTime.now().minus(refreshPeriod))) { + actualState.reset(); + actualState.increaseCount(); + return Optional.empty(); + } else { + return Optional.of(Duration.between(actualState.getLastRefreshTime(), LocalDateTime.now())); + } + } + + @Override + public void clear(ResourceID resourceID) { + limitData.remove(resourceID); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateLimiter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateLimiter.java new file mode 100644 index 0000000000..7281315e98 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateLimiter.java @@ -0,0 +1,22 @@ +package io.javaoperatorsdk.operator.processing.event.rate; + +import java.time.Duration; +import java.util.Optional; + +import io.javaoperatorsdk.operator.processing.event.ResourceID; + +public interface RateLimiter { + + /** + * @param resourceID id of the resource + * @return empty if permission acquired or minimal duration until a permission could be acquired + * again + */ + Optional acquirePermission(ResourceID resourceID); + + /** + * Cleanup state. Called when resource is deleted. + */ + void clear(ResourceID resourceID); + +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateState.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateState.java new file mode 100644 index 0000000000..f1302b077d --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/rate/RateState.java @@ -0,0 +1,35 @@ +package io.javaoperatorsdk.operator.processing.event.rate; + +import java.time.LocalDateTime; + +class RateState { + + private LocalDateTime lastRefreshTime; + private int count; + + public static RateState initialState() { + return new RateState(LocalDateTime.now(), 0); + } + + RateState(LocalDateTime lastRefreshTime, int count) { + this.lastRefreshTime = lastRefreshTime; + this.count = count; + } + + public void increaseCount() { + count = count + 1; + } + + public void reset() { + lastRefreshTime = LocalDateTime.now(); + count = 0; + } + + public LocalDateTime getLastRefreshTime() { + return lastRefreshTime; + } + + public int getCount() { + return count; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/AbstractResourceEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/AbstractResourceEventSource.java index 051a75ff20..d2e8ed8c1d 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/AbstractResourceEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/AbstractResourceEventSource.java @@ -1,5 +1,8 @@ package io.javaoperatorsdk.operator.processing.event.source; +import java.util.function.BiPredicate; +import java.util.function.Predicate; + import io.fabric8.kubernetes.api.model.HasMetadata; public abstract class AbstractResourceEventSource @@ -7,6 +10,11 @@ public abstract class AbstractResourceEventSource implements ResourceEventSource { private final Class resourceClass; + protected Predicate onAddFilter; + protected BiPredicate onUpdateFilter; + protected BiPredicate onDeleteFilter; + protected Predicate genericFilter; + protected AbstractResourceEventSource(Class resourceClass) { this.resourceClass = resourceClass; } @@ -15,4 +23,22 @@ protected AbstractResourceEventSource(Class resourceClass) { public Class resourceType() { return resourceClass; } + + public void setOnAddFilter(Predicate onAddFilter) { + this.onAddFilter = onAddFilter; + } + + public void setOnUpdateFilter( + BiPredicate onUpdateFilter) { + this.onUpdateFilter = onUpdateFilter; + } + + public void setOnDeleteFilter( + BiPredicate onDeleteFilter) { + this.onDeleteFilter = onDeleteFilter; + } + + public void setGenericFilter(Predicate genericFilter) { + this.genericFilter = genericFilter; + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/CachingEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/CachingEventSource.java deleted file mode 100644 index 55bd1ab920..0000000000 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/CachingEventSource.java +++ /dev/null @@ -1,51 +0,0 @@ -package io.javaoperatorsdk.operator.processing.event.source; - -import java.util.Optional; -import java.util.function.Predicate; -import java.util.stream.Stream; - -import io.fabric8.kubernetes.api.model.HasMetadata; -import io.javaoperatorsdk.operator.processing.event.ResourceID; - -/** - * Base class for event sources with caching capabilities. - *

- * - * @param represents the type of resources (usually external non-kubernetes ones) being handled. - */ -public abstract class CachingEventSource - extends AbstractResourceEventSource implements Cache { - - protected UpdatableCache cache; - - protected CachingEventSource(Class resourceClass) { - super(resourceClass); - cache = initCache(); - } - - @Override - public Optional get(ResourceID resourceID) { - return cache.get(resourceID); - } - - @Override - public boolean contains(ResourceID resourceID) { - return cache.contains(resourceID); - } - - @Override - public Stream keys() { - return cache.keys(); - } - - @Override - public Stream list(Predicate predicate) { - return cache.list(predicate); - } - - public Optional getCachedValue(ResourceID resourceID) { - return cache.get(resourceID); - } - - protected abstract UpdatableCache initCache(); -} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ExternalResourceCachingEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ExternalResourceCachingEventSource.java index f8a0cafcd8..275a41775a 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ExternalResourceCachingEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ExternalResourceCachingEventSource.java @@ -3,6 +3,7 @@ import java.util.*; import java.util.concurrent.ConcurrentHashMap; import java.util.stream.Collectors; +import java.util.stream.Stream; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -48,7 +49,7 @@ protected ExternalResourceCachingEventSource(Class resourceClass, protected synchronized void handleDelete(ResourceID primaryID) { var res = cache.remove(primaryID); - if (res != null) { + if (res != null && deleteAcceptedByFilter(res.values())) { getEventHandler().handleEvent(new Event(primaryID)); } } @@ -62,18 +63,19 @@ protected synchronized void handleDelete(ResourceID primaryID, R resource) { handleDelete(primaryID, Set.of(cacheKeyMapper.keyFor(resource))); } - protected synchronized void handleDelete(ResourceID primaryID, Set resourceID) { + protected synchronized void handleDelete(ResourceID primaryID, Set resourceIDs) { if (!isRunning()) { return; } var cachedValues = cache.get(primaryID); - var sizeBeforeRemove = cachedValues.size(); - resourceID.forEach(cachedValues::remove); + List removedResources = cachedValues == null ? Collections.emptyList() + : resourceIDs.stream() + .flatMap(id -> Stream.ofNullable(cachedValues.remove(id))).collect(Collectors.toList()); - if (cachedValues.isEmpty()) { + if (cachedValues != null && cachedValues.isEmpty()) { cache.remove(primaryID); } - if (sizeBeforeRemove > cachedValues.size()) { + if (!removedResources.isEmpty() && deleteAcceptedByFilter(removedResources)) { getEventHandler().handleEvent(new Event(primaryID)); } } @@ -90,7 +92,7 @@ protected synchronized void handleResources(Map> allNewResour var toDelete = cache.keySet().stream().filter(k -> !allNewResources.containsKey(k)) .collect(Collectors.toList()); toDelete.forEach(this::handleDelete); - allNewResources.forEach((primaryID, resources) -> handleResources(primaryID, resources)); + allNewResources.forEach(this::handleResources); } protected synchronized void handleResources(ResourceID primaryID, Set newResources, @@ -101,14 +103,76 @@ protected synchronized void handleResources(ResourceID primaryID, Set newReso return; } var cachedResources = cache.get(primaryID); + if (cachedResources == null) { + cachedResources = Collections.emptyMap(); + } var newResourcesMap = newResources.stream().collect(Collectors.toMap(cacheKeyMapper::keyFor, r -> r)); cache.put(primaryID, newResourcesMap); - if (propagateEvent && !newResourcesMap.equals(cachedResources)) { + if (propagateEvent && !newResourcesMap.equals(cachedResources) + && acceptedByFiler(cachedResources, newResourcesMap)) { getEventHandler().handleEvent(new Event(primaryID)); } } + private boolean acceptedByFiler(Map cachedResourceMap, + Map newResourcesMap) { + + var addedResources = new HashMap<>(newResourcesMap); + addedResources.keySet().removeAll(cachedResourceMap.keySet()); + if (onAddFilter != null || genericFilter != null) { + var anyAddAccepted = + addedResources.values().stream().anyMatch(r -> acceptedByGenericFiler(r) && + onAddFilter.test(r)); + if (anyAddAccepted) { + return true; + } + } else if (!addedResources.isEmpty()) { + return true; + } + + var deletedResource = new HashMap<>(cachedResourceMap); + deletedResource.keySet().removeAll(newResourcesMap.keySet()); + if (onDeleteFilter != null || genericFilter != null) { + var anyDeleteAccepted = + deletedResource.values().stream() + .anyMatch(r -> acceptedByGenericFiler(r) && onDeleteFilter.test(r, false)); + if (anyDeleteAccepted) { + return true; + } + } else if (!deletedResource.isEmpty()) { + return true; + } + + Map possibleUpdatedResources = new HashMap<>(cachedResourceMap); + possibleUpdatedResources.keySet().retainAll(newResourcesMap.keySet()); + possibleUpdatedResources = possibleUpdatedResources.entrySet().stream() + .filter(entry -> !newResourcesMap + .get(entry.getKey()).equals(entry.getValue())) + .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue)); + + if (onUpdateFilter != null || genericFilter != null) { + var anyUpdated = possibleUpdatedResources.entrySet().stream() + .anyMatch( + entry -> { + var newResource = newResourcesMap.get(entry.getKey()); + return acceptedByGenericFiler(newResource) && + onUpdateFilter.test(newResource, entry.getValue()); + }); + if (anyUpdated) { + return true; + } + } else if (!possibleUpdatedResources.isEmpty()) { + return true; + } + + return false; + } + + private boolean acceptedByGenericFiler(R resource) { + return genericFilter == null || genericFilter.test(resource); + } + @Override public synchronized void handleRecentResourceCreate(ResourceID primaryID, R resource) { var actualValues = cache.get(primaryID); @@ -163,4 +227,15 @@ public Optional getSecondaryResource(ResourceID primaryID) { public Map> getCache() { return Collections.unmodifiableMap(cache); } + + protected boolean deleteAcceptedByFilter(Collection res) { + if (onDeleteFilter == null) { + return true; + } + // it is enough if at least one event is accepted + // Cannot be sure about the final state in general, mainly for polled resources. This might be + // fine-tuned for + // other event sources. (For now just by overriding this method.) + return res.stream().anyMatch(r -> onDeleteFilter.test(r, false)); + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventSource.java index d57a662d82..2e19603bd4 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventSource.java @@ -2,6 +2,8 @@ import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import io.fabric8.kubernetes.api.model.HasMetadata; import io.javaoperatorsdk.operator.processing.ResourceOwner; @@ -22,4 +24,12 @@ default Optional getSecondaryResource(P primary) { } Set getSecondaryResources(P primary); + + void setOnAddFilter(Predicate onAddFilter); + + void setOnUpdateFilter(BiPredicate onUpdateFilter); + + void setOnDeleteFilter(BiPredicate onDeleteFilter); + + void setGenericFilter(Predicate genericFilter); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java index 45573542c2..63e77f0d0e 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java @@ -6,5 +6,5 @@ @FunctionalInterface public interface SecondaryToPrimaryMapper { - Set toPrimaryResourceIDs(R dependentResource); + Set toPrimaryResourceIDs(R secondaryResource); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java index 83afcc0be2..12d9d47fd5 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java @@ -2,6 +2,7 @@ import java.util.Optional; import java.util.Set; +import java.util.function.BiPredicate; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -19,6 +20,9 @@ import static io.javaoperatorsdk.operator.processing.KubernetesResourceUtils.getName; import static io.javaoperatorsdk.operator.processing.KubernetesResourceUtils.getUID; import static io.javaoperatorsdk.operator.processing.KubernetesResourceUtils.getVersion; +import static io.javaoperatorsdk.operator.processing.event.source.controller.InternalEventFilters.onUpdateFinalizerNeededAndApplied; +import static io.javaoperatorsdk.operator.processing.event.source.controller.InternalEventFilters.onUpdateGenerationAware; +import static io.javaoperatorsdk.operator.processing.event.source.controller.InternalEventFilters.onUpdateMarkedForDeletion; public class ControllerResourceEventSource extends ManagedInformerEventSource> @@ -27,22 +31,27 @@ public class ControllerResourceEventSource private static final Logger log = LoggerFactory.getLogger(ControllerResourceEventSource.class); private final Controller controller; - private final ResourceEventFilter filter; + private final ResourceEventFilter legacyFilters; @SuppressWarnings("unchecked") public ControllerResourceEventSource(Controller controller) { super(controller.getCRClient(), controller.getConfiguration()); this.controller = controller; - var filters = new ResourceEventFilter[] { - ResourceEventFilters.finalizerNeededAndApplied(), - ResourceEventFilters.markedForDeletion(), - ResourceEventFilters.generationAware(), - }; - if (controller.getConfiguration().getEventFilter() != null) { - filter = controller.getConfiguration().getEventFilter().and(ResourceEventFilters.or(filters)); - } else { - filter = ResourceEventFilters.or(filters); - } + + BiPredicate internalOnUpdateFilter = + (BiPredicate) onUpdateFinalizerNeededAndApplied(controller.useFinalizer(), + controller.getConfiguration().getFinalizerName()) + .or(onUpdateGenerationAware(controller.getConfiguration().isGenerationAware())) + .or(onUpdateMarkedForDeletion()); + + legacyFilters = controller.getConfiguration().getEventFilter(); + + // by default the on add should be processed in all cases regarding internal filters + controller.getConfiguration().onAddFilter().ifPresent(this::setOnAddFilter); + controller.getConfiguration().onUpdateFilter() + .ifPresentOrElse(filter -> setOnUpdateFilter(filter.and(internalOnUpdateFilter)), + () -> setOnUpdateFilter(internalOnUpdateFilter)); + controller.getConfiguration().genericFilter().ifPresent(this::setGenericFilter); } @Override @@ -60,7 +69,9 @@ public void eventReceived(ResourceAction action, T resource, T oldResource) { log.debug("Event received for resource: {}", getName(resource)); MDCUtils.addResourceInfo(resource); controller.getEventSourceManager().broadcastOnResourceEvent(action, resource, oldResource); - if (filter.acceptChange(controller, oldResource, resource)) { + if ((legacyFilters == null || + legacyFilters.acceptChange(controller, oldResource, resource)) + && isAcceptedByFilters(action, resource, oldResource)) { getEventHandler().handleEvent( new ResourceEvent(action, ResourceID.fromResource(resource), resource)); } else { @@ -72,6 +83,20 @@ public void eventReceived(ResourceAction action, T resource, T oldResource) { } } + private boolean isAcceptedByFilters(ResourceAction action, T resource, T oldResource) { + // delete event is filtered for generic filter only. + if (genericFilter != null && !genericFilter.test(resource)) { + return false; + } + switch (action) { + case ADDED: + return onAddFilter == null || onAddFilter.test(resource); + case UPDATED: + return onUpdateFilter.test(resource, oldResource); + } + return true; + } + @Override public void onAdd(T resource) { super.onAdd(resource); @@ -99,4 +124,10 @@ public Optional getSecondaryResource(T primary) { public Set getSecondaryResources(T primary) { throw new IllegalStateException("This method should not be called here. Primary: " + primary); } + + @Override + public void setOnDeleteFilter(BiPredicate onDeleteFilter) { + throw new IllegalStateException( + "onDeleteFilter is not supported for controller resource event source"); + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/InternalEventFilters.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/InternalEventFilters.java new file mode 100644 index 0000000000..97da768979 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/InternalEventFilters.java @@ -0,0 +1,43 @@ +package io.javaoperatorsdk.operator.processing.event.source.controller; + +import java.util.function.BiPredicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; + +public class InternalEventFilters { + + private InternalEventFilters() {} + + static BiPredicate onUpdateMarkedForDeletion() { + return (newResource, oldResource) -> newResource.isMarkedForDeletion(); + } + + static BiPredicate onUpdateGenerationAware( + boolean generationAware) { + + return (newResource, oldResource) -> { + if (!generationAware) { + return true; + } + return oldResource.getMetadata().getGeneration() < newResource + .getMetadata().getGeneration(); + }; + } + + static BiPredicate onUpdateFinalizerNeededAndApplied( + boolean useFinalizer, + String finalizerName) { + return (newResource, oldResource) -> { + if (useFinalizer) { + boolean oldFinalizer = oldResource.hasFinalizer(finalizerName); + boolean newFinalizer = newResource.hasFinalizer(finalizerName); + // accepts event if old did not have finalizer, since it was just added, so the event needs + // to + // be published. + return !newFinalizer || !oldFinalizer; + } else { + return false; + } + }; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilter.java index 17cb27fd71..08a86c92ae 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilter.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilter.java @@ -10,6 +10,7 @@ * * @param

the type of custom resources handled by this filter */ +@Deprecated(forRemoval = true) @FunctionalInterface public interface ResourceEventFilter

{ diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilters.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilters.java index 5a22cafac8..a02a278006 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilters.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ResourceEventFilters.java @@ -5,37 +5,12 @@ /** * Convenience implementations of, and utility methods for, {@link ResourceEventFilter}. */ +@Deprecated(forRemoval = true) public final class ResourceEventFilters { - private static final ResourceEventFilter USE_FINALIZER = - (controller, oldResource, newResource) -> { - if (controller.useFinalizer()) { - final var finalizer = controller.getConfiguration().getFinalizerName(); - boolean oldFinalizer = oldResource == null || oldResource.hasFinalizer(finalizer); - boolean newFinalizer = newResource.hasFinalizer(finalizer); - - return !newFinalizer || !oldFinalizer; - } else { - return false; - } - }; - - private static final ResourceEventFilter GENERATION_AWARE = - (controller, oldResource, newResource) -> { - final var generationAware = controller.getConfiguration().isGenerationAware(); - return oldResource == null || !generationAware || - oldResource.getMetadata().getGeneration() < newResource.getMetadata().getGeneration(); - }; - private static final ResourceEventFilter PASSTHROUGH = (configuration, oldResource, newResource) -> true; - private static final ResourceEventFilter NONE = - (configuration, oldResource, newResource) -> false; - - private static final ResourceEventFilter MARKED_FOR_DELETION = - (configuration, oldResource, newResource) -> newResource.isMarkedForDeletion(); - private ResourceEventFilters() {} /** @@ -49,117 +24,4 @@ public static ResourceEventFilter passthrough() { return (ResourceEventFilter) PASSTHROUGH; } - /** - * Retrieves a filter that reject all events. - * - * @param the type of custom resource the filter should handle - * @return a filter that reject all events - */ - @SuppressWarnings("unchecked") - public static ResourceEventFilter none() { - return (ResourceEventFilter) NONE; - } - - /** - * Retrieves a filter that accepts all events if generation-aware processing is not activated but - * only changes that represent a generation increase otherwise. - * - * @param the type of custom resource the filter should handle - * @return a filter accepting changes based on generation information - */ - @SuppressWarnings("unchecked") - public static ResourceEventFilter generationAware() { - return (ResourceEventFilter) GENERATION_AWARE; - } - - /** - * Retrieves a filter that accepts changes if the target controller uses a finalizer and that - * finalizer hasn't already been applied, rejecting them otherwise. - * - * @param the type of custom resource the filter should handle - * @return a filter accepting changes based on whether the finalizer is needed and has been - * applied - */ - @SuppressWarnings("unchecked") - public static ResourceEventFilter finalizerNeededAndApplied() { - return (ResourceEventFilter) USE_FINALIZER; - } - - /** - * Retrieves a filter that accepts changes if the custom resource is marked for deletion. - * - * @param the type of custom resource the filter should handle - * @return a filter accepting changes based on whether the Custom Resource is marked for deletion. - */ - @SuppressWarnings("unchecked") - public static ResourceEventFilter markedForDeletion() { - return (ResourceEventFilter) MARKED_FOR_DELETION; - } - - /** - * Combines the provided, potentially {@code null} filters with an AND logic, i.e. the resulting - * filter will only accept the change if all filters accept it, reject it otherwise. - *

- * Note that the evaluation of filters is lazy: the result is returned as soon as possible without - * evaluating all filters if possible. - * - * @param items the filters to combine - * @param the type of custom resources the filters are supposed to handle - * @return a combined filter implementing the AND logic combination of the provided filters - */ - @SafeVarargs - public static ResourceEventFilter and( - ResourceEventFilter... items) { - if (items == null) { - return none(); - } - - return (configuration, oldResource, newResource) -> { - for (ResourceEventFilter item : items) { - if (item == null) { - continue; - } - - if (!item.acceptChange(configuration, oldResource, newResource)) { - return false; - } - } - - return true; - }; - } - - /** - * Combines the provided, potentially {@code null} filters with an OR logic, i.e. the resulting - * filter will accept the change if any of the filters accepts it, rejecting it only if all reject - * it. - *

- * Note that the evaluation of filters is lazy: the result is returned as soon as possible without - * evaluating all filters if possible. - * - * @param items the filters to combine - * @param the type of custom resources the filters are supposed to handle - * @return a combined filter implementing the OR logic combination of both provided filters - */ - @SafeVarargs - public static ResourceEventFilter or( - ResourceEventFilter... items) { - if (items == null) { - return none(); - } - - return (configuration, oldResource, newResource) -> { - for (ResourceEventFilter item : items) { - if (item == null) { - continue; - } - - if (item.acceptChange(configuration, oldResource, newResource)) { - return true; - } - } - - return false; - }; - } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidGenericFilter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidGenericFilter.java new file mode 100644 index 0000000000..c02af4e421 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidGenericFilter.java @@ -0,0 +1,12 @@ +package io.javaoperatorsdk.operator.processing.event.source.filter; + +import java.util.function.Predicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; + +public class VoidGenericFilter implements Predicate { + @Override + public boolean test(HasMetadata hasMetadata) { + return true; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnAddFilter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnAddFilter.java new file mode 100644 index 0000000000..11c233e14c --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnAddFilter.java @@ -0,0 +1,12 @@ +package io.javaoperatorsdk.operator.processing.event.source.filter; + +import java.util.function.Predicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; + +public class VoidOnAddFilter implements Predicate { + @Override + public boolean test(HasMetadata hasMetadata) { + return true; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnDeleteFilter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnDeleteFilter.java new file mode 100644 index 0000000000..7dba4566b4 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnDeleteFilter.java @@ -0,0 +1,12 @@ +package io.javaoperatorsdk.operator.processing.event.source.filter; + +import java.util.function.BiPredicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; + +public class VoidOnDeleteFilter implements BiPredicate { + @Override + public boolean test(HasMetadata hasMetadata, Boolean aBoolean) { + return true; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnUpdateFilter.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnUpdateFilter.java new file mode 100644 index 0000000000..333bff2455 --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/filter/VoidOnUpdateFilter.java @@ -0,0 +1,12 @@ +package io.javaoperatorsdk.operator.processing.event.source.filter; + +import java.util.function.BiPredicate; + +import io.fabric8.kubernetes.api.model.HasMetadata; + +public class VoidOnUpdateFilter implements BiPredicate { + @Override + public boolean test(HasMetadata hasMetadata, HasMetadata hasMetadata2) { + return true; + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/AnnotationSecondaryToPrimaryMapper.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/AnnotationSecondaryToPrimaryMapper.java new file mode 100644 index 0000000000..b3ac008bac --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/AnnotationSecondaryToPrimaryMapper.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.processing.event.source.informer; + +import java.util.Set; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.processing.event.ResourceID; +import io.javaoperatorsdk.operator.processing.event.source.SecondaryToPrimaryMapper; + +public class AnnotationSecondaryToPrimaryMapper + implements SecondaryToPrimaryMapper { + @Override + public Set toPrimaryResourceIDs(R secondaryResource) { + return Mappers.fromDefaultAnnotations().toPrimaryResourceIDs(secondaryResource); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java index 4e0d96d5d9..55300f8f28 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java @@ -92,38 +92,52 @@ public InformerEventSource(InformerConfiguration configuration, KubernetesCli } else { primaryToSecondaryIndex = NOOPPrimaryToSecondaryIndex.getInstance(); } + onAddFilter = configuration.onAddFilter().orElse(null); + onUpdateFilter = configuration.onUpdateFilter().orElse(null); + onDeleteFilter = configuration.onDeleteFilter().orElse(null); + genericFilter = configuration.genericFilter().orElse(null); } @Override - public void onAdd(R resource) { + public void onAdd(R newResource) { if (log.isDebugEnabled()) { - log.debug("On add event received for resource id: {}", ResourceID.fromResource(resource)); + log.debug("On add event received for resource id: {} type: {}", + ResourceID.fromResource(newResource), + resourceType().getSimpleName()); } - primaryToSecondaryIndex.onAddOrUpdate(resource); - onAddOrUpdate("add", resource, () -> InformerEventSource.super.onAdd(resource)); + primaryToSecondaryIndex.onAddOrUpdate(newResource); + onAddOrUpdate(Operation.ADD, newResource, null, + () -> InformerEventSource.super.onAdd(newResource)); } @Override public void onUpdate(R oldObject, R newObject) { if (log.isDebugEnabled()) { - log.debug("On update event received for resource id: {}", ResourceID.fromResource(newObject)); + log.debug("On update event received for resource id: {} type: {}", + ResourceID.fromResource(newObject), + resourceType().getSimpleName()); } primaryToSecondaryIndex.onAddOrUpdate(newObject); - onAddOrUpdate("update", newObject, + onAddOrUpdate(Operation.UPDATE, newObject, oldObject, () -> InformerEventSource.super.onUpdate(oldObject, newObject)); } @Override public void onDelete(R resource, boolean b) { if (log.isDebugEnabled()) { - log.debug("On delete event received for resource id: {}", ResourceID.fromResource(resource)); + log.debug("On delete event received for resource id: {} type: {}", + ResourceID.fromResource(resource), + resourceType().getSimpleName()); } primaryToSecondaryIndex.onDelete(resource); super.onDelete(resource, b); - propagateEvent(resource); + if (acceptedByDeleteFilters(resource, b)) { + propagateEvent(resource); + } } - private synchronized void onAddOrUpdate(String operation, R newObject, Runnable superOnOp) { + private synchronized void onAddOrUpdate(Operation operation, R newObject, R oldObject, + Runnable superOnOp) { var resourceID = ResourceID.fromResource(newObject); if (eventRecorder.isRecordingFor(resourceID)) { log.debug("Recording event for: {}", resourceID); @@ -138,11 +152,15 @@ private synchronized void onAddOrUpdate(String operation, R newObject, Runnable superOnOp.run(); } else { superOnOp.run(); - log.debug( - "Propagating event for {}, resource with same version not result of a reconciliation. Resource ID: {}", - operation, - resourceID); - propagateEvent(newObject); + if (eventAcceptedByFilter(operation, newObject, oldObject)) { + log.debug( + "Propagating event for {}, resource with same version not result of a reconciliation. Resource ID: {}", + operation, + resourceID); + propagateEvent(newObject); + } else { + log.debug("Event filtered out for operation: {}, resourceID: {}", operation, resourceID); + } } } @@ -199,20 +217,21 @@ public InformerConfiguration getConfiguration() { @Override public synchronized void handleRecentResourceUpdate(ResourceID resourceID, R resource, R previousVersionOfResource) { - handleRecentCreateOrUpdate(resource, + handleRecentCreateOrUpdate(Operation.UPDATE, resource, previousVersionOfResource, () -> super.handleRecentResourceUpdate(resourceID, resource, previousVersionOfResource)); } @Override public synchronized void handleRecentResourceCreate(ResourceID resourceID, R resource) { - handleRecentCreateOrUpdate(resource, + handleRecentCreateOrUpdate(Operation.ADD, resource, null, () -> super.handleRecentResourceCreate(resourceID, resource)); } - private void handleRecentCreateOrUpdate(R resource, Runnable runnable) { + private void handleRecentCreateOrUpdate(Operation operation, R resource, R oldResource, + Runnable runnable) { primaryToSecondaryIndex.onAddOrUpdate(resource); if (eventRecorder.isRecordingFor(ResourceID.fromResource(resource))) { - handleRecentResourceOperationAndStopEventRecording(resource); + handleRecentResourceOperationAndStopEventRecording(operation, resource, oldResource); } else { runnable.run(); } @@ -233,23 +252,26 @@ private void handleRecentCreateOrUpdate(R resource, Runnable runnable) { * an event needs to be propagated to compensate. * * - * @param resource just created or updated resource + * @param newResource just created or updated resource */ - private void handleRecentResourceOperationAndStopEventRecording(R resource) { - ResourceID resourceID = ResourceID.fromResource(resource); + private void handleRecentResourceOperationAndStopEventRecording(Operation operation, + R newResource, R oldResource) { + ResourceID resourceID = ResourceID.fromResource(newResource); try { if (!eventRecorder.containsEventWithResourceVersion( - resourceID, resource.getMetadata().getResourceVersion())) { + resourceID, newResource.getMetadata().getResourceVersion())) { log.debug( "Did not found event in buffer with target version and resource id: {}", resourceID); - temporaryResourceCache.unconditionallyCacheResource(resource); + temporaryResourceCache.unconditionallyCacheResource(newResource); } else if (eventRecorder.containsEventWithVersionButItsNotLastOne( - resourceID, resource.getMetadata().getResourceVersion())) { + resourceID, newResource.getMetadata().getResourceVersion())) { R lastEvent = eventRecorder.getLastEvent(resourceID); log.debug( "Found events in event buffer but the target event is not last for id: {}. Propagating event.", resourceID); - propagateEvent(lastEvent); + if (eventAcceptedByFilter(operation, newResource, oldResource)) { + propagateEvent(lastEvent); + } } } finally { eventRecorder.stopEventRecording(resourceID); @@ -283,4 +305,25 @@ public synchronized void cleanupOnCreateOrUpdateEventFiltering(ResourceID resour public boolean allowsNamespaceChanges() { return getConfiguration().followControllerNamespaceChanges(); } + + + private boolean eventAcceptedByFilter(Operation operation, R newObject, R oldObject) { + if (genericFilter != null && !genericFilter.test(newObject)) { + return false; + } + if (operation == Operation.ADD) { + return onAddFilter == null || onAddFilter.test(newObject); + } else { + return onUpdateFilter == null || onUpdateFilter.test(newObject, oldObject); + } + } + + private enum Operation { + ADD, UPDATE + } + + private boolean acceptedByDeleteFilters(R resource, boolean b) { + return (onDeleteFilter == null || onDeleteFilter.test(resource, b)) && + (genericFilter == null || genericFilter.test(resource)); + } } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/ManagedInformerEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/ManagedInformerEventSource.java index 98e445032c..6ebd63a7eb 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/ManagedInformerEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/ManagedInformerEventSource.java @@ -20,18 +20,18 @@ import io.javaoperatorsdk.operator.api.config.ResourceConfiguration; import io.javaoperatorsdk.operator.api.reconciler.dependent.RecentOperationCacheFiller; import io.javaoperatorsdk.operator.processing.event.ResourceID; -import io.javaoperatorsdk.operator.processing.event.source.CachingEventSource; -import io.javaoperatorsdk.operator.processing.event.source.IndexerResourceCache; -import io.javaoperatorsdk.operator.processing.event.source.UpdatableCache; +import io.javaoperatorsdk.operator.processing.event.source.*; public abstract class ManagedInformerEventSource> - extends CachingEventSource - implements ResourceEventHandler, IndexerResourceCache, RecentOperationCacheFiller, + extends AbstractResourceEventSource + implements ResourceEventHandler, Cache, IndexerResourceCache, + RecentOperationCacheFiller, NamespaceChangeable { private static final Logger log = LoggerFactory.getLogger(ManagedInformerEventSource.class); protected TemporaryResourceCache temporaryResourceCache = new TemporaryResourceCache<>(this); + protected InformerManager cache = new InformerManager<>(); protected ManagedInformerEventSource( MixedOperation, Resource> client, C configuration) { @@ -54,13 +54,8 @@ public void onDelete(R obj, boolean deletedFinalStateUnknown) { temporaryResourceCache.removeResourceFromCache(obj); } - @Override - protected UpdatableCache initCache() { - return new InformerManager<>(); - } - protected InformerManager manager() { - return (InformerManager) cache; + return cache; } @Override @@ -103,11 +98,10 @@ public Optional get(ResourceID resourceID) { } else { log.debug("Resource not found in temporal cache reading it from informer cache," + " for Resource ID: {}", resourceID); - return super.get(resourceID); + return cache.get(resourceID); } } - @Override public Optional getCachedValue(ResourceID resourceID) { return get(resourceID); } @@ -128,4 +122,15 @@ public void addIndexers(Map>> indexers) { public List byIndex(String indexName, String indexKey) { return manager().byIndex(indexName, indexKey); } + + @Override + public Stream keys() { + return cache.keys(); + } + + @Override + public Stream list(Predicate predicate) { + return cache.list(predicate); + } + } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/Mappers.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/Mappers.java index 1c0150b084..a37f404f94 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/Mappers.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/Mappers.java @@ -9,6 +9,10 @@ public class Mappers { + public static final String DEFAULT_ANNOTATION_FOR_NAME = "io.javaoperatorsdk/primary-name"; + public static final String DEFAULT_ANNOTATION_FOR_NAMESPACE = + "io.javaoperatorsdk/primary-namespace"; + private Mappers() {} public static SecondaryToPrimaryMapper fromAnnotation( @@ -26,6 +30,10 @@ public static SecondaryToPrimaryMapper fromLabel( return fromMetadata(nameKey, null, true); } + public static SecondaryToPrimaryMapper fromDefaultAnnotations() { + return fromMetadata(DEFAULT_ANNOTATION_FOR_NAME, DEFAULT_ANNOTATION_FOR_NAMESPACE, false); + } + public static SecondaryToPrimaryMapper fromLabel( String nameKey, String namespaceKey) { return fromMetadata(nameKey, namespaceKey, true); diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/OwnerReferenceSecondaryToPrimaryMapper.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/OwnerReferenceSecondaryToPrimaryMapper.java new file mode 100644 index 0000000000..49a438d4ff --- /dev/null +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/OwnerReferenceSecondaryToPrimaryMapper.java @@ -0,0 +1,18 @@ +package io.javaoperatorsdk.operator.processing.event.source.informer; + +import java.util.Set; + +import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.processing.event.ResourceID; +import io.javaoperatorsdk.operator.processing.event.source.SecondaryToPrimaryMapper; + +/** + * @param resource type + */ +public class OwnerReferenceSecondaryToPrimaryMapper + implements SecondaryToPrimaryMapper { + @Override + public Set toPrimaryResourceIDs(R secondaryResource) { + return Mappers.fromOwnerReference().toPrimaryResourceIDs(secondaryResource); + } +} diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java index f55e7dd05e..44bab7a624 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java @@ -12,7 +12,6 @@ import io.javaoperatorsdk.operator.processing.event.ResourceID; import io.javaoperatorsdk.operator.processing.event.source.Cache; import io.javaoperatorsdk.operator.processing.event.source.CacheKeyMapper; -import io.javaoperatorsdk.operator.processing.event.source.CachingEventSource; import io.javaoperatorsdk.operator.processing.event.source.ExternalResourceCachingEventSource; import io.javaoperatorsdk.operator.processing.event.source.ResourceEventAware; @@ -22,7 +21,7 @@ * if there is no registerPredicate provided. If register predicate provided it is evaluated on * resource create and/or update to register polling for the event source. *

- * For other behavior see {@link CachingEventSource} + * For other behavior see {@link ExternalResourceCachingEventSource} * * @param the resource polled by the event source * @param

related custom resource diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PollingEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PollingEventSource.java index 09ff2e8b0e..94efbf25aa 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PollingEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PollingEventSource.java @@ -1,9 +1,6 @@ package io.javaoperatorsdk.operator.processing.event.source.polling; -import java.util.Map; -import java.util.Set; -import java.util.Timer; -import java.util.TimerTask; +import java.util.*; import org.slf4j.Logger; import org.slf4j.LoggerFactory; diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSource.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSource.java index 28acfdc3ac..f22400453a 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSource.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSource.java @@ -24,17 +24,20 @@ public class TimerEventSource private final AtomicBoolean running = new AtomicBoolean(); private final Map onceTasks = new ConcurrentHashMap<>(); - public void scheduleOnce(R resource, long delay) { + scheduleOnce(ResourceID.fromResource(resource), delay); + } + + public void scheduleOnce(ResourceID resourceID, long delay) { if (!running.get()) { throw new IllegalStateException("The TimerEventSource is not running"); } - ResourceID resourceUid = ResourceID.fromResource(resource); - if (onceTasks.containsKey(resourceUid)) { - cancelOnceSchedule(resourceUid); + + if (onceTasks.containsKey(resourceID)) { + cancelOnceSchedule(resourceID); } - EventProducerTimeTask task = new EventProducerTimeTask(resourceUid); - onceTasks.put(resourceUid, task); + EventProducerTimeTask task = new EventProducerTimeTask(resourceID); + onceTasks.put(resourceID, task); timer.schedule(task, delay); } diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/GenericRetry.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/GenericRetry.java index bed8e43c9d..7804586fa3 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/GenericRetry.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/GenericRetry.java @@ -3,9 +3,9 @@ import io.javaoperatorsdk.operator.api.config.RetryConfiguration; public class GenericRetry implements Retry { - private int maxAttempts = DEFAULT_MAX_ATTEMPTS; - private long initialInterval = DEFAULT_INITIAL_INTERVAL; - private double intervalMultiplier = DEFAULT_MULTIPLIER; + private int maxAttempts = RetryConfiguration.DEFAULT_MAX_ATTEMPTS; + private long initialInterval = RetryConfiguration.DEFAULT_INITIAL_INTERVAL; + private double intervalMultiplier = RetryConfiguration.DEFAULT_MULTIPLIER; private long maxInterval = -1; public static GenericRetry defaultLimitedExponentialRetry() { diff --git a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/Retry.java b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/Retry.java index b7911d1a74..3500a196f8 100644 --- a/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/Retry.java +++ b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/retry/Retry.java @@ -1,8 +1,7 @@ package io.javaoperatorsdk.operator.processing.retry; -import io.javaoperatorsdk.operator.api.config.RetryConfiguration; - -public interface Retry extends RetryConfiguration { +public interface Retry { RetryExecution initExecution(); + } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/ControllerManagerTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/ControllerManagerTest.java index ca02094d46..02a27b1c96 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/ControllerManagerTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/ControllerManagerTest.java @@ -63,7 +63,7 @@ private static class TestControllerConfiguration public TestControllerConfiguration(Reconciler controller, Class crClass) { super(null, getControllerName(controller), CustomResource.getCRDName(crClass), null, false, null, null, null, null, crClass, - null, null); + null, null, null, null, null, null, null); this.controller = controller; } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/TestUtils.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/TestUtils.java index a7b6b46d17..3b2c5354f5 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/TestUtils.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/TestUtils.java @@ -49,8 +49,9 @@ public static TestCustomResource testCustomResource(ResourceID id) { return resource; } - public static void markForDeletion(HasMetadata customResource) { + public static T markForDeletion(T customResource) { customResource.getMetadata().setDeletionTimestamp("2019-8-10"); + return customResource; } } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProviderTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProviderTest.java index f8c1a0f6aa..fbe5a3542e 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProviderTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceProviderTest.java @@ -55,7 +55,9 @@ void resetShouldResetAllState() { shouldBePossibleToOverrideConfigOnce(); ConfigurationServiceProvider.reset(); - assertEquals(ConfigurationServiceProvider.DEFAULT, ConfigurationServiceProvider.getDefault()); + // makes sure createDefault creates a new instance + assertNotEquals(ConfigurationServiceProvider.getDefault(), + ConfigurationServiceProvider.createDefault()); shouldBePossibleToOverrideConfigOnce(); } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/UtilsTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/UtilsTest.java index ccdc69ee9b..87e60b8aa6 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/UtilsTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/api/config/UtilsTest.java @@ -1,7 +1,5 @@ package io.javaoperatorsdk.operator.api.config; -import java.util.Optional; - import org.junit.jupiter.api.Test; import io.fabric8.kubernetes.api.model.ConfigMap; @@ -11,7 +9,7 @@ import io.javaoperatorsdk.operator.api.reconciler.Reconciler; import io.javaoperatorsdk.operator.api.reconciler.UpdateControl; import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; -import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; +import io.javaoperatorsdk.operator.processing.dependent.EmptyTestDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; @@ -88,7 +86,7 @@ void getsFirstTypeArgumentFromExtendedClass() { @Test void getsFirstTypeArgumentFromInterface() { - assertThat(Utils.getFirstTypeArgumentFromInterface(TestDependentResource.class, + assertThat(Utils.getFirstTypeArgumentFromInterface(EmptyTestDependentResource.class, DependentResource.class)) .isEqualTo(Deployment.class); } @@ -110,27 +108,6 @@ public UpdateControl reconcile(ConfigMap resource, Context } } - public static class TestDependentResource - implements DependentResource { - - @Override - public ReconcileResult reconcile(TestCustomResource primary, - Context context) { - return null; - } - - @Override - public Optional getSecondaryResource(TestCustomResource primaryResource) { - return Optional.empty(); - } - - @Override - public Class resourceType() { - return Deployment.class; - } - - } - public static class TestKubernetesDependentResource extends KubernetesDependentResource { diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/EmptyTestDependentResource.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/EmptyTestDependentResource.java new file mode 100644 index 0000000000..aa75849051 --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/EmptyTestDependentResource.java @@ -0,0 +1,31 @@ +package io.javaoperatorsdk.operator.processing.dependent; + +import java.util.Optional; + +import io.fabric8.kubernetes.api.model.apps.Deployment; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; +import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; + +public class EmptyTestDependentResource + implements DependentResource { + + @Override + public ReconcileResult reconcile(TestCustomResource primary, + Context context) { + return null; + } + + @Override + public Optional getSecondaryResource(TestCustomResource primaryResource) { + return Optional.empty(); + } + + @Override + public Class resourceType() { + return Deployment.class; + } + +} + diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/AbstractWorkflowExecutorTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/AbstractWorkflowExecutorTest.java new file mode 100644 index 0000000000..0ad559b7ca --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/AbstractWorkflowExecutorTest.java @@ -0,0 +1,126 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Optional; + +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; +import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; +import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; + +public class AbstractWorkflowExecutorTest { + public static final String VALUE = "value"; + + protected TestDependent dr1 = new TestDependent("DR_1"); + protected TestDependent dr2 = new TestDependent("DR_2"); + protected TestDeleterDependent drDeleter = new TestDeleterDependent("DR_DELETER"); + protected TestErrorDependent drError = new TestErrorDependent("ERROR_1"); + protected TestErrorDeleterDependent errorDD = new TestErrorDeleterDependent("ERROR_DELETER"); + + protected final Condition noMetDeletePostCondition = + (dependentResource, primary, context) -> false; + protected final Condition metDeletePostCondition = + (dependentResource, primary, context) -> true; + + protected List executionHistory = + Collections.synchronizedList(new ArrayList<>()); + + public class TestDependent implements DependentResource { + + private String name; + + public TestDependent(String name) { + this.name = name; + } + + @Override + public ReconcileResult reconcile(TestCustomResource primary, + Context context) { + executionHistory.add(new ReconcileRecord(this)); + return ReconcileResult.resourceCreated(VALUE); + } + + @Override + public Class resourceType() { + return String.class; + } + + @Override + public Optional getSecondaryResource(TestCustomResource primary) { + return Optional.of(VALUE); + } + + @Override + public String toString() { + return name; + } + } + + public class TestDeleterDependent extends TestDependent implements Deleter { + + public TestDeleterDependent(String name) { + super(name); + } + + @Override + public void delete(TestCustomResource primary, Context context) { + executionHistory.add(new ReconcileRecord(this, true)); + } + } + + public class GarbageCollectedDeleter extends TestDeleterDependent + implements GarbageCollected { + + public GarbageCollectedDeleter(String name) { + super(name); + } + } + + public class TestErrorDeleterDependent extends TestDependent + implements Deleter { + + public TestErrorDeleterDependent(String name) { + super(name); + } + + @Override + public void delete(TestCustomResource primary, Context context) { + executionHistory.add(new ReconcileRecord(this, true)); + throw new IllegalStateException("Test exception"); + } + } + + public class TestErrorDependent implements DependentResource { + private String name; + + public TestErrorDependent(String name) { + this.name = name; + } + + @Override + public ReconcileResult reconcile(TestCustomResource primary, + Context context) { + executionHistory.add(new ReconcileRecord(this)); + throw new IllegalStateException("Test exception"); + } + + @Override + public Class resourceType() { + return String.class; + } + + @Override + public Optional getSecondaryResource(TestCustomResource primary) { + return Optional.of(VALUE); + } + + @Override + public String toString() { + return name; + } + } +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ExecutionAssert.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ExecutionAssert.java new file mode 100644 index 0000000000..b928071bd7 --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ExecutionAssert.java @@ -0,0 +1,91 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.List; +import java.util.Optional; +import java.util.stream.Collectors; + +import org.assertj.core.api.AbstractAssert; + +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +public class ExecutionAssert + extends AbstractAssert> { + + public ExecutionAssert(List reconcileRecords) { + super(reconcileRecords, ExecutionAssert.class); + } + + public static ExecutionAssert assertThat(List actual) { + return new ExecutionAssert(actual); + } + + public ExecutionAssert reconciled(DependentResource... dependentResources) { + for (int i = 0; i < dependentResources.length; i++) { + var rr = getReconcileRecordFor(dependentResources[i]); + if (rr.isEmpty()) { + failWithMessage("Resource not reconciled: %s with index %d", dependentResources, i); + } else { + if (rr.get().isDeleted()) { + failWithMessage("Resource deleted: %s with index %d", dependentResources, i); + } + } + } + return this; + } + + public ExecutionAssert deleted(DependentResource... dependentResources) { + for (int i = 0; i < dependentResources.length; i++) { + var rr = getReconcileRecordFor(dependentResources[i]); + if (rr.isEmpty()) { + failWithMessage("Resource not reconciled: %s with index %d", dependentResources, i); + } else { + if (!rr.get().isDeleted()) { + failWithMessage("Resource not deleted: %s with index %d", dependentResources, i); + } + } + } + return this; + } + + private List getActualDependentResources() { + return actual.stream().map(rr -> rr.getDependentResource()).collect(Collectors.toList()); + } + + private Optional getReconcileRecordFor(DependentResource dependentResource) { + return actual.stream().filter(rr -> rr.getDependentResource() == dependentResource).findFirst(); + } + + public ExecutionAssert reconciledInOrder(DependentResource... dependentResources) { + if (dependentResources.length < 2) { + throw new IllegalArgumentException("At least two dependent resource needs to be specified"); + } + for (int i = 0; i < dependentResources.length - 1; i++) { + checkIfReconciled(i, dependentResources); + checkIfReconciled(i + 1, dependentResources); + if (getActualDependentResources() + .indexOf(dependentResources[i]) > getActualDependentResources() + .indexOf(dependentResources[i + 1])) { + failWithMessage( + "Dependent resource on index %d reconciled after the one on index %d", i, i + 1); + } + } + + return this; + } + + public ExecutionAssert notReconciled(DependentResource... dependentResources) { + for (int i = 0; i < dependentResources.length; i++) { + if (getActualDependentResources().contains(dependentResources[i])) { + failWithMessage("Resource was reconciled: %s with index %d", dependentResources, i); + } + } + return this; + } + + private void checkIfReconciled(int i, DependentResource[] dependentResources) { + if (!getActualDependentResources().contains(dependentResources[i])) { + failWithMessage("Dependent resource: %s, not reconciled on place %d", dependentResources[i], + i); + } + } +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowSupportTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowSupportTest.java new file mode 100644 index 0000000000..85f04f220f --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowSupportTest.java @@ -0,0 +1,159 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.Collections; +import java.util.List; +import java.util.stream.Collectors; + +import org.assertj.core.data.Index; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.OperatorException; +import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +import static io.javaoperatorsdk.operator.processing.dependent.workflow.ManagedWorkflowTestUtils.createDRS; +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; + +class ManagedWorkflowSupportTest { + + public static final String NAME_1 = "name1"; + public static final String NAME_2 = "name2"; + public static final String NAME_3 = "name3"; + public static final String NAME_4 = "name4"; + + ManagedWorkflowSupport managedWorkflowSupport = ManagedWorkflowSupport.instance(); + + @Test + void trivialCasesNameDuplicates() { + managedWorkflowSupport.checkForNameDuplication(null); + managedWorkflowSupport.checkForNameDuplication(Collections.emptyList()); + managedWorkflowSupport.checkForNameDuplication(List.of(createDRS(NAME_1))); + managedWorkflowSupport.checkForNameDuplication(List.of(createDRS(NAME_1), createDRS(NAME_2))); + } + + @Test + void checkFindsDuplicates() { + final var drs2 = createDRS(NAME_2); + final var drs1 = createDRS(NAME_1); + + Assertions.assertThrows(OperatorException.class, () -> managedWorkflowSupport + .checkForNameDuplication(List.of(drs2, drs2))); + + Assertions.assertThrows(OperatorException.class, + () -> managedWorkflowSupport.checkForNameDuplication( + List.of(drs1, drs2, drs2))); + + final var exception = Assertions.assertThrows(OperatorException.class, + () -> managedWorkflowSupport.checkForNameDuplication( + List.of(drs1, drs2, drs2, drs1))); + assertThat(exception.getMessage()).contains(NAME_1, NAME_2); + } + + @Test + void orderingTrivialCases() { + assertThat(managedWorkflowSupport.orderAndDetectCycles(List.of(createDRS(NAME_1)))) + .map(DependentResourceSpec::getName).containsExactly(NAME_1); + + assertThat(managedWorkflowSupport + .orderAndDetectCycles(List.of(createDRS(NAME_2, NAME_1), createDRS(NAME_1)))) + .map(DependentResourceSpec::getName).containsExactly(NAME_1, NAME_2); + } + + @Test + void orderingDiamondShape() { + String NAME_3 = "name3"; + String NAME_4 = "name4"; + + var res = managedWorkflowSupport + .orderAndDetectCycles(List.of(createDRS(NAME_2, NAME_1), createDRS(NAME_1), + createDRS(NAME_3, NAME_1), createDRS(NAME_4, NAME_2, NAME_3))) + .stream().map(DependentResourceSpec::getName).collect(Collectors.toList()); + + assertThat(res) + .containsExactlyInAnyOrder(NAME_1, NAME_2, NAME_3, NAME_4) + .contains(NAME_1, Index.atIndex(0)) + .contains(NAME_4, Index.atIndex(3)); + } + + + @Test + void orderingMultipleRoots() { + final var NAME_3 = "name3"; + final var NAME_4 = "name4"; + final var NAME_5 = "name5"; + final var NAME_6 = "name6"; + + var res = managedWorkflowSupport + .orderAndDetectCycles(List.of( + createDRS(NAME_2, NAME_1, NAME_5), + createDRS(NAME_1), + createDRS(NAME_3, NAME_1), + createDRS(NAME_4, NAME_2, NAME_3), + createDRS(NAME_5, NAME_1, NAME_6), + createDRS(NAME_6))) + .stream().map(DependentResourceSpec::getName).collect(Collectors.toList()); + + assertThat(res) + .containsExactlyInAnyOrder(NAME_1, NAME_5, NAME_6, NAME_2, NAME_3, NAME_4) + .contains(NAME_6, Index.atIndex(0)) + .contains(NAME_1, Index.atIndex(1)) + .contains(NAME_5, Index.atIndex(2)) + .contains(NAME_3, Index.atIndex(3)) + .contains(NAME_2, Index.atIndex(4)) + .contains(NAME_4, Index.atIndex(5)); + } + + @Test + void detectsCyclesTrivialCases() { + String NAME_3 = "name3"; + Assertions.assertThrows(OperatorException.class, () -> managedWorkflowSupport + .orderAndDetectCycles(List.of(createDRS(NAME_2, NAME_1), createDRS(NAME_1, NAME_2)))); + Assertions.assertThrows(OperatorException.class, + () -> managedWorkflowSupport + .orderAndDetectCycles(List.of(createDRS(NAME_2, NAME_1), createDRS(NAME_1, NAME_3), + createDRS(NAME_3, NAME_2)))); + } + + @Test + void detectsCycleOnSubTree() { + + Assertions.assertThrows(OperatorException.class, + () -> managedWorkflowSupport.orderAndDetectCycles(List.of(createDRS(NAME_1), + createDRS(NAME_2, NAME_1), + createDRS(NAME_3, NAME_1, NAME_4), + createDRS(NAME_4, NAME_3)))); + + Assertions.assertThrows(OperatorException.class, + () -> managedWorkflowSupport.orderAndDetectCycles(List.of( + createDRS(NAME_1), + createDRS(NAME_2, NAME_1, NAME_4), + createDRS(NAME_3, NAME_2), + createDRS(NAME_4, NAME_3)))); + } + + @Test + void createsWorkflow() { + var specs = List.of(createDRS(NAME_1), + createDRS(NAME_2, NAME_1), + createDRS(NAME_3, NAME_1), + createDRS(NAME_4, NAME_3, NAME_2)); + + var drByName = specs + .stream().collect(Collectors.toMap(DependentResourceSpec::getName, + spec -> managedWorkflowSupport.createAndConfigureFrom(spec, + mock(KubernetesClient.class)))); + + var workflow = managedWorkflowSupport.createWorkflow(specs, drByName); + + assertThat(workflow.getDependentResources()).containsExactlyInAnyOrder(drByName.values() + .toArray(new DependentResource[0])); + assertThat(workflow.getTopLevelDependentResources()) + .map(DependentResourceNode::getDependentResource).containsExactly(drByName.get(NAME_1)); + assertThat(workflow.getBottomLevelResource()).map(DependentResourceNode::getDependentResource) + .containsExactly(drByName.get(NAME_4)); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowTest.java new file mode 100644 index 0000000000..df556885b1 --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowTest.java @@ -0,0 +1,65 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.List; +import java.util.Set; + +import org.junit.jupiter.api.Test; + +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; + +import static io.javaoperatorsdk.operator.processing.dependent.workflow.ManagedWorkflowTestUtils.createDRS; +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.mockito.Mockito.withSettings; + +@SuppressWarnings({"rawtypes", "unchecked"}) +class ManagedWorkflowTest { + + public static final String NAME = "name"; + + ManagedWorkflowSupport managedWorkflowSupportMock = mock(ManagedWorkflowSupport.class); + KubernetesClient kubernetesClientMock = mock(KubernetesClient.class); + + @Test + void checksIfWorkflowEmpty() { + var mockWorkflow = mock(Workflow.class); + when(managedWorkflowSupportMock.createWorkflow(any(), any())).thenReturn(mockWorkflow); + when(managedWorkflowSupportMock.createAndConfigureFrom(any(), any())) + .thenReturn(mock(DependentResource.class)); + assertThat(managedWorkflow().isEmptyWorkflow()).isTrue(); + + when(mockWorkflow.getDependentResources()).thenReturn(Set.of(mock(DependentResource.class))); + assertThat(managedWorkflow(createDRS(NAME)).isEmptyWorkflow()).isFalse(); + } + + @Test + void isCleanerIfAtLeastOneDRIsDeleterAndNoGC() { + var mockWorkflow = mock(Workflow.class); + when(managedWorkflowSupportMock.createWorkflow(any(), any())).thenReturn(mockWorkflow); + when(managedWorkflowSupportMock.createAndConfigureFrom(any(), any())) + .thenReturn(mock(DependentResource.class)); + when(mockWorkflow.getDependentResources()).thenReturn(Set.of(mock(DependentResource.class))); + + assertThat(managedWorkflow(createDRS(NAME)).isCleaner()).isFalse(); + + when(mockWorkflow.getDependentResources()).thenReturn( + Set.of(mock(DependentResource.class, withSettings().extraInterfaces(Deleter.class)))); + assertThat(managedWorkflow(createDRS(NAME)).isCleaner()).isTrue(); + + when(mockWorkflow.getDependentResources()).thenReturn(Set.of(mock(DependentResource.class, + withSettings().extraInterfaces(Deleter.class, GarbageCollected.class)))); + assertThat(managedWorkflow(createDRS(NAME)).isCleaner()).isFalse(); + } + + ManagedWorkflow managedWorkflow(DependentResourceSpec... specs) { + return new DefaultManagedWorkflow(kubernetesClientMock, List.of(specs), + managedWorkflowSupportMock); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowTestUtils.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowTestUtils.java new file mode 100644 index 0000000000..2332778e2e --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ManagedWorkflowTestUtils.java @@ -0,0 +1,17 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.Set; + +import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec; +import io.javaoperatorsdk.operator.processing.dependent.EmptyTestDependentResource; + +@SuppressWarnings("rawtypes") +public class ManagedWorkflowTestUtils { + + @SuppressWarnings("unchecked") + public static DependentResourceSpec createDRS(String name, String... dependOns) { + return new DependentResourceSpec(EmptyTestDependentResource.class, + null, name, Set.of(dependOns), null, null, null); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ReconcileRecord.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ReconcileRecord.java new file mode 100644 index 0000000000..66e0b82d59 --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/ReconcileRecord.java @@ -0,0 +1,26 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; + +public class ReconcileRecord { + + private DependentResource dependentResource; + private final boolean deleted; + + public ReconcileRecord(DependentResource dependentResource) { + this(dependentResource, false); + } + + public ReconcileRecord(DependentResource dependentResource, boolean deleted) { + this.dependentResource = dependentResource; + this.deleted = deleted; + } + + public DependentResource getDependentResource() { + return dependentResource; + } + + public boolean isDeleted() { + return deleted; + } +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutorTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutorTest.java new file mode 100644 index 0000000000..6540f00157 --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutorTest.java @@ -0,0 +1,133 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import org.assertj.core.api.Assertions; +import org.junit.jupiter.api.Test; + +import io.javaoperatorsdk.operator.AggregatedOperatorException; +import io.javaoperatorsdk.operator.processing.dependent.workflow.builder.WorkflowBuilder; +import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; + +import static io.javaoperatorsdk.operator.processing.dependent.workflow.ExecutionAssert.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class WorkflowCleanupExecutorTest extends AbstractWorkflowExecutorTest { + + protected TestDeleterDependent dd1 = new TestDeleterDependent("DR_DELETER_1"); + protected TestDeleterDependent dd2 = new TestDeleterDependent("DR_DELETER_2"); + protected TestDeleterDependent dd3 = new TestDeleterDependent("DR_DELETER_3"); + + @Test + void cleanUpDiamondWorkflow() { + var workflow = new WorkflowBuilder() + .addDependentResource(dd1).build() + .addDependentResource(dr1).dependsOn(dd1).build() + .addDependentResource(dd2).dependsOn(dd1).build() + .addDependentResource(dd3).dependsOn(dr1, dd2).build() + .build(); + + var res = workflow.cleanup(new TestCustomResource(), null); + + assertThat(executionHistory).reconciledInOrder(dd3, dd2, dd1).notReconciled(dr1); + + Assertions.assertThat(res.getDeleteCalledOnDependents()).containsExactlyInAnyOrder(dd1, dd2, + dd3); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getPostConditionNotMetDependents()).isEmpty(); + } + + @Test + void dontDeleteIfDependentErrored() { + var workflow = new WorkflowBuilder() + .addDependentResource(dd1).build() + .addDependentResource(dd2).dependsOn(dd1).build() + .addDependentResource(dd3).dependsOn(dd2).build() + .addDependentResource(errorDD).dependsOn(dd2).build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.cleanup(new TestCustomResource(), null); + assertThrows(AggregatedOperatorException.class, + res::throwAggregateExceptionIfErrorsPresent); + + assertThat(executionHistory).deleted(dd3, errorDD).notReconciled(dd1, dd2); + + Assertions.assertThat(res.getDeleteCalledOnDependents()).containsExactlyInAnyOrder(dd3); + Assertions.assertThat(res.getErroredDependents()).containsOnlyKeys(errorDD); + Assertions.assertThat(res.getPostConditionNotMetDependents()).isEmpty(); + } + + + @Test + void cleanupConditionTrivialCase() { + var workflow = new WorkflowBuilder() + .addDependentResource(dd1).build() + .addDependentResource(dd2).dependsOn(dd1).withDeletePostcondition(noMetDeletePostCondition) + .build() + .build(); + + var res = workflow.cleanup(new TestCustomResource(), null); + + assertThat(executionHistory).deleted(dd2).notReconciled(dd1); + Assertions.assertThat(res.getDeleteCalledOnDependents()).containsExactlyInAnyOrder(dd2); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getPostConditionNotMetDependents()).containsExactlyInAnyOrder(dd2); + } + + @Test + void cleanupConditionMet() { + var workflow = new WorkflowBuilder() + .addDependentResource(dd1).build() + .addDependentResource(dd2).dependsOn(dd1).withDeletePostcondition(metDeletePostCondition) + .build() + .build(); + + var res = workflow.cleanup(new TestCustomResource(), null); + + assertThat(executionHistory).deleted(dd2, dd1); + + Assertions.assertThat(res.getDeleteCalledOnDependents()).containsExactlyInAnyOrder(dd1, dd2); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getPostConditionNotMetDependents()).isEmpty(); + } + + @Test + void cleanupConditionDiamondWorkflow() { + TestDeleterDependent dd4 = new TestDeleterDependent("DR_DELETER_4"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dd1).build() + .addDependentResource(dd2).dependsOn(dd1).build() + .addDependentResource(dd3).dependsOn(dd1).withDeletePostcondition(noMetDeletePostCondition) + .build() + .addDependentResource(dd4).dependsOn(dd2, dd3).build() + .build(); + + var res = workflow.cleanup(new TestCustomResource(), null); + + assertThat(executionHistory) + .reconciledInOrder(dd4, dd2) + .reconciledInOrder(dd4, dd3) + .notReconciled(dr1); + + Assertions.assertThat(res.getDeleteCalledOnDependents()).containsExactlyInAnyOrder(dd4, dd3, + dd2); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getPostConditionNotMetDependents()).containsExactlyInAnyOrder(dd3); + } + + @Test + void dontDeleteIfGarbageCollected() { + GarbageCollectedDeleter gcDel = new GarbageCollectedDeleter("GC_DELETER"); + var workflow = new WorkflowBuilder() + .addDependentResource(gcDel).build() + .build(); + + var res = workflow.cleanup(new TestCustomResource(), null); + + assertThat(executionHistory) + .notReconciled(gcDel); + + Assertions.assertThat(res.getDeleteCalledOnDependents()).isEmpty(); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutorTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutorTest.java new file mode 100644 index 0000000000..2c81bcb50b --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutorTest.java @@ -0,0 +1,467 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import org.assertj.core.api.Assertions; +import org.junit.jupiter.api.Test; + +import io.javaoperatorsdk.operator.AggregatedOperatorException; +import io.javaoperatorsdk.operator.processing.dependent.workflow.builder.WorkflowBuilder; +import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; + +import static io.javaoperatorsdk.operator.processing.dependent.workflow.ExecutionAssert.*; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class WorkflowReconcileExecutorTest extends AbstractWorkflowExecutorTest { + + private Condition met_reconcile_condition = + (dependentResource, primary, context) -> true; + private Condition not_met_reconcile_condition = + (dependentResource, primary, context) -> false; + + private Condition metReadyCondition = + (dependentResource, primary, context) -> true; + private Condition notMetReadyCondition = + (dependentResource, primary, context) -> false; + + private Condition notMetReadyConditionWithStatusUpdate = + (dependentResource, primary, context) -> false; + + @Test + void reconcileTopLevelResources() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory).reconciled(dr1, dr2); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2); + } + + @Test + void reconciliationWithSimpleDependsOn() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + assertThat(executionHistory).reconciledInOrder(dr1, dr2); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void reconciliationWithTwoTheDependsOns() { + TestDependent dr3 = new TestDependent("DR_3"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .addDependentResource(dr3).dependsOn(dr1).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + assertThat(executionHistory) + .reconciledInOrder(dr1, dr2).reconciledInOrder(dr1, dr3); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2, dr3); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void diamondShareWorkflowReconcile() { + TestDependent dr3 = new TestDependent("DR_3"); + TestDependent dr4 = new TestDependent("DR_4"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .addDependentResource(dr3).dependsOn(dr1).build() + .addDependentResource(dr4).dependsOn(dr3).dependsOn(dr2).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + assertThat(executionHistory) + .reconciledInOrder(dr1, dr2, dr4) + .reconciledInOrder(dr1, dr3, dr4); + + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2, dr3, + dr4); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void exceptionHandlingSimpleCases() { + var workflow = new WorkflowBuilder() + .addDependentResource(drError).build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThrows(AggregatedOperatorException.class, + res::throwAggregateExceptionIfErrorsPresent); + + assertThat(executionHistory).reconciled(drError); + Assertions.assertThat(res.getErroredDependents()).containsOnlyKeys(drError); + Assertions.assertThat(res.getReconciledDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void dependentsOnErroredResourceNotReconciled() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(drError).dependsOn(dr1).build() + .addDependentResource(dr2).dependsOn(drError).build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + assertThrows(AggregatedOperatorException.class, + res::throwAggregateExceptionIfErrorsPresent); + + assertThat(executionHistory).reconciled(dr1, drError).notReconciled(dr2); + Assertions.assertThat(res.getErroredDependents()).containsOnlyKeys(drError); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void oneBranchErrorsOtherCompletes() { + TestDependent dr3 = new TestDependent("DR_3"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(drError).dependsOn(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .addDependentResource(dr3).dependsOn(dr2).build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + assertThrows(AggregatedOperatorException.class, + res::throwAggregateExceptionIfErrorsPresent); + + assertThat(executionHistory).reconciledInOrder(dr1, dr2, dr3).reconciledInOrder(dr1, drError); + Assertions.assertThat(res.getErroredDependents()).containsOnlyKeys(drError); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2, dr3); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void onlyOneDependsOnErroredResourceNotReconciled() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(drError).build() + .addDependentResource(dr2).dependsOn(drError, dr1).build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + assertThrows(AggregatedOperatorException.class, + res::throwAggregateExceptionIfErrorsPresent); + + assertThat(executionHistory).notReconciled(dr2); + Assertions.assertThat(res.getErroredDependents()).containsKey(drError); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void simpleReconcileCondition() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).withReconcilePrecondition(not_met_reconcile_condition).build() + .addDependentResource(dr2).withReconcilePrecondition(met_reconcile_condition).build() + .addDependentResource(drDeleter).withReconcilePrecondition(not_met_reconcile_condition) + .build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory).notReconciled(dr1).reconciled(dr2).deleted(drDeleter); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr2); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + + @Test + void triangleOnceConditionNotMet() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .addDependentResource(drDeleter).withReconcilePrecondition(not_met_reconcile_condition) + .dependsOn(dr1) + .build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory).reconciledInOrder(dr1, dr2).deleted(drDeleter); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void reconcileConditionTransitiveDelete() { + TestDeleterDependent drDeleter2 = new TestDeleterDependent("DR_DELETER_2"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1) + .withReconcilePrecondition(not_met_reconcile_condition) + .build() + .addDependentResource(drDeleter).dependsOn(dr2) + .withReconcilePrecondition(met_reconcile_condition) + .build() + .addDependentResource(drDeleter2).dependsOn(drDeleter) + .withReconcilePrecondition(met_reconcile_condition).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + assertThat(executionHistory).notReconciled(dr2); + assertThat(executionHistory).reconciledInOrder(dr1, drDeleter2, drDeleter); + assertThat(executionHistory).deleted(drDeleter2, drDeleter); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void reconcileConditionAlsoErrorDependsOn() { + TestDeleterDependent drDeleter2 = new TestDeleterDependent("DR_DELETER_2"); + + var workflow = new WorkflowBuilder() + .addDependentResource(drError).build() + .addDependentResource(drDeleter).withReconcilePrecondition(not_met_reconcile_condition) + .build() + .addDependentResource(drDeleter2).dependsOn(drError, drDeleter) + .withReconcilePrecondition(met_reconcile_condition) + .build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + assertThrows(AggregatedOperatorException.class, + res::throwAggregateExceptionIfErrorsPresent); + + assertThat(executionHistory) + .deleted(drDeleter2, drDeleter) + .reconciled(drError); + + Assertions.assertThat(res.getErroredDependents()).containsOnlyKeys(drError); + Assertions.assertThat(res.getReconciledDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void oneDependsOnConditionNotMet() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).withReconcilePrecondition(not_met_reconcile_condition).build() + .addDependentResource(drDeleter).dependsOn(dr1, dr2).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + + assertThat(executionHistory).deleted(drDeleter).notReconciled(dr2).reconciled(dr1); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void deletedIfReconcileConditionNotMet() { + TestDeleterDependent drDeleter2 = new TestDeleterDependent("DR_DELETER_2"); + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(drDeleter).dependsOn(dr1) + .withReconcilePrecondition(not_met_reconcile_condition) + .build() + .addDependentResource(drDeleter2).dependsOn(dr1, drDeleter).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory) + .reconciledInOrder(dr1, drDeleter2, drDeleter) + .deleted(drDeleter2, drDeleter); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void deleteDoneInReverseOrder() { + TestDeleterDependent drDeleter2 = new TestDeleterDependent("DR_DELETER_2"); + TestDeleterDependent drDeleter3 = new TestDeleterDependent("DR_DELETER_3"); + TestDeleterDependent drDeleter4 = new TestDeleterDependent("DR_DELETER_4"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(drDeleter).withReconcilePrecondition(not_met_reconcile_condition) + .dependsOn(dr1) + .build() + .addDependentResource(drDeleter2).dependsOn(drDeleter).build() + .addDependentResource(drDeleter3).dependsOn(drDeleter).build() + .addDependentResource(drDeleter4).dependsOn(drDeleter3).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory) + .reconciledInOrder(dr1, drDeleter4, drDeleter3, drDeleter) + .reconciledInOrder(dr1, drDeleter2, drDeleter) + .deleted(drDeleter, drDeleter2, drDeleter3, drDeleter4); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void diamondDeleteWithPostConditionInMiddle() { + TestDeleterDependent drDeleter2 = new TestDeleterDependent("DR_DELETER_2"); + TestDeleterDependent drDeleter3 = new TestDeleterDependent("DR_DELETER_3"); + TestDeleterDependent drDeleter4 = new TestDeleterDependent("DR_DELETER_4"); + + var workflow = new WorkflowBuilder() + .addDependentResource(drDeleter).withReconcilePrecondition(not_met_reconcile_condition) + .build() + .addDependentResource(drDeleter2).dependsOn(drDeleter).build() + .addDependentResource(drDeleter3).dependsOn(drDeleter) + .withDeletePostcondition(noMetDeletePostCondition).build() + .addDependentResource(drDeleter4).dependsOn(drDeleter3, drDeleter2).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory).notReconciled(drDeleter) + .reconciledInOrder(drDeleter4, drDeleter2) + .reconciledInOrder(drDeleter4, drDeleter3); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void diamondDeleteErrorInMiddle() { + TestDeleterDependent drDeleter2 = new TestDeleterDependent("DR_DELETER_2"); + TestDeleterDependent drDeleter3 = new TestDeleterDependent("DR_DELETER_3"); + + var workflow = new WorkflowBuilder() + .addDependentResource(drDeleter).withReconcilePrecondition(not_met_reconcile_condition) + .build() + .addDependentResource(drDeleter2).dependsOn(drDeleter).build() + .addDependentResource(errorDD).dependsOn(drDeleter).build() + .addDependentResource(drDeleter3).dependsOn(errorDD, drDeleter2).build() + .withThrowExceptionFurther(false) + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory) + .notReconciled(drDeleter, drError) + .reconciledInOrder(drDeleter3, drDeleter2); + + Assertions.assertThat(res.getErroredDependents()).containsOnlyKeys(errorDD); + Assertions.assertThat(res.getReconciledDependents()).isEmpty(); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void readyConditionTrivialCase() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).withReadyPostcondition(metReadyCondition).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory).reconciledInOrder(dr1, dr2); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2); + Assertions.assertThat(res.getNotReadyDependents()).isEmpty(); + } + + @Test + void readyConditionNotMetTrivialCase() { + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).withReadyPostcondition(notMetReadyCondition).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + + assertThat(executionHistory).reconciled(dr1).notReconciled(dr2); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1); + Assertions.assertThat(res.getNotReadyDependents()).containsExactlyInAnyOrder(dr1); + } + + @Test + void readyConditionNotMetInOneParent() { + TestDependent dr3 = new TestDependent("DR_3"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).withReadyPostcondition(notMetReadyCondition).build() + .addDependentResource(dr2).build() + .addDependentResource(dr3).dependsOn(dr1, dr2).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + assertThat(executionHistory).reconciled(dr1, dr2).notReconciled(dr3); + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2); + Assertions.assertThat(res.getNotReadyDependents()).containsExactlyInAnyOrder(dr1); + } + + @Test + void diamondShareWithReadyCondition() { + TestDependent dr3 = new TestDependent("DR_3"); + TestDependent dr4 = new TestDependent("DR_4"); + + var workflow = new WorkflowBuilder() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).withReadyPostcondition(notMetReadyCondition) + .build() + .addDependentResource(dr3).dependsOn(dr1).build() + .addDependentResource(dr4).dependsOn(dr2, dr3).build() + .build(); + + var res = workflow.reconcile(new TestCustomResource(), null); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + assertThat(executionHistory).reconciledInOrder(dr1, dr2) + .reconciledInOrder(dr1, dr3) + .notReconciled(dr4); + + Assertions.assertThat(res.getErroredDependents()).isEmpty(); + Assertions.assertThat(res.getReconciledDependents()).containsExactlyInAnyOrder(dr1, dr2, dr3); + Assertions.assertThat(res.getNotReadyDependents()).containsExactlyInAnyOrder(dr2); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowTest.java new file mode 100644 index 0000000000..01b8bc619c --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowTest.java @@ -0,0 +1,58 @@ +package io.javaoperatorsdk.operator.processing.dependent.workflow; + +import java.util.Set; +import java.util.stream.Collectors; + +import org.junit.jupiter.api.Test; + +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.builder.WorkflowBuilder; +import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; + +@SuppressWarnings("rawtypes") +class WorkflowTest { + + @Test + void calculatesTopLevelResources() { + var dr1 = mock(DependentResource.class); + var dr2 = mock(DependentResource.class); + var independentDR = mock(DependentResource.class); + + var workflow = new WorkflowBuilder() + .addDependentResource(independentDR).build() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .build(); + + Set topResources = + workflow.getTopLevelDependentResources().stream() + .map(DependentResourceNode::getDependentResource) + .collect(Collectors.toSet()); + + assertThat(topResources).containsExactlyInAnyOrder(dr1, independentDR); + } + + @Test + void calculatesBottomLevelResources() { + var dr1 = mock(DependentResource.class); + var dr2 = mock(DependentResource.class); + var independentDR = mock(DependentResource.class); + + Workflow workflow = new WorkflowBuilder() + .addDependentResource(independentDR).build() + .addDependentResource(dr1).build() + .addDependentResource(dr2).dependsOn(dr1).build() + .build(); + + Set bottomResources = + workflow.getBottomLevelResource().stream() + .map(DependentResourceNode::getDependentResource) + .collect(Collectors.toSet()); + + assertThat(bottomResources).containsExactlyInAnyOrder(dr2, independentDR); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventProcessorTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventProcessorTest.java index 4d03059c6b..788325ebe8 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventProcessorTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventProcessorTest.java @@ -15,7 +15,10 @@ import org.slf4j.LoggerFactory; import io.fabric8.kubernetes.api.model.HasMetadata; +import io.javaoperatorsdk.operator.api.config.RetryConfiguration; import io.javaoperatorsdk.operator.api.monitoring.Metrics; +import io.javaoperatorsdk.operator.processing.event.rate.PeriodRateLimiter; +import io.javaoperatorsdk.operator.processing.event.rate.RateLimiter; import io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceAction; import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEvent; @@ -29,8 +32,19 @@ import static org.awaitility.Awaitility.await; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.ArgumentMatchers.isNull; -import static org.mockito.Mockito.*; - +import static org.mockito.Mockito.after; +import static org.mockito.Mockito.any; +import static org.mockito.Mockito.anyLong; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.spy; +import static org.mockito.Mockito.timeout; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@SuppressWarnings({"rawtypes", "unchecked"}) class EventProcessorTest { private static final Logger log = LoggerFactory.getLogger(EventProcessorTest.class); @@ -39,15 +53,16 @@ class EventProcessorTest { public static final int SEPARATE_EXECUTION_TIMEOUT = 450; public static final String TEST_NAMESPACE = "default-event-handler-test"; - private ReconciliationDispatcher reconciliationDispatcherMock = + private final ReconciliationDispatcher reconciliationDispatcherMock = mock(ReconciliationDispatcher.class); - private EventSourceManager eventSourceManagerMock = mock(EventSourceManager.class); - private TimerEventSource retryTimerEventSourceMock = mock(TimerEventSource.class); - private ControllerResourceEventSource controllerResourceEventSourceMock = + private final EventSourceManager eventSourceManagerMock = mock(EventSourceManager.class); + private final TimerEventSource retryTimerEventSourceMock = mock(TimerEventSource.class); + private final ControllerResourceEventSource controllerResourceEventSourceMock = mock(ControllerResourceEventSource.class); - private Metrics metricsMock = mock(Metrics.class); + private final Metrics metricsMock = mock(Metrics.class); private EventProcessor eventProcessor; private EventProcessor eventProcessorWithRetry; + private final RateLimiter rateLimiterMock = mock(RateLimiter.class); @BeforeEach void setup() { @@ -55,14 +70,15 @@ void setup() { .thenReturn(controllerResourceEventSourceMock); eventProcessor = spy(new EventProcessor(reconciliationDispatcherMock, eventSourceManagerMock, "Test", null, - null)); + rateLimiterMock, null)); eventProcessor.start(); eventProcessorWithRetry = spy(new EventProcessor(reconciliationDispatcherMock, eventSourceManagerMock, "Test", - GenericRetry.defaultLimitedExponentialRetry(), null)); + GenericRetry.defaultLimitedExponentialRetry(), rateLimiterMock, null)); eventProcessorWithRetry.start(); when(eventProcessor.retryEventSource()).thenReturn(retryTimerEventSourceMock); when(eventProcessorWithRetry.retryEventSource()).thenReturn(retryTimerEventSourceMock); + when(rateLimiterMock.acquirePermission(any())).thenReturn(Optional.empty()); } @Test @@ -84,7 +100,7 @@ void skipProcessingIfLatestCustomResourceNotInCache() { } @Test - void ifExecutionInProgressWaitsUntilItsFinished() throws InterruptedException { + void ifExecutionInProgressWaitsUntilItsFinished() { ResourceID resourceUid = eventAlreadyUnderProcessing(); eventProcessor.handleEvent(nonCREvent(resourceUid)); @@ -104,7 +120,8 @@ void schedulesAnEventRetryOnException() { eventProcessorWithRetry.eventProcessingFinished(executionScope, postExecutionControl); verify(retryTimerEventSourceMock, times(1)) - .scheduleOnce(eq(customResource), eq(GenericRetry.DEFAULT_INITIAL_INTERVAL)); + .scheduleOnce(eq(ResourceID.fromResource(customResource)), + eq(RetryConfiguration.DEFAULT_INITIAL_INTERVAL)); } @Test @@ -135,7 +152,8 @@ void executesTheControllerInstantlyAfterErrorIfNewEventsReceived() { List allValues = executionScopeArgumentCaptor.getAllValues(); assertThat(allValues).hasSize(2); verify(retryTimerEventSourceMock, never()) - .scheduleOnce(eq(customResource), eq(GenericRetry.DEFAULT_INITIAL_INTERVAL)); + .scheduleOnce(eq(ResourceID.fromResource(customResource)), + eq(RetryConfiguration.DEFAULT_INITIAL_INTERVAL)); } @Test @@ -197,7 +215,7 @@ void scheduleTimedEventIfInstructedByPostExecutionControl() { eventProcessor.handleEvent(prepareCREvent()); verify(retryTimerEventSourceMock, timeout(SEPARATE_EXECUTION_TIMEOUT).times(1)) - .scheduleOnce(any(), eq(testDelay)); + .scheduleOnce((ResourceID) any(), eq(testDelay)); } @Test @@ -213,7 +231,7 @@ void reScheduleOnlyIfNotExecutedEventsReceivedMeanwhile() throws InterruptedExce verify(retryTimerEventSourceMock, after((long) (FAKE_CONTROLLER_EXECUTION_DURATION * 1.5)).times(0)) - .scheduleOnce(any(), eq(testDelay)); + .scheduleOnce((ResourceID) any(), eq(testDelay)); } @Test @@ -240,6 +258,7 @@ void startProcessedMarkedEventReceivedBefore() { var crID = new ResourceID("test-cr", TEST_NAMESPACE); eventProcessor = spy(new EventProcessor(reconciliationDispatcherMock, eventSourceManagerMock, "Test", null, + new PeriodRateLimiter(), metricsMock)); when(controllerResourceEventSourceMock.get(eq(crID))) .thenReturn(Optional.of(testCustomResource())); @@ -326,6 +345,25 @@ void newResourceAfterMissedDeleteEvent() { verify(reconciliationDispatcherMock, timeout(50).times(1)).handleExecution(any()); } + @Test + void rateLimitsReconciliationSubmission() { + // the refresh period value does not matter here + var refreshPeriod = Duration.ofMillis(100); + var event = prepareCREvent(); + + when(rateLimiterMock.acquirePermission(event.getRelatedCustomResourceID())) + .thenReturn(Optional.empty()) + .thenReturn(Optional.of(refreshPeriod)); + + eventProcessor.handleEvent(event); + verify(reconciliationDispatcherMock, after(FAKE_CONTROLLER_EXECUTION_DURATION).times(1)) + .handleExecution(any()); + verify(retryTimerEventSourceMock, times(0)).scheduleOnce((ResourceID) any(), anyLong()); + + eventProcessor.handleEvent(event); + verify(retryTimerEventSourceMock, times(1)).scheduleOnce((ResourceID) any(), anyLong()); + } + private ResourceID eventAlreadyUnderProcessing() { when(reconciliationDispatcherMock.handleExecution(any())) .then( diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourceManagerTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourceManagerTest.java index 821efbd16a..779fe032f9 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourceManagerTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourceManagerTest.java @@ -11,10 +11,10 @@ import io.javaoperatorsdk.operator.api.config.informer.InformerConfiguration; import io.javaoperatorsdk.operator.api.reconciler.Reconciler; import io.javaoperatorsdk.operator.processing.Controller; -import io.javaoperatorsdk.operator.processing.event.source.CachingEventSource; import io.javaoperatorsdk.operator.processing.event.source.EventSource; import io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource; import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource; +import io.javaoperatorsdk.operator.processing.event.source.informer.ManagedInformerEventSource; import io.javaoperatorsdk.operator.processing.event.source.timer.TimerEventSource; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; @@ -79,7 +79,7 @@ void retrievingEventSourceForClassShouldWork() { assertThatExceptionOfType(IllegalArgumentException.class) .isThrownBy(() -> manager.getResourceEventSourceFor(HasMetadata.class, "unknown_name")); - CachingEventSource eventSource = mock(CachingEventSource.class); + ManagedInformerEventSource eventSource = mock(ManagedInformerEventSource.class); when(eventSource.resourceType()).thenReturn(String.class); manager.registerEventSource(eventSource); @@ -93,11 +93,11 @@ void shouldNotBePossibleToAddEventSourcesForSameTypeAndName() { EventSourceManager manager = initManager(); final var name = "name1"; - CachingEventSource eventSource = mock(CachingEventSource.class); + ManagedInformerEventSource eventSource = mock(ManagedInformerEventSource.class); when(eventSource.resourceType()).thenReturn(TestCustomResource.class); manager.registerEventSource(name, eventSource); - eventSource = mock(CachingEventSource.class); + eventSource = mock(ManagedInformerEventSource.class); when(eventSource.resourceType()).thenReturn(TestCustomResource.class); final var source = eventSource; @@ -114,11 +114,11 @@ void shouldNotBePossibleToAddEventSourcesForSameTypeAndName() { void retrievingAnEventSourceWhenMultipleAreRegisteredForATypeShouldRequireAQualifier() { EventSourceManager manager = initManager(); - CachingEventSource eventSource = mock(CachingEventSource.class); + ManagedInformerEventSource eventSource = mock(ManagedInformerEventSource.class); when(eventSource.resourceType()).thenReturn(TestCustomResource.class); manager.registerEventSource("name1", eventSource); - CachingEventSource eventSource2 = mock(CachingEventSource.class); + ManagedInformerEventSource eventSource2 = mock(ManagedInformerEventSource.class); when(eventSource2.resourceType()).thenReturn(TestCustomResource.class); manager.registerEventSource("name2", eventSource2); diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourcesTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourcesTest.java index ecc5c12079..7e1fc36967 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourcesTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/EventSourcesTest.java @@ -9,7 +9,6 @@ import io.javaoperatorsdk.operator.processing.Controller; import io.javaoperatorsdk.operator.processing.event.source.EventSource; -import static io.javaoperatorsdk.operator.processing.event.EventSources.CONTROLLER_RESOURCE_EVENT_SOURCE_NAME; import static io.javaoperatorsdk.operator.processing.event.EventSources.RETRY_RESCHEDULE_TIMER_EVENT_SOURCE_NAME; import static org.assertj.core.api.Assertions.assertThat; import static org.junit.jupiter.api.Assertions.*; @@ -29,34 +28,14 @@ void cannotAddTwoEventSourcesWithSameName() { }); } - @Test - void allEventSourcesShouldReturnAll() { - // initial state doesn't have ControllerResourceEventSource - assertThat(eventSources.eventSources()).containsExactly(eventSources.retryEventSource()); - - initControllerEventSource(); - - assertThat(eventSources.eventSources()).containsExactly( - eventSources.controllerResourceEventSource(), - eventSources.retryEventSource()); - - final var source = mock(EventSource.class); - eventSources.add(EVENT_SOURCE_NAME, source); - // order matters - assertThat(eventSources.eventSources()) - .containsExactly(eventSources.controllerResourceEventSource(), - eventSources.retryEventSource(), source); - } @Test - void eventSourcesIteratorShouldReturnControllerEventSourceAsFirst() { + void eventSourcesStreamShouldNotReturnControllerEventSource() { initControllerEventSource(); final var source = mock(EventSource.class); eventSources.add(EVENT_SOURCE_NAME, source); - assertThat(eventSources.iterator()).toIterable().containsExactly( - new NamedEventSource(eventSources.controllerResourceEventSource(), - CONTROLLER_RESOURCE_EVENT_SOURCE_NAME), + assertThat(eventSources.additionalNamedEventSources()).containsExactly( new NamedEventSource(eventSources.retryEventSource(), RETRY_RESCHEDULE_TIMER_EVENT_SOURCE_NAME), new NamedEventSource(source, EVENT_SOURCE_NAME)); diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcherTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcherTest.java index 166075b6bb..f19ba918f3 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcherTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcherTest.java @@ -25,7 +25,6 @@ import io.javaoperatorsdk.operator.api.config.ConfigurationServiceProvider; import io.javaoperatorsdk.operator.api.config.ControllerConfiguration; import io.javaoperatorsdk.operator.api.config.MockControllerConfiguration; -import io.javaoperatorsdk.operator.api.config.RetryConfiguration; import io.javaoperatorsdk.operator.api.reconciler.Cleaner; import io.javaoperatorsdk.operator.api.reconciler.Context; import io.javaoperatorsdk.operator.api.reconciler.DeleteControl; @@ -36,6 +35,7 @@ import io.javaoperatorsdk.operator.api.reconciler.UpdateControl; import io.javaoperatorsdk.operator.processing.Controller; import io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.CustomResourceFacade; +import io.javaoperatorsdk.operator.processing.retry.GenericRetry; import io.javaoperatorsdk.operator.sample.observedgeneration.ObservedGenCustomResource; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; @@ -110,7 +110,7 @@ private ReconciliationDispatcher init(R customResourc when(configuration.getFinalizerName()).thenReturn(DEFAULT_FINALIZER); when(configuration.getName()).thenReturn("EventDispatcherTestController"); when(configuration.getResourceClass()).thenReturn(resourceClass); - when(configuration.getRetryConfiguration()).thenReturn(RetryConfiguration.DEFAULT); + when(configuration.getRetry()).thenReturn(new GenericRetry()); when(configuration.reconciliationMaxInterval()) .thenReturn(Optional.of(Duration.ofHours(RECONCILIATION_MAX_INTERVAL))); diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiterTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiterTest.java new file mode 100644 index 0000000000..90ad8447cf --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/rate/PeriodRateLimiterTest.java @@ -0,0 +1,65 @@ +package io.javaoperatorsdk.operator.processing.event.rate; + +import java.time.Duration; + +import org.junit.jupiter.api.Test; + +import io.javaoperatorsdk.operator.TestUtils; +import io.javaoperatorsdk.operator.processing.event.ResourceID; + +import static org.assertj.core.api.Assertions.assertThat; + +class PeriodRateLimiterTest { + + public static final Duration REFRESH_PERIOD = Duration.ofMillis(300); + ResourceID resourceID = ResourceID.fromResource(TestUtils.testCustomResource()); + + @Test + void acquirePermissionForNewResource() { + var rl = new PeriodRateLimiter(REFRESH_PERIOD, 2); + var res = rl.acquirePermission(resourceID); + assertThat(res).isEmpty(); + res = rl.acquirePermission(resourceID); + assertThat(res).isEmpty(); + + res = rl.acquirePermission(resourceID); + assertThat(res).isNotEmpty(); + } + + @Test + void returnsMinimalDurationToAcquirePermission() { + var rl = new PeriodRateLimiter(REFRESH_PERIOD, 1); + var res = rl.acquirePermission(resourceID); + assertThat(res).isEmpty(); + + res = rl.acquirePermission(resourceID); + + assertThat(res).isPresent(); + assertThat(res.get()).isLessThan(REFRESH_PERIOD); + } + + @Test + void resetsPeriodAfterLimit() throws InterruptedException { + var rl = new PeriodRateLimiter(REFRESH_PERIOD, 1); + var res = rl.acquirePermission(resourceID); + assertThat(res).isEmpty(); + res = rl.acquirePermission(resourceID); + assertThat(res).isPresent(); + + // sleep plus some slack + Thread.sleep(REFRESH_PERIOD.toMillis() + REFRESH_PERIOD.toMillis() / 3); + + res = rl.acquirePermission(resourceID); + assertThat(res).isEmpty(); + } + + @Test + void rateLimitCanBeTurnedOff() { + var rl = new PeriodRateLimiter(REFRESH_PERIOD, PeriodRateLimiter.NO_LIMIT_PERIOD); + + var res = rl.acquirePermission(resourceID); + + assertThat(res).isEmpty(); + } + +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/CustomResourceSelectorTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/CustomResourceSelectorTest.java index 3de6578515..608b58d08a 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/CustomResourceSelectorTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/CustomResourceSelectorTest.java @@ -20,11 +20,8 @@ import io.javaoperatorsdk.operator.api.config.ConfigurationServiceProvider; import io.javaoperatorsdk.operator.api.config.DefaultControllerConfiguration; import io.javaoperatorsdk.operator.api.config.Version; -import io.javaoperatorsdk.operator.api.reconciler.Constants; -import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.api.reconciler.ControllerConfiguration; -import io.javaoperatorsdk.operator.api.reconciler.Reconciler; -import io.javaoperatorsdk.operator.api.reconciler.UpdateControl; +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.processing.event.rate.PeriodRateLimiter; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; import static org.assertj.core.api.Assertions.assertThat; @@ -55,7 +52,10 @@ void setUpResources() { configurationService = spy(ConfigurationService.class); when(configurationService.checkCRDAndValidateLocalModel()).thenReturn(false); when(configurationService.getVersion()).thenReturn(new Version("1", "1", new Date())); + // make sure not the same config instance is used for the controller, so rate limiter is not + // shared when(configurationService.getConfigurationFor(any(MyController.class))) + .thenReturn(new MyConfiguration()) .thenReturn(new MyConfiguration()); } @@ -136,7 +136,8 @@ public static class MyConfiguration extends DefaultControllerConfiguration false); + setUpSource(delFilteringEventSource); + // try without any resources added + source.handleDeletes(primaryID1(), Set.of(testResource1(), testResource2())); + source.handleResources(primaryID1(), Set.of(testResource1(), testResource2())); + // handling the add event + verify(eventHandler, times(1)).handleEvent(any()); + + source.handleDeletes(primaryID1(), Set.of(testResource1(), testResource2())); + + // no more invocation + verify(eventHandler, times(1)).handleEvent(any()); + } + + @Test + void filtersAddEvents() { + TestExternalCachingEventSource delFilteringEventSource = new TestExternalCachingEventSource(); + delFilteringEventSource.setOnAddFilter((res) -> false); + setUpSource(delFilteringEventSource); + + source.handleResources(primaryID1(), Set.of(testResource1())); + verify(eventHandler, times(0)).handleEvent(any()); + + source.handleResources(primaryID1(), Set.of(testResource1(), testResource2())); + verify(eventHandler, times(0)).handleEvent(any()); + } + + @Test + void filtersUpdateEvents() { + TestExternalCachingEventSource delFilteringEventSource = new TestExternalCachingEventSource(); + delFilteringEventSource.setOnUpdateFilter((res, res2) -> false); + setUpSource(delFilteringEventSource); + source.handleResources(primaryID1(), Set.of(testResource1())); + verify(eventHandler, times(1)).handleEvent(any()); + + var resource = testResource1(); + resource.setValue("changed value"); + source.handleResources(primaryID1(), Set.of(resource)); + + verify(eventHandler, times(1)).handleEvent(any()); + } + + @Test + void filtersImplicitDeleteEvents() { + TestExternalCachingEventSource delFilteringEventSource = new TestExternalCachingEventSource(); + delFilteringEventSource.setOnDeleteFilter((res, b) -> false); + setUpSource(delFilteringEventSource); + + source.handleResources(primaryID1(), Set.of(testResource1(), testResource2())); + verify(eventHandler, times(1)).handleEvent(any()); + + source.handleResources(primaryID1(), Set.of(testResource1())); + verify(eventHandler, times(1)).handleEvent(any()); + } + + @Test + void genericFilteringEvents() { + TestExternalCachingEventSource delFilteringEventSource = new TestExternalCachingEventSource(); + delFilteringEventSource.setGenericFilter(res -> false); + setUpSource(delFilteringEventSource); + + source.handleResources(primaryID1(), Set.of(testResource1())); + verify(eventHandler, times(0)).handleEvent(any()); + + source.handleResources(primaryID1(), Set.of(testResource1(), testResource2())); + verify(eventHandler, times(0)).handleEvent(any()); + + source.handleResources(primaryID1(), Set.of(testResource2())); + verify(eventHandler, times(0)).handleEvent(any()); + } + public static class TestExternalCachingEventSource extends ExternalResourceCachingEventSource { public TestExternalCachingEventSource() { - super(SampleExternalResource.class, (r) -> r.getName() + "#" + r.getValue()); + super(SampleExternalResource.class, (r) -> r.getName()); } } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventFilterTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventFilterTest.java index 1c4c2bcce1..d8309ae3e1 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventFilterTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/ResourceEventFilterTest.java @@ -56,7 +56,12 @@ public void eventFilteredByCustomPredicate() { cr.getMetadata().setGeneration(1L); cr.getStatus().setConfigMapStatus("1"); - eventSource.eventReceived(ResourceAction.UPDATED, cr, null); + TestCustomResource cr2 = TestUtils.testCustomResource(); + cr.getMetadata().setFinalizers(List.of(FINALIZER)); + cr.getMetadata().setGeneration(1L); + cr.getStatus().setConfigMapStatus("2"); + + eventSource.eventReceived(ResourceAction.UPDATED, cr, cr2); verify(eventHandler, times(1)).handleEvent(any()); cr.getMetadata().setGeneration(1L); @@ -139,8 +144,8 @@ public ControllerConfig(String finalizer, boolean generationAware, null, eventFilter, customResourceClass, - null, - null); + null, null, + null, null, null, null, null); } } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSourceTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSourceTest.java index 1c467e0490..db3b35420a 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSourceTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSourceTest.java @@ -2,6 +2,8 @@ import java.time.LocalDateTime; import java.util.List; +import java.util.function.BiPredicate; +import java.util.function.Predicate; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -16,10 +18,7 @@ import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; import static org.mockito.ArgumentMatchers.eq; -import static org.mockito.Mockito.any; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.*; class ControllerResourceEventSourceTest extends AbstractEventSourceTestBase, EventHandler> { @@ -90,7 +89,7 @@ void handlesAllEventIfNotGenerationAware() { } @Test - public void eventWithNoGenerationProcessedIfNoFinalizer() { + void eventWithNoGenerationProcessedIfNoFinalizer() { TestCustomResource customResource1 = TestUtils.testCustomResource(); source.eventReceived(ResourceAction.UPDATED, customResource1, customResource1); @@ -99,7 +98,7 @@ public void eventWithNoGenerationProcessedIfNoFinalizer() { } @Test - public void callsBroadcastsOnResourceEvents() { + void callsBroadcastsOnResourceEvents() { TestCustomResource customResource1 = TestUtils.testCustomResource(); source.eventReceived(ResourceAction.UPDATED, customResource1, customResource1); @@ -109,14 +108,53 @@ public void callsBroadcastsOnResourceEvents() { eq(customResource1)); } + @Test + void filtersOutEventsOnAddAndUpdate() { + TestCustomResource cr = TestUtils.testCustomResource(); + + Predicate onAddPredicate = (res) -> false; + BiPredicate onUpdatePredicate = (res, res2) -> false; + source = + new ControllerResourceEventSource<>( + new TestController(onAddPredicate, onUpdatePredicate, null)); + setUpSource(source); + + source.eventReceived(ResourceAction.ADDED, cr, null); + source.eventReceived(ResourceAction.UPDATED, cr, cr); + + verify(eventHandler, never()).handleEvent(any()); + } + + @Test + void genericFilterFiltersOutAddUpdateAndDeleteEvents() { + TestCustomResource cr = TestUtils.testCustomResource(); + + source = + new ControllerResourceEventSource<>(new TestController(null, null, res -> false)); + setUpSource(source); + + source.eventReceived(ResourceAction.ADDED, cr, null); + source.eventReceived(ResourceAction.UPDATED, cr, cr); + source.eventReceived(ResourceAction.DELETED, cr, cr); + + verify(eventHandler, never()).handleEvent(any()); + } + @SuppressWarnings("unchecked") private static class TestController extends Controller { private final EventSourceManager eventSourceManager = mock(EventSourceManager.class); + public TestController(Predicate onAddFilter, + BiPredicate onUpdateFilter, + Predicate genericFilter) { + super(null, new TestConfiguration(true, onAddFilter, onUpdateFilter, genericFilter), + MockKubernetesClient.client(TestCustomResource.class)); + } + public TestController(boolean generationAware) { - super(null, new TestConfiguration(generationAware), + super(null, new TestConfiguration(generationAware, null, null, null), MockKubernetesClient.client(TestCustomResource.class)); } @@ -134,7 +172,9 @@ public boolean useFinalizer() { private static class TestConfiguration extends DefaultControllerConfiguration { - public TestConfiguration(boolean generationAware) { + public TestConfiguration(boolean generationAware, Predicate onAddFilter, + BiPredicate onUpdateFilter, + Predicate genericFilter) { super( null, null, @@ -147,7 +187,7 @@ public TestConfiguration(boolean generationAware) { null, TestCustomResource.class, null, - null); + onAddFilter, onUpdateFilter, genericFilter, null, null, null); } } } diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/InternalEventFiltersTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/InternalEventFiltersTest.java new file mode 100644 index 0000000000..2f1fec6917 --- /dev/null +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/controller/InternalEventFiltersTest.java @@ -0,0 +1,60 @@ +package io.javaoperatorsdk.operator.processing.event.source.controller; + +import java.util.List; + +import org.junit.jupiter.api.Test; + +import io.javaoperatorsdk.operator.TestUtils; + +import static io.javaoperatorsdk.operator.TestUtils.markForDeletion; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.*; + +class InternalEventFiltersTest { + + public static final String FINALIZER = "finalizer"; + + @Test + void onUpdateMarkedForDeletion() { + var res = markForDeletion(TestUtils.testCustomResource()); + assertThat(InternalEventFilters.onUpdateMarkedForDeletion().test(res, res)).isTrue(); + } + + @Test + void generationAware() { + var res = TestUtils.testCustomResource1(); + var res2 = TestUtils.testCustomResource1(); + res2.getMetadata().setGeneration(2L); + + assertThat(InternalEventFilters.onUpdateGenerationAware(true).test(res2, res)).isTrue(); + assertThat(InternalEventFilters.onUpdateGenerationAware(true).test(res, res)).isFalse(); + assertThat(InternalEventFilters.onUpdateGenerationAware(false).test(res, res)).isTrue(); + } + + @Test + void finalizerCheckedIfConfigured() { + assertThat(InternalEventFilters.onUpdateFinalizerNeededAndApplied(true, FINALIZER) + .test(TestUtils.testCustomResource1(), TestUtils.testCustomResource1())).isTrue(); + + var res = TestUtils.testCustomResource1(); + res.getMetadata().setFinalizers(List.of(FINALIZER)); + + assertThat(InternalEventFilters.onUpdateFinalizerNeededAndApplied(true, FINALIZER) + .test(res, res)).isFalse(); + } + + @Test + void acceptsIfFinalizerWasJustAdded() { + var res = TestUtils.testCustomResource1(); + res.getMetadata().setFinalizers(List.of(FINALIZER)); + + assertThat(InternalEventFilters.onUpdateFinalizerNeededAndApplied(true, "finalizer") + .test(res, TestUtils.testCustomResource1())).isTrue(); + } + + @Test + void dontAcceptIfFinalizerNotUsed() { + assertThat(InternalEventFilters.onUpdateFinalizerNeededAndApplied(false, FINALIZER) + .test(TestUtils.testCustomResource1(), TestUtils.testCustomResource1())).isFalse(); + } +} diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSourceTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSourceTest.java index 173379f802..623705b609 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSourceTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSourceTest.java @@ -58,11 +58,13 @@ void setup() { .thenReturn(DEFAULT_NAMESPACES_SET); when(informerConfiguration.getSecondaryToPrimaryMapper()) .thenReturn(mock(SecondaryToPrimaryMapper.class)); + when(informerConfiguration.getResourceClass()).thenReturn(Deployment.class); informerEventSource = new InformerEventSource<>(informerConfiguration, clientMock); informerEventSource.setTemporalResourceCache(temporaryResourceCacheMock); informerEventSource.setEventHandler(eventHandlerMock); + SecondaryToPrimaryMapper secondaryToPrimaryMapper = mock(SecondaryToPrimaryMapper.class); when(informerConfiguration.getSecondaryToPrimaryMapper()) .thenReturn(secondaryToPrimaryMapper); @@ -204,6 +206,52 @@ void putsResourceOnTempCacheIfNoEventRecordedWithSameResourceVersion() { verify(temporaryResourceCacheMock, times(1)).unconditionallyCacheResource(any()); } + @Test + void genericFilterForEvents() { + informerEventSource.setGenericFilter(r -> false); + when(temporaryResourceCacheMock.getResourceFromCache(any())) + .thenReturn(Optional.empty()); + + informerEventSource.onAdd(testDeployment()); + informerEventSource.onUpdate(testDeployment(), testDeployment()); + informerEventSource.onDelete(testDeployment(), true); + + verify(eventHandlerMock, never()).handleEvent(any()); + } + + @Test + void filtersOnAddEvents() { + informerEventSource.setOnAddFilter(r -> false); + when(temporaryResourceCacheMock.getResourceFromCache(any())) + .thenReturn(Optional.empty()); + + informerEventSource.onAdd(testDeployment()); + + verify(eventHandlerMock, never()).handleEvent(any()); + } + + @Test + void filtersOnUpdateEvents() { + informerEventSource.setOnUpdateFilter((r1, r2) -> false); + when(temporaryResourceCacheMock.getResourceFromCache(any())) + .thenReturn(Optional.empty()); + + informerEventSource.onUpdate(testDeployment(), testDeployment()); + + verify(eventHandlerMock, never()).handleEvent(any()); + } + + @Test + void filtersOnDeleteEvents() { + informerEventSource.setOnDeleteFilter((r, b) -> false); + when(temporaryResourceCacheMock.getResourceFromCache(any())) + .thenReturn(Optional.empty()); + + informerEventSource.onDelete(testDeployment(), true); + + verify(eventHandlerMock, never()).handleEvent(any()); + } + Deployment testDeployment() { Deployment deployment = new Deployment(); deployment.setMetadata(new ObjectMeta()); diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/DefaultPrimaryToSecondaryIndexTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/PrimaryToSecondaryIndexTest.java similarity index 96% rename from operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/DefaultPrimaryToSecondaryIndexTest.java rename to operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/PrimaryToSecondaryIndexTest.java index da2d1b7cf0..4baa531274 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/DefaultPrimaryToSecondaryIndexTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/PrimaryToSecondaryIndexTest.java @@ -15,11 +15,11 @@ import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; -class DefaultPrimaryToSecondaryIndexTest { +class PrimaryToSecondaryIndexTest { private SecondaryToPrimaryMapper secondaryToPrimaryMapperMock = mock(SecondaryToPrimaryMapper.class); - private DefaultPrimaryToSecondaryIndex primaryToSecondaryIndex = + private PrimaryToSecondaryIndex primaryToSecondaryIndex = new DefaultPrimaryToSecondaryIndex<>(secondaryToPrimaryMapperMock); private ResourceID primaryID1 = new ResourceID("id1", "default"); diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSourceTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSourceTest.java index 2dc42c0b16..3fe3a5db58 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSourceTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSourceTest.java @@ -37,9 +37,9 @@ public void setup() { @Test public void schedulesOnce() { - TestCustomResource customResource = TestUtils.testCustomResource(); + var resourceID = ResourceID.fromResource(TestUtils.testCustomResource()); - source.scheduleOnce(customResource, PERIOD); + source.scheduleOnce(resourceID, PERIOD); untilAsserted(() -> assertThat(eventHandler.events).hasSize(1)); untilAsserted(PERIOD * 2, 0, () -> assertThat(eventHandler.events).hasSize(1)); @@ -47,20 +47,20 @@ public void schedulesOnce() { @Test public void canCancelOnce() { - TestCustomResource customResource = TestUtils.testCustomResource(); + var resourceID = ResourceID.fromResource(TestUtils.testCustomResource()); - source.scheduleOnce(customResource, PERIOD); - source.cancelOnceSchedule(ResourceID.fromResource(customResource)); + source.scheduleOnce(resourceID, PERIOD); + source.cancelOnceSchedule(resourceID); untilAsserted(() -> assertThat(eventHandler.events).isEmpty()); } @Test public void canRescheduleOnceEvent() { - TestCustomResource customResource = TestUtils.testCustomResource(); + var resourceID = ResourceID.fromResource(TestUtils.testCustomResource()); - source.scheduleOnce(customResource, PERIOD); - source.scheduleOnce(customResource, 2 * PERIOD); + source.scheduleOnce(resourceID, PERIOD); + source.scheduleOnce(resourceID, 2 * PERIOD); untilAsserted(PERIOD * 2, PERIOD, () -> assertThat(eventHandler.events).hasSize(1)); } @@ -69,7 +69,7 @@ public void canRescheduleOnceEvent() { public void deRegistersOnceEventSources() { TestCustomResource customResource = TestUtils.testCustomResource(); - source.scheduleOnce(customResource, PERIOD); + source.scheduleOnce(ResourceID.fromResource(customResource), PERIOD); source.onResourceDeleted(customResource); untilAsserted(() -> assertThat(eventHandler.events).isEmpty()); @@ -77,16 +77,16 @@ public void deRegistersOnceEventSources() { @Test public void eventNotRegisteredIfStopped() throws IOException { - TestCustomResource customResource = TestUtils.testCustomResource(); + var resourceID = ResourceID.fromResource(TestUtils.testCustomResource()); source.stop(); assertThatExceptionOfType(IllegalStateException.class).isThrownBy( - () -> source.scheduleOnce(customResource, PERIOD)); + () -> source.scheduleOnce(resourceID, PERIOD)); } @Test public void eventNotFiredIfStopped() throws IOException { - source.scheduleOnce(TestUtils.testCustomResource(), PERIOD); + source.scheduleOnce(ResourceID.fromResource(TestUtils.testCustomResource()), PERIOD); source.stop(); untilAsserted(() -> assertThat(eventHandler.events).isEmpty()); diff --git a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/retry/GenericRetryExecutionTest.java b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/retry/GenericRetryExecutionTest.java index 0f8e44d1b2..1dcd9df464 100644 --- a/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/retry/GenericRetryExecutionTest.java +++ b/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/retry/GenericRetryExecutionTest.java @@ -4,14 +4,16 @@ import org.junit.jupiter.api.Test; -import static io.javaoperatorsdk.operator.processing.retry.GenericRetry.DEFAULT_INITIAL_INTERVAL; +import io.javaoperatorsdk.operator.api.config.RetryConfiguration; + import static org.assertj.core.api.Assertions.assertThat; public class GenericRetryExecutionTest { @Test public void forFirstBackOffAlwaysReturnsInitialInterval() { - assertThat(getDefaultRetryExecution().nextDelay().get()).isEqualTo(DEFAULT_INITIAL_INTERVAL); + assertThat(getDefaultRetryExecution().nextDelay().get()) + .isEqualTo(RetryConfiguration.DEFAULT_INITIAL_INTERVAL); } @Test @@ -19,18 +21,19 @@ public void delayIsMultipliedEveryNextDelayCall() { RetryExecution retryExecution = getDefaultRetryExecution(); Optional res = callNextDelayNTimes(retryExecution, 1); - assertThat(res.get()).isEqualTo(DEFAULT_INITIAL_INTERVAL); + assertThat(res.get()).isEqualTo(RetryConfiguration.DEFAULT_INITIAL_INTERVAL); res = retryExecution.nextDelay(); assertThat(res.get()) - .isEqualTo((long) (DEFAULT_INITIAL_INTERVAL * GenericRetry.DEFAULT_MULTIPLIER)); + .isEqualTo((long) (RetryConfiguration.DEFAULT_INITIAL_INTERVAL + * RetryConfiguration.DEFAULT_MULTIPLIER)); res = retryExecution.nextDelay(); assertThat(res.get()) .isEqualTo( - (long) (DEFAULT_INITIAL_INTERVAL - * GenericRetry.DEFAULT_MULTIPLIER - * GenericRetry.DEFAULT_MULTIPLIER)); + (long) (RetryConfiguration.DEFAULT_INITIAL_INTERVAL + * RetryConfiguration.DEFAULT_MULTIPLIER + * RetryConfiguration.DEFAULT_MULTIPLIER)); } @Test diff --git a/operator-framework-junit5/pom.xml b/operator-framework-junit5/pom.xml index b1de5c892a..2800bfe1e1 100644 --- a/operator-framework-junit5/pom.xml +++ b/operator-framework-junit5/pom.xml @@ -5,7 +5,7 @@ java-operator-sdk io.javaoperatorsdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT 4.0.0 diff --git a/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/AbstractOperatorExtension.java b/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/AbstractOperatorExtension.java index 7164dd42af..1a2ff991c2 100644 --- a/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/AbstractOperatorExtension.java +++ b/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/AbstractOperatorExtension.java @@ -161,6 +161,8 @@ protected void afterAllImpl(ExtensionContext context) { } protected void afterEachImpl(ExtensionContext context) { + // resets the config service provider so the controller configs are reconstructed always + ConfigurationServiceProvider.reset(); if (!oneNamespacePerClass) { after(context); } diff --git a/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/ClusterOperatorExtension.java b/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/ClusterDeployedOperatorExtension.java similarity index 90% rename from operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/ClusterOperatorExtension.java rename to operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/ClusterDeployedOperatorExtension.java index 22e7974182..4b23fe0805 100644 --- a/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/ClusterOperatorExtension.java +++ b/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/ClusterDeployedOperatorExtension.java @@ -20,14 +20,15 @@ import io.fabric8.kubernetes.api.model.rbac.ClusterRoleBinding; import io.javaoperatorsdk.operator.api.config.ConfigurationService; -public class ClusterOperatorExtension extends AbstractOperatorExtension { +public class ClusterDeployedOperatorExtension extends AbstractOperatorExtension { - private static final Logger LOGGER = LoggerFactory.getLogger(ClusterOperatorExtension.class); + private static final Logger LOGGER = + LoggerFactory.getLogger(ClusterDeployedOperatorExtension.class); private final List operatorDeployment; private final Duration operatorDeploymentTimeout; - private ClusterOperatorExtension( + private ClusterDeployedOperatorExtension( ConfigurationService configurationService, List operatorDeployment, Duration operatorDeploymentTimeout, @@ -44,7 +45,7 @@ private ClusterOperatorExtension( } /** - * Creates a {@link Builder} to set up an {@link ClusterOperatorExtension} instance. + * Creates a {@link Builder} to set up an {@link ClusterDeployedOperatorExtension} instance. * * @return the builder. */ @@ -110,6 +111,7 @@ protected Builder() { this.deploymentTimeout = Duration.ofMinutes(1); } + @SuppressWarnings("unused") public Builder withDeploymentTimeout(Duration value) { deploymentTimeout = value; return this; @@ -127,13 +129,14 @@ public Builder withOperatorDeployment(List hm) { return this; } + @SuppressWarnings("unused") public Builder withOperatorDeployment(HasMetadata... hms) { operatorDeployment.addAll(Arrays.asList(hms)); return this; } - public ClusterOperatorExtension build() { - return new ClusterOperatorExtension( + public ClusterDeployedOperatorExtension build() { + return new ClusterDeployedOperatorExtension( configurationService, operatorDeployment, deploymentTimeout, diff --git a/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/LocalOperatorExtension.java b/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/LocallyRunOperatorExtension.java similarity index 96% rename from operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/LocalOperatorExtension.java rename to operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/LocallyRunOperatorExtension.java index 2eb0fa5900..e2f4234453 100644 --- a/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/LocalOperatorExtension.java +++ b/operator-framework-junit5/src/main/java/io/javaoperatorsdk/operator/junit/LocallyRunOperatorExtension.java @@ -28,9 +28,9 @@ import static io.javaoperatorsdk.operator.api.config.ControllerConfigurationOverrider.override; @SuppressWarnings("rawtypes") -public class LocalOperatorExtension extends AbstractOperatorExtension { +public class LocallyRunOperatorExtension extends AbstractOperatorExtension { - private static final Logger LOGGER = LoggerFactory.getLogger(LocalOperatorExtension.class); + private static final Logger LOGGER = LoggerFactory.getLogger(LocallyRunOperatorExtension.class); private final Operator operator; private final List reconcilers; @@ -39,7 +39,7 @@ public class LocalOperatorExtension extends AbstractOperatorExtension { private final List> additionalCustomResourceDefinitions; private final Map registeredControllers; - private LocalOperatorExtension( + private LocallyRunOperatorExtension( ConfigurationService configurationService, List reconcilers, List infrastructure, @@ -65,7 +65,7 @@ private LocalOperatorExtension( } /** - * Creates a {@link Builder} to set up an {@link LocalOperatorExtension} instance. + * Creates a {@link Builder} to set up an {@link LocallyRunOperatorExtension} instance. * * @return the builder. */ @@ -100,6 +100,7 @@ public RegisteredController getRegisteredControllerForReconcile( } @SuppressWarnings("unchecked") + @Override protected void before(ExtensionContext context) { super.before(context); @@ -162,6 +163,7 @@ private void applyCrd(String resourceTypeName) { } } + @Override protected void after(ExtensionContext context) { super.after(context); @@ -235,15 +237,15 @@ public Builder withPortForward(String namespace, String labelKey, String labelVa return this; } + public Builder withAdditionalCustomResourceDefinition( Class customResource) { additionalCustomResourceDefinitions.add(customResource); return this; } - - public LocalOperatorExtension build() { - return new LocalOperatorExtension( + public LocallyRunOperatorExtension build() { + return new LocallyRunOperatorExtension( configurationService, reconcilers, infrastructure, diff --git a/operator-framework/pom.xml b/operator-framework/pom.xml index 34c286797d..5720646d55 100644 --- a/operator-framework/pom.xml +++ b/operator-framework/pom.xml @@ -5,7 +5,7 @@ java-operator-sdk io.javaoperatorsdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT 4.0.0 diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ChangeNamespaceIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ChangeNamespaceIT.java index ca432b4410..a6a228bc5e 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ChangeNamespaceIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ChangeNamespaceIT.java @@ -12,7 +12,7 @@ import io.fabric8.kubernetes.api.model.NamespaceBuilder; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; import io.fabric8.kubernetes.client.KubernetesClient; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.changenamespace.ChangeNamespaceTestCustomResource; import io.javaoperatorsdk.operator.sample.changenamespace.ChangeNamespaceTestReconciler; @@ -26,9 +26,11 @@ class ChangeNamespaceIT { public static final String TEST_RESOURCE_NAME_3 = "test3"; public static final String ADDITIONAL_TEST_NAMESPACE = "additional-test-namespace"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new ChangeNamespaceTestReconciler()).build(); + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new ChangeNamespaceTestReconciler()) + .build(); + @SuppressWarnings("rawtypes") @Test void addNewAndRemoveOldNamespaceTest() { try { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanerForReconcilerIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanerForReconcilerIT.java index 8c48e2b5ab..65a828fd92 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanerForReconcilerIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanerForReconcilerIT.java @@ -4,7 +4,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.cleanerforreconciler.CleanerForReconcilerCustomResource; import io.javaoperatorsdk.operator.sample.cleanerforreconciler.CleanerForReconcilerTestReconciler; @@ -16,8 +16,8 @@ class CleanerForReconcilerIT { public static final String TEST_RESOURCE_NAME = "cleaner-for-reconciler-test1"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new CleanerForReconcilerTestReconciler()) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new CleanerForReconcilerTestReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanupConflictIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanupConflictIT.java index 65f783336f..d904698041 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanupConflictIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CleanupConflictIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.cleanupconflict.CleanupConflictCustomResource; import io.javaoperatorsdk.operator.sample.cleanupconflict.CleanupConflictReconciler; @@ -20,8 +20,8 @@ class CleanupConflictIT { public static final String TEST_RESOURCE_NAME = "test1"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new CleanupConflictReconciler()) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new CleanupConflictReconciler()) .build(); @Test @@ -30,10 +30,9 @@ void cleanupRemovesFinalizerWithoutConflict() throws InterruptedException { testResource.addFinalizer(ADDITIONAL_FINALIZER); testResource = operator.create(CleanupConflictCustomResource.class, testResource); - await().untilAsserted(() -> { - assertThat(operator.getReconcilerOfType(CleanupConflictReconciler.class) - .getNumberReconcileExecutions()).isEqualTo(1); - }); + await().untilAsserted( + () -> assertThat(operator.getReconcilerOfType(CleanupConflictReconciler.class) + .getNumberReconcileExecutions()).isEqualTo(1)); operator.delete(CleanupConflictCustomResource.class, testResource); Thread.sleep(WAIT_TIME / 2); @@ -42,10 +41,9 @@ void cleanupRemovesFinalizerWithoutConflict() throws InterruptedException { testResource.getMetadata().setResourceVersion(null); operator.replace(CleanupConflictCustomResource.class, testResource); - await().pollDelay(Duration.ofMillis(WAIT_TIME * 2)).untilAsserted(() -> { - assertThat(operator.getReconcilerOfType(CleanupConflictReconciler.class) - .getNumberOfCleanupExecutions()).isEqualTo(1); - }); + await().pollDelay(Duration.ofMillis(WAIT_TIME * 2)).untilAsserted( + () -> assertThat(operator.getReconcilerOfType(CleanupConflictReconciler.class) + .getNumberOfCleanupExecutions()).isEqualTo(1)); } private CleanupConflictCustomResource createTestResource() { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ConcurrencyIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ConcurrencyIT.java index f238d0d63b..b6216009b7 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ConcurrencyIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ConcurrencyIT.java @@ -10,7 +10,7 @@ import org.slf4j.LoggerFactory; import io.fabric8.kubernetes.api.model.ConfigMap; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; import io.javaoperatorsdk.operator.sample.simple.TestReconciler; import io.javaoperatorsdk.operator.support.TestUtils; @@ -26,8 +26,8 @@ class ConcurrencyIT { private static final Logger log = LoggerFactory.getLogger(ConcurrencyIT.class); @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new TestReconciler(true)).build(); + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new TestReconciler(true)).build(); @Test void manyResourcesGetCreatedUpdatedAndDeleted() throws InterruptedException { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ControllerExecutionIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ControllerExecutionIT.java index 9c8e786e0e..9bcea57f58 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ControllerExecutionIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ControllerExecutionIT.java @@ -7,7 +7,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ConfigMap; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; import io.javaoperatorsdk.operator.sample.simple.TestReconciler; import io.javaoperatorsdk.operator.support.TestUtils; @@ -18,8 +18,8 @@ class ControllerExecutionIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new TestReconciler(true)).build(); + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new TestReconciler(true)).build(); @Test void configMapGetsCreatedForTestCustomResource() { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CreateUpdateInformerEventSourceEventFilterIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CreateUpdateInformerEventSourceEventFilterIT.java index 1bd00e6d87..75310157f4 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CreateUpdateInformerEventSourceEventFilterIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CreateUpdateInformerEventSourceEventFilterIT.java @@ -7,7 +7,7 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.createupdateeventfilter.CreateUpdateEventFilterTestCustomResource; import io.javaoperatorsdk.operator.sample.createupdateeventfilter.CreateUpdateEventFilterTestCustomResourceSpec; import io.javaoperatorsdk.operator.sample.createupdateeventfilter.CreateUpdateEventFilterTestReconciler; @@ -19,8 +19,8 @@ class CreateUpdateInformerEventSourceEventFilterIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(new CreateUpdateEventFilterTestReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CustomResourceFilterIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CustomResourceFilterIT.java index e7fd4b60e4..1d45c808ac 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/CustomResourceFilterIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/CustomResourceFilterIT.java @@ -4,7 +4,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.customfilter.CustomFilteringTestReconciler; import io.javaoperatorsdk.operator.sample.customfilter.CustomFilteringTestResource; import io.javaoperatorsdk.operator.sample.customfilter.CustomFilteringTestResourceSpec; @@ -14,8 +14,9 @@ class CustomResourceFilterIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new CustomFilteringTestReconciler()).build(); + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new CustomFilteringTestReconciler()) + .build(); @Test void doesCustomFiltering() throws InterruptedException { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DeleterForManagedDependentResourcesOnlyIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DeleterForManagedDependentResourcesOnlyIT.java index 6b7ea41a4e..22e1b0f2be 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DeleterForManagedDependentResourcesOnlyIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DeleterForManagedDependentResourcesOnlyIT.java @@ -4,7 +4,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.cleanermanageddependent.CleanerForManagedDependentCustomResource; import io.javaoperatorsdk.operator.sample.cleanermanageddependent.CleanerForManagedDependentTestReconciler; import io.javaoperatorsdk.operator.sample.cleanermanageddependent.ConfigMapDependentResource; @@ -17,8 +17,8 @@ class DeleterForManagedDependentResourcesOnlyIT { public static final String TEST_RESOURCE_NAME = "cleaner-for-reconciler-test1"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(new CleanerForManagedDependentTestReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentAnnotationSecondaryMapperIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentAnnotationSecondaryMapperIT.java new file mode 100644 index 0000000000..ddc61a8590 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentAnnotationSecondaryMapperIT.java @@ -0,0 +1,64 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.dependentannotationsecondarymapper.DependentAnnotationSecondaryMapperReconciler; +import io.javaoperatorsdk.operator.sample.dependentannotationsecondarymapper.DependentAnnotationSecondaryMapperResource; + +import static io.javaoperatorsdk.operator.processing.event.source.informer.Mappers.DEFAULT_ANNOTATION_FOR_NAME; +import static io.javaoperatorsdk.operator.processing.event.source.informer.Mappers.DEFAULT_ANNOTATION_FOR_NAMESPACE; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +class DependentAnnotationSecondaryMapperIT { + + public static final String TEST_RESOURCE_NAME = "test1"; + + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() + .withReconciler(DependentAnnotationSecondaryMapperReconciler.class) + .build(); + + @Test + void mapsSecondaryByAnnotation() { + operator.create(DependentAnnotationSecondaryMapperResource.class, testResource()); + + var reconciler = + operator.getReconcilerOfType(DependentAnnotationSecondaryMapperReconciler.class); + + await().pollDelay(Duration.ofMillis(150)) + .untilAsserted(() -> assertThat(reconciler.getNumberOfExecutions()).isEqualTo(1)); + var configMap = operator.get(ConfigMap.class, TEST_RESOURCE_NAME); + + var annotations = configMap.getMetadata().getAnnotations(); + + assertThat(annotations) + .containsEntry(DEFAULT_ANNOTATION_FOR_NAME, TEST_RESOURCE_NAME) + .containsEntry(DEFAULT_ANNOTATION_FOR_NAMESPACE, operator.getNamespace()); + + assertThat(configMap.getMetadata().getOwnerReferences()).isEmpty(); + + configMap.getData().put("additional_data", "data"); + operator.replace(ConfigMap.class, configMap); + + await().pollDelay(Duration.ofMillis(150)) + .untilAsserted(() -> assertThat(reconciler.getNumberOfExecutions()).isEqualTo(2)); + } + + + DependentAnnotationSecondaryMapperResource testResource() { + var res = new DependentAnnotationSecondaryMapperResource(); + res.setMetadata(new ObjectMetaBuilder() + .withName(TEST_RESOURCE_NAME) + .build()); + return res; + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentFilterIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentFilterIT.java new file mode 100644 index 0000000000..a900d0645d --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentFilterIT.java @@ -0,0 +1,60 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; +import java.util.Map; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestCustomResource; +import io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestReconciler; +import io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestResourceSpec; + +import static io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestReconciler.CM_VALUE_KEY; +import static io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestReconciler.CONFIG_MAP_FILTER_VALUE; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +class DependentFilterIT { + + public static final String RESOURCE_NAME = "test1"; + + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(DependentFilterTestReconciler.class) + .build(); + + @Test + void filtersUpdateOnConfigMap() { + var resource = createResource(); + operator.create(DependentFilterTestCustomResource.class, resource); + + await().pollDelay(Duration.ofMillis(150)).untilAsserted(() -> { + assertThat(operator.getReconcilerOfType(DependentFilterTestReconciler.class) + .getNumberOfExecutions()).isEqualTo(1); + }); + + var configMap = operator.get(ConfigMap.class, RESOURCE_NAME); + configMap.setData(Map.of(CM_VALUE_KEY, CONFIG_MAP_FILTER_VALUE)); + operator.replace(ConfigMap.class, configMap); + + await().pollDelay(Duration.ofMillis(150)).untilAsserted(() -> { + assertThat(operator.getReconcilerOfType(DependentFilterTestReconciler.class) + .getNumberOfExecutions()).isEqualTo(1); + }); + } + + DependentFilterTestCustomResource createResource() { + DependentFilterTestCustomResource resource = new DependentFilterTestCustomResource(); + resource.setMetadata(new ObjectMetaBuilder() + .withName(RESOURCE_NAME) + .build()); + resource.setSpec(new DependentFilterTestResourceSpec()); + resource.getSpec().setValue("value1"); + return resource; + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentOperationEventFilterIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentOperationEventFilterIT.java index 4fe7adf954..6dd8164952 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentOperationEventFilterIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentOperationEventFilterIT.java @@ -7,7 +7,7 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.dependentoperationeventfiltering.ConfigMapDependentResource; import io.javaoperatorsdk.operator.sample.dependentoperationeventfiltering.DependentOperationEventFilterCustomResource; import io.javaoperatorsdk.operator.sample.dependentoperationeventfiltering.DependentOperationEventFilterCustomResourceSpec; @@ -23,8 +23,8 @@ class DependentOperationEventFilterIT { public static final String SPEC_VAL_2 = "val2"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(new DependentOperationEventFilterCustomResourceTestReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentPrimaryIndexerIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentPrimaryIndexerIT.java index 50090f5929..44d88aaecf 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentPrimaryIndexerIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentPrimaryIndexerIT.java @@ -1,12 +1,12 @@ package io.javaoperatorsdk.operator; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.primaryindexer.DependentPrimaryIndexerTestReconciler; public class DependentPrimaryIndexerIT extends PrimaryIndexerIT { - protected LocalOperatorExtension buildOperator() { - return LocalOperatorExtension.builder() + protected LocallyRunOperatorExtension buildOperator() { + return LocallyRunOperatorExtension.builder() .withReconciler(new DependentPrimaryIndexerTestReconciler()) .build(); } diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentResourceCrossRefIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentResourceCrossRefIT.java index 1571ccd249..fc4118f909 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentResourceCrossRefIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/DependentResourceCrossRefIT.java @@ -8,7 +8,7 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; import io.fabric8.kubernetes.api.model.Secret; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.dependentresourcecrossref.DependentResourceCrossRefReconciler; import io.javaoperatorsdk.operator.sample.dependentresourcecrossref.DependentResourceCrossRefResource; @@ -21,8 +21,8 @@ class DependentResourceCrossRefIT { public static final int EXECUTION_NUMBER = 50; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(new DependentResourceCrossRefReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ErrorStatusHandlerIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ErrorStatusHandlerIT.java index 8468f81ba8..529b34a7f5 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ErrorStatusHandlerIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ErrorStatusHandlerIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.processing.retry.GenericRetry; import io.javaoperatorsdk.operator.sample.errorstatushandler.ErrorStatusHandlerTestCustomResource; import io.javaoperatorsdk.operator.sample.errorstatushandler.ErrorStatusHandlerTestReconciler; @@ -20,8 +20,8 @@ class ErrorStatusHandlerIT { ErrorStatusHandlerTestReconciler reconciler = new ErrorStatusHandlerTestReconciler(); @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(reconciler, new GenericRetry().setMaxAttempts(MAX_RETRY_ATTEMPTS).withLinearRetry()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceByAnnotationIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceByAnnotationIT.java new file mode 100644 index 0000000000..e77b8f8454 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceByAnnotationIT.java @@ -0,0 +1,57 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; +import java.util.Map; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.eventsourcebyannotation.observedgeneration.EventSourceByAnnotationCustomResource; +import io.javaoperatorsdk.operator.sample.eventsourcebyannotation.observedgeneration.EventSourceByAnnotationReconciler; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +public class EventSourceByAnnotationIT { + + public static final String NAME = "test1"; + + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(EventSourceByAnnotationReconciler.class) + .build(); + + @Test + void registerEventSourceWithAnnotation() { + operator.create(EventSourceByAnnotationCustomResource.class, customResource()); + await().pollDelay(Duration.ofMillis(150)).untilAsserted(() -> { + assertThat(operator.getReconcilerOfType(EventSourceByAnnotationReconciler.class) + .getNumberOfExecution()) + .isEqualTo(2); // first reconcile + config map creation + }); + + var cm = operator.get(ConfigMap.class, NAME); + cm.setData(Map.of("key", "newvalue")); + operator.replace(ConfigMap.class, cm); + + await().pollDelay(Duration.ofMillis(150)).untilAsserted(() -> { + assertThat(operator.getReconcilerOfType(EventSourceByAnnotationReconciler.class) + .getNumberOfExecution()) + .isEqualTo(3); + }); + } + + EventSourceByAnnotationCustomResource customResource() { + EventSourceByAnnotationCustomResource resource = new EventSourceByAnnotationCustomResource(); + resource.setMetadata(new ObjectMetaBuilder() + .withName(NAME) + .build()); + return resource; + } + + + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceIT.java index 38c79d005f..b4ff4fc04d 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/EventSourceIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.event.EventSourceTestCustomReconciler; import io.javaoperatorsdk.operator.sample.event.EventSourceTestCustomResource; import io.javaoperatorsdk.operator.sample.event.EventSourceTestCustomResourceSpec; @@ -17,8 +17,8 @@ class EventSourceIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(EventSourceTestCustomReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(EventSourceTestCustomReconciler.class) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/FilterIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/FilterIT.java new file mode 100644 index 0000000000..2dea399448 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/FilterIT.java @@ -0,0 +1,73 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.filter.FilterTestCustomResource; +import io.javaoperatorsdk.operator.sample.filter.FilterTestReconciler; +import io.javaoperatorsdk.operator.sample.filter.FilterTestResourceSpec; + +import static io.javaoperatorsdk.operator.sample.filter.FilterTestReconciler.CONFIG_MAP_FILTER_VALUE; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +class FilterIT { + + public static final String RESOURCE_NAME = "test1"; + public static final int POLL_DELAY = 150; + + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(FilterTestReconciler.class) + .build(); + + @Test + void filtersControllerResourceUpdate() { + var res = operator.create(FilterTestCustomResource.class, createResource()); + // One for CR create event other for ConfigMap event + await().pollDelay(Duration.ofMillis(POLL_DELAY)) + .untilAsserted(() -> assertThat(operator.getReconcilerOfType(FilterTestReconciler.class) + .getNumberOfExecutions()).isEqualTo(2)); + + res.getSpec().setValue(FilterTestReconciler.CUSTOM_RESOURCE_FILTER_VALUE); + operator.replace(FilterTestCustomResource.class, res); + + // not more reconciliation with the filtered value + await().pollDelay(Duration.ofMillis(POLL_DELAY)) + .untilAsserted(() -> assertThat(operator.getReconcilerOfType(FilterTestReconciler.class) + .getNumberOfExecutions()).isEqualTo(2)); + } + + @Test + void filtersSecondaryResourceUpdate() { + var res = operator.create(FilterTestCustomResource.class, createResource()); + // One for CR create event other for ConfigMap event + await().pollDelay(Duration.ofMillis(POLL_DELAY)) + .untilAsserted(() -> assertThat(operator.getReconcilerOfType(FilterTestReconciler.class) + .getNumberOfExecutions()).isEqualTo(2)); + + res.getSpec().setValue(CONFIG_MAP_FILTER_VALUE); + operator.replace(FilterTestCustomResource.class, res); + + // the CM event filtered out + await().pollDelay(Duration.ofMillis(POLL_DELAY)) + .untilAsserted(() -> assertThat(operator.getReconcilerOfType(FilterTestReconciler.class) + .getNumberOfExecutions()).isEqualTo(3)); + } + + + FilterTestCustomResource createResource() { + FilterTestCustomResource resource = new FilterTestCustomResource(); + resource.setMetadata(new ObjectMetaBuilder() + .withName(RESOURCE_NAME) + .build()); + resource.setSpec(new FilterTestResourceSpec()); + resource.getSpec().setValue("value1"); + return resource; + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/InformerEventSourceIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/InformerEventSourceIT.java index fb8f81172c..8f05b4b124 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/InformerEventSourceIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/InformerEventSourceIT.java @@ -8,11 +8,13 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.informereventsource.InformerEventSourceTestCustomReconciler; import io.javaoperatorsdk.operator.sample.informereventsource.InformerEventSourceTestCustomResource; -import static io.javaoperatorsdk.operator.sample.informereventsource.InformerEventSourceTestCustomReconciler.*; +import static io.javaoperatorsdk.operator.sample.informereventsource.InformerEventSourceTestCustomReconciler.MISSING_CONFIG_MAP; +import static io.javaoperatorsdk.operator.sample.informereventsource.InformerEventSourceTestCustomReconciler.RELATED_RESOURCE_NAME; +import static io.javaoperatorsdk.operator.sample.informereventsource.InformerEventSourceTestCustomReconciler.TARGET_CONFIG_MAP_KEY; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.fail; import static org.awaitility.Awaitility.await; @@ -24,8 +26,8 @@ class InformerEventSourceIT { public static final String UPDATE_STATUS_MESSAGE = "Updated Status"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(new InformerEventSourceTestCustomReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesDependentGarbageCollectionIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesDependentGarbageCollectionIT.java new file mode 100644 index 0000000000..a524089009 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesDependentGarbageCollectionIT.java @@ -0,0 +1,83 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection.DependentGarbageCollectionTestCustomResource; +import io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection.DependentGarbageCollectionTestCustomResourceSpec; +import io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection.DependentGarbageCollectionTestReconciler; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +class KubernetesDependentGarbageCollectionIT { + + public static final String TEST_RESOURCE_NAME = "test1"; + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() + .withReconciler(new DependentGarbageCollectionTestReconciler()) + .build(); + + + @Test + void resourceSecondaryResourceIsGarbageCollected() { + var resource = customResource(); + var createdResources = + operator.create(DependentGarbageCollectionTestCustomResource.class, resource); + + await().untilAsserted(() -> { + ConfigMap configMap = operator.get(ConfigMap.class, TEST_RESOURCE_NAME); + assertThat(configMap).isNotNull(); + }); + + ConfigMap configMap = operator.get(ConfigMap.class, TEST_RESOURCE_NAME); + assertThat(configMap.getMetadata().getOwnerReferences()).hasSize(1); + assertThat(configMap.getMetadata().getOwnerReferences().get(0).getName()) + .isEqualTo(TEST_RESOURCE_NAME); + + operator.delete(DependentGarbageCollectionTestCustomResource.class, createdResources); + + await().atMost(Duration.ofSeconds(30)).untilAsserted(() -> { + ConfigMap cm = operator.get(ConfigMap.class, TEST_RESOURCE_NAME); + assertThat(cm).isNull(); + }); + } + + @Test + void deletesSecondaryResource() { + var resource = customResource(); + var createdResources = + operator.create(DependentGarbageCollectionTestCustomResource.class, resource); + + await().untilAsserted(() -> { + ConfigMap configMap = operator.get(ConfigMap.class, TEST_RESOURCE_NAME); + assertThat(configMap).isNotNull(); + }); + + createdResources.getSpec().setCreateConfigMap(false); + operator.replace(DependentGarbageCollectionTestCustomResource.class, createdResources); + + await().untilAsserted(() -> { + ConfigMap cm = operator.get(ConfigMap.class, TEST_RESOURCE_NAME); + assertThat(cm).isNull(); + }); + } + + DependentGarbageCollectionTestCustomResource customResource() { + DependentGarbageCollectionTestCustomResource resource = + new DependentGarbageCollectionTestCustomResource(); + resource.setMetadata(new ObjectMetaBuilder() + .withName(TEST_RESOURCE_NAME) + .build()); + resource.setSpec(new DependentGarbageCollectionTestCustomResourceSpec()); + resource.getSpec().setCreateConfigMap(true); + return resource; + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesResourceStatusUpdateIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesResourceStatusUpdateIT.java index 81b1fb20b6..10541437ae 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesResourceStatusUpdateIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/KubernetesResourceStatusUpdateIT.java @@ -8,10 +8,16 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.RegisterExtension; -import io.fabric8.kubernetes.api.model.*; +import io.fabric8.kubernetes.api.model.Container; +import io.fabric8.kubernetes.api.model.ContainerPort; +import io.fabric8.kubernetes.api.model.LabelSelector; +import io.fabric8.kubernetes.api.model.ObjectMeta; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.fabric8.kubernetes.api.model.PodSpec; +import io.fabric8.kubernetes.api.model.PodTemplateSpec; import io.fabric8.kubernetes.api.model.apps.Deployment; import io.fabric8.kubernetes.api.model.apps.DeploymentSpec; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.deployment.DeploymentReconciler; import static io.javaoperatorsdk.operator.sample.deployment.DeploymentReconciler.STATUS_MESSAGE; @@ -21,8 +27,8 @@ class KubernetesResourceStatusUpdateIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new DeploymentReconciler()).build(); + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new DeploymentReconciler()).build(); @Test void testReconciliationOfNonCustomResourceAndStatusUpdate() { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MaxIntervalIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MaxIntervalIT.java index 2914a0f2da..767219ca70 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MaxIntervalIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MaxIntervalIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.maxinterval.MaxIntervalTestCustomResource; import io.javaoperatorsdk.operator.sample.maxinterval.MaxIntervalTestReconciler; @@ -15,8 +15,8 @@ class MaxIntervalIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new MaxIntervalTestReconciler()).build(); + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new MaxIntervalTestReconciler()).build(); @Test void reconciliationTriggeredBasedOnMaxInterval() { diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultiVersionCRDIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultiVersionCRDIT.java index 177da1c3fd..36533ae661 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultiVersionCRDIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultiVersionCRDIT.java @@ -7,8 +7,13 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; -import io.javaoperatorsdk.operator.sample.multiversioncrd.*; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.multiversioncrd.MultiVersionCRDTestCustomResource1; +import io.javaoperatorsdk.operator.sample.multiversioncrd.MultiVersionCRDTestCustomResource2; +import io.javaoperatorsdk.operator.sample.multiversioncrd.MultiVersionCRDTestCustomResourceSpec1; +import io.javaoperatorsdk.operator.sample.multiversioncrd.MultiVersionCRDTestCustomResourceSpec2; +import io.javaoperatorsdk.operator.sample.multiversioncrd.MultiVersionCRDTestReconciler1; +import io.javaoperatorsdk.operator.sample.multiversioncrd.MultiVersionCRDTestReconciler2; import static org.assertj.core.api.AssertionsForClassTypes.assertThat; import static org.awaitility.Awaitility.await; @@ -19,8 +24,8 @@ class MultiVersionCRDIT { public static final String CR_V2_NAME = "crv2"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(MultiVersionCRDTestReconciler1.class) .withReconciler(MultiVersionCRDTestReconciler2.class) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleDependentResourceIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleDependentResourceIT.java index 78e422f81e..8dbe73da68 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleDependentResourceIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleDependentResourceIT.java @@ -8,7 +8,7 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.multipledependentresource.MultipleDependentResourceConfigMap; import io.javaoperatorsdk.operator.sample.multipledependentresource.MultipleDependentResourceCustomResource; import io.javaoperatorsdk.operator.sample.multipledependentresource.MultipleDependentResourceReconciler; @@ -20,8 +20,9 @@ class MultipleDependentResourceIT { public static final String TEST_RESOURCE_NAME = "multipledependentresource-testresource"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(MultipleDependentResourceReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() + .withReconciler(MultipleDependentResourceReconciler.class) .waitForNamespaceDeletion(true) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleSecondaryEventSourceIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleSecondaryEventSourceIT.java index 82bcfc0cbd..278547d303 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleSecondaryEventSourceIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/MultipleSecondaryEventSourceIT.java @@ -7,7 +7,7 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.multiplesecondaryeventsource.MultipleSecondaryEventSourceCustomResource; import io.javaoperatorsdk.operator.sample.multiplesecondaryeventsource.MultipleSecondaryEventSourceReconciler; @@ -17,8 +17,9 @@ class MultipleSecondaryEventSourceIT { public static final String TEST_RESOURCE_NAME = "testresource"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(MultipleSecondaryEventSourceReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() + .withReconciler(MultipleSecondaryEventSourceReconciler.class) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ObservedGenerationHandlingIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ObservedGenerationHandlingIT.java index c9fc464a46..edf89112f8 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/ObservedGenerationHandlingIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/ObservedGenerationHandlingIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.observedgeneration.ObservedGenerationTestCustomResource; import io.javaoperatorsdk.operator.sample.observedgeneration.ObservedGenerationTestReconciler; @@ -15,8 +15,8 @@ class ObservedGenerationHandlingIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new ObservedGenerationTestReconciler()) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new ObservedGenerationTestReconciler()) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/OrderedManagedDependentIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/OrderedManagedDependentIT.java index b6a623b052..3292dd3d92 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/OrderedManagedDependentIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/OrderedManagedDependentIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.orderedmanageddependent.ConfigMapDependentResource1; import io.javaoperatorsdk.operator.sample.orderedmanageddependent.ConfigMapDependentResource2; import io.javaoperatorsdk.operator.sample.orderedmanageddependent.OrderedManagedDependentCustomResource; @@ -18,8 +18,9 @@ class OrderedManagedDependentIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new OrderedManagedDependentTestReconciler()) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() + .withReconciler(new OrderedManagedDependentTestReconciler()) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryIndexerIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryIndexerIT.java index 3241551dea..e4bdd14d51 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryIndexerIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryIndexerIT.java @@ -7,7 +7,7 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMeta; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.primaryindexer.AbstractPrimaryIndexerTestReconciler; import io.javaoperatorsdk.operator.sample.primaryindexer.PrimaryIndexerTestCustomResource; import io.javaoperatorsdk.operator.sample.primaryindexer.PrimaryIndexerTestCustomResourceSpec; @@ -23,10 +23,10 @@ class PrimaryIndexerIT { public static final String RESOURCE_NAME2 = "test2"; @RegisterExtension - LocalOperatorExtension operator = buildOperator(); + LocallyRunOperatorExtension operator = buildOperator(); - protected LocalOperatorExtension buildOperator() { - return LocalOperatorExtension.builder().withReconciler(new PrimaryIndexerTestReconciler()) + protected LocallyRunOperatorExtension buildOperator() { + return LocallyRunOperatorExtension.builder().withReconciler(new PrimaryIndexerTestReconciler()) .build(); } diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryToSecondaryIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryToSecondaryIT.java index b54923c6e6..ebb7c3bee5 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryToSecondaryIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/PrimaryToSecondaryIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.primarytosecondary.Cluster; import io.javaoperatorsdk.operator.sample.primarytosecondary.Job; import io.javaoperatorsdk.operator.sample.primarytosecondary.JobReconciler; @@ -21,8 +21,8 @@ class PrimaryToSecondaryIT { public static final int MIN_DELAY = 150; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withAdditionalCustomResourceDefinition(Cluster.class) .withReconciler(new JobReconciler()) .build(); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/RateLimitIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/RateLimitIT.java new file mode 100644 index 0000000000..25509155de --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/RateLimitIT.java @@ -0,0 +1,62 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; +import java.util.stream.IntStream; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.ratelimit.RateLimitCustomResource; +import io.javaoperatorsdk.operator.sample.ratelimit.RateLimitCustomResourceSpec; +import io.javaoperatorsdk.operator.sample.ratelimit.RateLimitReconciler; + +import static io.javaoperatorsdk.operator.sample.ratelimit.RateLimitReconciler.REFRESH_PERIOD; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +class RateLimitIT { + + private final static Logger log = LoggerFactory.getLogger(RateLimitIT.class); + + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() + .withReconciler(new RateLimitReconciler()) + .build(); + + @Test + void rateLimitsExecution() { + var res = operator.create(RateLimitCustomResource.class, createResource()); + IntStream.rangeClosed(1, 5).forEach(i -> { + log.debug("replacing resource version: {}", i); + var resource = createResource(); + resource.getSpec().setNumber(i); + operator.replace(RateLimitCustomResource.class, resource); + }); + await().pollInterval(Duration.ofMillis(100)) + .pollDelay(Duration.ofMillis(REFRESH_PERIOD / 2)) + .untilAsserted(() -> assertThat( + operator.getReconcilerOfType(RateLimitReconciler.class).getNumberOfExecutions()) + .isEqualTo(1)); + + await().pollDelay(Duration.ofMillis(REFRESH_PERIOD)) + .untilAsserted(() -> assertThat( + operator.getReconcilerOfType(RateLimitReconciler.class).getNumberOfExecutions()) + .isEqualTo(2)); + } + + public RateLimitCustomResource createResource() { + RateLimitCustomResource res = new RateLimitCustomResource(); + res.setMetadata(new ObjectMetaBuilder() + .withName("test") + .build()); + res.setSpec(new RateLimitCustomResourceSpec()); + res.getSpec().setNumber(0); + return res; + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryIT.java index cef7a51d1f..9050601378 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.processing.retry.GenericRetry; import io.javaoperatorsdk.operator.sample.retry.RetryTestCustomReconciler; import io.javaoperatorsdk.operator.sample.retry.RetryTestCustomResource; @@ -24,8 +24,8 @@ class RetryIT { public static final int NUMBER_FAILED_EXECUTIONS = 3; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler( new RetryTestCustomReconciler(NUMBER_FAILED_EXECUTIONS), new GenericRetry().setInitialInterval(RETRY_INTERVAL).withLinearRetry() diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryMaxAttemptIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryMaxAttemptIT.java index bbd3d09eb5..fd2fa864f2 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryMaxAttemptIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/RetryMaxAttemptIT.java @@ -3,7 +3,7 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.RegisterExtension; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.processing.retry.GenericRetry; import io.javaoperatorsdk.operator.sample.retry.RetryTestCustomReconciler; import io.javaoperatorsdk.operator.sample.retry.RetryTestCustomResource; @@ -20,8 +20,8 @@ class RetryMaxAttemptIT { RetryTestCustomReconciler reconciler = new RetryTestCustomReconciler(ALL_EXECUTION_TO_FAIL); @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder() + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder() .withReconciler(reconciler, new GenericRetry().setInitialInterval(RETRY_INTERVAL).withLinearRetry() .setMaxAttempts(MAX_RETRY_ATTEMPTS)) diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/StandaloneDependentResourceIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/StandaloneDependentResourceIT.java index 71bcafd569..f1e38ce6eb 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/StandaloneDependentResourceIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/StandaloneDependentResourceIT.java @@ -8,7 +8,7 @@ import io.fabric8.kubernetes.api.model.ObjectMeta; import io.fabric8.kubernetes.api.model.apps.Deployment; import io.javaoperatorsdk.operator.api.config.ConfigurationService; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.standalonedependent.StandaloneDependentTestCustomResource; import io.javaoperatorsdk.operator.sample.standalonedependent.StandaloneDependentTestCustomResourceSpec; import io.javaoperatorsdk.operator.sample.standalonedependent.StandaloneDependentTestReconciler; @@ -16,13 +16,13 @@ import static org.assertj.core.api.Assertions.assertThat; import static org.awaitility.Awaitility.await; -class StandaloneDependentResourceIT { +public class StandaloneDependentResourceIT { public static final String DEPENDENT_TEST_NAME = "dependent-test1"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(new StandaloneDependentTestReconciler()) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(new StandaloneDependentTestReconciler()) .build(); @Test @@ -32,7 +32,8 @@ void dependentResourceManagesDeployment() { customResource.setSpec(new StandaloneDependentTestCustomResourceSpec()); customResource.setMetadata(new ObjectMeta()); customResource.getMetadata().setName(DEPENDENT_TEST_NAME); - var createdCR = operator.create(StandaloneDependentTestCustomResource.class, customResource); + + operator.create(StandaloneDependentTestCustomResource.class, customResource); awaitForDeploymentReadyReplicas(1); assertThat( diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusPatchNotLockingIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusPatchNotLockingIT.java index 7a556d28ff..5cc7e7f065 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusPatchNotLockingIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusPatchNotLockingIT.java @@ -7,7 +7,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.statuspatchnonlocking.StatusPatchLockingCustomResource; import io.javaoperatorsdk.operator.sample.statuspatchnonlocking.StatusPatchLockingCustomResourceSpec; import io.javaoperatorsdk.operator.sample.statuspatchnonlocking.StatusPatchLockingReconciler; @@ -22,8 +22,8 @@ class StatusPatchNotLockingIT { public static final String TEST_RESOURCE_NAME = "test"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(StatusPatchLockingReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(StatusPatchLockingReconciler.class) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusUpdateLockingIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusUpdateLockingIT.java index 99668778e0..fa84469a09 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusUpdateLockingIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/StatusUpdateLockingIT.java @@ -7,7 +7,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.statusupdatelocking.StatusUpdateLockingCustomResource; import io.javaoperatorsdk.operator.sample.statusupdatelocking.StatusUpdateLockingReconciler; @@ -20,8 +20,8 @@ class StatusUpdateLockingIT { public static final String TEST_RESOURCE_NAME = "test"; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(StatusUpdateLockingReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(StatusUpdateLockingReconciler.class) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/SubResourceUpdateIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/SubResourceUpdateIT.java index 34621aaab0..8e5fe0a1dd 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/SubResourceUpdateIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/SubResourceUpdateIT.java @@ -7,7 +7,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.subresource.SubResourceTestCustomReconciler; import io.javaoperatorsdk.operator.sample.subresource.SubResourceTestCustomResource; import io.javaoperatorsdk.operator.sample.subresource.SubResourceTestCustomResourceSpec; @@ -23,8 +23,8 @@ class SubResourceUpdateIT { public static final int EVENT_RECEIVE_WAIT = 200; @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(SubResourceTestCustomReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(SubResourceTestCustomReconciler.class) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/UpdatingResAndSubResIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/UpdatingResAndSubResIT.java index 65e4c56621..b1d82d9b46 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/UpdatingResAndSubResIT.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/UpdatingResAndSubResIT.java @@ -6,7 +6,7 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.doubleupdate.DoubleUpdateTestCustomReconciler; import io.javaoperatorsdk.operator.sample.doubleupdate.DoubleUpdateTestCustomResource; import io.javaoperatorsdk.operator.sample.doubleupdate.DoubleUpdateTestCustomResourceSpec; @@ -18,8 +18,8 @@ class UpdatingResAndSubResIT { @RegisterExtension - LocalOperatorExtension operator = - LocalOperatorExtension.builder().withReconciler(DoubleUpdateTestCustomReconciler.class) + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(DoubleUpdateTestCustomReconciler.class) .build(); @Test diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/WorkflowAllFeatureIT.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/WorkflowAllFeatureIT.java new file mode 100644 index 0000000000..2a8289101f --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/WorkflowAllFeatureIT.java @@ -0,0 +1,125 @@ +package io.javaoperatorsdk.operator; + +import java.time.Duration; +import java.util.HashMap; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.fabric8.kubernetes.api.model.apps.Deployment; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; +import io.javaoperatorsdk.operator.sample.workflowallfeature.WorkflowAllFeatureCustomResource; +import io.javaoperatorsdk.operator.sample.workflowallfeature.WorkflowAllFeatureReconciler; +import io.javaoperatorsdk.operator.sample.workflowallfeature.WorkflowAllFeatureSpec; + +import static io.javaoperatorsdk.operator.sample.workflowallfeature.ConfigMapDependentResource.READY_TO_DELETE_ANNOTATION; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +public class WorkflowAllFeatureIT { + + public static final String RESOURCE_NAME = "test"; + private static final Duration ONE_MINUTE = Duration.ofMinutes(1); + + @RegisterExtension + LocallyRunOperatorExtension operator = + LocallyRunOperatorExtension.builder().withReconciler(WorkflowAllFeatureReconciler.class) + .build(); + + @Test + void configMapNotReconciledUntilDeploymentReady() { + operator.create(WorkflowAllFeatureCustomResource.class, customResource(true)); + await().untilAsserted( + () -> { + assertThat(operator + .getReconcilerOfType(WorkflowAllFeatureReconciler.class) + .getNumberOfReconciliationExecution()) + .isPositive(); + assertThat(operator.get(Deployment.class, RESOURCE_NAME)).isNotNull(); + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNull(); + }); + + await().atMost(ONE_MINUTE).untilAsserted(() -> { + assertThat(operator + .getReconcilerOfType(WorkflowAllFeatureReconciler.class) + .getNumberOfReconciliationExecution()) + .isGreaterThan(1); + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNotNull(); + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME) + .getStatus().getReady()).isTrue(); + }); + + markConfigMapForDelete(); + } + + + @Test + void configMapNotReconciledIfReconcileConditionNotMet() { + var resource = operator.create(WorkflowAllFeatureCustomResource.class, customResource(false)); + + await().atMost(ONE_MINUTE).untilAsserted(() -> { + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNull(); + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME) + .getStatus().getReady()).isTrue(); + }); + + resource.getSpec().setCreateConfigMap(true); + operator.replace(WorkflowAllFeatureCustomResource.class, resource); + + await().untilAsserted(() -> { + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNotNull(); + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME) + .getStatus().getReady()).isTrue(); + }); + } + + + @Test + void configMapNotDeletedUntilNotMarked() { + var resource = operator.create(WorkflowAllFeatureCustomResource.class, customResource(true)); + + await().atMost(ONE_MINUTE).untilAsserted(() -> { + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME).getStatus()) + .isNotNull(); + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME) + .getStatus().getReady()).isTrue(); + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNotNull(); + }); + + operator.delete(WorkflowAllFeatureCustomResource.class, resource); + + await().pollDelay(Duration.ofMillis(300)).untilAsserted(() -> { + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNotNull(); + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME)).isNotNull(); + }); + + markConfigMapForDelete(); + + await().atMost(ONE_MINUTE).untilAsserted(() -> { + assertThat(operator.get(ConfigMap.class, RESOURCE_NAME)).isNull(); + assertThat(operator.get(WorkflowAllFeatureCustomResource.class, RESOURCE_NAME)).isNull(); + }); + } + + private void markConfigMapForDelete() { + var cm = operator.get(ConfigMap.class, RESOURCE_NAME); + if (cm.getMetadata().getAnnotations() == null) { + cm.getMetadata().setAnnotations(new HashMap<>()); + } + cm.getMetadata().getAnnotations().put(READY_TO_DELETE_ANNOTATION, "true"); + operator.replace(ConfigMap.class, cm); + } + + private WorkflowAllFeatureCustomResource customResource(boolean createConfigMap) { + var res = new WorkflowAllFeatureCustomResource(); + res.setMetadata(new ObjectMetaBuilder() + .withName(RESOURCE_NAME) + .build()); + res.setSpec(new WorkflowAllFeatureSpec()); + res.getSpec().setCreateConfigMap(createConfigMap); + return res; + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/cleanermanageddependent/ConfigMapDependentResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/cleanermanageddependent/ConfigMapDependentResource.java index ff1d7f2cc8..9941779784 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/cleanermanageddependent/ConfigMapDependentResource.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/cleanermanageddependent/ConfigMapDependentResource.java @@ -6,10 +6,16 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMeta; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.processing.dependent.Creator; +import io.javaoperatorsdk.operator.processing.dependent.Updater; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; public class ConfigMapDependentResource extends - CRUDKubernetesDependentResource { + KubernetesDependentResource + implements Creator, + Updater, + Deleter { private static final AtomicInteger numberOfCleanupExecutions = new AtomicInteger(0); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/createupdateeventfilter/CreateUpdateEventFilterTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/createupdateeventfilter/CreateUpdateEventFilterTestReconciler.java index 256d861791..988b1466c1 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/createupdateeventfilter/CreateUpdateEventFilterTestReconciler.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/createupdateeventfilter/CreateUpdateEventFilterTestReconciler.java @@ -105,7 +105,8 @@ public Map prepareEventSources( InformerConfiguration.from(ConfigMap.class) .withLabelSelector("integrationtest = " + this.getClass().getSimpleName()) .build(); - informerEventSource = new InformerEventSource<>(informerConfiguration, client); + informerEventSource = + new InformerEventSource<>(informerConfiguration, client); return EventSourceInitializer.nameEventSources(informerEventSource); } diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperReconciler.java new file mode 100644 index 0000000000..ec4a2c86b9 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperReconciler.java @@ -0,0 +1,58 @@ +package io.javaoperatorsdk.operator.sample.dependentannotationsecondarymapper; + +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent; +import io.javaoperatorsdk.operator.processing.dependent.Creator; +import io.javaoperatorsdk.operator.processing.dependent.Updater; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.support.TestExecutionInfoProvider; + +@ControllerConfiguration(dependents = @Dependent( + type = DependentAnnotationSecondaryMapperReconciler.ConfigMapDependentResource.class)) +public class DependentAnnotationSecondaryMapperReconciler + implements Reconciler, TestExecutionInfoProvider { + + private final AtomicInteger numberOfExecutions = new AtomicInteger(0); + + @Override + public UpdateControl reconcile( + DependentAnnotationSecondaryMapperResource resource, + Context context) { + numberOfExecutions.addAndGet(1); + return UpdateControl.noUpdate(); + } + + public int getNumberOfExecutions() { + return numberOfExecutions.get(); + } + + public static class ConfigMapDependentResource extends + KubernetesDependentResource + implements Creator, + Updater, + Deleter { + + public ConfigMapDependentResource() { + super(ConfigMap.class); + } + + @Override + protected ConfigMap desired(DependentAnnotationSecondaryMapperResource primary, + Context context) { + ConfigMap configMap = new ConfigMap(); + configMap.setMetadata(new ObjectMetaBuilder() + .withName(primary.getMetadata().getName()) + .withNamespace(primary.getMetadata().getNamespace()) + .build()); + configMap.setData(Map.of("data", primary.getMetadata().getName())); + return configMap; + } + } + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperResource.java new file mode 100644 index 0000000000..22ff6256ae --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperResource.java @@ -0,0 +1,17 @@ +package io.javaoperatorsdk.operator.sample.dependentannotationsecondarymapper; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.Kind; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@Kind("MaxIntervalTestCustomResource") +@ShortNames("mit") +public class DependentAnnotationSecondaryMapperResource + extends CustomResource + implements Namespaced { +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperResourceStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperResourceStatus.java new file mode 100644 index 0000000000..33ea00e819 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentannotationsecondarymapper/DependentAnnotationSecondaryMapperResourceStatus.java @@ -0,0 +1,5 @@ +package io.javaoperatorsdk.operator.sample.dependentannotationsecondarymapper; + +public class DependentAnnotationSecondaryMapperResourceStatus { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestCustomResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestCustomResource.java new file mode 100644 index 0000000000..0930c29774 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestCustomResource.java @@ -0,0 +1,19 @@ +package io.javaoperatorsdk.operator.sample.dependentfilter; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@ShortNames("dft") +public class DependentFilterTestCustomResource + extends CustomResource + implements Namespaced { + + public String getConfigMapName(int id) { + return "configmap" + id; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestReconciler.java new file mode 100644 index 0000000000..114491d9b9 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestReconciler.java @@ -0,0 +1,29 @@ +package io.javaoperatorsdk.operator.sample.dependentfilter; + +import java.util.concurrent.atomic.AtomicInteger; + +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent; + +@ControllerConfiguration(onUpdateFilter = UpdateFilter.class, + dependents = {@Dependent(type = FilteredDependentConfigMap.class)}) +public class DependentFilterTestReconciler + implements Reconciler { + + public static final String CONFIG_MAP_FILTER_VALUE = "config_map_skip_this"; + public static final String CM_VALUE_KEY = "value"; + + private final AtomicInteger numberOfExecutions = new AtomicInteger(0); + + @Override + public UpdateControl reconcile( + DependentFilterTestCustomResource resource, + Context context) { + numberOfExecutions.addAndGet(1); + return UpdateControl.noUpdate(); + } + + public int getNumberOfExecutions() { + return numberOfExecutions.get(); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestResourceSpec.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestResourceSpec.java new file mode 100644 index 0000000000..cf6b02e936 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestResourceSpec.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.sample.dependentfilter; + +public class DependentFilterTestResourceSpec { + + private String value; + + public String getValue() { + return value; + } + + public DependentFilterTestResourceSpec setValue(String value) { + this.value = value; + return this; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestResourceStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestResourceStatus.java new file mode 100644 index 0000000000..99e8d54514 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/DependentFilterTestResourceStatus.java @@ -0,0 +1,5 @@ +package io.javaoperatorsdk.operator.sample.dependentfilter; + +public class DependentFilterTestResourceStatus { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/FilteredDependentConfigMap.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/FilteredDependentConfigMap.java new file mode 100644 index 0000000000..ee4bd4cde7 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/FilteredDependentConfigMap.java @@ -0,0 +1,32 @@ +package io.javaoperatorsdk.operator.sample.dependentfilter; + +import java.util.Map; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; + +import static io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestReconciler.CM_VALUE_KEY; + +@KubernetesDependent(onUpdateFilter = UpdateFilter.class) +public class FilteredDependentConfigMap + extends CRUDKubernetesDependentResource { + + public FilteredDependentConfigMap() { + super(ConfigMap.class); + } + + @Override + protected ConfigMap desired(DependentFilterTestCustomResource primary, + Context context) { + ConfigMap configMap = new ConfigMap(); + configMap.setMetadata(new ObjectMetaBuilder() + .withName(primary.getMetadata().getName()) + .withNamespace(primary.getMetadata().getNamespace()) + .build()); + configMap.setData(Map.of(CM_VALUE_KEY, primary.getSpec().getValue())); + return configMap; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/UpdateFilter.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/UpdateFilter.java new file mode 100644 index 0000000000..60dc00ce8e --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentfilter/UpdateFilter.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.sample.dependentfilter; + +import java.util.function.BiPredicate; + +import io.fabric8.kubernetes.api.model.ConfigMap; + +import static io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestReconciler.CM_VALUE_KEY; +import static io.javaoperatorsdk.operator.sample.dependentfilter.DependentFilterTestReconciler.CONFIG_MAP_FILTER_VALUE; + +public class UpdateFilter + implements BiPredicate { + @Override + public boolean test(ConfigMap resource, ConfigMap oldResource) { + return !resource.getData().get(CM_VALUE_KEY).equals(CONFIG_MAP_FILTER_VALUE); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentoperationeventfiltering/ConfigMapDependentResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentoperationeventfiltering/ConfigMapDependentResource.java index 17dbe20ddf..34fbd21c50 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentoperationeventfiltering/ConfigMapDependentResource.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentoperationeventfiltering/ConfigMapDependentResource.java @@ -5,10 +5,10 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ObjectMeta; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; public class ConfigMapDependentResource extends - CRUKubernetesDependentResource { + CRUDKubernetesDependentResource { public static final String KEY = "key1"; diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentresourcecrossref/DependentResourceCrossRefReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentresourcecrossref/DependentResourceCrossRefReconciler.java index 96e3029548..bb319741b3 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentresourcecrossref/DependentResourceCrossRefReconciler.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/dependentresourcecrossref/DependentResourceCrossRefReconciler.java @@ -10,15 +10,20 @@ import io.fabric8.kubernetes.api.model.Secret; import io.javaoperatorsdk.operator.api.reconciler.*; import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; + +import static io.javaoperatorsdk.operator.sample.dependentresourcecrossref.DependentResourceCrossRefReconciler.SECRET_NAME; @ControllerConfiguration(dependents = { - @Dependent(type = DependentResourceCrossRefReconciler.SecretDependentResource.class), - @Dependent(type = DependentResourceCrossRefReconciler.ConfigMapDependentResource.class)}) + @Dependent(name = SECRET_NAME, + type = DependentResourceCrossRefReconciler.SecretDependentResource.class), + @Dependent(type = DependentResourceCrossRefReconciler.ConfigMapDependentResource.class, + dependsOn = SECRET_NAME)}) public class DependentResourceCrossRefReconciler implements Reconciler, ErrorStatusHandler { + public static final String SECRET_NAME = "secret"; private final AtomicInteger numberOfExecutions = new AtomicInteger(0); private volatile boolean errorHappened = false; @@ -47,7 +52,7 @@ public boolean isErrorHappened() { } public static class SecretDependentResource extends - CRUKubernetesDependentResource { + CRUDKubernetesDependentResource { public SecretDependentResource() { super(Secret.class); @@ -67,8 +72,7 @@ protected Secret desired(DependentResourceCrossRefResource primary, } public static class ConfigMapDependentResource extends - CRUKubernetesDependentResource { - + CRUDKubernetesDependentResource { public ConfigMapDependentResource() { super(ConfigMap.class); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationCustomResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationCustomResource.java new file mode 100644 index 0000000000..15802a8c00 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationCustomResource.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.sample.eventsourcebyannotation.observedgeneration; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@ShortNames("esa") +public class EventSourceByAnnotationCustomResource + extends CustomResource + implements Namespaced { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationCustomResourceStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationCustomResourceStatus.java new file mode 100644 index 0000000000..fd3474eada --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationCustomResourceStatus.java @@ -0,0 +1,7 @@ +package io.javaoperatorsdk.operator.sample.eventsourcebyannotation.observedgeneration; + +import io.javaoperatorsdk.operator.api.ObservedGenerationAwareStatus; + +public class EventSourceByAnnotationCustomResourceStatus extends ObservedGenerationAwareStatus { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationReconciler.java new file mode 100644 index 0000000000..6df5eb11e9 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/eventsourcebyannotation/observedgeneration/EventSourceByAnnotationReconciler.java @@ -0,0 +1,59 @@ +package io.javaoperatorsdk.operator.sample.eventsourcebyannotation.observedgeneration; + +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.junit.KubernetesClientAware; + +@ControllerConfiguration( + eventSources = @EventSources(informers = @Informer(resourceType = ConfigMap.class, + followNamespaceChanges = true))) +public class EventSourceByAnnotationReconciler + implements Reconciler, KubernetesClientAware { + + private KubernetesClient kubernetesClient; + private final AtomicInteger numberOfExecution = new AtomicInteger(); + + + @Override + public UpdateControl reconcile( + EventSourceByAnnotationCustomResource resource, + Context context) { + numberOfExecution.addAndGet(1); + if (kubernetesClient.configMaps().inNamespace(resource.getMetadata().getNamespace()) + .withName(resource.getMetadata().getName()).get() == null) { + kubernetesClient.configMaps().inNamespace(resource.getMetadata().getNamespace()) + .createOrReplace(configMap(resource)); + } + return UpdateControl.noUpdate(); + } + + private ConfigMap configMap(EventSourceByAnnotationCustomResource primary) { + ConfigMap configMap = new ConfigMap(); + configMap.setMetadata(new ObjectMetaBuilder().withName(primary.getMetadata() + .getName()) + .withNamespace(primary.getMetadata().getNamespace()) + .build()); + configMap.addOwnerReference(primary); + configMap.setData(Map.of("key", "value")); + return configMap; + } + + public int getNumberOfExecution() { + return numberOfExecution.get(); + } + + @Override + public KubernetesClient getKubernetesClient() { + return kubernetesClient; + } + + @Override + public void setKubernetesClient(KubernetesClient kubernetesClient) { + this.kubernetesClient = kubernetesClient; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestCustomResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestCustomResource.java new file mode 100644 index 0000000000..3314861ee5 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestCustomResource.java @@ -0,0 +1,19 @@ +package io.javaoperatorsdk.operator.sample.filter; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@ShortNames("ftc") +public class FilterTestCustomResource + extends CustomResource + implements Namespaced { + + public String getConfigMapName(int id) { + return "configmap" + id; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestReconciler.java new file mode 100644 index 0000000000..7189b10c7f --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestReconciler.java @@ -0,0 +1,77 @@ +package io.javaoperatorsdk.operator.sample.filter; + +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.api.config.informer.InformerConfiguration; +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.junit.KubernetesClientAware; +import io.javaoperatorsdk.operator.processing.event.source.EventSource; +import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource; + +@ControllerConfiguration(onUpdateFilter = UpdateFilter.class) +public class FilterTestReconciler + implements Reconciler, + EventSourceInitializer, + KubernetesClientAware { + + public static final String CONFIG_MAP_FILTER_VALUE = "config_map_skip_this"; + public static final String CUSTOM_RESOURCE_FILTER_VALUE = "custom_resource_skip_this"; + + public static final String CM_VALUE_KEY = "value"; + private final AtomicInteger numberOfExecutions = new AtomicInteger(0); + private KubernetesClient client; + + @Override + public UpdateControl reconcile( + FilterTestCustomResource resource, + Context context) { + numberOfExecutions.addAndGet(1); + client.configMaps().inNamespace(resource.getMetadata().getNamespace()) + .createOrReplace(createConfigMap(resource)); + return UpdateControl.noUpdate(); + } + + private ConfigMap createConfigMap(FilterTestCustomResource resource) { + ConfigMap configMap = new ConfigMap(); + configMap.setMetadata(new ObjectMetaBuilder() + .withName(resource.getMetadata().getName()) + .withNamespace(resource.getMetadata().getNamespace()) + .build()); + configMap.addOwnerReference(resource); + configMap.setData(Map.of(CM_VALUE_KEY, resource.getSpec().getValue())); + return configMap; + } + + + public int getNumberOfExecutions() { + return numberOfExecutions.get(); + } + + @Override + public Map prepareEventSources( + EventSourceContext context) { + + InformerEventSource configMapES = + new InformerEventSource<>(InformerConfiguration + .from(ConfigMap.class, context) + .withOnUpdateFilter((newCM, oldCM) -> !newCM.getData().get(CM_VALUE_KEY) + .equals(CONFIG_MAP_FILTER_VALUE)) + .build(), context); + + return EventSourceInitializer.nameEventSources(configMapES); + } + + @Override + public void setKubernetesClient(KubernetesClient kubernetesClient) { + this.client = kubernetesClient; + } + + @Override + public KubernetesClient getKubernetesClient() { + return client; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestResourceSpec.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestResourceSpec.java new file mode 100644 index 0000000000..044b0ea883 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestResourceSpec.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.sample.filter; + +public class FilterTestResourceSpec { + + private String value; + + public String getValue() { + return value; + } + + public FilterTestResourceSpec setValue(String value) { + this.value = value; + return this; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestResourceStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestResourceStatus.java new file mode 100644 index 0000000000..cf0d24aa2c --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/FilterTestResourceStatus.java @@ -0,0 +1,5 @@ +package io.javaoperatorsdk.operator.sample.filter; + +public class FilterTestResourceStatus { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/UpdateFilter.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/UpdateFilter.java new file mode 100644 index 0000000000..6b8a91f2c7 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/filter/UpdateFilter.java @@ -0,0 +1,13 @@ +package io.javaoperatorsdk.operator.sample.filter; + +import java.util.function.BiPredicate; + +import static io.javaoperatorsdk.operator.sample.filter.FilterTestReconciler.CUSTOM_RESOURCE_FILTER_VALUE; + +public class UpdateFilter + implements BiPredicate { + @Override + public boolean test(FilterTestCustomResource resource, FilterTestCustomResource oldResource) { + return !resource.getSpec().getValue().equals(CUSTOM_RESOURCE_FILTER_VALUE); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResource.java new file mode 100644 index 0000000000..5f1e5a0435 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResource.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@ShortNames("dgc") +public class DependentGarbageCollectionTestCustomResource + extends + CustomResource + implements Namespaced { +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResourceSpec.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResourceSpec.java new file mode 100644 index 0000000000..9c29ebbacc --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResourceSpec.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection; + +public class DependentGarbageCollectionTestCustomResourceSpec { + + private boolean createConfigMap; + + public boolean isCreateConfigMap() { + return createConfigMap; + } + + public DependentGarbageCollectionTestCustomResourceSpec setCreateConfigMap( + boolean createConfigMap) { + this.createConfigMap = createConfigMap; + return this; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResourceStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResourceStatus.java new file mode 100644 index 0000000000..79f67c017e --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestCustomResourceStatus.java @@ -0,0 +1,5 @@ +package io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection; + +public class DependentGarbageCollectionTestCustomResourceStatus { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestReconciler.java new file mode 100644 index 0000000000..e9b947a83b --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/kubernetesdependentgarbagecollection/DependentGarbageCollectionTestReconciler.java @@ -0,0 +1,102 @@ +package io.javaoperatorsdk.operator.sample.kubernetesdependentgarbagecollection; + +import java.util.Map; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.KubernetesClientException; +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected; +import io.javaoperatorsdk.operator.junit.KubernetesClientAware; +import io.javaoperatorsdk.operator.processing.dependent.Creator; +import io.javaoperatorsdk.operator.processing.dependent.Updater; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.event.source.EventSource; + +@ControllerConfiguration +public class DependentGarbageCollectionTestReconciler + implements Reconciler, + EventSourceInitializer, + KubernetesClientAware, ErrorStatusHandler { + + private KubernetesClient kubernetesClient; + private volatile boolean errorOccurred = false; + + ConfigMapDependentResource configMapDependent; + + public DependentGarbageCollectionTestReconciler() { + configMapDependent = new ConfigMapDependentResource(); + } + + @Override + public Map prepareEventSources( + EventSourceContext context) { + return EventSourceInitializer + .nameEventSources(configMapDependent.initEventSource(context)); + } + + @Override + public UpdateControl reconcile( + DependentGarbageCollectionTestCustomResource primary, + Context context) { + + if (primary.getSpec().isCreateConfigMap()) { + configMapDependent.reconcile(primary, context); + } else { + configMapDependent.delete(primary, context); + } + + return UpdateControl.noUpdate(); + } + + @Override + public void setKubernetesClient(KubernetesClient kubernetesClient) { + this.kubernetesClient = kubernetesClient; + configMapDependent.setKubernetesClient(kubernetesClient); + } + + @Override + public KubernetesClient getKubernetesClient() { + return this.kubernetesClient; + } + + @Override + public ErrorStatusUpdateControl updateErrorStatus( + DependentGarbageCollectionTestCustomResource resource, + Context context, Exception e) { + // this can happen when a namespace is terminated in test + if (e instanceof KubernetesClientException) { + return ErrorStatusUpdateControl.noStatusUpdate(); + } + errorOccurred = true; + return ErrorStatusUpdateControl.noStatusUpdate(); + } + + public boolean isErrorOccurred() { + return errorOccurred; + } + + private static class ConfigMapDependentResource extends + KubernetesDependentResource + implements Creator, + Updater, + GarbageCollected { + + public ConfigMapDependentResource() { + super(ConfigMap.class); + } + + @Override + protected ConfigMap desired(DependentGarbageCollectionTestCustomResource primary, + Context context) { + ConfigMap configMap = new ConfigMap(); + configMap.setMetadata(new ObjectMetaBuilder() + .withName(primary.getMetadata().getName()) + .withNamespace(primary.getMetadata().getNamespace()) + .build()); + configMap.setData(Map.of("key", "data")); + return configMap; + } + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/multipledependentresource/MultipleDependentResourceConfigMap.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/multipledependentresource/MultipleDependentResourceConfigMap.java index 0f4a3a861e..4cdc2e457d 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/multipledependentresource/MultipleDependentResourceConfigMap.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/multipledependentresource/MultipleDependentResourceConfigMap.java @@ -6,12 +6,12 @@ import io.fabric8.kubernetes.api.model.ConfigMap; import io.fabric8.kubernetes.api.model.ConfigMapBuilder; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; @KubernetesDependent public class MultipleDependentResourceConfigMap - extends CRUKubernetesDependentResource { + extends CRUDKubernetesDependentResource { public static final String DATA_KEY = "key"; private final int value; diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource1.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource1.java index 19fd28b631..14530cf17e 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource1.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource1.java @@ -7,12 +7,12 @@ import io.fabric8.kubernetes.api.model.ObjectMeta; import io.javaoperatorsdk.operator.api.reconciler.Context; import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; @KubernetesDependent(labelSelector = "dependent = cm1") public class ConfigMapDependentResource1 extends - CRUKubernetesDependentResource { + CRUDKubernetesDependentResource { public ConfigMapDependentResource1() { super(ConfigMap.class); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource2.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource2.java index 2bffdfa8c1..35ae69586e 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource2.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/ConfigMapDependentResource2.java @@ -7,12 +7,12 @@ import io.fabric8.kubernetes.api.model.ObjectMeta; import io.javaoperatorsdk.operator.api.reconciler.Context; import io.javaoperatorsdk.operator.api.reconciler.dependent.ReconcileResult; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; @KubernetesDependent(labelSelector = "dependent = cm2") public class ConfigMapDependentResource2 extends - CRUKubernetesDependentResource { + CRUDKubernetesDependentResource { public ConfigMapDependentResource2() { super(ConfigMap.class); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/OrderedManagedDependentTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/OrderedManagedDependentTestReconciler.java index 8ca8f0651d..f7172ca44d 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/OrderedManagedDependentTestReconciler.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/orderedmanageddependent/OrderedManagedDependentTestReconciler.java @@ -16,8 +16,8 @@ @ControllerConfiguration( namespaces = Constants.WATCH_CURRENT_NAMESPACE, dependents = { - @Dependent(type = ConfigMapDependentResource1.class), - @Dependent(type = ConfigMapDependentResource2.class) + @Dependent(type = ConfigMapDependentResource1.class, name = "cm1"), + @Dependent(type = ConfigMapDependentResource2.class, dependsOn = "cm1") }) public class OrderedManagedDependentTestReconciler implements Reconciler, diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/primaryindexer/DependentPrimaryIndexerTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/primaryindexer/DependentPrimaryIndexerTestReconciler.java index 59b6594846..8c333dbc9b 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/primaryindexer/DependentPrimaryIndexerTestReconciler.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/primaryindexer/DependentPrimaryIndexerTestReconciler.java @@ -30,8 +30,8 @@ public ReadOnlyConfigMapDependent() { } @Override - public Set toPrimaryResourceIDs(ConfigMap dependentResource) { - return cache.byIndex(CONFIG_MAP_RELATION_INDEXER, dependentResource.getMetadata().getName()) + public Set toPrimaryResourceIDs(ConfigMap secondaryResource) { + return cache.byIndex(CONFIG_MAP_RELATION_INDEXER, secondaryResource.getMetadata().getName()) .stream() .map(ResourceID::fromResource) .collect(Collectors.toSet()); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResource.java new file mode 100644 index 0000000000..60456e6a4d --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResource.java @@ -0,0 +1,16 @@ +package io.javaoperatorsdk.operator.sample.ratelimit; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@ShortNames("rlc") +public class RateLimitCustomResource + extends CustomResource + implements Namespaced { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResourceSpec.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResourceSpec.java new file mode 100644 index 0000000000..7dbee7f75a --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResourceSpec.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.sample.ratelimit; + +public class RateLimitCustomResourceSpec { + + private int number; + + public int getNumber() { + return number; + } + + public RateLimitCustomResourceSpec setNumber(int number) { + this.number = number; + return this; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResourceStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResourceStatus.java new file mode 100644 index 0000000000..087408fc16 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitCustomResourceStatus.java @@ -0,0 +1,7 @@ +package io.javaoperatorsdk.operator.sample.ratelimit; + +import io.javaoperatorsdk.operator.api.ObservedGenerationAwareStatus; + +public class RateLimitCustomResourceStatus extends ObservedGenerationAwareStatus { + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitReconciler.java new file mode 100644 index 0000000000..b4a77a8fc0 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/ratelimit/RateLimitReconciler.java @@ -0,0 +1,30 @@ +package io.javaoperatorsdk.operator.sample.ratelimit; + +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; + +import io.javaoperatorsdk.operator.api.reconciler.*; + +@ControllerConfiguration(rateLimit = @RateLimit(limitForPeriod = 1, + refreshPeriod = RateLimitReconciler.REFRESH_PERIOD, + refreshPeriodTimeUnit = TimeUnit.MILLISECONDS)) +public class RateLimitReconciler + implements Reconciler { + + public static final int REFRESH_PERIOD = 3000; + + private final AtomicInteger numberOfExecutions = new AtomicInteger(0); + + @Override + public UpdateControl reconcile( + RateLimitCustomResource resource, + Context context) { + + numberOfExecutions.addAndGet(1); + return UpdateControl.noUpdate(); + } + + public int getNumberOfExecutions() { + return numberOfExecutions.get(); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/standalonedependent/StandaloneDependentTestReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/standalonedependent/StandaloneDependentTestReconciler.java index 2ecaa5cc27..af6b2d7e25 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/standalonedependent/StandaloneDependentTestReconciler.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/standalonedependent/StandaloneDependentTestReconciler.java @@ -7,6 +7,7 @@ import io.fabric8.kubernetes.client.KubernetesClient; import io.fabric8.kubernetes.client.KubernetesClientException; import io.javaoperatorsdk.operator.ReconcilerUtils; +import io.javaoperatorsdk.operator.StandaloneDependentResourceIT; import io.javaoperatorsdk.operator.api.reconciler.Context; import io.javaoperatorsdk.operator.api.reconciler.ControllerConfiguration; import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusHandler; @@ -16,9 +17,7 @@ import io.javaoperatorsdk.operator.api.reconciler.Reconciler; import io.javaoperatorsdk.operator.api.reconciler.UpdateControl; import io.javaoperatorsdk.operator.junit.KubernetesClientAware; -import io.javaoperatorsdk.operator.processing.dependent.Creator; -import io.javaoperatorsdk.operator.processing.dependent.Updater; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.event.source.EventSource; @ControllerConfiguration @@ -88,9 +87,7 @@ public boolean isErrorOccurred() { } private static class DeploymentDependentResource extends - KubernetesDependentResource - implements Creator, - Updater { + CRUDKubernetesDependentResource { public DeploymentDependentResource() { super(Deployment.class); @@ -100,7 +97,8 @@ public DeploymentDependentResource() { protected Deployment desired(StandaloneDependentTestCustomResource primary, Context context) { Deployment deployment = - ReconcilerUtils.loadYaml(Deployment.class, getClass(), "nginx-deployment.yaml"); + ReconcilerUtils.loadYaml(Deployment.class, StandaloneDependentResourceIT.class, + "nginx-deployment.yaml"); deployment.getMetadata().setName(primary.getMetadata().getName()); deployment.getSpec().setReplicas(primary.getSpec().getReplicaCount()); deployment.getMetadata().setNamespace(primary.getMetadata().getNamespace()); diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapDeletePostCondition.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapDeletePostCondition.java new file mode 100644 index 0000000000..c5c908dfe6 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapDeletePostCondition.java @@ -0,0 +1,24 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; + +public class ConfigMapDeletePostCondition + implements Condition { + + private static final Logger log = LoggerFactory.getLogger(ConfigMapDeletePostCondition.class); + + @Override + public boolean isMet( + DependentResource dependentResource, + WorkflowAllFeatureCustomResource primary, Context context) { + var configMapDeleted = dependentResource.getSecondaryResource(primary).isEmpty(); + log.debug("Config Map Deleted: {}", configMapDeleted); + return configMapDeleted; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapDependentResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapDependentResource.java new file mode 100644 index 0000000000..0620d6a753 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapDependentResource.java @@ -0,0 +1,59 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import java.util.Map; +import java.util.Optional; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Deleter; +import io.javaoperatorsdk.operator.processing.dependent.Creator; +import io.javaoperatorsdk.operator.processing.dependent.Updater; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.event.ResourceID; + +public class ConfigMapDependentResource + extends KubernetesDependentResource + implements Creator, + Updater, + Deleter { + + public static final String READY_TO_DELETE_ANNOTATION = "ready-to-delete"; + + private static final Logger log = LoggerFactory.getLogger(ConfigMapDependentResource.class); + + public ConfigMapDependentResource() { + super(ConfigMap.class); + } + + @Override + protected ConfigMap desired(WorkflowAllFeatureCustomResource primary, + Context context) { + ConfigMap configMap = new ConfigMap(); + configMap.setMetadata(new ObjectMetaBuilder() + .withName(primary.getMetadata().getName()) + .withNamespace(primary.getMetadata().getNamespace()) + .build()); + configMap.setData(Map.of("key", "data")); + return configMap; + } + + @Override + public void delete(WorkflowAllFeatureCustomResource primary, + Context context) { + Optional optionalConfigMap = context.getSecondaryResource(ConfigMap.class); + if (optionalConfigMap.isEmpty()) { + log.debug("Config Map not found for primary: {}", ResourceID.fromResource(primary)); + return; + } + optionalConfigMap.ifPresent((configMap -> { + if (configMap.getMetadata().getAnnotations() != null + && configMap.getMetadata().getAnnotations().get(READY_TO_DELETE_ANNOTATION) != null) { + client.resource(configMap).delete(); + } + })); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapReconcileCondition.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapReconcileCondition.java new file mode 100644 index 0000000000..65409efc36 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/ConfigMapReconcileCondition.java @@ -0,0 +1,17 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; + +public class ConfigMapReconcileCondition + implements Condition { + + @Override + public boolean isMet( + DependentResource dependentResource, + WorkflowAllFeatureCustomResource primary, Context context) { + return primary.getSpec().isCreateConfigMap(); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/DeploymentDependentResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/DeploymentDependentResource.java new file mode 100644 index 0000000000..61cf18f57b --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/DeploymentDependentResource.java @@ -0,0 +1,29 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import io.fabric8.kubernetes.api.model.apps.Deployment; +import io.javaoperatorsdk.operator.ReconcilerUtils; +import io.javaoperatorsdk.operator.WorkflowAllFeatureIT; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDNoGCKubernetesDependentResource; + +public class DeploymentDependentResource extends + CRUDNoGCKubernetesDependentResource { + + public DeploymentDependentResource() { + super(Deployment.class); + } + + @Override + protected Deployment desired(WorkflowAllFeatureCustomResource primary, + Context context) { + Deployment deployment = + ReconcilerUtils.loadYaml(Deployment.class, WorkflowAllFeatureIT.class, + "nginx-deployment.yaml"); + deployment.getMetadata().setName(primary.getMetadata().getName()); + deployment.getSpec().setReplicas(2); + deployment.getMetadata().setNamespace(primary.getMetadata().getNamespace()); + return deployment; + } +} + + diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/DeploymentReadyCondition.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/DeploymentReadyCondition.java new file mode 100644 index 0000000000..c8646f6ea5 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/DeploymentReadyCondition.java @@ -0,0 +1,20 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import io.fabric8.kubernetes.api.model.apps.Deployment; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; + +public class DeploymentReadyCondition + implements Condition { + @Override + public boolean isMet( + DependentResource dependentResource, + WorkflowAllFeatureCustomResource primary, Context context) { + + var deployment = dependentResource.getSecondaryResource(primary).orElseThrow(); + var readyReplicas = deployment.getStatus().getReadyReplicas(); + + return readyReplicas != null && deployment.getSpec().getReplicas().equals(readyReplicas); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureCustomResource.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureCustomResource.java new file mode 100644 index 0000000000..ee764f4681 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureCustomResource.java @@ -0,0 +1,18 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import io.fabric8.kubernetes.api.model.Namespaced; +import io.fabric8.kubernetes.client.CustomResource; +import io.fabric8.kubernetes.model.annotation.Group; +import io.fabric8.kubernetes.model.annotation.ShortNames; +import io.fabric8.kubernetes.model.annotation.Version; + +@Group("sample.javaoperatorsdk") +@Version("v1") +@ShortNames("waf") +public class WorkflowAllFeatureCustomResource + extends CustomResource + implements Namespaced { + + + +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureReconciler.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureReconciler.java new file mode 100644 index 0000000000..2c25d13924 --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureReconciler.java @@ -0,0 +1,57 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +import java.util.concurrent.atomic.AtomicInteger; + +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent; + +import static io.javaoperatorsdk.operator.sample.workflowallfeature.WorkflowAllFeatureReconciler.DEPLOYMENT_NAME; + +@ControllerConfiguration(dependents = { + @Dependent(name = DEPLOYMENT_NAME, type = DeploymentDependentResource.class, + readyPostcondition = DeploymentReadyCondition.class), + @Dependent(type = ConfigMapDependentResource.class, + reconcilePrecondition = ConfigMapReconcileCondition.class, + deletePostcondition = ConfigMapDeletePostCondition.class, + dependsOn = DEPLOYMENT_NAME) +}) +public class WorkflowAllFeatureReconciler + implements Reconciler, + Cleaner { + + public static final String DEPLOYMENT_NAME = "deployment"; + + private final AtomicInteger numberOfReconciliationExecution = new AtomicInteger(0); + private final AtomicInteger numberOfCleanupExecution = new AtomicInteger(0); + + @Override + public UpdateControl reconcile( + WorkflowAllFeatureCustomResource resource, + Context context) { + numberOfReconciliationExecution.addAndGet(1); + if (resource.getStatus() == null) { + resource.setStatus(new WorkflowAllFeatureStatus()); + } + resource.getStatus() + .setReady( + context.managedDependentResourceContext() + .getWorkflowReconcileResult().orElseThrow() + .allDependentResourcesReady()); + return UpdateControl.patchStatus(resource); + } + + public int getNumberOfReconciliationExecution() { + return numberOfReconciliationExecution.get(); + } + + public int getNumberOfCleanupExecution() { + return numberOfCleanupExecution.get(); + } + + @Override + public DeleteControl cleanup(WorkflowAllFeatureCustomResource resource, + Context context) { + numberOfCleanupExecution.addAndGet(1); + return DeleteControl.defaultDelete(); + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureSpec.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureSpec.java new file mode 100644 index 0000000000..6d5cfd7a5a --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureSpec.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +public class WorkflowAllFeatureSpec { + + private boolean createConfigMap = false; + + public boolean isCreateConfigMap() { + return createConfigMap; + } + + public WorkflowAllFeatureSpec setCreateConfigMap(boolean createConfigMap) { + this.createConfigMap = createConfigMap; + return this; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureStatus.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureStatus.java new file mode 100644 index 0000000000..11d0798fca --- /dev/null +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/sample/workflowallfeature/WorkflowAllFeatureStatus.java @@ -0,0 +1,15 @@ +package io.javaoperatorsdk.operator.sample.workflowallfeature; + +public class WorkflowAllFeatureStatus { + + private Boolean ready; + + public Boolean getReady() { + return ready; + } + + public WorkflowAllFeatureStatus setReady(Boolean ready) { + this.ready = ready; + return this; + } +} diff --git a/operator-framework/src/test/java/io/javaoperatorsdk/operator/support/TestUtils.java b/operator-framework/src/test/java/io/javaoperatorsdk/operator/support/TestUtils.java index 1ba0007643..ea3a72043b 100644 --- a/operator-framework/src/test/java/io/javaoperatorsdk/operator/support/TestUtils.java +++ b/operator-framework/src/test/java/io/javaoperatorsdk/operator/support/TestUtils.java @@ -4,7 +4,7 @@ import java.util.UUID; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.simple.TestCustomResource; import io.javaoperatorsdk.operator.sample.simple.TestCustomResourceSpec; @@ -56,7 +56,7 @@ public static void waitXms(int x) { } } - public static int getNumberOfExecutions(LocalOperatorExtension extension) { + public static int getNumberOfExecutions(LocallyRunOperatorExtension extension) { return ((TestExecutionInfoProvider) extension.getReconcilers().get(0)).getNumberOfExecutions(); } } diff --git a/operator-framework/src/test/resources/io/javaoperatorsdk/operator/sample/standalonedependent/nginx-deployment.yaml b/operator-framework/src/test/resources/io/javaoperatorsdk/operator/nginx-deployment.yaml similarity index 100% rename from operator-framework/src/test/resources/io/javaoperatorsdk/operator/sample/standalonedependent/nginx-deployment.yaml rename to operator-framework/src/test/resources/io/javaoperatorsdk/operator/nginx-deployment.yaml diff --git a/pom.xml b/pom.xml index 400a49b3bd..9c3b036cbc 100644 --- a/pom.xml +++ b/pom.xml @@ -5,7 +5,7 @@ io.javaoperatorsdk java-operator-sdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT Operator SDK for Java Java SDK for implementing Kubernetes operators pom diff --git a/sample-operators/mysql-schema/pom.xml b/sample-operators/mysql-schema/pom.xml index c6ee96e8ea..68beecdc65 100644 --- a/sample-operators/mysql-schema/pom.xml +++ b/sample-operators/mysql-schema/pom.xml @@ -7,7 +7,7 @@ io.javaoperatorsdk sample-operators - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT sample-mysql-schema-operator diff --git a/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/MySQLSchemaReconciler.java b/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/MySQLSchemaReconciler.java index 08b65162fd..05187aea2b 100644 --- a/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/MySQLSchemaReconciler.java +++ b/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/MySQLSchemaReconciler.java @@ -21,8 +21,9 @@ @ControllerConfiguration( dependents = { - @Dependent(type = SecretDependentResource.class), - @Dependent(type = SchemaDependentResource.class, name = SchemaDependentResource.NAME) + @Dependent(type = SecretDependentResource.class, name = SecretDependentResource.NAME), + @Dependent(type = SchemaDependentResource.class, name = SchemaDependentResource.NAME, + dependsOn = SecretDependentResource.NAME) }) public class MySQLSchemaReconciler implements Reconciler, ErrorStatusHandler { diff --git a/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SchemaDependentResource.java b/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SchemaDependentResource.java index 5bef4adb04..48e3f37abe 100644 --- a/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SchemaDependentResource.java +++ b/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SchemaDependentResource.java @@ -29,6 +29,7 @@ public class SchemaDependentResource implements EventSourceProvider, DependentResourceConfigurator, Creator, Deleter { + public static final String NAME = "schema"; private static final Logger log = LoggerFactory.getLogger(SchemaDependentResource.class); diff --git a/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SecretDependentResource.java b/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SecretDependentResource.java index b1c516df8e..e7e5550669 100644 --- a/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SecretDependentResource.java +++ b/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/dependent/SecretDependentResource.java @@ -1,6 +1,7 @@ package io.javaoperatorsdk.operator.sample.dependent; import java.util.Base64; +import java.util.Set; import org.apache.commons.lang3.RandomStringUtils; @@ -10,12 +11,15 @@ import io.javaoperatorsdk.operator.processing.dependent.Creator; import io.javaoperatorsdk.operator.processing.dependent.Matcher.Result; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.event.ResourceID; +import io.javaoperatorsdk.operator.processing.event.source.SecondaryToPrimaryMapper; import io.javaoperatorsdk.operator.sample.MySQLSchema; public class SecretDependentResource extends KubernetesDependentResource - implements Creator { - - public static final String SECRET_FORMAT = "%s-secret"; + implements Creator, SecondaryToPrimaryMapper { + public static final String NAME = "secret"; + public static final String SECRET_SUFFIX = "-secret"; + public static final String SECRET_FORMAT = "%s" + SECRET_SUFFIX; public static final String USERNAME_FORMAT = "%s-user"; public static final String MYSQL_SECRET_USERNAME = "mysql.secret.user.name"; public static final String MYSQL_SECRET_PASSWORD = "mysql.secret.user.password"; @@ -55,4 +59,11 @@ public Result match(Secret actual, MySQLSchema primary, Context toPrimaryResourceIDs(Secret secondaryResource) { + String name = secondaryResource.getMetadata().getName(); + return Set.of(new ResourceID(name.substring(0, name.length() - SECRET_SUFFIX.length()), + secondaryResource.getMetadata().getNamespace())); + } } diff --git a/sample-operators/mysql-schema/src/test/java/io/javaoperatorsdk/operator/sample/MySQLSchemaOperatorE2E.java b/sample-operators/mysql-schema/src/test/java/io/javaoperatorsdk/operator/sample/MySQLSchemaOperatorE2E.java index 07eb2c0a54..93a0097b6c 100644 --- a/sample-operators/mysql-schema/src/test/java/io/javaoperatorsdk/operator/sample/MySQLSchemaOperatorE2E.java +++ b/sample-operators/mysql-schema/src/test/java/io/javaoperatorsdk/operator/sample/MySQLSchemaOperatorE2E.java @@ -16,8 +16,8 @@ import io.fabric8.kubernetes.client.DefaultKubernetesClient; import io.fabric8.kubernetes.client.KubernetesClient; import io.javaoperatorsdk.operator.junit.AbstractOperatorExtension; -import io.javaoperatorsdk.operator.junit.ClusterOperatorExtension; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.ClusterDeployedOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import io.javaoperatorsdk.operator.sample.dependent.ResourcePollerConfig; import io.javaoperatorsdk.operator.sample.dependent.SchemaDependentResource; @@ -62,7 +62,7 @@ boolean isLocal() { @RegisterExtension AbstractOperatorExtension operator = isLocal() - ? LocalOperatorExtension.builder() + ? LocallyRunOperatorExtension.builder() .withReconciler( new MySQLSchemaReconciler(), c -> c.replacingNamedDependentResourceConfig( @@ -73,7 +73,7 @@ boolean isLocal() { .withInfrastructure(infrastructure) .withPortForward(MY_SQL_NS, "app", "mysql", 3306, LOCAL_PORT) .build() - : ClusterOperatorExtension.builder() + : ClusterDeployedOperatorExtension.builder() .withOperatorDeployment(client.load(new FileInputStream("k8s/operator.yaml")).get()) .withInfrastructure(infrastructure) .build(); diff --git a/sample-operators/pom.xml b/sample-operators/pom.xml index 31af37c86b..7773257c72 100644 --- a/sample-operators/pom.xml +++ b/sample-operators/pom.xml @@ -7,7 +7,7 @@ io.javaoperatorsdk java-operator-sdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT sample-operators diff --git a/sample-operators/tomcat-operator/pom.xml b/sample-operators/tomcat-operator/pom.xml index a5385abfee..b91ebfa131 100644 --- a/sample-operators/tomcat-operator/pom.xml +++ b/sample-operators/tomcat-operator/pom.xml @@ -7,7 +7,7 @@ io.javaoperatorsdk sample-operators - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT sample-tomcat-operator diff --git a/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java b/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java index 94726d40ae..25e46fad16 100644 --- a/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java +++ b/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java @@ -5,12 +5,12 @@ import io.fabric8.kubernetes.api.model.apps.DeploymentBuilder; import io.javaoperatorsdk.operator.ReconcilerUtils; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; @KubernetesDependent(labelSelector = "app.kubernetes.io/managed-by=tomcat-operator") public class DeploymentDependentResource - extends CRUKubernetesDependentResource { + extends CRUDKubernetesDependentResource { public DeploymentDependentResource() { super(Deployment.class); diff --git a/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java b/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java index 8efaadc0a8..3b526d02bc 100644 --- a/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java +++ b/sample-operators/tomcat-operator/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java @@ -5,12 +5,11 @@ import io.fabric8.kubernetes.api.model.ServiceBuilder; import io.javaoperatorsdk.operator.ReconcilerUtils; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.Creator; -import io.javaoperatorsdk.operator.processing.dependent.Updater; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; -public class ServiceDependentResource extends KubernetesDependentResource - implements Creator, Updater { +@KubernetesDependent(labelSelector = "app.kubernetes.io/managed-by=tomcat-operator") +public class ServiceDependentResource extends CRUDKubernetesDependentResource { public ServiceDependentResource() { super(Service.class); @@ -23,6 +22,7 @@ protected Service desired(Tomcat tomcat, Context context) { .editMetadata() .withName(tomcatMetadata.getName()) .withNamespace(tomcatMetadata.getNamespace()) + .addToLabels("app.kubernetes.io/managed-by", "tomcat-operator") .endMetadata() .editSpec() .addToSelector("app", tomcatMetadata.getName()) diff --git a/sample-operators/tomcat-operator/src/test/java/io/javaoperatorsdk/operator/sample/TomcatOperatorE2E.java b/sample-operators/tomcat-operator/src/test/java/io/javaoperatorsdk/operator/sample/TomcatOperatorE2E.java index c04cbed72c..10959c4c48 100644 --- a/sample-operators/tomcat-operator/src/test/java/io/javaoperatorsdk/operator/sample/TomcatOperatorE2E.java +++ b/sample-operators/tomcat-operator/src/test/java/io/javaoperatorsdk/operator/sample/TomcatOperatorE2E.java @@ -8,12 +8,14 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import io.fabric8.kubernetes.api.model.*; -import io.fabric8.kubernetes.client.*; +import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; +import io.fabric8.kubernetes.client.DefaultKubernetesClient; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.KubernetesClientException; import io.javaoperatorsdk.operator.junit.AbstractOperatorExtension; -import io.javaoperatorsdk.operator.junit.ClusterOperatorExtension; +import io.javaoperatorsdk.operator.junit.ClusterDeployedOperatorExtension; import io.javaoperatorsdk.operator.junit.InClusterCurl; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import static java.util.concurrent.TimeUnit.MINUTES; import static org.awaitility.Awaitility.await; @@ -40,12 +42,12 @@ boolean isLocal() { } @RegisterExtension - AbstractOperatorExtension operator = isLocal() ? LocalOperatorExtension.builder() + AbstractOperatorExtension operator = isLocal() ? LocallyRunOperatorExtension.builder() .waitForNamespaceDeletion(false) .withReconciler(new TomcatReconciler()) .withReconciler(new WebappReconciler(client)) .build() - : ClusterOperatorExtension.builder() + : ClusterDeployedOperatorExtension.builder() .waitForNamespaceDeletion(false) .withOperatorDeployment( client.load(new FileInputStream("k8s/operator.yaml")).get()) diff --git a/sample-operators/webpage/pom.xml b/sample-operators/webpage/pom.xml index bb1c757518..2446663c88 100644 --- a/sample-operators/webpage/pom.xml +++ b/sample-operators/webpage/pom.xml @@ -7,7 +7,7 @@ io.javaoperatorsdk sample-operators - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT sample-webpage-operator diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ConfigMapDependentResource.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ConfigMapDependentResource.java index cfe0f79a0e..cf997f86d8 100644 --- a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ConfigMapDependentResource.java +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ConfigMapDependentResource.java @@ -10,7 +10,7 @@ import io.fabric8.kubernetes.api.model.ConfigMapBuilder; import io.fabric8.kubernetes.api.model.ObjectMetaBuilder; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; import static io.javaoperatorsdk.operator.sample.Utils.configMapName; @@ -19,7 +19,8 @@ // this annotation only activates when using managed dependents and is not otherwise needed @KubernetesDependent(labelSelector = SELECTOR) -public class ConfigMapDependentResource extends CRUKubernetesDependentResource { +public class ConfigMapDependentResource + extends CRUDKubernetesDependentResource { private static final Logger log = LoggerFactory.getLogger(ConfigMapDependentResource.class); diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java index 4991171f12..8986660bdf 100644 --- a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/DeploymentDependentResource.java @@ -6,7 +6,7 @@ import io.fabric8.kubernetes.api.model.ConfigMapVolumeSourceBuilder; import io.fabric8.kubernetes.api.model.apps.Deployment; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; import static io.javaoperatorsdk.operator.ReconcilerUtils.loadYaml; @@ -15,9 +15,9 @@ import static io.javaoperatorsdk.operator.sample.WebPageManagedDependentsReconciler.SELECTOR; // this annotation only activates when using managed dependents and is not otherwise needed -@KubernetesDependent(labelSelector = WebPageManagedDependentsReconciler.SELECTOR) +@KubernetesDependent(labelSelector = SELECTOR) public class DeploymentDependentResource - extends CRUKubernetesDependentResource { + extends CRUDKubernetesDependentResource { public DeploymentDependentResource() { super(Deployment.class); diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ExposedIngressCondition.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ExposedIngressCondition.java new file mode 100644 index 0000000000..218d1f8ca2 --- /dev/null +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ExposedIngressCondition.java @@ -0,0 +1,14 @@ +package io.javaoperatorsdk.operator.sample; + +import io.fabric8.kubernetes.api.model.networking.v1.Ingress; +import io.javaoperatorsdk.operator.api.reconciler.Context; +import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition; + +public class ExposedIngressCondition implements Condition { + @Override + public boolean isMet(DependentResource dependentResource, WebPage primary, + Context context) { + return primary.getSpec().getExposed(); + } +} diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/IngressDependentResource.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/IngressDependentResource.java index 074f36cffb..703d3aceb1 100644 --- a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/IngressDependentResource.java +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/IngressDependentResource.java @@ -2,14 +2,14 @@ import io.fabric8.kubernetes.api.model.networking.v1.Ingress; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; -import static io.javaoperatorsdk.operator.sample.Utils.*; +import static io.javaoperatorsdk.operator.sample.Utils.makeDesiredIngress; // this annotation only activates when using managed dependents and is not otherwise needed @KubernetesDependent(labelSelector = WebPageManagedDependentsReconciler.SELECTOR) -public class IngressDependentResource extends CRUKubernetesDependentResource { +public class IngressDependentResource extends CRUDKubernetesDependentResource { public IngressDependentResource() { super(Ingress.class); diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java index a914aa5994..1080b1b461 100644 --- a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/ServiceDependentResource.java @@ -5,7 +5,6 @@ import io.fabric8.kubernetes.api.model.Service; import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUKubernetesDependentResource; import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent; import static io.javaoperatorsdk.operator.ReconcilerUtils.loadYaml; @@ -14,8 +13,9 @@ import static io.javaoperatorsdk.operator.sample.WebPageManagedDependentsReconciler.SELECTOR; // this annotation only activates when using managed dependents and is not otherwise needed -@KubernetesDependent(labelSelector = WebPageManagedDependentsReconciler.SELECTOR) -public class ServiceDependentResource extends CRUKubernetesDependentResource { +@KubernetesDependent(labelSelector = SELECTOR) +public class ServiceDependentResource extends + io.javaoperatorsdk.operator.processing.dependent.kubernetes.CRUDKubernetesDependentResource { public ServiceDependentResource() { super(Service.class); diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageDependentsWorkflowReconciler.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageDependentsWorkflowReconciler.java new file mode 100644 index 0000000000..e9c5218cf8 --- /dev/null +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageDependentsWorkflowReconciler.java @@ -0,0 +1,89 @@ +package io.javaoperatorsdk.operator.sample; + +import java.util.Arrays; +import java.util.Map; + +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.Service; +import io.fabric8.kubernetes.api.model.apps.Deployment; +import io.fabric8.kubernetes.api.model.networking.v1.Ingress; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.javaoperatorsdk.operator.api.reconciler.*; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource; +import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfig; +import io.javaoperatorsdk.operator.processing.dependent.workflow.Workflow; +import io.javaoperatorsdk.operator.processing.dependent.workflow.builder.WorkflowBuilder; +import io.javaoperatorsdk.operator.processing.event.source.EventSource; + +import static io.javaoperatorsdk.operator.sample.Utils.*; + +/** + * Shows how to implement reconciler using standalone dependent resources. + */ +@ControllerConfiguration( + labelSelector = WebPageDependentsWorkflowReconciler.DEPENDENT_RESOURCE_LABEL_SELECTOR) +public class WebPageDependentsWorkflowReconciler + implements Reconciler, ErrorStatusHandler, EventSourceInitializer { + + public static final String DEPENDENT_RESOURCE_LABEL_SELECTOR = "!low-level"; + + private KubernetesDependentResource configMapDR; + private KubernetesDependentResource deploymentDR; + private KubernetesDependentResource serviceDR; + private KubernetesDependentResource ingressDR; + + private Workflow workflow; + + public WebPageDependentsWorkflowReconciler(KubernetesClient kubernetesClient) { + initDependentResources(kubernetesClient); + workflow = new WorkflowBuilder() + .addDependentResource(configMapDR).build() + .addDependentResource(deploymentDR).build() + .addDependentResource(serviceDR).build() + .addDependentResource(ingressDR).withReconcilePrecondition(new ExposedIngressCondition()) + .build() + .build(); + } + + @Override + public Map prepareEventSources(EventSourceContext context) { + return EventSourceInitializer.nameEventSources(configMapDR.initEventSource(context), + deploymentDR.initEventSource(context), serviceDR.initEventSource(context), + ingressDR.initEventSource(context)); + } + + @Override + public UpdateControl reconcile(WebPage webPage, Context context) + throws Exception { + simulateErrorIfRequested(webPage); + + workflow.reconcile(webPage, context); + + webPage.setStatus( + createStatus( + configMapDR.getSecondaryResource(webPage).orElseThrow().getMetadata().getName())); + return UpdateControl.patchStatus(webPage); + } + + @Override + public ErrorStatusUpdateControl updateErrorStatus( + WebPage resource, Context retryInfo, Exception e) { + return handleError(resource, e); + } + + private void initDependentResources(KubernetesClient client) { + this.configMapDR = new ConfigMapDependentResource(); + this.deploymentDR = new DeploymentDependentResource(); + this.serviceDR = new ServiceDependentResource(); + this.ingressDR = new IngressDependentResource(); + + Arrays.asList(configMapDR, deploymentDR, serviceDR, ingressDR).forEach(dr -> { + dr.setKubernetesClient(client); + dr.configureWith(new KubernetesDependentResourceConfig() + .setLabelSelector(DEPENDENT_RESOURCE_LABEL_SELECTOR)); + }); + } + + + +} diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageManagedDependentsReconciler.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageManagedDependentsReconciler.java index ac89ebd269..b2e0963ef2 100644 --- a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageManagedDependentsReconciler.java +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageManagedDependentsReconciler.java @@ -20,7 +20,9 @@ dependents = { @Dependent(type = ConfigMapDependentResource.class), @Dependent(type = DeploymentDependentResource.class), - @Dependent(type = ServiceDependentResource.class) + @Dependent(type = ServiceDependentResource.class), + @Dependent(type = IngressDependentResource.class, + reconcilePrecondition = ExposedIngressCondition.class) }) public class WebPageManagedDependentsReconciler implements Reconciler, ErrorStatusHandler { diff --git a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageReconciler.java b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageReconciler.java index 19b130da88..2aa1674564 100644 --- a/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageReconciler.java +++ b/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageReconciler.java @@ -3,6 +3,7 @@ import java.util.HashMap; import java.util.Map; import java.util.Objects; +import java.util.concurrent.TimeUnit; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; @@ -18,14 +19,7 @@ import io.fabric8.kubernetes.client.KubernetesClient; import io.javaoperatorsdk.operator.ReconcilerUtils; import io.javaoperatorsdk.operator.api.config.informer.InformerConfiguration; -import io.javaoperatorsdk.operator.api.reconciler.Context; -import io.javaoperatorsdk.operator.api.reconciler.ControllerConfiguration; -import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusHandler; -import io.javaoperatorsdk.operator.api.reconciler.ErrorStatusUpdateControl; -import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext; -import io.javaoperatorsdk.operator.api.reconciler.EventSourceInitializer; -import io.javaoperatorsdk.operator.api.reconciler.Reconciler; -import io.javaoperatorsdk.operator.api.reconciler.UpdateControl; +import io.javaoperatorsdk.operator.api.reconciler.*; import io.javaoperatorsdk.operator.processing.event.source.EventSource; import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource; @@ -33,7 +27,8 @@ import static io.javaoperatorsdk.operator.sample.WebPageManagedDependentsReconciler.SELECTOR; /** Shows how to implement reconciler using the low level api directly. */ -@ControllerConfiguration +@ControllerConfiguration(rateLimit = @RateLimit(limitForPeriod = 2, refreshPeriod = 3, + refreshPeriodTimeUnit = TimeUnit.SECONDS)) public class WebPageReconciler implements Reconciler, ErrorStatusHandler, EventSourceInitializer { diff --git a/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorE2E.java b/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorE2E.java index 2183fba43e..a39d1cc054 100644 --- a/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorE2E.java +++ b/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorE2E.java @@ -10,8 +10,8 @@ import io.fabric8.kubernetes.api.model.EnvVar; import io.fabric8.kubernetes.api.model.apps.Deployment; import io.javaoperatorsdk.operator.junit.AbstractOperatorExtension; -import io.javaoperatorsdk.operator.junit.ClusterOperatorExtension; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.ClusterDeployedOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; import static io.javaoperatorsdk.operator.sample.WebPageOperator.WEBPAGE_CLASSIC_RECONCILER_ENV_VALUE; import static io.javaoperatorsdk.operator.sample.WebPageOperator.WEBPAGE_RECONCILER_ENV; @@ -24,11 +24,11 @@ public WebPageOperatorE2E() throws FileNotFoundException {} @RegisterExtension AbstractOperatorExtension operator = isLocal() - ? LocalOperatorExtension.builder() + ? LocallyRunOperatorExtension.builder() .waitForNamespaceDeletion(false) .withReconciler(new WebPageReconciler(client)) .build() - : ClusterOperatorExtension.builder() + : ClusterDeployedOperatorExtension.builder() .waitForNamespaceDeletion(false) .withOperatorDeployment(client.load(new FileInputStream("k8s/operator.yaml")).get(), resources -> { diff --git a/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorManagedDependentResourcesE2E.java b/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorManagedDependentResourcesE2E.java index 1b40fccc80..3fbf877d64 100644 --- a/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorManagedDependentResourcesE2E.java +++ b/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorManagedDependentResourcesE2E.java @@ -10,10 +10,11 @@ import io.fabric8.kubernetes.api.model.EnvVar; import io.fabric8.kubernetes.api.model.apps.Deployment; import io.javaoperatorsdk.operator.junit.AbstractOperatorExtension; -import io.javaoperatorsdk.operator.junit.ClusterOperatorExtension; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.ClusterDeployedOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; -import static io.javaoperatorsdk.operator.sample.WebPageOperator.*; +import static io.javaoperatorsdk.operator.sample.WebPageOperator.WEBPAGE_MANAGED_DEPENDENT_RESOURCE_ENV_VALUE; +import static io.javaoperatorsdk.operator.sample.WebPageOperator.WEBPAGE_RECONCILER_ENV; class WebPageOperatorManagedDependentResourcesE2E extends WebPageOperatorAbstractTest { @@ -22,11 +23,11 @@ public WebPageOperatorManagedDependentResourcesE2E() throws FileNotFoundExceptio @RegisterExtension AbstractOperatorExtension operator = isLocal() - ? LocalOperatorExtension.builder() + ? LocallyRunOperatorExtension.builder() .waitForNamespaceDeletion(false) .withReconciler(new WebPageManagedDependentsReconciler()) .build() - : ClusterOperatorExtension.builder() + : ClusterDeployedOperatorExtension.builder() .waitForNamespaceDeletion(false) .withOperatorDeployment(client.load(new FileInputStream("k8s/operator.yaml")).get(), resources -> { diff --git a/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorStandaloneDependentResourcesE2E.java b/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorStandaloneDependentResourcesE2E.java index 2175e33a41..1b7a08d71b 100644 --- a/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorStandaloneDependentResourcesE2E.java +++ b/sample-operators/webpage/src/test/java/io/javaoperatorsdk/operator/sample/WebPageOperatorStandaloneDependentResourcesE2E.java @@ -6,8 +6,8 @@ import org.junit.jupiter.api.extension.RegisterExtension; import io.javaoperatorsdk.operator.junit.AbstractOperatorExtension; -import io.javaoperatorsdk.operator.junit.ClusterOperatorExtension; -import io.javaoperatorsdk.operator.junit.LocalOperatorExtension; +import io.javaoperatorsdk.operator.junit.ClusterDeployedOperatorExtension; +import io.javaoperatorsdk.operator.junit.LocallyRunOperatorExtension; class WebPageOperatorStandaloneDependentResourcesE2E extends WebPageOperatorAbstractTest { @@ -16,11 +16,11 @@ public WebPageOperatorStandaloneDependentResourcesE2E() throws FileNotFoundExcep @RegisterExtension AbstractOperatorExtension operator = isLocal() - ? LocalOperatorExtension.builder() + ? LocallyRunOperatorExtension.builder() .waitForNamespaceDeletion(false) .withReconciler(new WebPageStandaloneDependentsReconciler(client)) .build() - : ClusterOperatorExtension.builder() + : ClusterDeployedOperatorExtension.builder() .waitForNamespaceDeletion(false) .withOperatorDeployment(client.load(new FileInputStream("k8s/operator.yaml")).get()) .build(); diff --git a/smoke-test-samples/common/pom.xml b/smoke-test-samples/common/pom.xml index e0debc66e0..5d0babe88f 100644 --- a/smoke-test-samples/common/pom.xml +++ b/smoke-test-samples/common/pom.xml @@ -6,7 +6,7 @@ io.javaoperatorsdk java-operator-sdk-smoke-test-samples - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT operator-framework-smoke-test-samples-common diff --git a/smoke-test-samples/pom.xml b/smoke-test-samples/pom.xml index f514fc8681..434856b1a0 100644 --- a/smoke-test-samples/pom.xml +++ b/smoke-test-samples/pom.xml @@ -6,7 +6,7 @@ io.javaoperatorsdk java-operator-sdk - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT java-operator-sdk-smoke-test-samples diff --git a/smoke-test-samples/pure-java/pom.xml b/smoke-test-samples/pure-java/pom.xml index 41aca289ec..c9f639c0b4 100644 --- a/smoke-test-samples/pure-java/pom.xml +++ b/smoke-test-samples/pure-java/pom.xml @@ -6,7 +6,7 @@ io.javaoperatorsdk java-operator-sdk-smoke-test-samples - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT operator-framework-smoke-test-samples-pure-java diff --git a/smoke-test-samples/spring-boot-plain/pom.xml b/smoke-test-samples/spring-boot-plain/pom.xml index dc694a52a7..a6c3db970b 100644 --- a/smoke-test-samples/spring-boot-plain/pom.xml +++ b/smoke-test-samples/spring-boot-plain/pom.xml @@ -6,7 +6,7 @@ io.javaoperatorsdk java-operator-sdk-smoke-test-samples - 3.0.4-SNAPSHOT + 3.1.0-SNAPSHOT operator-framework-smoke-test-samples-spring-boot