Repository navigation
leader-election support multi cluster #1817
Description
Bug Report
What did you do?
an instance watch multi-cluster event, but only one election
What did you expect to see?
https://github.com/java-operator-sdk/java-operator-sdk/blob/3812a7fe73f81033d87297afd3081a93760a7abd/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/Operator.java#L76
can support custom kubernetes-client
Activity
csviri commentedon Mar 14, 2023
Hi @hkmdxlftjf , I'm not sure I uderstand what are you asking for, could you describe your use case?
hkmdxlftjf commentedon Mar 14, 2023
we have multi k8s clusters, I want to deploy the operator on a cluster and namespace is xxxx, watch all cluster cr events,
eg:
Map<String, Operator> clusterWithOperator = new HashMap<>();when isLeaderElectionEnabled is true, new Operator() will create lease on a different cluster，expect to create a lease only on the deployed cluster, or i can specify the cluster(kubernetes client)
csviri commentedon Mar 14, 2023
Ok, so currently you can do this just not using the leader election capabilities of the JOSDK, and just setup leader election with fabric8 client, and start the operators if becomes the leader.
The downside is that the non-leader operator does prepares the caches, so when one them becomes the leader it might take some time until the resources are processed on startup on large clusters.
So if I understand the problem what you asking for is to be able to specify the client for leader election.
Will have to take a look because this not necessarily will solve the problem. Since you will have multiple operators competing for the lease, even if they have the same ID (in terms of leader election), this might not work. Will have to think about it / check it.
hkmdxlftjf commentedon Mar 14, 2023
can i try add attributein leaderelectionconfiguration(KubernetesClient client), and provide a method to use modify this attribute
csviri commentedon Mar 14, 2023
You are always free to create a pull request, and we can take a look. But would require a test for this, pls see current leader election tests.
I will also take a look on the implementation of the leader election, since there might be conceptual issue with this approach. Since in this case it is more like about having multiple leaders instead of one with same
identityand the leader election was not designed for this, so it anyways feels like a hack to me. (But will try the best to support your use case).cc @shawkins
csviri commentedon Mar 15, 2023
Will take a look but I think the proper solution here is to share the
LeaderElectorbetween multiple controllers, not createLeaderElectorfor eachOperatorwith the same client. Will take a look how feasible is this to implement.csviri commentedon Mar 15, 2023
@hkmdxlftjf Also note that leader election does not give you that much, so if pod crashes and restarts is good enough in most of cases.
csviri commentedon Apr 3, 2023
@hkmdxlftjf can we close this issue too?
hkmdxlftjf commentedon Apr 3, 2023
@csviri yes