Custom Thread Pool for Reconciler Execution #2263

Currently ThreadPoolExecutor is used to manage threads for the reconciliation.

Controllers are a little bit specific since at startup all the resources are reconciled, which can mean a large initial burst, while for the rest of the execution Thread utilization might be much smaller.

This situation might be covered relatively fine with a configuration for ThreadPoolExecutor that is created similar to the newCachedThreadPool
That provides a virtually unbound number of Threads, which are discarded from the pool if not used actively anymore. Since reconciliation is ideally fast (at least the majority on startup), the number of threads should never grow too big. Note that rejecting a task from the executor is not something that we can allow now.
On the other hand, it is not uncommon, that some controllers manage hundreds of thousands of resources and the reconciliation might not be that fast in all cases. So we would like to limit the number of threads even if that limit is quite high, and on startup have the rest of the resource queued and processed. Unfortunately, there is no such thing as, "queue the rest of incoming tasks" if the maximum thread count is reached in ThreadPoolExecutor.

So ideally the executor that we would need is very similar to the ThreadPool executor, but with an additional (maybe bounded) queue, that will queue up the tasks even if the maximum number of Threads is reached in the pool.

Notes:
see also: #2262