Repository navigation
[Stack Monitoring] Deprecate internal collection topics #555
Description
We now have the ability to monitor Elasticsearch, Kibana, Logstash, Beats, and APM Server with Metricbeat. The previous method of monitoring stack products, using collection code internal to each product that shipped monitoring data to a custom Monitoring Bulk API endpoint, is now deprecated.
We should reflect this deprecation in documentation as well, specifically in these areas:
- Stack docs: https://www.elastic.co/guide/en/elastic-stack-overview/current/xpack-monitoring.html
- Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/monitor-elasticsearch-cluster.html
- Kibana: https://www.elastic.co/guide/en/kibana/current/configuring-monitoring.html
- Beats: https://www.elastic.co/guide/en/beats/heartbeat/current/monitoring.html (Using Heartbeat as a sample here, but applicable to all Beats and APM Server documentation)
- Logstash: TBD; PR introducing Metricbeat collection is currently WIP: [Doc]Add metricbeat as monitoring option logstash#11033
Activity
cachedout commentedon Dec 2, 2019
Hi @lcawl. I've started some of this:
Beats: elastic/beats#14887
Elasticsearch: elastic/elasticsearch#49758
cachedout commentedon Dec 2, 2019
Elasticsearch done: elastic/elasticsearch#49758
insukcho commentedon May 12, 2023
Just want to make sure that Beat also has the same policy. So, we need to use separate Metricbeat to monitor Metricbeat, right?
If yes, we should change the documentation content because users could be misleading that we recommend using deprecated internal collection:
consulthys commentedon Oct 9, 2024
Agree with @insukcho as we (Stack Monitoring) start getting some reports (e.g. https://github.com/elastic/sdh-beats/issues/5232) that Beats internal monitoring doesn't show deprecated in the docs.
I see there was an initial attempt here by @cachedout but it was closed. How can we make it happen? Thanks!
consulthys commentedon Oct 10, 2024
The last comment in that PR intrigues me.
@dedemorton could you provide some context concerning the discussions you had and how we should progress? Thanks a lot
dedemorton commentedon Oct 10, 2024
@consulthys I'm sorry, it's been too long since I made that comment, and I can't remember the details. Plus Metricbeat has evolved so much over the years that whatever concerns were raised back then are probably no longer valid. I have not worked in that area for quite awhile, so I cannot provide any more info.
consulthys commentedon Oct 11, 2024
@dedemorton thank you so much for your quick answer!
No worries, we'll figure it out eventually.