feat!: OpenTelemetry metrics #13265
Open
feat!: OpenTelemetry metrics #13265
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Joibel
added
area/controller
Led by #12589. As discussed in #12589 OpenTelemetry is the future of observability. This PR changes the underlying codebase to collect metrics using the otel libraries, whilst attempting to retain compatibility with prometheus scraping where this makes sense. It helps lay the groundwork for adding workflow tracing using otel. This PR amends and extends the built in metrics to be more useful and correctly named. This PR does not attempt do do anything else with otel, nor does it attempt to change custom metrics. Mostly removed prometheus libraries, and replaced with otel libraries. Allows use of both prometheus `/metrics` scraping and opentelemetry protocol transmission. Extends the workqueue metrics to include all of them exposed by [client-go/workqueue](https://github.com/kubernetes/client-go/blob/babfe96631905312244f34500d2fd1a8a10b186c/util/workqueue/metrics.go#L173) Removed `argo_workflows_workflows_processed_count` as a non-functioning metric. `originName` enhancements proposed in #12589 were not added, nor `argo_workflows_workflowtemplate_used_total` Note to reviewers: this is part of a stack of reviews for metrics changes. Please don't merge until the rest of the stack is also ready. Signed-off-by: Alan Clucas <alan@clucas.org>
Joibel
force-pushed
the
opentelemetry
branch
2 times, most recently
from
066e9a7
to
5974967
Compare
Signed-off-by: Alan Clucas <alan@clucas.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Led by #12589.
As discussed in #12589 OpenTelemetry is the future of observability. This PR changes the underlying codebase to collect metrics using the otel libraries, whilst attempting to retain compatibility with prometheus scraping where this makes sense.
It helps lay the groundwork for adding workflow tracing using otel.
This PR amends and extends the built in metrics to be more useful and correctly named.
This PR does not attempt do do anything else with otel, nor does it attempt to change custom metrics.
Mostly removed prometheus libraries, and replaced with otel libraries.
Allows use of both prometheus
/metricsscraping and opentelemetry protocol transmission.Extends the workqueue metrics to include all of them exposed by client-go/workqueue
Removed
argo_workflows_workflows_processed_countas a non-functioning metric.originNameenhancements proposed in #12589 were not added, norargo_workflows_workflowtemplate_used_totalmetricsTTLfor histogram custom metrics isn't implementable as the opentelemetry api doesn't provide a way to delete metrics by design. This only works for other custom metrics types by making them all asynchronous/observable (even for non realtime metrics) and just not reporting the TTL expired metrics.Note to reviewers: this is part of a stack of reviews for metrics changes. Please don't merge until the rest of the stack is also ready.
Supersedes #13232