OTEL workflow span attributes #6699
Conversation
stephanos
force-pushed
the
otel-span-wf-attrs
branch
4 times, most recently
from
8a6fd36
to
fa77802
Compare
| fx.Provide( | ||
| func(r *otelresource.Resource, sps []otelsdktrace.SpanProcessor) []otelsdktrace.TracerProviderOption { | ||
| opts := make([]otelsdktrace.TracerProviderOption, 0, len(sps)+1) | ||
| opts = append(opts, otelsdktrace.WithResource(r)) | ||
| for _, sp := range sps { | ||
| opts = append(opts, otelsdktrace.WithSpanProcessor(sp)) | ||
| } | ||
| return opts | ||
| }, | ||
| ), |
There was a problem hiding this comment.
Inlined into the next provider to be able to check if there are span processors at all.
| fx.Provide(func(lc fx.Lifecycle, opts []otelsdktrace.TracerProviderOption) trace.TracerProvider { | ||
| fx.Provide(func(lc fx.Lifecycle, r *otelresource.Resource, sps []otelsdktrace.SpanProcessor) trace.TracerProvider { | ||
| if len(sps) == 0 { | ||
| return noop.NewTracerProvider() |
There was a problem hiding this comment.
No need to trace anything if there are no processors (since there are no exporters)
| // annotate span with workflow tags (same ones the Temporal SDKs use) | ||
| for _, tag := range c.tags.Extract(s.Payload, methodName) { | ||
| var k string | ||
| switch tag.Key() { | ||
| case "wf-id": | ||
| k = "temporalWorkflowID" | ||
| case "wf-run-id": | ||
| k = "temporalRunID" | ||
| default: | ||
| continue | ||
| } | ||
| span.SetAttributes(attribute.Key(k).String(tag.Value().(string))) | ||
| } |
There was a problem hiding this comment.
This part is new net behavior. It applies the two attributes, when present.
| if !isEnabled(tp) { | ||
| return nil | ||
| } |
There was a problem hiding this comment.
This is new behavior: when tracing is disabled, it returns nil.
| if !isEnabled(tp) { | ||
| return nil | ||
| } |
There was a problem hiding this comment.
This is new behavior: when tracing is disabled, it returns nil.
stephanos
force-pushed
the
otel-span-wf-attrs
branch
from
fa77802
to
e527f21
Compare
| tp trace.TracerProvider, | ||
| tmp propagation.TextMapPropagator, | ||
| ) ServerTraceInterceptor { | ||
| //nolint:staticcheck | ||
| otelInterceptor := otelgrpc.UnaryServerInterceptor( |
There was a problem hiding this comment.
otelgrpc.UnaryServerInterceptor is deprecated.
| tmp propagation.TextMapPropagator, | ||
| ) ClientTraceInterceptor { | ||
| return ClientTraceInterceptor( | ||
| otelgrpc.UnaryClientInterceptor( |
There was a problem hiding this comment.
otelgrpc.UnaryClientInterceptor is deprecated.
| } | ||
|
|
||
| // annotate span with workflow tags (same ones the Temporal SDKs use) | ||
| for _, tag := range c.tags.Extract(s.Payload, methodName) { |
There was a problem hiding this comment.
This is re-using the existing generated code that can extract Workflow tags from gRPC requests. It seems to do exactly what we need, so I'm not inclined to clone/fork that code.
stephanos
force-pushed
the
otel-span-wf-attrs
branch
from
e527f21
to
f2ef687
Compare
stephanos
force-pushed
the
otel-span-wf-attrs
branch
from
f2ef687
to
afcc861
Compare
What changed?
(1) Adds the following OpenTelemetry (OTEL) span attributes:
temporalWorkflowIDtemporalRunID(2) Replaces the deprecated approach of OTEL interceptors with
stats.Handlers(3) If there are no exporters, disables the
stats.Handlerscompletely.Why?
The workflow id allows to correlate traces (which spans are part of) that belong to the same workflow id.
How did you test it?
(1) Added unit tests
(2) Manually
Using
it outputs
Potential risks
Breaking change?
Documentation
Is hotfix candidate?