Optimize Encoding by Grouping Multiple Spans into OTEL TraceData

[making]

Feature

Optimize Encoding by Grouping Multiple Spans into OTEL TraceData

Rationale

The current implementation of the encoder converts each Zipkin Span into a OTEL TraceData object. As a result, TraceData with only a single Span is sent each time. In OTEL, multiple Spans are typically grouped within ResourceSpans, and multiple ResourceSpans can be wrapped within a single TraceData. This makes the current implementation appear inefficient.

By converting MutableSpan to io.opentelemetry.proto.trace.v1.Span within the encoder, and then aggregating multiple Spans into a single TraceData on the zipkin2.reporter.Encoding or zipkin2.reporter.internal.AsyncReporter side, the efficiency of the encoding process could be significantly improved.

Example Scenario

In the current setup, each individual Zipkin Span results in a separate TraceData object being sent, even though OTEL is designed to handle multiple Spans together within ResourceSpans. By adjusting the implementation to aggregate multiple Spans into a single TraceData object, it would reduce overhead and improve processing efficiency, especially in environments where a large volume of Spans is being reported.

[codefromthecrypt]

this is an interesting feature idea. in brave, there is a localRootId which was made for aggregating by trace ID, just not span ID. I'm not sure if there is something in otel side to aggregate once already converted to otel. Mainly, the challenge on this is to ensure you release/timeout what you are collecting.

here are some breadcrumbs
https://github.com/openzipkin/brave/blob/69003dfc811418f0dbc42e9e17ff880ebe1f4b02/brave/src/test/java/brave/features/handler/SkeletalSpansTest.java
https://github.com/openzipkin/brave/blob/69003dfc811418f0dbc42e9e17ff880ebe1f4b02/brave/src/test/java/brave/features/handler/CountingChildrenTest.java
https://github.com/openzipkin-contrib/zipkin-storage-kafka/blob/master/storage/src/main/java/zipkin2/storage/kafka/streams/SpanAggregationTopology.java