Is Spandex still being maintained?

[san650]

I see low activity on this library and I'm wondering if it is still actively maintained? Is there a way to help?

[atonse]

I hope so – we're relying on this for our elixir app and it's been extremely shaky to get this integrated. We've spent 4 weeks getting it working with our app so there are definitely papercuts with it.

Happy to contribute as we get more familiar because we really do like Datadog.

[GregMefford]

The short answer is yes and no. There's not any major development going on, because the long-term solution is to migrate eventually to OpenTelemetry instead, which works with Datadog today, though it still has a few rough edges for Datadog specifically.

For small bugfixes and general maintenance, I am still planning to slowly keep things moving forward, but my time is limited and I'm no longer actually using Spandex at work, since we've already moved to OpenTelemetry.

[atonse]

Thank you for your work on this library @GregMefford.

It's great news that the community is moving towards an agnostic library, because it gives us even more options to switch vendors without it being a much bigger internal project to get the APM of a new vendor working.

And if this is truly the case and we are steering everyone towards OpenTelemetry, would you be able to add that note to the ReadMe on the front page of this project? It seems like this project would be entirely redundant at that point. (And who knows, maybe could've saved us a few weeks in even trying to adopt this library given all its quirks, if it's already abandoned).

OpenTelemetry looks great but still looks like it's quite a ways away from being an easy to integrate solution especially for Elixir devs, and seems to be more heavy on the erlang side than the elixir side (as far as docs and notes go). Is that accurate?

We're at least all tapped out for the next few weeks after having spent a few weeks getting Datadog APM to just work on our system. We'll revisit this again later to see how we can convert over to OpenTelemetry.

[san650]

Thank you @GregMefford -- I'll start looking into OpenTelemetry then

[warmwaffles]

@GregMefford perhaps a migration guide could be written for those who wish to transition to OpenTelemetry?

[tsloughter]

👋 from OpenTelemetry

I'd love to have a migration from Spandex guide put together, but I'd like to guarantee we have feature parity. If anyone sees this and has done or is going to do the migration please let me know if you find anything missing and what the pain points are. I'm also happy to help and answer any questions for the OpenTelemetry side.

[atonse]

@tsloughter From the links earlier to openTelemetry, all the examples seem to be pretty erlang focused, are you aware of a clean, easy to follow readme for Elixir devs?

[tsloughter]

@atonse the link to the github repo? The website has example snippets in both Erlang and Elixir https://opentelemetry.io/docs/instrumentation/erlang/

I'm reworking these docs as I don't think they flow that well, but it'll be a bit before I finish.

[atonse]

@tsloughter ahhhhhh yes I clicked on the link from the earlier post and the GitHub readme only had erlang code so I must've just moved along thinking the elixir side just isn't worked out yet.

Thank you!

[rossvz]

Found my way here after today's Elixir Merge newsletter included a link to this talk: https://www.youtube.com/watch?v=OVAe7vjlVVY

which uses the Spandex library. We've had numerous issues configuring our elixir/absinthe app with datadog using OpenTelemetry so I was curious if Spandex was a better option. @GregMefford I know you mentioned already having migrated - I'm guessing its feature-complete enough for you to not need Spandex, but we're still missing a lot of useful info like seen on the youtube video above.

[tsloughter]

@rossvz could you describe the missing info and configuration issues? I'd love to get Otel working for you. May want to open an issue in the otel repo instead though if you haven't already for these issues, or ping me on slack if you want.

[GregMefford]

Actually, since I mentioned before that I had moved to OTel, I have moved to a new job in a company that is using Spandex again, so I'm currently going through this process again, including an application that uses Absinthe, which is something I hadn't personally gone through a migration for before. I haven't finished that process yet, but my intention is to continue to contribute improvements to the OTel instrumentation for the various libraries that we're using, so that they'll continue to improve for others.

I think the issue with Absinthe in particular is just that the telemetry it emits is so verbose as to be almost useless without some amount of post-processing to make sense of it, which requires a lot of understanding about how Absinthe/DataLoader specifically, and GraphQL more generally, are designed. I will say though that the basics are in place with the OTel integrations, and it should be just as easy with OTel as it is with Spandex to overlay your own tweaks to the spans that it generates, so I would recommend to get started with OTel rather than instrumenting with Spandex only to migrate away shortly after.

[rossvz]

Thanks @GregMefford - do you have any resources or guides you've been using as far as setting that up correctly? As far as I know at the moment we basically just have the absinthe opentelemetry library added, with very minimal config:

  config :opentelemetry_absinthe,
    trace_options: [trace_response_result: false, trace_response_errors: true]

I'm trying to figure out what we would need to do to get things like resolver-level spans or info into Dataloader.

Also, around errors/crashes, anytime a query errors we just get something like

or in some cases when its a crash:

Is that stuff that we should be able to configure in Opentelemetry somehow?

[GregMefford]

Yeah, at the moment, that's basically all we have as well in our OTel spans, but we have a lot of custom stuff that's attaching to Absinthe's Telemetry events and creating custom spans/tags using Spandex. My plan is to figure out which parts of that is generally useful to others and get it upstreamed into opentelemetry_absinthe where it makes sense, so that we can trim down stuff that we have internally customized and also share with the community.