Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenObserve as a Sink for Vector #21514

Open
chaitanya-sistla opened this issue Oct 16, 2024 · 2 comments · May be fixed by #21531
Open

OpenObserve as a Sink for Vector #21514

chaitanya-sistla opened this issue Oct 16, 2024 · 2 comments · May be fixed by #21531
Labels
sink: new A request for a new sink type: feature A value-adding code addition that introduce new functionality.

Comments

@chaitanya-sistla
Copy link

chaitanya-sistla commented Oct 16, 2024

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Users are already successfully ingesting data from Vector into OpenObserve using the existing http sink. The goal of this request is to have OpenObserve officially documented as a supported sink in Vector, providing clear guidance for users on how to set it up. This will streamline the integration process for new users, reduce confusion, and ensure OpenObserve is recognized alongside other supported sinks in the documentation.

By including OpenObserve in the official documentation, we can provide a more straightforward path for users to ingest observability data like logs, metrics, and traces into OpenObserve.

Attempted Solutions

We have been using the http sink with the following configuration to ingest data into OpenObserve, which works without any issues:

[sinks.openobserve]
type = "http"
inputs = ["my_source"]
uri = "http://localhost:5080/api/default/default/_json"
method = "post"
auth.strategy = "basic"
auth.user = "[email protected]"
auth.password = "password"
compression = "gzip"
encoding.codec = "json"
encoding.timestamp_format = "rfc3339"
healthcheck.enabled = false

This setup works perfectly for current users. The ingestion process has been stable, and no breaking changes would be introduced with the official inclusion of OpenObserve in the Vector documentation. The main limitation we are facing is the lack of official recognition and ease of setup guidance for new users.

Proposal

We propose adding OpenObserve to the list of officially supported sinks in Vector's documentation. This would require updating the documentation with a simple example configuration, similar to the one above, and highlighting the use of OpenObserve’s HTTP-based API for data ingestion.

No new functionality is required, as the current http sink serves the purpose well. However, official recognition and clear setup instructions in the Vector documentation would significantly enhance the user experience and promote OpenObserve as an observability solution.

By doing this, users can confidently use Vector to send data to OpenObserve, knowing it’s a supported use case with clear guidance from both the Vector and OpenObserve teams.

References

No response

Version

No response

@chaitanya-sistla chaitanya-sistla added the type: feature A value-adding code addition that introduce new functionality. label Oct 16, 2024
@jszwedko jszwedko added the sink: new A request for a new sink label Oct 16, 2024
@jszwedko
Copy link
Member

Thanks @chaitanya-sistla ! I think we'd be open to this as a contribution if you like. The recent rewrite of the Axiom sink to wrap the HTTP sink could be useful as a reference: #21362. There isn't a precedent, at the moment, for having component pages that just document how to configure another component to be used with the service, but that could be another option. The closest example we have is the mention of Azure Event Hubs on the Kafka sink: https://vector.dev/docs/reference/configuration/sinks/kafka/#azure-event-hubs

@chaitanya-sistla
Copy link
Author

Thank you @jszwedko I will get to it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sink: new A request for a new sink type: feature A value-adding code addition that introduce new functionality.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants