Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin' into feature/mqtt_sink_from_astro
Browse files Browse the repository at this point in the history
Signed-off-by: Stephen Wakely <[email protected]>
  • Loading branch information
StephenWakely committed Jan 10, 2024
2 parents 272ef31 + 8f504b3 commit 4212e7e
Show file tree
Hide file tree
Showing 6,842 changed files with 117,476 additions and 140,786 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
5 changes: 5 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
[alias]
vdev = "run --quiet --package vdev --"

[env]
# Build with large pages so that Vector runs on systems with 64k pages or less (e.g. 4k) to support
# CentOS 7, 8, and a few other Linux distributions.
JEMALLOC_SYS_WITH_LG_PAGE = "16"

[target.'cfg(all())']
rustflags = [
"-Dclippy::print_stdout",
Expand Down
1 change: 1 addition & 0 deletions .config/nextest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ fail-fast = false
[profile.default.junit]
# output test results at target/nextest/default/junit.xml
path = "junit.xml"
store-failure-output = true
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ Makefile linguist-detectable=false

distribution/helm/*/charts/*.tgz filter=lfs diff=lfs merge=lfs -text

lib/codecs/tests/data/** linguist-generated=true
tests/helm-snapshots/**/snapshot.yaml linguist-generated=true
100 changes: 100 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
docs/ @vectordotdev/ux-team
lib/dnsmsg-parser/ @vectordotdev/integrations-team
lib/file-source/ @vectordotdev/integrations-team
lib/k8s-e2e-tests/ @vectordotdev/integrations-team
lib/k8s-test-framework/ @vectordotdev/integrations-team
lib/opentelemetry-proto/ @vectordotdev/integrations-team
lib/vector-common/ @vectordotdev/core-team
lib/vector-config/ @vectordotdev/core-team
lib/vector-config-common/ @vectordotdev/core-team
lib/vector-config-macros/ @vectordotdev/core-team
lib/vector-core/ @vectordotdev/core-team
lib/vector-vrl-functions/ @vectordotdev/processing-team
lib/vrl/ @vectordotdev/processing-team
src/config/ @vectordotdev/core-team
src/internal_telemetry/ @vectordotdev/core-team
src/sinks/ @vectordotdev/integrations-team
src/sinks/amqp/ @StephenWakely @vectordotdev/integrations-team
src/sinks/appsignal/ @neuronull @vectordotdev/integrations-team
src/sinks/aws_cloudwatch_logs/ @vectordotdev/integrations-team
src/sinks/aws_cloudwatch_metrics/ @vectordotdev/integrations-team
src/sinks/aws_kinesis/ @vectordotdev/integrations-team # sink_aws_kinesis_firehose,sink_aws_kinesis_stream
src/sinks/aws_s3/ @vectordotdev/integrations-team
src/sinks/aws_sqs/ @vectordotdev/integrations-team
src/sinks/axiom.rs @vectordotdev/integrations-team
src/sinks/azure_blob/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/azure_monitor_logs.rs @dsmith3197 @vectordotdev/integrations-team
src/sinks/blackhole/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/clickhouse/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/console/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/databend/ @vectordotdev/integrations-team
src/sinks/datadog_events/ @neuronull @vectordotdev/integrations-team
src/sinks/datadog_logs/ @neuronull @vectordotdev/integrations-team
src/sinks/datadog_metrics/ @neuronull @vectordotdev/integrations-team
src/sinks/datadog_traces/ @neuronull @vectordotdev/integrations-team
src/sinks/elasticsearch/ @vectordotdev/integrations-team
src/sinks/file/ @vectordotdev/integrations-team
src/sinks/gcp/ @StephenWakely @vectordotdev/integrations-team # sink_gcp_chronicle_unstructured,sink_gcp_cloud_storage,sink_gcp_pubsub,sink_gcp_stackdriver_logs,sink_gcp_stackdriver_metrics
src/sinks/honeycomb.rs @vectordotdev/integrations-team
src/sinks/http.rs @neuronull @vectordotdev/integrations-team
src/sinks/humio/ @StephenWakely @vectordotdev/integrations-team # sink_humio_logs,sink_humio_metrics
src/sinks/influxdb/ @dsmith3197 @vectordotdev/integrations-team # sink_influxdb_logs,sink_influxdb_metrics
src/sinks/kafka/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/logdna.rs @neuronull @vectordotdev/integrations-team
src/sinks/loki/ @vectordotdev/integrations-team
src/sinks/nats.rs @StephenWakely @vectordotdev/integrations-team
src/sinks/new_relic/ @dsmith3197 @vectordotdev/integrations-team # sink_newrelix,sink_newrelic_logs
src/sinks/papertrail.rs @StephenWakely @vectordotdev/integrations-team
src/sinks/prometheus/ @StephenWakely @vectordotdev/integrations-team # sink_prometheus_exporter,sink_prometheus_remote_write
src/sinks/pulsar.rs @dsmith3197 @vectordotdev/integrations-team
src/sinks/redis.rs @StephenWakely @vectordotdev/integrations-team
src/sinks/sematext/ @vectordotdev/integrations-team # sink_sematext_logs,sink_sematext_metrics
src/sinks/socket.rs @neuronull @vectordotdev/integrations-team
src/sinks/splunk_hec/ @StephenWakely @vectordotdev/integrations-team # sink_splunk_hec_logs,sink_splunk_hec_metrics
src/sinks/statsd.rs @neuronull @vectordotdev/integrations-team
src/sinks/vector/ @neuronull @vectordotdev/integrations-team
src/sinks/websocket/ @neuronull @vectordotdev/integrations-team
src/source_sender/ @vectordotdev/core-team
src/sources/ @vectordotdev/integrations-team
src/sources/amqp.rs @StephenWakely @vectordotdev/integrations-team
src/sources/apache_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/aws_ecs_metrics/ @vectordotdev/integrations-team
src/sources/aws_kinesis_firehose/ @vectordotdev/integrations-team
src/sources/aws_s3/ @vectordotdev/integrations-team
src/sources/aws_sqs/ @vectordotdev/integrations-team
src/sources/datadog_agent/ @neuronull @vectordotdev/integrations-team
src/sources/demo_logs.rs @StephenWakely @vectordotdev/integrations-team
src/sources/dnstap/ @StephenWakely @vectordotdev/integrations-team
src/sources/docker_logs/ @vectordotdev/integrations-team
src/sources/eventstoredb_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/exec/ @dsmith3197 @vectordotdev/integrations-team
src/sources/file.rs @vectordotdev/integrations-team
src/sources/file_descriptors/ @dsmith3197 @vectordotdev/integrations-team # source_file_descriptor,source_stdin
src/sources/fluent/ @neuronull @vectordotdev/integrations-team
src/sources/gcp_pubsub.rs @StephenWakely @vectordotdev/integrations-team
src/sources/heroku_logs.rs @vectordotdev/integrations-team
src/sources/host_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/http_client/ @neuronull @vectordotdev/integrations-team
src/sources/http_server.rs @neuronull @vectordotdev/integrations-team
src/sources/internal_logs.rs @neuronull @vectordotdev/integrations-team
src/sources/internal_metrics.rs @neuronull @vectordotdev/integrations-team
src/sources/journald.rs @vectordotdev/integrations-team
src/sources/kafka.rs @dsmith3197 @vectordotdev/integrations-team
src/sources/kubernetes_logs/ @vectordotdev/integrations-team
src/sources/logstash.rs @neuronull @vectordotdev/integrations-team
src/sources/mongodb_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/nats.rs @StephenWakely @vectordotdev/integrations-team
src/sources/nginx_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/opentelemetry/ @vectordotdev/integrations-team
src/sources/postgresql_metrics.rs @dsmith3197 @vectordotdev/integrations-team
src/sources/prometheus/ @StephenWakely @vectordotdev/integrations-team # source_prometheus_remote_write,source_prometheus_scrape
src/sources/redis/ @StephenWakely @vectordotdev/integrations-team
src/sources/socket/ @neuronull @vectordotdev/integrations-team
src/sources/splunk_hec/ @StephenWakely @vectordotdev/integrations-team
src/sources/statsd/ @neuronull @vectordotdev/integrations-team
src/sources/syslog.rs @StephenWakely @vectordotdev/integrations-team
src/sources/vector/ @neuronull @vectordotdev/integrations-team
src/test_util/ @vectordotdev/core-team
src/topology/ @vectordotdev/core-team
src/transforms/ @vectordotdev/processing-team
website/ @vectordotdev/ux-team @vectordotdev/documentation
54 changes: 54 additions & 0 deletions .github/ISSUE_TEMPLATE/minor-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
name: Vector minor release
about: Use this template for a new minor release.
title: "Vector [version] release"
labels: "domain: releasing"
---

The week before the release:

- [ ] Check for any outstanding deprecation actions in [DEPRECATIONS.md](docs/DEPRECATIONS.md) and
take them (or have someone help you take them)
- [ ] Create a new release branch from master to freeze commits
- `git fetch && git checkout origin/master && git checkout -b v0.<new version number> && git push -u`
- [ ] Create a new release preparation branch from `master`
- `git checkout -b prepare-v0.<new version number> && git push -u`
- [ ] Check if there is a newer version of Alpine or Debian available to update the release images
in `distribution/docker/`. Update if so.
- [ ] Run `cargo vdev build release-cue` to generate a new cue file for the release
- [ ] Add description key to the generated cue file with a description of the release (see
previous releases for examples).
- [ ] Ensure any breaking changes are highlighted in the release upgrade guide
- [ ] Ensure any deprecations are highlighted in the release upgrade guide
- [ ] Update version number in `website/cue/reference/administration/interfaces/kubectl.cue`
- [ ] Update version number in `distribution/install.sh`
- [ ] Add new version to `website/cue/reference/versions.cue`
- [ ] Create new release md file by copying an existing one in `./website/content/en/releases/` and
updating version number
- [ ] Commit these changes
- [ ] Open PR against the release branch (`v0.<new version number>`) for review
- [ ] PR approval

On the day of release:

- [ ] Rebase the release preparation branch on the release branch
- [ ] Squash the release preparation commits (but not the cherry-picked commits!) to a single
commit. This makes it easier to cherry-pick to master after the release. 
- [ ] Ensure release date in cue matches current date.
- [ ] Merge release preparation branch into the release branch
- `git co v0.<new version number> && git merge --ff-only prepare-v0.<new version number>`
- [ ] Tag new release
- [ ] `git tag v0.<minor>.0 -a -m v0.<minor>.0`
- [ ] `git push origin v0.<minor>.0`
- [ ] Wait for release workflow to complete
- Discoverable via [https://github.com/timberio/vector/actions/workflows/release.yml](https://github.com/timberio/vector/actions/workflows/release.yml)
- [ ] Release Linux packages. See [`vector-release` usage](https://github.com/DataDog/vector-release#usage).
- [ ] Release updated Helm chart. See [releasing Helm chart](https://github.com/vectordotdev/helm-charts#releasing).
- [ ] Once Helm chart is released, updated Vector manifests
- Run `cargo vdev build manifests` and open a PR with changes
- [ ] Add docker images to [https://github.com/DataDog/images](https://github.com/DataDog/images/tree/master/vector) to have them available internally.
- [ ] Cherry-pick any release commits from the release branch that are not on `master`, to `master`
- [ ] Bump the release number in the `Cargo.toml` on master to the next major release
- [ ] Drop a note in the #websites Slack channel to request an update of the branch deployed
at https://vector.dev to the new release branch.
- [ ] Kick-off post-mortems for any regressions resolved by the release
54 changes: 54 additions & 0 deletions .github/ISSUE_TEMPLATE/patch-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
name: Vector patch release
about: Use this template for a new patch release.
title: "Vector [version] release"
labels: "domain: releasing"
---

Before the release:

- [ ] Create a new release preparation branch from the current release branch
- `git fetch && git checkout v0.<current minor version> && git checkout -b prepare-v0.<new version number>`
- [ ] Cherry-pick in all commits to be released from the associated release milestone
- If any merge conflicts occur, attempt to solve them and if needed enlist the aid of those familiar with the conflicting commits.
- [ ] Run `cargo vdev build release-cue` to generate a new cue file for the release
- [ ] Add `changelog` key to generated cue file
- [ ] `git log --no-merges --cherry-pick --right-only <last release tag>...`
- [ ] Should be hand-written list of changes
([example](https://github.com/vectordotdev/vector/blob/9fecdc8b5c45c613de2d01d4d2aee22be3a2e570/website/cue/reference/releases/0.19.0.cue#L44))
- [ ] Add description key to the generated cue file with a description of the release (see
previous releases for examples).
- [ ] Update version number in `distribution/install.sh`
- [ ] Add new version to `website/cue/reference/versions.cue`
- [ ] Create new release md file by copying an existing one in `./website/content/en/releases/` and
updating version number
- [ ] Bump the release number in the `Cargo.toml` to the current version number
- [ ] Run `cargo check` to regenerate `Cargo.lock` file
- [ ] Commit these changes
- [ ] Open PR against the release branch (`v0.<new version number>`) for review
- [ ] PR approval

On the day of release:

- [ ] Ensure release date in cue matches current date.
- [ ] Rebase the release preparation branch on the release branch
- Squash the release preparation commits (but not the cherry-picked commits!) to a single
commit. This makes it easier to cherry-pick to master after the release.
- `git checkout prepare-v0.<new version number> && git rebase -i v0.<current minor version>`
- [ ] Merge release preparation branch into the release branch
- `git co v0.<current minor version> && git merge --ff-only prepare-v0.<current minor version>.<patch>`
- [ ] Tag new release
- [ ] `git tag v0.<minor>.<patch> -a -m v0.<minor>.<patch>`
- [ ] `git push origin v0.<minor>.<patch>`
- [ ] Wait for release workflow to complete
- Discoverable via [https://github.com/timberio/vector/actions/workflows/release.yml](https://github.com/timberio/vector/actions/workflows/release.yml)
- [ ] Release Linux packages. See [`vector-release` usage](https://github.com/DataDog/vector-release#usage).
- [ ] Push the release branch to update the remote (This should close the preparation branch PR).
- `git checkout v0.<current minor version> && git push`
- [ ] Release updated Helm chart. See [releasing Helm chart](https://github.com/vectordotdev/helm-charts#releasing).
- [ ] Once Helm chart is released, updated Vector manifests
- Run `cargo vdev build manifests` and open a PR with changes
- [ ] Add docker images to [https://github.com/DataDog/images](https://github.com/DataDog/images/tree/master/vector) to have them available internally.
- Follow the [instructions at the top of the mirror.yaml file](https://github.com/DataDog/images/blob/fbf12868e90d52e513ebca0389610dea8a3c7e1a/mirror.yaml#L33-L49).
- [ ] Cherry-pick any release commits from the release branch that are not on `master`, to `master`
- [ ] Kick-off post-mortems for any regressions resolved by the release
106 changes: 104 additions & 2 deletions .github/SUPPORT.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,107 @@
# Support

Vector is a growing open source project with a variety of people willing to help you through various mediums.
Vector is a growing open source project with a variety of people willing to help
you through various mediums.

Take a look at those mediums listed at https://vector.dev/community
Take a look at those mediums listed at <https://vector.dev/community>

## How to ask a question about Vector

Whether in our community [Discord][discord] server or [GitHub Discussions][discussions], framing
your question well and providing the right level of details will improve your chances of getting
your question answered. Here are some tips:

### Before asking

Check the [Vector documentation](https://vector.dev/docs/) first to see if it answers your question.
If your question is about [VRL](https://vector.dev/docs/reference/vrl/#learn), you can also try out
the [VRL playground](vrl_playground).

If the docs do not answer your question, try using the search feature on [Discord][discord] or
[GitHub][vector], to search for keywords related to your issue. It is quite possible someone has
already asked your question before. This is especially useful if you have a specific error message
you are observing.

### Provide details

Essential details to include:

- Errors- for each error, please provide the full error message snippet, and
details such as where the error is observed, at what stage in the process
(e.g. at boot time, after some specific condition etc.).
- What is the version of Vector (and the Helm chart if deploying via Helm) and
the versions of any other systems in use (like Elasticsearch, NATS, etc.).
- What is your Vector configuration. See the below section on [how to format
your config](#formatting).
- How are you [deploying](https://vector.dev/docs/setup/deployment/) Vector?
- What is your complete deployment architecture? For example: I have Logstash
agents sending to Vector over syslog that is being forwarded to Loki.

Situation specific (not exhaustive):

- Did it occur after upgrading to a new version of Vector?
- Are you trying out Vector for the first time, or did you have a previous
working configuration?

These are just some examples of questions that may or may not apply to your
situation.

### Formatting

When providing snippets of configuration files, or log messages, format these with backticks to
improve the legibility for readers.

#### Blocks

Blocks should be formatted with three backticks (\`\`\`)

This should be used for any configuration snippets of Vector, Helm values, etc.
and for Vector console log messages.

See the [markdown documentation](https://www.markdownguide.org/basic-syntax/#fenced-code-blocks)
for an explanation.

For example:

[sinks.sink0]

inputs = ["source0"]

target = "stdout"

type = "console"

should be written as

```toml
[sinks.sink0]
inputs = ["source0"]
target = "stdout"
type = "console"
```

#### Single word or phrase

Formatting things like component names, versions etc. is done with single
backticks (\`) around the word or phrase.

This is less critical but also helps readability a lot and is greatly
appreciated.

See the [markdown documentation](https://www.markdownguide.org/basic-syntax/#code)
for an explanation.

For example:

> We upgraded from v24 to 25.1 and are seeing the following error output from
kafka.

should be written as

> We upgraded from `v0.24.0` to `v0.25.1` and are seeing the following error
output from the `kafka` sink.

[discord]: https://chat.vector.dev
[discussions]: https://github.com/vectordotdev/vector/discussions
[vector]: https://github.com/vectordotdev/vector
[vrl_playground]: https://playground.vrl.dev
Loading

0 comments on commit 4212e7e

Please sign in to comment.