Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collector not working when k8sattributes in use #35879

Open
clintonb opened this issue Oct 18, 2024 · 1 comment
Open

Collector not working when k8sattributes in use #35879

clintonb opened this issue Oct 18, 2024 · 1 comment
Labels
bug Something isn't working needs triage New item requiring triage processor/k8sattributes k8s Attributes processor

Comments

@clintonb
Copy link

Component(s)

processor/k8sattributes

What happened?

Description

I am trying to add k8sattributes to a gateway collector, but the collector and health check are not functioning. The collector appears to start, but refuses connections on the receiving ports. The health check endpoint returns a 503 with {"status":"Server not available","upSince":"0001-01-01T00:00:00Z","uptime":""}.

Steps to Reproduce

  1. Build an image with the configuration below.
  2. Run it.

Expected Result

  1. Collector responds to health checks at curl -v http://localhost:13133.
  2. Received traces include k8s attributes.

Actual Result

The collector never becomes healthy, and does not accept any signals.

Collector version

0.111.0

Environment information

Environment

OS: macOS 15.0.1 (Docker), and GKE Autopilot

OpenTelemetry Collector configuration

extensions:
  health_check:
    endpoint: "0.0.0.0:13133"

receivers:
  otlp:
    protocols:
      http:
        endpoint: "0.0.0.0:4318"
        include_metadata: true

processors:
  k8sattributes:
    auth_type: "serviceAccount"
    extract:
      metadata:
#        - k8s.namespace.name
#        - k8s.deployment.name
#        - k8s.statefulset.name
#        - k8s.daemonset.name
#        - k8s.cronjob.name
#        - k8s.job.name
#        - k8s.node.name
        - k8s.pod.name
#        - k8s.pod.uid
#        - k8s.pod.start_time
    passthrough: false
#    pod_association:
#      - sources:
#          - from: resource_attribute
#            name: k8s.pod.ip
#      - sources:
#          - from: resource_attribute
#            name: k8s.pod.uid
#      - sources:
#          - from: connection

exporters:
  # NOTE: Add this to the list of pipeline exporters to see the collector's debug logs
  debug:
    verbosity: detailed
service:
  extensions: [ health_check ]
  # https://opentelemetry.io/docs/collector/configuration/#telemetry
  telemetry:
    # This controls log verbosity of the collector itself.
    logs:
      encoding: json
      level: "debug"
  pipelines:
    traces:
      receivers: [ otlp ]
      processors: [ k8sattributes ]
      exporters: [ debug ]

Log output

otel-collector-1  | {"level":"info","ts":1729289848.7740626,"caller":"[email protected]/service.go:136","msg":"Setting up own telemetry..."}
otel-collector-1  | {"level":"info","ts":1729289848.7742362,"caller":"telemetry/metrics.go:70","msg":"Serving metrics","address":"localhost:8888","metrics level":"Normal"}
otel-collector-1  | {"level":"info","ts":1729289848.7743702,"caller":"builders/builders.go:26","msg":"Development component. May change in the future.","kind":"exporter","data_type":"traces","name":"debug"}
otel-collector-1  | {"level":"debug","ts":1729289848.774658,"caller":"builders/builders.go:24","msg":"Beta component. May change in the future.","kind":"processor","name":"k8sattributes","pipeline":"traces"}
otel-collector-1  | {"level":"debug","ts":1729289848.774745,"caller":"builders/builders.go:24","msg":"Stable component.","kind":"receiver","name":"otlp","data_type":"traces"}
otel-collector-1  | {"level":"debug","ts":1729289848.7748547,"caller":"builders/extension.go:48","msg":"Beta component. May change in the future.","kind":"extension","name":"health_check"}
otel-collector-1  | {"level":"info","ts":1729289848.7753859,"caller":"[email protected]/service.go:208","msg":"Starting otelcol-contrib...","Version":"0.111.0","NumCPU":16}
otel-collector-1  | {"level":"info","ts":1729289848.775412,"caller":"extensions/extensions.go:39","msg":"Starting extensions..."}
otel-collector-1  | {"level":"info","ts":1729289848.775442,"caller":"extensions/extensions.go:42","msg":"Extension is starting...","kind":"extension","name":"health_check"}
otel-collector-1  | {"level":"info","ts":1729289848.7754776,"caller":"[email protected]/healthcheckextension.go:33","msg":"Starting health_check extension","kind":"extension","name":"health_check","config":{"Endpoint":"0.0.0.0:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":{"Enabled":false,"Interval":"5m","ExporterFailureThreshold":5}}}
otel-collector-1  | {"level":"warn","ts":1729289848.7757652,"caller":"[email protected]/warning.go:40","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.","kind":"extension","name":"health_check","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
otel-collector-1  | {"level":"info","ts":1729289848.7758508,"caller":"extensions/extensions.go:59","msg":"Extension started.","kind":"extension","name":"health_check"}


### Additional context

I run `curl -v http://localhost:13133` to check if the collector is healthy.
@clintonb clintonb added bug Something isn't working needs triage New item requiring triage labels Oct 18, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the processor/k8sattributes k8s Attributes processor label Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage New item requiring triage processor/k8sattributes k8s Attributes processor
Projects
None yet
Development

No branches or pull requests

1 participant