Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CRD monitoring behaves unexpectedly #84

Closed
Antrakos opened this issue Aug 29, 2024 · 3 comments · Fixed by #85
Closed

CRD monitoring behaves unexpectedly #84

Antrakos opened this issue Aug 29, 2024 · 3 comments · Fixed by #85

Comments

@Antrakos
Copy link

Hi. We've been using robusta for quite some time already for tracking updates to deployments and stateful sets. With our platform expansion there's a need to monitor a few more resources, namely PodTemplate and flux HelmRelease.

In recent kubewatch release CRD monitoring support was added (#76), but I can't manage to make it work in my cluster. Chart version is 3.4.0 and image is v2.8.0. kubewatch is deployed to monitoring namespace.

Here's my initial config:

rbac:
  create: true
  customRoles:
    - apiGroups: [ "helm.toolkit.fluxcd.io" ]
      resources: [ "helmreleases" ]
      verbs: [ "get", "list", "watch" ]
    - apiGroups: [ "" ]
      resources: [ "podtemplates" ]
      verbs: [ "get", "list", "watch" ]
resourcesToWatch:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: false
  pod: false
  job: false
  node: false
  clusterrole: false
  clusterrolebinding: false
  serviceaccount: false
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false
  coreevent: false
  event: false
customresources:
  - group: helm.toolkit.fluxcd.io
    version: v2beta2
    resource: helmreleases
  - group: ''
    version: v1
    resource: podtemplates
slack:
  enabled: false
cloudevent:
  enabled: true
  url: "http://robusta-runner:80/api/handle"

With this configuration two controllers are started, but logs suggest that kubewatch didn't find any resources even though they do exist in multiple namespaces. There's also no logs when I make changes to any HelmRelease.

time="2024-08-29T06:19:20Z" level=info msg="Starting kubewatch controller" pkg=kubewatch-podtemplates
time="2024-08-29T06:19:20Z" level=info msg="Starting kubewatch controller" pkg=kubewatch-helmreleases
time="2024-08-29T06:19:20Z" level=info msg="Kubewatch controller synced and ready" pkg=kubewatch-helmreleases
time="2024-08-29T06:19:20Z" level=info msg="Kubewatch controller synced and ready" pkg=kubewatch-podtemplates

What was even more strange is when I tried to add PrometheusRule example from the docs to my config:

rbac:
  create: true
  customRoles:
    - apiGroups: [ "helm.toolkit.fluxcd.io" ]
      resources: [ "helmreleases" ]
      verbs: [ "get", "list", "watch" ]
    - apiGroups: [ "" ]
      resources: [ "podtemplates" ]
      verbs: [ "get", "list", "watch" ]
    - apiGroups: [ "monitoring.coreos.com" ]
      resources: [ "prometheusrules" ]
      verbs: [ "get", "list", "watch" ]
resourcesToWatch:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: false
  pod: false
  job: false
  node: false
  clusterrole: false
  clusterrolebinding: false
  serviceaccount: false
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false
  coreevent: false
  event: false
customresources:
  - group: helm.toolkit.fluxcd.io
    version: v2beta2
    resource: helmreleases
  - group: ''
    version: v1
    resource: podtemplates
  - group: monitoring.coreos.com
    version: v1
    resource: prometheusrules
slack:
  enabled: false
cloudevent:
  enabled: true
  url: "http://robusta-runner:80/api/handle"

With this config kubewatch started behaving unexpectedly. It found all prometheus rules, but then duplicated them for HelmRelease and PodTemplate.

time="2024-08-29T06:02:40Z" level=info msg="Starting kubewatch controller" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Starting kubewatch controller" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Starting kubewatch controller" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-alertmanager.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-config-reloaders" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-etcd" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-alertmanager.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-config-reloaders" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-etcd" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-general.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-general.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.container-cpu-usage-second" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.container-cpu-usage-second" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-cache" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-rss" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-swap" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-working-s" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-cache" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-rss" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.container-resource" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-k8s.rules.pod-owner" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-alertmanager.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-config-reloaders" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-etcd" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-general.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.container-cpu-usage-second" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-apiserver-availability.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-apiserver-burnrate.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-swap" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-working-s" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-cache" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.container-resource" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-rss" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-swap" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.container-memory-working-s" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-k8s.rules.pod-owner" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.container-resource" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-apiserver-availability.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-k8s.rules.pod-owner" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-apiserver-availability.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-apiserver-burnrate.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-apiserver-histogram.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-apiserver-burnrate.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-apiserver-histogram.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-apiserver-slos" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-prometheus-general.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-prometheus-node-recording.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-apiserver-slos" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-scheduler.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kube-state-metrics" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-prometheus-general.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubelet.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-apiserver-histogram.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-apiserver-slos" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-prometheus-general.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-prometheus-node-recording.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-scheduler.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-apps" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-resources" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-prometheus-node-recording.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-scheduler.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kube-state-metrics" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubelet.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-apps" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-storage" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-system" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-system-apiserver" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-system-controller-manager" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-resources" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-system-kube-proxy" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-storage" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-system-kubelet" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-system" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-system-apiserver" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-kubernetes-system-scheduler" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-system-controller-manager" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-node-exporter" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-system-kube-proxy" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-system-kubelet" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-kubernetes-system-scheduler" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-node-exporter.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-node-network" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-node-exporter" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-node-exporter.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-node-network" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-node.rules" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-node.rules" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-prometheus" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to podtemplates: monitoring/monitoring-prometheus-kube-prometheus-operator" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-prometheus" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to helmreleases: monitoring/monitoring-prometheus-kube-prometheus-operator" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kube-state-metrics" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubelet.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-apps" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-resources" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-storage" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-system" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-system-apiserver" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-system-controller-manager" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-system-kube-proxy" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-system-kubelet" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-kubernetes-system-scheduler" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-node-exporter" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-node-exporter.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-node-network" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-node.rules" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-prometheus" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Processing add to prometheusrules: monitoring/monitoring-prometheus-kube-prometheus-operator" pkg=kubewatch-prometheusrules
time="2024-08-29T06:02:40Z" level=info msg="Kubewatch controller synced and ready" pkg=kubewatch-helmreleases
time="2024-08-29T06:02:40Z" level=info msg="Kubewatch controller synced and ready" pkg=kubewatch-podtemplates
time="2024-08-29T06:02:40Z" level=info msg="Kubewatch controller synced and ready" pkg=kubewatch-prometheusrules

Moreover any changes to HelmRelease are ingnored, but when updating PrometheusRule all 3 controllers output change event:

time="2024-08-29T06:25:40Z" level=info msg="Processing update to helmreleases: monitoring/monitoring-prometheus-kube-config-reloaders" pkg=kubewatch-helmreleases
time="2024-08-29T06:25:40Z" level=info msg="Processing update to podtemplates: monitoring/monitoring-prometheus-kube-config-reloaders" pkg=kubewatch-podtemplates
time="2024-08-29T06:25:40Z" level=info msg="Processing update to prometheusrules: monitoring/monitoring-prometheus-kube-config-reloaders" pkg=kubewatch-prometheusrules
time="2024-08-29T06:25:40Z" level=info msg="Message successfully sent to http://robusta-runner:80/api/handle at 2024-08-29 06:25:40.137941159 +0000 UTC m=+77.421358676 "
time="2024-08-29T06:25:40Z" level=info msg="Message successfully sent to http://robusta-runner:80/api/handle at 2024-08-29 06:25:40.139041805 +0000 UTC m=+77.422459329 "
time="2024-08-29T06:25:40Z" level=info msg="Message successfully sent to http://robusta-runner:80/api/handle at 2024-08-29 06:25:40.140189514 +0000 UTC m=+77.423607027 "

@aantn @arikalon1 Could you help me investigate this? Did I misconfigure something or there's some concurrency issue in controllers?
@arikalon1
Copy link
Collaborator

Hey @Antrakos

thanks for reporting it
It does look like a bug related to monitoring multiple CRDs
We'll try to look into it soon

@arikalon1
Copy link
Collaborator

Hey @Antrakos

This image fixes the bug: us-central1-docker.pkg.dev/genuine-flight-317411/devel/kubewatch:arik-test-fixed
We'll release it soon

But, it's only kubewatch that supports CRDs, the runner doesn't support it yet.
So trying to use the runner to send those won't work.
At the moment, you can only send it directly from kubewatch

@arikalon1 arikalon1 mentioned this issue Aug 30, 2024
Merged
@aantn aantn closed this as completed in #85 Aug 30, 2024
@Antrakos
Copy link
Author

Thanks. I can confirm the issue is fixed. When do you plan to do a bugfix release?

Robusta does indeed fail to process events for these new resources. I filed a Github issue there as well robusta-dev/robusta#1536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fixed bug when multiple CRDs are configured. robusta-dev/kubewatch
2 participants
@Antrakos @arikalon1