filter_log_to_metrics: Add setting "label" #7739

wndhydrnt · 2023-07-23T16:59:05Z

Add setting label to filter "Log to Metrics". It allows a user to define a custom label name and a key from which the value of the label is read.

Addresses #7737.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

Example configuration file for the change
Debug log output from testing the change

Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

[N/A] Run local packaging test showing all targets (including any new ones) build.
[N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

Documentation required for this feature

fluent/fluent-bit-docs#1167

Backporting

[N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

wndhydrnt · 2023-07-23T17:20:11Z

Configuration:

[SERVICE]
    Flush     1
    Log_Level debug

[INPUT]
    Name  dummy
    Dummy {"message":"This is a test","kubernetes":{"labels":{"app":"testservice"}}}

[FILTER]
    Name               log_to_metrics
    Match              *
    Tag                test_metric
    Label              app $kubernetes['labels']['app']
    Metric_mode        counter
    Metric_name        count_all_dummy_messages
    Metric_description This metric counts dummy messages

[OUTPUT]
    Name  stdout
    Match *

[OUTPUT]
    Name  prometheus_exporter
    Match test_metric
    Host  0.0.0.0
    Port  2021

wndhydrnt · 2023-07-23T17:21:53Z

Debug output:

fluent-bit@35b0d2ad42e2:/workspaces/fluent-bit/build$ ./bin/fluent-bit -c /fluent-bit/etc/fluent-bit.conf
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/07/23 17:19:17] [ info] Configuration:
[2023/07/23 17:19:17] [ info]  flush time     | 1.000000 seconds
[2023/07/23 17:19:17] [ info]  grace          | 5 seconds
[2023/07/23 17:19:17] [ info]  daemon         | 0
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  inputs:
[2023/07/23 17:19:17] [ info]      dummy
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  filters:
[2023/07/23 17:19:17] [ info]      log_to_metrics.0
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  outputs:
[2023/07/23 17:19:17] [ info]      stdout.0
[2023/07/23 17:19:17] [ info]      prometheus_exporter.1
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  collectors:
[2023/07/23 17:19:17] [ info] [fluent bit] version=2.1.8, commit=, pid=11199
[2023/07/23 17:19:17] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2023/07/23 17:19:17] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/07/23 17:19:17] [ info] [cmetrics] version=0.6.3
[2023/07/23 17:19:17] [ info] [ctraces ] version=0.3.1
[2023/07/23 17:19:17] [ info] [input:dummy:dummy.0] initializing
[2023/07/23 17:19:17] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/07/23 17:19:17] [debug] [dummy:dummy.0] created event channels: read=21 write=22
[2023/07/23 17:19:17] [ info] [input:emitter:emitter.1] initializing
[2023/07/23 17:19:17] [ info] [input:emitter:emitter.1] storage_strategy='memory' (memory only)
[2023/07/23 17:19:17] [debug] [emitter:emitter.1] created event channels: read=23 write=24
[2023/07/23 17:19:17] [debug] [stdout:stdout.0] created event channels: read=25 write=26
[2023/07/23 17:19:17] [debug] [prometheus_exporter:prometheus_exporter.1] created event channels: read=32 write=33
[2023/07/23 17:19:17] [ info] [output:stdout:stdout.0] worker #0 started
[2023/07/23 17:19:17] [ info] [output:prometheus_exporter:prometheus_exporter.1] listening iface=0.0.0.0 tcp_port=2021
[2023/07/23 17:19:17] [debug] [router] match rule dummy.0:stdout.0
[2023/07/23 17:19:17] [debug] [router] match rule emitter.1:stdout.0
[2023/07/23 17:19:17] [ info] [sp] stream processor started
[2023/07/23 17:19:18] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:18] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[0] dummy.0: [[1690132758.358001759, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:19] [debug] [task] created task=0xffff98038910 id=0 OK
[2023/07/23 17:19:19] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:19:19] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:19] [debug] [task] created task=0xffff98038990 id=1 OK
[2023/07/23 17:19:19] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[2023/07/23 17:19:19] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:19] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:19] [debug] [task] destroy task=0xffff98038910 (task_id=0)
2023-07-23T17:19:18.358371342Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 1
[2023/07/23 17:19:19] [debug] [out flush] cb_destroy coro_id=1
[2023/07/23 17:19:19] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:19:19] [debug] [task] destroy task=0xffff98038990 (task_id=1)
[0] dummy.0: [[1690132759.358301218, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:20] [debug] [task] created task=0xffff98075ad0 id=0 OK
[2023/07/23 17:19:20] [debug] [out flush] cb_destroy coro_id=2
[2023/07/23 17:19:20] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:20] [debug] [task] created task=0xffff98075b50 id=1 OK
[2023/07/23 17:19:20] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
2023-07-23T17:19:19.358409093Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 2
[2023/07/23 17:19:20] [debug] [out flush] cb_destroy coro_id=3
[2023/07/23 17:19:20] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:20] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:20] [debug] [task] destroy task=0xffff98075ad0 (task_id=0)
[2023/07/23 17:19:20] [debug] [out flush] cb_destroy coro_id=1
[2023/07/23 17:19:20] [debug] [task] destroy task=0xffff98075b50 (task_id=1)
[2023/07/23 17:19:21] [debug] [task] created task=0xffff98045950 id=0 OK
[2023/07/23 17:19:21] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:21] [debug] [task] created task=0xffff980459d0 id=1 OK
[2023/07/23 17:19:21] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[0] dummy.0: [[1690132760.357341760, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
2023-07-23T17:19:20.357457968Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 3
[2023/07/23 17:19:21] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:21] [debug] [out flush] cb_destroy coro_id=4
[2023/07/23 17:19:21] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:21] [debug] [out flush] cb_destroy coro_id=5
[2023/07/23 17:19:21] [debug] [out flush] cb_destroy coro_id=2
[2023/07/23 17:19:21] [debug] [task] destroy task=0xffff98045950 (task_id=0)
[2023/07/23 17:19:21] [debug] [task] destroy task=0xffff980459d0 (task_id=1)
[0] dummy.0: [[1690132761.355765760, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:22] [debug] [task] created task=0xffff980448d0 id=0 OK
[2023/07/23 17:19:22] [debug] [out flush] cb_destroy coro_id=6
[2023/07/23 17:19:22] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:22] [debug] [task] created task=0xffff980459d0 id=1 OK
[2023/07/23 17:19:22] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
2023-07-23T17:19:21.356098510Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 4
[2023/07/23 17:19:22] [debug] [out flush] cb_destroy coro_id=7
[2023/07/23 17:19:22] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:22] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:22] [debug] [task] destroy task=0xffff980448d0 (task_id=0)
[2023/07/23 17:19:22] [debug] [out flush] cb_destroy coro_id=3
[2023/07/23 17:19:22] [debug] [task] destroy task=0xffff980459d0 (task_id=1)
[2023/07/23 17:19:23] [debug] [task] created task=0xffff980459d0 id=0 OK
[2023/07/23 17:19:23] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] dummy.0: [[1690132762.358517552, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:23] [debug] [out flush] cb_destroy coro_id=8
2023-07-23T17:19:22.358604219Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 5
[2023/07/23 17:19:23] [debug] [task] created task=0xffff98045f20 id=1 OK
[2023/07/23 17:19:23] [debug] [out flush] cb_destroy coro_id=9
[2023/07/23 17:19:23] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[2023/07/23 17:19:23] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:23] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:23] [debug] [task] destroy task=0xffff980459d0 (task_id=0)
[2023/07/23 17:19:23] [debug] [out flush] cb_destroy coro_id=4
[2023/07/23 17:19:23] [debug] [task] destroy task=0xffff98045f20 (task_id=1)
^C[2023/07/23 17:19:24] [engine] caught signal (SIGINT)
[2023/07/23 17:19:24] [debug] [task] created task=0xffff980459d0 id=0 OK
[2023/07/23 17:19:24] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:24] [debug] [task] created task=0xffff98045e70 id=1 OK
[2023/07/23 17:19:24] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[0] dummy.0: [[1690132763.360203886, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
2023-07-23T17:19:23.360298136Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 6
[2023/07/23 17:19:24] [ warn] [engine] service will shutdown in max 5 seconds
[2023/07/23 17:19:24] [debug] [out flush] cb_destroy coro_id=10
[2023/07/23 17:19:24] [ info] [input] pausing dummy.0
[2023/07/23 17:19:24] [debug] [out flush] cb_destroy coro_id=11
[2023/07/23 17:19:24] [debug] [task] destroy task=0xffff980459d0 (task_id=0)
[2023/07/23 17:19:24] [debug] [out flush] cb_destroy coro_id=5
[2023/07/23 17:19:24] [debug] [task] destroy task=0xffff98045e70 (task_id=1)
[2023/07/23 17:19:24] [ info] [engine] service has stopped (0 pending tasks)
[2023/07/23 17:19:24] [ info] [input] pausing dummy.0
[2023/07/23 17:19:24] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/07/23 17:19:24] [ info] [output:stdout:stdout.0] thread worker #0 stopped

wndhydrnt · 2023-07-23T17:25:50Z

Valgrind output:

$ valgrind --leak-check=full ./bin/fluent-bit -c ../.dev/fluent-bit.conf
==11206== Memcheck, a memory error detector
==11206== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==11206== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==11206== Command: ./bin/fluent-bit -c ../.dev/fluent-bit.conf
==11206==
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/07/23 17:24:23] [ info] Configuration:
[2023/07/23 17:24:23] [ info]  flush time     | 1.000000 seconds
[2023/07/23 17:24:23] [ info]  grace          | 5 seconds
[2023/07/23 17:24:23] [ info]  daemon         | 0
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  inputs:
[2023/07/23 17:24:23] [ info]      dummy
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  filters:
[2023/07/23 17:24:23] [ info]      log_to_metrics.0
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  outputs:
[2023/07/23 17:24:23] [ info]      stdout.0
[2023/07/23 17:24:23] [ info]      prometheus_exporter.1
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  collectors:
[2023/07/23 17:24:23] [ info] [fluent bit] version=2.1.8, commit=, pid=11206
[2023/07/23 17:24:23] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2023/07/23 17:24:23] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/07/23 17:24:23] [ info] [output:stdout:stdout.0] worker #0 started
[2023/07/23 17:24:23] [ info] [cmetrics] version=0.6.3
[2023/07/23 17:24:23] [ info] [ctraces ] version=0.3.1
[2023/07/23 17:24:23] [ info] [input:dummy:dummy.0] initializing
[2023/07/23 17:24:23] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/07/23 17:24:23] [debug] [dummy:dummy.0] created event channels: read=21 write=22
[2023/07/23 17:24:23] [ info] [input:emitter:emitter.1] initializing
[2023/07/23 17:24:23] [ info] [input:emitter:emitter.1] storage_strategy='memory' (memory only)
[2023/07/23 17:24:23] [debug] [emitter:emitter.1] created event channels: read=23 write=24
[2023/07/23 17:24:23] [debug] [stdout:stdout.0] created event channels: read=25 write=26
[2023/07/23 17:24:23] [debug] [prometheus_exporter:prometheus_exporter.1] created event channels: read=32 write=33
[2023/07/23 17:24:24] [ info] [output:prometheus_exporter:prometheus_exporter.1] listening iface=0.0.0.0 tcp_port=2021
[2023/07/23 17:24:24] [debug] [router] match rule dummy.0:stdout.0
[2023/07/23 17:24:24] [debug] [router] match rule emitter.1:stdout.0
[2023/07/23 17:24:24] [ info] [sp] stream processor started
[2023/07/23 17:24:24] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:24:24] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:24:25] [debug] [task] created task=0x52d0a30 id=0 OK
[0] dummy.0: [[1690133064.382956803, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
2023-07-23T17:24:24.424676637Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 1
[2023/07/23 17:24:25] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:24:25] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:24:25] [debug] [task] created task=0x52d0e80 id=1 OK
[2023/07/23 17:24:25] [debug] [out flush] cb_destroy coro_id=1
[2023/07/23 17:24:25] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[2023/07/23 17:24:25] [debug] [task] destroy task=0x52d0a30 (task_id=0)
[2023/07/23 17:24:25] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:24:25] [debug] [task] destroy task=0x52d0e80 (task_id=1)
^C[2023/07/23 17:24:28] [engine] caught signal (SIGINT)
[2023/07/23 17:24:28] [ warn] [engine] service will shutdown in max 5 seconds
[2023/07/23 17:24:28] [ info] [input] pausing dummy.0
[2023/07/23 17:24:29] [ info] [engine] service has stopped (0 pending tasks)
[2023/07/23 17:24:29] [ info] [input] pausing dummy.0
[2023/07/23 17:24:29] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/07/23 17:24:29] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==11206==
==11206== HEAP SUMMARY:
==11206==     in use at exit: 0 bytes in 0 blocks
==11206==   total heap usage: 2,582 allocs, 2,582 frees, 1,950,769 bytes allocated
==11206==
==11206== All heap blocks were freed -- no leaks are possible
==11206==
==11206== For lists of detected and suppressed errors, rerun with: -s
==11206== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Signed-off-by: Markus Meyer <[email protected]>

edsiper · 2023-09-23T17:36:17Z

merging this PR, note that I will rename label option to add_label to be compliant with other plugins that performs the same operation

wndhydrnt · 2023-09-24T11:02:48Z

Thank you for the review and the merge @edsiper. Unfortunately, some lines were missed in #7964. I have created #7967 to address the issue. Please take a look.

fluent/fluent-bit-docs#1167 has been updated as well to reflect the rename.

--------- Signed-off-by: Markus Meyer <[email protected]>

github-actions bot added the docs-required label Jul 23, 2023

wndhydrnt temporarily deployed to pr July 23, 2023 16:59 — with GitHub Actions Inactive

wndhydrnt added 4 commits July 23, 2023 19:27

Fix formatting

d69485b

Signed-off-by: Markus Meyer <[email protected]>

Implement

bdbd88f

Signed-off-by: Markus Meyer <[email protected]>

Simplify test

18d60e1

Signed-off-by: Markus Meyer <[email protected]>

Fix issues highlighted by valgrind

ed60c66

Signed-off-by: Markus Meyer <[email protected]>

wndhydrnt force-pushed the log_to_metrics-label-key branch from 5806941 to ed60c66 Compare July 23, 2023 17:27

wndhydrnt temporarily deployed to pr July 23, 2023 17:27 — with GitHub Actions Inactive

wndhydrnt temporarily deployed to pr July 23, 2023 17:53 — with GitHub Actions Inactive

wndhydrnt mentioned this pull request Jul 23, 2023

filter_log_to_metrics: Add documentation for add_label fluent/fluent-bit-docs#1167

Merged

wndhydrnt marked this pull request as ready for review July 23, 2023 18:02

wndhydrnt requested review from edsiper, leonardo-albertovich, fujimotos and koleini as code owners July 23, 2023 18:02

edsiper merged commit b18587b into fluent:master Sep 23, 2023
41 of 43 checks passed

wndhydrnt deleted the log_to_metrics-label-key branch September 24, 2023 10:10

wndhydrnt mentioned this pull request Sep 24, 2023

filter_log_to_metrics: Fix config key after rename #7967

Open

4 tasks

Porkepix mentioned this pull request Sep 28, 2023

fluent-bit 2.1.10 Homebrew/homebrew-core#146545

Merged

leonardo-albertovich pushed a commit that referenced this pull request Oct 5, 2023

filter_log_to_metrics: Add setting "label" (#7739)

43ebe08

--------- Signed-off-by: Markus Meyer <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

filter_log_to_metrics: Add setting "label" #7739

filter_log_to_metrics: Add setting "label" #7739

wndhydrnt commented Jul 23, 2023 •

edited

Loading

wndhydrnt commented Jul 23, 2023

wndhydrnt commented Jul 23, 2023 •

edited

Loading

wndhydrnt commented Jul 23, 2023

edsiper commented Sep 23, 2023

wndhydrnt commented Sep 24, 2023

filter_log_to_metrics: Add setting "label" #7739

filter_log_to_metrics: Add setting "label" #7739

Conversation

wndhydrnt commented Jul 23, 2023 • edited Loading

wndhydrnt commented Jul 23, 2023

wndhydrnt commented Jul 23, 2023 • edited Loading

wndhydrnt commented Jul 23, 2023

edsiper commented Sep 23, 2023

wndhydrnt commented Sep 24, 2023

wndhydrnt commented Jul 23, 2023 •

edited

Loading

wndhydrnt commented Jul 23, 2023 •

edited

Loading