Convert logs from UTF-16 to UTF-8 before sending to New Relic Log API

[nedl86]

According to the documentation, the New Relic Log API requires UTF-8 encoding. Forwarding UTF-16 logs doesn't result in a failure, but inserts \u0000 (NULL) characters between each character due to the extra bits, making it impossible to query/parse the resulting logs in New Relic.

[GHMeyer0]

Hu, in the input settings there are two parameters "encoding" and "from_encoding", you could use them to do the conversion.

encoding UTF8
from_encoding UTF16

https://docs.fluentd.org/input/tail#encoding-from_encoding

[jdearana-nr]

@GHMeyer0 Currently, FluentBit does not natively support encoding conversions. FluentBit is designed to work with UTF-8 encoded data and doesn't have built-in capabilities for converting between other encoding types like UTF-16 and UTF-8. I assume the ask for this Issue is to provide a configurable feature in this output plugin to convert logs encoded in UTF-16 to UTF-8 before the logs are sent to New Relic.