One of the pods keeps crashing on startup and shows this in the logs "too many open files"

[bigdaddy5414]

`Fluent Bit v3.0.4

• Copyright (C) 2015-2024 The Fluent Bit Authors
• Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
• https://fluentbit.io
  . __ . __ ________
  _ ____/| | __ __ ____ / | _ _|/ | ___ _ \
  | __) | | | | _/ __ \ / \ __\ | | / \ _\ \ / / ( <
  | \ | || | /\ /| | \ | | | \ || | \ / /
  _ / |/____/ _ >| /| |____ /||| _/ /______ /
  / / / / /
  [2024/08/20 20:09:57] [ info] [fluent bit] version=3.0.4, commit=7de2c45227, pid=1
  [2024/08/20 20:09:57] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
  [2024/08/20 20:09:57] [ info] [cmetrics] version=0.9.0
  [2024/08/20 20:09:57] [ info] [ctraces ] version=0.5.1
  [2024/08/20 20:09:57] [ info] [input:tail:pod-logs-tailer] initializing
  [2024/08/20 20:09:57] [ info] [input:tail:pod-logs-tailer] storage_strategy='memory' (memory only)
  [2024/08/20 20:09:57] [ info] [input:tail:pod-logs-tailer] multiline core started
  [2024/08/20 20:09:57] [error] [/src/fluent-bit/plugins/in_tail/tail_fs_inotify.c:360 errno=24] Too many open files
  [2024/08/20 20:09:57] [error] failed initialize input tail.0
  [2024/08/20 20:09:57] [error] [engine] input initialization failed`

This is a brand new install using the guided Kubernetes install. It is fine on 5 out of 6 nodes it gets installed on.