Log sink is a configuration that exports logs to the outside or creates metrics.
Lobster
defines this configuration as a kubernetes custom resource and can be managed by user namespaces in multiple clusters.
You can create a LobsterSink
directly in kubernetes via kubectl
with yaml files or via the API-server of the Lobster operator
.
For APIs, please refer to the API documentation.
LobsterSink
defines types for log metrics/exports.
You can see how to use LobsterSink by type through the examples below.
Below is an example of creating a LobsterSink
called metric
in the log-test
namespace.
- The rule named
include-error-for-tc-container
counts logs containingerror
from logs produced bytc-container
in the pods labeledapp=sampleA
in thelog-test
namespace - The rule named
exclude-GET
counts logs produced by all containers in the pods labeledapp=sampleB
in thelog-test
namespace inclusterA and clusterB
, excludingGET
apiVersion: lobster.io/v1
kind: LobsterSink
metadata:
name: metric
namespace: log-test
spec:
type: logMetricRules
logMetricRules:
- name: include-error-for-tc-container
filter:
include: "error”
labels:
- app: sampleA
namespace: log-test
container: tc-container
- name: exclude-GET
filter:
clusters:
- clusterA
- clusterB
exclude: "GET”
labels:
- app: sampleB
namespace: log-test
Below is an example of creating a LobsterSink
called export
in the log-test
namespace.
- The rule named
include-error-for-tc-container
sends logs containingerror
from the logs produced bytc-container
among the pods labeledapp=sampleA
in thelog-test
namespace to{bucket destination}
every minute. If the bucket supports path configuration, the root path can be/
. - The rule named
exclude-GET
sends logs exceptGET
from the logs produced by all containers of the pods labeledapp=sampleB
in thelog-test
namespace ofclusterA and clusterB
to{bucket destination}
every hour. If the bucket supports path configuration, the root path can be/
. logMetricRules
supportsbasicBucket
(multi-part upload) ands3Bucket
apiVersion: lobster.io/v1
kind: LobsterSink
metadata:
name: export
namespace: log-test
spec:
name: match-labels-exclude
type: logMetricRules
logMetricRules:
- name: include-error-for-tc-container
interval: 1m
filter:
include: "error”
labels:
- app: sampleA
namespace: log-test
container: tc-container
basicBucket:
destination: http://{basicBucket endpoint}
rootPath: /
timeLayoutOfSubDirectory: {time layout of sub-directory; default `2006-01`}
- name: exclude-GET
interval: 1h
filter:
clusters:
- clusterA
- clusterB
exclude: "GET”
labels:
- app: sampleB
namespace: log-test
s3Bucket:
destination: http://{s3 bucket endpoint}
rootPath: /
timeLayoutOfSubDirectory: {time layout of sub-directory; default `2006-01`}
bucketName: {s3 bucket name}
accessKey: {s3 bucket access key}
secretKey: {s3 bucket secret key}
Tags: # supports multiple tags
{label1}: {value1}
{label2}: {value2}
...
LobsterSink
rule generated through Lobster operator
is synchronized to each cluster to perform log metrics/export.
Lobster operator
manages a custom resource calledLobsterSink
and has a built-in API serverLobster syncer
gets rules fromLobster operator
, and eachmatcher
andexporter
request a part of these rules based on the namespace where the latest log occurs- If you want to use these features in a single cluster, you can install and link
Lobster operator
andLobster syncer
in the same cluster
- If you want to use these features in a single cluster, you can install and link
matcher
compares the currently tailed logs with the metric rule and produces a Prometheus metric. Please refer to the metrics guide for the metric contents.matcher
operates as a module withinstore
andexporter
operates as a sidecar forstore
exporter
periodically sends logs as a file to external storage. Since information about transfer volume or transfer failure is produced as Prometheus metrics, please refer to the metrics guide for the metric contents- If a bucket has a directory structure, the directory can follow the rules below
{Root path}/{time layout of sub-directory}/{Namespace}/{LobsterSink name}/{Contents name}/{Pod}/{Container}/{Start time of log}_{End time of log}.log
- If a bucket has a directory structure, the directory can follow the rules below
time layout of sub-directory
is an option(default2006-01
) that sets the name of the sub-directory following{Root path}
to a time-based layout
Logs can be stored in a specific directory according to the rendering results of the layout, such as time or date- Here are some examples
now: 2009-11-10T23:00:00Z // You can separate the directories into small time chunks, or group them by day or month, as shown below // If you have a layout that represents time in seconds, // lobster will create directories in your bucket with names in seconds layout: 20060102150405 | result: 20091110230000 layout: 15:04:05 01-02-2006 | result: 23:00:00 11-10-2009 layout: 2006-01-02-15:04:05 | result: 2009-11-10-23:00:00 layout: 2006-01-02 15:04:05 | result: 2009-11-10 23:00:00 layout: 2006-01-02_15:04:05 | result: 2009-11-10_23:00:00 // If you have a layout that represents time in days, // lobster will create directories in your bucket with names in days layout: 2006010215 | result: 2009111023 layout: 15 01-02-2006 | result: 23 11-10-2009 layout: 2006-01-02-15 | result: 2009-11-10-23 layout: 2006-01-02 15 | result: 2009-11-10 23 layout: 2006-01-02_15 | result: 2009-11-10_23
- Please refer to the below documents for more information about the time layout
- Here are some examples