FluentBit: add dynamic es index - fluent-bit

Question

I have a working fluent-bit:1.7
I need OUTPUT to Elasticsearch and create a dynamic index based on the k8s label = name.
I want the following convention for the index:
infra-${app_name}-yyyy.mm.dd
Example: infra-mongodb-2021.01.01, infra-postgresql-2021.01.01, infra-kafka-2021.01.01 etc...
This is my FILTER and OUTPUT config:
[FILTER]
Name kubernetes
Match kube.*
Merge_Log Off
Keep_Log Off
[OUTPUT]
Name es
Match kube.*
Host ${ES_HOST}
Logstash_Format On
Logstash_Prefix_Key kubernetes['labels']['name']
But it generates the following index: mongodb-2021.01.01
It's almost done, I just need to always add the infra- prefix.
Just to be clear, I need something like:
Logstash_Prefix_Key infra-${kubernetes['labels']['name']}

Answer 1

I did this using Lua plugin to create a field in the record with the name of the index then using this field as Logstash_Prefix_Key
Lua script (based on https://github.com/fluent/fluent-bit/blob/master/scripts/append_tag.lua):
function append_es_index(tag, timestamp, record)
new_record = record
if (record["cluster_name"] ~= nil) then
es_index = record["cluster_name"]
else
es_index = "k8s"
end
if (record["kubernetes"] ~= nil) then
kube = record["kubernetes"]
if (kube["namespace_name"] ~= nil and string.len(kube["namespace_name"]) > 0) then
es_index = es_index .. "." .. kube["namespace_name"]
end
if (kube["labels"] ~= nil) then
labels = kube["labels"]
if (labels["app"] ~= nil and string.len(labels["app"]) > 0) then
es_index = es_index .. "." .. labels["app"]
elseif (labels["k8s-app"] ~= nil and string.len(labels["k8s-app"]) > 0) then
es_index = es_index .. "." .. labels["k8s-app"]
elseif (labels["name"] ~= nil and string.len(labels["name"]) > 0) then
es_index = es_index .. "." .. labels["name"]
end
end
end
new_record["es_index"] = es_index
return 1, timestamp, new_record
end
Fluentbit filters config:
[FILTER]
Name kubernetes
...
[FILTER]
Name record_modifier
Match *
Record cluster_name my-test-cluster
[FILTER]
Name lua
Match *
script /fluent-bit/scripts/append_es_index.lua
call append_es_index

Answer 2

Another way to achieve similar result is to create a label or an annotation for your Pods
for example
[OUTPUT]
Name es
Logstash_Prefix_Key kubernetes['labels']['log-key']
...
deployment yml based on docs
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
log-key: infra-nginx # log-key label will be used in Logstash_Prefix_Key
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
this way you will be able to combine multiple "dynamic" data for example also the namespace