Fluent Bit

OBSESC speaks the Fluent Forward protocol natively, so Fluent Bit and Fluentd can send to it with their built-in forward output. Add one [OUTPUT] block and keep your existing output as it is.

Endpoint

Setting	Value
Protocol	Fluent Forward (msgpack over TCP, not HTTP)
Default port	24224 (ingest.fluent_port)
Modes accepted	Forward, PackedForward, CompressedPackedForward (gzip)
Authentication	Shared-key handshake (security.fluent_shared_key)
Acknowledgements	Chunk acks, sent after the data is durable

The stock CloudFormation stack doesn’t open port 24224. Add an inbound security group rule from your shipper subnets first. See Network configuration.

Fluent Bit configuration

[SERVICE]
    Flush        1
    Log_Level    info

[INPUT]
    Name              tail
    Path              /var/log/app/*.log
    Tag               checkout
    # Assemble multiline events here. Without this, a Java stack trace
    # arrives as one event per line.
    multiline.parser  java

# OBSESC destination (added). Your existing [OUTPUT] stays below.
[OUTPUT]
    Name                  forward
    Match                 *
    Host                  obsesc.your-domain.internal
    Port                  24224
    # Ask for an ack per chunk so retries are safe (see below):
    Require_ack_response  On
    # When the node sets security.fluent_shared_key:
    # Shared_Key          ${OBSESC_FLUENT_SHARED_KEY}
    # Self_Hostname       my-host

# Existing destination, unchanged
# [OUTPUT]
#     Name   splunk
#     Match  *
#     ...

The tag is the service

The Fluent tag becomes the OBSESC service for every record in the frame. Tag deliberately, with one tag per service rather than one per file. If you tail several services with a wildcard input, use Tag_Regex or a rewrite_tag filter so that each service gets its own tag.

Records under a generic tag can still be split. Set ingest.service_from to an attribute key such as kubernetes.namespace_name, and it’ll be used for events that would otherwise have no service.

Fluentd configuration

<match **>
  @type forward
  require_ack_response true
  <server>
    host obsesc.your-domain.internal
    port 24224
  </server>
  # When the node sets security.fluent_shared_key:
  # <security>
  #   self_hostname my-host
  #   shared_key "#{ENV['OBSESC_FLUENT_SHARED_KEY']}"
  # </security>
</match>

To dual-write from Fluentd, wrap this and your existing <store> in a copy output.

How records are mapped

Forward field	OBSESC field
Tag	service
Entry time (integer seconds or EventTime with nanoseconds)	timestamp
message or log key	body
Every other key	Attributes. Nested maps are flattened to dotted keys, such as kubernetes.pod_name

With the Fluent Bit Kubernetes filter, kubernetes.host and kubernetes.namespace_name are also read into the built-in host and namespace dimensions.

Authentication

Set security.fluent_shared_key on the node (or put fluent_shared_key in the node’s Secrets Manager secret). Every connection must then complete the Forward protocol’s shared-key handshake before it can send events. The node doesn’t offer username and password authentication, so leave those fields empty on the client.

If no shared key is set, connections skip the handshake and anyone who can reach the port can send events. In that case, limit access to port 24224 with security groups.

For encryption in transit, configure TLS on the node (see Encryption) and turn on TLS in the output:

[OUTPUT]
    Name        forward
    Match       *
    Host        obsesc.your-domain.internal
    Port        24224
    tls         On
    tls.verify  On

Delivery and retries

• With Require_ack_response On, the node replies with {"ack": "<chunk>"} only after the chunk is durably stored. Under backpressure, while draining or on a write error, it holds the ack back, and Fluent Bit resends the chunk.
• A resent chunk is recognised as a retry and isn’t stored twice, provided it arrives soon after the original.
• Without acks, delivery is at-least-once, and a reconnect-and-resend can store a chunk twice.
• A malformed frame closes the connection. Fluent Bit reconnects and retries, which is its normal backpressure behaviour.

Alternative: Elasticsearch output

If you’d rather use HTTP, Fluent Bit’s es output also works against the Elasticsearch bulk listener on port 9200:

[OUTPUT]
    Name                es
    Match               *
    Host                obsesc.your-domain.internal
    Port                9200
    Index               logs
    Suppress_Type_Name  On

Fluent Bit’s es output can’t send an ApiKey or Bearer header, though. If the node enforces security.ingest_tokens, use forward with a shared key instead.

Node-side multiline (fallback)

Assemble multiline events in Fluent Bit wherever you can. If a source can’t have a multiline parser in front of it, the node can join continuation lines for Fluent Forward records itself (ingest.multiline.enabled: true). See Configuration examples for the trade-offs.