Vector

OBSESC implements Vector’s native gRPC protocol, so a Vector agent can send to it with the built-in vector sink. Events sent this way aren’t stored twice when Vector retries soon after a failed send.

Endpoint

Setting	Value
Protocol	Vector native, gRPC vector.Vector/PushEvents
Default port	9000 (ingest.vector_port)
Event types	Logs. Metric and trace events are accepted but skipped
Authentication	authorization: Bearer <token> metadata, checked against security.ingest_tokens

The stock CloudFormation stack doesn’t open port 9000. Add an inbound security group rule for your agents first. See Network configuration.

Configuration

[sources.app_logs]
type = "file"
include = ["/var/log/app/*.log"]

# Assemble multiline events in Vector, not downstream.
[sources.app_logs.multiline]
start_pattern = '^\d{4}-\d{2}-\d{2}'
mode = "halt_before"
condition_pattern = '^\d{4}-\d{2}-\d{2}'
timeout_ms = 1000

[transforms.tag_service]
type = "remap"
inputs = ["app_logs"]
source = '.service = "checkout"'

# OBSESC destination (added)
[sinks.obsesc]
type = "vector"
inputs = ["tag_service"]
address = "obsesc.your-domain.internal:9000"

# Existing sink, unchanged. Both sinks read the same input.
# [sinks.splunk]
# type = "splunk_hec_logs"
# inputs = ["tag_service"]
# ...

Set a service field

OBSESC takes the service from the event’s service field. If that’s missing, it uses the Vector source type (for example file or journald). If there’s no source type either, the event lands in ingest.default_service. Set .service in a remap transform, as shown above, so each application gets its own service.

How events are mapped

Vector field	OBSESC field
service	service
message (or log)	body
timestamp	timestamp. If it’s missing, the node’s receive time is used
Every other field	Attributes. Nested maps and arrays are flattened to dotted keys such as kubernetes.pod_namespace

Both current and older Vector agents are supported. Fields added by the kubernetes_logs source, such as kubernetes.pod_namespace and kubernetes.pod_node_name, are read into the built-in namespace and host dimensions.

Delivery guarantee

The vector sink gives each event a source_event_id that stays the same across retries. OBSESC uses it to drop a retried event it has already stored, so a retry that arrives soon after the original isn’t stored twice. A retry that arrives much later, or after the node restarts, may be. Events without that ID are at-least-once and never dropped.

The node acknowledges a batch only after it’s durably stored. Under backpressure, Vector’s sink retries automatically.

Authentication and TLS

Vector’s vector sink can’t attach an Authorization header. If the node enforces security.ingest_tokens, you have three options:

1. Use the Elasticsearch sink instead, which can send a token (below).
2. Use Vector’s loki sink with the Loki listener (ingest.loki), which accepts a bearer token.
3. Keep the native sink and limit who can reach port 9000 with security groups.

When TLS is configured on the node (see Encryption), the native listener serves TLS only. Turn it on in the sink:

[sinks.obsesc]
type = "vector"
inputs = ["tag_service"]
address = "obsesc.your-domain.internal:9000"
tls.enabled = true

Token-authenticated alternative: Elasticsearch sink

[sinks.obsesc]
type = "elasticsearch"
inputs = ["tag_service"]
endpoints = ["http://obsesc.your-domain.internal:9200"]
mode = "bulk"
bulk.index = "logs"
api_version = "v8"
request.headers.Authorization = "ApiKey ${OBSESC_INGEST_TOKEN}"

Events sent this way are at-least-once unless you set a stable document ID. See Elasticsearch bulk for the field mapping, including how service is chosen.

journald and Windows Event Log

Vector is the recommended shipper for sources the node has no listener for:

• journald: the journald source with a remap that sets .service from _SYSTEMD_UNIT.
• Windows Event Log: the windows_event_log source, with the channel or provider as the service.

Full configs are in Configuration examples.