Configuration reference

An OBSESC node reads a single YAML file. On a Marketplace deployment this is /etc/obsesc/config.yaml, which the CloudFormation stack writes on first boot from your stack parameters. Most deployments only need the stack parameters; this page lists the keys you’re most likely to set yourself, with their defaults.

How configuration is loaded

Values are resolved in this order, and each layer overrides the one before it:

1. Built-in defaults.
2. The YAML file, /etc/obsesc/config.yaml on the AMI.
3. Environment variables prefixed OBSESC_, with __ separating nested keys.

# Equivalent to security.query_api_token in YAML
OBSESC_SECURITY__QUERY_API_TOKEN=...

# Equivalent to storage.retention_window_days
OBSESC_STORAGE__RETENTION_WINDOW_DAYS=365

On the AMI, the node also reads /etc/obsesc/obsesc-node.env if that file exists. It’s a good place for environment overrides.

Unknown keys are rejected. A misspelt key stops the node from starting, and the error names the key. If you’re unsure about a change, contact us via the contact page before you restart.

The stack also sets local data paths and other settings that you shouldn’t change. Leave any key that isn’t on this page as the stack wrote it.

ingest

Listener ports

Key	Default	Protocol
ingest.otlp_http_port	4318	OTLP/HTTP (/v1/logs, /v1/traces), protobuf only
ingest.otlp_grpc_port	4317	OTLP/gRPC
ingest.es_bulk_port	9200	Elasticsearch _bulk
ingest.hec_port	8088	Splunk HTTP Event Collector (HEC)
ingest.fluent_port	24224	Fluent Forward
ingest.vector_port	9000	Vector native gRPC
ingest.query_api_port	18080 (set by the stack)	Web console and API

Event handling

Key	Default	Description
ingest.default_service	unknown	Service name given to events whose shipper didn’t supply one
ingest.service_from	unset	Attribute to take the service name from when the shipper didn’t supply one, for example kubernetes.namespace_name
ingest.max_event_bytes	8 MiB	Size cap per event. Any batch containing an oversized event is rejected. 0 disables the cap
ingest.max_attrs_per_event	4096	Attribute-count cap per event. 0 disables the cap
ingest.max_future_skew_secs	604800 (7 days)	How far ahead of the current time a timestamp may be. 0 disables the check
ingest.max_event_age_secs	0 (off)	How old a timestamp may be
ingest.timestamp_action	clamp	What to do with an out-of-range timestamp: clamp moves it to the nearest boundary, reject refuses the batch
ingest.max_inflight_bytes	256 MiB	Total request body bytes being processed across all HTTP listeners at once. Requests over this get 503 with Retry-After
ingest.dedup_window_events	built in	Retries that carry a stable event ID are de-duplicated within a recent window. The window is counted in events, so it covers less time as volume rises. 0 disables de-duplication
ingest.hec_max_body_bytes	16 MiB	Maximum HEC request body
ingest.redaction	[]	Redaction rules applied before anything is written. Each rule sets field (an attribute key) or regex, plus a replacement
ingest.dimensions.enabled	true	Extracts the host, env, namespace and tenant dimensions
ingest.multiline.enabled	false	Assembles multi-line events on the node, for HEC /raw and Fluent Forward. Assembling in the shipper is preferred where possible

ingest:
  redaction:
    - field: "user.email"
      replacement: "[redacted]"
    - regex: "\\b\\d{3}-\\d{2}-\\d{4}\\b"
      replacement: "***-**-****"

Pull and AWS-native sources

These are all off by default. Each one is a subsection of ingest. Setup is covered in Supported sources.

Section	Key settings (defaults)
ingest.firehose	enabled, port (8555), access_key (required; requests without it are refused), max_body_bytes (64 MiB), service
ingest.loki	enabled, port (3100), max_body_bytes (32 MiB)
ingest.gelf	enabled, port (12201), udp (true), tcp (true), max_message_bytes (8 MiB)
ingest.kinesis	enabled, stream_name, region, start (latest or earliest)
ingest.sqs	enabled, queue_url, region, max_object_bytes (256 MiB)
ingest.kafka	enabled, brokers, topic, partitions, start, tls, sasl_mechanism (plain, scram-sha-256 or scram-sha-512), sasl_username, sasl_password

The pull sources have further polling settings with sensible defaults. Contact us via the contact page if you need to change them.

To backfill historical archives from S3, contact us via the contact page.

storage

Key	Default	Description
storage.s3_bucket	none (required)	S3 bucket for your raw events
storage.s3_prefix	raw/	Key prefix for raw events
storage.s3_sse_kms_key_arn	unset	KMS key used for SSE-KMS on everything OBSESC writes to the bucket
storage.s3_region	the instance’s Region	Set this when the bucket is in another Region
storage.assume_role_arn	unset	IAM role the node assumes for S3 access, used for buckets in another account
storage.expected_bucket_owner	looked up at startup	The 12-digit account ID that must own the bucket. S3 refuses requests to a bucket owned by any other account
storage.retention_window_days	unset (keep forever)	Global retention, measured by event time
storage.lifecycle	unset	Tells OBSESC your bucket’s lifecycle rules: standard_ia_after_days and glacier_ir_after_days
storage.object_lock	unset	mode (governance or compliance) and retain_days. The node won’t start if the bucket doesn’t have Object Lock enabled

retention

Per-service retention rules, layered on top of storage.retention_window_days. See Storage and formats and Compliance.

retention:
  min_window_days: 90          # compliance floor; a config below it will not start
  policies:
    - match: { service_glob: "audit-*" }
      retention_window_days: 2555
    - match: { service_glob: "debug-*" }
      retention_window_days: 90

When several rules match a service, the most specific one wins. Raw files hold events from many services, so each raw file is kept until the longest applicable retention period has passed. Legal holds override every retention rule.

Erasure

Erasure is off by default and must be turned on before you can use it. Contact us via the contact page to enable erasure before you need it.

Custody verification

Custody verification is optional and off by default. When it’s on, OBSESC keeps a cryptographic record of the history it stores, and you can check in the console that it hasn’t been altered. It doesn’t cover the raw events in your S3 bucket; use S3 Object Lock to protect those.

To turn it on, set the EvaluationFeatures stack parameter to Enabled. This also turns on other optional console features. Changing this parameter on an existing stack replaces the node, so plan the update. If you want custody verification on its own, contact us via the contact page. See Audit and verification.

security

Key	Default	Description
security.query_api_token	unset	Bearer token for /v1/*
security.ingest_tokens	[]	Tokens accepted by OTLP, Elasticsearch _bulk, Loki and Vector. An empty list means ingest is unauthenticated
security.hec_tokens	[]	Splunk HEC tokens. An empty list rejects every HEC request
security.fluent_shared_key	unset	Shared key for the Fluent Forward handshake
security.secrets_manager_secret_arn	unset	Secrets Manager secret holding a JSON object with any of query_api_token, hec_tokens, ingest_tokens and fluent_shared_key. Values in the secret override the file
security.tls.cert_path / key_path	unset	PEM certificate and key. When set, every TCP listener serves TLS only. GELF over UDP is not encrypted
security.tls.reload_interval_secs	unset	Reloads certificates from disk on this interval, without a restart
security.authz	unset	Role-based access control: enabled, roles, token_role_map, oidc_claim, oidc_role_map, plus the load balancer settings that the stack writes for sign-in
security.audit.enabled	true	Records every /v1/* request except health checks
security.audit.record_query_text	false	Stores query text in audit records. By default, records store only a hash and the length

security:
  secrets_manager_secret_arn: "arn:aws:secretsmanager:us-east-1:123456789012:secret:obsesc-auth"
  authz:
    enabled: true
    oidc_claim: "groups"
    oidc_role_map:
      - { value: "obsesc-admins", role: admin }
      - { value: "*", role: viewer }

With EnableUiPerUserRbac, the stack writes the sign-in settings for you from its parameters. See Roles for what each built-in role can do.

alerting

Built-in alerts. Every node ships with OBSESC’s built-in alert rules, which tell you when the deployment itself needs attention. Set the AlertmanagerEndpoint stack parameter to deliver them to your Alertmanager. If you leave it empty, the alerts aren’t delivered anywhere.

Your own alert rules. Alert rules over your logs are off by default (alerting.enabled: false). Rules are evaluated every few minutes, so alert latency is measured in minutes, not seconds.

alerting:
  enabled: true
  rules:
    - name: checkout-error-rate
      predicate: threshold
      service_glob: "checkout*"
      threshold:
        metric: { kind: dimension_ratio, key: level, value: error }
        op: gt
        value: 0.02
      destinations:
        - kind: slack
          url: "https://hooks.slack.com/services/..."

• Predicates: threshold (a measure crosses a value) and absence (a service goes quiet).
• Threshold metrics: event_count, dimension_count, dimension_ratio and quantile.
• Destination kinds: webhook, slack, pagerduty, opsgenie, sns and email.
• Secrets: destination secrets, such as a PagerDuty routing key, are given only as a reference (secret_ref: env:VAR, file:/path or secretsmanager:<arn>), never inline.

CloudFormation parameters that write config

Parameter	What it sets
RawBucketName	storage.s3_bucket
KmsKeyArn	storage.s3_sse_kms_key_arn (and EBS encryption)
S3Region, S3AssumeRoleArn, RawBucketAccountId	storage.s3_region, storage.assume_role_arn, storage.expected_bucket_owner
CreateRawBucket, StandardIaAfterDays, GlacierIrAfterDays	storage.lifecycle
EnableObjectLock, ObjectLockMode, ObjectLockRetainDays	storage.object_lock
AuthSecretArn	security.secrets_manager_secret_arn and security.authz.enabled: true
EnableUiPerUserRbac, NodeApiOidcClaim, NodeApiOidcAdminValue, NodeApiOidcViewerValue	Console sign-in through your identity provider, with each user mapped to their own role under security.authz
TlsCertSsmParameter, TlsKeySsmParameter	security.tls
EvaluationFeatures	Optional console features, including custody verification
AlertmanagerEndpoint	Where the built-in alerts are delivered