Docs: Limits
Documentation / Reference

Limits

Size, rate and range limits enforced by OBSESC at ingest, query time and in storage, and which ones you can change.

OBSESC enforces limits on each request and on each event. They protect ingest from runaway queries, and protect queries from oversized requests. A request that breaks a limit is rejected with a clear error. OBSESC doesn’t quietly truncate the request, except for the documented timestamp clamp.

Limits with a config key can be changed in the node configuration. Full details are in the Configuration reference. Contact us via the contact page if you need to change any other limit.

Ingest

Request and event size

LimitValueConfig key
OTLP/HTTP request body16 MiBFixed
Elasticsearch _bulk request body100 MiBFixed
Splunk HEC request body16 MiBingest.hec_max_body_bytes
Loki push request body32 MiBingest.loki.max_body_bytes
Firehose HTTP endpoint request body64 MiBingest.firehose.max_body_bytes
GELF message8 MiBingest.gelf.max_message_bytes
S3 object read by the SQS source256 MiBingest.sqs.max_object_bytes
Size of one event (body, service name and attributes)8 MiBingest.max_event_bytes
Attributes per event4,096ingest.max_attrs_per_event

An event over the size or attribute cap causes its whole batch to be rejected. Nothing from that batch is stored.

Timestamps

LimitDefaultConfig key
Maximum time in the future7 daysingest.max_future_skew_secs
Maximum ageNoneingest.max_event_age_secs
Action for an out-of-range timestampclamp (moved to the nearest allowed value)ingest.timestamp_action (clamp or reject)

Negative timestamps are always out of range.

Back-pressure

LimitDefaultConfig key
Total request body bytes being processed at once, across all HTTP listeners256 MiBingest.max_inflight_bytes
Data received but not yet written out to S3Built inContact us

When either limit is reached, ingest returns 503 with Retry-After until there is room again. Configure your shippers to retry on 503. See Configuration examples. OBSESC’s built-in alerts will tell you if a node is pushing back on ingest for a sustained period.

De-duplication

Retries that carry a stable event ID are de-duplicated within a recent window (ingest.dedup_window_events), so the event is stored once. The window is counted in events, so it covers less time as volume rises. Older retries, and events with no stable ID, are stored at least once.

Query

SQL scan limit

LimitDefaultHow to change it
Most data a POST /v1/sql query may read100 GiBmax_scan_bytes in the request
Longest a POST /v1/sql query may takeNonemax_scan_seconds in the request

A query over the limit returns 412 and reads nothing. Resubmit it with "confirm": true to run it anyway.

Console

Searches, comparisons and other investigation views in the console each cover a bounded amount of data per request. When there’s more, the console tells you, and you can continue or narrow the time range.

Some figures in Explore, such as distinct counts and percentiles, are close approximations rather than exact counts.

Storage and retention

LimitValue
RetentionUnlimited by default. Set storage.retention_window_days, or per-service retention.policies
Compliance floorretention.min_window_days. A configuration that would delete data sooner than the floor will not start
S3 Object Lock retention (stack parameter)1–36,500 days
Standard-IA transition (stack-created bucket)At least 30 days
Glacier Instant Retrieval transition (stack-created bucket)At least 60 days, and at least 30 days after the Standard-IA transition
EBS volume sizesSee System requirements

AWS charges a minimum storage duration on the colder S3 classes: 30 days for Standard-IA and 90 days for Glacier Instant Retrieval. It also bills small objects as if they were 128 KB. If your retention period is shorter than an object’s minimum storage duration, deleting it early incurs an AWS charge. The node warns about this at startup when storage.lifecycle is declared.