Limits
Size, rate and range limits enforced by OBSESC at ingest, query time and in storage, and which ones you can change.
OBSESC enforces limits on each request and on each event. They protect ingest from runaway queries, and protect queries from oversized requests. A request that breaks a limit is rejected with a clear error. OBSESC doesn’t quietly truncate the request, except for the documented timestamp clamp.
Limits with a config key can be changed in the node configuration. Full details are in the Configuration reference. Contact us via the contact page if you need to change any other limit.
Ingest
Request and event size
| Limit | Value | Config key |
|---|---|---|
| OTLP/HTTP request body | 16 MiB | Fixed |
Elasticsearch _bulk request body | 100 MiB | Fixed |
| Splunk HEC request body | 16 MiB | ingest.hec_max_body_bytes |
| Loki push request body | 32 MiB | ingest.loki.max_body_bytes |
| Firehose HTTP endpoint request body | 64 MiB | ingest.firehose.max_body_bytes |
| GELF message | 8 MiB | ingest.gelf.max_message_bytes |
| S3 object read by the SQS source | 256 MiB | ingest.sqs.max_object_bytes |
| Size of one event (body, service name and attributes) | 8 MiB | ingest.max_event_bytes |
| Attributes per event | 4,096 | ingest.max_attrs_per_event |
An event over the size or attribute cap causes its whole batch to be rejected. Nothing from that batch is stored.
Timestamps
| Limit | Default | Config key |
|---|---|---|
| Maximum time in the future | 7 days | ingest.max_future_skew_secs |
| Maximum age | None | ingest.max_event_age_secs |
| Action for an out-of-range timestamp | clamp (moved to the nearest allowed value) | ingest.timestamp_action (clamp or reject) |
Negative timestamps are always out of range.
Back-pressure
| Limit | Default | Config key |
|---|---|---|
| Total request body bytes being processed at once, across all HTTP listeners | 256 MiB | ingest.max_inflight_bytes |
| Data received but not yet written out to S3 | Built in | Contact us |
When either limit is reached, ingest returns 503 with Retry-After until there is room again. Configure your shippers to retry on 503. See Configuration examples. OBSESC’s built-in alerts will tell you if a node is pushing back on ingest for a sustained period.
De-duplication
Retries that carry a stable event ID are de-duplicated within a recent window (ingest.dedup_window_events), so the event is stored once. The window is counted in events, so it covers less time as volume rises. Older retries, and events with no stable ID, are stored at least once.
Query
SQL scan limit
| Limit | Default | How to change it |
|---|---|---|
Most data a POST /v1/sql query may read | 100 GiB | max_scan_bytes in the request |
Longest a POST /v1/sql query may take | None | max_scan_seconds in the request |
A query over the limit returns 412 and reads nothing. Resubmit it with "confirm": true to run it anyway.
Console
Searches, comparisons and other investigation views in the console each cover a bounded amount of data per request. When there’s more, the console tells you, and you can continue or narrow the time range.
Some figures in Explore, such as distinct counts and percentiles, are close approximations rather than exact counts.
Storage and retention
| Limit | Value |
|---|---|
| Retention | Unlimited by default. Set storage.retention_window_days, or per-service retention.policies |
| Compliance floor | retention.min_window_days. A configuration that would delete data sooner than the floor will not start |
| S3 Object Lock retention (stack parameter) | 1–36,500 days |
| Standard-IA transition (stack-created bucket) | At least 30 days |
| Glacier Instant Retrieval transition (stack-created bucket) | At least 60 days, and at least 30 days after the Standard-IA transition |
| EBS volume sizes | See System requirements |
AWS charges a minimum storage duration on the colder S3 classes: 30 days for Standard-IA and 90 days for Glacier Instant Retrieval. It also bills small objects as if they were 128 KB. If your retention period is shorter than an object’s minimum storage duration, deleting it early incurs an AWS charge. The node warns about this at startup when storage.lifecycle is declared.