CASE 111 · SENTRY · 2024
Logs we retain for a year, queried for the price of S3.
A B2B SaaS company had been paying Datadog for full-fidelity log indexing across the entire fleet, with 90-day retention. The logs cost $42k/month. We split the path: keep the recent 14 days in Datadog for incident response, archive everything to S3 with Athena for the long-tail forensics.
B2B SaaS
COST
2024
RESULTS
What changed, by the numbers.
LOGS BILL
−74%
INCIDENT-RESPONSE WINDOW
14d
FORENSICS WINDOW
1y
QUERY TIME (HOT)
UNCHANGED
HOW IT WENT
The team didn’t actually need Datadog for the 16-89-day-old logs. Investigations against logs older than two weeks were rare and could afford to be slower. But they did happen — compliance investigations and customer-reported issues sometimes needed the older data.
We reduced Datadog’s retention to 14 days and added a Firehose pipeline that wrote all logs to S3 as compressed Parquet, partitioned by date and service. Glue Catalog made the partitions discoverable; Athena queries surfaced the long-tail data on demand.
Bill dropped 74% — Datadog was billed on indexed volume, which is what we cut. Hot incident response in Datadog is unchanged. The handful of cold investigations per month run from Athena in under a minute and cost cents. Total annual saving funds another engineer.
RELATED · SAME DOMAIN
Other engagements in this space.
READY WHEN YOU ARE
Let's get your AWS bill (and architecture) in order.
The discovery call is free. You walk away with at least one concrete idea — even if we never work together.