The same analytics design works from a thousand events a day to billions: if you change the right piece at the right time. The four stages, what breaks at each, and when Iceberg, S3 Tables, Parquet and a warehouse start earning their cost.
Collector into Firehose, gzipped JSON into S3 by date, Athena with partition projection. Everything is serverless, nothing needs sizing, and the cost is dominated by minimums. The only discipline needed is a stable event schema, names and property keys you never repurpose, because every property is a column someone will query.
Queries slow down and Athena's $5-per-terabyte-scanned bill becomes noticeable, because JSON makes every query read every byte. Convert to Parquet — Firehose can do the conversion on delivery against a Glue schema — and partition by date and event name. Typical analytical queries then read a small fraction of the data. Keep raw JSON for a short window as a safety net.
Book a free 30-minute call. You'll leave knowing what we'd do first and roughly what it costs, whether or not you hire us.
Now the problems are small files, late data and corrections. Apache Iceberg tables solve all three: compaction merges small files, partitions can evolve without rewriting history, and deletes and updates work — which matters the day someone asks you to remove a user's data. Firehose has delivered directly into Iceberg tables since October 2024. S3 Tables, launched in December 2024, are Iceberg tables managed by S3 with automatic compaction and snapshot clean-up; compaction is billed per object and gigabyte processed — $0.002 per 1,000 objects and $0.005 per GB for the default strategy — in exchange for never thinking about it.
Several teams need the same stream in real time — fraud detection, personalisation, dashboards — and replay matters. This is the point to put Kinesis Data Streams or MSK between the collector and storage, fan out to consumers, and add a warehouse such as Redshift for the heavily used aggregates while Iceberg stays the source of truth. Do not start here: most companies never reach it, and the operating cost is real.
Once a month: what changed in tech that month and what it means for you, from cloud and AI to security and the tools you build with. Every item links to its source. No sales pitch, unsubscribe in one click.
Book a free 30-minute call about it. Bring your setup and your bill; you'll leave knowing what we'd do first and roughly what it costs, whether or not you hire us.
Or keep reading
First-Party Analytics on AWS for a Few Dollars a Month: The Pipeline Behind This Site
A visitor id, a session id and a page view id; a collector on our own domain; Firehose into S3; Athena with saved queries. How we built analytics that ad blockers do not drop, nobody else holds, and that costs less than a coffee at our traffic, and the three bugs we found on the way.
Read
Aurora PostgreSQL Now Queries Iceberg and Parquet Directly, With DuckDB Inside: When It Replaces a Pipeline
Since September 30, 2026, Aurora PostgreSQL 17 and 18 can query Apache Iceberg and Parquet in S3, S3 Tables and Glue-catalogued lakes in the same SQL as live operational data, using an embedded DuckDB engine. No extra charge. What it is good for, its limits, and when you still want Redshift or Athena.
Read
As of October 26, 2026. Provided as is, for general information, and accurate to the best of our knowledge on the date shown. Not legal, compliance, tax, financial or other professional advice. Prices, limits and vendor services change; check current vendor documentation before relying on anything here. Terms of use.