A flight data warehouse on a home server
I track aircraft as a hobby. Over 2026 that hobby turned into adsb-db, a private system that ingests ADS-B position reports from several sources, consolidates them into one archive, folds them into flight legs, and serves the lot through a read-only API with alerting on top. It runs on one home server. These are the numbers as of 8 November 2026, measured rather than estimated.
The numbers
| Position reports in the cold archive | 9,162,425,594 (310 daily Parquet files, 1 Jan to 8 Nov 2026) |
| Daily volume since October | ~37M rows a day median, 40M peak, about 430 rows a second |
| Flight legs folded from those positions | 46,132,340 |
| Distinct airframes across those legs | 574,830 |
| Airframes in the aircraft store | 303,894 |
| Cold archive on disk | 93 GB, about 10 bytes per position row |
| Leg store on disk | 43 GB |
| Hardware | 16 threads, 32 GB RAM, archive on a SATA SSD mirror |
Today’s hot tier is another ~37M rows on top and is not in the archive count.
Where the data comes from
Nothing here is bought. The feeds are the free community aggregators (adsb.lol and adsb.fi), my own receiver near Stansted, and an unofficial FR24 gRPC feed used sparingly as a gap filler for a coverage hole over the Gulf (one bounding box search a minute). I deliberately don’t use ADS-B Exchange because they charge for the API. I also keep the full adsb.lol archive (about 1 TB) locally while the system is in active development, because I’ve had to re-download it twice when a new feature needed the raw data refolded, and I’d rather not do that a third time.
How it works
Positions land in a hot tier partitioned by the hour, then roll into one Parquet file per day in the cold archive. The hourly partitioning wasn’t the first design. The original hot tier was a single table, and pruning old rows blocked the pollers. Splitting by hour fixed that outright.
The fold turns positions into flight legs: contact runs are closed on gaps and ground state changes, legs are keyed to airframes, and a separate aircraft store carries registration and type data. Refolding a whole day currently takes about 34 seconds, which matters because every rule change to leg closing means refolding history.
The storage figure, around 10 bytes a position, comes from a phase-aware retention scheme rather than a flat time sample. Within 120 seconds of the end of a contact run or any ground state change, every position is kept at full resolution. In cruise, a position is only stored if the aircraft has moved more than 0.05 nm laterally or 100 ft vertically since the last stored one (a sleeve filter), with a five minute backstop so nothing goes silent for too long. Against the raw feed that’s roughly 4.8x compression, and 44% smaller than the one-point-per-15-seconds floor I started with, at little cost in track fidelity. Known gaps: go-arounds and diversions aren’t yet exempted from the sleeve, and the merge step still uses a flat 120 second rule.
The whole thing sits behind Cloudflare with a Google login on the edge, so it is reachable from my phone without the app knowing anything about auth. Alerts are rules in a database table with an API, not a config file, so sibling projects can register their own watches. Notifications go out over a self-hosted ntfy. Pressing one button on an aircraft’s hex gives me a push every time it goes airborne.
Optimisation, since that’s the fun part
The live fold’s CPU use went from 42% of a core to 18% in October. The test suite went from 219 seconds to 49. The hot tier partitioning stopped the prune from blocking ingestion. None of this is exotic, it is the usual work of looking at what the box is actually doing and removing the thing it does most often for no reason. It’s also the part of the project I’d point at if someone asked what I do all day.
What’s next
Worldwide METARs as another time series (IEM ASOS archive for the backfill, aviationweather.gov for live). Whisper transcription of my airband recordings on a GPU in the same box, catalogued against the legs. An MCP server so I can ask it questions in plain English. And at some point a decision about retention, because 37M rows a day does not stop.
Why I’m writing this down
Partly because the numbers surprised me when I measured them. Partly because it’s a better description of how I think about infrastructure than anything on my CV: one machine, no budget, a real-time feed that never pauses, and a long run of small decisions about what to keep, what to throw away, and where the time actually goes.