Petabyte databases don’t back up themselves. They demand infrastructure that scales with them. PlanetScale has built one such system for its sharded Postgres offering, known internally as Neki. The result? Backups complete in hours at speeds exceeding 50 GB/s. And the approach reveals lessons for anyone running large Postgres deployments.
Every 12 hours, PlanetScale’s system must capture a consistent, encrypted snapshot of busy production databases. No impact on queries. No smeared data. Customers see a simple schedule or button click. Behind the scenes sits careful orchestration of cloud instances, object storage, and Postgres internals. Parallelism across shards makes the difference.
Ben Dicken described the process in detail three days ago. In “Massively parallel Postgres backups” on the PlanetScale blog, he walks through the lifecycle for sharded Postgres. The post builds on an earlier piece from 2024 that showed similar gains for MySQL via Vitess. That one, also by Dicken, reported a 20-terabyte sharded database backing up in under two hours at 6.7 GB/s overall. See “Faster backups with sharding”.
Sharding spreads data across many independent Postgres primaries. Each shard operates its own WAL stream. PlanetScale treats regular Postgres backups as building blocks. Then it runs them in parallel. One backup node per shard. Spun up on demand from EC2 or equivalent. The nodes pull prior backups from S3, replay recent changes, freeze a consistent point, encrypt, and push the new snapshot back to object storage.
Why not back up on the primary or a replica? IOPS and compute load would hit production traffic. Better to isolate the work. So the system launches fresh instances. They restore the previous backup from S3. That proves restorability every cycle. Then they catch up.
Catch-up uses a hybrid method. Most WAL comes from archived streams in S3 via wal-g. The final few minutes stream directly from the primary. An archive_timeout of five minutes ensures recent segments reach storage quickly. The last leg finishes in seconds. Not minutes. Not hours. Once all nodes reach timestamp T, replication stops. The backup freezes. Nodes encrypt their state and upload. Then they shut down.
First backups differ. They start with pg_basebackup on each new node to seed from the primary. Replication follows until catch-up. Then wal-g handles formatting, encryption, and storage. Subsequent backups reuse the prior snapshot. Steady state achieved.
Numbers tell the story. Consider an unsharded 32-terabyte database. Assume compression reduces it to 20 TB. The backup cycle moves roughly 40,000 GB in and out of storage. At 500 MB/s sustained, that takes 22 hours. Twice-daily backups would overlap. RPO goals break.
Split the same data across eight shards of 4 TB each. Parallel nodes handle the transfers simultaneously. Time drops to 2.8 hours. Add more shards and it shrinks further. Thirty-two shards finish in 42 minutes. One hundred terabytes across 100 shards backs up at speeds comparable to a single-terabyte single-shard instance. Speed scales with shard count.
The PlanetScale docs confirm the operational side. Automatic backups run every 12 hours on production and development branches. No extra charge for the included ones. WAL logs stay available for the same window as the oldest backup. Storage lives off-cluster in the same region and cloud provider for durability. PITR reaches any point from the oldest backup up to five minutes before now. PlanetScale Postgres backups documentation.
Backups do more than protect against disaster. They power routine operations. On Metal instances that use local NVMe, resizing spins up new larger nodes. Each pulls the latest backup from S3, replays WAL, catches up as a standby, then switches over. Node replacement after failure follows the same restore-and-replay path. For clusters with hundreds of shards, statistical failure rates make this routine. Backups keep it manageable.
Recent coverage echoes the interest. A Medium post from early this week reviewed top Postgres backup tools for 2026. It highlights pgBackRest for parallel physical backups on large clusters and notes wal-g for fast compression and upload. Parallelism appears repeatedly as a requirement for scale. See the piece by Rostislav Dugin at “Top 5 PostgreSQL® backup tools in 2026”.
Microsoft’s June update on Postgres offerings added long-term retention up to 10 years and highlighted throughput gains on Azure. Yet managed services still wrestle with the same physics PlanetScale addresses through sharding and on-demand nodes. Incremental and continuous strategies matter. So does avoiding overlap in backup windows.
PlanetScale’s approach isn’t unique in concept. pgBackRest users have long pursued parallel backup and restore. Wal-g brings S3-native speed. The combination here, tied to sharded architecture and dynamic compute, delivers the headline numbers. Petabyte-scale in hours. Over 50 GB/s aggregate.
Challenges remain. Object storage transfers add cost. Temporary instances add cost. Hybrid WAL replay requires tight coordination. Archive timeouts must balance freshness against overhead. Encryption and compression run on the backup nodes. Still, the system prioritizes production stability. Customers pay for managed simplicity.
Discussions on X this week reinforced the point. PlanetScale’s own posts highlighted that backups matter most when never needed. Engineers noted the hidden complexity. One thread compared self-managed Postgres to the DBA overhead avoided by using a service that handles orchestration at this scale.
Industry articles from the past month stress that AI-era workloads push databases harder. Larger models mean larger datasets. Operational tasks like backup, resize, and recovery must not become bottlenecks. Sharding plus parallel infrastructure offers one path. Not every team will shard their Postgres. Those who do, or who run on platforms that do it for them, gain from these techniques.
The orchestration itself deserves appreciation. Spinning hundreds of instances on schedule. Streaming terabytes from S3 without saturating networks. Coordinating WAL replay across shards so they all stop at the same logical time. Freezing consistent snapshots. Encrypting on the fly. Decommissioning cleanly. All while production traffic continues uninterrupted.
PlanetScale wants backups to feel boring. Click, schedule, restore when required. The engineering underneath is anything but. It shows what becomes possible when sharding, object storage, and Postgres tooling combine at cloud scale. Other operators running large Postgres instances would do well to study the patterns. Parallel backup nodes. Hybrid WAL sources. Proof-of-restorability in every cycle. These ideas travel beyond any single vendor.
Recent conversations on Reddit’s r/PostgreSQL asked what the best backup solution looks like in 2026. Answers circled pgBackRest, wal-g, and managed services. Performance on very large databases drove the recommendations. Parallelism and incremental options topped the list. The PlanetScale implementation puts concrete numbers to those preferences.
So the next time a backup completes without drama, remember the instances that spun up, the data that flowed from S3, the WAL that replayed in parallel, and the snapshot that landed encrypted and ready. It all happens so the production database never has to pause. That’s the point.
WebProNews is an iEntry Publication