在POSETTE 2026上，赛·斯里拉姆普尔(Sai Srirampur)提出了"Postgres isn't Slow. 您的存储是。 一个谈话检查存储如何影响 PostgreSQL 的性能。 这个帖子概括了基准测试，结果揭示的内容，以及提出在本地NVMe生产中运行PostgreSQL的架构。 您可以在下面的视频中观看完整的演讲。
讨论的起点是几个规模的Postgres运行的症状小组可能认识到：摄入速度放慢，P95读取延迟变得不可预测，自动真空下降，检查站与你的实际工作量争夺资源，或者逻辑复制积累滞后。 这些问题并不总是有相同的原因，但它们可以有一个容易忽略的共同原因：存储性能。
为了理解原因，赛伊运行了基准测试，比较了基线gp3 EBS和当地NVMe上相同的3.3亿行PostgreSQL工作量。 我们将走过设置和赛在分析结果时发现的东西，然后接近于为在当地NVMe上运行Postgres而提出的生产模式。
五个缩放症状
一旦PostgreSQL的工作量达到数百千兆字节或千兆字节，并随着实际的货币和吞吐量而运行，就经常出现五个共同的问题。
- 摄入缓慢第一种是重在更新型和UPSERT的摄入管道。 工作一旦数秒完成，就可以随着下面的数据量的增加而延伸成分钟。
- 不连贯的阅读延迟。 下一个问题出现在工作量方面，大多数查询速度仍然很快，但有些却大大放慢。 P95耐久从毫秒移动到秒，仪表板超时，不均匀的减速使得原因更难分离。
- VAUUM落在后面。 第三期是随着UPDATE和UPSERT的吞吐量的增长，自动真空下降，从千兆字节累积到兆字节。
- 较慢的检查站。 检查点也可以花费更长的时间，与I/O和CPU的客户端工作量竞争，将以前隐形的操作变成反复出现的潜伏性突起。
-逻辑复制滞后。 最后一个症状出现在PostgreSQL数据流到Kafka或ClickHouse等系统时。 WAL的解码可能很慢，不断增长的复制槽会使问题复杂化。
这些症状如何连接到存储
由于工作设置将PostgreSQL可用的内存和操作系统页面缓存排出，更多的数据读取需要物理I/O. 在较慢的存储上，这些读数的较高成本和排队会增加平均和尾部的耐久性。
客户端查询和背景维护也使用相同的存储预算。 高压和指数写作，WAL同步，检查站，VACUUM，逻辑解码溢出争夺带宽，IOPS，以及队列容量。
一旦储存达到其中一个限度，业务便开始相互拖延。 这就是缓慢存储会成为几个显然无关的PostgreSQL问题的根源。
本地 NVMe 有何变化
当地NVMe的情况并非是它消除了I/O，而是可以大幅降低成本。 比较对比了网络附着存储上的毫秒级操作与本地NVMe上可能以数十微秒完成的操作。 缓存缺失、WAL冲洗和维护I/O仍在发生
基准测试：本地NVMe对EBS
为了更好地理解为什么这样，赛伊建立了一个简单的实验，以比较在EBS和本地NVMe上运行的PostgreSQL的类似工作量。
设置
为了隔离存储作为变量，基准测试使用了八个相同的PostgreSQL集群：四个使用实例存储NVMe，四个使用基线gp3 EBS，以3000 IOPS提供。 每个集群运行在一个m6id.4x大实例上，有16 vCPU,64 GiB of RAM,16 GiB of share buffers. 我们用多个实例来规范基准测试运行的变异性。
该设置使用了482 GiB pgbench counts堆积，包含33亿行。 然后，每个集群在五分钟内进行高度同步的随机-UPDATE工作量，有64个客户和16个工人线条：
\ set 援助随机(1, 3300000000)
更新 pgbench counts SET 余额=余额+1 援助=：援助；
简而言之，两组人之间唯一的区别是存储类。
基准测试显示的
基准测试为NVMe集群生产了中位数16,030 TPS，而EBS上的TPS为1,734 TPS——这个测试的差数为9.24×. 中间交易期从36.9毫秒降至4.0毫秒。
对于不同的操作，NVMe在366秒内完成了一个超过10GBbloat的VACUM，而EBS上的是964秒，并在139秒内排出一个10GB逻辑复制的积压，而不是238秒。
一个资格问题：这些结果适用于这一工作量和这些存储配置。 EBS方面使用在3000 IOPS中提供的基线gp3，不同的EBS设置极有可能产生不同的结果。
解释差距
通过观察EBS交易时空37毫秒的位置，通量差距变得更容易理解。
在这张基准测试中，页面读取在EBS上记录了大约19 ms，而NVMe上记录了0.3 ms. WAL fsync at content又在EBS上增加了11 ms，在NVMe上增加了1.5 ms，而锁和排程器的间接费用则贡献了约5 ms，对0.2 ms. CPU的工作本身大致相同——在这两种情况下大约为2 ms.
换句话说，附加的EBS空档大多来自等待存储，而不是PostgreSQL做更多的CPU工作。
一个中运行的 pg stat 活动快照从另一个角度显示了相同的模式。 在64个后端中，EBS上的29个在IO:DataFileRead上等待，而NVMe上的9个是大约45%对14%。 这是一次即时的快照，而不是整个运行的平均数，但它表明，当时还有多少次EBS会议因阅读而停顿。
CPU的剖面然后新增是反直观的观察。 在大约240秒的剖析窗口上，NVMe主机累计了2,253个CPU秒——大约9.4个核心繁忙——而EBS主机则只积累了251个CPU秒，或约一个核心繁忙。 一些PostgreSQL的功能在EBS上出现比例更热，但绝对的CPU时间显示，EBS进程在等待存储时会花费更多的时间-CPU.
NVMe主机让更多的CPU能力忙于做有用的工作。 这有助于解释高得多的吞吐量：缓存缺失，WAL冲洗，维护I/O仍然发生，但它们完成得更快，使得PostgreSQL在存储上花费的时间较少。
持久性问题
性能结果后，谈话转为明显的反对：实例存储NVMe与机器的寿命联系在一起，因此局部数据随节点消失。 操作问题在于PostgreSQL是否能够将NVMe latency与生产级可用性和回收性相结合。
简短的回答是 " 是 "，因为耐久性是在集群一级提供的，而不是由任何单个的Postgres节点提供的。
在生产时运行 NVMe 支持的 Postgres
在本地的NVMe上运行 PostgreSQL 在生产中意味着围绕存储进行设计，存储速度快但与机器的寿命有关。 失去一个节点意味着失去其本地的数据副本；实例存储卷不能通过EBS API进行快照；可用容量取决于选定的实例类型。
谈话中呈现的制作架构通过三个主要要求来解决这个问题：replicate数据库，这样服务就可以在节点丢失后幸存下来，在单个机器之外保持备份和恢复数据，以及规划每个实例类型上NVMe周围可用的容量。 这些模式共同使得NVMe支持的PostgreSQL能够自信地运行生产。
配备两个备用机的可用性很高
第一个模式是以法定人数为基础，与两名待命候选人同步复制。 如果配置像Ah 1(待命1，备用2)，交易可以在永久确认其WAL之后进行，而不是等待两者中较慢者。 将节点设置在可用区之间，防止AZ故障。
为耐久性持续备份对象存储
对于数据库节点以外的耐久性，WAL-G等开源工具可以提供物理基础备份和连续WAL存档。 在低纬度WAL航运的情况下，这种架构可以瞄准以秒计的RPO，并支持保留窗口内的点对点回收。
备份和WAL也应保存在单独的故障域中，当恢复计划需要从全区域断电中存活下来时，还有另一个区域副本。
作为恢复基础的备份
基础备份和存档的WAL不仅仅是一个应急机制：同样的恢复历史可以支持点对点恢复，孤立的分支，部署调整大小，读取-replica种子，以及灾难恢复。
备份和复制可以解决不同的问题。 在谈话中描述的架构中，备用提供快速故障，而备份则防止腐败，操作员错误，以及整个集群的故障；两个恢复路径都需要定期测试。
生产实例
这种模式不仅是理论性的，谈话引用了Instacart的公开例子，其中描述了NVMe上的PostgreSQL对于耐久性敏感的搜索工作量，以及Datadog，其中讨论了本地-NVMe PostgreSQL对于需要这些工作量的例子。 反复出现的模式是局部存储性能，同步复制可用性，以及独立存储的基础备份和WAL档案进行恢复。
主外卖
谈话最后提出了诊断性建议：当PostgreSQL看起来停止缩放时，检查数据库并一起存储。 消化缓慢，读取延迟性不可预测，真空压力，干扰性检查点，复制滞后等可能是储存层达到极限的信号。
底线故意比标题更窄：本地NVMe可以大幅降低存储工作量的I/O罚，但它会带有不同的操作模式。 会谈中概述的架构使用基于法定人数的复制，以备可用，并独立存储的基备份加上存档的WAL供回收。
今天开始使用 ClickHouse 管理 Postgres
有兴趣看看 ClickHouse 管理 Postgres 如何在您的数据上工作吗？几分钟后从ClickHouse Cloud开始免费获得300美元的信贷
报名
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
At POSETTE 2026, Sai Srirampur presented "Postgres Isn’t Slow. Your Storage Is." a talk examining how storage affects PostgreSQL performance at scale. This post recaps the benchmark, what the results revealed, and the architecture proposed for running PostgreSQL on local NVMe in production. You can watch the full talk in the video linked below.
The talk starts with several symptoms teams running Postgres at scale may recognize: ingestion slows down, P95 read latency becomes unpredictable, autovacuum falls behind, checkpoints compete with your actual workload for resources, or logical replication accumulates lag. These problems do not always share the same cause, but they can have a common one that is easy to overlook: storage performance.
To understand why, Sai ran a benchmark comparing the same 3.3-billion-row PostgreSQL workload on baseline gp3 EBS and local NVMe. We will walk through the setup and what Sai discovered when analyzing the results, then close with the production patterns proposed for running Postgres on local NVMe.
Five scaling symptoms
There are five common issues that often appear once a PostgreSQL workload reaches hundreds of gigabytes or terabytes and is running with real concurrency and throughput.
- Slow ingestion. The first one is ingestion pipelines heavy on UPDATEs and UPSERTs. Jobs that once finished in seconds can stretch into minutes as the volume of data underneath them grows.
- Inconsistent read latency. Next one appears in workloads where most queries remain fast but a few become dramatically slower. P95 latency moves from milliseconds to seconds, dashboards time out, and the uneven slowdown makes the cause harder to isolate.
- VACUUM falling behind. A third issue is autovacuum falling behind as UPDATE and UPSERT throughput grows, with table bloat accumulating from gigabytes into terabytes.
- Slower checkpoints. Checkpoints can also take longer and compete with the client workload for I/O and CPU, turning an operation that used to be invisible into a recurring latency spike.
- Logical replication lag. The last symptom appears when PostgreSQL data is streaming to systems such as Kafka or ClickHouse. WAL decoding can be slow and a growing replication slot can compound the problem.
How do these symptoms connect to storage
As the working set outgrows the memory available to PostgreSQL and the operating-system page cache, more data read require physical I/O. On slower storage, the higher cost and queuing of those reads can increase both average and tail latency.
Client queries and background maintenance also use the same storage budget. Heap and index writes, WAL synchronization, checkpoints, VACUUM, and logical-decoding spills compete for bandwidth, IOPS, and queue capacity.
Once storage reaches one of those limits, operations begin delaying one another. That is how slow storage can be the root cause of several apparently unrelated PostgreSQL problems.
What changes with local NVMe
The case for local NVMe is not that it eliminates I/O, but that it can sharply reduce its cost. The comparison contrasts millisecond-scale operations on network-attached storage with operations that may complete in tens of microseconds on local NVMe. Cache misses, WAL flushes, and maintenance I/O still happen, but they simply don’t impact the overall system as much.
The benchmark: Local NVMe vs. EBS
To better understand why this is the case, Sai set up a simple experiment to compare a similar workload on PostgreSQL running on EBS and local NVMe.
Setup
To isolate storage as the variable, the benchmark used eight identical PostgreSQL clusters: four using instance-store NVMe and four using baseline gp3 EBS provisioned at 3,000 IOPS. Each cluster ran on an m6id.4xlarge instance with 16 vCPUs, 64 GiB of RAM, and 16 GiB of shared_buffers. We used multiple instances for each storage type to normalize variability across benchmark runs.
The setup used a 482 GiB pgbench_accounts heap containing 3.3 billion rows. Each cluster then ran the same highly concurrent random-UPDATE workload for five minutes, with 64 clients and 16 worker threads:
\set aid random(1, 3300000000)
UPDATE pgbench_accounts SET abalance = abalance + 1 WHERE aid = :aid;
In short, the only difference between the two groups was the storage class.
What the benchmark showed
The benchmark produced a median 16,030 TPS for the NVMe clusters, compared with 1,734 TPS on EBS—a 9.24× difference in this test. Median transaction latency fell from 36.9 ms to 4.0 ms.
For different operations, NVMe completed a VACUUM over 10 GB of bloat in 366 seconds, compared with 964 seconds on EBS, and drained a 10 GB logical-replication backlog in 139 seconds instead of 238 seconds.
One qualification matters: these results apply to this workload and these storage configurations. The EBS side used baseline gp3 provisioned at 3,000 IOPS, different EBS setup would most likely produce different results.
Explaining the gap
The throughput gap becomes easier to understand by looking at where the 37 ms of EBS transaction latency went.
In this benchmark, page reads accounted for about 19 ms on EBS, compared with 0.3 ms on NVMe. WAL fsync at commit added another 11 ms on EBS versus 1.5 ms on NVMe, while lock and scheduler overhead contributed about 5 ms versus 0.2 ms. CPU work itself was roughly the same—about 2 ms in both cases.
In other words, most of the additional EBS latency came from waiting on storage, not from PostgreSQL doing more CPU work.
A mid-run pg_stat_activity snapshot showed the same pattern from another angle. Of the 64 backends, 29 on EBS were waiting on IO:DataFileRead, compared with nine on NVMe—roughly 45% versus 14%. This was a point-in-time snapshot rather than an average across the run, but it illustrates how many more EBS sessions were stalled on reads at that moment.
The CPU profile then adds a counterintuitive observation. Over a profiling window of roughly 240 seconds, the NVMe host accumulated 2,253 CPU-seconds—about 9.4 cores busy—while the EBS host accumulated only 251 CPU-seconds, or about one core busy. Some PostgreSQL functions appeared proportionally hotter on EBS, but the absolute CPU time showed that EBS processes were spending far more time off-CPU waiting for storage.
The NVMe host kept more of its CPU capacity busy doing useful work. That helps explain the much higher throughput: cache misses, WAL flushes, and maintenance I/O still occurred, but they completed faster and left PostgreSQL spending less time blocked on storage.
The durability question
After the performance results, the talk turns to the obvious objection: instance-store NVMe is tied to the lifetime of a machine, so the local data disappears with the node. The operational question is whether PostgreSQL can combine NVMe latency with production-grade availability and recovery.
The short answer is yes, given durability is provided at the cluster level rather than by any individual Postgres node.
Running NVMe-backed Postgres on production
Running PostgreSQL on local NVMe in production means designing around storage that is fast but tied to the lifetime of a machine. Losing a node means losing its local copy of the data; instance-store volumes cannot be snapshotted through the EBS API; and available capacity depends on the selected instance type.
The production architecture presented in the talk addresses this through three main requirements: replicate the database so the service can survive node loss, maintain backups and recovery data outside the individual machines, and plan capacity around the NVMe available on each instance type. Together, these patterns make it possible to run NVMe-backed PostgreSQL confidently in production.
High availability with two standbys
The first pattern is quorum-based synchronous replication with two standby candidates. With a configuration such as ANY 1 (standby1, standby2), a transaction can commit after either standby durably acknowledges its WAL rather than waiting for the slower of the two. Placing the nodes across availability zones protects against an AZ failure.
Continuous backups to Object Storage for durability
For durability outside the database nodes, open-source tools such as WAL-G can provide physical base backups and continuous WAL archiving. With low-latency WAL shipping, this architecture can target an RPO measured in seconds and support point-in-time recovery within the retention window.
Backups and WAL should also be kept in a separate failure domain, with another regional copy when the recovery plan needs to survive a full regional outage.
Backups as the recovery foundation
Base backups and archived WAL are more than an emergency mechanism: the same recovery history can support point-in-time restores, isolated branches, deployment resizing, read-replica seeding, and disaster recovery.
Backups and replication solve different problems. In the architecture described in the talk, standbys provide rapid failover, while backups protect against corruption, operator error, and cluster-wide failure; both recovery paths need regular testing.
Examples from production
This pattern is not only theoretical, the talk cites public examples from Instacart, which has described PostgreSQL on NVMe for a latency-sensitive search workload, and Datadog, which has discussed local-NVMe PostgreSQL instances for workloads that need them. The recurring pattern is local storage for performance, synchronous replication for availability, and independently stored base backups and WAL archives for recovery.
The main takeaway
The talk closes with a diagnostic suggestion: when PostgreSQL appears to stop scaling, inspect the database and storage together. Slow ingestion, unpredictable read latency, vacuum pressure, disruptive checkpoints, and replication lag can be signals that the storage layer has reached a limit.
The bottom line is deliberately narrower than the title: local NVMe can sharply reduce the I/O penalty for storage-bound workloads, but it comes with a different operating model. The architecture outlined in the talk uses quorum-based replication for availability and independently stored base backups plus archived WAL for recovery.
Get started with ClickHouse Managed Postgres today
Interested in seeing how ClickHouse Managed Postgres works on your data? Get started with ClickHouse Cloud in minutes and receive $300 in free credits.
Sign up
Auto-fetched and translated by AI Atlas. All rights belong to the original author.