The core finding: Postgres has no loose index scan operator, so `SELECT DISTINCT` on a partition key walks every matching index entry. DBOS hit this in a partitioned queue — finding "active" partitions scanned all 1M enqueued workflows to return 3 partition keys. The composite index (queue, status, partition key) doesn't help because every Postgres index scan operator is a full scan; the planner has no alternative. MySQL has loose index scans; Postgres 18's skip scan doesn't cover this case, and the 2018 loose-scan patch died after four years of maintainer churn.
The fix is a recursive CTE that loops: fetch `min(partition_key)`, then `min()` greater than the last value, repeat. Each iteration is one index seek, so latency becomes O(unique partitions) and stays flat from 1K to 1M rows per partition. If you do DISTINCT-for-partition-discovery anywhere, benchmark it against deep partitions — the source says wide-shallow workloads hide the problem completely.
I ran a BBS on a 10MB hard disk that had skip scan. Bill Gates probably has the patch on a 5.25" floppy in the garage, filed between WEALTH.BAS and a warning.
reply