Blog

The Update That Locked Rows It Never Touched

A SQL Server UPDATE that touched rows 1 to 8000 blocked a point read of row 150000 for three full seconds. Lock escalation collapsed 8000 row locks into a single table lock, and the read baseline of 33ms became a 3001ms wait. Below the cliff, the same read was 0.5ms. Batching the update kept it there.

The Three Ways Postgres Reaches a Row

The same point lookup ran at 134 ms with a Seq Scan and 0.22 ms with an Index Scan, 600x faster touching 8 buffers instead of 41,667. A covering index did nothing until I ran VACUUM, and past 10% selectivity the index lost to the full scan it was supposed to replace. Measured on PostgreSQL 16.14 over 5,000,000 rows.

The Thread That Was Never Actually Idle

A non-blocking queue.poll() in a hot loop pegged a consumer thread at 91.8% CPU for a queue that was empty almost the whole time. I expected the flame graph to show an empty loop spinning on nothing. It showed lock contention instead, 77% of samples in AbstractQueuedSynchronizer.

The Rows That Never Left

Soft-deleting a row (UPDATE deleted_at instead of DELETE) doesn't make the row go away, it makes it permanent. A Postgres benchmark across 50 churn cycles shows the soft-delete table's heap at 26.7x the size of the hard-delete equivalent for the same 5,000 active rows, the query cost that comes with it, and the partial index that gets most of it back.

The Query That Paid For Everyone Else's Changes

A one-row indexed lookup query against a Hibernate session holding 8,000 managed entities ran at 1,371 checks/sec. The same query with FlushMode.COMMIT instead of the default AUTO ran at 878,355 checks/sec, 640.7x faster, for identical SQL.

The Profiler That Doesn't Wait for a Safe Moment

Why async-profiler's samples land where your code actually runs instead of where the JVM parks it. Safepoint bias, AsyncGetCallTrace, the four clocks it can sample on, and why you'd want it running in production all the time.

The JVM That Was Up and Doing Nothing

Why a JVM stuck in a garbage collection death spiral passes every health check, why the JVM's own OOM guards don't fire, how Netflix's jvmquake catches it with a debt counter, and what Uber's GC tuning numbers actually teach.

The Gentle Fix That Wasn't

Rebuilding and reorganizing a 99%-fragmented SQL Server index landed at nearly the same fragmentation number, 0.16% vs 0.46%. Rebuild took 0.22 seconds and 0.4MB of transaction log. Reorganize took 5.42 seconds and about 255MB, roughly 615x more log for a comparable result.

The Garbage the Collector Couldn't Collect

A refactor moved one object from being built once at startup to being built new on every request. Nothing referenced the old copies, so I assumed the garbage collector would clean them up. It couldn't. Each one prestarted two threads, and a live thread is a GC root, so 189 requests left 378 threads pinning 190 MB the collector was helpless to reclaim, and the heap walked straight into an OutOfMemoryError.

The Flame Graph That Found 25 Years a Day

A plain walkthrough of what a flame graph actually shows, why Java was invisible to Linux perf for years, and how Netflix read one graph well enough to hand back 13 million minutes of CPU time a day.

The CPU Number That Didn't Move

I built the classic unanchored-regex CPU bug on purpose and profiled it with async-profiler. Process CPU load barely moved, 9.9% to 10.05%, while throughput dropped 8x, 1.86M lines/sec to 14.79M. top would have told me nothing was wrong.

The Counter That Lost 336 Of Its 400 Increments

Eight workers incremented a Postgres counter 400 times under READ COMMITTED. Every transaction committed, not one error was raised, and the final balance was 64. The same workload at REPEATABLE READ lost nothing silently, it just told me about all 319 conflicts instead. Then write skew broke an invariant in 200 out of 200 trials at REPEATABLE READ and 0 out of 200 at SERIALIZABLE.

Want to get blog posts over email?

Enter your email address and get notified when there's a new post!