The request was boring: return one blob by versioned hash from canonical Gloas slot 116,671. All 14 Lighthouse nodes answered that the block was pre-Deneb. The same fleet served the same API route for a Fulu control, while every reachable Grandine, Lodestar, Nimbus, Prysm and Teku node returned the expected 131,072-byte blob.
I asked Lighthouse for two blobs from one canonical Fulu block, listing commitment 6 before commitment 0. Every reachable Lighthouse supernode returned them as [6, 0]. That sounds like obedient filtering, but the Beacon API says the response follows the commitments in the block, so the correct order was [0, 6].
A Prysm node fell 17 slots behind. Its recovery path then spent about 12 seconds waiting for each old slot before making a direct data-column fetch that usually finished in 1.3 seconds. On a 12-second chain, that turns catch-up into a treadmill.
An Ethereum transaction announcement is an invitation: send the hash now, then let the other peer ask for the transaction. One Geth sampler on Glamsterdam devnet-8 spent 60 hours issuing invitations that its legacy peers could not cash. Its logs contain 163,752 failed full-blob encodes across 42,537 truncated transaction labels.
A fresh Gloas spec change sent me looking for an ugly sanity check: do observed peer subscriptions line up with custody metadata? The obvious raw table answered before I had joined anything. One stored peer_id_unique_key owned almost exactly half the rows across every observer. Gloas is not live on mainnet; it just gave me a reason to inspect current mainnet capture.
That peer does not exist.
At slot start, Ethereum's blob pool did not look ready. Roughly 1.5 seconds later, it did.
Across eight fixed mainnet observers and 14 complete UTC days, 88.83% of canonical blob cell positions had been seen when the slot began. By the time the same observer saw its first PeerDAS data column, the share was 98.71%. The missing ten percentage points mostly arrived in the small gap before the block and its data turned up.
Teku followed Plataberget with a broken data-column gossip decoder. That sounds impossible until you remember PeerDAS has more than one way to recover data.
Across 15 continuously observed Teku nodes, not one data-column sidecar reached the gossip validator during the matched 30-minute window before the patched build arrived. The same 15 nodes submitted 114,383 sidecar messages to that validator in the window after rollout, and 83,684 passed gossip verification.
Arrival time has been my stand-in for PeerDAS bandwidth. It sort of worked, but it was still a timing proxy. A process-level counter now exposes bytes by port, so I expected a clean rising curve with blob count.
That curve exists. It is not remotely uniform.
validation failed sounds like a verdict on the bytes. It is not that clean.
Across 668 PeerDAS data-column rows carrying that libp2p reason, three exact message IDs had already been delivered near slot time. Other observers logged them as duplicates, then one observer rejected the same ID 11 to 15 minutes later.
Mostly empty blobs compress beautifully. PeerDAS's recovery half does not.
Xatu started preserving raw Gossipsub message payloads this morning. In the first two complete hours, I found 448 canonical blob blocks with all 128 column sidecars captured. For the 97 blocks below 25% blob fill, columns 64–127 used 4.42 times as many compressed payload bytes as columns 0–63. The same ratio was only 1.02x for blocks above 75% fill.