QUIC restarts, slow problems: udpgrm to the rescue

In this project I wrote quite some eBPF. I was constantly hitting verifier limits, and like everyone initially just reordering variables and sprinkling "inline" or "noinline" everywhere. Of course that wasn't sustainable.

And it turns out - there is now some reasonable tooling to understand verifier!

For stack problems, clang accepts `-s` which prints stack requirement per function, like so:

  ebpf/ebpf_aes128.c:180  AES_ECB_encrypt        32   static
  ebpf/ebpf_sha256.c:34   sha256_calc_chunk      64   static
  ebpf/ebpf_sha256.c:123  sha256_hmac            40   static
  ebpf/ebpf_quic.c:86     compute_hp_mask        24   static
  ebpf/ebpf_quic.c:242    decrypt_quic           16   static
  ebpf/ebpf_quic.c:193    _do_decrypt_quic_loop  16   static

And for instruction count, I was able to feed the logs from verbose verifier (during loading) into code-coverage tooling, and count stuff up. Anyway, the reuseorg prog takes 100k verifier instruction count/paths:

  \*\* verifier instruction count \*\*
  udpgrm_reuseport_prog  processed  103486  insns  
  udpgrm_setsockopt      processed  9260    insns  
  udpgrm_getsockopt      processed  4215    insns  
  udpgrm_bpf_bind6       processed  75      insns

While 100k is lower than 1m instr count limit, it's still a lot. And reordering some loop or introducing some "if" often makes that count baloon.

Anyway - with stack and instr statistics it's way easier to make sense of verifier problems.