perf(mpsc): batch writev flush (threshold=8) for TCP — +7% pps

Add configurable batch flush threshold to SpinSink. When threshold > 1,
MpscTunnelSender::send accumulates packets in FramedWriter's BufList
without flushing. After N packets, poll_flush triggers a single writev()
syscall instead of N individual write() syscalls.

Implementation:
- SpinSink: pending_count + batch_threshold atomics
- MpscTunnelSender::send: flush every N packets via writev
- Default threshold=1 (per-packet flush, safe for handshake/control)
- Settable via set_batch_threshold() through PeerConn → Peer → PeerManager
- Bench: HOTPATH_BATCH env var, set after convergence

Batch threshold must be 1 during handshake (control packets are
request-response, can't be delayed). Bench sets threshold=8 only after
routes converge.

Benchmark (no hotpath, 3 runs avg):
  TCP batch=1:  985K pps
  TCP batch=8:  1,053K pps (+7%)
  Ring:         unchanged (flush is no-op for RingSink)
  UDP:          unchanged (flush is no-op for RingSink)

MpscTunnelSender::send avg: 343ns → 213ns (-38%, with hotpath) —
writev writes 8 Bytes in one syscall vs 8 write() calls.

All 210 peers tests pass. 6 netns tests fail (require root, unchanged).
This commit is contained in:
fanyang
2026-06-29 02:58:08 +08:00
parent 0e665eafc6
commit 90c45d2964
5 changed files with 60 additions and 11 deletions
+11
View File
@@ -126,6 +126,17 @@ async fn main() {
let pm = inst_a.get_peer_manager();
let send_pkt = make_data_packet(src, "10.144.144.2", pkt_size);
let batch_threshold: u32 = std::env::var("HOTPATH_BATCH")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(1);
// After convergence, enable batch flush for writev optimization
if converged && batch_threshold > 1 {
pm.set_peer_conn_batch_threshold(batch_threshold);
println!("cpu_hotspot_ring: batch_threshold={}", batch_threshold);
}
let pipeline_depth: usize = std::env::var("HOTPATH_PIPELINE")
.ok()
.and_then(|s| s.parse().ok())