perf(data-plane): restore native throughput after host portability (#2452)

* perf(core): make data-plane idle check constant time Avoid scanning every DashMap shard for each peer packet when no data-plane flows are active. Publish the flow count before insertion and release it after removal so an Acquire load is a safe O(1) idle signal. Reject count overflow and underflow instead of silently saturating. * test(perf): add repeatable two-node netns benchmark Create isolated underlay namespaces, pin both EasyTier cores and iperf3 endpoints, and measure a single TCP flow in both directions over either UDP or TCP peer transport. Keep every iperf3 JSON result and emit directional medians while cleaning up processes and namespaces on every exit path. * perf(tcp): preserve native owned stream halves Let each VirtualTcpSocket adapter consume itself into independent read and write halves. Portable adapters retain the generic shared split as a default. Use lock-free Tokio owned halves for native TCP and Unix streams so tunnel I/O no longer takes the generic split mutex on every poll. Cover full-duplex traffic and write-half shutdown. * perf(packet): preserve ownership across the Host seam Introduce an opaque, move-only HostPacket that retains core packet storage while exposing only the raw IP payload. Clear private headers before handing storage back to a native TUN adapter. Use an ownership-preserving bounded channel for native ingress and egress. Keep explicit copy adapters for Vec and WASI boundaries, and verify allocation identity, backpressure, shutdown, and end-to-end delivery. * perf(udp): preserve packet ownership through sessions Carry EasyTier tunnel packets through UDP session queues as owned values. Reuse the existing tunnel header for session framing instead of copying payloads into a second packet and rebuilding them on receive. Keep completion delivery for the public datagram socket API while removing the unused completion channel from streaming tunnel sends. Avoid the unconditional receive-side clone before QUIC routing is known. * perf(peer): publish packet filters as immutable snapshots Replace per-packet async and synchronous registry locks with ArcSwap snapshots. Permanent filters now need no activity checks, while managed registrations retain explicit acquire/release visibility. Closing a managed registration marks it inactive before atomically removing it. Existing snapshots keep in-flight filters alive, and registration mutations prune inactive entries while preserving newest-first order. * perf(instance): give native hosts direct packet egress Let the core create one bounded HostPacket channel and transfer its receiver directly to a PacketEgressHost during startup. Native TUN runtimes now consume that receiver without the intermediate PacketSink channel and forwarding task. Keep PacketSinkEgress as the compatibility adapter for callback and test hosts, and make receiver installation one-shot across desktop, mobile, and disabled runtimes. * perf(crypto): restore accelerated native AEAD backends Move Ring and OpenSSL implementations behind the core Encryptor seam. Portable builds continue selecting only supported backends. Restore historical precedence: OpenSSL, Ring, then RustCrypto. Keep backend availability consistent across secure transports and cover fixed-nonce wire compatibility between implementations. * perf(udp): receive native datagrams into owned buffers Extend the portable UDP socket seam with an owned-datagram receive path. Keep a compatible default for portable hosts. Native Unix sockets write recvmsg output directly into the final BytesMut allocation. This removes the per-packet stack-to-heap copy introduced by the portable socket boundary without exposing native socket resources to core. * perf(data-plane): remove portable hot-path overhead Restore native throughput lost while generalizing the host and UDP session layers. Read packet policy once per send, update traffic counters through registry guards, and preserve packet ownership while UDP dispatch borrows stable session state. Move UDP shutdown monitoring into a control task so forwarding avoids a select future per packet. Bound native datagram storage to 8 KiB, reject oversized sends, and drop truncated Unix receives. Keep accelerated AEAD selection warning-free when portable crypto features are also built. Cover session bounds, truncation, and idle shutdown with regression tests. * fix(udp): preserve portable datagram receive semantics Keep the public portable receive capacity at the theoretical UDP maximum instead of silently shrinking it to the native fast-path limit. Apply the 8 KiB session boundary after a complete portable receive, so Windows cannot turn an oversized datagram into a fatal listener error and other adapters cannot dispatch a truncated prefix. Cover dropping an oversized packet while the same portable socket continues to deliver the following valid datagram. * fix(ci): align feature gating with backend selection Compile the Ring implementation in production only when OpenSSL is not selected, while retaining it for cross-backend unit tests. Remove stale test imports and assert UDP dispatch results so the strict workspace Clippy job passes without suppressing diagnostics.
2026-09-03 09:35:41 +00:00 · 2026-07-26 22:54:43 +08:00
parent dc11298558
commit 7fb42c3b73
48 changed files with 2182 additions and 541 deletions
@@ -421,7 +421,18 @@ mod tests {
    }

    #[test]
-    #[cfg(all(feature = "aes-gcm", feature = "chacha20"))]
+    #[cfg(all(
+        any(
+            feature = "aes-gcm",
+            feature = "openssl-crypto",
+            feature = "ring-crypto"
+        ),
+        any(
+            feature = "chacha20",
+            feature = "openssl-crypto",
+            feature = "ring-crypto"
+        )
+    ))]
    fn peer_session_supports_asymmetric_algorithms() {
        let a: PeerId = 10;
        let b: PeerId = 20;
@@ -528,6 +528,27 @@ impl Default for TrustedKeyMapManager {
 /// `PeerContext` is intentionally scoped to `easytier-core::peers`; other core
 /// modules should depend on their own narrow DTOs or traits instead of treating
 /// this as a core-wide global context.
+#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
+pub(crate) struct PeerPacketPolicy {
+    pub(crate) disable_relay_data: bool,
+    pub(crate) p2p_only: bool,
+    pub(crate) latency_first: bool,
+    pub(crate) disable_p2p: bool,
+    pub(crate) lazy_p2p: bool,
+}
+
+impl PeerPacketPolicy {
+    fn from_flags(flags: &FlagsInConfig) -> Self {
+        Self {
+            disable_relay_data: flags.disable_relay_data,
+            p2p_only: flags.p2p_only,
+            latency_first: flags.latency_first && !flags.p2p_only,
+            disable_p2p: flags.disable_p2p,
+            lazy_p2p: flags.lazy_p2p,
+        }
+    }
+}
+
 pub(crate) trait PeerContext: Send + Sync {
    fn host_routing_policy(&self) -> HostRoutingPolicy {
        HostRoutingPolicy::default()
@@ -543,8 +564,12 @@ pub(crate) trait PeerContext: Send + Sync {
        FlagsInConfig::default()
    }

+    fn packet_policy(&self) -> PeerPacketPolicy {
+        PeerPacketPolicy::from_flags(&self.flags())
+    }
+
    fn disable_relay_data(&self) -> bool {
-        self.flags().disable_relay_data
+        self.packet_policy().disable_relay_data
    }

    fn secure_mode(&self) -> Option<SecureModeConfig> {
@@ -583,15 +608,6 @@ pub(crate) trait PeerContext: Send + Sync {
        }
    }

-    fn p2p_only(&self) -> bool {
-        self.flags().p2p_only
-    }
-
-    fn latency_first(&self) -> bool {
-        let flags = self.flags();
-        flags.latency_first && !flags.p2p_only
-    }
-
    fn proxy_cidrs(&self) -> Vec<Ipv4Cidr> {
        Vec::new()
    }
@@ -746,6 +762,10 @@ impl PeerContext for CorePeerContext {
        self.snapshot().flags.clone()
    }

+    fn packet_policy(&self) -> PeerPacketPolicy {
+        PeerPacketPolicy::from_flags(&self.snapshot().flags)
+    }
+
    fn host_routing_policy(&self) -> HostRoutingPolicy {
        self.snapshot().runtime.host_routing
    }
@@ -9,8 +9,8 @@ use std::{
 };

 use anyhow::Context;
+use arc_swap::ArcSwap;
 use dashmap::DashMap;
-use parking_lot::RwLock as SyncRwLock;
 use quanta::Instant;
 use serde::{Deserialize, Serialize};
 use tokio::sync::{
@@ -26,6 +26,7 @@ use crate::{
    config::{P2pPolicyFlags, PeerId, ProxyNetworkConfig},
    events::CoreEventSink,
    foundation::task::ExternalTaskSignal,
+    host::packet::{HostPacket, HostPacketSender},
    packet::{
        CompressorAlgo, PacketType, ZCPacket,
        compressor::{Compressor as _, DefaultCompressor},
@@ -42,8 +43,7 @@ use crate::{
 };

 use super::{
-    BoxNicPacketFilter, BoxPeerPacketFilter, PacketRecvChan, PacketRecvChanReceiver,
-    PeerPacketFilter,
+    BoxNicPacketFilter, BoxPeerPacketFilter, PacketRecvChanReceiver, PeerPacketFilter,
    acl::AclFilter,
    conn::{
        peer_conn::{PeerConn, PeerConnId},
@@ -52,7 +52,7 @@ use super::{
    },
    context::{
        ArcPeerContext, CorePeerContext, CorePeerContextAdapters, NetworkIdentity, PeerContext,
-        PeerStunInfoSource,
+        PeerPacketPolicy, PeerStunInfoSource,
    },
    credential_manager::{CredentialManager, CredentialStorage},
    error::Error,
@@ -401,7 +401,7 @@ pub(crate) async fn close_untrusted_credential_peers<F>(
 }

 struct NicPacketProcessor {
-    nic_channel: PacketRecvChan,
+    nic_channel: HostPacketSender,
 }

 #[async_trait::async_trait]
@@ -420,7 +420,10 @@ impl PeerPacketFilter for NicPacketProcessor {
                return None;
            }
            tracing::trace!(?packet, "send packet to nic channel");
-            let _ = self.nic_channel.send(packet).await;
+            let _ = self
+                .nic_channel
+                .send(HostPacket::from_core_packet(packet))
+                .await;
            None
        } else {
            Some(packet)
@@ -448,26 +451,58 @@ impl PeerPacketFilter for PeerRpcPacketProcessor {
    }
 }

+#[derive(Clone)]
 pub(crate) struct PeerPipelineEntry {
-    active: Arc<AtomicBool>,
-    filter: Arc<SyncRwLock<Option<Arc<dyn PeerPacketFilter + Send + Sync>>>>,
+    active: Option<Arc<AtomicBool>>,
+    filter: Arc<dyn PeerPacketFilter + Send + Sync>,
 }

+#[derive(Clone)]
 pub(crate) struct NicPipelineEntry {
-    active: Arc<AtomicBool>,
-    filter: Arc<SyncRwLock<Option<Arc<dyn super::NicPacketFilter + Send + Sync>>>>,
+    active: Option<Arc<AtomicBool>>,
+    filter: Arc<dyn super::NicPacketFilter + Send + Sync>,
+}
+
+type PeerPacketPipeline = Arc<ArcSwap<Vec<PeerPipelineEntry>>>;
+type NicPacketPipeline = Arc<ArcSwap<Vec<NicPipelineEntry>>>;
+
+impl PeerPipelineEntry {
+    fn filter_if_active(&self) -> Option<&Arc<dyn PeerPacketFilter + Send + Sync>> {
+        if self
+            .active
+            .as_ref()
+            .is_some_and(|active| !active.load(Ordering::Acquire))
+        {
+            return None;
+        }
+        Some(&self.filter)
+    }
+}
+
+impl NicPipelineEntry {
+    fn filter_if_active(&self) -> Option<&Arc<dyn super::NicPacketFilter + Send + Sync>> {
+        if self
+            .active
+            .as_ref()
+            .is_some_and(|active| !active.load(Ordering::Acquire))
+        {
+            return None;
+        }
+        Some(&self.filter)
+    }
 }

 #[derive(Clone)]
 pub(crate) struct PipelineRegistrationGuard {
    active: Arc<AtomicBool>,
-    release_filter: Arc<dyn Fn() + Send + Sync>,
+    unregister: Arc<dyn Fn() + Send + Sync>,
 }

 impl PipelineRegistrationGuard {
    pub fn close(&self) {
-        self.active.store(false, Ordering::Release);
-        (self.release_filter)();
+        if self.active.swap(false, Ordering::AcqRel) {
+            (self.unregister)();
+        }
    }
 }

@@ -477,36 +512,52 @@ impl Drop for PipelineRegistrationGuard {
    }
 }

-fn permanent_peer_pipeline_entry(filter: BoxPeerPacketFilter) -> Arc<PeerPipelineEntry> {
-    Arc::new(PeerPipelineEntry {
-        active: Arc::new(AtomicBool::new(true)),
-        filter: Arc::new(SyncRwLock::new(Some(Arc::from(filter)))),
-    })
+fn permanent_peer_pipeline_entry(filter: BoxPeerPacketFilter) -> PeerPipelineEntry {
+    PeerPipelineEntry {
+        active: None,
+        filter: Arc::from(filter),
+    }
 }

-fn permanent_nic_pipeline_entry(filter: BoxNicPacketFilter) -> Arc<NicPipelineEntry> {
-    Arc::new(NicPipelineEntry {
-        active: Arc::new(AtomicBool::new(true)),
-        filter: Arc::new(SyncRwLock::new(Some(Arc::from(filter)))),
-    })
+fn permanent_nic_pipeline_entry(filter: BoxNicPacketFilter) -> NicPipelineEntry {
+    NicPipelineEntry {
+        active: None,
+        filter: Arc::from(filter),
+    }
 }

 fn managed_peer_pipeline_entry(
    filter: BoxPeerPacketFilter,
-) -> (Arc<PeerPipelineEntry>, PipelineRegistrationGuard) {
+    pipeline: &PeerPacketPipeline,
+) -> (PeerPipelineEntry, PipelineRegistrationGuard) {
    let active = Arc::new(AtomicBool::new(true));
-    let filter = Arc::new(SyncRwLock::new(Some(Arc::from(filter))));
-    let release_filter = filter.clone();
+    let weak_pipeline = Arc::downgrade(pipeline);
+    let registration = active.clone();
    (
-        Arc::new(PeerPipelineEntry {
-            active: active.clone(),
-            filter,
-        }),
+        PeerPipelineEntry {
+            active: Some(active.clone()),
+            filter: Arc::from(filter),
+        },
        PipelineRegistrationGuard {
            active,
-            release_filter: Arc::new(move || {
-                let filter = release_filter.write().take();
-                drop(filter);
+            unregister: Arc::new(move || {
+                let Some(pipeline) = weak_pipeline.upgrade() else {
+                    return;
+                };
+                pipeline.rcu(|current| {
+                    Arc::new(
+                        current
+                            .iter()
+                            .filter(|entry| {
+                                entry
+                                    .active
+                                    .as_ref()
+                                    .is_none_or(|active| !Arc::ptr_eq(active, &registration))
+                            })
+                            .cloned()
+                            .collect(),
+                    )
+                });
            }),
        },
    )
@@ -515,63 +566,101 @@ fn managed_peer_pipeline_entry(
 #[cfg(any(feature = "proxy-packet", test))]
 fn managed_nic_pipeline_entry(
    filter: BoxNicPacketFilter,
-) -> (Arc<NicPipelineEntry>, PipelineRegistrationGuard) {
+    pipeline: &NicPacketPipeline,
+) -> (NicPipelineEntry, PipelineRegistrationGuard) {
    let active = Arc::new(AtomicBool::new(true));
-    let filter = Arc::new(SyncRwLock::new(Some(Arc::from(filter))));
-    let release_filter = filter.clone();
+    let weak_pipeline = Arc::downgrade(pipeline);
+    let registration = active.clone();
    (
-        Arc::new(NicPipelineEntry {
-            active: active.clone(),
-            filter,
-        }),
+        NicPipelineEntry {
+            active: Some(active.clone()),
+            filter: Arc::from(filter),
+        },
        PipelineRegistrationGuard {
            active,
-            release_filter: Arc::new(move || {
-                let filter = release_filter.write().take();
-                drop(filter);
+            unregister: Arc::new(move || {
+                let Some(pipeline) = weak_pipeline.upgrade() else {
+                    return;
+                };
+                pipeline.rcu(|current| {
+                    Arc::new(
+                        current
+                            .iter()
+                            .filter(|entry| {
+                                entry
+                                    .active
+                                    .as_ref()
+                                    .is_none_or(|active| !Arc::ptr_eq(active, &registration))
+                            })
+                            .cloned()
+                            .collect(),
+                    )
+                });
            }),
        },
    )
 }

-#[cfg(any(feature = "proxy-packet", test))]
-async fn remove_managed_nic_pipeline_entry(
-    pipeline: &RwLock<Vec<Arc<NicPipelineEntry>>>,
-    registration: &PipelineRegistrationGuard,
-) {
-    registration.close();
-    pipeline
-        .write()
-        .await
-        .retain(|entry| !Arc::ptr_eq(&entry.active, &registration.active));
+fn append_peer_pipeline(pipeline: &PeerPacketPipeline, entry: PeerPipelineEntry) {
+    pipeline.rcu(|current| {
+        let mut next = Vec::with_capacity(current.len() + 1);
+        next.extend(
+            current
+                .iter()
+                .filter(|entry| {
+                    entry
+                        .active
+                        .as_ref()
+                        .is_none_or(|active| active.load(Ordering::Acquire))
+                })
+                .cloned(),
+        );
+        next.push(entry.clone());
+        Arc::new(next)
+    });
+}
+
+fn append_nic_pipeline(pipeline: &NicPacketPipeline, entry: NicPipelineEntry) {
+    pipeline.rcu(|current| {
+        let mut next = Vec::with_capacity(current.len() + 1);
+        next.extend(
+            current
+                .iter()
+                .filter(|entry| {
+                    entry
+                        .active
+                        .as_ref()
+                        .is_none_or(|active| active.load(Ordering::Acquire))
+                })
+                .cloned(),
+        );
+        next.push(entry.clone());
+        Arc::new(next)
+    });
 }

 async fn init_packet_process_pipeline(
-    peer_packet_process_pipeline: &RwLock<Vec<Arc<PeerPipelineEntry>>>,
-    nic_channel: PacketRecvChan,
+    peer_packet_process_pipeline: &PeerPacketPipeline,
+    nic_channel: HostPacketSender,
    peer_rpc_tspt_sender: UnboundedSender<ZCPacket>,
 ) {
    // for tun/tap ip/eth packet.
-    peer_packet_process_pipeline
-        .write()
-        .await
-        .push(permanent_peer_pipeline_entry(Box::new(
-            NicPacketProcessor { nic_channel },
-        )));
+    append_peer_pipeline(
+        peer_packet_process_pipeline,
+        permanent_peer_pipeline_entry(Box::new(NicPacketProcessor { nic_channel })),
+    );

    // for peer rpc packet
-    peer_packet_process_pipeline
-        .write()
-        .await
-        .push(permanent_peer_pipeline_entry(Box::new(
-            PeerRpcPacketProcessor {
-                peer_rpc_tspt_sender,
-            },
-        )));
+    append_peer_pipeline(
+        peer_packet_process_pipeline,
+        permanent_peer_pipeline_entry(Box::new(PeerRpcPacketProcessor {
+            peer_rpc_tspt_sender,
+        })),
+    );
 }

 async fn add_route<T>(
-    peer_packet_process_pipeline: &RwLock<Vec<Arc<PeerPipelineEntry>>>,
+    peer_packet_process_pipeline: &PeerPacketPipeline,
    peers: Arc<PeerMap>,
    foreign_network_client: Arc<ForeignNetworkClient>,
    foreign_network_manager: Arc<ForeignNetworkManager>,
@@ -581,10 +670,10 @@ async fn add_route<T>(
    T: Route + PeerPacketFilter + Send + Sync + 'static,
 {
    // for route
-    peer_packet_process_pipeline
-        .write()
-        .await
-        .push(permanent_peer_pipeline_entry(Box::new(route.clone())));
+    append_peer_pipeline(
+        peer_packet_process_pipeline,
+        permanent_peer_pipeline_entry(Box::new(route.clone())),
+    );

    let _route_id = route
        .open(Box::new(PeerManagerRouteInterface {
@@ -614,9 +703,9 @@ pub struct PeerManagerCore {
    peers: Arc<PeerMap>,
    peer_rpc_mgr: Arc<super::peer_rpc::PeerRpcManager>,
    peer_rpc_tspt: Arc<RpcTransport>,
-    peer_packet_process_pipeline: Arc<RwLock<Vec<Arc<PeerPipelineEntry>>>>,
-    nic_packet_process_pipeline: Arc<RwLock<Vec<Arc<NicPipelineEntry>>>>,
-    nic_channel: PacketRecvChan,
+    peer_packet_process_pipeline: PeerPacketPipeline,
+    nic_packet_process_pipeline: NicPacketPipeline,
+    nic_channel: HostPacketSender,
    route_algo_inst: RouteAlgoInst,
    foreign_network_client: Arc<ForeignNetworkClient>,
    foreign_network_manager: Arc<ForeignNetworkManager>,
@@ -671,7 +760,7 @@ impl PeerManagerCore {
        mut config: PortablePeerManagerConfig,
        runtime_config: CoreRuntimeConfigStore,
        stun_info_source: Arc<dyn PeerStunInfoSource>,
-        nic_channel: PacketRecvChan,
+        nic_channel: HostPacketSender,
        public_ipv6_runtime: Arc<CorePublicIpv6Runtime>,
        events: Arc<dyn CoreEventSink>,
        credential_storage: Option<Arc<dyn CredentialStorage>>,
@@ -800,7 +889,7 @@ impl PeerManagerCore {
        my_peer_id: PeerId,
        core_context: Arc<CorePeerContext>,
        public_ipv6_runtime: Arc<dyn PublicIpv6Runtime>,
-        nic_channel: PacketRecvChan,
+        nic_channel: HostPacketSender,
        encryptor: Arc<dyn Encryptor + 'static>,
        is_secure_mode_enabled: bool,
        data_compress_algo: CompressorAlgo,
@@ -928,8 +1017,8 @@ impl PeerManagerCore {
                }
            },
        ));
-        let peer_packet_process_pipeline = Arc::new(RwLock::new(Vec::new()));
-        let nic_packet_process_pipeline = Arc::new(RwLock::new(Vec::new()));
+        let peer_packet_process_pipeline = Arc::new(ArcSwap::from_pointee(Vec::new()));
+        let nic_packet_process_pipeline = Arc::new(ArcSwap::from_pointee(Vec::new()));
        let exit_nodes = Arc::new(RwLock::new(exit_nodes));
        let relay_peer_map = super::relay_peer_map::new_relay_peer_map(
            peers.clone(),
@@ -1171,7 +1260,7 @@ impl PeerManagerCore {
        self.peer_session_store.clone()
    }

-    pub fn get_nic_channel(&self) -> PacketRecvChan {
+    pub(crate) fn get_nic_channel(&self) -> HostPacketSender {
        self.nic_channel.clone()
    }

@@ -1271,28 +1360,27 @@ impl PeerManagerCore {

    pub async fn add_packet_process_pipeline(&self, pipeline: BoxPeerPacketFilter) {
        // newest pipeline will be executed first
-        self.peer_packet_process_pipeline
-            .write()
-            .await
-            .push(permanent_peer_pipeline_entry(pipeline));
+        append_peer_pipeline(
+            &self.peer_packet_process_pipeline,
+            permanent_peer_pipeline_entry(pipeline),
+        );
    }

    pub async fn add_nic_packet_process_pipeline(&self, pipeline: BoxNicPacketFilter) {
        // newest pipeline will be executed first
-        self.nic_packet_process_pipeline
-            .write()
-            .await
-            .push(permanent_nic_pipeline_entry(pipeline));
+        append_nic_pipeline(
+            &self.nic_packet_process_pipeline,
+            permanent_nic_pipeline_entry(pipeline),
+        );
    }

    pub(crate) async fn add_managed_packet_process_pipeline(
        &self,
        pipeline: BoxPeerPacketFilter,
    ) -> PipelineRegistrationGuard {
-        let (entry, guard) = managed_peer_pipeline_entry(pipeline);
-        let mut pipelines = self.peer_packet_process_pipeline.write().await;
-        pipelines.retain(|pipeline| pipeline.active.load(Ordering::Acquire));
-        pipelines.push(entry);
+        let (entry, guard) =
+            managed_peer_pipeline_entry(pipeline, &self.peer_packet_process_pipeline);
+        append_peer_pipeline(&self.peer_packet_process_pipeline, entry);
        guard
    }

@@ -1301,10 +1389,9 @@ impl PeerManagerCore {
        &self,
        pipeline: BoxNicPacketFilter,
    ) -> PipelineRegistrationGuard {
-        let (entry, guard) = managed_nic_pipeline_entry(pipeline);
-        let mut pipelines = self.nic_packet_process_pipeline.write().await;
-        pipelines.retain(|pipeline| pipeline.active.load(Ordering::Acquire));
-        pipelines.push(entry);
+        let (entry, guard) =
+            managed_nic_pipeline_entry(pipeline, &self.nic_packet_process_pipeline);
+        append_nic_pipeline(&self.nic_packet_process_pipeline, entry);
        guard
    }

@@ -1313,7 +1400,7 @@ impl PeerManagerCore {
        &self,
        registration: &PipelineRegistrationGuard,
    ) {
-        remove_managed_nic_pipeline_entry(&self.nic_packet_process_pipeline, registration).await;
+        registration.close();
    }

    pub async fn add_route<T>(&self, route: Arc<T>)
@@ -1321,7 +1408,7 @@ impl PeerManagerCore {
        T: Route + PeerPacketFilter + Send + Sync + 'static,
    {
        add_route(
-            self.peer_packet_process_pipeline.as_ref(),
+            &self.peer_packet_process_pipeline,
            self.peers.clone(),
            self.foreign_network_client.clone(),
            self.foreign_network_manager.clone(),
@@ -1332,16 +1419,28 @@ impl PeerManagerCore {
    }

    pub async fn remove_nic_packet_process_pipeline(&self, id: String) -> Result<(), Error> {
-        let mut pipelines = self.nic_packet_process_pipeline.write().await;
-        if let Some(pos) = pipelines.iter().position(|pipeline| {
-            let filter = pipeline.filter.read().clone();
-            filter.is_some_and(|filter| filter.id() == id)
-        }) {
-            pipelines.remove(pos);
-            Ok(())
-        } else {
-            Err(Error::NotFound)
-        }
+        let snapshot = self.nic_packet_process_pipeline.load_full();
+        let Some(target) = snapshot
+            .iter()
+            .find(|entry| {
+                entry
+                    .filter_if_active()
+                    .is_some_and(|filter| filter.id() == id)
+            })
+            .map(|entry| entry.filter.clone())
+        else {
+            return Err(Error::NotFound);
+        };
+        self.nic_packet_process_pipeline.rcu(|current| {
+            Arc::new(
+                current
+                    .iter()
+                    .filter(|entry| !Arc::ptr_eq(&entry.filter, &target))
+                    .cloned()
+                    .collect(),
+            )
+        });
+        Ok(())
    }

    pub async fn send_msg_for_proxy(
@@ -1428,10 +1527,9 @@ impl PeerManagerCore {

    pub(crate) async fn clear_resources(&self) {
        self.stop().await;
-        let mut peer_pipeline = self.peer_packet_process_pipeline.write().await;
-        peer_pipeline.clear();
-        let mut nic_pipeline = self.nic_packet_process_pipeline.write().await;
-        nic_pipeline.clear();
+        self.peer_packet_process_pipeline
+            .store(Arc::new(Vec::new()));
+        self.nic_packet_process_pipeline.store(Arc::new(Vec::new()));

        self.peer_rpc_mgr.rpc_server().registry().unregister_all();
    }
@@ -1490,7 +1588,7 @@ impl PeerManagerCore {
        }

        init_packet_process_pipeline(
-            self.peer_packet_process_pipeline.as_ref(),
+            &self.peer_packet_process_pipeline,
            self.nic_channel.clone(),
            self.peer_rpc_tspt.packet_sender(),
        )
@@ -2061,7 +2159,7 @@ pub(crate) struct PeerOutboundPacketRouter {
    route: ArcRoute,
    foreign_network_client: Arc<ForeignNetworkClient>,
    relay_peer_map: Arc<RelayPeerMap>,
-    nic_packet_process_pipeline: Arc<RwLock<Vec<Arc<NicPipelineEntry>>>>,
+    nic_packet_process_pipeline: NicPacketPipeline,
    encryptor: Arc<dyn Encryptor>,
    data_compress_algo: CompressorAlgo,
    exit_nodes: Arc<RwLock<Vec<IpAddr>>>,
@@ -2081,7 +2179,7 @@ impl PeerOutboundPacketRouter {
        route: ArcRoute,
        foreign_network_client: Arc<ForeignNetworkClient>,
        relay_peer_map: Arc<RelayPeerMap>,
-        nic_packet_process_pipeline: Arc<RwLock<Vec<Arc<NicPipelineEntry>>>>,
+        nic_packet_process_pipeline: NicPacketPipeline,
        encryptor: Arc<dyn Encryptor>,
        data_compress_algo: CompressorAlgo,
        exit_nodes: Arc<RwLock<Vec<IpAddr>>>,
@@ -2128,12 +2226,17 @@ impl PeerOutboundPacketRouter {
        }
    }

-    fn mark_recent_traffic(&self, dst_peer_id: PeerId) {
-        let flags = self.context.flags();
-        self.recent_traffic
-            .mark(dst_peer_id, flags.disable_p2p, flags.lazy_p2p, |peer_id| {
-                self.has_directly_connected_conn(peer_id)
-            });
+    fn mark_recent_traffic_with_policy(
+        &self,
+        dst_peer_id: PeerId,
+        packet_policy: PeerPacketPolicy,
+    ) {
+        self.recent_traffic.mark(
+            dst_peer_id,
+            packet_policy.disable_p2p,
+            packet_policy.lazy_p2p,
+            |peer_id| self.has_directly_connected_conn(peer_id),
+        );
    }

    async fn run_nic_packet_process_pipeline(&self, data: &mut ZCPacket) -> bool {
@@ -2148,12 +2251,9 @@ impl PeerOutboundPacketRouter {
            return false;
        }

-        for pipeline in self.nic_packet_process_pipeline.read().await.iter().rev() {
-            if !pipeline.active.load(Ordering::Acquire) {
-                continue;
-            }
-            let filter = pipeline.filter.read().clone();
-            if let Some(filter) = filter {
+        let pipelines = self.nic_packet_process_pipeline.load_full();
+        for pipeline in pipelines.iter().rev() {
+            if let Some(filter) = pipeline.filter_if_active() {
                let _ = filter.try_process_packet_from_nic(data).await;
            }
        }
@@ -2161,8 +2261,12 @@ impl PeerOutboundPacketRouter {
        true
    }

-    fn check_p2p_only_before_send(&self, dst_peer_id: PeerId) -> Result<(), Error> {
-        if self.context.p2p_only() && !self.peers.has_peer(dst_peer_id) {
+    fn check_p2p_only_before_send(
+        &self,
+        dst_peer_id: PeerId,
+        packet_policy: PeerPacketPolicy,
+    ) -> Result<(), Error> {
+        if packet_policy.p2p_only && !self.peers.has_peer(dst_peer_id) {
            return Err(Error::RouteError(None));
        }
        Ok(())
@@ -2235,8 +2339,9 @@ impl PeerOutboundPacketRouter {
        mut msg: ZCPacket,
        dst_peer_id: PeerId,
    ) -> Result<(), Error> {
-        self.mark_recent_traffic(dst_peer_id);
-        self.check_p2p_only_before_send(dst_peer_id)?;
+        let packet_policy = self.context.packet_policy();
+        self.mark_recent_traffic_with_policy(dst_peer_id, packet_policy);
+        self.check_p2p_only_before_send(dst_peer_id, packet_policy)?;

        self.counters
            .compress_tx_bytes_before
@@ -2397,9 +2502,10 @@ impl PeerOutboundPacketRouter {
        if !self.run_nic_packet_process_pipeline(&mut msg).await {
            return Ok(());
        }
+        let packet_policy = self.context.packet_policy();
        let cur_to_peer_id = msg.peer_manager_header().unwrap().to_peer_id.into();
        if cur_to_peer_id != 0 {
-            self.mark_recent_traffic(cur_to_peer_id);
+            self.mark_recent_traffic_with_policy(cur_to_peer_id, packet_policy);
            return send_msg_internal(
                self.peers.as_ref(),
                &self.foreign_network_client,
@@ -2437,10 +2543,9 @@ impl PeerOutboundPacketRouter {
            .compress_tx_bytes_after
            .add(msg.buf_len() as u64);

-        let is_latency_first = self.context.latency_first();
        msg.mut_peer_manager_header()
            .unwrap()
-            .set_latency_first(is_latency_first)
+            .set_latency_first(packet_policy.latency_first)
            .set_exit_node(is_exit_node);

        let mut errs: Vec<Error> = vec![];
@@ -2449,9 +2554,9 @@ impl PeerOutboundPacketRouter {
        let should_mark_recent_traffic = should_mark_recent_traffic_for_fanout(total_dst_peers);
        for (i, peer_id) in dst_peers.iter().enumerate() {
            if should_mark_recent_traffic {
-                self.mark_recent_traffic(*peer_id);
+                self.mark_recent_traffic_with_policy(*peer_id, packet_policy);
            }
-            if let Err(e) = self.check_p2p_only_before_send(*peer_id) {
+            if let Err(e) = self.check_p2p_only_before_send(*peer_id, packet_policy) {
                errs.push(e);
                continue;
            }
@@ -2523,7 +2628,7 @@ pub(crate) struct PeerPacketRouter {
    packet_recv: PacketRecvChanReceiver,
    my_peer_id: PeerId,
    peers: Arc<PeerMap>,
-    peer_packet_process_pipeline: Arc<RwLock<Vec<Arc<PeerPipelineEntry>>>>,
+    peer_packet_process_pipeline: PeerPacketPipeline,
    foreign_client: Arc<ForeignNetworkClient>,
    relay_peer_map: Arc<RelayPeerMap>,
    foreign_network_manager: Arc<ForeignNetworkManager>,
@@ -2545,7 +2650,7 @@ impl PeerPacketRouter {
        packet_recv: PacketRecvChanReceiver,
        my_peer_id: PeerId,
        peers: Arc<PeerMap>,
-        peer_packet_process_pipeline: Arc<RwLock<Vec<Arc<PeerPipelineEntry>>>>,
+        peer_packet_process_pipeline: PeerPacketPipeline,
        foreign_client: Arc<ForeignNetworkClient>,
        relay_peer_map: Arc<RelayPeerMap>,
        foreign_network_manager: Arc<ForeignNetworkManager>,
@@ -2786,12 +2891,9 @@ impl PeerPacketRouter {
            let mut processed = false;
            let mut zc_packet = Some(ret);
            tracing::trace!(?zc_packet, "try_process_packet_from_peer");
-            for pipeline in self.peer_packet_process_pipeline.read().await.iter().rev() {
-                if !pipeline.active.load(Ordering::Acquire) {
-                    continue;
-                }
-                let filter = pipeline.filter.read().clone();
-                if let Some(filter) = filter {
+            let pipelines = self.peer_packet_process_pipeline.load_full();
+            for pipeline in pipelines.iter().rev() {
+                if let Some(filter) = pipeline.filter_if_active() {
                    zc_packet = filter
                        .try_process_packet_from_peer(zc_packet.unwrap())
                        .await;
@@ -3099,17 +3201,15 @@ mod tests {
    use crate::{
        config::runtime::CoreRuntimeConfig,
        config::{CoreConfig, IpPrefix, NetworkIdentity, NodeConfig, ProxyNetworkConfig},
-        peers::{
-            context::{PeerContext, PeerEvent},
-            create_packet_recv_chan,
-        },
+        host::packet::{HostPacketSender, host_packet_channel},
+        peers::context::{PeerContext, PeerEvent},
        proto::common::{PeerFeatureFlag, StunInfo},
    };

    impl PeerManagerCore {
        pub(crate) fn new_portable_for_test(
            config: PortablePeerManagerConfig,
-            nic_channel: PacketRecvChan,
+            nic_channel: HostPacketSender,
        ) -> anyhow::Result<Self> {
            let runtime_config = CoreRuntimeConfigStore::new(
                CoreRuntimeConfig::default(),
@@ -3182,40 +3282,35 @@ mod tests {
        }
    }

-    #[tokio::test]
-    async fn managed_nic_pipeline_removal_waits_for_readers_and_drops_filter() {
+    #[test]
+    fn managed_nic_pipeline_removal_preserves_in_flight_snapshot() {
        let drops = Arc::new(AtomicUsize::new(0));
+        let pipeline = Arc::new(ArcSwap::from_pointee(Vec::new()));
        let (entry, registration) =
-            managed_nic_pipeline_entry(Box::new(DropCountingNicFilter(drops.clone())));
-        let pipeline = Arc::new(RwLock::new(vec![entry]));
-        let reader = pipeline.read().await;
-        let active_filter = reader[0].filter.read().clone().unwrap();
-        let remove_pipeline = pipeline.clone();
-        let remove_registration = registration.clone();
-        let removal = tokio::spawn(async move {
-            remove_managed_nic_pipeline_entry(&remove_pipeline, &remove_registration).await;
-        });
+            managed_nic_pipeline_entry(Box::new(DropCountingNicFilter(drops.clone())), &pipeline);
+        append_nic_pipeline(&pipeline, entry);
+        let reader = pipeline.load_full();

-        tokio::task::yield_now().await;
-        assert!(!removal.is_finished());
+        registration.close();
+
+        assert!(pipeline.load().is_empty());
        assert_eq!(drops.load(Ordering::Relaxed), 0);

        drop(reader);
-        drop(active_filter);
-        removal.await.unwrap();
-        assert!(pipeline.read().await.is_empty());
        assert_eq!(drops.load(Ordering::Relaxed), 1);
    }

    #[test]
    fn managed_pipeline_guard_releases_filter_without_a_runtime() {
        let drops = Arc::new(AtomicUsize::new(0));
+        let pipeline = Arc::new(ArcSwap::from_pointee(Vec::new()));
        let (entry, registration) =
-            managed_nic_pipeline_entry(Box::new(DropCountingNicFilter(drops.clone())));
+            managed_nic_pipeline_entry(Box::new(DropCountingNicFilter(drops.clone())), &pipeline);
+        append_nic_pipeline(&pipeline, entry);

        drop(registration);

-        assert!(entry.filter.read().is_none());
+        assert!(pipeline.load().is_empty());
        assert_eq!(drops.load(Ordering::Relaxed), 1);
    }

@@ -3258,7 +3353,7 @@ mod tests {
    fn build_portable_config_for_test(
        config: PortablePeerManagerConfig,
    ) -> anyhow::Result<PeerManagerCore> {
-        let (packet_tx, _packet_rx) = create_packet_recv_chan();
+        let (packet_tx, _packet_rx) = host_packet_channel();
        PeerManagerCore::new_portable_for_test(config, packet_tx)
    }

@@ -3305,7 +3400,11 @@ mod tests {
        );
    }

-    #[cfg(not(feature = "aes-gcm"))]
+    #[cfg(not(any(
+        feature = "aes-gcm",
+        feature = "openssl-crypto",
+        feature = "ring-crypto"
+    )))]
    #[tokio::test]
    async fn portable_peer_manager_rejects_requested_unavailable_aes() {
        let mut config = PortablePeerManagerConfig::new(portable_runtime_config("portable-net"));
@@ -3382,7 +3481,7 @@ mod tests {
        let public_ipv6_runtime =
            CorePublicIpv6Runtime::new(runtime_config.clone(), Arc::new(()), Arc::new(()));
        let events = Arc::new(CountingPeerEventSink::default());
-        let (packet_tx, _packet_rx) = create_packet_recv_chan();
+        let (packet_tx, _packet_rx) = host_packet_channel();

        let core = PeerManagerCore::new(
            config,
@@ -3417,7 +3516,7 @@ mod tests {
            }),
        };
        config.snapshot.set_acl_groups(Some(&acl));
-        let (packet_tx, _packet_rx) = create_packet_recv_chan();
+        let (packet_tx, _packet_rx) = host_packet_channel();

        let core = PeerManagerCore::new_portable_for_test(config, packet_tx).unwrap();

@@ -3563,7 +3662,7 @@ mod tests {
    async fn portable_peer_manager_rejects_inconsistent_network_names() {
        let mut runtime = portable_runtime_config("identity-net");
        runtime.core.node.network_name = "node-net".to_owned();
-        let (packet_tx, _packet_rx) = create_packet_recv_chan();
+        let (packet_tx, _packet_rx) = host_packet_channel();

        let result = PeerManagerCore::new_portable_for_test(
            PortablePeerManagerConfig::new(runtime),
@@ -37,6 +37,13 @@ enum CachedPeerTrafficCounters {
 }

 impl CachedPeerTrafficCounters {
+    fn add_sample(&self, bytes: u64) {
+        match self {
+            CachedPeerTrafficCounters::Unknown(counters)
+            | CachedPeerTrafficCounters::Resolved(counters) => counters.add_sample(bytes),
+        }
+    }
+
    fn counters(&self) -> TrafficCounters {
        match self {
            CachedPeerTrafficCounters::Unknown(counters)
@@ -95,7 +102,7 @@ impl LogicalTrafficMetrics {
        if let Some(entry) = self.per_peer.get(&peer_id)
            && entry.value().is_resolved()
        {
-            entry.value().counters().add_sample(bytes);
+            entry.value().add_sample(bytes);
            return;
        }