MpscTunnelSender now supports two modes:
- Channel mode (existing): try_send to tokio mpsc → receiver task → sink
- Direct mode (new): MpscTunnelSender holds Arc<Mutex<sink>> directly,
bypassing the channel + receiver task entirely
PeerConn uses new_direct to skip the channel intermediary.
Benchmark result: pps unchanged (~245K). The async fn overhead of
Mutex::lock().await + SinkExt::feed().await + SinkExt::flush().await
(~2us) is comparable to channel try_send (~2us). The bottleneck is
the Sink trait's async poll machinery, not the channel itself.
However, this change provides:
- RingSink timing now fully visible (start_send 10ns, poll_ready 13ns,
poll_flush 17ns = 40ns/pkt total)
- Reduced architectural complexity (no receiver task for PeerConn)
- Foundation for a sync fast path using RingSink::try_send directly
Reveals RingSink operation costs:
poll_ready: 13ns/call (0.57% total)
start_send: <26ns/call (below top-15 threshold)
poll_flush: below top-15 threshold
Confirms ring tunnel sink operations are ~33ns/pkt total,
vs 2.04us for MpscTunnelSender::send (which routes through
tokio mpsc channel). The channel intermediary adds ~2us/pkt
of pure overhead.
channel(32) was frequently full under high pps, causing try_send to
fail and fall back to send().await (semaphore wait). Increasing to 1024
reduces fallback frequency.
Benchmark (4 threads, 1400B, 15s):
MpscTunnelSender::send avg: 2.23us → 2.01us (-220ns)
MpscTunnelSender::send P95: 6.39us → 5.74us (-650ns)
send_msg_internal avg: 2.67us → 2.45us (-220ns)
pps: ~250K (unchanged, receiver-bound)
pps unchanged because bottleneck moved to receiver (forward_one_round →
sink.feed/flush). The 220ns/pkt saving is pure CPU efficiency gain.
Splits the 1.04us gap between send_msg_by_ip and send_msg_internal:
try_compress_and_encrypt: 386ns (37%) ← AES-GCM encrypt + zstd compress
get_msg_dst_peer_ipv4: 161ns (15%) ← IP→peer_id route lookup
run_nic_packet_process_pipeline: 121ns (12%) ← ACL check
other (fill_hdr + counters): ~440ns (36%)
The 386ns encrypt is the largest optimization opportunity in this gap:
ring tunnel is in-process, so application-layer encryption may be skippable.
Adds hotpath::measure to two critical blind spots in the send chain:
1. MpscTunnelSender::send — the tokio mpsc channel send point, which
accounts for 84% of PeerConn::send_msg wall time (2.33us/pkt).
2. PeerManager::send_msg_by_ip — the top-level packet send entry point,
revealing a 1.04us gap between send_msg_by_ip and send_msg_internal
(encryption + routing + ACL + fan-out).
Full send chain timing now visible:
send_msg_by_ip: 3.83us
└─ send_msg_internal: 2.79us (gap: 1.04us = encrypt + route + ACL)
└─ MpscTunnelSender::send: 2.33us (84% of internal)
Two optimizations to reduce per-packet TrafficMetricRecorder overhead:
1. Batch CounterHandle updates (TRAFFIC_BATCH_SIZE=128): accumulate
bytes/packets in AtomicU64, flush to CounterHandle (and its
touch()/Instant::now()) only every 128 packets. Reduces touch
calls from 4/pkt to 0.03/pkt.
2. Sync fast path: record_tx_fast/record_rx_fast handle the common
case (peer already resolved) without entering async fn or cloning
TrafficCounters. Falls back to async record_tx/record_rx only for
first packet to a new/unresolved peer.
Tests use BATCH_SIZE=1 via cfg(test) for exact counter validation.
Benchmark (4 threads, 1400B, 15s, 3 runs):
Before: 246K pps, send_msg_internal avg 3.13us
After: 250K pps, send_msg_internal avg 3.05us
Delta: +1.6% pps, -80ns/pkt
All 11 traffic_metrics + send_msg_internal tests pass.
Replace all UnsafeCell-based counters with safe alternatives:
- New ShardedCounter: per-thread TLS accumulation (thread_local crate)
with periodic publish to static AtomicU64. Zero atomic RMW on hot path.
- ACL RuleStatsTracker: remove unsafe Arc<RuleStats> raw pointer
mutation, use two ShardedCounter fields.
- ZCPacket: replace set_len on uninitialized BytesMut with
write_bytes + copy_nonoverlapping before set_len.
- StatsManager UnsafeCounter/MetricData: remove UnsafeCell and
unsafe impl Send/Sync, wrap ShardedCounter. last_updated uses
AtomicU64 epoch millis.
- Throughput: replace UnsafeCell with AtomicU64.
- secure_datagram: fix grace-window test timestamp.
MpscTunnelSender::send now tries try_send first, falling back to
send().await only when the channel is full. try_send bypasses the
tokio batch_semaphore Acquire::poll + add_permits_locked machinery
(~9.4% of CPU in samply profiling), which is pure overhead when the
channel has capacity.
In the ring-tunnel bench (4 threads, 1400B, 15s) the channel(32) fast
path hits >99%, so the fallback rarely triggers.
Benchmark improvement:
pps: 230K -> 246K (+7.0%)
send_msg_internal avg: 3.25us -> 3.13us (-120ns/pkt)
forward_one_round calls: 2.46M -> 706K (-71%, bigger batches)
All mpsc tests pass.
Replace has_peer(dst_peer_id) + send_msg_directly() with a single
get_peer_by_id() call, eliminating one redundant dashmap contains_key
query (~50-100ns) per packet on the direct-peer happy path.
send_msg_directly is no longer called from send_msg_internal but remains
available for other callers. All 7 send_msg_internal tests pass.
Benchmark (4 threads, 1400B pkts, 15s):
Before: 234K pps, send_msg_internal avg 3.26us
After: 230K pps, send_msg_internal avg 3.25us
Delta within noise; ~10-50ns/pkt saved as expected for one fewer hash.
Replace std::time::Instant with quanta::Instant on per-packet, per-RPC,
and per-session paths. TSC-based, ~5ns vs ~25ns per now() call.
Reuses the existing `extern crate self as hotpath` alias so
`use hotpath::instant::Instant;` resolves to the same quanta type with
or without the hotpath feature. Leaves tokio::time::Instant and
smoltcp::time::Instant untouched.
* refactor(web): use generated proto network types
* fix(core): preserve dumped config flags
* test(web): cover config flag save paths
* fix(ci): use system protoc before frontend codegen
* fix(ci): serialize frontend-lib builds
Handle TUN receive errors by marking the fake TCP stack closed and
clearing registered sockets instead of panicking.
Refuse new sockets on closed stacks and let listeners recreate stacks
when the reader task exits.
Remove quinn-plaintext to fix connection errors caused
by different hash values across platforms.
On x64, maintain compatibility with quinn-plaintext.
Add config server client support for the C FFI and Android JNI bindings.
Reuse the existing easytier::web_client::run_web_client path and
NetworkInstanceManager; OHOS is unchanged.
Report successful remote config apply/delete operations through a
callback, with one JSON event per affected instance.
Keep the config server client and FFI data plane mutually exclusive: once
either side is in use, the other side returns an error instead of sharing
lifecycle state.
Use pbjson to support string deserialization for enum fields
This allows TOML configs like:
chainType = "Inbound"
instead of:
chainType = 1
- Maintain backward compatibility with integer values
- Default serialization format is now string
TomlConfigLoader::new_from_str() always calls NetworkIdentity::new()
with unwrap_or_default() on network_secret, converting None to ''.
This creates a non-zero SHA256 digest, causing credential nodes loaded
from TOML to be misidentified as regular nodes (with network_secret),
which breaks Noise handshake authentication.
Fix: check if secure_mode is enabled AND network_secret is absent/empty,
and call NetworkIdentity::new_credential() in that case.
The same detection already exists in:
- core.rs (CLI path, via --credential flag)
- launcher.rs (GUI/web path, via gen_config)
This makes TOML config loading consistent with the other two entry points.
1. Overview
This PR adds data plane APIs to easytier-ffi:
TCP Outbound:
- data_plane_tcp_connect
- data_plane_tcp_read
- data_plane_tcp_write
- data_plane_tcp_close
TCP Listener:
- data_plane_tcp_bind
- data_plane_tcp_accept
- data_plane_tcp_listener_close
UDP:
- data_plane_udp_bind
- data_plane_udp_send_to
- data_plane_udp_recv_from
- data_plane_udp_close
2. Key Changes
The main changes are focused on:
- easytier-contrib/easytier-ffi/src/lib.rs: Added FFI interfaces;
made ERROR_MSG thread-safe.
- easytier/src/gateway/socks5.rs: Bridges the data plane to the
existing Socks5 server logic.
- Added EasyTierUdpSocket, mainly wrapping ref-counting and
critical object (e.g., Socks5EntrySet) hold & drop logic,
and exposing common fields (e.g., local_addr).
- Extended Socks5Server functionality to expose TCP and UDP
socket creation interfaces for FFI calls.
- Other files: Mostly pass-through logic.
- Added a relatively large Go usage example.
Adds a Linux-only socket_mark u32 config flag (CLI: --socket-mark, env:
ET_SOCKET_MARK, TOML/proto: flags.socket_mark, 0 = disabled) that is
applied as SO_MARK to every outbound underlay socket EasyTier creates:
TCP, UDP, QUIC, WebSocket, WireGuard connectors and listeners, plus the
FakeTCP decoy socket. Lets the host policy-route or filter EasyTier
underlay traffic with 'ip rule fwmark ...' or iptables -m mark.
Plumbing mirrors the existing bind_device pattern:
- FlagsInConfig.socket_mark (proto) + default 0 in gen_default_flags
- bind() builder gets a socket_mark arg; setup_socket2_ext calls
apply_socket_mark which is a no-op for mark=0 and on non-Linux
- TunnelConnector trait gets set_socket_mark(u32) default-no-op method
- IP-based connectors override; create_listener_by_url and the connector
factory pass mark from global_ctx flags
- QUIC threads mark through QuicEndpointManager::{server,connect}
- WebSocket/FakeTCP/TCP default-bind bypass paths apply mark via
socket2::SockRef::from(&tokio_socket)
- ForeignNetworkEntry propagates parent socket_mark into its derived ctx
Includes a Linux smoke test plus a CAP_NET_ADMIN-gated test that does a
getsockopt(SO_MARK) round-trip to confirm the kernel applied the value.
SO_MARK requires CAP_NET_ADMIN; ignored silently on non-Linux. FakeTCP's
TUN-written segments are not covered (kernel doesn't tag raw TUN
writes); operators relying on fwmark for FakeTCP must apply an iptables
rule on the FakeTCP TUN device separately.
Co-authored-by: Claude <noreply@anthropic.com>
This may helps games to find rooms in virtual network.
- add opt-in Windows UDP broadcast relay config flag and CLI/env plumbing
- capture local UDP broadcasts with Windows raw sockets, normalize packets, and inject them via PeerManager
Stop sharing PeerCenterServer state through a process-global map so local and foreign-network services cannot mix peer-center data when peer ids overlap.
When a node has public IPv6 addresses allocated by EasyTier, those addresses
are installed on the host's network interfaces. The system would then pick
them up as candidate source/destination addresses for underlay connections
(direct peer, UDP hole punch, bind addresses), causing overlay traffic to
loop back into the overlay itself.
Add a central predicate is_ip_easytier_managed_ipv6() and apply it at every
point where IPv6 addresses are selected for underlay use:
- Filter managed IPv6 from DNS-resolved connector addresses, including a
UDP socket getsockname check to detect whether the OS would route through
the overlay to reach a destination
- Skip managed IPv6 in bind address selection and STUN candidate filtering
- Strip managed IPv6 from GetIpListResponse RPC so peers never learn them
- Pass pre-resolved addresses to tunnel connectors to avoid re-resolution
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Avoid resolving Url::as_ref() to the full URL string before TunnelScheme
conversion. Add regression coverage for owned/borrowed URLs and the UDP
IPv6 hole-punch branch condition.
Co-authored-by: KKRainbow <443152178@qq.com>
fanyang