mirror/Easytier
1
0
Fork 0
mirror of https://github.com/EasyTier/EasyTier.git synced 2026-09-03 01:25:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(mobile): add embedded runtime and managed network updates (#2532)

Browse Source 
* feat(mobile): add embedded iOS runtime API

Add a thin panic-safe C ABI crate for embedding no-TUN instances
on iOS. Expose lifecycle, status, JSON-RPC, string ownership, and
error handling.

Build device and simulator XCFramework static libraries on macOS.
Add exact named-instance deletion to the iOS and Android wrappers.
Cover wrapper lifecycle and the port-forward patch flow on host
targets.

* fix(gateway): recover TCP port-forward listeners

Release an unusable TCP port-forward listener after an accept
failure. Retry binding until the forward is cancelled. Keep the old
listener released while rebinding so mobile sockets can recover.

Expose opt-in iOS diagnostics for listener and connection events.
Trace configuration removal and adapter shutdown. Add tests for
recovery, release-before-rebind, and cancellation.

* feat(web): persist incremental managed config patches

Add a revision-CAS PATCH contract for managed configs while keeping
the existing Full PUT path for compatibility and recovery.

Apply Full and Patch mutations with their revision in one SQLite
transaction. Reject ownership conflicts and invalidate revisions on
alternate web-owned writes.

Document limits, failure semantics, rollout order, and verification.
Cover delta updates, conflicts, idempotency, and transaction rollback.

* feat(web): apply managed config patches to live sessions

Carry Patch fences and touched instance IDs into live sessions.
Reconcile only those instances when the applied revision matches the
Patch base. Fall back to Full reconciliation for gaps and restarts.

Invalidate the applied revision around every direct runtime mutation.
Fence revision advancement with the runtime cache epoch so stale
reconcile rounds cannot overwrite a newer invalidation.

Require deletion responses to confirm each requested instance before
advancing the revision. Raise the managed PUT and PATCH body limit to
32 MiB and return typed conflicts for publisher recovery.

* fix(core): retry transient accepted TCP errors

Keep TCP tunnel listeners alive when an accepted socket fails during
upgrade with a retryable connection-state error.

Share the retryable I/O classifier with the socket listener. Cover a
rejected connection followed by success and propagation of permanent
errors.

* feat(core): add internal Peer Relay edge projection

Derive the local advertised OSPF row from physical adjacency and transport-authenticated credential relay coverage. Keep full local adjacency only in the temporary SPF snapshot so direct destinations retain a fallback route.

Leave Peer Relay disabled at the public configuration seam. A follow-up change can expose the preference without coupling route projection to credential reauthorization.

feat(config): expose Peer Relay routing preference

Add prefer_peer_relay to public protobuf, TOML, management patch, and
hosted runtime surfaces.

Read the preference from live peer context so runtime config updates take
effect. Refresh authenticated peer metadata when the option is enabled.

Cover dynamic enable and disable in a five-node, dual-admin credential
topology, including forwarded relay coverage and local fallback.
This commit is contained in:
KKRainbow
2026-08-28 00:43:26 +08:00
committed by GitHub
parent abf03ca521
commit 4a10d1c2b9
38 changed files with 5614 additions and 554 deletions
Download Patch File Download Diff File Expand all files Collapse all files
easytier-web/src/client_manager/managed_config.rs
+401 -182
View File
@@ -3,7 +3,6 @@ use std::{
sync::{Arc, Weak},
};
use anyhow::Context as _;
use dashmap::{DashMap, mapref::entry::Entry};
use easytier::{
common::config::ConfigSource,
@@ -13,11 +12,13 @@ use easytier::{
},
};
use easytier_core::management::config_source_from_rpc;
use easytier_core::management::remote_client::{
ListNetworkProps, PersistentConfig as _, Storage as _,
};
use easytier_core::management::remote_client::{PersistentConfig as _, Storage as _};
use super::storage::Storage;
use crate::db::{
ManagedConfigApplyResult, ManagedConfigExpectedRevision, ManagedConfigUpdate,
ManagedConfigUpsert,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) enum PersistedConfigSource {
@@ -31,7 +32,9 @@ pub(super) enum ExpectedConfigRevision<'a> {
}
#[derive(Debug, thiserror::Error)]
pub(super) enum ManagedConfigError {
pub(crate) enum ManagedConfigError {
#[error("invalid managed config update: {0}")]
Invalid(String),
#[error(
"managed config revision changed while reconciling: expected {expected:?}, current {current:?}"
)]
@@ -39,6 +42,16 @@ pub(super) enum ManagedConfigError {
expected: Option<String>,
current: Option<String>,
},
#[error("managed config instance {instance_id} is user-owned")]
OwnershipConflict { instance_id: uuid::Uuid },
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(super) enum ManagedConfigApplyStatus {
Applied {
deleted_web_instance_ids: Vec<uuid::Uuid>,
},
AlreadyApplied,
}
impl PersistedConfigSource {
@@ -111,8 +124,12 @@ fn remove_unused_managed_config_reconcile_lock(
});
}
pub(super) fn is_revision_conflict(error: &anyhow::Error) -> bool {
error.downcast_ref::<ManagedConfigError>().is_some()
#[cfg(test)]
fn is_revision_conflict(error: &anyhow::Error) -> bool {
matches!(
error.downcast_ref::<ManagedConfigError>(),
Some(ManagedConfigError::RevisionConflict { .. })
)
}
fn snake_to_lower_camel(key: &str) -> Option<String> {
@@ -188,103 +205,36 @@ fn normalize_network_config(
Ok(serde_json::from_value::<NetworkConfig>(network_config)?)
}
struct ExistingConfigSources {
sources: HashMap<uuid::Uuid, PersistedConfigSource>,
web_ids: HashSet<uuid::Uuid>,
}
struct NormalizedWebConfigs {
desired_ids: HashSet<uuid::Uuid>,
configs: HashMap<uuid::Uuid, NetworkConfig>,
}
async fn ensure_expected_config_revision(
storage: &Storage,
user_id: i32,
machine_id: uuid::Uuid,
expected_config_revision: ExpectedConfigRevision<'_>,
) -> anyhow::Result<()> {
let ExpectedConfigRevision::Exact(expected) = expected_config_revision else {
return Ok(());
};
let current = storage
.db()
.get_managed_config_revision((user_id, machine_id))
.await
.map_err(|e| anyhow::anyhow!("failed to get managed config revision: {:?}", e))?;
if current.as_deref() != expected {
return Err(ManagedConfigError::RevisionConflict {
expected: expected.map(str::to_string),
current,
}
.into());
}
Ok(())
}
async fn load_existing_config_sources(
storage: &Storage,
user_id: i32,
machine_id: uuid::Uuid,
) -> anyhow::Result<ExistingConfigSources> {
let existing_configs = storage
.db()
.list_network_configs((user_id, machine_id), ListNetworkProps::All)
.await
.map_err(|e| anyhow::anyhow!("failed to list existing network configs: {:?}", e))?;
let sources = existing_configs
.iter()
.filter_map(|cfg| {
uuid::Uuid::parse_str(&cfg.network_instance_id)
.ok()
.map(|inst_id| (inst_id, PersistedConfigSource::from_db(&cfg.source)))
})
.collect::<HashMap<_, _>>();
let web_ids = sources
.iter()
.filter_map(|(inst_id, source)| (*source == PersistedConfigSource::Web).then_some(*inst_id))
.collect::<HashSet<_>>();
Ok(ExistingConfigSources { sources, web_ids })
configs: Vec<ManagedConfigUpsert>,
}
fn normalize_desired_web_configs(
user_id: i32,
machine_id: uuid::Uuid,
desired_configs: Vec<crate::webhook::ManagedNetworkConfig>,
config_revision: Option<&str>,
existing_sources: &HashMap<uuid::Uuid, PersistedConfigSource>,
) -> anyhow::Result<NormalizedWebConfigs> {
let mut desired_ids = HashSet::with_capacity(desired_configs.len());
let mut configs = HashMap::with_capacity(desired_configs.len());
let mut configs = Vec::with_capacity(desired_configs.len());
for desired in desired_configs {
let inst_id = uuid::Uuid::parse_str(&desired.instance_id).with_context(|| {
format!(
let inst_id = uuid::Uuid::parse_str(&desired.instance_id).map_err(|_| {
ManagedConfigError::Invalid(format!(
"invalid desired web config instance id: {}",
desired.instance_id
)
))
})?;
if let Some(PersistedConfigSource::User) = existing_sources.get(&inst_id) {
if config_revision.is_some() {
anyhow::bail!(
"cannot persist managed config revision because instance {} is user-owned",
inst_id
);
}
tracing::warn!(
?user_id,
?machine_id,
instance_id = %inst_id,
"skip web config because a user-owned config already exists"
);
continue;
if !desired_ids.insert(inst_id) {
return Err(ManagedConfigError::Invalid(format!(
"duplicate managed config instance id: {inst_id}"
))
.into());
}
let config = normalize_network_config(desired.network_config, inst_id)?;
desired_ids.insert(inst_id);
configs.insert(inst_id, config);
let config = normalize_network_config(desired.network_config, inst_id)
.map_err(|error| ManagedConfigError::Invalid(error.to_string()))?;
configs.push(ManagedConfigUpsert {
instance_id: inst_id,
network_config: config,
});
}
Ok(NormalizedWebConfigs {
@@ -293,72 +243,21 @@ fn normalize_desired_web_configs(
})
}
async fn upsert_web_configs(
storage: &Storage,
user_id: i32,
machine_id: uuid::Uuid,
configs: HashMap<uuid::Uuid, NetworkConfig>,
) -> anyhow::Result<()> {
for (inst_id, config) in configs {
let updated = storage
.db()
.insert_or_update_web_network_config((user_id, machine_id), inst_id, config)
.await
.map_err(|e| {
anyhow::anyhow!("failed to persist web network config {}: {:?}", inst_id, e)
})?;
if !updated {
anyhow::bail!(
"cannot persist managed config revision because instance {} is user-owned",
inst_id
);
fn map_apply_result(result: ManagedConfigApplyResult) -> anyhow::Result<ManagedConfigApplyStatus> {
match result {
ManagedConfigApplyResult::Applied {
deleted_web_instance_ids,
} => Ok(ManagedConfigApplyStatus::Applied {
deleted_web_instance_ids,
}),
ManagedConfigApplyResult::AlreadyApplied => Ok(ManagedConfigApplyStatus::AlreadyApplied),
ManagedConfigApplyResult::RevisionConflict { expected, current } => {
Err(ManagedConfigError::RevisionConflict { expected, current }.into())
}
ManagedConfigApplyResult::OwnershipConflict { instance_id } => {
Err(ManagedConfigError::OwnershipConflict { instance_id }.into())
}
}
Ok(())
}
async fn delete_stale_web_configs(
storage: &Storage,
user_id: i32,
machine_id: uuid::Uuid,
existing_web_ids: &HashSet<uuid::Uuid>,
desired_ids: &HashSet<uuid::Uuid>,
) -> anyhow::Result<()> {
let stale_ids = existing_web_ids
.difference(desired_ids)
.copied()
.collect::<Vec<_>>();
if stale_ids.is_empty() {
return Ok(());
}
storage
.db()
.delete_web_network_configs((user_id, machine_id), &stale_ids)
.await
.map_err(|e| anyhow::anyhow!("failed to delete stale network configs: {:?}", e))?;
Ok(())
}
async fn persist_config_revision(
storage: &Storage,
user_id: i32,
machine_id: uuid::Uuid,
config_revision: Option<&str>,
) -> anyhow::Result<()> {
let Some(config_revision) = config_revision else {
return Ok(());
};
storage
.db()
.set_managed_config_revision((user_id, machine_id), config_revision)
.await
.map_err(|e| anyhow::anyhow!("failed to persist managed config revision: {:?}", e))?;
Ok(())
}
pub(super) async fn reconcile_web_source_configs(
@@ -368,34 +267,111 @@ pub(super) async fn reconcile_web_source_configs(
desired_configs: Vec<crate::webhook::ManagedNetworkConfig>,
config_revision: Option<&str>,
expected_config_revision: ExpectedConfigRevision<'_>,
) -> anyhow::Result<()> {
) -> anyhow::Result<ManagedConfigApplyStatus> {
if config_revision.is_some_and(|revision| revision.trim().is_empty()) {
return Err(
ManagedConfigError::Invalid("config_revision must not be empty".to_string()).into(),
);
}
let normalized = normalize_desired_web_configs(desired_configs)?;
let expected_revision = match expected_config_revision {
ExpectedConfigRevision::Any => ManagedConfigExpectedRevision::Any,
ExpectedConfigRevision::Exact(revision) => {
ManagedConfigExpectedRevision::Exact(revision.map(str::to_string))
}
};
let key = (user_id, machine_id);
let reconcile_lock = managed_config_reconcile_lock(key);
let result = async {
let _guard = reconcile_lock.lock().await;
ensure_expected_config_revision(storage, user_id, machine_id, expected_config_revision)
.await?;
let existing = load_existing_config_sources(storage, user_id, machine_id).await?;
let normalized = normalize_desired_web_configs(
user_id,
machine_id,
desired_configs,
config_revision,
&existing.sources,
)?;
upsert_web_configs(storage, user_id, machine_id, normalized.configs).await?;
delete_stale_web_configs(
storage,
user_id,
machine_id,
&existing.web_ids,
&normalized.desired_ids,
)
.await?;
persist_config_revision(storage, user_id, machine_id, config_revision).await?;
let result = storage
.db()
.apply_managed_config_update(
(user_id, machine_id),
ManagedConfigUpdate::Full {
upserts: normalized.configs,
target_revision: config_revision.map(str::to_string),
expected_revision,
},
)
.await
.map_err(|error| anyhow::anyhow!("failed to apply managed config Full: {error}"))?;
map_apply_result(result)
}
.await;
remove_unused_managed_config_reconcile_lock(key, &reconcile_lock);
result
}
Ok(())
pub(super) async fn patch_web_source_configs(
storage: &Storage,
user_id: i32,
machine_id: uuid::Uuid,
upserts: Vec<crate::webhook::ManagedNetworkConfig>,
delete_instance_ids: Vec<uuid::Uuid>,
config_revision: &str,
expected_config_revision: &str,
) -> anyhow::Result<ManagedConfigApplyStatus> {
let config_revision = config_revision.trim();
let expected_config_revision = expected_config_revision.trim();
if config_revision.is_empty() || expected_config_revision.is_empty() {
return Err(
ManagedConfigError::Invalid("Patch revisions must not be empty".to_string()).into(),
);
}
if config_revision == expected_config_revision {
return Err(ManagedConfigError::Invalid(
"Patch target revision must differ from expected revision".to_string(),
)
.into());
}
let normalized = normalize_desired_web_configs(upserts)?;
let mut delete_ids = HashSet::with_capacity(delete_instance_ids.len());
for instance_id in delete_instance_ids {
if !delete_ids.insert(instance_id) {
return Err(ManagedConfigError::Invalid(format!(
"duplicate managed config delete instance id: {instance_id}"
))
.into());
}
}
if let Some(instance_id) = delete_ids
.intersection(&normalized.desired_ids)
.next()
.copied()
{
return Err(ManagedConfigError::Invalid(format!(
"managed config instance {instance_id} cannot be upserted and deleted"
))
.into());
}
if normalized.configs.is_empty() && delete_ids.is_empty() {
return Err(ManagedConfigError::Invalid(
"Patch must contain an upsert or delete".to_string(),
)
.into());
}
let key = (user_id, machine_id);
let reconcile_lock = managed_config_reconcile_lock(key);
let result = async {
let _guard = reconcile_lock.lock().await;
let result = storage
.db()
.apply_managed_config_update(
(user_id, machine_id),
ManagedConfigUpdate::Patch {
upserts: normalized.configs,
delete_instance_ids: delete_ids.into_iter().collect(),
target_revision: config_revision.to_string(),
expected_revision: expected_config_revision.to_string(),
},
)
.await
.map_err(|error| anyhow::anyhow!("failed to apply managed config Patch: {error}"))?;
map_apply_result(result)
}
.await;
remove_unused_managed_config_reconcile_lock(key, &reconcile_lock);
@@ -516,6 +492,19 @@ mod tests {
use super::*;
fn managed_config(
instance_id: uuid::Uuid,
network_name: &str,
) -> crate::webhook::ManagedNetworkConfig {
crate::webhook::ManagedNetworkConfig {
instance_id: instance_id.to_string(),
network_config: json!({
"instance_id": instance_id.to_string(),
"network_name": network_name
}),
}
}
#[tokio::test]
async fn reconcile_web_source_configs_upserts_and_deletes_exact_set() {
let storage = Storage::new(crate::db::Db::memory_db().await);
@@ -812,12 +801,11 @@ mod tests {
let conflict = err
.downcast_ref::<ManagedConfigError>()
.expect("expected typed revision conflict");
match conflict {
ManagedConfigError::RevisionConflict { expected, current } => {
assert_eq!(expected.as_deref(), Some("rev-old"));
assert_eq!(current.as_deref(), Some("rev-new"));
}
}
let ManagedConfigError::RevisionConflict { expected, current } = conflict else {
panic!("unexpected managed config error: {conflict:?}");
};
assert_eq!(expected.as_deref(), Some("rev-old"));
assert_eq!(current.as_deref(), Some("rev-new"));
assert_eq!(
storage
.db()
@@ -837,6 +825,237 @@ mod tests {
);
}
#[tokio::test]
async fn patch_web_source_configs_applies_delta_and_is_idempotent() {
let storage = Storage::new(crate::db::Db::memory_db().await);
let user_id = storage
.db()
.auto_create_user("web-user-patch")
.await
.unwrap()
.id;
let machine_id = uuid::Uuid::new_v4();
let update_id = uuid::Uuid::new_v4();
let delete_id = uuid::Uuid::new_v4();
let missing_delete_id = uuid::Uuid::new_v4();
let add_id = uuid::Uuid::new_v4();
reconcile_web_source_configs(
&storage,
user_id,
machine_id,
vec![
managed_config(update_id, "before"),
managed_config(delete_id, "delete"),
],
Some("rev-1"),
ExpectedConfigRevision::Any,
)
.await
.unwrap();
let status = patch_web_source_configs(
&storage,
user_id,
machine_id,
vec![
managed_config(update_id, "after"),
managed_config(add_id, "added"),
],
vec![delete_id, missing_delete_id],
"rev-2",
"rev-1",
)
.await
.unwrap();
assert_eq!(
status,
ManagedConfigApplyStatus::Applied {
deleted_web_instance_ids: vec![delete_id],
}
);
let retry_status = patch_web_source_configs(
&storage,
user_id,
machine_id,
vec![managed_config(update_id, "ignored-on-idempotent-retry")],
vec![delete_id],
"rev-2",
"rev-1",
)
.await
.unwrap();
assert_eq!(retry_status, ManagedConfigApplyStatus::AlreadyApplied);
let updated = storage
.db()
.get_network_config((user_id, machine_id), &update_id.to_string())
.await
.unwrap()
.unwrap()
.get_network_config()
.unwrap();
assert_eq!(updated.network_name.as_deref(), Some("after"));
assert!(
storage
.db()
.get_network_config((user_id, machine_id), &delete_id.to_string())
.await
.unwrap()
.is_none()
);
assert!(
storage
.db()
.get_network_config((user_id, machine_id), &add_id.to_string())
.await
.unwrap()
.is_some()
);
assert_eq!(
storage
.db()
.get_managed_config_revision((user_id, machine_id))
.await
.unwrap()
.as_deref(),
Some("rev-2")
);
}
#[tokio::test]
async fn patch_web_source_configs_rejects_conflict_and_user_owned_delete() {
let storage = Storage::new(crate::db::Db::memory_db().await);
let user_id = storage
.db()
.auto_create_user("web-user-patch-conflict")
.await
.unwrap()
.id;
let machine_id = uuid::Uuid::new_v4();
let user_owned_id = uuid::Uuid::new_v4();
storage
.db()
.insert_or_update_user_network_config(
(user_id, machine_id),
user_owned_id,
NetworkConfig {
network_name: Some("user-owned".to_string()),
..Default::default()
},
ConfigSource::User,
)
.await
.unwrap();
storage
.db()
.set_managed_config_revision((user_id, machine_id), "rev-current")
.await
.unwrap();
let revision_error = patch_web_source_configs(
&storage,
user_id,
machine_id,
vec![managed_config(uuid::Uuid::new_v4(), "new")],
Vec::new(),
"rev-next",
"rev-stale",
)
.await
.unwrap_err();
assert!(is_revision_conflict(&revision_error));
let ownership_error = patch_web_source_configs(
&storage,
user_id,
machine_id,
Vec::new(),
vec![user_owned_id],
"rev-next",
"rev-current",
)
.await
.unwrap_err();
assert!(matches!(
ownership_error.downcast_ref::<ManagedConfigError>(),
Some(ManagedConfigError::OwnershipConflict { instance_id })
if *instance_id == user_owned_id
));
assert!(
storage
.db()
.get_network_config((user_id, machine_id), &user_owned_id.to_string())
.await
.unwrap()
.is_some()
);
assert_eq!(
storage
.db()
.get_managed_config_revision((user_id, machine_id))
.await
.unwrap()
.as_deref(),
Some("rev-current")
);
}
#[tokio::test]
async fn reconcile_web_source_configs_rolls_back_rows_when_revision_write_fails() {
let storage = Storage::new(crate::db::Db::memory_db().await);
let user_id = storage
.db()
.auto_create_user("web-user-rollback")
.await
.unwrap()
.id;
let machine_id = uuid::Uuid::new_v4();
let instance_id = uuid::Uuid::new_v4();
sqlx::query(
r#"
CREATE TRIGGER reject_managed_revision
BEFORE INSERT ON managed_config_revisions
WHEN NEW.config_revision = 'reject-revision'
BEGIN
SELECT RAISE(ABORT, 'forced revision failure');
END
"#,
)
.execute(&storage.db().inner())
.await
.unwrap();
reconcile_web_source_configs(
&storage,
user_id,
machine_id,
vec![managed_config(instance_id, "must-rollback")],
Some("reject-revision"),
ExpectedConfigRevision::Any,
)
.await
.unwrap_err();
assert!(
storage
.db()
.get_network_config((user_id, machine_id), &instance_id.to_string())
.await
.unwrap()
.is_none()
);
assert!(
storage
.db()
.get_managed_config_revision((user_id, machine_id))
.await
.unwrap()
.is_none()
);
}
#[test]
fn managed_config_reconcile_lock_reuses_live_entry_and_replaces_stale_entry() {
let key = (i32::MIN, uuid::Uuid::new_v4());
easytier-web/src/client_manager/mod.rs
+127 -10
View File
@@ -19,7 +19,7 @@ use easytier_core::{
tunnel::{Tunnel, web_security},
};
use maxminddb::geoip2;
use session::{Location, Session};
use session::{Location, ManagedConfigRevisionDelta, Session};
use storage::{Storage, StorageToken};
use crate::FeatureFlags;
@@ -28,15 +28,13 @@ use tokio::task::JoinSet;
use crate::db::{Db, UserIdInDb, entity::user_running_network_configs};
pub(crate) use managed_config::ManagedConfigError;
#[derive(rust_embed::Embed)]
#[folder = "resources/"]
#[include = "geoip2-cn.mmdb"]
struct GeoipDb;
pub fn is_managed_config_revision_conflict(error: &anyhow::Error) -> bool {
managed_config::is_revision_conflict(error)
}
fn load_geoip_db(geoip_db: Option<String>) -> Option<maxminddb::Reader<Vec<u8>>> {
if let Some(path) = geoip_db {
match maxminddb::Reader::open_readfile(&path) {
@@ -228,7 +226,7 @@ impl ClientManager {
Some("") => managed_config::ExpectedConfigRevision::Exact(None),
Some(revision) => managed_config::ExpectedConfigRevision::Exact(Some(revision)),
};
managed_config::reconcile_web_source_configs(
let status = managed_config::reconcile_web_source_configs(
&self.storage,
user_id,
machine_id,
@@ -237,16 +235,80 @@ impl ClientManager {
expected_config_revision,
)
.await?;
if let Some(config_revision) = config_revision
if matches!(
status,
managed_config::ManagedConfigApplyStatus::Applied { .. }
) && let Some(config_revision) = config_revision
&& let Some(session) = self.get_session_by_machine_id(user_id, &machine_id)
{
session
.notify_config_revision_changed(user_id, machine_id, config_revision)
.notify_full_config_revision_changed(user_id, machine_id, config_revision)
.await;
}
Ok(())
}
pub async fn patch_managed_network_configs(
&self,
user_id: UserIdInDb,
machine_id: uuid::Uuid,
upserts: Vec<ManagedNetworkConfig>,
delete_instance_ids: Vec<uuid::Uuid>,
config_revision: String,
expected_config_revision: String,
) -> anyhow::Result<()> {
let config_revision = config_revision.trim().to_string();
let expected_config_revision = expected_config_revision.trim().to_string();
let upsert_instance_ids = upserts
.iter()
.map(|config| config.instance_id.clone())
.collect();
let status = managed_config::patch_web_source_configs(
&self.storage,
user_id,
machine_id,
upserts,
delete_instance_ids,
&config_revision,
&expected_config_revision,
)
.await?;
if let managed_config::ManagedConfigApplyStatus::Applied {
deleted_web_instance_ids,
} = status
&& let Some(session) = self.get_session_by_machine_id(user_id, &machine_id)
{
session
.notify_patch_config_revision_changed(
user_id,
machine_id,
ManagedConfigRevisionDelta {
expected_revision: expected_config_revision,
target_revision: config_revision,
upsert_instance_ids,
delete_instance_ids: deleted_web_instance_ids
.into_iter()
.map(|instance_id| instance_id.to_string())
.collect(),
},
)
.await;
}
Ok(())
}
pub async fn invalidate_applied_config_revision(
&self,
user_id: UserIdInDb,
machine_id: uuid::Uuid,
) {
if let Some(session) = self.get_session_by_machine_id(user_id, &machine_id) {
session
.invalidate_applied_config_revision(user_id, machine_id)
.await;
}
}
pub async fn get_heartbeat_requests(&self, client_url: &url::Url) -> Option<HeartbeatRequest> {
let s = self.client_sessions.get(client_url)?.clone();
s.data().read().await.req()
@@ -390,7 +452,10 @@ mod tests {
use axum::{Json, Router, extract::State, routing::post};
use easytier::{
common::{MachineIdOptions, config::NetworkConfigExt},
common::{
MachineIdOptions,
config::{ConfigSource, NetworkConfigExt},
},
instance::factory::{
NativeInstanceManager, native_compact_instance_manager_with_runtime,
native_instance_manager,
@@ -402,7 +467,9 @@ mod tests {
},
web_client::{WebClient, run_web_client},
};
use easytier_core::management::remote_client::Storage as RemoteStorage;
use easytier_core::management::remote_client::{
RemoteClientManager as _, Storage as RemoteStorage,
};
use serde_json::json;
use sqlx::Executor;
@@ -693,6 +760,29 @@ mod tests {
.unwrap()
}
async fn wait_for_applied_revision(
manager: &ClientManager,
user_id: i32,
machine_id: uuid::Uuid,
revision: &str,
) {
tokio::time::timeout(Duration::from_secs(12), async {
loop {
let applied = manager
.get_session_by_machine_id(user_id, &machine_id)
.map(|session| async move { session.applied_config_revision().await });
if let Some(applied) = applied
&& applied.await.as_deref() == Some(revision)
{
break;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
})
.await
.unwrap();
}
async fn start_web_client_for_test(
config_server_addr: std::net::SocketAddr,
machine_id: uuid::Uuid,
@@ -1081,6 +1171,33 @@ mod tests {
config.network_name.as_deref() == Some("managed-initial")
})
.await;
wait_for_applied_revision(&mgr, user_id, machine_id, "rev-initial").await;
// Runtime-only mutations do not change SQLite. Invalidate the Session
// applied fence and verify the existing revision is fully reconciled
// before a later targeted Patch may rely on it as a base.
let mut drifted: NetworkConfig =
serde_json::from_value(initial_managed_network_config(instance_id)).unwrap();
drifted.network_name = Some("runtime-only-drift".to_string());
mgr.handle_run_network_instance_with_source(
(user_id, machine_id),
drifted,
false,
ConfigSource::Web,
)
.await
.unwrap();
wait_for_runtime_config(&core_manager, instance_id, |config| {
config.network_name.as_deref() == Some("runtime-only-drift")
})
.await;
mgr.invalidate_applied_config_revision(user_id, machine_id)
.await;
wait_for_runtime_config(&core_manager, instance_id, |config| {
config.network_name.as_deref() == Some("managed-initial")
})
.await;
wait_for_applied_revision(&mgr, user_id, machine_id, "rev-initial").await;
// Online revision update: web-owned running config is fully overwritten
// when non-hot-patch flags such as enable_kcp_proxy change.
easytier-web/src/client_manager/runtime_reconcile.rs
+25
View File
@@ -64,6 +64,7 @@ fn hot_patch_base(config: &NetworkConfig) -> anyhow::Result<NetworkConfig> {
config.port_forwards.clear();
config.proxy_cidrs.clear();
config.disable_relay_data = None;
config.prefer_peer_relay = None;
// VPN portal clients are diffed separately; the listener identity
// (address and private key) decides between patch and recreate.
config.vpn_portal_config = None;
@@ -211,6 +212,10 @@ fn normalized_disable_relay_data(config: &NetworkConfig) -> anyhow::Result<bool>
Ok(config.gen_config()?.get_flags().disable_relay_data)
}
fn normalized_prefer_peer_relay(config: &NetworkConfig) -> anyhow::Result<bool> {
Ok(config.gen_config()?.get_flags().prefer_peer_relay)
}
fn normalized_vpn_portal(config: &NetworkConfig) -> anyhow::Result<Option<RuntimeVpnPortalConfig>> {
Ok(config.gen_config()?.get_vpn_portal_config())
}
@@ -316,6 +321,12 @@ fn web_source_runtime_patch(
patch.disable_relay_data = Some(desired_disable_relay_data);
}
let current_prefer_peer_relay = normalized_prefer_peer_relay(current)?;
let desired_prefer_peer_relay = normalized_prefer_peer_relay(desired)?;
if current_prefer_peer_relay != desired_prefer_peer_relay {
patch.prefer_peer_relay = Some(desired_prefer_peer_relay);
}
match (
normalized_vpn_portal(current)?,
normalized_vpn_portal(desired)?,
@@ -943,6 +954,20 @@ mod tests {
assert_eq!(patch.disable_relay_data, Some(true));
}
#[test]
fn runtime_patch_updates_peer_relay_preference_independently() {
let current = config_with_port_forwards(Vec::new());
let mut desired = current.clone();
desired.prefer_peer_relay = Some(true);
let patch = web_source_runtime_patch(&current, &desired)
.expect("build patch")
.expect("hot patch");
assert_eq!(patch.prefer_peer_relay, Some(true));
assert_eq!(patch.disable_relay_data, None);
}
#[test]
fn runtime_patch_still_rejects_unsupported_flag_change() {
let current = config_with_port_forwards(Vec::new());
easytier-web/src/client_manager/session.rs
+98 -2
View File
@@ -1,4 +1,5 @@
use std::{
collections::HashSet,
fmt::Debug,
str::FromStr as _,
sync::Arc,
@@ -42,6 +43,14 @@ enum SessionAuthState {
Invalid,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(super) struct ManagedConfigRevisionDelta {
pub expected_revision: String,
pub target_revision: String,
pub upsert_instance_ids: HashSet<String>,
pub delete_instance_ids: HashSet<String>,
}
impl SessionAuthState {
fn is_authorized(self) -> bool {
matches!(self, Self::Authorized)
@@ -58,6 +67,8 @@ pub struct SessionData {
storage_token: Option<StorageToken>,
binding_version: Option<u64>,
applied_config_revision: Option<String>,
pending_managed_config_delta: Option<ManagedConfigRevisionDelta>,
runtime_config_epoch: u64,
notifier: broadcast::Sender<HeartbeatRequest>,
req: Option<HeartbeatRequest>,
location: Option<Location>,
@@ -88,6 +99,8 @@ impl SessionData {
storage_token: None,
binding_version: None,
applied_config_revision: None,
pending_managed_config_delta: None,
runtime_config_epoch: 0,
notifier: tx,
req: None,
location,
@@ -698,14 +711,14 @@ impl Session {
self.scoped_client::<ConfigRpcClientFactory<BaseController>>()
}
pub async fn notify_config_revision_changed(
pub(super) async fn notify_full_config_revision_changed(
&self,
user_id: i32,
machine_id: uuid::Uuid,
config_revision: String,
) {
let notify = {
let data = self.data.read().await;
let mut data = self.data.write().await;
if !data.auth_state.is_authorized() {
return;
}
@@ -719,6 +732,84 @@ impl Session {
if data.applied_config_revision.as_deref() == Some(config_revision.as_str()) {
return;
}
data.pending_managed_config_delta = None;
data.req.clone().map(|req| (data.notifier.clone(), req))
};
if let Some((notifier, req)) = notify {
let _ = notifier.send(req);
}
}
pub(super) async fn notify_patch_config_revision_changed(
&self,
user_id: i32,
machine_id: uuid::Uuid,
delta: ManagedConfigRevisionDelta,
) {
let notify = {
let mut data = self.data.write().await;
if !data.auth_state.is_authorized() {
return;
}
if !data
.storage_token
.as_ref()
.is_some_and(|token| token.user_id == user_id && token.machine_id == machine_id)
{
return;
}
if data.applied_config_revision.as_deref() == Some(delta.target_revision.as_str()) {
return;
}
// A Patch may drive a targeted runtime reconcile only when the
// connected Session has applied its exact base and no earlier
// Patch is still pending. Otherwise the normal Full reconcile is
// the safe convergence path.
data.pending_managed_config_delta = (data.applied_config_revision.as_deref()
== Some(delta.expected_revision.as_str())
&& data.pending_managed_config_delta.is_none())
.then_some(delta);
data.req.clone().map(|req| (data.notifier.clone(), req))
};
if let Some((notifier, req)) = notify {
let _ = notifier.send(req);
}
}
pub(super) async fn invalidate_applied_config_revision(
&self,
user_id: i32,
machine_id: uuid::Uuid,
) {
let notify = {
let mut data = self.data.write().await;
if !data
.storage_token
.as_ref()
.is_some_and(|token| token.user_id == user_id && token.machine_id == machine_id)
{
return;
}
data.applied_config_revision = None;
data.pending_managed_config_delta = None;
data.runtime_config_epoch = data.runtime_config_epoch.wrapping_add(1);
data.req.clone().map(|req| (data.notifier.clone(), req))
};
if let Some((notifier, req)) = notify {
let _ = notifier.send(req);
}
}
pub(crate) async fn invalidate_runtime_config_for_direct_mutation(&self) {
let notify = {
let mut data = self.data.write().await;
if data.storage_token.is_none() {
return;
}
data.applied_config_revision = None;
data.pending_managed_config_delta = None;
data.runtime_config_epoch = data.runtime_config_epoch.wrapping_add(1);
data.req.clone().map(|req| (data.notifier.clone(), req))
};
if let Some((notifier, req)) = notify {
@@ -733,6 +824,11 @@ impl Session {
pub async fn get_heartbeat_req(&self) -> Option<HeartbeatRequest> {
self.data.read().await.req()
}
#[cfg(test)]
pub(super) async fn applied_config_revision(&self) -> Option<String> {
self.data.read().await.applied_config_revision.clone()
}
}
#[cfg(test)]
easytier-web/src/client_manager/session/runtime_revision.rs
+507 -51
View File
@@ -2,8 +2,9 @@ use std::collections::{HashMap, HashSet};
use easytier::proto::{
api::manage::{
DeleteNetworkInstanceRequest, ListNetworkInstanceMetaRequest, ListNetworkInstanceRequest,
NetworkConfig, NetworkMeta, RunNetworkInstanceRequest,
DeleteNetworkInstanceRequest, DeleteNetworkInstanceResponse,
ListNetworkInstanceMetaRequest, ListNetworkInstanceRequest, NetworkConfig, NetworkMeta,
RunNetworkInstanceRequest,
},
rpc_types::controller::BaseController,
web::HeartbeatRequest,
@@ -11,7 +12,10 @@ use easytier::proto::{
use easytier_core::management::remote_client::{ListNetworkProps, Storage as _};
use tokio::sync::{RwLock, broadcast};
use super::{SessionConfigClient, SessionData, SessionRpcClient, SessionRpcService};
use super::{
ManagedConfigRevisionDelta, SessionConfigClient, SessionData, SessionRpcClient,
SessionRpcService,
};
use crate::client_manager::{
managed_config::{self, PersistedConfigSource},
runtime_reconcile,
@@ -75,48 +79,105 @@ pub(super) async fn reconcile_network_configs_on_heartbeat(
RoundStatus::Skip => continue,
RoundStatus::Stop => return,
};
if cache.runtime_config_epoch != round.runtime_config_epoch {
cache = ReconcileCache {
runtime_config_epoch: round.runtime_config_epoch,
..Default::default()
};
}
let running_metas =
match sync_running_sources_for_round(&mut rpc_client, &storage, &mut round).await {
RoundStatus::Ready(running_metas) => running_metas,
RoundStatus::Skip => continue,
RoundStatus::Stop => return,
};
let mut mutation_fence = RuntimeMutationFence::default();
let context = ReconcileRoundContext {
session_data: &session_data,
round: &round,
};
let desired_web_inst_ids =
managed_config::desired_web_source_instance_ids(&round.local_configs);
cache.runtime_configs.retain_desired(&desired_web_inst_ids);
let mut outcome = match cleanup_stale_web_source_instances(
&session_data,
&storage,
&mut rpc_client,
&round,
running_metas.as_deref(),
&desired_web_inst_ids,
&mut cache,
)
.await
{
RoundStatus::Ready(outcome) => outcome,
RoundStatus::Skip => continue,
RoundStatus::Stop => return,
let mut outcome = match &round.scope {
ReconcileScope::Full => {
let desired_web_inst_ids =
managed_config::desired_web_source_instance_ids(&round.local_configs);
cache.runtime_configs.retain_desired(&desired_web_inst_ids);
match cleanup_stale_web_source_instances(
&context,
&storage,
&mut rpc_client,
running_metas.as_deref(),
&desired_web_inst_ids,
&mut cache,
&mut mutation_fence,
)
.await
{
RoundStatus::Ready(outcome) => outcome,
RoundStatus::Skip => continue,
RoundStatus::Stop => return,
}
}
ReconcileScope::Patch {
delete_instance_ids,
..
} => {
match cleanup_patch_deleted_instances(
&session_data,
&mut rpc_client,
&round,
running_metas.as_deref(),
delete_instance_ids,
&mut cache,
&mut mutation_fence,
)
.await
{
RoundStatus::Ready(outcome) => outcome,
RoundStatus::Skip => continue,
RoundStatus::Stop => return,
}
}
};
outcome.merge(
reconcile_desired_runtime_configs(
&session_data,
&context,
&mut rpc_client,
&mut config_client,
&round,
&mut cache,
&mut mutation_fence,
)
.await,
);
if !outcome.has_failed {
cache.last_desired_web_inst_ids = Some(desired_web_inst_ids);
match &round.scope {
ReconcileScope::Full => {
cache.last_desired_web_inst_ids = Some(
managed_config::desired_web_source_instance_ids(&round.local_configs),
);
}
ReconcileScope::Patch {
upsert_instance_ids,
delete_instance_ids,
} => {
if let Some(last) = &mut cache.last_desired_web_inst_ids {
last.retain(|id| !delete_instance_ids.contains(id));
last.extend(upsert_instance_ids.iter().cloned());
}
}
}
}
match mark_config_revision_applied_if_current(&session_data, &storage, &round, &outcome)
.await
match mark_config_revision_applied_if_current(
&session_data,
&storage,
&round,
&outcome,
&mutation_fence,
)
.await
{
RoundStatus::Ready(()) | RoundStatus::Skip => {}
RoundStatus::Stop => return,
@@ -138,6 +199,7 @@ enum ConfigActionResult {
#[derive(Default)]
struct ReconcileCache {
runtime_config_epoch: u64,
cleaned_web_source_instances: bool,
last_desired_web_inst_ids: Option<HashSet<String>>,
runtime_configs: SessionRuntimeConfigCache,
@@ -191,6 +253,11 @@ struct ReconcileOutcome {
managed_revision_failed: bool,
}
#[derive(Default)]
struct RuntimeMutationFence {
started: bool,
}
impl ReconcileOutcome {
fn record_failure(&mut self, managed_revision_failed: bool) {
self.has_failed = true;
@@ -211,6 +278,41 @@ struct ReconcileRound {
local_configs: Vec<crate::db::entity::user_running_network_configs::Model>,
target_config_revision: Option<String>,
should_apply_runtime_revision: bool,
scope: ReconcileScope,
runtime_config_epoch: u64,
}
struct ReconcileRoundContext<'a> {
session_data: &'a std::sync::Weak<RwLock<SessionData>>,
round: &'a ReconcileRound,
}
#[derive(Debug, Clone, PartialEq, Eq)]
enum ReconcileScope {
Full,
Patch {
upsert_instance_ids: HashSet<String>,
delete_instance_ids: HashSet<String>,
},
}
fn select_reconcile_scope(
applied_revision: Option<&str>,
target_revision: Option<&str>,
pending_delta: Option<&ManagedConfigRevisionDelta>,
) -> ReconcileScope {
match pending_delta {
Some(delta)
if applied_revision == Some(delta.expected_revision.as_str())
&& target_revision == Some(delta.target_revision.as_str()) =>
{
ReconcileScope::Patch {
upsert_instance_ids: delta.upsert_instance_ids.clone(),
delete_instance_ids: delta.delete_instance_ids.clone(),
}
}
_ => ReconcileScope::Full,
}
}
async fn prepare_reconcile_round(
@@ -244,11 +346,16 @@ async fn prepare_reconcile_round(
}
};
let applied_config_revision = {
let (applied_config_revision, pending_delta, runtime_config_epoch) = {
let Some(data) = session_data.upgrade() else {
return RoundStatus::Stop;
};
data.read().await.applied_config_revision.clone()
let data = data.read().await;
(
data.applied_config_revision.clone(),
data.pending_managed_config_delta.clone(),
data.runtime_config_epoch,
)
};
let target_config_revision = match storage
.db
@@ -263,6 +370,15 @@ async fn prepare_reconcile_round(
};
let should_apply_runtime_revision =
target_config_revision.is_some() && target_config_revision != applied_config_revision;
let mut scope = if should_apply_runtime_revision {
select_reconcile_scope(
applied_config_revision.as_deref(),
target_config_revision.as_deref(),
pending_delta.as_ref(),
)
} else {
ReconcileScope::Full
};
let running_inst_ids = match running_instance_ids_for_round(
rpc_client,
&req,
@@ -277,14 +393,29 @@ async fn prepare_reconcile_round(
RoundStatus::Stop => return RoundStatus::Stop,
};
let local_configs = match storage
.db
.list_network_configs((user_id, machine_id), ListNetworkProps::EnabledOnly)
.await
{
Ok(configs) => configs,
let local_configs = match load_round_configs(storage, user_id, machine_id, &scope).await {
Ok(Some(configs)) => configs,
Ok(None) => {
tracing::warn!(
?user_id,
?machine_id,
"Managed config Patch no longer matches persisted rows; using Full reconcile"
);
scope = ReconcileScope::Full;
match storage
.db
.list_network_configs((user_id, machine_id), ListNetworkProps::EnabledOnly)
.await
{
Ok(configs) => configs,
Err(e) => {
tracing::error!("Failed to list network configs, error: {:?}", e);
return RoundStatus::Stop;
}
}
}
Err(e) => {
tracing::error!("Failed to list network configs, error: {:?}", e);
tracing::error!("Failed to load managed config Patch rows, error: {:?}", e);
return RoundStatus::Stop;
}
};
@@ -297,9 +428,50 @@ async fn prepare_reconcile_round(
local_configs,
target_config_revision,
should_apply_runtime_revision,
scope,
runtime_config_epoch,
})
}
async fn load_round_configs(
storage: &StorageInner,
user_id: i32,
machine_id: uuid::Uuid,
scope: &ReconcileScope,
) -> Result<Option<Vec<crate::db::entity::user_running_network_configs::Model>>, sea_orm::DbErr> {
let ReconcileScope::Patch {
upsert_instance_ids,
..
} = scope
else {
return storage
.db
.list_network_configs((user_id, machine_id), ListNetworkProps::EnabledOnly)
.await
.map(Some);
};
let mut instance_ids = upsert_instance_ids.iter().collect::<Vec<_>>();
instance_ids.sort_unstable();
let mut configs = Vec::with_capacity(instance_ids.len());
for instance_id in instance_ids {
let Some(config) = storage
.db
.get_network_config((user_id, machine_id), instance_id)
.await?
else {
return Ok(None);
};
if config.disabled
|| PersistedConfigSource::from_db(&config.source) != PersistedConfigSource::Web
{
return Ok(None);
}
configs.push(config);
}
Ok(Some(configs))
}
async fn running_instance_ids_for_round(
rpc_client: &mut SessionRpcClient,
req: &HeartbeatRequest,
@@ -375,7 +547,7 @@ async fn sync_running_sources_for_round(
%e,
"Failed to sync running network config sources"
);
} else if !metas.is_empty() {
} else if !metas.is_empty() && matches!(round.scope, ReconcileScope::Full) {
round.local_configs = match storage
.db
.list_network_configs(
@@ -408,14 +580,16 @@ async fn sync_running_sources_for_round(
}
async fn cleanup_stale_web_source_instances(
session_data: &std::sync::Weak<RwLock<SessionData>>,
context: &ReconcileRoundContext<'_>,
storage: &StorageInner,
rpc_client: &mut SessionRpcClient,
round: &ReconcileRound,
running_metas: Option<&[NetworkMeta]>,
desired_web_inst_ids: &HashSet<String>,
cache: &mut ReconcileCache,
mutation_fence: &mut RuntimeMutationFence,
) -> RoundStatus<ReconcileOutcome> {
let session_data = context.session_data;
let round = context.round;
let desired_changed = cache
.last_desired_web_inst_ids
.as_ref()
@@ -450,10 +624,10 @@ async fn cleanup_stale_web_source_instances(
let mut outcome = ReconcileOutcome::default();
if !should_delete_ids.is_empty() {
if !SessionRpcService::runtime_heartbeat_is_current(session_data, &round.req).await {
if !begin_managed_runtime_mutation(session_data, round, mutation_fence).await {
tracing::debug!(
machine_id = ?round.machine_id,
"skip stale cleanup because webhook session is no longer current"
"skip stale cleanup because the managed runtime fence is no longer current"
);
return RoundStatus::Skip;
}
@@ -471,10 +645,23 @@ async fn cleanup_stale_web_source_instances(
ret,
round.req.user_token
);
if ret.is_err() {
outcome.record_failure(true);
} else {
cache.runtime_configs.forget_many(&should_delete_inst_ids);
match ret {
Err(_) => outcome.record_failure(true),
Ok(response) => {
let undeleted_instance_ids =
retained_requested_instance_ids(response, &should_delete_inst_ids);
if undeleted_instance_ids.is_empty() {
cache.runtime_configs.forget_many(&should_delete_inst_ids);
} else {
tracing::warn!(
user_id = ?round.user_id,
machine_id = ?round.machine_id,
instance_ids = ?undeleted_instance_ids,
"Stale managed instances were retained by the runtime"
);
outcome.record_failure(true);
}
}
}
}
@@ -486,13 +673,128 @@ async fn cleanup_stale_web_source_instances(
RoundStatus::Ready(outcome)
}
async fn reconcile_desired_runtime_configs(
async fn cleanup_patch_deleted_instances(
session_data: &std::sync::Weak<RwLock<SessionData>>,
rpc_client: &mut SessionRpcClient,
config_client: &mut SessionConfigClient,
round: &ReconcileRound,
running_metas: Option<&[NetworkMeta]>,
delete_instance_ids: &HashSet<String>,
cache: &mut ReconcileCache,
mutation_fence: &mut RuntimeMutationFence,
) -> RoundStatus<ReconcileOutcome> {
let running_web_instance_ids: HashSet<String> = match running_metas {
Some(metas) => managed_config::running_web_source_instance_ids(
&round.running_inst_ids,
delete_instance_ids,
Some(metas),
)
.intersection(delete_instance_ids)
.cloned()
.collect(),
None => round
.running_inst_ids
.intersection(delete_instance_ids)
.cloned()
.collect(),
};
if running_web_instance_ids.is_empty() {
cache
.runtime_configs
.forget_many(delete_instance_ids.iter());
return RoundStatus::Ready(ReconcileOutcome::default());
}
if !begin_managed_runtime_mutation(session_data, round, mutation_fence).await {
tracing::debug!(
machine_id = ?round.machine_id,
"skip managed config Patch cleanup because the runtime fence is no longer current"
);
return RoundStatus::Skip;
}
let ret = rpc_client
.delete_network_instance(
BaseController::default(),
DeleteNetworkInstanceRequest {
inst_ids: managed_config::parse_instance_ids(
running_web_instance_ids.iter().cloned(),
),
},
)
.await;
tracing::info!(
user_id = ?round.user_id,
deleted_instance_ids = ?running_web_instance_ids,
"Apply managed config Patch deletions at runtime: {:?}",
ret
);
let mut outcome = ReconcileOutcome::default();
match ret {
Err(_) => outcome.record_failure(true),
Ok(response) => {
let undeleted_instance_ids =
retained_requested_instance_ids(response, &running_web_instance_ids);
if undeleted_instance_ids.is_empty() {
cache
.runtime_configs
.forget_many(delete_instance_ids.iter());
} else {
tracing::warn!(
user_id = ?round.user_id,
machine_id = ?round.machine_id,
instance_ids = ?undeleted_instance_ids,
"Managed config Patch deletion was retained by the runtime"
);
outcome.record_failure(true);
}
}
}
RoundStatus::Ready(outcome)
}
async fn begin_managed_runtime_mutation(
session_data: &std::sync::Weak<RwLock<SessionData>>,
round: &ReconcileRound,
mutation_fence: &mut RuntimeMutationFence,
) -> bool {
let Some(data) = session_data.upgrade() else {
return false;
};
let mut data = data.write().await;
if !SessionRpcService::runtime_heartbeat_is_current_locked(&data, &round.req)
|| data.runtime_config_epoch != round.runtime_config_epoch
{
return false;
}
if !mutation_fence.started {
data.applied_config_revision = None;
data.pending_managed_config_delta = None;
mutation_fence.started = true;
}
true
}
fn retained_requested_instance_ids(
response: DeleteNetworkInstanceResponse,
requested_instance_ids: &HashSet<String>,
) -> HashSet<String> {
response
.remain_inst_ids
.into_iter()
.map(|instance_id| uuid::Uuid::from(instance_id).to_string())
.filter(|instance_id| requested_instance_ids.contains(instance_id))
.collect()
}
async fn reconcile_desired_runtime_configs(
context: &ReconcileRoundContext<'_>,
rpc_client: &mut SessionRpcClient,
config_client: &mut SessionConfigClient,
cache: &mut ReconcileCache,
mutation_fence: &mut RuntimeMutationFence,
) -> ReconcileOutcome {
let session_data = context.session_data;
let round = context.round;
let mut outcome = ReconcileOutcome::default();
// After stale web-owned instances are removed, start every enabled
@@ -529,13 +831,13 @@ async fn reconcile_desired_runtime_configs(
let action_result = if should_reconcile_running_web_config {
reconcile_running_web_config(
session_data,
context,
rpc_client,
config_client,
round,
config,
desired_config,
&mut cache.runtime_configs,
mutation_fence,
)
.await
} else {
@@ -548,6 +850,7 @@ async fn reconcile_desired_runtime_configs(
round,
config,
desired_config.clone(),
mutation_fence,
)
.await;
if matches!(action_result, ConfigActionResult::Success)
@@ -599,14 +902,16 @@ async fn reconcile_desired_runtime_configs(
}
async fn reconcile_running_web_config(
session_data: &std::sync::Weak<RwLock<SessionData>>,
context: &ReconcileRoundContext<'_>,
rpc_client: &mut SessionRpcClient,
config_client: &mut SessionConfigClient,
round: &ReconcileRound,
config: &crate::db::entity::user_running_network_configs::Model,
desired_config: NetworkConfig,
runtime_config_cache: &mut SessionRuntimeConfigCache,
mutation_fence: &mut RuntimeMutationFence,
) -> ConfigActionResult {
let session_data = context.session_data;
let round = context.round;
if !SessionRpcService::runtime_heartbeat_is_current(session_data, &round.req).await {
tracing::debug!(
machine_id = ?round.machine_id,
@@ -633,6 +938,11 @@ async fn reconcile_running_web_config(
if !SessionRpcService::runtime_heartbeat_is_current(session_data, &round.req).await {
anyhow::bail!("webhook session is no longer current before runtime reconcile apply");
}
if !matches!(action, runtime_reconcile::RuntimeReconcileAction::None)
&& !begin_managed_runtime_mutation(session_data, round, mutation_fence).await
{
anyhow::bail!("managed runtime mutation fence is no longer current");
}
let observed_config = runtime_reconcile::apply_web_source_runtime_reconcile(
&mut *rpc_client,
&mut *config_client,
@@ -667,6 +977,7 @@ async fn run_missing_network_config(
round: &ReconcileRound,
config: &crate::db::entity::user_running_network_configs::Model,
desired_config: NetworkConfig,
mutation_fence: &mut RuntimeMutationFence,
) -> ConfigActionResult {
if !SessionRpcService::runtime_heartbeat_is_current(session_data, &round.req).await {
tracing::debug!(
@@ -677,6 +988,18 @@ async fn run_missing_network_config(
return ConfigActionResult::StopRound;
}
let source = PersistedConfigSource::from_db(&config.source);
if source == PersistedConfigSource::Web
&& !begin_managed_runtime_mutation(session_data, round, mutation_fence).await
{
tracing::debug!(
machine_id = ?round.machine_id,
instance_id = %config.network_instance_id,
"skip run network instance because the managed runtime fence is no longer current"
);
return ConfigActionResult::StopRound;
}
let ret = rpc_client
.run_network_instance(
BaseController::default(),
@@ -684,7 +1007,7 @@ async fn run_missing_network_config(
inst_id: Some(config.network_instance_id.clone().into()),
config: Some(desired_config),
overwrite: false,
source: PersistedConfigSource::from_db(&config.source).auto_run_rpc_source() as i32,
source: source.auto_run_rpc_source() as i32,
},
)
.await;
@@ -739,8 +1062,11 @@ async fn mark_config_revision_applied_if_current(
storage: &StorageInner,
round: &ReconcileRound,
outcome: &ReconcileOutcome,
mutation_fence: &RuntimeMutationFence,
) -> RoundStatus<()> {
if outcome.managed_revision_failed || !round.should_apply_runtime_revision {
if outcome.managed_revision_failed
|| (!round.should_apply_runtime_revision && !mutation_fence.started)
{
return RoundStatus::Ready(());
}
@@ -765,7 +1091,11 @@ async fn mark_config_revision_applied_if_current(
if !SessionRpcService::runtime_heartbeat_is_current_locked(&data, &round.req) {
return RoundStatus::Ready(());
}
if data.runtime_config_epoch != round.runtime_config_epoch {
return RoundStatus::Ready(());
}
data.applied_config_revision = round.target_config_revision.clone();
data.pending_managed_config_delta = None;
RoundStatus::Ready(())
}
@@ -798,6 +1128,132 @@ mod tests {
}
}
#[test]
fn patch_delete_requires_runtime_to_remove_every_requested_instance() {
let deleted_id = uuid::Uuid::new_v4();
let requested = HashSet::from([deleted_id.to_string()]);
assert_eq!(
retained_requested_instance_ids(
DeleteNetworkInstanceResponse {
remain_inst_ids: vec![deleted_id.into()],
},
&requested,
),
requested
);
assert!(
retained_requested_instance_ids(
DeleteNetworkInstanceResponse {
remain_inst_ids: Vec::new(),
},
&requested,
)
.is_empty()
);
}
#[tokio::test]
async fn managed_runtime_mutation_clears_old_applied_revision_before_side_effects() {
let machine_id = uuid::Uuid::new_v4();
let req = HeartbeatRequest {
user_token: "token".to_string(),
machine_id: Some(machine_id.into()),
..Default::default()
};
let storage =
crate::client_manager::storage::Storage::new(crate::db::Db::memory_db().await);
let client_url = url::Url::parse("http://127.0.0.1").unwrap();
let mut data = SessionData::new(
storage.weak_ref(),
client_url.clone(),
None,
std::sync::Arc::new(crate::FeatureFlags::default()),
std::sync::Arc::new(crate::webhook::WebhookConfig::new(
None, None, None, None, None,
)),
);
data.storage_token = Some(crate::client_manager::storage::StorageToken {
token: req.user_token.clone(),
client_url,
machine_id,
user_id: 7,
});
data.req = Some(req.clone());
data.auth_state = super::super::SessionAuthState::Authorized;
data.applied_config_revision = Some("rev-a".to_string());
data.pending_managed_config_delta = Some(revision_delta("rev-a", "rev-b"));
data.runtime_config_epoch = 11;
let session_data = std::sync::Arc::new(RwLock::new(data));
let round = ReconcileRound {
req,
machine_id,
user_id: 7,
running_inst_ids: HashSet::new(),
local_configs: Vec::new(),
target_config_revision: Some("rev-b".to_string()),
should_apply_runtime_revision: true,
scope: ReconcileScope::Full,
runtime_config_epoch: 11,
};
let mut mutation_fence = RuntimeMutationFence::default();
assert!(
begin_managed_runtime_mutation(
&std::sync::Arc::downgrade(&session_data),
&round,
&mut mutation_fence,
)
.await
);
let data = session_data.read().await;
assert!(mutation_fence.started);
assert_eq!(data.applied_config_revision, None);
assert_eq!(data.pending_managed_config_delta, None);
assert_eq!(data.runtime_config_epoch, 11);
}
fn revision_delta(base: &str, target: &str) -> ManagedConfigRevisionDelta {
ManagedConfigRevisionDelta {
expected_revision: base.to_string(),
target_revision: target.to_string(),
upsert_instance_ids: HashSet::from(["upsert".to_string()]),
delete_instance_ids: HashSet::from(["delete".to_string()]),
}
}
#[test]
fn exact_revision_delta_selects_targeted_reconcile() {
let delta = revision_delta("rev-1", "rev-2");
assert_eq!(
select_reconcile_scope(Some("rev-1"), Some("rev-2"), Some(&delta)),
ReconcileScope::Patch {
upsert_instance_ids: HashSet::from(["upsert".to_string()]),
delete_instance_ids: HashSet::from(["delete".to_string()]),
}
);
}
#[test]
fn revision_gap_uses_full_reconcile() {
let delta = revision_delta("rev-1", "rev-2");
assert_eq!(
select_reconcile_scope(Some("older"), Some("rev-2"), Some(&delta)),
ReconcileScope::Full
);
assert_eq!(
select_reconcile_scope(Some("rev-1"), Some("newer"), Some(&delta)),
ReconcileScope::Full
);
assert_eq!(
select_reconcile_scope(Some("rev-1"), Some("rev-2"), None),
ReconcileScope::Full
);
}
#[test]
fn session_runtime_config_cache_misses_unknown_instance() {
let cache = SessionRuntimeConfigCache::default();
easytier-web/src/client_manager/session/webhook_validation.rs
+1
View File
@@ -248,6 +248,7 @@ pub(super) async fn apply_rejected(
data.webhook_validation_dirty = false;
data.binding_version = None;
data.applied_config_revision = None;
data.pending_managed_config_delta = None;
let storage_token = data.storage_token.clone();
let disconnect_notification = storage_token.as_ref().and_then(|storage_token| {
data.webhook_connected_binding_version
easytier-web/src/db/mod.rs
+541 -133
View File
@@ -7,10 +7,11 @@ use easytier_core::management::remote_client::{ListNetworkProps, Storage};
use entity::user_running_network_configs;
use sea_orm::{
ColumnTrait as _, DatabaseConnection, DbErr, EntityTrait, QueryFilter as _, Set,
SqlxSqliteConnector, TransactionTrait as _, prelude::Expr, sea_query::OnConflict,
SqlxSqliteConnector, TransactionTrait as _, sea_query::OnConflict,
};
use sea_orm_migration::MigratorTrait as _;
use sqlx::{Sqlite, SqlitePool, migrate::MigrateDatabase as _, types::chrono};
use std::collections::{HashMap, HashSet};
#[cfg(unix)]
use std::os::unix::fs::PermissionsExt as _;
use uuid::Uuid;
@@ -20,6 +21,179 @@ use async_trait::async_trait;
pub type UserIdInDb = i32;
#[derive(Debug)]
pub(crate) struct ManagedConfigUpsert {
pub instance_id: Uuid,
pub network_config: NetworkConfig,
}
#[derive(Debug, Clone)]
pub(crate) enum ManagedConfigExpectedRevision {
Any,
Exact(Option<String>),
}
#[derive(Debug)]
pub(crate) enum ManagedConfigUpdate {
Full {
upserts: Vec<ManagedConfigUpsert>,
target_revision: Option<String>,
expected_revision: ManagedConfigExpectedRevision,
},
Patch {
upserts: Vec<ManagedConfigUpsert>,
delete_instance_ids: Vec<Uuid>,
target_revision: String,
expected_revision: String,
},
}
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum ManagedConfigApplyResult {
Applied {
deleted_web_instance_ids: Vec<Uuid>,
},
AlreadyApplied,
RevisionConflict {
expected: Option<String>,
current: Option<String>,
},
OwnershipConflict {
instance_id: Uuid,
},
}
fn sqlx_db_error(error: sqlx::Error) -> DbErr {
DbErr::Custom(error.to_string())
}
async fn read_managed_config_revision(
transaction: &mut sqlx::Transaction<'_, Sqlite>,
user_id: UserIdInDb,
device_id: Uuid,
) -> Result<Option<String>, DbErr> {
sqlx::query_scalar(
r#"
SELECT config_revision
FROM managed_config_revisions
WHERE user_id = ? AND device_id = ?
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.fetch_optional(&mut **transaction)
.await
.map_err(sqlx_db_error)
}
async fn clear_managed_config_revision(
transaction: &mut sqlx::Transaction<'_, Sqlite>,
user_id: UserIdInDb,
device_id: Uuid,
) -> Result<(), DbErr> {
sqlx::query(
r#"
DELETE FROM managed_config_revisions
WHERE user_id = ? AND device_id = ?
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.execute(&mut **transaction)
.await
.map_err(sqlx_db_error)?;
Ok(())
}
async fn write_managed_config_revision(
transaction: &mut sqlx::Transaction<'_, Sqlite>,
user_id: UserIdInDb,
device_id: Uuid,
config_revision: &str,
) -> Result<(), DbErr> {
let now = chrono::Local::now().fixed_offset();
sqlx::query(
r#"
INSERT INTO managed_config_revisions (
user_id, device_id, config_revision, create_time, update_time
) VALUES (?, ?, ?, ?, ?)
ON CONFLICT(user_id, device_id) DO UPDATE SET
config_revision = excluded.config_revision,
update_time = excluded.update_time
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.bind(config_revision)
.bind(now)
.bind(now)
.execute(&mut **transaction)
.await
.map_err(sqlx_db_error)?;
Ok(())
}
async fn read_config_source(
transaction: &mut sqlx::Transaction<'_, Sqlite>,
user_id: UserIdInDb,
device_id: Uuid,
instance_id: Uuid,
) -> Result<Option<String>, DbErr> {
sqlx::query_scalar(
r#"
SELECT source
FROM user_running_network_configs
WHERE user_id = ? AND device_id = ? AND network_instance_id = ?
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.bind(instance_id.to_string())
.fetch_optional(&mut **transaction)
.await
.map_err(sqlx_db_error)
}
async fn upsert_network_config(
transaction: &mut sqlx::Transaction<'_, Sqlite>,
user_id: UserIdInDb,
device_id: Uuid,
instance_id: Uuid,
network_config: &str,
source: ConfigSource,
web_only_update: bool,
) -> Result<bool, DbErr> {
let now = chrono::Local::now().fixed_offset();
let mut query = r#"
INSERT INTO user_running_network_configs (
user_id, device_id, network_instance_id, network_config,
source, disabled, create_time, update_time
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(user_id, device_id, network_instance_id) DO UPDATE SET
network_config = excluded.network_config,
source = excluded.source,
disabled = excluded.disabled,
update_time = excluded.update_time
"#
.to_string();
if web_only_update {
query.push_str(" WHERE user_running_network_configs.source = 'web'");
}
let result = sqlx::query(&query)
.bind(user_id)
.bind(device_id.to_string())
.bind(instance_id.to_string())
.bind(network_config)
.bind(source.as_str())
.bind(false)
.bind(now)
.bind(now)
.execute(&mut **transaction)
.await
.map_err(sqlx_db_error)?;
Ok(result.rows_affected() > 0)
}
#[cfg(unix)]
fn restrict_database_file_permissions(db_path: &str) -> anyhow::Result<()> {
if db_path.ends_with(":memory:") || db_path.contains("mode=memory") {
@@ -214,44 +388,205 @@ impl Db {
Ok(())
}
pub async fn insert_or_update_web_network_config(
pub(crate) async fn apply_managed_config_update(
&self,
(user_id, device_id): (UserIdInDb, Uuid),
network_inst_id: Uuid,
network_config: NetworkConfig,
) -> Result<bool, DbErr> {
let now = chrono::Local::now().fixed_offset();
let network_config =
serde_json::to_string(&network_config).map_err(|e| DbErr::Json(e.to_string()))?;
let source = ConfigSource::Web.as_str();
let result = sqlx::query(
r#"
INSERT INTO user_running_network_configs (
user_id, device_id, network_instance_id, network_config,
source, disabled, create_time, update_time
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(user_id, device_id, network_instance_id) DO UPDATE SET
network_config = excluded.network_config,
source = excluded.source,
disabled = excluded.disabled,
update_time = excluded.update_time
WHERE user_running_network_configs.source = ?
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.bind(network_inst_id.to_string())
.bind(network_config)
.bind(source)
.bind(false)
.bind(now)
.bind(now)
.bind(source)
.execute(&self.db)
.await
.map_err(|e| DbErr::Custom(e.to_string()))?;
update: ManagedConfigUpdate,
) -> Result<ManagedConfigApplyResult, DbErr> {
let (upserts, target_revision, expected_revision) = match &update {
ManagedConfigUpdate::Full {
upserts,
target_revision,
expected_revision,
} => (
upserts,
target_revision.as_deref(),
expected_revision.clone(),
),
ManagedConfigUpdate::Patch {
upserts,
target_revision,
expected_revision,
..
} => (
upserts,
Some(target_revision.as_str()),
ManagedConfigExpectedRevision::Exact(Some(expected_revision.clone())),
),
};
let serialized_upserts = upserts
.iter()
.map(|upsert| {
serde_json::to_string(&upsert.network_config)
.map(|config| (upsert.instance_id, config))
.map_err(|error| DbErr::Json(error.to_string()))
})
.collect::<Result<Vec<_>, _>>()?;
Ok(result.rows_affected() > 0)
let mut transaction = self
.db
.begin_with("BEGIN IMMEDIATE")
.await
.map_err(sqlx_db_error)?;
let current_revision =
read_managed_config_revision(&mut transaction, user_id, device_id).await?;
if target_revision.is_some() && current_revision.as_deref() == target_revision {
transaction.commit().await.map_err(sqlx_db_error)?;
return Ok(ManagedConfigApplyResult::AlreadyApplied);
}
if let ManagedConfigExpectedRevision::Exact(expected) = &expected_revision
&& current_revision.as_ref() != expected.as_ref()
{
let result = ManagedConfigApplyResult::RevisionConflict {
expected: expected.clone(),
current: current_revision,
};
transaction.commit().await.map_err(sqlx_db_error)?;
return Ok(result);
}
let mut existing_sources = HashMap::new();
match &update {
ManagedConfigUpdate::Full { .. } => {
let rows = sqlx::query_as::<_, (String, String)>(
r#"
SELECT network_instance_id, source
FROM user_running_network_configs
WHERE user_id = ? AND device_id = ?
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.fetch_all(&mut *transaction)
.await
.map_err(sqlx_db_error)?;
for (instance_id, source) in rows {
if let Ok(instance_id) = Uuid::parse_str(&instance_id) {
existing_sources.insert(instance_id, source);
}
}
}
ManagedConfigUpdate::Patch {
delete_instance_ids,
..
} => {
for instance_id in upserts
.iter()
.map(|upsert| upsert.instance_id)
.chain(delete_instance_ids.iter().copied())
{
if let Some(source) =
read_config_source(&mut transaction, user_id, device_id, instance_id)
.await?
{
existing_sources.insert(instance_id, source);
}
}
}
}
let strict_ownership = target_revision.is_some();
if strict_ownership
&& let Some(instance_id) = serialized_upserts
.iter()
.map(|(instance_id, _)| *instance_id)
.chain(match &update {
ManagedConfigUpdate::Patch {
delete_instance_ids,
..
} => delete_instance_ids.iter().copied(),
ManagedConfigUpdate::Full { .. } => [].iter().copied(),
})
.find(|instance_id| {
existing_sources
.get(instance_id)
.is_some_and(|source| source != ConfigSource::Web.as_str())
})
{
transaction.commit().await.map_err(sqlx_db_error)?;
return Ok(ManagedConfigApplyResult::OwnershipConflict { instance_id });
}
let desired_ids = serialized_upserts
.iter()
.map(|(instance_id, _)| *instance_id)
.collect::<HashSet<_>>();
for (instance_id, network_config) in &serialized_upserts {
if !strict_ownership
&& existing_sources
.get(instance_id)
.is_some_and(|source| source != ConfigSource::Web.as_str())
{
continue;
}
let updated = upsert_network_config(
&mut transaction,
user_id,
device_id,
*instance_id,
network_config,
ConfigSource::Web,
true,
)
.await?;
if !updated {
transaction.rollback().await.map_err(sqlx_db_error)?;
return Ok(ManagedConfigApplyResult::OwnershipConflict {
instance_id: *instance_id,
});
}
}
let delete_instance_ids = match &update {
ManagedConfigUpdate::Full { .. } => existing_sources
.iter()
.filter_map(|(instance_id, source)| {
(source == ConfigSource::Web.as_str() && !desired_ids.contains(instance_id))
.then_some(*instance_id)
})
.collect::<Vec<_>>(),
ManagedConfigUpdate::Patch {
delete_instance_ids,
..
} => delete_instance_ids
.iter()
.filter(|instance_id| {
existing_sources
.get(instance_id)
.is_some_and(|source| source == ConfigSource::Web.as_str())
})
.copied()
.collect(),
};
for instance_id in &delete_instance_ids {
sqlx::query(
r#"
DELETE FROM user_running_network_configs
WHERE user_id = ? AND device_id = ? AND network_instance_id = ?
AND source = 'web'
"#,
)
.bind(user_id)
.bind(device_id.to_string())
.bind(instance_id.to_string())
.execute(&mut *transaction)
.await
.map_err(sqlx_db_error)?;
}
match target_revision {
Some(revision) => {
write_managed_config_revision(&mut transaction, user_id, device_id, revision)
.await?;
}
None => {
clear_managed_config_revision(&mut transaction, user_id, device_id).await?;
}
}
transaction.commit().await.map_err(sqlx_db_error)?;
Ok(ManagedConfigApplyResult::Applied {
deleted_web_instance_ids: delete_instance_ids,
})
}
pub async fn delete_web_network_configs(
@@ -259,19 +594,32 @@ impl Db {
(user_id, device_id): (UserIdInDb, Uuid),
network_inst_ids: &[Uuid],
) -> Result<(), DbErr> {
use entity::user_running_network_configs as urnc;
urnc::Entity::delete_many()
.filter(urnc::Column::UserId.eq(user_id))
.filter(urnc::Column::DeviceId.eq(device_id.to_string()))
.filter(urnc::Column::Source.eq(ConfigSource::Web.as_str()))
.filter(
urnc::Column::NetworkInstanceId
.is_in(network_inst_ids.iter().map(|id| id.to_string())),
let mut transaction = self
.db
.begin_with("BEGIN IMMEDIATE")
.await
.map_err(sqlx_db_error)?;
let mut deleted = false;
for instance_id in network_inst_ids {
let result = sqlx::query(
r#"
DELETE FROM user_running_network_configs
WHERE user_id = ? AND device_id = ? AND network_instance_id = ?
AND source = 'web'
"#,
)
.exec(self.orm_db())
.await?;
.bind(user_id)
.bind(device_id.to_string())
.bind(instance_id.to_string())
.execute(&mut *transaction)
.await
.map_err(sqlx_db_error)?;
deleted |= result.rows_affected() > 0;
}
if deleted {
clear_managed_config_revision(&mut transaction, user_id, device_id).await?;
}
transaction.commit().await.map_err(sqlx_db_error)?;
Ok(())
}
}
@@ -285,42 +633,31 @@ impl Storage<(UserIdInDb, Uuid), user_running_network_configs::Model, DbErr> for
network_config: NetworkConfig,
source: ConfigSource,
) -> Result<(), DbErr> {
let txn = self.orm_db().begin().await?;
use entity::user_running_network_configs as urnc;
let on_conflict = OnConflict::columns([
urnc::Column::UserId,
urnc::Column::DeviceId,
urnc::Column::NetworkInstanceId,
])
.update_columns([
urnc::Column::NetworkConfig,
urnc::Column::Source,
urnc::Column::Disabled,
urnc::Column::UpdateTime,
])
.to_owned();
let insert_m = urnc::ActiveModel {
user_id: sea_orm::Set(user_id),
device_id: sea_orm::Set(device_id.to_string()),
network_instance_id: sea_orm::Set(network_inst_id.to_string()),
network_config: sea_orm::Set(
serde_json::to_string(&network_config).map_err(|e| DbErr::Json(e.to_string()))?,
),
source: sea_orm::Set(source.as_str().to_string()),
disabled: sea_orm::Set(false),
create_time: sea_orm::Set(chrono::Local::now().fixed_offset()),
update_time: sea_orm::Set(chrono::Local::now().fixed_offset()),
..Default::default()
};
urnc::Entity::insert(insert_m)
.on_conflict(on_conflict)
.do_nothing()
.exec(&txn)
.await?;
txn.commit().await
let network_config =
serde_json::to_string(&network_config).map_err(|e| DbErr::Json(e.to_string()))?;
let mut transaction = self
.db
.begin_with("BEGIN IMMEDIATE")
.await
.map_err(sqlx_db_error)?;
let previous_source =
read_config_source(&mut transaction, user_id, device_id, network_inst_id).await?;
upsert_network_config(
&mut transaction,
user_id,
device_id,
network_inst_id,
&network_config,
source,
false,
)
.await?;
if source == ConfigSource::Web
|| previous_source.as_deref() == Some(ConfigSource::Web.as_str())
{
clear_managed_config_revision(&mut transaction, user_id, device_id).await?;
}
transaction.commit().await.map_err(sqlx_db_error)
}
async fn delete_network_configs(
@@ -328,18 +665,35 @@ impl Storage<(UserIdInDb, Uuid), user_running_network_configs::Model, DbErr> for
(user_id, device_id): (UserIdInDb, Uuid),
network_inst_ids: &[Uuid],
) -> Result<(), DbErr> {
use entity::user_running_network_configs as urnc;
urnc::Entity::delete_many()
.filter(urnc::Column::UserId.eq(user_id))
.filter(urnc::Column::DeviceId.eq(device_id.to_string()))
.filter(
urnc::Column::NetworkInstanceId
.is_in(network_inst_ids.iter().map(|id| id.to_string())),
let mut transaction = self
.db
.begin_with("BEGIN IMMEDIATE")
.await
.map_err(sqlx_db_error)?;
let mut deleted_web_config = false;
for instance_id in network_inst_ids {
deleted_web_config |=
read_config_source(&mut transaction, user_id, device_id, *instance_id)
.await?
.as_deref()
== Some(ConfigSource::Web.as_str());
sqlx::query(
r#"
DELETE FROM user_running_network_configs
WHERE user_id = ? AND device_id = ? AND network_instance_id = ?
"#,
)
.exec(self.orm_db())
.await?;
.bind(user_id)
.bind(device_id.to_string())
.bind(instance_id.to_string())
.execute(&mut *transaction)
.await
.map_err(sqlx_db_error)?;
}
if deleted_web_config {
clear_managed_config_revision(&mut transaction, user_id, device_id).await?;
}
transaction.commit().await.map_err(sqlx_db_error)?;
Ok(())
}
@@ -349,20 +703,32 @@ impl Storage<(UserIdInDb, Uuid), user_running_network_configs::Model, DbErr> for
network_inst_id: Uuid,
disabled: bool,
) -> Result<(), DbErr> {
use entity::user_running_network_configs as urnc;
urnc::Entity::update_many()
.filter(urnc::Column::UserId.eq(user_id))
.filter(urnc::Column::DeviceId.eq(device_id.to_string()))
.filter(urnc::Column::NetworkInstanceId.eq(network_inst_id.to_string()))
.col_expr(urnc::Column::Disabled, Expr::value(disabled))
.col_expr(
urnc::Column::UpdateTime,
Expr::value(chrono::Local::now().fixed_offset()),
)
.exec(self.orm_db())
.await?;
let mut transaction = self
.db
.begin_with("BEGIN IMMEDIATE")
.await
.map_err(sqlx_db_error)?;
let source =
read_config_source(&mut transaction, user_id, device_id, network_inst_id).await?;
let result = sqlx::query(
r#"
UPDATE user_running_network_configs
SET disabled = ?, update_time = ?
WHERE user_id = ? AND device_id = ? AND network_instance_id = ?
"#,
)
.bind(disabled)
.bind(chrono::Local::now().fixed_offset())
.bind(user_id)
.bind(device_id.to_string())
.bind(network_inst_id.to_string())
.execute(&mut *transaction)
.await
.map_err(sqlx_db_error)?;
if result.rows_affected() > 0 && source.as_deref() == Some(ConfigSource::Web.as_str()) {
clear_managed_config_revision(&mut transaction, user_id, device_id).await?;
}
transaction.commit().await.map_err(sqlx_db_error)?;
Ok(())
}
@@ -600,17 +966,73 @@ mod tests {
}
#[tokio::test]
async fn test_web_network_config_does_not_replace_user_owned_config() {
async fn web_owned_mutations_invalidate_managed_revision() {
let db = Db::memory_db().await;
let user_id = db.auto_create_user("user-web-race").await.unwrap().id;
let user_id = db
.auto_create_user("managed-revision-invalidation")
.await
.unwrap()
.id;
let device_id = uuid::Uuid::new_v4();
let inst_id = uuid::Uuid::new_v4();
db.insert_or_update_user_network_config(
(user_id, device_id),
inst_id,
NetworkConfig {
network_name: Some("managed".to_string()),
..Default::default()
},
ConfigSource::Web,
)
.await
.unwrap();
db.set_managed_config_revision((user_id, device_id), "rev-before-disable")
.await
.unwrap();
db.update_network_config_state((user_id, device_id), inst_id, true)
.await
.unwrap();
assert!(
db.get_managed_config_revision((user_id, device_id))
.await
.unwrap()
.is_none()
);
db.set_managed_config_revision((user_id, device_id), "rev-before-delete")
.await
.unwrap();
db.delete_network_configs((user_id, device_id), &[inst_id])
.await
.unwrap();
assert!(
db.get_managed_config_revision((user_id, device_id))
.await
.unwrap()
.is_none()
);
}
#[tokio::test]
async fn user_owned_mutation_preserves_managed_revision() {
let db = Db::memory_db().await;
let user_id = db
.auto_create_user("user-revision-preserved")
.await
.unwrap()
.id;
let device_id = uuid::Uuid::new_v4();
let inst_id = uuid::Uuid::new_v4();
db.set_managed_config_revision((user_id, device_id), "rev-user")
.await
.unwrap();
db.insert_or_update_user_network_config(
(user_id, device_id),
inst_id,
NetworkConfig {
network_name: Some("user-owned".to_string()),
network_name: Some("user".to_string()),
..Default::default()
},
ConfigSource::User,
@@ -618,26 +1040,12 @@ mod tests {
.await
.unwrap();
let updated = db
.insert_or_update_web_network_config(
(user_id, device_id),
inst_id,
NetworkConfig {
network_name: Some("web-owned".to_string()),
..Default::default()
},
)
.await
.unwrap();
assert!(!updated);
let saved = db
.get_network_config((user_id, device_id), &inst_id.to_string())
.await
.unwrap()
.unwrap();
assert_eq!(saved.get_network_config_source(), ConfigSource::User);
let saved_config = saved.get_network_config().unwrap();
assert_eq!(saved_config.network_name.as_deref(), Some("user-owned"));
assert_eq!(
db.get_managed_config_revision((user_id, device_id))
.await
.unwrap()
.as_deref(),
Some("rev-user")
);
}
}
easytier-web/src/restful/mod.rs
+6
View File
@@ -86,6 +86,10 @@ struct ParseConfigResponse {
#[derive(Debug, serde::Deserialize, serde::Serialize)]
pub struct Error {
message: String,
#[serde(skip_serializing_if = "Option::is_none")]
code: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
current_config_revision: Option<String>,
}
type RpcError = rpc_types::error::Error;
type HttpHandleError = (StatusCode, Json<Error>);
@@ -93,6 +97,8 @@ type HttpHandleError = (StatusCode, Json<Error>);
pub fn other_error<T: ToString>(error_message: T) -> Error {
Error {
message: error_message.to_string(),
code: None,
current_config_revision: None,
}
}
easytier-web/src/restful/network.rs
+177 -37
View File
@@ -1,6 +1,6 @@
use axum::extract::Path;
use axum::extract::{DefaultBodyLimit, Path};
use axum::http::StatusCode;
use axum::routing::{delete, post};
use axum::routing::{delete, post, put};
use axum::{Json, Router, extract::State, routing::get};
use axum_login::AuthUser;
use easytier::common::config::{
@@ -21,6 +21,8 @@ use super::{
AppState, AppStateInner, Error, HttpHandleError, RpcError, convert_db_error, other_error,
};
const MAX_MANAGED_CONFIG_REQUEST_BODY_SIZE: usize = 32 * 1024 * 1024;
fn convert_rpc_error(e: RpcError) -> (StatusCode, Json<Error>) {
let status_code = match &e {
RpcError::ExecutionError(_) => StatusCode::BAD_REQUEST,
@@ -29,6 +31,8 @@ fn convert_rpc_error(e: RpcError) -> (StatusCode, Json<Error>) {
};
let error = Error {
message: format!("{:?}", e),
code: None,
current_config_revision: None,
};
(status_code, Json(error))
}
@@ -98,6 +102,14 @@ struct ReconcileManagedNetworkConfigsJsonReq {
expected_config_revision: Option<String>,
}
#[derive(Debug, serde::Deserialize, serde::Serialize)]
struct PatchManagedNetworkConfigsJsonReq {
upserts: Vec<ManagedNetworkConfigJson>,
delete_instance_ids: Vec<uuid::Uuid>,
config_revision: String,
expected_config_revision: String,
}
#[derive(Debug, serde::Deserialize, serde::Serialize)]
struct ListMachineItem {
client_url: Option<url::Url>,
@@ -113,6 +125,37 @@ struct ListMachineJsonResp {
pub struct NetworkApi;
impl NetworkApi {
fn convert_managed_config_error(error: anyhow::Error) -> HttpHandleError {
let (status, code, current_config_revision) =
match error.downcast_ref::<crate::client_manager::ManagedConfigError>() {
Some(crate::client_manager::ManagedConfigError::Invalid(_)) => {
(StatusCode::BAD_REQUEST, None, None)
}
Some(crate::client_manager::ManagedConfigError::RevisionConflict {
current,
..
}) => (
StatusCode::CONFLICT,
Some("managed_config_revision_conflict".to_string()),
current.clone(),
),
Some(crate::client_manager::ManagedConfigError::OwnershipConflict { .. }) => (
StatusCode::CONFLICT,
Some("managed_config_ownership_conflict".to_string()),
None,
),
None => (StatusCode::INTERNAL_SERVER_ERROR, None, None),
};
(
status,
Json(Error {
message: error.to_string(),
code,
current_config_revision,
}),
)
}
fn get_user_id(auth_session: &AuthSession) -> Result<UserIdInDb, (StatusCode, Json<Error>)> {
let Some(user_id) = auth_session.user.as_ref().map(|x| x.id()) else {
return Err((
@@ -145,15 +188,22 @@ impl NetworkApi {
Path(machine_id): Path<uuid::Uuid>,
Json(payload): Json<RunNetworkJsonReq>,
) -> Result<Json<Void>, HttpHandleError> {
let user_id = Self::get_user_id(&auth_session)?;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
let result = client_mgr
.handle_run_network_instance_with_source(
(Self::get_user_id(&auth_session)?, machine_id),
(user_id, machine_id),
payload.config,
payload.save,
RuntimeConfigSource::Web,
)
.await
.map_err(convert_error)?;
.await;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
result.map_err(convert_error)?;
Ok(Void::default().into())
}
@@ -205,13 +255,18 @@ impl NetworkApi {
State(client_mgr): AppState,
Path((machine_id, inst_id)): Path<(uuid::Uuid, uuid::Uuid)>,
) -> Result<(), HttpHandleError> {
let user_id = Self::get_user_id(&auth_session)?;
client_mgr
.handle_remove_network_instances(
(Self::get_user_id(&auth_session)?, machine_id),
vec![inst_id],
)
.await
.map_err(convert_error)
.invalidate_applied_config_revision(user_id, machine_id)
.await;
let result = client_mgr
.handle_remove_network_instances((user_id, machine_id), vec![inst_id])
.await;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
result.map_err(convert_error)?;
Ok(())
}
async fn handle_list_machines(
@@ -251,14 +306,18 @@ impl NetworkApi {
));
};
let user_id = Self::get_user_id(&auth_session)?;
client_mgr
.handle_update_network_state(
(auth_session.user.unwrap().id(), machine_id),
inst_id,
payload.disabled,
)
.await
.map_err(convert_error)
.invalidate_applied_config_revision(user_id, machine_id)
.await;
let result = client_mgr
.handle_update_network_state((user_id, machine_id), inst_id, payload.disabled)
.await;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
result.map_err(convert_error)?;
Ok(())
}
async fn handle_get_network_metas(
@@ -290,15 +349,23 @@ impl NetworkApi {
other_error("Instance ID mismatch".to_string()).into(),
));
}
let user_id = Self::get_user_id(&auth_session)?;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
let result = client_mgr
.handle_save_network_config_with_source(
(Self::get_user_id(&auth_session)?, machine_id),
(user_id, machine_id),
inst_id,
payload.config,
RuntimeConfigSource::Web,
)
.await
.map_err(convert_error)
.await;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
result.map_err(convert_error)?;
Ok(())
}
async fn handle_get_network_config(
@@ -325,14 +392,20 @@ impl NetworkApi {
.and_then(config_source_from_rpc)
.unwrap_or(RuntimeConfigSource::Web);
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
let result = client_mgr
.handle_run_network_instance_with_source(
(user_id, machine_id),
payload.config,
payload.save,
source,
)
.await
.map_err(convert_error)?;
.await;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
result.map_err(convert_error)?;
Ok(Void::default().into())
}
@@ -341,16 +414,23 @@ impl NetworkApi {
Path((user_id, machine_id, inst_id)): Path<(UserIdInDb, uuid::Uuid, uuid::Uuid)>,
) -> Result<(), HttpHandleError> {
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
let result = client_mgr
.handle_remove_network_instances((user_id, machine_id), vec![inst_id])
.await
.map_err(convert_error)
.await;
client_mgr
.invalidate_applied_config_revision(user_id, machine_id)
.await;
result.map_err(convert_error)?;
Ok(())
}
async fn handle_reconcile_managed_network_configs_internal(
State(client_mgr): AppState,
Path((user_id, machine_id)): Path<(UserIdInDb, uuid::Uuid)>,
Json(payload): Json<ReconcileManagedNetworkConfigsJsonReq>,
) -> Result<Json<Void>, HttpHandleError> {
) -> Result<StatusCode, HttpHandleError> {
let desired = payload
.managed_network_configs
.into_iter()
@@ -368,15 +448,35 @@ impl NetworkApi {
payload.expected_config_revision,
)
.await
.map_err(|err| {
let status = if crate::client_manager::is_managed_config_revision_conflict(&err) {
StatusCode::CONFLICT
} else {
StatusCode::INTERNAL_SERVER_ERROR
};
(status, other_error(err.to_string()).into())
})?;
Ok(Void::default().into())
.map_err(Self::convert_managed_config_error)?;
Ok(StatusCode::NO_CONTENT)
}
async fn handle_patch_managed_network_configs_internal(
State(client_mgr): AppState,
Path((user_id, machine_id)): Path<(UserIdInDb, uuid::Uuid)>,
Json(payload): Json<PatchManagedNetworkConfigsJsonReq>,
) -> Result<StatusCode, HttpHandleError> {
let upserts = payload
.upserts
.into_iter()
.map(|item| crate::webhook::ManagedNetworkConfig {
instance_id: item.instance_id.to_string(),
network_config: item.network_config,
})
.collect();
client_mgr
.patch_managed_network_configs(
user_id,
machine_id,
upserts,
payload.delete_instance_ids,
payload.config_revision,
payload.expected_config_revision,
)
.await
.map_err(Self::convert_managed_config_error)?;
Ok(StatusCode::NO_CONTENT)
}
async fn handle_list_network_instance_ids_internal(
@@ -406,8 +506,10 @@ impl NetworkApi {
Router::new()
.route(
"/api/internal/users/:user-id/machines/:machine-id/networks",
post(Self::handle_run_network_instance_internal)
.put(Self::handle_reconcile_managed_network_configs_internal)
put(Self::handle_reconcile_managed_network_configs_internal)
.patch(Self::handle_patch_managed_network_configs_internal)
.layer(DefaultBodyLimit::max(MAX_MANAGED_CONFIG_REQUEST_BODY_SIZE))
.post(Self::handle_run_network_instance_internal)
.get(Self::handle_list_network_instance_ids_internal),
)
.route(
@@ -453,3 +555,41 @@ impl NetworkApi {
)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn revision_conflict_response_exposes_machine_readable_current_revision() {
let error = crate::client_manager::ManagedConfigError::RevisionConflict {
expected: Some("rev-1".to_string()),
current: Some("rev-2".to_string()),
};
let (status, Json(body)) = NetworkApi::convert_managed_config_error(error.into());
assert_eq!(status, StatusCode::CONFLICT);
assert_eq!(
body.code.as_deref(),
Some("managed_config_revision_conflict")
);
assert_eq!(body.current_config_revision.as_deref(), Some("rev-2"));
}
#[test]
fn ownership_conflict_is_distinct_from_revision_conflict() {
let error = crate::client_manager::ManagedConfigError::OwnershipConflict {
instance_id: uuid::Uuid::new_v4(),
};
let (status, Json(body)) = NetworkApi::convert_managed_config_error(error.into());
assert_eq!(status, StatusCode::CONFLICT);
assert_eq!(
body.code.as_deref(),
Some("managed_config_ownership_conflict")
);
assert_eq!(body.current_config_revision, None);
}
}
easytier-web/src/restful/rpc.rs
+86
View File
@@ -39,6 +39,13 @@ async fn handle_proxy_rpc_by_session(
scope,
} = req;
let mutates_runtime_config = proxy_rpc_mutates_runtime_config(&service_name, &method_name);
if mutates_runtime_config {
session
.invalidate_runtime_config_for_direct_mutation()
.await;
}
let resp = match service_name.as_str() {
"api.manage.WebClientService" => match_service!(
easytier::proto::api::manage::WebClientServiceClientFactory<BaseController>,
@@ -134,6 +141,12 @@ async fn handle_proxy_rpc_by_session(
}
};
if mutates_runtime_config {
session
.invalidate_runtime_config_for_direct_mutation()
.await;
}
match resp {
Ok(v) => Ok(Json(v)),
Err(e) => Err((
@@ -143,6 +156,32 @@ async fn handle_proxy_rpc_by_session(
}
}
fn proxy_rpc_mutates_runtime_config(service_name: &str, method_name: &str) -> bool {
matches!(
(service_name, method_name),
(
"api.manage.WebClientService",
"run_network_instance"
| "RunNetworkInstance"
| "retain_network_instance"
| "RetainNetworkInstance"
| "delete_network_instance"
| "DeleteNetworkInstance"
) | (
"api.config.ConfigRpcService",
"patch_config" | "PatchConfig"
) | (
"api.instance.CredentialManageRpcService",
"generate_credential"
| "GenerateCredential"
| "revoke_credential"
| "RevokeCredential"
| "upsert_credential"
| "UpsertCredential"
)
)
}
pub async fn handle_proxy_rpc(
auth_session: super::users::AuthSession,
State(client_mgr): AppState,
@@ -192,3 +231,50 @@ pub fn router_internal() -> Router<super::AppStateInner> {
post(handle_proxy_rpc_internal),
)
}
#[cfg(test)]
mod tests {
use super::proxy_rpc_mutates_runtime_config;
#[test]
fn runtime_config_mutation_detection_covers_proxy_rpc_aliases() {
for (service, method) in [
("api.manage.WebClientService", "run_network_instance"),
("api.manage.WebClientService", "RetainNetworkInstance"),
("api.manage.WebClientService", "delete_network_instance"),
("api.config.ConfigRpcService", "PatchConfig"),
(
"api.instance.CredentialManageRpcService",
"generate_credential",
),
(
"api.instance.CredentialManageRpcService",
"RevokeCredential",
),
(
"api.instance.CredentialManageRpcService",
"upsert_credential",
),
] {
assert!(
proxy_rpc_mutates_runtime_config(service, method),
"{service}/{method} must invalidate the managed revision fence"
);
}
for (service, method) in [
("api.manage.WebClientService", "list_network_instance"),
("api.config.ConfigRpcService", "get_config"),
(
"api.instance.CredentialManageRpcService",
"list_credentials",
),
("api.instance.StatsRpcService", "get_stats"),
] {
assert!(
!proxy_rpc_mutates_runtime_config(service, method),
"{service}/{method} must remain read-only"
);
}
}
}