fix(deploy): preserve legacy environment and diagnose prepared plans safely
The old initializer defaults HN_ENVIRONMENT to development, while migration validation required the literal production and combined database/platform prerequisites into one opaque error. Preserve supported environment identity namespaces without editing the live .env; still require PostgreSQL and fixed HTTPS/WSS endpoints for every existing-server migration. Match supported boolean spellings and report failing field names without credential values. Add read-only diagnose for an existing prepared plan so an earlier failed prepare can be reviewed without deleting or regenerating its keyring. Move future prerequisite validation before private directory/key generation. Validation: 253 passed, 3 skipped (real PostgreSQL unavailable), 30 existing warnings; 40 deployment-tool tests including 11 new compatibility/redaction cases. Python 3.7 syntax and staged diff checks pass. Actual ECS diagnosis must still confirm the current failing field; no live config or key changed.
This commit is contained in:
@@ -178,3 +178,13 @@ Windows / Python 3.12.10,平台桩、临时证书/密钥、SQLite及带Lua的F
|
||||
修订Compose命令统一显式指定`--profile longconn`,不启用未知profile;保留服务集合严格校验并在不符时仅输出missing/extra服务名。后续up仍限connector/router且带--no-deps。新增3项回归模拟被profile隐藏的服务及缺失/多出服务的安全诊断,本地工具测试29项通过;现机实际解析结果仍需更新后重跑确认,不将该推断当作已经现场复现的唯一原因。
|
||||
|
||||
修订后完整本地回归:**242 passed,3 skipped,30 warnings**(48.50秒)。跳过与warnings类型同上,宿主脚本Python3.7语法及diff检查通过。
|
||||
|
||||
### 准备阶段配置诊断兼容修订
|
||||
|
||||
现场prepare在platform/database配置检查处停止,未切换容器;原流程在最后验证前已生成私有plan/keyring,不得删除重建来绕过。核对旧初始化器默认environment=development,首版工具错误地要求字面production,可能误拒绝现有测试部署;同时数据库/开关条件合并导致现场无法辨别具体字段。
|
||||
|
||||
新版按现有Settings接受production/development/test,保持原身份命名空间和原.env,所有部署环境仍强制PostgreSQL、固定HTTPS平台/公开入口和WSS,不降低网络校验。布尔开关对齐Settings接受true/1/yes/on。错误只列字段,不回显数据库URL等敏感值;新增diagnose只读已有plan,无需重生成keyring。未来prepare在生成私有目录前先验证必要条件。
|
||||
|
||||
新增11项工具回归,覆盖合法环境保持、HTTPS/WSS/数据库失败在写密钥前拒绝、只读diagnose保留全部准备文件、错误不泄露数据库凭据;工具专项40项通过。实际现场哪项原始条件不符仍以更新后的diagnose输出为准,不将历史默认值当作当前配置证明。
|
||||
|
||||
修订后完整本地回归:**253 passed,3 skipped,30 warnings**(49.46秒);真实PostgreSQL仍未提供,warnings类型同上,Python3.7语法与diff检查通过。
|
||||
|
||||
@@ -53,6 +53,17 @@ prepare生成独立的root私有目录、随机AES密钥keyring、独立router
|
||||
|
||||
## 4. 检查与离线构建:仍不切换
|
||||
|
||||
如果旧版prepare返回`broker production platform/database configuration incomplete`,不要删除目录或重跑prepare。旧初始化器曾使用`HN_ENVIRONMENT=development`,首版工具把production写死,且把多个条件合并成同一错误。新版保留经过校验的原environment命名空间(production/development/test),所有模式仍要求PostgreSQL及HTTPS/WSS;不会为通过检查修改运行.env或将凭据换一个身份空间。
|
||||
|
||||
先更新暂存仓库,再对已经生成的计划执行只读诊断:
|
||||
|
||||
```bash
|
||||
sudo python3 deployable/scripts/broker_deploy.py diagnose \
|
||||
--plan /home/cnooc/hainuo-broker-prepared-20261104
|
||||
```
|
||||
|
||||
只显示environment合法枚举、数据库类型及开关状态,不显示URL、账号密码、Secret或Token。通过后原地继续check,沿用原keyring和配置;若仍有错误按字段处理,不关闭HTTPS校验、不改用SQLite。新prepare在生成密钥前检查这些条件;旧目录因原检查失败留下的密钥不会被自动覆盖。
|
||||
|
||||
```bash
|
||||
cd /home/cnooc/ecs-relay-service-staging
|
||||
sudo python3 deployable/scripts/broker_deploy.py check \
|
||||
|
||||
@@ -15,6 +15,7 @@ import secrets
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
|
||||
class DeployError(RuntimeError):
|
||||
@@ -93,6 +94,37 @@ def profiles_from(path):
|
||||
raise DeployError("invalid product permission profiles; use the product-approved JSON object") from None
|
||||
|
||||
|
||||
def enabled(value):
|
||||
return str(value).strip().lower() in {"true","1","yes","on"}
|
||||
|
||||
|
||||
def platform_problems(env):
|
||||
problems = []
|
||||
if not str(env.get("HN_DATABASE_URL", "")).startswith(("postgresql://","postgres://")):
|
||||
problems.append("HN_DATABASE_URL: PostgreSQL connection required (value withheld)")
|
||||
if env.get("HN_ENVIRONMENT") not in {"production","development","test"}:
|
||||
problems.append("HN_ENVIRONMENT: expected production/development/test; preserve the reviewed environment")
|
||||
for name in ("HN_FEISHU_ENABLED","HN_OAUTH_ENABLED"):
|
||||
if not enabled(env.get(name)):
|
||||
problems.append(name+": must be enabled")
|
||||
for name,scheme in (("HN_PUBLIC_URL","https"),("HN_ROUTER_URL","wss"),
|
||||
("HN_FEISHU_API_BASE","https"),("HN_FEISHU_AUTHORIZE_URL","https")):
|
||||
try:
|
||||
value = urlsplit(str(env.get(name,"")))
|
||||
valid = value.scheme == scheme and value.hostname and not value.username and not value.password
|
||||
except ValueError:
|
||||
valid = False
|
||||
if not valid:
|
||||
problems.append(name+": fixed "+scheme.upper()+" endpoint required (value withheld)")
|
||||
return problems
|
||||
|
||||
|
||||
def require_platform(env):
|
||||
problems = platform_problems(env)
|
||||
if problems:
|
||||
raise DeployError("broker platform configuration incomplete; "+"; ".join(problems))
|
||||
|
||||
|
||||
def validate_overlay(base, merged, candidate):
|
||||
old, new = base["services"], merged["services"]
|
||||
require_services(old)
|
||||
@@ -128,11 +160,10 @@ def validate_overlay(base, merged, candidate):
|
||||
or env.get("HN_CREDENTIAL_KEYRING_FILE") != "/run/hainuo-credentials/keyring.json"
|
||||
or new["router"]["environment"].get("HN_CONNECTOR_INTERNAL_URL") != "http://connector:8000"):
|
||||
raise DeployError("application/environment or credential routing mismatch")
|
||||
if (not env.get("HN_DATABASE_URL", "").startswith(("postgresql://","postgres://"))
|
||||
or env.get("HN_ENVIRONMENT") != "production"
|
||||
or str(env.get("HN_FEISHU_ENABLED", "")).lower() != "true"
|
||||
or str(env.get("HN_OAUTH_ENABLED", "")).lower() != "true"):
|
||||
raise DeployError("broker production platform/database configuration incomplete")
|
||||
# Old installer initializes HN_ENVIRONMENT=development. Preserve this identity
|
||||
# namespace rather than silently converting it to production. All modes still
|
||||
# require PostgreSQL and HTTPS/WSS for this existing-server migration.
|
||||
require_platform(env)
|
||||
for key in ("HN_ADMIN_SECRET","HN_INTERNAL_SECRET","HN_FEISHU_APP_SECRET"):
|
||||
if env["HN_ROUTER_ADMISSION_SECRET"] == env.get(key):
|
||||
raise DeployError("router admission credential must be independent")
|
||||
@@ -217,6 +248,7 @@ def prepare(args):
|
||||
base = run(compose(plan)+["config","--format","json"],True)
|
||||
if base["services"]["connector"].get("environment",{}).get("HN_CREDENTIAL_MODE","legacy") != "legacy":
|
||||
raise DeployError("already broker: preserve the existing keyring and migration plan")
|
||||
require_platform({**base["services"]["connector"].get("environment",{}),"HN_OAUTH_ENABLED":"true"})
|
||||
output.mkdir(mode=0o700,parents=False)
|
||||
os.chmod(str(output),0o700)
|
||||
keyring = {"active_key_id":"initial-v1","keys":{"initial-v1":base64.b64encode(os.urandom(32)).decode("ascii")}}
|
||||
@@ -269,6 +301,22 @@ def load_plan(root):
|
||||
return plan
|
||||
|
||||
|
||||
def diagnose(root,plan):
|
||||
config = run(compose(plan,root/"compose.broker.json")+["config","--format","json"],True)
|
||||
env = config["services"]["connector"].get("environment",{})
|
||||
value = env.get("HN_ENVIRONMENT")
|
||||
print("HN_ENVIRONMENT: "+(value if value in {"production","development","test"} else "missing or invalid"))
|
||||
print("HN_DATABASE_URL: "+("PostgreSQL configured (value withheld)" if str(env.get("HN_DATABASE_URL","")).startswith(("postgresql://","postgres://")) else "missing or unsupported (value withheld)"))
|
||||
for name in ("HN_FEISHU_ENABLED","HN_OAUTH_ENABLED"):
|
||||
print(name+": "+("enabled" if enabled(env.get(name)) else "disabled or missing"))
|
||||
problems = platform_problems(env)
|
||||
for problem in problems:
|
||||
print("CHECK: "+problem)
|
||||
print("Read-only diagnosis; existing preparation and keys preserved; no container restart.")
|
||||
if problems:
|
||||
raise DeployError("review the named fields above; do not paste configuration or regenerate keys")
|
||||
|
||||
|
||||
def check(root, plan, baseline=True):
|
||||
if os.name != "nt":
|
||||
if root.stat().st_mode & 0o077:
|
||||
@@ -367,7 +415,7 @@ def main():
|
||||
prep.add_argument("--profiles",required=True)
|
||||
prep.add_argument("--output",required=True)
|
||||
prep.add_argument("--connector-container",default="hainuo-relay-connector-1")
|
||||
for name in ("check","build","activate","rollback"):
|
||||
for name in ("diagnose","check","build","activate","rollback"):
|
||||
cmd = sub.add_parser(name)
|
||||
cmd.add_argument("--plan",required=True)
|
||||
if name == "activate":
|
||||
@@ -384,7 +432,9 @@ def main():
|
||||
return
|
||||
root = Path(args.plan).resolve()
|
||||
plan = load_plan(root)
|
||||
if args.action == "check":
|
||||
if args.action == "diagnose":
|
||||
diagnose(root,plan)
|
||||
elif args.action == "check":
|
||||
check(root,plan)
|
||||
elif args.action == "build":
|
||||
build(root,plan)
|
||||
|
||||
@@ -33,6 +33,8 @@ def scenario(tmp_path,monkeypatch):
|
||||
env = {"HN_ENVIRONMENT":"production","HN_DATABASE_URL":"postgresql://fake@postgres/fake",
|
||||
"HN_FEISHU_ENABLED":"true","HN_OAUTH_ENABLED":"true","HN_FEISHU_APP_ID":"cli_test",
|
||||
"HN_FEISHU_APP_SECRET":"synthetic-app-secret","HN_ADMIN_SECRET":"a"*40,"HN_INTERNAL_SECRET":"b"*40}
|
||||
env.update(HN_PUBLIC_URL="https://relay.example.test",HN_ROUTER_URL="wss://relay.example.test/device/ws",
|
||||
HN_FEISHU_API_BASE="https://work.example.test",HN_FEISHU_AUTHORIZE_URL="https://work.example.test/open-apis/authen/v1/index")
|
||||
base = {"services":{name:{"image":"old:tag","environment":{},"networks":{"backend":None},"volumes":[]}
|
||||
for name in [*roles,"postgres","redis"]},"volumes":{"metadata":{}},"networks":{"backend":{}}}
|
||||
base["services"]["connector"]["environment"] = env
|
||||
@@ -152,6 +154,50 @@ def test_prepare_refuses_existing_output_and_preserves_key(scenario):
|
||||
assert (scenario["output"]/"credential-keyring.json").read_bytes() == key
|
||||
|
||||
|
||||
@pytest.mark.parametrize("environment", ["development","test","production"])
|
||||
def test_preserves_existing_environment_namespace_with_tls(scenario,environment):
|
||||
scenario["base"]["services"]["connector"]["environment"]["HN_ENVIRONMENT"] = environment
|
||||
plan = prepared(scenario)
|
||||
deploy.check(scenario["output"],plan)
|
||||
merged = deploy.run(deploy.compose(plan,scenario["output"]/"compose.broker.json")+["config"],True)
|
||||
assert merged["services"]["connector"]["environment"]["HN_ENVIRONMENT"] == environment
|
||||
assert merged["services"]["router"]["environment"]["HN_ENVIRONMENT"] == environment
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,value", [("HN_DATABASE_URL","sqlite://synthetic-secret"),
|
||||
("HN_ENVIRONMENT","invalid-private-value"),("HN_FEISHU_ENABLED","false"),
|
||||
("HN_PUBLIC_URL","http://synthetic-secret"),("HN_ROUTER_URL","ws://synthetic-secret"),
|
||||
("HN_FEISHU_API_BASE","http://synthetic-secret")])
|
||||
def test_prerequisite_failure_reports_field_before_generating_keys(scenario,name,value):
|
||||
scenario["base"]["services"]["connector"]["environment"][name] = value
|
||||
with pytest.raises(deploy.DeployError) as error:
|
||||
deploy.prepare(scenario["args"])
|
||||
assert name in str(error.value) and "synthetic-secret" not in str(error.value)
|
||||
assert "invalid-private-value" not in str(error.value)
|
||||
assert not scenario["output"].exists()
|
||||
|
||||
|
||||
def test_diagnose_preserves_previously_generated_key_and_never_restarts(scenario,capsys):
|
||||
plan = prepared(scenario)
|
||||
before = {p.name:p.read_bytes() for p in scenario["output"].iterdir() if p.is_file()}
|
||||
scenario["base"]["services"]["connector"]["environment"]["HN_ENVIRONMENT"] = "development"
|
||||
scenario["calls"].clear()
|
||||
deploy.diagnose(scenario["output"],plan)
|
||||
assert "development" in capsys.readouterr().out
|
||||
assert before == {p.name:p.read_bytes() for p in scenario["output"].iterdir() if p.is_file()}
|
||||
assert not any("up" in c or "tag" in c or "build" in c for c in scenario["calls"])
|
||||
|
||||
|
||||
def test_diagnose_never_echoes_database_credentials(scenario,capsys):
|
||||
plan = prepared(scenario)
|
||||
env = scenario["base"]["services"]["connector"]["environment"]
|
||||
env["HN_DATABASE_URL"] = "unsupported://synthetic-db-secret@host"
|
||||
with pytest.raises(deploy.DeployError):
|
||||
deploy.diagnose(scenario["output"],plan)
|
||||
output = capsys.readouterr().out
|
||||
assert "HN_DATABASE_URL" in output and "synthetic-db-secret" not in output
|
||||
|
||||
|
||||
def test_prepare_refuses_live_subdirectory(scenario):
|
||||
scenario["args"].output = str(scenario["live"]/"broker")
|
||||
with pytest.raises(deploy.DeployError,match="outside"):
|
||||
|
||||
Reference in New Issue
Block a user