diff --git a/deployable/APPROVAL_ACCEPTANCE_LOCAL_20261117.md b/deployable/APPROVAL_ACCEPTANCE_LOCAL_20261117.md new file mode 100644 index 0000000..4bee753 --- /dev/null +++ b/deployable/APPROVAL_ACCEPTANCE_LOCAL_20261117.md @@ -0,0 +1,60 @@ +# 审批闭环与异常恢复:本地联调记录 + +日期:2026-11-17。范围:客户端feat/phase-A-local-20261116与ECS feat/approval-foundation-local-20261117,均为本地未提交改动。不代表master或已安装产品具备此能力。 + +## 结果 + +最终针对性测试:162项通过,1项跳过。ECS/两仓联合100项通过、1项跳过;客户端62项通过。联合测试自身25项通过,其余是认证、任务交接、卡片及客户端既有行为回归。 + +跳过项:test_postgres_authorization_and_binding_generation,原因是没有提供真实测试PostgreSQL地址。本轮审批账本使用临时SQLite;不声称PostgreSQL审批并发已通过。 + +## 已核对行为 + +| 场景 | 实测结果 | +| --- | --- | +| 手机决定的合成回调允许 | 业务MockTransport收到一次请求,审批消费为consumed | +| 拒绝 | 未发送业务请求,同操作再次提交不生成第二次执行 | +| 重复按钮、重复操作、并发消费 | 仅一次许可可消费,业务请求无重复 | +| 消费响应丢失、轮询断线 | 客户端未发送业务请求;重建客户端仍读原操作记录,不重新执行 | +| 业务响应丢失 | 原请求计数为1,返回unknown;重建客户端不重放 | +| 本机停止、绑定变化、服务端执行终态、过期 | 不发送业务请求 | +| 假操作者/应用/租户/Verification Token/时效/会话/卡片/nonce/自报身份 | 未形成批准,原审批保持pending | +| 缺设备签名、签名重放、错任务 | 请求被拒绝,不发送业务请求 | +| 卡片创建响应不明 | 不重发第二张确认卡 | +| 过期卡片更新响应不明 | 只重试更新已知原卡片,删除按钮后不再更新;不新发卡 | +| 审批worker及数据库连接重建 | 查询原审批/卡片继续处理,不重新发卡或替换操作ID | +| 已核定只读 | 不生成审批卡片,沿正常读取流程 | +| 默认关闭 | 没有审批/内部回调路由,不创建审批表,协议不宣称启用 | +| 原生run状态投影 | 仅当前任务运行态投影waiting_for_approval;completed不回退 | + +## 方法与边界 + +使用真实客户端BrokerWorkClient、设备Ed25519签名、ECS ASGI路由、临时数据库与专用审批worker。Redis使用fakeredis,平台卡片及业务API使用合成服务,不连接正式Work,不创建真实日程、不发邮件。 + +客户端新增可注入的http_transport仅供构造测试;网络和模型不能覆盖传输目标。测试中的原生请求补齐loopback来源/Host,模拟Agent补停止回调,没有放宽正式安全检查。 + +测试曾因固定轮数的轮询窗口过早读取发送中间态失败,改为单调时钟的实际期限后完整重跑通过。正式“回执不明不重发”规则未修改。 + +未覆盖:正式长连接SDK传递真人点击、平台三秒响应上限的现场表现、手机卡片呈现、完整ECS进程/主机重启、正式PostgreSQL和所有工具统一审批。这里的worker重建测试不等于服务器重启验收。 + +## 复现与证据 + +ECS deployable目录使用现有测试Python执行: + +```powershell +python -m pytest tests/test_approval_integration.py tests/test_task_delivery.py tests/test_credential_api.py tests/test_task_cards.py -q --junitxml=build-support/approval-integration-20261117.xml +``` + +客户端仓库根目录: + +```powershell +python -m pytest integrations/work-relay/tests/test_broker_client.py integrations/work-relay/tests/test_task_journal.py integrations/work-relay/tests/test_task_executor_context.py integrations/work-relay/tests/test_broker_task_context.py -q --junitxml=build-support/approval-client-regression-20261117.xml +``` + +若成对客户端源码不在同级默认目录,通过HAINUO_APPROVAL_PC_SOURCE指定其仓库根目录。缺源码时联合模块明确skip,不能用跳过代替通过。 + +两端XML保留具体测试名、结果与耗时;git diff --check通过。本轮没有提交、推送、部署、出包或修改已安装客户端。两端审批开关仍默认关闭。 + +## 下一步 + +准备独立测试ECS和客户端版本,复用唯一机器人长连接,核对应用回调订阅后做真人允许/拒绝和断线测试。部署需覆盖原longconn及connector,不能照旧只升级connector/router,也不能增加第二个机器人连接。此步骤需要测试环境和平台配置配合;本轮未擅自执行部署。 diff --git a/deployable/APPROVAL_LOCAL_20261117.md b/deployable/APPROVAL_LOCAL_20261117.md new file mode 100644 index 0000000..171926c --- /dev/null +++ b/deployable/APPROVAL_LOCAL_20261117.md @@ -0,0 +1,41 @@ +# C2远程确认本地接入记录 + +基线d2881b8,分支feat/approval-foundation-local-20261117。此前按用户要求仅开发源码;用户随后明确要求“审批闭环与异常恢复联调”,本轮已开始本地合成联调。不修改线上订阅、不申请权限、不发真实卡片、不部署或推送。条件路由注册与执行闸门开关默认关闭;本地通过不能代替平台真人验收。 + +approval_store.py为独立审批账本基础:绑定本人、任务、设备、代次与动作摘要;动作、显示文案与随机按钮nonce采用现有keyring加密。审批有效期30—600秒且不超过任务期限;相同操作不能换动作;仅pending允许决定,approved只能消费一次;拒绝、过期、取消及已消费不再允许执行。已消费回执丢失不自动再执行,后续仍需PC操作账本保护。 + +VerifiedDecision仅由approval_callback.py在核对可信SDK内部通道、应用、租户、Verification Token、时效、真实操作者/会话/卡片字段后构造。必须继续经账本核对原任务与nonce,不接受按钮自报身份。 + +用户提供的内网文档已读取:[更新消息流卡片按钮](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/group/im-v2/chat_button/update),PUT /open-apis/im/v2/chat_button,使用tenant_access_token,权限im:app_feed_card:write,10次/秒,最多2个按钮,action_type支持webhook及url_page,action_map为字符串字典。这是消息流卡片快捷操作接口,不能直接替代当前消息回复卡片的交互协议,也未为本功能申请这一权限。 + +已从同一平台目录继续核对以下协议: + +- [卡片回传交互回调](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukzMukzMukzM/feishu-cards/card-callback-communication):schema 2.0、card.action.trigger;header包含app_id、tenant_key、Verification Token、event_id和微秒create_time;真实操作者来自event.operator.open_id,卡片/会话来自event.context.open_message_id/open_chat_id。event.token仅为更新卡片凭证,不用作身份认证。 +- [长连接接收回调](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/event-subscription-guide/callback-subscription/step-1-choose-a-subscription-mode/send-callbacks-to-developers-server):官方Python SDK register_p2_card_action_trigger,返回P2CardActionTriggerResponse;响应必须在3秒内完成。复用现有唯一SDK连接,不增加第二个长连接消费者。 +- [接收并处理回调](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/event-subscription-guide/callback-subscription/receive-and-handle-callbacks):签名为SHA256(timestamp + nonce + Encrypt Key的UTF-8字节 + 原始body);先验签再用SHA256(Encrypt Key)作为AES-256-CBC密钥解密,密文前16字节为IV,PKCS7去填充。 + +approval_callback.py提供严格边界:限制64KB、拒绝重复JSON键、应用/租户/Verification Token校验、事件5分钟时效、仅im_message/button。按钮只携带审批引用、随机nonce和选择,不携带本人或设备身份。当前路由只接现有SDK长连接的内部密钥通道;签名加密Webhook验证方法保留为独立实现,没有接入公开Webhook路由或URL challenge。 + +审批事件摘要与event_id在同一决定事务中记账;相同回调返回原状态,事件ID换动作/换审批拒绝。确认文案由服务端从实际动作生成,客户端不能用自拟“低风险”文案改变内容。卡片使用plain_text;发送前持久化意图,超时/取消/回执丢失后不自动重发;实际回复卡片ID必须绑定成功才能接受点击。已知卡片可重试PATCH去掉过期按钮,不用新卡代替未知发送结果。 + +approval_worker.py使用专用单线程及该线程独享的数据库连接,不共享connector主连接事务。工作槽最多8个,排队50ms、调用等待1.5秒,工作开始前核对排队期限;SQLite busy_timeout 500ms,PostgreSQL lock_timeout 500ms/statement_timeout 800ms。超时不声称事务已回滚,也不批准客户端重试执行。SDK回调使用最多2个独立内部请求worker,等待2.3秒后返回待核对toast;卡片发送/更新不放在回调等待链上。3秒平台时限尚未实测,数据库/网络故障下的预算仍需验收。 + +条件路由为POST /work/v2/approvals/create、status、consume、cancel;复用现有设备证书签名、请求正文摘要、90秒proof期限、Redis nonce去重与绑定认证。每次操作同时核对任务ID/本人/设备/代次/动作摘要;任务已终止、失联未知、解绑或超期时不准消费。内部回调为POST /internal/approval-callback,仅接所选longconn来源及内部密钥。创建与回调各自限制body大小。 + +客户端对应分支feat/phase-A-local-20261116已在BrokerWorkClient.api的真实平台HTTP发送前接入approval_transport.py。已核定只读沿用读取;需要确认的请求先冻结参数,等待本人决定,消费前登记本地意图;只有收到本次消费成功且动作逐字节相同,任务未停止、绑定未变时才继续原请求。原写操作账本仍保留,重复操作不会因重新审批而重放。同步执行入口不能安全等待、接口404、断网、无停止控制或回执不明均停止,不回退为免确认发送。此闸门覆盖该Work API边界,不等于终端、execute_code等所有工具已完成统一风险接入;OpenCLI仍禁用。 + +等待状态通过可信任务作用域在原生run查询中投影为waiting_for_approval,沿原进度协议传至卡片;不覆盖终态。取消/拒绝/过期/已消费卡片删除按钮,consumed仅说明许可已交PC,不冒充业务成功。维护任务只补尚未登记发送意图的卡片、重试已知原卡片更新,每轮各5项;进程内最多32个去重卡片作业。审批审计保留期清理尚未实现。 + +配置约束:ECS的HN_WORK_REMOTE_APPROVALS_ENABLED默认false;启用需broker模式、现有longconn、明确的HN_FEISHU_TENANT_KEY和HN_FEISHU_VERIFICATION_TOKEN。客户端可信release.json中remote_approvals默认false。没有修改现有release/Compose/安装实例。后续部署若启用,需要同步更新原唯一longconn容器的源码和配置,不能只更新connector/router,也不能额外启动第二个机器人连接。应用回调订阅仍需在独立验收时核对;本轮没有操作应用后台。 + +此前检查仅为AST/diff;本轮增加tests/test_approval_integration.py,成对加载客户端源码,使用真实Ed25519设备签名、ASGI认证路由、Redis合成服务及临时SQLite数据库连接审批worker;平台发卡和业务HTTP均由合成服务替代。测试不是正式Work长连接、真实手机按钮或线上数据库验收。 + +已覆盖:允许/拒绝、重复按钮与重复请求、并发决定/消费、未经签名及签名重放、跨任务、假操作者/应用/租户/token/时效/会话/卡片/nonce、停止/本机绑定变化/执行终态/过期、审批消费回执丢失/轮询断线/业务回执丢失后的客户端重建、确认卡创建回执不明不重发、已知过期卡片去按钮更新回执不明可重试、已核定只读免确认、默认关闭无审批路由。客户端既有模拟请求补全真实入口要求的loopback Host/remote,模拟Agent补停止回调,没有放宽正式入口。 + +测试XML:deployable/build-support/approval-integration-20261117.xml。成对源码可通过HAINUO_APPROVAL_PC_SOURCE指定客户端仓库根目录;缺少时明确skip,不能将缺少跨仓执行当通过。独立PostgreSQL用例因未提供测试地址仍跳过;完整服务重启、SDK三秒响应、手机UI显示及真实平台回调需后续隔离环境验收。不能据此标记C201—C204全部完成或可上线。 + +最终本轮ECS/联合回归100项通过、1项跳过,客户端62项通过(合计162通过、1跳过);详细范围与复现见APPROVAL_ACCEPTANCE_LOCAL_20261117.md。没有提交、推送、部署或出包。 + +后续M6本地增量:用户暂停手机现场测试后,明确继续M6开发。本轮补C207最小审计、事务失败不放行、默认关闭的保留/清理策略及独立PostgreSQL竞争用例。组合回归192通过2跳过;清理游标最终调整专项另核。详见M6_AUDIT_RETENTION_20261117.md。本轮代码未部署,不能将新审计能力写成现网已启用。 + +后续独立安全复核与真实PG验证:复现并修复同任务换操作ID重复申请、确认目标不明确/被截断两类问题。隔离本机PostgreSQL 17.7下执行4/16连接竞争、决定冲突、真实SQL审计失败回滚及不同ID同意图竞争,临时实例已停止。最终组合205项通过0跳过,取代本轮“PG缺地址”的未验证状态,但不代表ECS现场或手机验收。详见M6_SECURITY_REVIEW_POSTGRES_20261117.md;修复仍是本地未提交源码,未部署。 diff --git a/deployable/APPROVAL_TEST_DEPLOY_20261117.md b/deployable/APPROVAL_TEST_DEPLOY_20261117.md new file mode 100644 index 0000000..5a0bbcf --- /dev/null +++ b/deployable/APPROVAL_TEST_DEPLOY_20261117.md @@ -0,0 +1,77 @@ +# 新ECS审批候选部署 + +目标经用户2026-11-17贴回的docker inspect核对为ecs-fd74,当前镜像hainuo-relay:broker-update-6c8b7ecfc8976fcc。父计划:/home/cnooc/hainuo-broker-work-20261116。Compose基线:/home/cnooc/ecs-relay-source/deployable/compose.yaml和该父计划compose.broker.json。不要使用旧ECS的20261112计划。 + +候选来自feat/approval-foundation-local-20261117,未经合并;执行时应使用交付命令中固定的提交。客户端保持原状,之后单独准备配套测试代码。测试包不包含本机密钥或服务端身份文件。 + +## 执行前必须具备 + +1. 应用事件与回调中支持并配置长连接接收card.action.trigger。复用当前唯一longconn,不能新开第二个连接。 +2. Verification Token从当前connector本地配置复用,确实缺失才在ECS私密输入。tenant_key不再要求用户填写:脚本在现有connector中以只读数据库连接读取当前应用/环境、未撤销绑定的有效授权,调用既有user_info接口并核对open_id,再提取tenant_key。不会刷新/修改授权、不新增scope、不读取业务正文;令牌不离开容器。没有可核实的有效授权时停止并提示检查Work授权,不允许猜填。 +3. 当前客户端具备broker v2协议。本轮先升级ECS,手机审批实际请求需要后续客户端开启remote_approvals;该客户端开关仍默认关闭,服务端升级不等于真人审批已可用。 + +## 准备与切换 + +从当前源码仓库fetch候选后,在新的独立worktree执行,不覆盖/home/cnooc/ecs-relay-source。候选worktree:/home/cnooc/ecs-relay-approvals-20261117;新计划:/home/cnooc/hainuo-broker-approvals-20261117。 + +在候选worktree中: + +```bash +sudo python3 deployable/scripts/approval_identity.py \ + --output /home/cnooc/hainuo-approval-callback-20261117.json + +sudo python3 deployable/scripts/broker_deploy.py prepare-update \ + --plan /home/cnooc/hainuo-broker-work-20261116 \ + --output /home/cnooc/hainuo-broker-approvals-20261117 \ + --enable-remote-approvals \ + --callback-identity /home/cnooc/hainuo-approval-callback-20261117.json + +sudo python3 deployable/scripts/broker_deploy.py build \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 + +sudo python3 deployable/scripts/broker_deploy.py activate \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 \ + --client-v2-ready + +sudo python3 deployable/scripts/broker_deploy.py state \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 +``` + +输出目录或私有身份文件已经存在时停止,不覆盖或删除原计划来重来。构建失败后保留现场,未激活前服务不变。切换失败可能部分生效,先运行state;不要自动重复activate。 + +本次仅重建connector/router/原longconn。数据库、edge、signer、绑定、工作材料及既有凭据keyring保留。connector/router复用当前connector候选运行时;longconn使用自己的原镜像作基础,分别构建并保存摘要。两种镜像都不在线下载依赖,候选预检不接网络;任一SDK/密钥/配置预检失败即停止。 + +## 状态核对 + +connector/router应为candidate、running、healthy;longconn应为candidate、running,platform_connected observed应为yes。longconn无healthcheck时health=none不是独立通过证据。启动租约最多会等待旧租约失效,稍后再次state只读检查,不启动其他消费者。启动信号仅导出固定类别,不直接贴含令牌的原始日志。 + +随后进行配套客户端和手机允许/拒绝/超时/重复点击/断线的合成动作验收,不能用真实发送邮件作为第一项。 + +## 回滚 + +停止正在进行的审批测试/任务后,在同一候选worktree运行: + +```bash +sudo python3 deployable/scripts/broker_deploy.py rollback \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 \ + --stop-broker-use + +sudo python3 deployable/scripts/broker_deploy.py state \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 +``` + +--stop-broker-use是操作者已经停止测试的声明,脚本不替你停止电脑。恢复三服务原镜像和父配置,不回滚数据库、不覆盖refresh token、不重新生成身份密钥。新审批表保留,回滚不承诺撤销已发生业务操作;配套客户端须同时关闭remote_approvals。不要运行父计划的rollback,那会回到更早基线。 + +## 保密与边界 + +身份JSON、plan.json、compose.broker.json含私有信息,只保存在ECS受限目录,勿发聊天或提交Git。tenant_key由脚本自动核实;Verification Token确实缺失时才由本人在平台本地核对,不需要把账号密码交给助手。真实PostgreSQL/SDK回调/手机UI尚待本次环境验收,不能凭本地测试数宣称上线。 + +2026-11-17现场修正:c80c6a0的旧准备脚本曾要求用户输入tenant_key,用户未知该运维字段而中止,未生成候选/切换服务。修复后移除tenant输入;12项自动发现测试通过,覆盖只读加密SQL、有效授权用户匹配、过期/403/重定向/缺字段拒绝、应用/环境匹配,以及不输出身份正文或令牌。私有化user_info文档已确认tenant_key字段:[获取用户信息](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/reference/authen-v1/user_info/get)。测试不代表现场已成功取得tenant_key。 + +现场3132c07自动发现返回通用失败,尚无证据判定是授权过期或平台网络。新增--diagnose只读入口,不要求--output、不写身份文件、不询问Verification Token、不重启容器。失败仅输出TENANT_DIAGNOSTIC的固定stage、计数、HTTP/平台数字错误码及白名单异常类;不输出数据库地址、账号、token、异常正文或平台用户信息。15项针对性测试通过。 + +```bash +sudo python3 deployable/scripts/approval_identity.py --diagnose +``` + +阶段为container_execution说明容器内探测未完成;credentials_query/database_connect指数据库读取阶段;matched_bindings为0表示没有匹配当前应用/环境的未撤销绑定授权;expired_access表示已有access过期但没有尝试refresh;platform_rejected/http_status/platform_code表示平台拒绝;missing_tenant/user_mismatch分别是返回缺字段/用户身份不匹配。只有拿到诊断事实后才决定修复,不因通用失败要求用户解绑、重建身份或猜填租户。 diff --git a/deployable/M6_AUDIT_RETENTION_20261117.md b/deployable/M6_AUDIT_RETENTION_20261117.md new file mode 100644 index 0000000..aa85ab1 --- /dev/null +++ b/deployable/M6_AUDIT_RETENTION_20261117.md @@ -0,0 +1,43 @@ +# M6审批审计与保留开发记录 + +日期:2026-11-17。范围:C207及C201至C204的本地回归。手机现场验收继续暂缓,线上服务仍为之前部署的03d34f1,本次没有部署、推送、出包或修改本机已安装Work组件。 + +## 本轮实现 + +增加work_approval_audit_v1,记录创建、允许、拒绝、许可消费、过期、取消、卡片绑定、发送结果不明及内容清理。审计写入与对应状态修改使用同一数据库事务;写审计失败即撤回该事务,不下发许可。重复回调/重复消费不会生成第二次决定或消费事件。 + +审计字段仅含环境/应用、审批引用、任务/本人/设备的作用域摘要、绑定代次、动作摘要、固定动作类别、事件、状态及记录时间。不保存动作请求正文、卡片文案、账号姓名、原始用户/设备ID、访问/刷新令牌或按钮nonce。摘要是关联标识,不宣称不可重识别。审计中的consumed仅代表执行许可已消费,不代表业务已成功;业务结果仍由原任务和操作记录核对。 + +没有新增公开的审计查询、任意SQL或管理员执行接口;无效回调在原校验边界拒绝,本轮未增加未经验证操作者的身份审计。 + +## 保留与清理候选 + +| 配置 | 候选默认 | 行为 | +| --- | --- | --- | +| HN_APPROVAL_RETENTION_ENABLED | false | 未评审前不启用后台清理 | +| HN_APPROVAL_PAYLOAD_RETENTION_DAYS | 30 | 已关闭审批的加密动作/文案/nonce及对应密钥引用到期后清空 | +| HN_APPROVAL_AUDIT_RETENTION_DAYS | 90 | 已清理审批的过期审计事件有限删除 | + +30/90天只是研发候选值,应按团队正式留存要求评审,不作为公司合规期限。要求1 <= 内容保留天数 <= 审计保留天数 <= 365;启用清理还要求审批功能已开启。现有部署配置未被修改,部署脚本也未新增自动开启清理的参数。 + +只选择当前应用/环境内rejected、expired、cancelled、consumed审批;pending/approved不清。consumed但原任务仍inflight、unknown或已查不到任务时不清,保留恢复核对材料。使用游标分批扫描,避免待核对任务挡住其他可清项;每批最多20项,有本轮耗时预算,真正变更前再持有本人锁重读状态。 + +不删除审批/操作防重放记录,不改状态回pending,不删除callback发送意图/回执,不碰用户原稿、附件或任务目录。清理完成后同操作仍返回原关闭状态,不能重新取得执行许可。清理是数据库字段更新,不承诺底层磁盘备份或历史数据库页物理抹除。 + +## 测试与证据 + +组合针对性回归192项通过、2项跳过;包括审批闭环、设备认证、卡片、交付、部署和身份发现。跳过的2项均需独立PostgreSQL测试地址:原授权/绑定测试,以及本轮新增的审批并发测试。本轮未连接生产数据库或真实平台业务写接口。 + +新增审计/清理用例覆盖:字段最小化、重复决定去重、清理后重放拒绝、活跃审批不清、环境隔离、审计失败消费回滚、配置限制以及业务结果未知的consumed保留。最终游标修订后另跑该专项,结果见build-support/m6-retention-final-20261117.xml;组合记录为build-support/m6-audit-retention-20261117.xml,两批不能重复累计为功能完成数。 + +tests/test_approval_postgres.py已准备独立随机schema的真实数据库竞争测试:同审批并发允许/拒绝、四连接竞争消费、重开连接后再次消费及审计决定唯一。仅显式设置HN_TEST_DATABASE_URL才运行,凭据不放命令行/交付说明;只建和清理本次随机测试schema,不使用现网配置。没有测试地址,不能将跳过写作通过。 + +## M6仍待完成 + +1. 独立安全复核:回调来源、操作者/卡片/nonce、任务与代次,以及“模型改ID”对重复请求的影响。 +2. 独立PostgreSQL执行新旧竞争用例,测试锁等待/连接重建和事务失败,不只验证SQLite。 +3. 真实手机允许/拒绝、超时/重复点击、客户端停止/失联后的旧卡片,按用户恢复验收的时间安排。 +4. 应用SDK三秒回调时限及正式部署回滚;桌面主dashboard候选版本另行核对。 +5. 审计留存参数评审、受控运维查询与释放策略。审计/清理代码完成不代表C207已验收或整个M6已完成。 + +后续更新:用户明确要求安全复核与PostgreSQL验证后,已在独立本机PostgreSQL 17.7补跑原跳过用例,并新增对抗修复;最终205通过0跳过,临时实例已停止。测试与剩余边界以M6_SECURITY_REVIEW_POSTGRES_20261117.md为准。这里记录的192通过2跳过是前一批历史结果,不再作为最新结果。 diff --git a/deployable/M6_FIX_TEST_DEPLOY_20261117.md b/deployable/M6_FIX_TEST_DEPLOY_20261117.md new file mode 100644 index 0000000..2b08933 --- /dev/null +++ b/deployable/M6_FIX_TEST_DEPLOY_20261117.md @@ -0,0 +1,60 @@ +# M6修复测试环境升级 + +对象:ecs-fd74。父计划/home/cnooc/hainuo-broker-approvals-20261117已由用户确认三服务candidate、connector/router healthy及longconn platform_connected=yes;最后已部署源码为03d34f1。新计划使用/home/cnooc/hainuo-broker-approvals-m6-fix-20261117,不能改用更早的work计划。 + +## 修复内容与证据 + +- 同任务同动作意图不能通过换operation_id及自动日历幂等键重新申请;真正不同动作可另行待确认。 +- 确认卡显示具体资源路径,超长目标拒绝,不静默截断。 +- 申请/决定/消费/关闭等事件增加最小审计,和状态变更处于同一事务。旧审批的历史事件不伪造补录。 +- 保留与清理逻辑已实现,但本次候选显式HN_APPROVAL_RETENTION_ENABLED=false,不删除既有审批内容或记录。手机现场测试继续暂缓。 + +205项核心组合回归通过,包含本机独立PostgreSQL 17.7的4/16连接竞争、SQL审计报错回滚和不同ID同意图竞争,0跳过。升级脚本修改后71项部署回归通过,包含从已启用审批的父计划再次升级及回滚;两批存在重叠,不能合算为276个不同功能验收。来源记录见M6_SECURITY_REVIEW_POSTGRES_20261117.md。 + +2026-11-17准备时本机到10.77.81.160的443请求超时,不能据此判定ECS服务故障;部署由用户在现有ECS shell执行,脚本会先核对实际镜像和父计划。此文档不表示升级已发生。 + +## 新候选准备与切换 + +先按交付命令fetch测试分支并用固定提交创建独立worktree:/home/cnooc/ecs-relay-approvals-m6-fix-20261117。下面步骤在该worktree内执行: + +```bash +sudo python3 deployable/scripts/broker_deploy.py state \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 + +sudo python3 deployable/scripts/broker_deploy.py prepare-update \ + --plan /home/cnooc/hainuo-broker-approvals-20261117 \ + --output /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 \ + --enable-remote-approvals \ + --callback-identity /home/cnooc/hainuo-approval-callback-20261117.json + +sudo python3 deployable/scripts/broker_deploy.py build \ + --plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 + +sudo python3 deployable/scripts/broker_deploy.py activate \ + --plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 \ + --client-v2-ready + +sudo python3 deployable/scripts/broker_deploy.py state \ + --plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 +``` + +复用现有私有callback identity文件,不需再次输入/复制令牌。新目录若已存在停止,不自动删除、覆盖或重建。prepare/build不切换服务,activate仅定向重建原connector/router/longconn。各角色分别复用当前运行镜像作基础,不在线拉依赖,不启动第二个机器人连接。 + +保留数据库、材料、绑定和keyring;准备目录包含私有配置,禁止贴回或提交Git。启动会新增审计/保留元数据表,但清理开关为false,不做旧数据清理。首次启动时旧审批后续事件可能标为unknown类别,不据此编造历史created事件。 + +## 核对及回滚 + +输出应包含connector/router candidate running healthy、longconn candidate running及platform_connected observed=yes。SDK连接信号延迟时再次只读state,不重复activate。失败可能部分切换,先state查看,不自动删除数据或回退数据库。 + +回滚前停止进行中的审批任务,然后在本次新worktree执行: + +```bash +sudo python3 deployable/scripts/broker_deploy.py rollback \ + --plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 \ + --stop-broker-use + +sudo python3 deployable/scripts/broker_deploy.py state \ + --plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 +``` + +仅恢复父计划三服务镜像和配置,不回退绑定、OAuth刷新数据或数据库,不移除新增表。此回滚应保留父计划已有审批功能,不回到最早未启用审批的环境。--stop-broker-use是操作者已停止任务的声明,脚本不会代停电脑或撤销已发生业务动作。 diff --git a/deployable/M6_SECURITY_REVIEW_POSTGRES_20261117.md b/deployable/M6_SECURITY_REVIEW_POSTGRES_20261117.md new file mode 100644 index 0000000..210e1c5 --- /dev/null +++ b/deployable/M6_SECURITY_REVIEW_POSTGRES_20261117.md @@ -0,0 +1,74 @@ +# M6安全复核与PostgreSQL并发验证 + +日期:2026-11-17。对象:feat/approval-foundation-local-20261117工作树中未提交的M6审计与审批增量,基于03d34f1。此处“独立复核”指从功能流程之外按对抗场景重新检查,不是另一位团队验收人的签字或第三方测评。 + +## 结论 + +复现并修复两类问题,三项初始负向检查均从失败转为通过。最终组合回归205项通过,0失败、0跳过。数据库相关用例在真实PostgreSQL 17.7执行,未再以缺测试地址跳过。 + +修改仅在本地源码,未提交/推送、未部署到ECS、未改变已安装客户端或清理开关。手机现场验收继续暂缓。当前已部署03d34f1不包含本轮修复,不能将本报告当作现网升级回执。 + +## 复核发现及修复 + +| 编号 | 初始可复现行为 | 影响与修复 | 回归结果 | +| --- | --- | --- | --- | +| R1 | 原审批拒绝后,换operation_id并随之更换客户端自动生成的日历idempotency_key,同一动作可再次申请成功并生成新审批。 | 没有证明能不经批准直接执行,但可绕开原拒绝反复发卡。新增同一可信任务内的动作意图校验,覆盖等待、拒绝、过期及已消费的原申请;更换操作ID不能抹掉原决定。 | pending、rejected、consumed下换ID均拒绝;同ID换参数拒绝;真正不同的新动作仍可待确认;PostgreSQL两连接同动作不同ID竞争只建一条。 | +| R2 | 确认卡只显示路径最后一段,两个不同日历都显示events;长资源标识静默截断。 | 用户不能判断真正目标。改为显示完整资源相对路径,保留日历/文件/成员等对象标识。超过300字符的目标直接拒绝,不能截断隐藏差异。 | 两个不同日历文案可区分;过长目标拒绝。 | + +R1保留完整operation_digest用于许可消费时逐字节核对。意图规范化仅去除已经核实为客户端按owner和operation_id生成的日历幂等键;不任意忽略业务参数。身份来源、请求正文、nonce、真正操作者及卡片校验均保留。 + +相同动作只能在同一个任务内复用原审批,不允许模型换ID重新发卡;用户明确发起的新任务是新的授权范围。每任务最多50个不同审批意图,超过预算停止。旧加密内容不可校验时拒绝新申请,而不是猜测放行。 + +## 复核边界 + +- 平台来源:只接所选长连接的内部认证通道;核对应用、租户、Verification Token、事件时效及回调类型。不把action.value中的身份或event.token更新凭据当来源证明。 +- 决定归属:操作者、私聊、原卡片、随机nonce、本人设备、绑定代次、原task及动作摘要都要匹配。错误应用/租户/用户/卡片、旧任务和旧绑定不能决定。 +- 单次许可:决定、消费和审计在数据库事务中执行,按本人锁串行化;已关闭审批不能复活。重投只返回已有事实,未知结果不重新执行。 +- 内容与日志:审计不保存正文或token;动作类别固定。清理默认关闭,不删除活跃或结果未知任务所需内容,保留防重放记录;清理不构成物理擦除承诺。 +- 恢复:停止控制取实际Agent,许可返回后再核对停止/绑定。许可consumed不宣称业务成功,网络中断不承诺撤销已有外部请求。 + +## 真实PostgreSQL验证 + +从官方EDB HTTPS地址获取便携PostgreSQL 17.7,解包到D盘测试目录,不安装Windows服务。每轮创建独立数据目录、随机密码、本机随机端口,仅监听127.0.0.1。测试数据库为合成专用hainuo_m6_test,测试表位于随机schema;没有使用任何生产DSN、应用密钥或真实业务记录。 + +数据库并发用例用Barrier同时放行独立连接,不是单连接顺序调用: + +1. 两连接竞争允许/拒绝,只有一个最终决定;覆盖自然竞争、预先允许及预先拒绝三种情况。 +2. 分别4和16连接竞争同一许可;允许时恰好一个execute=true,拒绝时全部false。 +3. 重开数据库连接后再次消费仍为false,原决定的审计事件保持唯一。 +4. 用PostgreSQL真实触发器让consumed审计INSERT报错,确认前面的状态UPDATE随同事务回滚,之后移除合成故障再按正常规则竞争。 +5. 两连接同时以不同操作ID申请同一动作,只建立一个审批,另一个拒绝。 +6. 原有PostgreSQL OAuth/绑定代次回归真实执行,不再跳过。 + +以上为6个参数化审批用例及1个授权/绑定用例;其他198项包含合成平台接口、SQLite应用链路、安全负向、卡片、部署与身份发现回归,不把205项都声称为PostgreSQL端到端业务实测。 + +初轮针对性验证12项通过。完整回归曾出现1个旧清理测试失败:它在清空原动作后,仍在同任务中申请新动作,与“无法校验旧意图则拒绝”的新保护冲突。调整为清理前创建活跃审批来验证不误清,未放宽正式校验。最终完整重跑205项全部通过。 + +## 证据和复现 + +最终XML:build-support/m6-independent-review-pg-final-20261117.xml。包含205项结果及耗时,未通过数与跳过数均为0。前序针对性XML:build-support/m6-postgres-20261117.xml。成对客户端源码仍通过HAINUO_APPROVAL_PC_SOURCE定位,缺少时不能宣称联合通过。 + +本地临时数据库的启动/运行/停止由工作区outputs/install-1087b-20261117/run_isolated_pg_tests.py完成;运行回执pg-test-receipt.json不含密码。下载来源与ZIP SHA256在D盘postgres-runtime/runtime-receipt.json,临时数据目录保留供复查,明文临时密码文件已移除。 + +部署仓库deployable目录的复现选择器: + +```text +tests/test_approval_postgres.py +tests/test_approval_security_review.py +tests/test_approval_audit.py +tests/test_approval_integration.py +tests/test_task_delivery.py +tests/test_task_cards.py +tests/test_credential_api.py +tests/test_broker_deploy.py +tests/test_approval_identity.py +``` + +PostgreSQL测试只读取明确指定的HN_TEST_DATABASE_URL。测试schema名称由代码固定前缀加随机十六进制组成,清理仅作用于该schema,不读取现网配置。最终运行后pg_ctl已停止该独立测试实例。 + +## 尚不能据此放行的事项 + +- 真实手机点击、私有化SDK三秒回调时限,以及ECS ARM64与现场PostgreSQL版本的整链路行为仍未验收;Windows本地PostgreSQL通过不代替这些事实。 +- 真实服务器在提交中途退出、平台业务回执真实丢失、数据库网络分区等仍需要独立环境故障注入;当前证明是账本事务、并发和合成失联行为。 +- 当前审批执行闸门覆盖Work API边界,不是任意execute_code/终端工具的沙箱。M2/M8仍需完成所有工具统一风险规则与外部材料指令隔离,不能宣称整个客户端已安全放行。 +- 团队仍需另一位具备权限的工程师复核补丁并验收;本轮没有合并master或部署修复。M6整体保持待验收。 diff --git a/deployable/relay/approval_api.py b/deployable/relay/approval_api.py new file mode 100644 index 0000000..a107538 --- /dev/null +++ b/deployable/relay/approval_api.py @@ -0,0 +1,106 @@ +"""Opt-in device-authenticated approval API, with no business execution route.""" +import asyncio +from fastapi import APIRouter,HTTPException,Request +from fastapi.responses import JSONResponse +from pydantic import Field + +from .approval_callback import ApprovalCallbackVerifier,decision_toast +from .approval_cards import ApprovalCardSender +from .approval_policy import describe_action +from .approval_store import ApprovalError +from .common import no_cache,require_secret +from .task_delivery import TaskRequest + + +class CreateApproval(TaskRequest): + operation_id:str=Field(pattern=r'^[A-Za-z0-9_-]{8,100}$') + action:dict=Field(repr=False) + ttl:int=Field(default=120,strict=True,ge=30,le=600) + + +class ApprovalReference(TaskRequest): + approval_id:str=Field(pattern=r'^apr_[a-f0-9]{64}$') + operation_digest:str=Field(pattern=r'^[a-f0-9]{64}$') + + +class ApprovalAPI: + def __init__(self,auth,worker,platform,spawn): + self.auth,self.worker,self.spawn=auth,worker,spawn + self.sender=ApprovalCardSender(worker,platform) + self.scheduled=set() + s=auth.s + self.verifier=ApprovalCallbackVerifier(environment=s.environment,app_id=s.feishu_app_id, + tenant_key=s.feishu_tenant_key,verification_token=s.feishu_verification_token, + internal_secret=s.internal_secret,encrypt_key=s.feishu_encrypt_key) + self.router=APIRouter() + for name in ('create','status','consume','cancel'): + self.router.add_api_route('/work/v2/approvals/'+name,getattr(self,name),methods=['POST']) + self.router.add_api_route('/internal/approval-callback',self.callback,methods=['POST']) + + async def _call(self,method,*args): + try:return await self.worker.call(method,*args) + except ApprovalError: + raise HTTPException(409,'approval_unavailable_or_unconfirmed')from None + + def _schedule(self,kind,approval_id): + key=(kind,approval_id) + if key in self.scheduled or len(self.scheduled)>=32:return + self.scheduled.add(key) + async def deliver(): + try: + if kind=='send':await self.sender.send(approval_id) + else:await self.sender.update_closed(approval_id) + finally:self.scheduled.discard(key) + self.spawn(deliver()) + + async def create(self,body:CreateApproval,request:Request): + await self.auth.authenticate(body,request) + try:summary,impact=describe_action(body.action) + except (ApprovalError,TypeError,ValueError,RecursionError): + raise HTTPException(400,'unsupported_approval_action')from None + result=await self._call('create',body.task_id,body.binding_owner,body.host_id, + body.binding_generation,body.operation_id,body.action,summary,impact,body.ttl) + self._schedule('send',result['approval_id']) + return no_cache(JSONResponse(result)) + + async def _reference(self,method,body,request): + await self.auth.authenticate(body,request) + args=(body.approval_id,body.task_id,body.binding_owner,body.host_id,body.binding_generation) + # Cancel requires the same request digest too, not just visibility of an ID. + if method=='cancel': + await self._call('status',*args,body.operation_digest) + result=await self._call('cancel',*args) + else:result=await self._call(method,*args,body.operation_digest) + if method in {'consume','cancel'}:self._schedule('close',body.approval_id) + return no_cache(JSONResponse(result)) + + async def status(self,body:ApprovalReference,request:Request): + return await self._reference('status',body,request) + + async def consume(self,body:ApprovalReference,request:Request): + return await self._reference('consume',body,request) + + async def cancel(self,body:ApprovalReference,request:Request): + return await self._reference('cancel',body,request) + + async def callback(self,request:Request): + if self.auth.s.feishu_event_source!='longconn':raise HTTPException(404,'not_found') + supplied=request.headers.get('X-Internal-Secret','') + require_secret(supplied,self.auth.s.internal_secret) + try: + raw=await asyncio.wait_for(request.body(),timeout=.2) + decision=self.verifier.from_sdk_channel(raw,supplied) + result=await self.worker.call('decide',decision) + except (ApprovalError,asyncio.TimeoutError): + return no_cache(JSONResponse({'toast':{'type':'warning', + 'content':'确认未完成或结果待核对,请查看原任务状态,勿重复发起操作。'}})) + self._schedule('close',decision.approval_id) + return no_cache(JSONResponse(decision_toast(result))) + + async def maintain(self): + await self.worker.call('close_unavailable') + for approval_id in await self.worker.call('cards_to_send'):self._schedule('send',approval_id) + for approval_id in await self.worker.call('cards_to_close'):self._schedule('close',approval_id) + if self.auth.s.approval_retention_enabled: + await self.worker.call('maintain_retention',self.auth.s.approval_payload_retention_days, + self.auth.s.approval_audit_retention_days) diff --git a/deployable/relay/approval_audit.py b/deployable/relay/approval_audit.py new file mode 100644 index 0000000..9bbcd38 --- /dev/null +++ b/deployable/relay/approval_audit.py @@ -0,0 +1,40 @@ +"""Minimal approval audit; no action bodies, tokens, names or card text.""" +import hashlib +from .crypto import canonical + +EVENTS={'created','approved','rejected','consumed','expired','cancelled','card_bound', + 'card_send_unknown','payload_purged'} +STATES={'pending','approved','rejected','consumed','expired','cancelled'} +ACTION_CLASSES={'work_api_read','work_api_write','work_api_delete','unknown'} + + +class ApprovalAudit: + def __init__(self,db,clock): + self.db,self.clock=db,clock + with db: + db.execute('''CREATE TABLE IF NOT EXISTS work_approval_audit_v1( + event_key TEXT PRIMARY KEY,environment TEXT NOT NULL,app_id TEXT NOT NULL, + approval_id TEXT NOT NULL,task_scope TEXT NOT NULL,actor_scope TEXT NOT NULL, + device_scope TEXT NOT NULL,binding_generation BIGINT NOT NULL, + operation_digest TEXT NOT NULL,event TEXT NOT NULL,state TEXT NOT NULL,action_class TEXT NOT NULL, + created_at DOUBLE PRECISION NOT NULL)''') + db.execute('''CREATE INDEX IF NOT EXISTS work_approval_audit_time_v1 + ON work_approval_audit_v1(environment,app_id,created_at)''') + + def record(self,row,event,action_class=None): + # Caller owns the approval transaction and owner lock. Failure rolls + # back the associated decision/consume instead of losing its audit. + if event not in EVENTS or row['state']not in STATES:raise ValueError('invalid_approval_audit') + scope=[row['environment'],row['app_id']] + if action_class is None: + created=self.db.execute('''SELECT action_class FROM work_approval_audit_v1 + WHERE environment=? AND app_id=? AND approval_id=? AND event='created' ''', + (*scope,row['approval_id'])).fetchone() + action_class=created['action_class']if created else'unknown' + if action_class not in ACTION_CLASSES:raise ValueError('invalid_approval_audit_class') + def digest(kind,value):return hashlib.sha256(canonical([*scope,kind,value])).hexdigest() + key=digest('audit',[row['approval_id'],event]) + self.db.execute('''INSERT INTO work_approval_audit_v1 VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?) + ON CONFLICT DO NOTHING''',(key,*scope,row['approval_id'],digest('task',row['task_id']), + digest('actor',row['owner']),digest('device',row['host_id']),row['binding_generation'], + row['operation_digest'],event,row['state'],action_class,self.clock())) diff --git a/deployable/relay/approval_callback.py b/deployable/relay/approval_callback.py new file mode 100644 index 0000000..42b279f --- /dev/null +++ b/deployable/relay/approval_callback.py @@ -0,0 +1,153 @@ +"""C2 callback verifier; HTTP registration is opt-in in approval_api. + +Only the SDK worker's authenticated internal channel or a signed platform +webhook may enter here. action.value is a reference, never actor identity. +""" +import base64 +import hashlib +import hmac +import json +import re +import time + +from cryptography.hazmat.primitives import padding +from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes + +from .approval_store import ApprovalError, VerifiedDecision +from .crypto import canonical + + +class CallbackError(ApprovalError): + pass + + +def _pairs(pairs): + result = {} + for key, value in pairs: + if key in result: + raise CallbackError('ambiguous_callback') + result[key] = value + return result + + +def _json(raw): + if not isinstance(raw, bytes) or not 0 < len(raw) <= 65536: + raise CallbackError('invalid_callback_budget') + try: + data = json.loads(raw, object_pairs_hook=_pairs, + parse_constant=lambda _: (_ for _ in ()).throw(ValueError())) + if not isinstance(data, dict): + raise ValueError() + return data + except (ValueError, RecursionError, UnicodeError): + raise CallbackError('invalid_callback_json') from None + + +def _text(value, pattern=r'[A-Za-z0-9_-]{1,128}'): + if not isinstance(value, str) or not re.fullmatch(pattern, value): + raise CallbackError('invalid_callback_field') + return value + + +def _equal_secret(actual, expected): + return (isinstance(actual, str) and isinstance(expected, str) and bool(expected) + and hmac.compare_digest(actual.encode(), expected.encode())) + + +class ApprovalCallbackVerifier: + def __init__(self, *, environment, app_id, tenant_key, verification_token, + internal_secret='', encrypt_key='', clock=time.time): + self.environment = _text(environment) + self.app_id = _text(app_id) + self.tenant_key = _text(tenant_key) + if not isinstance(verification_token, str) or not verification_token: + raise CallbackError('callback_verifier_not_configured') + self.verification_token = verification_token + self.internal_secret = internal_secret + self.encrypt_key = encrypt_key + self.clock = clock + + def from_sdk_channel(self, raw, supplied_secret): + """Future internal handler only; never expose as an unauthenticated API. + + Caller must enforce selected event source, a bounded body and the + private network boundary. The secret is not read from callback JSON. + """ + if len(self.internal_secret) < 32 or not _equal_secret(supplied_secret, self.internal_secret): + raise CallbackError('unauthenticated_callback_channel') + return self._decision(_json(raw)) + + def from_webhook(self, raw, *, timestamp, nonce, signature): + """Verify the original bytes before decrypting. No token-only fallback.""" + if not self.encrypt_key: + raise CallbackError('callback_verifier_not_configured') + _text(timestamp, r'[0-9]{1,12}') + _text(nonce) + _text(signature, r'[a-f0-9]{64}') + if abs(self.clock() - int(timestamp)) > 300: + raise CallbackError('stale_callback') + if not isinstance(raw, bytes) or not 0 < len(raw) <= 65536: + raise CallbackError('invalid_callback_budget') + expected = hashlib.sha256((timestamp + nonce + self.encrypt_key).encode() + raw).hexdigest() + if not hmac.compare_digest(expected, signature): + raise CallbackError('unauthenticated_callback_channel') + envelope = _json(raw) + try: + if set(envelope) != {'encrypt'} or not isinstance(envelope['encrypt'], str): + raise ValueError() + ciphertext = base64.b64decode(envelope['encrypt'], validate=True) + if len(ciphertext) < 32 or len(ciphertext) % 16: + raise ValueError() + decryptor = Cipher(algorithms.AES(hashlib.sha256(self.encrypt_key.encode()).digest()), + modes.CBC(ciphertext[:16])).decryptor() + padded = decryptor.update(ciphertext[16:]) + decryptor.finalize() + unpadder = padding.PKCS7(128).unpadder() + plaintext = unpadder.update(padded) + unpadder.finalize() + except (ValueError, TypeError): + raise CallbackError('invalid_callback_ciphertext') from None + return self._decision(_json(plaintext)) + + def _decision(self, data): + header, event = data.get('header'), data.get('event') + if data.get('schema') != '2.0' or not isinstance(header, dict) or not isinstance(event, dict): + raise CallbackError('unsupported_callback') + if (header.get('event_type') != 'card.action.trigger' + or header.get('app_id') != self.app_id + or header.get('tenant_key') != self.tenant_key + or not _equal_secret(header.get('token'), self.verification_token)): + raise CallbackError('callback_application_conflict') + event_id = _text(header.get('event_id')) + created = _text(header.get('create_time'), r'[0-9]{1,20}') + if abs(self.clock() - int(created) / 1000000) > 300: + raise CallbackError('stale_callback') + operator, context, action = event.get('operator'), event.get('context'), event.get('action') + if not all(isinstance(item, dict) for item in (operator, context, action)): + raise CallbackError('invalid_callback_context') + if (event.get('host') != 'im_message' or action.get('tag') != 'button' + or operator.get('tenant_key') != self.tenant_key): + raise CallbackError('unsupported_callback_context') + value = action.get('value') + if isinstance(value, str): + value = _json(value.encode()) + if not isinstance(value, dict) or set(value) != {'kind', 'approval_id', 'nonce', 'choice'}: + raise CallbackError('invalid_approval_reference') + if value['kind'] != 'hainuo.approval.v1' or value['choice'] not in ('allow', 'deny'): + raise CallbackError('invalid_approval_choice') + fields = dict(environment=self.environment, app_id=self.app_id, + actor_open_id=_text(operator.get('open_id'), r'ou_[A-Za-z0-9_-]{1,120}'), + chat_id=_text(context.get('open_chat_id'), r'oc_[A-Za-z0-9_-]{1,120}'), + card_message_id=_text(context.get('open_message_id'), r'om_[A-Za-z0-9_-]{1,120}'), + approval_id=_text(value['approval_id'], r'apr_[a-f0-9]{64}'), + nonce=_text(value['nonce'], r'[A-Za-z0-9_-]{43}'), choice=value['choice']) + # No token, raw callback or user text is persisted in the receipt. + digest = hashlib.sha256(canonical(fields)).hexdigest() + return VerifiedDecision(**fields, event_id=event_id, event_digest=digest) + + +def decision_toast(status): + """Bounded platform response; never starts a task or executes an action.""" + state = status.get('state') if isinstance(status, dict) else None + messages = {'approved': '已允许本次操作,等待客户端继续。', + 'rejected': '已拒绝本次操作。', 'expired': '确认已过期,请重新发起。', + 'consumed': '本次确认已使用,请勿重复操作。', 'cancelled': '本次操作已取消。'} + return {'toast': {'type': 'info', 'content': messages.get(state, '确认未完成,请核对任务状态。')}} diff --git a/deployable/relay/approval_cards.py b/deployable/relay/approval_cards.py new file mode 100644 index 0000000..19406a8 --- /dev/null +++ b/deployable/relay/approval_cards.py @@ -0,0 +1,81 @@ +"""C2 approval presentation and delivery through the dedicated DB worker.""" +import asyncio +import hashlib +import math +import re +import time + +from .approval_store import ApprovalError, _public_text + + +def approval_card(material, clock=time.time): + reference, nonce = material.get('approval_id'), material.get('nonce') + if (not isinstance(reference, str) or not re.fullmatch(r'apr_[a-f0-9]{64}', reference) + or not isinstance(nonce, str) or not re.fullmatch(r'[A-Za-z0-9_-]{43}', nonce)): + raise ApprovalError('invalid_approval_reference') + summary = _public_text(material.get('summary'), 200) + impact = _public_text(material.get('impact'), 500) + expiry = material.get('expires_at') + if type(expiry) not in (int, float) or not math.isfinite(expiry) or expiry <= clock(): + raise ApprovalError('approval_task_expired') + seconds = min(600, max(1, math.ceil(expiry - clock()))) + buttons = [] + for choice, label, style in (('allow', '允许本次', 'primary'), ('deny', '拒绝', 'default')): + buttons.append({'tag': 'button', 'text': {'tag': 'plain_text', 'content': label}, + 'type': style, 'value': {'kind': 'hainuo.approval.v1', 'approval_id': reference, + 'nonce': nonce, 'choice': choice}}) + return {'config': {'wide_screen_mode': True, 'enable_forward': False}, + 'header': {'template': 'orange', 'title': {'tag': 'plain_text', 'content': '海诺操作确认'}}, + 'elements': [ + {'tag': 'div', 'text': {'tag': 'plain_text', 'content': '准备做什么:' + summary}}, + {'tag': 'div', 'text': {'tag': 'plain_text', 'content': '可能的影响:' + impact}}, + {'tag': 'div', 'text': {'tag': 'plain_text', 'content': '仅允许本次操作;约' + str(seconds) + '秒后失效。'}}, + {'tag': 'action', 'actions': buttons}]} + + +class ApprovalCardSender: + def __init__(self, approvals, platform): + self.approvals, self.platform = approvals, platform + + async def send(self, approval_id): + material = await self.approvals.call('card_material',approval_id) + # Prepare before claiming: invalid presentation must not consume an intent. + card = approval_card(material, self.approvals.clock) + token = await self.platform.tenant_token() + if not await self.approvals.call('claim_card',approval_id): + return {'state': 'already_sent_or_unconfirmed'} + delivery_id = hashlib.sha256(('approval-card-v1\0' + approval_id).encode()).hexdigest()[:32] + try: + # Receipt is the actual reply card ID, never the source user message. + message_id = await asyncio.wait_for(self.platform.reply_task_card( + material['source_message_id'], card, delivery_id, token=token), timeout=10) + await self.approvals.call('bind_card',approval_id, message_id) + except BaseException: + # Cancellation/timeout can occur after platform acceptance. Never + # infer "not sent" and automatically issue another confirmation. + await self.approvals.call('card_send_unknown',approval_id) + raise + return {'state': 'sent'} + + async def update_closed(self,approval_id): + receipt=await self.approvals.call('closed_card_receipt',approval_id) + if not receipt:return {'state':'no_closed_card'} + card=closed_approval_card(receipt['state']) + token=await self.platform.tenant_token() + # Updating the known original message is retryable; sending a new + # confirmation card after a lost create receipt is not. + await asyncio.wait_for(self.platform.update_task_card( + receipt['message_id'],card,token=token),timeout=5) + await self.approvals.call('mark_card_closed',approval_id,receipt['state']) + return {'state':'updated'} + + +def closed_approval_card(state): + messages={'rejected':'你已拒绝本次操作。', + 'expired':'本次确认已过期,未获得新的执行许可;需要时请重新发起任务。', + 'cancelled':'本次确认已取消,不能再通过这张卡片批准。', + 'consumed':'本次许可已交给客户端;是否执行及执行结果请查看任务进度。'} + if state not in messages:raise ApprovalError('approval_not_closed') + return {'config':{'wide_screen_mode':True,'enable_forward':False}, + 'header':{'template':'grey','title':{'tag':'plain_text','content':'海诺操作确认已结束'}}, + 'elements':[{'tag':'div','text':{'tag':'plain_text','content':messages[state]}}]} diff --git a/deployable/relay/approval_policy.py b/deployable/relay/approval_policy.py new file mode 100644 index 0000000..8cc8cb2 --- /dev/null +++ b/deployable/relay/approval_policy.py @@ -0,0 +1,34 @@ +"""Describe exactly the request being approved; never trust model risk prose.""" +import json +import re +from .approval_store import ApprovalError + + +def describe_action(action): + if (not isinstance(action,dict)or set(action)!={'kind','method','path','params','body'} + or action['kind']!='work-api-v1' or action['method']not in ('GET','POST','PUT','PATCH','DELETE') + or not isinstance(action['path'],str)or len(action['path'])>1500 + or not re.fullmatch(r'/open-apis/[A-Za-z0-9_./@!:$-]+',action['path']) + or '..'in action['path']or '//'in action['path'] + or not isinstance(action['params'],dict)or not isinstance(action['body'],dict)): + raise ApprovalError('unsupported_approval_action') + encoded=json.dumps(action,sort_keys=True,ensure_ascii=True,separators=(',',':'),allow_nan=False).encode() + if len(encoded)>32000:raise ApprovalError('operation_budget_exceeded') + parts=action['path'].split('/');namespace=parts[2] + targets={'calendar':'日历','contact':'通讯录','search':'搜索','drive':'云盘', + 'docx':'文档','wiki':'知识库','bitable':'多维表格','task':'任务','sheets':'电子表格','vc':'会议'} + if namespace not in targets:raise ApprovalError('unsupported_approval_action') + method=action['method'];label=targets[namespace] + verb={'GET':'查询','POST':'提交','PUT':'更新','PATCH':'修改','DELETE':'删除'}[method] + subject=next((action['body'][key].strip()[:80]for key in ('summary','title','name') + if isinstance(action['body'].get(key),str)and action['body'][key].strip()),'') + if any(ord(char)<32 for char in subject):subject='' + summary=verb+label+'中的指定数据'+(':'+subject if subject else'') + resource='/'.join(parts[4:]) + if len(resource)>300:raise ApprovalError('approval_target_display_budget_exceeded') + impact=('本次是查询,但该接口尚未列入自动放行范围。'if method=='GET'else + '可能修改平台数据或触发通知;已经发生的操作不能保证自动撤销。') + if method=='DELETE':impact='会删除指定数据或移除关联,可能影响其他人,且不一定能够恢复。' + if re.search(r'/(permissions|members|collaborators|share)(/|$)',action['path']): + impact='可能改变其他人的访问权限或成员关系,请核对授权范围。' + return summary,impact+'\n目标范围:'+label+';目标标识:'+resource diff --git a/deployable/relay/approval_store.py b/deployable/relay/approval_store.py new file mode 100644 index 0000000..91dfb84 --- /dev/null +++ b/deployable/relay/approval_store.py @@ -0,0 +1,385 @@ +"""Offline foundation for C2: task-scoped approvals with one-time consumption. + +No HTTP routes, event subscription, platform message or execution adapter is +registered by this module. VerifiedDecision must only be constructed by a +platform callback verifier; a browser/button payload alone is not evidence. +""" +from dataclasses import dataclass +import hashlib +import hmac +import json +import re +import secrets +import time +from .crypto import canonical +from .approval_audit import ApprovalAudit + + +class ApprovalError(ValueError): + pass + + +@dataclass(frozen=True) +class VerifiedDecision: + environment: str + app_id: str + actor_open_id: str + chat_id: str + card_message_id: str + approval_id: str + nonce: str + choice: str + event_id: str + event_digest: str + + +def _public_text(value,limit): + if not isinstance(value,str)or not value.strip()or len(value)>limit or any(ord(c)<32 and c!='\n'for c in value): + raise ApprovalError('invalid_approval_text') + return value + + +class ApprovalStore: + def __init__(self,state,keyring,clock=time.time,enforce_execution_state=False): + self.state,self.db,self.keyring,self.clock=state,state.db,keyring,clock + self.enforce_execution_state=enforce_execution_state + self._sweep_after='' + self._retention_after='' + with self.db: + self.db.execute('''CREATE TABLE IF NOT EXISTS work_approvals_v3( + approval_id TEXT PRIMARY KEY,environment TEXT NOT NULL,app_id TEXT NOT NULL, + task_id TEXT NOT NULL,owner TEXT NOT NULL,host_id TEXT NOT NULL, + binding_generation BIGINT NOT NULL,operation_id TEXT NOT NULL, + operation_digest TEXT NOT NULL,nonce_digest TEXT NOT NULL, + key_id TEXT NOT NULL,ciphertext TEXT NOT NULL,card_message_id TEXT NOT NULL, + state TEXT NOT NULL,expires_at DOUBLE PRECISION NOT NULL, + updated_at DOUBLE PRECISION NOT NULL, + UNIQUE(environment,app_id,task_id,operation_id))''') + self.db.execute('''CREATE TABLE IF NOT EXISTS work_approval_callbacks_v1( + environment TEXT NOT NULL,app_id TEXT NOT NULL,event_id TEXT NOT NULL, + approval_id TEXT NOT NULL,event_digest TEXT NOT NULL, + received_at DOUBLE PRECISION NOT NULL, + PRIMARY KEY(environment,app_id,event_id))''') + self.db.execute('''CREATE TABLE IF NOT EXISTS work_approval_card_outbox_v1( + approval_id TEXT PRIMARY KEY,state TEXT NOT NULL, + updated_at DOUBLE PRECISION NOT NULL)''') + self.db.execute('''CREATE INDEX IF NOT EXISTS work_approvals_expiry_v3 + ON work_approvals_v3(environment,app_id,state,expires_at)''') + self.db.execute('''CREATE TABLE IF NOT EXISTS work_approval_retention_v1( + approval_id TEXT PRIMARY KEY,purged_at DOUBLE PRECISION NOT NULL)''') + self.audit=ApprovalAudit(self.db,self.clock) + + def _task(self,task_id,owner,host,generation): + if type(generation)is not int or generation<1: + raise ApprovalError('invalid_binding_generation') + task=self.state.task(task_id) + if (not self.state.task_current(task)or not task['chat_id']or not task['source_message_id']or + (task['owner'],task['host_id'],task['binding_generation'])!=(owner,host,generation)or + task['status']!='inflight'or self._execution_closed(task_id)): + raise ApprovalError('approval_task_unavailable') + return task + + def _execution_closed(self,task_id): + if not self.enforce_execution_state:return False + row=self.db.execute('SELECT execution_state FROM work_execution_state_v3 WHERE task_id=?',(task_id,)).fetchone() + return bool(row and row['execution_state']in {'completed','failed','cancelled','interrupted','unknown'}) + + def _aad(self,row): + return canonical(['work-approval-v3',*[row[k]for k in ('environment','app_id','task_id', + 'owner','host_id','binding_generation','operation_id','operation_digest','approval_id')]]) + + @staticmethod + def _intent(action,owner,operation_id): + normalized=json.loads(json.dumps(action,sort_keys=True,ensure_ascii=True,allow_nan=False)) + # The calendar key is generated by our PC boundary from owner+op ID; + # changing that generated key must not turn a denied action into a new + # business intent. No arbitrary caller field is discarded otherwise. + if (normalized.get('kind')=='work-api-v1'and normalized.get('method')=='POST' + and isinstance(normalized.get('path'),str) + and re.fullmatch(r'/open-apis/calendar/v4/calendars/[A-Za-z0-9_.@!:$-]+/events',normalized['path']) + and isinstance(normalized.get('params'),dict)): + expected=hashlib.sha256(canonical([owner,operation_id])).hexdigest() + if normalized['params'].get('idempotency_key')==expected: + normalized['params'].pop('idempotency_key') + return hashlib.sha256(canonical(normalized)).hexdigest() + + def _row(self,approval_id): + if not isinstance(approval_id,str)or not re.fullmatch(r'apr_[a-f0-9]{64}',approval_id): + raise ApprovalError('invalid_approval_reference') + row=self.db.execute('SELECT * FROM work_approvals_v3 WHERE approval_id=? AND environment=? AND app_id=?', + (approval_id,self.state.environment,self.state.app_id)).fetchone() + if not row:raise ApprovalError('approval_not_found') + return dict(row) + + def _expire(self,row): + if row['expires_at']<=self.clock()and row['state']in {'pending','approved'}: + self.db.execute("UPDATE work_approvals_v3 SET state='expired',updated_at=? WHERE approval_id=? AND state IN ('pending','approved')", + (self.clock(),row['approval_id'])) + row=self._row(row['approval_id']) + self.audit.record(row,'expired') + return row + + @staticmethod + def _status(row): + return {k:row[k]for k in ('approval_id','state','expires_at','operation_digest')} + + def create(self,task_id,owner,host,generation,operation_id,action,summary,impact,ttl=120): + if not isinstance(operation_id,str)or not re.fullmatch(r'[A-Za-z0-9_-]{8,100}',operation_id): + raise ApprovalError('invalid_operation_id') + if type(ttl)is not int or not 30<=ttl<=600:raise ApprovalError('invalid_approval_lifetime') + if not isinstance(action,dict)or not action:raise ApprovalError('invalid_operation') + encoded=json.dumps(action,sort_keys=True,ensure_ascii=True,separators=(',',':'),allow_nan=False).encode() + if len(encoded)>32000:raise ApprovalError('operation_budget_exceeded') + summary=_public_text(summary,200);impact=_public_text(impact,500) + digest=hashlib.sha256(encoded).hexdigest() + approval_id='apr_'+hashlib.sha256(canonical([self.state.environment,self.state.app_id,task_id,operation_id])).hexdigest() + with self.db: + self.state._owner_lock(owner) + task=self._task(task_id,owner,host,generation) + old=self.db.execute('SELECT * FROM work_approvals_v3 WHERE approval_id=?',(approval_id,)).fetchone() + if old: + row=self._expire(dict(old)) + if (row['owner'],row['host_id'],row['binding_generation'],row['operation_digest'])!=(owner,host,generation,digest): + raise ApprovalError('operation_identity_conflict') + return self._status(row) + # Scope to the SAME authenticated task. Legitimate subsequent user + # tasks are independent; changing a model-chosen operation ID inside + # this task cannot erase a rejection, expiry or consumed grant. + intent=self._intent(action,owner,operation_id) + previous=self.db.execute('''SELECT * FROM work_approvals_v3 + WHERE environment=? AND app_id=? AND task_id=? LIMIT 51''', + (self.state.environment,self.state.app_id,task_id)).fetchall() + if len(previous)>=50:raise ApprovalError('approval_task_budget_exceeded') + for prior in previous: + prior=dict(prior) + if not prior['ciphertext']:raise ApprovalError('approval_intent_unavailable') + original=json.loads(self.keyring.decrypt(prior['key_id'],prior['ciphertext'],self._aad(prior))) + if hmac.compare_digest(intent,self._intent(original['action'],prior['owner'],prior['operation_id'])): + raise ApprovalError('approval_intent_already_registered') + nonce=secrets.token_urlsafe(32) + row={'approval_id':approval_id,'environment':task['environment'],'app_id':task['app_id'], + 'task_id':task_id,'owner':owner,'host_id':host,'binding_generation':generation, + 'operation_id':operation_id,'operation_digest':digest} + key,cipher=self.keyring.encrypt(canonical({'action':action,'summary':summary,'impact':impact,'nonce':nonce}),self._aad(row)) + expires=min(self.clock()+ttl,task['deadline']) + if expires<=self.clock():raise ApprovalError('approval_task_expired') + self.db.execute('INSERT INTO work_approvals_v3 VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', + (approval_id,row['environment'],row['app_id'],task_id,owner,host,generation,operation_id,digest, + hashlib.sha256(nonce.encode()).hexdigest(),key,cipher,'','pending',expires,self.clock())) + action_class=('work_api_delete'if action.get('method')=='DELETE'else + 'work_api_read'if action.get('method')=='GET'else'work_api_write')if action.get('kind')=='work-api-v1'else'unknown' + self.audit.record(self._row(approval_id),'created',action_class) + return self._status(self._row(approval_id)) + + def card_material(self,approval_id): + """Internal sender only: private task origin + fixed confirmation text.""" + with self.db: + row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id)) + task=self._task(row['task_id'],row['owner'],row['host_id'],row['binding_generation']) + if row['state']!='pending':raise ApprovalError('approval_not_pending') + material=json.loads(self.keyring.decrypt(row['key_id'],row['ciphertext'],self._aad(row))) + return {'source_message_id':task['source_message_id'],'chat_id':task['chat_id'], + 'approval_id':approval_id,'summary':material['summary'],'impact':material['impact'], + 'nonce':material['nonce'],'expires_at':row['expires_at']} + + def bind_card(self,approval_id,message_id): + if not isinstance(message_id,str)or not re.fullmatch(r'om_[A-Za-z0-9_-]{1,120}',message_id): + raise ApprovalError('invalid_card_receipt') + with self.db: + row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id)) + task=self.state.task(row['task_id']) + if row['state']in {'pending','approved'}and(not self.state.task_current(task)or task['status']!='inflight'or self._execution_closed(row['task_id'])): + self.db.execute("UPDATE work_approvals_v3 SET state='cancelled',updated_at=? WHERE approval_id=?",(self.clock(),approval_id)) + self.audit.record(self._row(approval_id),'cancelled') + if not self.db.execute('SELECT approval_id FROM work_approval_card_outbox_v1 WHERE approval_id=?',(approval_id,)).fetchone(): + raise ApprovalError('approval_send_intent_required') + if row['card_message_id']not in ('',message_id):raise ApprovalError('approval_card_conflict') + self.db.execute('UPDATE work_approvals_v3 SET card_message_id=?,updated_at=? WHERE approval_id=?', + (message_id,self.clock(),approval_id)) + self.db.execute("UPDATE work_approval_card_outbox_v1 SET state='sent',updated_at=? WHERE approval_id=?", + (self.clock(),approval_id)) + self.audit.record(self._row(approval_id),'card_bound') + + def claim_card(self,approval_id): + """Persist send intent before network I/O; lost receipt is not resendable.""" + with self.db: + row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id)) + self._task(row['task_id'],row['owner'],row['host_id'],row['binding_generation']) + if row['state']!='pending':raise ApprovalError('approval_not_pending') + if row['card_message_id']:return False + previous=self.db.execute('SELECT state FROM work_approval_card_outbox_v1 WHERE approval_id=?',(approval_id,)).fetchone() + if previous:return False + self.db.execute("INSERT INTO work_approval_card_outbox_v1 VALUES(?,'sending',?)",(approval_id,self.clock())) + return True + + def card_send_unknown(self,approval_id): + with self.db: + changed=self.db.execute("UPDATE work_approval_card_outbox_v1 SET state='unknown',updated_at=? WHERE approval_id=? AND state='sending'", + (self.clock(),approval_id)).rowcount + if changed:self.audit.record(self._row(approval_id),'card_send_unknown') + + def close_unavailable(self,limit=100): + """Future bounded maintenance: expiry or invalid task never grants consent.""" + if type(limit)is not int or not 1<=limit<=100:raise ApprovalError('invalid_approval_budget') + rows=self.db.execute("SELECT approval_id FROM work_approvals_v3 WHERE environment=? AND app_id=? AND state IN ('pending','approved') AND approval_id>? ORDER BY approval_id LIMIT ?", + (self.state.environment,self.state.app_id,self._sweep_after,limit)).fetchall() + if not rows:self._sweep_after='' + closed=[];started=time.monotonic() + for candidate in rows: + if time.monotonic()-started>=.5:break + with self.db: + row=self._row(candidate['approval_id']);self.state._owner_lock(row['owner']);row=self._row(row['approval_id']) + if row['state']not in {'pending','approved'}:continue + task=self.state.task(row['task_id']) + expired=row['expires_at']<=self.clock()or bool(task and task['deadline']<=self.clock()) + invalid=(not self.state.task_current(task)or task['status']!='inflight'or self._execution_closed(row['task_id'])or + (task['owner'],task['host_id'],task['binding_generation'])!= + (row['owner'],row['host_id'],row['binding_generation'])) + if expired or invalid: + state='expired'if expired else'cancelled' + self.db.execute("UPDATE work_approvals_v3 SET state=?,updated_at=? WHERE approval_id=? AND state IN ('pending','approved')", + (state,self.clock(),row['approval_id'])) + closed.append({'approval_id':row['approval_id'],'state':state}) + self.audit.record(self._row(row['approval_id']),state) + self._sweep_after=candidate['approval_id'] + return closed + + def closed_card_receipt(self,approval_id): + """Internal renderer only. Never return action/nonce or imply execution success.""" + with self.db: + row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id)) + if row['state']not in {'rejected','expired','cancelled','consumed'}or not row['card_message_id']: + return None + sent=self.db.execute('SELECT state FROM work_approval_card_outbox_v1 WHERE approval_id=?',(approval_id,)).fetchone() + if sent and sent['state']=='closed':return None + self.db.execute('UPDATE work_approval_card_outbox_v1 SET updated_at=? WHERE approval_id=?', + (self.clock(),approval_id)) + return {'message_id':row['card_message_id'],'state':row['state']} + + def mark_card_closed(self,approval_id,state): + with self.db: + row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._row(approval_id) + if row['state']!=state or state not in {'rejected','expired','cancelled','consumed'}: + raise ApprovalError('approval_state_changed') + self.db.execute("UPDATE work_approval_card_outbox_v1 SET state='closed',updated_at=? WHERE approval_id=?", + (self.clock(),approval_id)) + + def cards_to_send(self): + rows=self.db.execute("""SELECT a.approval_id FROM work_approvals_v3 a + WHERE a.environment=? AND a.app_id=? AND a.state='pending' AND a.expires_at>? + AND a.card_message_id='' AND NOT EXISTS(SELECT 1 FROM work_approval_card_outbox_v1 o WHERE o.approval_id=a.approval_id) + ORDER BY a.updated_at LIMIT 5""",(self.state.environment,self.state.app_id,self.clock())).fetchall() + return [row['approval_id']for row in rows] + + def cards_to_close(self): + rows=self.db.execute("""SELECT a.approval_id FROM work_approvals_v3 a + JOIN work_approval_card_outbox_v1 o ON o.approval_id=a.approval_id + WHERE a.environment=? AND a.app_id=? AND a.state IN ('rejected','expired','cancelled','consumed') + AND a.card_message_id<>'' AND o.state<>'closed' ORDER BY o.updated_at LIMIT 5""", + (self.state.environment,self.state.app_id)).fetchall() + return [row['approval_id']for row in rows] + + def decide(self,decision): + if not isinstance(decision,VerifiedDecision):raise ApprovalError('verified_callback_required') + if decision.choice not in {'allow','deny'}:raise ApprovalError('invalid_approval_choice') + if (not isinstance(decision.event_id,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',decision.event_id) + or not isinstance(decision.event_digest,str)or not re.fullmatch(r'[a-f0-9]{64}',decision.event_digest)): + raise ApprovalError('invalid_callback_receipt') + fields={key:getattr(decision,key)for key in ('environment','app_id','actor_open_id', + 'chat_id','card_message_id','approval_id','nonce','choice')} + if not hmac.compare_digest(hashlib.sha256(canonical(fields)).hexdigest(),decision.event_digest): + raise ApprovalError('callback_receipt_conflict') + with self.db: + row=self._row(decision.approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(decision.approval_id)) + task=self._task(row['task_id'],row['owner'],row['host_id'],row['binding_generation']) + if (decision.environment,decision.app_id,decision.actor_open_id,decision.chat_id,decision.card_message_id)!=\ + (row['environment'],row['app_id'],row['owner'],task['chat_id'],row['card_message_id'])or not row['card_message_id']: + raise ApprovalError('callback_identity_conflict') + if not isinstance(decision.nonce,str)or len(decision.nonce)>128 or not hmac.compare_digest(hashlib.sha256(decision.nonce.encode()).hexdigest(),row['nonce_digest']): + raise ApprovalError('invalid_approval_nonce') + # Owner lock serializes duplicate clicks for this approval. Reused + # event IDs for a different approval also fail the database PK. + receipt=self.db.execute('SELECT approval_id,event_digest FROM work_approval_callbacks_v1 WHERE environment=? AND app_id=? AND event_id=?', + (row['environment'],row['app_id'],decision.event_id)).fetchone() + if receipt: + if (receipt['approval_id'],receipt['event_digest'])!=(row['approval_id'],decision.event_digest): + raise ApprovalError('callback_receipt_conflict') + return self._status(row) + target='approved'if decision.choice=='allow'else'rejected' + if row['state']not in {'pending',target}:raise ApprovalError('approval_already_closed') + self.db.execute('INSERT INTO work_approval_callbacks_v1 VALUES(?,?,?,?,?,?)', + (row['environment'],row['app_id'],decision.event_id,row['approval_id'],decision.event_digest,self.clock())) + self.db.execute("UPDATE work_approvals_v3 SET state=?,updated_at=? WHERE approval_id=? AND state='pending'", + (target,self.clock(),row['approval_id'])) + self.audit.record(self._row(row['approval_id']),target) + return self._status(self._row(row['approval_id'])) + + def status(self,approval_id,task_id,owner,host,generation,operation_digest): + """Future device-authenticated polling endpoint; no nonce/action returned.""" + with self.db: + self.state._owner_lock(owner);row=self._expire(self._row(approval_id)) + self._task(task_id,owner,host,generation) + if (row['task_id'],row['owner'],row['host_id'],row['binding_generation'],row['operation_digest'])!=\ + (task_id,owner,host,generation,operation_digest):raise ApprovalError('operation_identity_conflict') + return self._status(row) + + def consume(self,approval_id,task_id,owner,host,generation,operation_digest): + with self.db: + self.state._owner_lock(owner) + row=self._expire(self._row(approval_id)) + self._task(task_id,owner,host,generation) + if (row['task_id'],row['owner'],row['host_id'],row['binding_generation'],row['operation_digest'])!=(task_id,owner,host,generation,operation_digest): + raise ApprovalError('operation_identity_conflict') + if row['state']!='approved':return {**self._status(row),'execute':False} + material=json.loads(self.keyring.decrypt(row['key_id'],row['ciphertext'],self._aad(row))) + changed=self.db.execute("UPDATE work_approvals_v3 SET state='consumed',updated_at=? WHERE approval_id=? AND state='approved'", + (self.clock(),approval_id)).rowcount + if changed!=1:return {**self._status(self._row(approval_id)),'execute':False} + self.audit.record(self._row(approval_id),'consumed') + return {**self._status(self._row(approval_id)),'execute':True,'action':material['action']} + + def cancel(self,approval_id,task_id,owner,host,generation): + with self.db: + self.state._owner_lock(owner);row=self._row(approval_id) + self._task(task_id,owner,host,generation) + if (row['task_id'],row['owner'],row['host_id'],row['binding_generation'])!=(task_id,owner,host,generation):raise ApprovalError('approval_identity_conflict') + changed=self.db.execute("UPDATE work_approvals_v3 SET state='cancelled',updated_at=? WHERE approval_id=? AND state IN ('pending','approved')", + (self.clock(),approval_id)).rowcount + if changed:self.audit.record(self._row(approval_id),'cancelled') + return self._status(self._row(approval_id)) + + def maintain_retention(self,payload_days=30,audit_days=90,limit=10): + """Prune CLOSED payloads/audit only. Durable tombstones fence replay.""" + if (type(payload_days)is not int or type(audit_days)is not int + or not 1<=payload_days<=audit_days<=365 or type(limit)is not int or not 1<=limit<=20): + raise ApprovalError('invalid_approval_retention') + cutoff=self.clock()-payload_days*86400 + rows=self.db.execute('''SELECT approval_id FROM work_approvals_v3 a WHERE environment=? AND app_id=? + AND state IN ('rejected','expired','cancelled','consumed') AND updated_at? + AND NOT EXISTS(SELECT 1 FROM work_approval_retention_v1 r WHERE r.approval_id=a.approval_id) + ORDER BY approval_id LIMIT ?''',(self.state.environment,self.state.app_id,cutoff,self._retention_after,limit)).fetchall() + if not rows:self._retention_after='' + purged=0;started=time.monotonic() + for item in rows: + if time.monotonic()-started>.5:break + self._retention_after=item['approval_id'] + with self.db: + row=self._row(item['approval_id']);self.state._owner_lock(row['owner']);row=self._row(row['approval_id']) + if row['state']not in {'rejected','expired','cancelled','consumed'}or row['updated_at']>=cutoff:continue + if row['state']=='consumed': + task=self.state.task(row['task_id']) + if not task or task['status']in {'inflight','unknown'}:continue + marked=self.db.execute('INSERT INTO work_approval_retention_v1 VALUES(?,?) ON CONFLICT DO NOTHING', + (row['approval_id'],self.clock())).rowcount + if not marked:continue + self.db.execute("UPDATE work_approvals_v3 SET ciphertext='',key_id='',nonce_digest='' WHERE approval_id=?",(row['approval_id'],)) + self.audit.record(row,'payload_purged');purged+=1 + # Audit removal is bounded too; approval and operation tombstones stay. + with self.db: + removed=self.db.execute('''DELETE FROM work_approval_audit_v1 WHERE event_key IN ( + SELECT x.event_key FROM work_approval_audit_v1 x JOIN work_approval_retention_v1 r + ON r.approval_id=x.approval_id WHERE x.environment=? AND x.app_id=? AND x.created_atdeadline or self.store is None: + raise ApprovalError('approval_temporarily_unavailable') + try: + # Postgres may reconnect after a lost connection. Reapply this + # worker's limits before opening a new approval transaction. + if self.settings.database_url: + db=self.connection.db + if db.conn.closed or db.conn.broken: + db.execute("SET lock_timeout TO '500ms'") + db.execute("SET statement_timeout TO '800ms'") + return getattr(self.store,method)(*args) + except ApprovalError:raise + except Exception:raise ApprovalError('approval_temporarily_unavailable')from None + + async def call(self,method,*args): + allowed={'create','status','consume','cancel','decide','card_material','claim_card', + 'bind_card','card_send_unknown','close_unavailable','closed_card_receipt', + 'mark_card_closed','cards_to_send','cards_to_close','maintain_retention'} + if method not in allowed:raise ApprovalError('unsupported_approval_operation') + try:await asyncio.wait_for(self.slots.acquire(),timeout=.05) + except asyncio.TimeoutError:raise ApprovalError('approval_temporarily_unavailable')from None + future=asyncio.get_running_loop().run_in_executor(self.executor,self._invoke, + time.monotonic()+1.2,method,args) + # Keep the queue slot until the worker really stops, even when a caller + # times out. A timeout is not proof that a database commit did not occur. + def settled(completed): + self.slots.release() + if not completed.cancelled():completed.exception() + future.add_done_callback(settled) + try:return await asyncio.wait_for(asyncio.shield(future),timeout=1.5) + except asyncio.TimeoutError:raise ApprovalError('approval_result_unconfirmed')from None + + async def close(self): + def finish(): + if self.store is not None:self.connection.close() + self.store=None + try:await asyncio.get_running_loop().run_in_executor(self.executor,finish) + finally:self.executor.shutdown(wait=False,cancel_futures=True) diff --git a/deployable/relay/config.py b/deployable/relay/config.py index 1cb0c8d..71c1d12 100644 --- a/deployable/relay/config.py +++ b/deployable/relay/config.py @@ -42,6 +42,11 @@ class Settings(BaseSettings): max_result: int = 16000 work_files_enabled: bool = False work_task_cards_enabled: bool = False + work_remote_approvals_enabled: bool = False + feishu_tenant_key: str = '' + approval_retention_enabled: bool = False + approval_payload_retention_days: int = 30 + approval_audit_retention_days: int = 90 work_file_max_bytes: int = 10*1024*1024 work_file_batch_limit: int = 20 work_file_retention: int = 86400 @@ -57,6 +62,15 @@ class Settings(BaseSettings): @model_validator(mode="after") def validate_runtime(self): + if not 1<=self.approval_payload_retention_days<=self.approval_audit_retention_days<=365: + raise ValueError('approval retention requires 1 <= payload days <= audit days <= 365') + if self.approval_retention_enabled and not self.work_remote_approvals_enabled: + raise ValueError('approval retention requires remote approvals') + if self.work_remote_approvals_enabled: + if (self.credential_mode!='broker'or self.feishu_event_source!='longconn' + or not self.feishu_enabled or not self.feishu_verification_token + or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',self.feishu_tenant_key)): + raise ValueError('remote approvals require broker, longconn and explicit callback identity') if self.work_task_cards_enabled and self.credential_mode!='broker':raise ValueError('task cards require broker mode') if not 1024<=self.work_file_max_bytes<=100*1024*1024 or not 1<=self.work_file_batch_limit<=50 or not 300<=self.work_file_retention<=7*86400: raise ValueError('invalid work file deployment limits') diff --git a/deployable/relay/connector.py b/deployable/relay/connector.py index edc7c65..7cac28d 100644 --- a/deployable/relay/connector.py +++ b/deployable/relay/connector.py @@ -265,6 +265,8 @@ def create_app(settings=None, redis=None, platform=None): lifecycle = None from .notifications import router as notification_router background = set() + approvals = None + approval_worker = None def spawn(coro): task = asyncio.create_task(coro) @@ -273,9 +275,16 @@ def create_app(settings=None, redis=None, platform=None): # Retrieve errors without displaying possibly sensitive exception text. task.add_done_callback(lambda t: t.exception() if not t.cancelled() else None) + if s.work_remote_approvals_enabled: + from .approval_api import ApprovalAPI + from .approval_worker import ApprovalWorker + approval_worker=ApprovalWorker(s,credential_store.keyring) + approvals=ApprovalAPI(oauth,approval_worker,f,spawn) + @asynccontextmanager async def lifespan(app): await r.ping() + if approval_worker:await approval_worker.start() if lifecycle: await lifecycle.maintain() await deliveries.recover_interrupted_sends() @@ -302,6 +311,7 @@ def create_app(settings=None, redis=None, platform=None): await deliveries.recover_interrupted_sends() if broker.files:await broker.files.flush() await deliveries.flush() + if approvals:await approvals.maintain() except Exception: # No credentials or raw exception text in background logs. pass @@ -323,14 +333,19 @@ def create_app(settings=None, redis=None, platform=None): await r.delete("hn:task-worker:"+broker.worker_id) await pub.aclose() await f.close() + if approval_worker:await approval_worker.close() store.close() if redis is None: await r.aclose() app = FastAPI(title="Hainuo B1 connector", lifespan=lifespan, docs_url=None, redoc_url=None) app.add_middleware(BodyLimitMiddleware, max_body=s.max_body, - path_limits={'/work/v2/files/upload':s.work_file_max_bytes}if broker.files else {}) + path_limits={**({'/work/v2/files/upload':s.work_file_max_bytes}if broker.files else {}), + **({'/internal/approval-callback':65536,'/work/v2/approvals/create':49152}if approvals else {})}) app.include_router(oauth.router) + if approvals: + app.include_router(approvals.router) + app.state.approvals=approvals if lifecycle: app.include_router(deliveries.router) app.state.deliveries=deliveries diff --git a/deployable/relay/credential_api.py b/deployable/relay/credential_api.py index 1567bb3..b95faee 100644 --- a/deployable/relay/credential_api.py +++ b/deployable/relay/credential_api.py @@ -121,6 +121,7 @@ class CredentialAPI: raise HTTPException(503, "credential_temporarily_unavailable") from None self.require_binding(body.host_id, device["owner"], device["binding_generation"]) return no_cache(JSONResponse({"credential_mode": "broker", "protocol_version": 2, + "approval_protocol": "work-approval-v1" if self.s.work_remote_approvals_enabled else None, "binding_owner": device["owner"], "binding_generation": device["binding_generation"], "permission_profiles": sorted(self.s.credential_permission_profiles), **result})) diff --git a/deployable/relay/longconn.py b/deployable/relay/longconn.py index 6d7be70..851b8d8 100644 --- a/deployable/relay/longconn.py +++ b/deployable/relay/longconn.py @@ -4,6 +4,7 @@ import logging import os import threading import time +from concurrent.futures import ThreadPoolExecutor,TimeoutError as FutureTimeout import httpx import redis @@ -51,7 +52,52 @@ def main(): raise RuntimeError("connector did not accept event") print(json.dumps({"event": "platform_event_accepted", "type": data.get("header", {}).get("event_type")}), flush=True) - handler = lark.EventDispatcherHandler.builder("", "").register_p2_im_message_receive_v1(received).build() + builder = lark.EventDispatcherHandler.builder("", "").register_p2_im_message_receive_v1(received) + callback_pool=None + if s.work_remote_approvals_enabled: + from lark_oapi.event.callback.model.p2_card_action_trigger import P2CardActionTriggerResponse + callback_pool=ThreadPoolExecutor(max_workers=2,thread_name_prefix='work-card-callback') + callback_slots=threading.BoundedSemaphore(2) + def forward_card(raw): + deadline=time.monotonic()+2.1 + # Own this client until the worker ends. The SDK response deadline + # does not imply an in-flight authenticated decision was rolled back. + with httpx.Client(base_url=url,trust_env=False,follow_redirects=False, + timeout=httpx.Timeout(.8,connect=.2,pool=.1))as callback_http: + with callback_http.stream('POST','/internal/approval-callback',content=raw, + headers={'X-Internal-Secret':s.internal_secret,'Content-Type':'application/json'})as response: + if response.status_code!=200:raise ValueError() + body=bytearray() + for chunk in response.iter_bytes(): + body.extend(chunk) + if len(body)>4096 or time.monotonic()>deadline:raise ValueError() + return json.loads(body) + def decision(event): + # Same authenticated SDK connection/lease, no second consumer. + # Never log callback bodies or the card-update credential. + fallback={'toast':{'type':'warning','content':'确认结果待核对,请查看原任务状态。'}} + try: + raw=lark.JSON.marshal(event).encode('utf-8') + if len(raw)>65536:return P2CardActionTriggerResponse(fallback) + if not callback_slots.acquire(blocking=False):return P2CardActionTriggerResponse(fallback) + try:future=callback_pool.submit(forward_card,raw) + except BaseException: + callback_slots.release() + raise + def settled(completed): + callback_slots.release() + if not completed.cancelled():completed.exception() + future.add_done_callback(settled) + try:payload=future.result(timeout=2.3) + except FutureTimeout:return P2CardActionTriggerResponse(fallback) + toast=payload.get('toast')if isinstance(payload,dict)else None + if (not isinstance(toast,dict)or toast.get('type')not in {'info','success','warning','error'} + or not isinstance(toast.get('content'),str)or len(toast['content'])>200): + return P2CardActionTriggerResponse(fallback) + return P2CardActionTriggerResponse({'toast':toast}) + except Exception:return P2CardActionTriggerResponse(fallback) + builder=builder.register_p2_card_action_trigger(decision) + handler=builder.build() class ObservableClient(lark.ws.Client): async def _connect(self): try: @@ -69,6 +115,7 @@ def main(): finally: done.set() client.close() + if callback_pool:callback_pool.shutdown(wait=False,cancel_futures=True) try: lease.release() except redis.exceptions.LockError: diff --git a/deployable/relay/task_cards.py b/deployable/relay/task_cards.py index 98173d4..756d174 100644 --- a/deployable/relay/task_cards.py +++ b/deployable/relay/task_cards.py @@ -10,7 +10,7 @@ DELIVERY={None:'尚未接收结果','ready':'已接收结果,等待投递','se 'replied':'结果已发送到当前会话','unknown':'结果投递待核对','discarded':'结果已停止交付'} -def card_content(task_id,execution,delivery,files=None,connection_lost=False): +def card_content(task_id,execution,delivery,files=None,connection_lost=False,remote_approvals=False): execution=execution if execution in EXECUTION else 'unknown' color='green' if execution=='completed' else 'red' if execution in {'failed','interrupted'} else 'orange' if execution in {'waiting_for_approval','unknown'} else 'blue' label=EXECUTION[execution] @@ -20,7 +20,9 @@ def card_content(task_id,execution,delivery,files=None,connection_lost=False): {'tag':'div','text':{'tag':'plain_text','content':'文字回复:'+DELIVERY.get(delivery,'状态待核对')}}, {'tag':'note','elements':[{'tag':'plain_text','content':'任务编号:'+task_id}]}] if execution=='waiting_for_approval': - elements.insert(1,{'tag':'div','text':{'tag':'plain_text','content':'请在客户端查看并确认具体操作。本卡片暂不提供远程批准。'}}) + hint=('请在本任务的操作确认卡片中核对动作和影响,选择允许本次或拒绝。'if remote_approvals else + '请在客户端查看并确认具体操作。本卡片暂不提供远程批准。') + elements.insert(1,{'tag':'div','text':{'tag':'plain_text','content':hint}}) if execution=='completed' and delivery!='replied': elements.insert(1,{'tag':'div','text':{'tag':'plain_text','content':'任务已执行完成,结果交付请看下方状态。'}}) return {'config':{'wide_screen_mode':True,'enable_forward':False}, @@ -28,9 +30,10 @@ def card_content(task_id,execution,delivery,files=None,connection_lost=False): class TaskCards: - def __init__(self,state,platform,worker_id,clock=time.time,files_enabled=False): + def __init__(self,state,platform,worker_id,clock=time.time,files_enabled=False,remote_approvals=False): self.state,self.db,self.platform,self.worker_id,self.clock=state,state.db,platform,worker_id,clock self.files_enabled=files_enabled + self.remote_approvals=remote_approvals with self.db: self.db.execute('''CREATE TABLE IF NOT EXISTS work_task_connection_v3( task_id TEXT PRIMARY KEY,reason TEXT NOT NULL)''') @@ -57,7 +60,8 @@ class TaskCards: if task['status']=='unknown' and execution not in {'completed','failed','cancelled','interrupted'}:execution='unknown' if task['deadline']<=self.clock() and execution not in {'completed','failed','cancelled','interrupted'}:execution='unknown' lost=self.db.execute('SELECT reason FROM work_task_connection_v3 WHERE task_id=?',(task['turn_id'],)).fetchone() - return card_content(task['turn_id'],execution,delivery['status'] if delivery else None,connection_lost=bool(lost)) + return card_content(task['turn_id'],execution,delivery['status'] if delivery else None, + connection_lost=bool(lost),remote_approvals=self.remote_approvals) async def flush(self): # Bound one maintenance pass; business result delivery remains independent. diff --git a/deployable/relay/task_delivery.py b/deployable/relay/task_delivery.py index f684794..8b99a4c 100644 --- a/deployable/relay/task_delivery.py +++ b/deployable/relay/task_delivery.py @@ -43,7 +43,8 @@ class TaskDelivery: self.cards=None if auth.s.work_task_cards_enabled: from .task_cards import TaskCards - self.cards=TaskCards(state,platform,broker.worker_id,files_enabled=auth.s.work_files_enabled) + self.cards=TaskCards(state,platform,broker.worker_id,files_enabled=auth.s.work_files_enabled, + remote_approvals=auth.s.work_remote_approvals_enabled) broker.task_cards=self.cards def current(self, body): diff --git a/deployable/scripts/approval_identity.py b/deployable/scripts/approval_identity.py new file mode 100644 index 0000000..53b5147 --- /dev/null +++ b/deployable/scripts/approval_identity.py @@ -0,0 +1,178 @@ +#!/usr/bin/env python3 +"""Prepare a private callback identity file without printing the token.""" +import argparse +import getpass +import json +import os +from pathlib import Path +import re +import subprocess + + +# Runs inside the EXISTING connector, not a new SDK/consumer container. All +# stored credentials stay there. SQL is read-only; never refresh or authorize. +DISCOVERY_CODE = r""" +import json,re,sqlite3,time +from urllib.parse import urlsplit +import httpx +from relay.config import Settings +from relay.credential_store import CredentialStore,CredentialKeyring +from relay.store import Postgres + +diag={'stage':'settings','matched_bindings':0,'expired_access':0,'credential_errors':0, + 'user_mismatch':0,'missing_tenant':0,'platform_rejected':0,'network_errors':0} + +def discover(): + s=Settings(role='connector') + target=urlsplit(s.feishu_api_base) + if (target.scheme!='https' or not target.hostname or target.username or target.password + or target.path not in ('','/') or target.query or target.fragment): + raise ValueError() + target.port + if s.credential_mode!='broker' or not s.credential_keyring_file:raise ValueError() + diag['stage']='database_connect' + if s.database_url: + db=Postgres(s.database_url) + db.execute('SET default_transaction_read_only TO on') + db.execute("SET statement_timeout TO '2000ms'") + else: + db=sqlite3.connect((s.data_dir/'metadata.sqlite').resolve().as_uri()+'?mode=ro',uri=True,timeout=2) + db.row_factory=sqlite3.Row + try: + diag['stage']='credentials_query' + rows=db.execute('''SELECT c.principal_id FROM work_credentials_v2 c + JOIN devices d ON d.owner=c.principal_id + WHERE c.environment=? AND c.app_id=? AND d.revoked=0 + ORDER BY c.updated_at DESC LIMIT 3''',(s.environment,s.feishu_app_id)).fetchall() + diag['matched_bindings']=len(rows) + # Reuse the existing getter WITHOUT running its schema constructor. + diag['stage']='keyring' + credentials=CredentialStore.__new__(CredentialStore) + credentials.db=db;credentials.keyring=CredentialKeyring.from_file(s.credential_keyring_file) + with httpx.Client(base_url=s.feishu_api_base.rstrip('/'),trust_env=False, + follow_redirects=False,timeout=httpx.Timeout(3,connect=2))as client: + for row in rows: + try: + diag['stage']='credential_read' + value=credentials.get(s.environment,s.feishu_app_id,row['principal_id']) + if not value or value.access_expires_at<=time.time()+10: + diag['expired_access']+=1 + continue + diag['stage']='user_info' + with client.stream('GET','/open-apis/authen/v1/user_info', + headers={'Authorization':'Bearer '+value.access_token})as response: + diag['http_status']=response.status_code + if response.status_code!=200: + diag['platform_rejected']+=1 + continue + body=bytearray() + for chunk in response.iter_bytes(): + body.extend(chunk) + if len(body)>65536:raise ValueError() + data=json.loads(body) + info=data.get('data',{}) + if type(data.get('code'))is int:diag['platform_code']=data['code'] + if data.get('code')!=0 or not isinstance(info,dict): + diag['platform_rejected']+=1 + continue + if info.get('open_id')!=row['principal_id']: + diag['user_mismatch']+=1 + continue + tenant=info.get('tenant_key') + if not isinstance(tenant,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',tenant): + diag['missing_tenant']+=1 + continue + return {'status':'verified','tenant_key':tenant,'app_id':s.feishu_app_id,'environment':s.environment} + except httpx.HTTPError as error: + diag['network_errors']+=1 + diag['error_class']=type(error).__name__ + continue + except Exception as error: + diag['credential_errors']+=1 + diag['error_class']=type(error).__name__ + continue + raise ValueError() + finally:db.close() + +try:print(json.dumps(discover())) +except Exception as error: + if not any(diag.get(k)for k in ('expired_access','credential_errors','user_mismatch','missing_tenant','platform_rejected','network_errors')): + diag['error_class']=type(error).__name__ + print(json.dumps({'status':'tenant_discovery_unavailable','diagnostic':diag})) +""" + + +def safe_diagnostic(value): + if not isinstance(value,dict):return {} + result={} + if value.get('stage')in {'settings','database_connect','credentials_query','keyring','credential_read','user_info'}: + result['stage']=value['stage'] + if value.get('error_class')in {'ValidationError','OperationalError','ProgrammingError','UndefinedTable', + 'UndefinedColumn','CredentialError','PermissionError','FileNotFoundError','ReadOnlySqlTransaction', + 'ConnectError','ConnectTimeout','ReadTimeout','SSLError','ValueError','TypeError'}: + result['error_class']=value['error_class'] + for key in ('matched_bindings','expired_access','credential_errors','user_mismatch','missing_tenant', + 'platform_rejected','network_errors','http_status','platform_code'): + number=value.get(key) + if type(number)is int and 0<=number<=2147483647:result[key]=number + return result + + +def discover_tenant(container,env): + result=subprocess.run(['docker','exec','-i',container,'python','-'],input=DISCOVERY_CODE, + stdout=subprocess.PIPE,stderr=subprocess.PIPE,universal_newlines=True,timeout=25) + try: + if result.returncode or len(result.stdout)>4096: + print('TENANT_DIAGNOSTIC '+json.dumps({'stage':'container_execution','exit_code':result.returncode})) + raise ValueError() + value=json.loads(result.stdout) + if isinstance(value,dict)and value.get('status')=='tenant_discovery_unavailable': + print('TENANT_DIAGNOSTIC '+json.dumps(safe_diagnostic(value.get('diagnostic')))) + raise ValueError() + if (set(value)!={'status','tenant_key','app_id','environment'}or value['status']!='verified' + or value['app_id']!=env.get('HN_FEISHU_APP_ID') + or value['environment']!=env.get('HN_ENVIRONMENT','production') + or not isinstance(value['tenant_key'],str) + or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',value['tenant_key'])): + raise ValueError() + return value['tenant_key'] + except (ValueError,TypeError,KeyError): + raise ValueError('tenant discovery unavailable; preserve current services and check Work authorization')from None + + +def main(): + parser=argparse.ArgumentParser() + parser.add_argument('--output') + parser.add_argument('--diagnose',action='store_true',help='Read-only discovery check; no output file or token prompt.') + parser.add_argument('--container',default='hainuo-relay-connector-1') + args=parser.parse_args();path=Path(args.output)if args.output else None + if not args.diagnose and(path is None or not path.is_absolute()or path.exists()or path.is_symlink()): + raise ValueError('output must be a new absolute file') + # Keep existing fields in process; never print docker Env or token values. + result=subprocess.run(['docker','inspect',args.container,'--format','{{json .Config.Env}}'], + stdout=subprocess.PIPE,stderr=subprocess.PIPE,universal_newlines=True) + if result.returncode:raise ValueError('cannot inspect current connector') + env=dict(item.split('=',1)for item in json.loads(result.stdout)if '='in item) + tenant=discover_tenant(args.container,env) + if args.diagnose: + print('TENANT_DIAGNOSTIC '+json.dumps({'status':'verified', + 'verification_token_configured':bool(env.get('HN_FEISHU_VERIFICATION_TOKEN'))})) + return + configured=env.get('HN_FEISHU_TENANT_KEY') + if configured and configured!=tenant: + raise ValueError('configured tenant does not match verified platform identity') + token=env.get('HN_FEISHU_VERIFICATION_TOKEN')or getpass.getpass('请输入应用“事件与回调”中的 Verification Token(输入不显示):') + if (not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',tenant)or not 8<=len(token)<=512 + or any(ord(c)<33 or ord(c)>126 for c in token)): + raise ValueError('invalid callback identity; no values displayed') + fd=os.open(str(path),os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600) + with os.fdopen(fd,'w',encoding='utf-8')as stream: + json.dump({'tenant_key':tenant,'verification_token':token},stream) + stream.write('\n') + print('Tenant verified automatically from existing authorization; callback identity prepared privately. No services changed.') + + +if __name__=='__main__': + try:main() + except (ValueError,OSError,EOFError,KeyboardInterrupt,subprocess.TimeoutExpired): + raise SystemExit('STOP: callback identity not verified; current services preserved. Check Work authorization or platform connectivity; do not guess tenant_key.')from None diff --git a/deployable/scripts/broker_deploy.py b/deployable/scripts/broker_deploy.py index 8e1ce57..32e511b 100644 --- a/deployable/scripts/broker_deploy.py +++ b/deployable/scripts/broker_deploy.py @@ -180,14 +180,33 @@ def router_public_key(config): raise DeployError('router connector public key is not a verified read-only trust mount') -def validate_overlay(base, merged, candidate, router_key_required=True): +def targets(plan): + return ('connector','router','longconn')if plan.get('remote_approvals_managed')else('connector','router') + + +def target_tag(plan,name): + return plan['longconn_candidate_image']if name=='longconn'else plan['candidate_image'] + + +def receipt_file(root,name): + return root/('candidate-longconn.json'if name=='longconn'else'candidate.json') + + +def validate_overlay(base, merged, candidate, router_key_required=True, approval_plan=None): old, new = base["services"], merged["services"] require_services(old) require_services(new) - for name in set(old)-{"connector","router"}: + managed=targets(approval_plan or {}) + for name in set(old)-set(managed): if old[name] != new[name]: raise DeployError("overlay changes an unrelated relay service: "+name) - for name in ("connector","router"): + for name in managed: + if name=='longconn': + expected_env=dict(old[name].get('environment',{})) + expected_env.update((approval_plan or {})['longconn_approval_environment']) + expected=dict(old[name]);expected.update(image=approval_plan['longconn_candidate_image'],environment=expected_env) + if new[name]!=expected:raise DeployError('approval overlay changes unrelated longconn settings') + continue for field in set(old[name]) | set(new[name]): if field not in {"environment","image","volumes"} and old[name].get(field) != new[name].get(field): raise DeployError("overlay changes non-migration service settings") @@ -196,7 +215,7 @@ def validate_overlay(base, merged, candidate, router_key_required=True): env = new[name]["environment"] for key, value in old[name].get("environment", {}).items(): if key not in {"HN_CREDENTIAL_MODE","HN_ROUTER_ADMISSION_SECRET","HN_CREDENTIAL_KEYRING_FILE", - "HN_CREDENTIAL_PERMISSION_PROFILES","HN_CONNECTOR_INTERNAL_URL","HN_FEISHU_APP_ID","HN_OAUTH_ENABLED","HN_WORK_FILES_ENABLED","HN_WORK_TASK_CARDS_ENABLED"}: + "HN_CREDENTIAL_PERMISSION_PROFILES","HN_CONNECTOR_INTERNAL_URL","HN_FEISHU_APP_ID","HN_OAUTH_ENABLED","HN_WORK_FILES_ENABLED","HN_WORK_TASK_CARDS_ENABLED","HN_WORK_REMOTE_APPROVALS_ENABLED","HN_FEISHU_TENANT_KEY","HN_FEISHU_VERIFICATION_TOKEN","HN_APPROVAL_RETENTION_ENABLED"}: if env.get(key) != value: raise DeployError("existing environment changed: "+key) old_volumes = {v["target"]:v for v in old[name].get("volumes", [])} @@ -219,6 +238,14 @@ def validate_overlay(base, merged, candidate, router_key_required=True): # namespace rather than silently converting it to production. All modes still # require PostgreSQL and HTTPS/WSS for this existing-server migration. require_platform(env) + approval_keys={'HN_WORK_REMOTE_APPROVALS_ENABLED','HN_FEISHU_TENANT_KEY','HN_FEISHU_VERIFICATION_TOKEN','HN_APPROVAL_RETENTION_ENABLED'} + for name in ('connector','router'): + before=old[name].get('environment',{});after=new[name]['environment'] + if name=='connector'and approval_plan and approval_plan.get('remote_approvals_managed'): + wanted=approval_plan['connector_approval_environment'] + if any(after.get(k)!=v for k,v in wanted.items()):raise DeployError('approval identity mismatch') + elif any(after.get(k)!=before.get(k)for k in approval_keys): + raise DeployError('unreviewed approval environment change') for key in ("HN_ADMIN_SECRET","HN_INTERNAL_SECRET","HN_FEISHU_APP_SECRET"): if env["HN_ROUTER_ADMISSION_SECRET"] == env.get(key): raise DeployError("router admission credential must be independent") @@ -377,6 +404,31 @@ def prepare_update(args): raise DeployError('work files implementation missing from update source') if getattr(args,'enable_task_cards',False)and not (source/'relay/task_cards.py').is_file(): raise DeployError('task cards implementation missing from update source') + enable_approvals=bool(getattr(args,'enable_remote_approvals',False)) + if parent.get('remote_approvals_managed')and not enable_approvals: + raise DeployError('approval parent requires explicit --enable-remote-approvals and callback identity on each update') + approval_identity=None + if enable_approvals: + if any(not (source/'relay'/name).is_file()for name in ('approval_api.py','approval_worker.py','approval_callback.py')): + raise DeployError('remote approval implementation missing') + identity_path=Path(args.callback_identity) + if identity_path.is_symlink()or not identity_path.is_file()or identity_path.stat().st_size>8192: + raise DeployError('invalid private callback identity file') + if os.name!='nt'and identity_path.stat().st_mode&0o077: + raise DeployError('callback identity file must be private (chmod 600)') + approval_identity=json.loads(identity_path.read_text(encoding='utf-8')) + if (not isinstance(approval_identity,dict)or set(approval_identity)!={'tenant_key','verification_token'} + or not isinstance(approval_identity['tenant_key'],str) + or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',approval_identity['tenant_key']) + or not isinstance(approval_identity['verification_token'],str) + or not 8<=len(approval_identity['verification_token'])<=512 + or any(ord(c)<33 or ord(c)>126 for c in approval_identity['verification_token'])): + raise DeployError('invalid callback identity fields; values withheld') + live=run(compose(parent,root/'compose.broker.json')+['config','--format','json'],True) + a=live['services']['connector']['environment'];b=live['services']['longconn']['environment'] + if (a.get('HN_FEISHU_APP_ID')!=b.get('HN_FEISHU_APP_ID')or a.get('HN_INTERNAL_SECRET')!=b.get('HN_INTERNAL_SECRET') + or b.get('HN_FEISHU_EVENT_SOURCE')!='longconn'): + raise DeployError('existing unique longconn does not match connector identity') output=output.resolve() output.mkdir(mode=0o700) os.chmod(output,0o700) @@ -394,6 +446,28 @@ def prepare_update(args): if mount['target']=='/run/hainuo-credentials/keyring.json': mount['source']=str(output/'credential-keyring.json') rollback=json.loads(json.dumps(overlay)) + if enable_approvals: + plan['remote_approvals_managed']=True + plan['longconn_candidate_image']='hainuo-relay:approval-longconn-'+plan_id + # Keep separate runtime ancestry: the connector image may lack lark-oapi. + plan['longconn_base_image']=roles['longconn']['image'] + shared={'HN_WORK_REMOTE_APPROVALS_ENABLED':'true', + 'HN_FEISHU_TENANT_KEY':approval_identity['tenant_key'], + 'HN_FEISHU_VERIFICATION_TOKEN':approval_identity['verification_token']} + # Retention remains disabled for this candidate even if a deployment + # environment contains an unrelated setting. Enabling needs review. + plan['connector_approval_environment']={**shared,'HN_APPROVAL_RETENTION_ENABLED':'false'} + plan['longconn_approval_environment']={**shared,'HN_CREDENTIAL_MODE':'broker'} + overlay['services']['connector']['environment'].update(plan['connector_approval_environment']) + overlay['services']['connector']['environment']['HN_WORK_TASK_CARDS_ENABLED']='true' + plan['work_task_cards_enabled']=True + original_long=live['services']['longconn'] + overlay['services']['longconn']={'image':plan['longconn_candidate_image'], + 'environment':plan['longconn_approval_environment']} + defaults={'HN_WORK_REMOTE_APPROVALS_ENABLED':'false','HN_CREDENTIAL_MODE':'legacy', + 'HN_FEISHU_TENANT_KEY':'','HN_FEISHU_VERIFICATION_TOKEN':''} + rollback['services']['longconn']={'image':roles['longconn']['image'], + 'environment':{k:original_long.get('environment',{}).get(k,defaults[k])for k in plan['longconn_approval_environment']}} if getattr(args,'enable_work_files',False): overlay['services']['connector']['environment']['HN_WORK_FILES_ENABLED']='true' plan['work_files_enabled']=True @@ -474,14 +548,24 @@ def deployment_state(root,plan): state = entry.get("state",{}) status = state.get("status") if state.get("status") in {'created','running','paused','restarting','removing','exited','dead'} else 'unknown' health = state.get("health") if state.get("health") in {'none','starting','healthy','unhealthy'} else 'unknown' - image = 'candidate' if expected and entry['image']==expected else ( + role_expected=expected + if name=='longconn'and plan.get('remote_approvals_managed')and receipt_file(root,name).is_file(): + role_expected=json.loads(receipt_file(root,name).read_text(encoding='utf-8')).get('image') + image = 'candidate' if role_expected and entry['image']==role_expected else ( 'original' if entry['image']==plan['roles'][name]['image'] else 'different') numbers = {k:v for k,v in state.items() if k in {'exit_code','restart_count'} and type(v) is int} print(name+': image='+image+' status='+status+' health='+health+' '+json.dumps(numbers,sort_keys=True)) - if name in {'connector','router'}: + if name in targets(plan): raw = run(['docker','logs','--tail','100',entry['id']],optional=True) if raw is not None: print(' startup signals: '+(', '.join(log_signals(raw)) or 'no recognized error class; raw logs withheld')) + if name=='longconn': + observed=False + for line in raw.splitlines(): + try:record=json.loads(line) + except ValueError:continue + if isinstance(record,dict)and record.get('event')=='platform_connected':observed=True + print(' platform_connected observed: '+('yes'if observed else'not yet; inspect again after startup')) print('STATE inspection only; no files, keys or containers changed. Healthy candidate services need no reactivation; inspect mixed/unhealthy states before further actions.') @@ -504,18 +588,19 @@ def check(root, plan, baseline=True, router_key_required=True): if digest(root/path) != expected: raise DeployError("prepared keyring or overlay changed; do not regenerate credential keys") for name, old in plan["roles"].items(): - current = service(plan["project"],name,include_stopped=not baseline and name in {"connector","router"}) - if (baseline or name not in {"connector","router"}) and current["image"] != old["image"]: + current = service(plan["project"],name,include_stopped=not baseline and name in targets(plan)) + if (baseline or name not in targets(plan)) and current["image"] != old["image"]: raise DeployError("running service image changed since preparation: "+name+"; this plan may already be activated. Run state; do not rebuild or reactivate a healthy candidate") - if not baseline and name in {"connector","router"}: + if not baseline and name in targets(plan): allowed = {old['image']} - if (root/'candidate.json').is_file(): - allowed.add(json.loads((root/'candidate.json').read_text(encoding='utf-8'))['image']) + receipt=receipt_file(root,name) + if receipt.is_file(): + allowed.add(json.loads(receipt.read_text(encoding='utf-8'))['image']) if current['image'] not in allowed: raise DeployError('unreviewed target image; selected parent plan does not match running '+name+'; use the latest successfully activated plan for a new update. No services changed') base = run(compose(plan)+["config","--format","json"],True) merged = run(compose(plan,root/"compose.broker.json")+["config","--format","json"],True) - validate_overlay(base,merged,plan["candidate_image"],router_key_required=router_key_required) + validate_overlay(base,merged,plan["candidate_image"],router_key_required=router_key_required,approval_plan=plan) # Require the current edge's explicit private-path block, before catch-all handling. mounts = base["services"]["edge"].get("volumes",[]) caddy = next((v["source"] for v in mounts if v.get("target") == "/etc/caddy/Caddyfile" and v.get("type") == "bind"),None) @@ -541,8 +626,17 @@ def build(root,plan): image = run(["docker","image","inspect",plan["candidate_image"],"--format","{{.Id}}"]) run(["docker","run","--rm","--network","none","--read-only","--entrypoint","python", plan["candidate_image"],"-c","import relay.credential_api, relay.binding_state, relay.device_lifecycle"]) + long_image=None + if plan.get('remote_approvals_managed'): + tag=plan['longconn_candidate_image'] + if run(['docker','image','inspect',tag],optional=True)is not None: + raise DeployError('longconn candidate already exists; do not overwrite') + run(['docker','build','--pull=false','--network','none','--build-arg', + 'BASE_IMAGE='+plan['longconn_base_image'],'-t',tag,str(root/'build')]) + long_image=run(['docker','image','inspect',tag,'--format','{{.Id}}']) runtime_check(root,plan) write_json(root/"candidate.json",{"image":image,"tag":plan["candidate_image"]}) + if long_image:write_json(root/'candidate-longconn.json',{'image':long_image,'tag':plan['longconn_candidate_image']}) print("BUILT: "+plan["candidate_image"]+" (offline runtime reuse; not activated)") @@ -558,12 +652,17 @@ def runtime_check(root,plan): " s=Settings(role=data['role'])\n" " if data['role']=='connector': CredentialKeyring.from_file(s.credential_keyring_file)\n" " if data['role']=='router': load_public(s.connector_public_key)\n" + " if data['role']=='connector' and s.work_remote_approvals_enabled: import relay.approval_api, relay.approval_worker\n" + " if data['role']=='longconn':\n" + " import lark_oapi as lark\n" + " from lark_oapi.event.callback.model.p2_card_action_trigger import P2CardActionTriggerResponse\n" + " lark.EventDispatcherHandler.builder('','').register_p2_card_action_trigger(lambda e:P2CardActionTriggerResponse({})).build()\n" "except Exception:\n sys.exit(1)\n") - for role in ("connector","router"): + for role in targets(plan): env = {key:str(value) for key,value in merged["services"][role]["environment"].items() if value is not None} command=["docker","run","--rm","-i","--network","none","--read-only","--entrypoint","python", "-v",str(root/"credential-keyring.json")+":/run/hainuo-credentials/keyring.json:ro", - "-v",str(public_source)+":"+public_key+":ro",plan["candidate_image"],"-c",code] + "-v",str(public_source)+":"+public_key+":ro",target_tag(plan,role),"-c",code] print('Runtime check: '+role+' configuration and required key files',flush=True) run(command,input_text=json.dumps({"role":role,"environment":env})) @@ -614,12 +713,17 @@ def activate(root,plan,ready): image = run(["docker","image","inspect",plan["candidate_image"],"--format","{{.Id}}"]) if built != {"image":image,"tag":plan["candidate_image"]}: raise DeployError("candidate image identity changed") + if plan.get('remote_approvals_managed'): + tag=plan['longconn_candidate_image'] + saved=json.loads((root/'candidate-longconn.json').read_text(encoding='utf-8')) + actual=run(['docker','image','inspect',tag,'--format','{{.Id}}']) + if saved!={'image':actual,'tag':tag}:raise DeployError('longconn candidate image identity changed') print('ACTIVATION step: isolated candidate configuration/keyring check (no service switch)',flush=True) runtime_check(root,plan) - print('ACTIVATION step: recreate connector/router; failures may leave a partial switch, inspect state',flush=True) + print('ACTIVATION step: recreate '+','.join(targets(plan))+'; failures may leave a partial switch, inspect state',flush=True) run(compose(plan,root/"compose.broker.json")+["up","-d","--no-deps","--no-build","--pull","never", - "--wait","--wait-timeout","180","connector","router"]) - print("ACTIVATED: connector/router only. Verify real OAuth, v2 client, unbind and task routing.") + "--wait","--wait-timeout","180",*targets(plan)]) + print('ACTIVATED: '+','.join(targets(plan))+' only. Verify real platform connection and task routing.') def container_permission_profiles(container): @@ -720,6 +824,8 @@ def main(): update.add_argument("--output",required=True) update.add_argument('--enable-work-files',action='store_true',help='Opt in after paired-client preparation; rollback retains the previous flag.') update.add_argument('--enable-task-cards',action='store_true',help='Opt in to paired-client status cards; rollback retains the previous flag.') + update.add_argument('--enable-remote-approvals',action='store_true',help='Update the existing longconn too; needs private callback identity file.') + update.add_argument('--callback-identity',help='Private JSON file with tenant_key and verification_token; never paste values into command arguments.') permissions=sub.add_parser('update-permissions') permissions.add_argument('--plan',required=True) permissions.add_argument('--profiles',required=True) @@ -764,7 +870,7 @@ def main(): raise DeployError("rollback requires stopping broker clients; old refresh credentials are not restored") check(root,plan,baseline=False,router_key_required=False) run(compose(plan,root/"compose.rollback.json")+["up","-d","--no-deps","--no-build","--pull","never", - "--wait","--wait-timeout","180","connector","router"]) + "--wait","--wait-timeout","180",*targets(plan)]) print("ROLLED BACK containers only; preserve keyring/database. Client mode and reauthorization must be coordinated.") except (DeployError,OSError,ValueError,KeyError,TypeError): error = sys.exc_info()[1] diff --git a/deployable/tests/test_approval_audit.py b/deployable/tests/test_approval_audit.py new file mode 100644 index 0000000..05e9d7b --- /dev/null +++ b/deployable/tests/test_approval_audit.py @@ -0,0 +1,119 @@ +"""M6 audit and cleanup on temporary encrypted approval stores only.""" +import json +import pytest +from test_credential_api import env +from test_approval_integration import approval_env,pair +from relay.approval_store import ApprovalStore,ApprovalError + + +async def test_audit_is_minimal_and_repeated_decisions_do_not_duplicate(pair): + p=pair;job=p['launch']();row=await p['card']();response,body=await p['click']() + await job + e=p['env'];await e['client'].post('/internal/approval-callback',json=body, + headers={'X-Internal-Secret':e['settings'].internal_secret}) + events=[dict(r)for r in e['store'].db.execute('SELECT * FROM work_approval_audit_v1').fetchall()] + assert {r['event']for r in events}>={'created','card_bound','approved','consumed'} + assert sum(r['event']=='approved'for r in events)==1 + assert sum(r['event']=='consumed'for r in events)==1 + encoded=json.dumps(events) + for secret in ('ou_a','oc_synthetic','om_source','合成测试','fake-access','verification-synthetic',row['ciphertext']): + assert secret not in encoded + assert all(len(r['actor_scope'])==64 and len(r['device_scope'])==64 for r in events) + assert all(r['action_class']=='work_api_write'for r in events) + + +async def test_closed_payload_purge_keeps_replay_tombstone_and_active_payload(pair): + p=pair;job=p['launch']();row=await p['card']();await p['click']('deny') + from test_approval_integration import pc_errors + with pytest.raises(pc_errors.WorkError):await job + e=p['env'];now=__import__('time').time() + with e['store'].db:e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=?',(now-31*86400,)) + api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now) + active=api.create('task-approval','ou_a','a'*32,1,'operation-active',{'kind':'synthetic'},'合成动作','仅测试') + with e['store'].db:e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=? WHERE approval_id=?', + (now-32*86400,active['approval_id'])) + result=api.maintain_retention() + assert result['payloads_purged']==1 and result['tombstones_preserved'] + saved=e['store'].db.execute('SELECT * FROM work_approvals_v3').fetchone() + assert saved['state']=='rejected'and saved['ciphertext']==''and saved['nonce_digest']=='' + assert api.maintain_retention()['payloads_purged']==0 + status=api.status(row['approval_id'],'task-approval','ou_a','a'*32,1,row['operation_digest']) + assert status['state']=='rejected' + assert api.consume(row['approval_id'],'task-approval','ou_a','a'*32,1,row['operation_digest'])['execute']is False + action=json.loads(e['api'].broker.store.keyring.decrypt(row['key_id'],row['ciphertext'],api._aad(row)))['action'] + assert api.create('task-approval','ou_a','a'*32,1,'operation-synthetic',action,'同动作','不得重新放行')['state']=='rejected' + # A pre-existing pending approval is not selected even when timestamp is old. + assert api.maintain_retention()['payloads_purged']==0 + assert e['store'].db.execute('SELECT ciphertext FROM work_approvals_v3 WHERE approval_id=?', + (active['approval_id'],)).fetchone()['ciphertext'] + + +async def test_retention_is_environment_scoped_and_bounded(pair): + p=pair;job=p['launch']();row=await p['card']();await p['click']('deny') + from test_approval_integration import pc_errors + with pytest.raises(pc_errors.WorkError):await job + e=p['env'];now=__import__('time').time() + api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now) + with e['store'].db: + e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=?',(now-100*86400,)) + e['store'].db.execute('UPDATE work_approval_audit_v1 SET created_at=?',(now-100*86400,)) + result=api.maintain_retention(limit=1) + assert result['audit_events_removed']==1 + assert e['store'].db.execute('SELECT COUNT(*) n FROM work_approvals_v3').fetchone()['n']==1 + assert e['store'].db.execute('SELECT COUNT(*) n FROM work_approval_callbacks_v1').fetchone()['n']==1 + for kwargs in ({'payload_days':0},{'audit_days':366},{'payload_days':91,'audit_days':90},{'limit':100}): + with pytest.raises(ApprovalError):api.maintain_retention(**kwargs) + + +async def test_audit_write_failure_rolls_back_consume_and_never_sends(pair,monkeypatch): + from relay.approval_audit import ApprovalAudit + from test_approval_integration import pc_errors + original=ApprovalAudit.record + def fail_consume(self,row,event,action_class=None): + if event=='consumed':raise RuntimeError('synthetic audit storage failure') + return original(self,row,event,action_class) + monkeypatch.setattr(ApprovalAudit,'record',fail_consume) + p=pair;job=p['launch']();await p['click']() + with pytest.raises(pc_errors.WorkError):await job + e=p['env'] + assert not p['calls'] + assert e['store'].db.execute("SELECT state FROM work_approvals_v3").fetchone()['state']!='consumed' + assert not e['store'].db.execute("SELECT * FROM work_approval_audit_v1 WHERE event='consumed'").fetchall() + + +async def test_retention_never_touches_another_environment(pair): + from test_approval_integration import pc_errors + p=pair;job=p['launch']();await p['click']('deny') + with pytest.raises(pc_errors.WorkError):await job + e=p['env'];now=__import__('time').time() + with e['store'].db:e['store'].db.execute("UPDATE work_approvals_v3 SET environment='development',updated_at=?",(now-100*86400,)) + api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now) + assert api.maintain_retention()['payloads_purged']==0 + assert e['store'].db.execute('SELECT ciphertext FROM work_approvals_v3').fetchone()['ciphertext'] + + +def test_retention_configuration_is_explicit_and_ordered(): + from relay.config import Settings + from pydantic import ValidationError + settings=Settings(role='test',environment='test') + assert settings.approval_retention_enabled is False + assert (settings.approval_payload_retention_days,settings.approval_audit_retention_days)==(30,90) + for changes in ({'approval_retention_enabled':True}, + {'approval_payload_retention_days':91,'approval_audit_retention_days':90}, + {'approval_audit_retention_days':366}): + with pytest.raises(ValidationError):Settings(role='test',environment='test',**changes) + + +async def test_consumed_but_result_unknown_is_not_purged(pair): + p=pair;job=p['launch']();await p['click']();await job + e=p['env'];now=__import__('time').time() + api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now) + with e['store'].db: + e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=?',(now-100*86400,)) + e['store'].db.execute("UPDATE work_tasks_v2 SET status='unknown'") + assert api.maintain_retention()['payloads_purged']==0 + assert e['store'].db.execute('SELECT ciphertext FROM work_approvals_v3').fetchone()['ciphertext'] + with e['store'].db:e['store'].db.execute("UPDATE work_tasks_v2 SET status='completed'") + # Bounded keyset scan wraps on the next maintenance pass. + outcomes=[api.maintain_retention()['payloads_purged']for _ in range(2)] + assert sum(outcomes)==1 diff --git a/deployable/tests/test_approval_identity.py b/deployable/tests/test_approval_identity.py new file mode 100644 index 0000000..d4bef70 --- /dev/null +++ b/deployable/tests/test_approval_identity.py @@ -0,0 +1,106 @@ +"""Automatic tenant lookup: real encrypted temp SQL, synthetic user-info API.""" +import importlib.util +import json +import sys +from pathlib import Path +from types import SimpleNamespace + +import httpx +import pytest +from test_credential_api import env,authorize + +spec=importlib.util.spec_from_file_location('approval_identity_test',Path(__file__).resolve().parents[1]/'scripts/approval_identity.py') +identity=importlib.util.module_from_spec(spec);spec.loader.exec_module(identity) + + +@pytest.mark.parametrize('failure',[None,'wrong_user','expired','permission','missing_tenant','redirect']) +async def test_in_container_lookup_is_readonly_and_never_refreshes(env,monkeypatch,capsys,failure): + await authorize(env) + from relay import config + monkeypatch.setattr(config,'Settings',lambda **kw:env['settings']) + credentials=env['api'].broker.store + if failure=='expired': + old=credentials.get('test','cli_test','ou_a') + credentials.save('test','cli_test','ou_a',expected_version=old.version, + access_token='fake-expired-access',refresh_token='fake-refresh-private', + access_expires_at=1,refresh_expires_at=9999999999,scopes=['read']) + before='\n'.join(env['store'].db.iterdump());calls=[] + def platform(request): + calls.append(request) + assert request.method=='GET'and request.url.path=='/open-apis/authen/v1/user_info' + assert request.url.host=='work.example.test' + if failure=='permission':return httpx.Response(403,json={'code':99991679}) + if failure=='redirect':return httpx.Response(302,headers={'Location':'https://unapproved.example.test'}) + data={'open_id':'ou_other'if failure=='wrong_user'else'ou_a', + 'tenant_key':'tenant_verified','name':'synthetic-private-user','email':'synthetic-private@example.test'} + if failure=='missing_tenant':data.pop('tenant_key') + return httpx.Response(200,json={'code':0,'data':data}) + original=httpx.Client + monkeypatch.setattr(httpx,'Client',lambda **kwargs:original(**kwargs,transport=httpx.MockTransport(platform))) + exec(identity.DISCOVERY_CODE,{}) + output=capsys.readouterr().out;result=json.loads(output) + if failure is None: + assert result=={'status':'verified','tenant_key':'tenant_verified','app_id':'cli_test','environment':'test'} + else: + assert result['status']=='tenant_discovery_unavailable' + diagnostic=result['diagnostic'] + if failure=='expired':assert diagnostic['expired_access']==1 + elif failure=='wrong_user':assert diagnostic['user_mismatch']==1 + elif failure=='missing_tenant':assert diagnostic['missing_tenant']==1 + else:assert diagnostic['platform_rejected']==1 + assert len(calls)==(0 if failure=='expired'else 1) + assert '\n'.join(env['store'].db.iterdump())==before + assert env['platform'].refreshes==0 + assert all(secret not in output for secret in ('fake-access','fake-refresh','synthetic-private','email')) + + +@pytest.mark.parametrize('mutate',[{'app_id':'different'},{'environment':'production'}, + {'tenant_key':'invalid tenant'},{'status':'unavailable'},{'extra':'forbidden'}]) +def test_lookup_rejects_mismatched_container_metadata(monkeypatch,mutate): + body={'status':'verified','tenant_key':'tenant_verified','app_id':'cli_test','environment':'test',**mutate} + calls=[] + def run(args,**kwargs):calls.append((args,kwargs));return SimpleNamespace(returncode=0,stdout=json.dumps(body)) + monkeypatch.setattr(identity.subprocess,'run',run) + with pytest.raises(ValueError):identity.discover_tenant('synthetic-connector',{'HN_FEISHU_APP_ID':'cli_test','HN_ENVIRONMENT':'test'}) + assert calls[0][0]==['docker','exec','-i','synthetic-connector','python','-'] + + +def test_prepare_never_asks_user_for_tenant_or_prints_verification_token(tmp_path,monkeypatch,capsys): + output=tmp_path/'callback.json' + env=['HN_FEISHU_APP_ID=cli_test','HN_ENVIRONMENT=test','HN_FEISHU_VERIFICATION_TOKEN=synthetic-private-verify'] + monkeypatch.setattr(sys,'argv', + ['approval_identity.py','--output',str(output)]) + monkeypatch.setattr(identity.subprocess,'run',lambda *a,**kw:SimpleNamespace(returncode=0,stdout=json.dumps(env))) + monkeypatch.setattr(identity,'discover_tenant',lambda *a:'tenant_verified') + def forbidden(*a):raise AssertionError('No interactive identity fields should be requested') + monkeypatch.setattr('builtins.input',forbidden) + monkeypatch.setattr(identity.getpass,'getpass',forbidden) + identity.main() + assert json.loads(output.read_text())=={'tenant_key':'tenant_verified','verification_token':'synthetic-private-verify'} + assert 'synthetic-private-verify'not in capsys.readouterr().out + + +def test_diagnostic_whitelist_drops_secrets_and_untrusted_strings(): + assert identity.safe_diagnostic({'stage':'user_info','http_status':403,'platform_code':99991679, + 'email':'private@example.test','access_token':'private-secret','error_class':'private-secret', + 'matched_bindings':True,'expired_access':-1})=={'stage':'user_info','http_status':403,'platform_code':99991679} + + +def test_diagnose_does_not_prompt_or_write(tmp_path,monkeypatch,capsys): + monkeypatch.setattr(sys,'argv',['approval_identity.py','--diagnose']) + monkeypatch.setattr(identity.subprocess,'run',lambda *a,**kw:SimpleNamespace(returncode=0,stdout='[]')) + monkeypatch.setattr(identity,'discover_tenant',lambda *a:'tenant_verified') + def forbidden(*a):raise AssertionError('Diagnostic must not prompt') + monkeypatch.setattr(identity.getpass,'getpass',forbidden) + identity.main() + assert '"status": "verified"'in capsys.readouterr().out + assert not list(tmp_path.iterdir()) + + +def test_failed_discovery_reports_only_safe_stage_counts(monkeypatch,capsys): + body={'status':'tenant_discovery_unavailable','diagnostic':{'stage':'user_info', + 'expired_access':2,'network_errors':0,'access_token':'do-not-print','name':'do-not-print'}} + monkeypatch.setattr(identity.subprocess,'run',lambda *a,**kw:SimpleNamespace(returncode=0,stdout=json.dumps(body))) + with pytest.raises(ValueError):identity.discover_tenant('connector',{}) + output=capsys.readouterr().out + assert '"expired_access": 2'in output and 'do-not-print'not in output diff --git a/deployable/tests/test_approval_integration.py b/deployable/tests/test_approval_integration.py new file mode 100644 index 0000000..c7884a0 --- /dev/null +++ b/deployable/tests/test_approval_integration.py @@ -0,0 +1,350 @@ +"""Two-repo offline approval integration: real proofs/SQL, synthetic platform only.""" +import asyncio +import importlib +import importlib.util +import json +import os +from pathlib import Path +import secrets +import sys +import threading +import time + +import httpx +import pytest +import pytest_asyncio + +from test_credential_api import env,wire,authorize,Platform +from relay.connector import create_app +from relay.crypto import pem_private +from relay.approval_store import ApprovalStore +from relay.store import Store +from relay.binding_state import BindingState +from relay.approval_worker import ApprovalWorker + + +PC_SERVICE=Path(os.environ.get('HAINUO_APPROVAL_PC_SOURCE',str( + Path(__file__).resolve().parents[3]/'hainuo-build-phase-A-local-20261116')))/'integrations/work-relay/service/relay' +if not (PC_SERVICE/'approval_transport.py').is_file(): + pytest.skip('paired PC source missing; set HAINUO_APPROVAL_PC_SOURCE to run joint acceptance',allow_module_level=True) +# An explicit checkout selection keeps imports independent of cwd and installed +# runtime. Missing paired sources are reported as skipped, never a joint pass. +spec=importlib.util.spec_from_file_location('approval_pc',PC_SERVICE/'__init__.py', + submodule_search_locations=[str(PC_SERVICE)]) +pc=importlib.util.module_from_spec(spec);sys.modules['approval_pc']=pc;spec.loader.exec_module(pc) +pc_broker=importlib.import_module('approval_pc.broker_client') +pc_gate=importlib.import_module('approval_pc.approval_transport') +pc_config=importlib.import_module('approval_pc.release_config') +pc_context=importlib.import_module('approval_pc.task_context') +pc_errors=importlib.import_module('approval_pc.work_errors') + + +class CardPlatform(Platform): + def __init__(self): + super().__init__();self.cards=[];self.updates=[];self.lose_card_receipt=False + + async def reply_task_card(self,source,card,delivery_id,*,token): + self.cards.append((source,card,delivery_id)) + if self.lose_card_receipt:raise ConnectionError('synthetic lost card receipt') + return 'om_approval_'+str(len(self.cards)) + + async def update_task_card(self,message_id,card,*,token): + self.updates.append((message_id,card)) + + +@pytest_asyncio.fixture +async def approval_env(env): + settings=env['settings'].model_copy(update=dict(work_remote_approvals_enabled=True, + feishu_tenant_key='tenant_synthetic',feishu_verification_token='verification-synthetic')) + platform=CardPlatform();app=create_app(settings,env['redis'],platform) + async with app.router.lifespan_context(app): + async with httpx.AsyncClient(transport=httpx.ASGITransport(app),base_url=settings.public_url)as client: + current={**env,'settings':settings,'platform':platform,'app':app,'client':client, + 'store':app.state.store,'api':app.state.oauth} + await authorize(current) + yield current + + +@pytest_asyncio.fixture +async def pair(approval_env,tmp_path,monkeypatch): + e=approval_env;directory=tmp_path/'pc';directory.mkdir() + host='a'*32;device=e['store'].device(host) + (directory/'server.pem').write_bytes(e['settings'].ca_cert.read_bytes()) + (directory/'device.key').write_bytes(pem_private(e['keys'][host])) + (directory/'device.json').write_text(json.dumps({'host_id':host,'owner':'ou_a', + 'binding_generation':device['binding_generation']})) + release=pc_config.validate_config(dict(public_url=e['settings'].public_url,credential_mode='broker', + app_id='cli_test',environment='test',permission_profile='product-v1', + work_api_base='https://work.example.test',remote_approvals=True)) + monkeypatch.setattr(pc_config,'load_config',lambda:release) + monkeypatch.setattr(pc_broker,'load_config',lambda:release) + loop=asyncio.get_running_loop();faults=set();calls=[];trace=[];stop=threading.Event() + def forward(request): + async def dispatch(): + response=await e['client'].request(request.method,request.url.path, + headers=dict(request.headers),content=request.content) + return httpx.Response(response.status_code,headers=response.headers,content=response.content,request=request) + response=asyncio.run_coroutine_threadsafe(dispatch(),loop).result(timeout=10) + trace.append((request.url.path,response.status_code)) + if request.url.path.endswith('/consume')and 'consume_receipt_lost'in faults: + raise httpx.ReadTimeout('synthetic lost consume response') + if request.url.path.endswith('/status')and '/approvals/'in request.url.path and 'poll_lost'in faults: + raise httpx.ConnectError('synthetic offline') + return response + transport=httpx.MockTransport(forward) + actual_transport=pc_gate.ApprovalTransport + monkeypatch.setattr(pc_gate,'ApprovalTransport',lambda d,r:actual_transport(d,r,http_transport=transport)) + def platform(request): + assert request.url.host=='work.example.test' + assert request.headers['authorization'].startswith('Bearer fake-access') + calls.append(request) + if 'business_receipt_lost'in faults:raise httpx.ReadTimeout('synthetic lost business response') + return httpx.Response(200,json={'code':0,'data':{'event':{'event_id':'synthetic-event'}}}) + client=pc_broker.BrokerWorkClient(directory,transport=transport,api_transport=httpx.MockTransport(platform)) + task=e['store'].lifecycle.create_task('task-approval','ou_a',host,device['binding_generation'], + 'synthetic-session',e['app'].state.broker.worker_id,time.time()+600,'oc_synthetic','om_source') + context={k:task[k]for k in ('environment','app_id','owner','host_id','binding_generation', + 'conversation_id','chat_id','source_message_id')} + context.update(task_id=task['turn_id'],channel='work-dm') + marker=pc_context.CURRENT_TASK.set(context);control=pc_context.CURRENT_TASK_STOP.set(stop.is_set) + jobs=[] + def launch(operation='operation-synthetic'): + nonlocal loop + loop=asyncio.get_running_loop() + job=asyncio.create_task(asyncio.to_thread(client.api,'POST', + '/open-apis/calendar/v4/calendars/cal_synthetic/events',{}, + {'summary':'合成测试:不创建真实日程'},operation)) + jobs.append(job);return job + async def card(bound=True): + deadline=time.monotonic()+10 + while time.monotonic()