Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1b3fded4ae | ||
|
|
61db139c72 | ||
|
|
7d2a15eaba | ||
|
|
75c609ded6 | ||
|
|
03d34f1d55 | ||
|
|
3132c07256 | ||
|
|
c80c6a0d27 | ||
|
|
d2881b82b0 | ||
|
|
b497f8604e | ||
|
|
43e7b97c0f | ||
|
|
a33ce02fa6 | ||
|
|
26bb8de927 | ||
|
|
abe2550dd0 | ||
|
|
2e0d5987c0 |
@@ -0,0 +1,60 @@
|
||||
# 审批闭环与异常恢复:本地联调记录
|
||||
|
||||
日期:2026-11-17。范围:客户端feat/phase-A-local-20261116与ECS feat/approval-foundation-local-20261117,均为本地未提交改动。不代表master或已安装产品具备此能力。
|
||||
|
||||
## 结果
|
||||
|
||||
最终针对性测试:162项通过,1项跳过。ECS/两仓联合100项通过、1项跳过;客户端62项通过。联合测试自身25项通过,其余是认证、任务交接、卡片及客户端既有行为回归。
|
||||
|
||||
跳过项:test_postgres_authorization_and_binding_generation,原因是没有提供真实测试PostgreSQL地址。本轮审批账本使用临时SQLite;不声称PostgreSQL审批并发已通过。
|
||||
|
||||
## 已核对行为
|
||||
|
||||
| 场景 | 实测结果 |
|
||||
| --- | --- |
|
||||
| 手机决定的合成回调允许 | 业务MockTransport收到一次请求,审批消费为consumed |
|
||||
| 拒绝 | 未发送业务请求,同操作再次提交不生成第二次执行 |
|
||||
| 重复按钮、重复操作、并发消费 | 仅一次许可可消费,业务请求无重复 |
|
||||
| 消费响应丢失、轮询断线 | 客户端未发送业务请求;重建客户端仍读原操作记录,不重新执行 |
|
||||
| 业务响应丢失 | 原请求计数为1,返回unknown;重建客户端不重放 |
|
||||
| 本机停止、绑定变化、服务端执行终态、过期 | 不发送业务请求 |
|
||||
| 假操作者/应用/租户/Verification Token/时效/会话/卡片/nonce/自报身份 | 未形成批准,原审批保持pending |
|
||||
| 缺设备签名、签名重放、错任务 | 请求被拒绝,不发送业务请求 |
|
||||
| 卡片创建响应不明 | 不重发第二张确认卡 |
|
||||
| 过期卡片更新响应不明 | 只重试更新已知原卡片,删除按钮后不再更新;不新发卡 |
|
||||
| 审批worker及数据库连接重建 | 查询原审批/卡片继续处理,不重新发卡或替换操作ID |
|
||||
| 已核定只读 | 不生成审批卡片,沿正常读取流程 |
|
||||
| 默认关闭 | 没有审批/内部回调路由,不创建审批表,协议不宣称启用 |
|
||||
| 原生run状态投影 | 仅当前任务运行态投影waiting_for_approval;completed不回退 |
|
||||
|
||||
## 方法与边界
|
||||
|
||||
使用真实客户端BrokerWorkClient、设备Ed25519签名、ECS ASGI路由、临时数据库与专用审批worker。Redis使用fakeredis,平台卡片及业务API使用合成服务,不连接正式Work,不创建真实日程、不发邮件。
|
||||
|
||||
客户端新增可注入的http_transport仅供构造测试;网络和模型不能覆盖传输目标。测试中的原生请求补齐loopback来源/Host,模拟Agent补停止回调,没有放宽正式安全检查。
|
||||
|
||||
测试曾因固定轮数的轮询窗口过早读取发送中间态失败,改为单调时钟的实际期限后完整重跑通过。正式“回执不明不重发”规则未修改。
|
||||
|
||||
未覆盖:正式长连接SDK传递真人点击、平台三秒响应上限的现场表现、手机卡片呈现、完整ECS进程/主机重启、正式PostgreSQL和所有工具统一审批。这里的worker重建测试不等于服务器重启验收。
|
||||
|
||||
## 复现与证据
|
||||
|
||||
ECS deployable目录使用现有测试Python执行:
|
||||
|
||||
```powershell
|
||||
python -m pytest tests/test_approval_integration.py tests/test_task_delivery.py tests/test_credential_api.py tests/test_task_cards.py -q --junitxml=build-support/approval-integration-20261117.xml
|
||||
```
|
||||
|
||||
客户端仓库根目录:
|
||||
|
||||
```powershell
|
||||
python -m pytest integrations/work-relay/tests/test_broker_client.py integrations/work-relay/tests/test_task_journal.py integrations/work-relay/tests/test_task_executor_context.py integrations/work-relay/tests/test_broker_task_context.py -q --junitxml=build-support/approval-client-regression-20261117.xml
|
||||
```
|
||||
|
||||
若成对客户端源码不在同级默认目录,通过HAINUO_APPROVAL_PC_SOURCE指定其仓库根目录。缺源码时联合模块明确skip,不能用跳过代替通过。
|
||||
|
||||
两端XML保留具体测试名、结果与耗时;git diff --check通过。本轮没有提交、推送、部署、出包或修改已安装客户端。两端审批开关仍默认关闭。
|
||||
|
||||
## 下一步
|
||||
|
||||
准备独立测试ECS和客户端版本,复用唯一机器人长连接,核对应用回调订阅后做真人允许/拒绝和断线测试。部署需覆盖原longconn及connector,不能照旧只升级connector/router,也不能增加第二个机器人连接。此步骤需要测试环境和平台配置配合;本轮未擅自执行部署。
|
||||
@@ -0,0 +1,41 @@
|
||||
# C2远程确认本地接入记录
|
||||
|
||||
基线d2881b8,分支feat/approval-foundation-local-20261117。此前按用户要求仅开发源码;用户随后明确要求“审批闭环与异常恢复联调”,本轮已开始本地合成联调。不修改线上订阅、不申请权限、不发真实卡片、不部署或推送。条件路由注册与执行闸门开关默认关闭;本地通过不能代替平台真人验收。
|
||||
|
||||
approval_store.py为独立审批账本基础:绑定本人、任务、设备、代次与动作摘要;动作、显示文案与随机按钮nonce采用现有keyring加密。审批有效期30—600秒且不超过任务期限;相同操作不能换动作;仅pending允许决定,approved只能消费一次;拒绝、过期、取消及已消费不再允许执行。已消费回执丢失不自动再执行,后续仍需PC操作账本保护。
|
||||
|
||||
VerifiedDecision仅由approval_callback.py在核对可信SDK内部通道、应用、租户、Verification Token、时效、真实操作者/会话/卡片字段后构造。必须继续经账本核对原任务与nonce,不接受按钮自报身份。
|
||||
|
||||
用户提供的内网文档已读取:[更新消息流卡片按钮](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/group/im-v2/chat_button/update),PUT /open-apis/im/v2/chat_button,使用tenant_access_token,权限im:app_feed_card:write,10次/秒,最多2个按钮,action_type支持webhook及url_page,action_map为字符串字典。这是消息流卡片快捷操作接口,不能直接替代当前消息回复卡片的交互协议,也未为本功能申请这一权限。
|
||||
|
||||
已从同一平台目录继续核对以下协议:
|
||||
|
||||
- [卡片回传交互回调](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukzMukzMukzM/feishu-cards/card-callback-communication):schema 2.0、card.action.trigger;header包含app_id、tenant_key、Verification Token、event_id和微秒create_time;真实操作者来自event.operator.open_id,卡片/会话来自event.context.open_message_id/open_chat_id。event.token仅为更新卡片凭证,不用作身份认证。
|
||||
- [长连接接收回调](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/event-subscription-guide/callback-subscription/step-1-choose-a-subscription-mode/send-callbacks-to-developers-server):官方Python SDK register_p2_card_action_trigger,返回P2CardActionTriggerResponse;响应必须在3秒内完成。复用现有唯一SDK连接,不增加第二个长连接消费者。
|
||||
- [接收并处理回调](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/event-subscription-guide/callback-subscription/receive-and-handle-callbacks):签名为SHA256(timestamp + nonce + Encrypt Key的UTF-8字节 + 原始body);先验签再用SHA256(Encrypt Key)作为AES-256-CBC密钥解密,密文前16字节为IV,PKCS7去填充。
|
||||
|
||||
approval_callback.py提供严格边界:限制64KB、拒绝重复JSON键、应用/租户/Verification Token校验、事件5分钟时效、仅im_message/button。按钮只携带审批引用、随机nonce和选择,不携带本人或设备身份。当前路由只接现有SDK长连接的内部密钥通道;签名加密Webhook验证方法保留为独立实现,没有接入公开Webhook路由或URL challenge。
|
||||
|
||||
审批事件摘要与event_id在同一决定事务中记账;相同回调返回原状态,事件ID换动作/换审批拒绝。确认文案由服务端从实际动作生成,客户端不能用自拟“低风险”文案改变内容。卡片使用plain_text;发送前持久化意图,超时/取消/回执丢失后不自动重发;实际回复卡片ID必须绑定成功才能接受点击。已知卡片可重试PATCH去掉过期按钮,不用新卡代替未知发送结果。
|
||||
|
||||
approval_worker.py使用专用单线程及该线程独享的数据库连接,不共享connector主连接事务。工作槽最多8个,排队50ms、调用等待1.5秒,工作开始前核对排队期限;SQLite busy_timeout 500ms,PostgreSQL lock_timeout 500ms/statement_timeout 800ms。超时不声称事务已回滚,也不批准客户端重试执行。SDK回调使用最多2个独立内部请求worker,等待2.3秒后返回待核对toast;卡片发送/更新不放在回调等待链上。3秒平台时限尚未实测,数据库/网络故障下的预算仍需验收。
|
||||
|
||||
条件路由为POST /work/v2/approvals/create、status、consume、cancel;复用现有设备证书签名、请求正文摘要、90秒proof期限、Redis nonce去重与绑定认证。每次操作同时核对任务ID/本人/设备/代次/动作摘要;任务已终止、失联未知、解绑或超期时不准消费。内部回调为POST /internal/approval-callback,仅接所选longconn来源及内部密钥。创建与回调各自限制body大小。
|
||||
|
||||
客户端对应分支feat/phase-A-local-20261116已在BrokerWorkClient.api的真实平台HTTP发送前接入approval_transport.py。已核定只读沿用读取;需要确认的请求先冻结参数,等待本人决定,消费前登记本地意图;只有收到本次消费成功且动作逐字节相同,任务未停止、绑定未变时才继续原请求。原写操作账本仍保留,重复操作不会因重新审批而重放。同步执行入口不能安全等待、接口404、断网、无停止控制或回执不明均停止,不回退为免确认发送。此闸门覆盖该Work API边界,不等于终端、execute_code等所有工具已完成统一风险接入;OpenCLI仍禁用。
|
||||
|
||||
等待状态通过可信任务作用域在原生run查询中投影为waiting_for_approval,沿原进度协议传至卡片;不覆盖终态。取消/拒绝/过期/已消费卡片删除按钮,consumed仅说明许可已交PC,不冒充业务成功。维护任务只补尚未登记发送意图的卡片、重试已知原卡片更新,每轮各5项;进程内最多32个去重卡片作业。审批审计保留期清理尚未实现。
|
||||
|
||||
配置约束:ECS的HN_WORK_REMOTE_APPROVALS_ENABLED默认false;启用需broker模式、现有longconn、明确的HN_FEISHU_TENANT_KEY和HN_FEISHU_VERIFICATION_TOKEN。客户端可信release.json中remote_approvals默认false。没有修改现有release/Compose/安装实例。后续部署若启用,需要同步更新原唯一longconn容器的源码和配置,不能只更新connector/router,也不能额外启动第二个机器人连接。应用回调订阅仍需在独立验收时核对;本轮没有操作应用后台。
|
||||
|
||||
此前检查仅为AST/diff;本轮增加tests/test_approval_integration.py,成对加载客户端源码,使用真实Ed25519设备签名、ASGI认证路由、Redis合成服务及临时SQLite数据库连接审批worker;平台发卡和业务HTTP均由合成服务替代。测试不是正式Work长连接、真实手机按钮或线上数据库验收。
|
||||
|
||||
已覆盖:允许/拒绝、重复按钮与重复请求、并发决定/消费、未经签名及签名重放、跨任务、假操作者/应用/租户/token/时效/会话/卡片/nonce、停止/本机绑定变化/执行终态/过期、审批消费回执丢失/轮询断线/业务回执丢失后的客户端重建、确认卡创建回执不明不重发、已知过期卡片去按钮更新回执不明可重试、已核定只读免确认、默认关闭无审批路由。客户端既有模拟请求补全真实入口要求的loopback Host/remote,模拟Agent补停止回调,没有放宽正式入口。
|
||||
|
||||
测试XML:deployable/build-support/approval-integration-20261117.xml。成对源码可通过HAINUO_APPROVAL_PC_SOURCE指定客户端仓库根目录;缺少时明确skip,不能将缺少跨仓执行当通过。独立PostgreSQL用例因未提供测试地址仍跳过;完整服务重启、SDK三秒响应、手机UI显示及真实平台回调需后续隔离环境验收。不能据此标记C201—C204全部完成或可上线。
|
||||
|
||||
最终本轮ECS/联合回归100项通过、1项跳过,客户端62项通过(合计162通过、1跳过);详细范围与复现见APPROVAL_ACCEPTANCE_LOCAL_20261117.md。没有提交、推送、部署或出包。
|
||||
|
||||
后续M6本地增量:用户暂停手机现场测试后,明确继续M6开发。本轮补C207最小审计、事务失败不放行、默认关闭的保留/清理策略及独立PostgreSQL竞争用例。组合回归192通过2跳过;清理游标最终调整专项另核。详见M6_AUDIT_RETENTION_20261117.md。本轮代码未部署,不能将新审计能力写成现网已启用。
|
||||
|
||||
后续独立安全复核与真实PG验证:复现并修复同任务换操作ID重复申请、确认目标不明确/被截断两类问题。隔离本机PostgreSQL 17.7下执行4/16连接竞争、决定冲突、真实SQL审计失败回滚及不同ID同意图竞争,临时实例已停止。最终组合205项通过0跳过,取代本轮“PG缺地址”的未验证状态,但不代表ECS现场或手机验收。详见M6_SECURITY_REVIEW_POSTGRES_20261117.md;修复仍是本地未提交源码,未部署。
|
||||
@@ -0,0 +1,77 @@
|
||||
# 新ECS审批候选部署
|
||||
|
||||
目标经用户2026-11-17贴回的docker inspect核对为ecs-fd74,当前镜像hainuo-relay:broker-update-6c8b7ecfc8976fcc。父计划:/home/cnooc/hainuo-broker-work-20261116。Compose基线:/home/cnooc/ecs-relay-source/deployable/compose.yaml和该父计划compose.broker.json。不要使用旧ECS的20261112计划。
|
||||
|
||||
候选来自feat/approval-foundation-local-20261117,未经合并;执行时应使用交付命令中固定的提交。客户端保持原状,之后单独准备配套测试代码。测试包不包含本机密钥或服务端身份文件。
|
||||
|
||||
## 执行前必须具备
|
||||
|
||||
1. 应用事件与回调中支持并配置长连接接收card.action.trigger。复用当前唯一longconn,不能新开第二个连接。
|
||||
2. Verification Token从当前connector本地配置复用,确实缺失才在ECS私密输入。tenant_key不再要求用户填写:脚本在现有connector中以只读数据库连接读取当前应用/环境、未撤销绑定的有效授权,调用既有user_info接口并核对open_id,再提取tenant_key。不会刷新/修改授权、不新增scope、不读取业务正文;令牌不离开容器。没有可核实的有效授权时停止并提示检查Work授权,不允许猜填。
|
||||
3. 当前客户端具备broker v2协议。本轮先升级ECS,手机审批实际请求需要后续客户端开启remote_approvals;该客户端开关仍默认关闭,服务端升级不等于真人审批已可用。
|
||||
|
||||
## 准备与切换
|
||||
|
||||
从当前源码仓库fetch候选后,在新的独立worktree执行,不覆盖/home/cnooc/ecs-relay-source。候选worktree:/home/cnooc/ecs-relay-approvals-20261117;新计划:/home/cnooc/hainuo-broker-approvals-20261117。
|
||||
|
||||
在候选worktree中:
|
||||
|
||||
```bash
|
||||
sudo python3 deployable/scripts/approval_identity.py \
|
||||
--output /home/cnooc/hainuo-approval-callback-20261117.json
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py prepare-update \
|
||||
--plan /home/cnooc/hainuo-broker-work-20261116 \
|
||||
--output /home/cnooc/hainuo-broker-approvals-20261117 \
|
||||
--enable-remote-approvals \
|
||||
--callback-identity /home/cnooc/hainuo-approval-callback-20261117.json
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py build \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py activate \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117 \
|
||||
--client-v2-ready
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py state \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117
|
||||
```
|
||||
|
||||
输出目录或私有身份文件已经存在时停止,不覆盖或删除原计划来重来。构建失败后保留现场,未激活前服务不变。切换失败可能部分生效,先运行state;不要自动重复activate。
|
||||
|
||||
本次仅重建connector/router/原longconn。数据库、edge、signer、绑定、工作材料及既有凭据keyring保留。connector/router复用当前connector候选运行时;longconn使用自己的原镜像作基础,分别构建并保存摘要。两种镜像都不在线下载依赖,候选预检不接网络;任一SDK/密钥/配置预检失败即停止。
|
||||
|
||||
## 状态核对
|
||||
|
||||
connector/router应为candidate、running、healthy;longconn应为candidate、running,platform_connected observed应为yes。longconn无healthcheck时health=none不是独立通过证据。启动租约最多会等待旧租约失效,稍后再次state只读检查,不启动其他消费者。启动信号仅导出固定类别,不直接贴含令牌的原始日志。
|
||||
|
||||
随后进行配套客户端和手机允许/拒绝/超时/重复点击/断线的合成动作验收,不能用真实发送邮件作为第一项。
|
||||
|
||||
## 回滚
|
||||
|
||||
停止正在进行的审批测试/任务后,在同一候选worktree运行:
|
||||
|
||||
```bash
|
||||
sudo python3 deployable/scripts/broker_deploy.py rollback \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117 \
|
||||
--stop-broker-use
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py state \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117
|
||||
```
|
||||
|
||||
--stop-broker-use是操作者已经停止测试的声明,脚本不替你停止电脑。恢复三服务原镜像和父配置,不回滚数据库、不覆盖refresh token、不重新生成身份密钥。新审批表保留,回滚不承诺撤销已发生业务操作;配套客户端须同时关闭remote_approvals。不要运行父计划的rollback,那会回到更早基线。
|
||||
|
||||
## 保密与边界
|
||||
|
||||
身份JSON、plan.json、compose.broker.json含私有信息,只保存在ECS受限目录,勿发聊天或提交Git。tenant_key由脚本自动核实;Verification Token确实缺失时才由本人在平台本地核对,不需要把账号密码交给助手。真实PostgreSQL/SDK回调/手机UI尚待本次环境验收,不能凭本地测试数宣称上线。
|
||||
|
||||
2026-11-17现场修正:c80c6a0的旧准备脚本曾要求用户输入tenant_key,用户未知该运维字段而中止,未生成候选/切换服务。修复后移除tenant输入;12项自动发现测试通过,覆盖只读加密SQL、有效授权用户匹配、过期/403/重定向/缺字段拒绝、应用/环境匹配,以及不输出身份正文或令牌。私有化user_info文档已确认tenant_key字段:[获取用户信息](https://open.w.cnooc.com.cn/document/uAjLw4CM/ukTMukTMukTM/reference/authen-v1/user_info/get)。测试不代表现场已成功取得tenant_key。
|
||||
|
||||
现场3132c07自动发现返回通用失败,尚无证据判定是授权过期或平台网络。新增--diagnose只读入口,不要求--output、不写身份文件、不询问Verification Token、不重启容器。失败仅输出TENANT_DIAGNOSTIC的固定stage、计数、HTTP/平台数字错误码及白名单异常类;不输出数据库地址、账号、token、异常正文或平台用户信息。15项针对性测试通过。
|
||||
|
||||
```bash
|
||||
sudo python3 deployable/scripts/approval_identity.py --diagnose
|
||||
```
|
||||
|
||||
阶段为container_execution说明容器内探测未完成;credentials_query/database_connect指数据库读取阶段;matched_bindings为0表示没有匹配当前应用/环境的未撤销绑定授权;expired_access表示已有access过期但没有尝试refresh;platform_rejected/http_status/platform_code表示平台拒绝;missing_tenant/user_mismatch分别是返回缺字段/用户身份不匹配。只有拿到诊断事实后才决定修复,不因通用失败要求用户解绑、重建身份或猜填租户。
|
||||
@@ -0,0 +1,43 @@
|
||||
# M6审批审计与保留开发记录
|
||||
|
||||
日期:2026-11-17。范围:C207及C201至C204的本地回归。手机现场验收继续暂缓,线上服务仍为之前部署的03d34f1,本次没有部署、推送、出包或修改本机已安装Work组件。
|
||||
|
||||
## 本轮实现
|
||||
|
||||
增加work_approval_audit_v1,记录创建、允许、拒绝、许可消费、过期、取消、卡片绑定、发送结果不明及内容清理。审计写入与对应状态修改使用同一数据库事务;写审计失败即撤回该事务,不下发许可。重复回调/重复消费不会生成第二次决定或消费事件。
|
||||
|
||||
审计字段仅含环境/应用、审批引用、任务/本人/设备的作用域摘要、绑定代次、动作摘要、固定动作类别、事件、状态及记录时间。不保存动作请求正文、卡片文案、账号姓名、原始用户/设备ID、访问/刷新令牌或按钮nonce。摘要是关联标识,不宣称不可重识别。审计中的consumed仅代表执行许可已消费,不代表业务已成功;业务结果仍由原任务和操作记录核对。
|
||||
|
||||
没有新增公开的审计查询、任意SQL或管理员执行接口;无效回调在原校验边界拒绝,本轮未增加未经验证操作者的身份审计。
|
||||
|
||||
## 保留与清理候选
|
||||
|
||||
| 配置 | 候选默认 | 行为 |
|
||||
| --- | --- | --- |
|
||||
| HN_APPROVAL_RETENTION_ENABLED | false | 未评审前不启用后台清理 |
|
||||
| HN_APPROVAL_PAYLOAD_RETENTION_DAYS | 30 | 已关闭审批的加密动作/文案/nonce及对应密钥引用到期后清空 |
|
||||
| HN_APPROVAL_AUDIT_RETENTION_DAYS | 90 | 已清理审批的过期审计事件有限删除 |
|
||||
|
||||
30/90天只是研发候选值,应按团队正式留存要求评审,不作为公司合规期限。要求1 <= 内容保留天数 <= 审计保留天数 <= 365;启用清理还要求审批功能已开启。现有部署配置未被修改,部署脚本也未新增自动开启清理的参数。
|
||||
|
||||
只选择当前应用/环境内rejected、expired、cancelled、consumed审批;pending/approved不清。consumed但原任务仍inflight、unknown或已查不到任务时不清,保留恢复核对材料。使用游标分批扫描,避免待核对任务挡住其他可清项;每批最多20项,有本轮耗时预算,真正变更前再持有本人锁重读状态。
|
||||
|
||||
不删除审批/操作防重放记录,不改状态回pending,不删除callback发送意图/回执,不碰用户原稿、附件或任务目录。清理完成后同操作仍返回原关闭状态,不能重新取得执行许可。清理是数据库字段更新,不承诺底层磁盘备份或历史数据库页物理抹除。
|
||||
|
||||
## 测试与证据
|
||||
|
||||
组合针对性回归192项通过、2项跳过;包括审批闭环、设备认证、卡片、交付、部署和身份发现。跳过的2项均需独立PostgreSQL测试地址:原授权/绑定测试,以及本轮新增的审批并发测试。本轮未连接生产数据库或真实平台业务写接口。
|
||||
|
||||
新增审计/清理用例覆盖:字段最小化、重复决定去重、清理后重放拒绝、活跃审批不清、环境隔离、审计失败消费回滚、配置限制以及业务结果未知的consumed保留。最终游标修订后另跑该专项,结果见build-support/m6-retention-final-20261117.xml;组合记录为build-support/m6-audit-retention-20261117.xml,两批不能重复累计为功能完成数。
|
||||
|
||||
tests/test_approval_postgres.py已准备独立随机schema的真实数据库竞争测试:同审批并发允许/拒绝、四连接竞争消费、重开连接后再次消费及审计决定唯一。仅显式设置HN_TEST_DATABASE_URL才运行,凭据不放命令行/交付说明;只建和清理本次随机测试schema,不使用现网配置。没有测试地址,不能将跳过写作通过。
|
||||
|
||||
## M6仍待完成
|
||||
|
||||
1. 独立安全复核:回调来源、操作者/卡片/nonce、任务与代次,以及“模型改ID”对重复请求的影响。
|
||||
2. 独立PostgreSQL执行新旧竞争用例,测试锁等待/连接重建和事务失败,不只验证SQLite。
|
||||
3. 真实手机允许/拒绝、超时/重复点击、客户端停止/失联后的旧卡片,按用户恢复验收的时间安排。
|
||||
4. 应用SDK三秒回调时限及正式部署回滚;桌面主dashboard候选版本另行核对。
|
||||
5. 审计留存参数评审、受控运维查询与释放策略。审计/清理代码完成不代表C207已验收或整个M6已完成。
|
||||
|
||||
后续更新:用户明确要求安全复核与PostgreSQL验证后,已在独立本机PostgreSQL 17.7补跑原跳过用例,并新增对抗修复;最终205通过0跳过,临时实例已停止。测试与剩余边界以M6_SECURITY_REVIEW_POSTGRES_20261117.md为准。这里记录的192通过2跳过是前一批历史结果,不再作为最新结果。
|
||||
@@ -0,0 +1,60 @@
|
||||
# M6修复测试环境升级
|
||||
|
||||
对象:ecs-fd74。父计划/home/cnooc/hainuo-broker-approvals-20261117已由用户确认三服务candidate、connector/router healthy及longconn platform_connected=yes;最后已部署源码为03d34f1。新计划使用/home/cnooc/hainuo-broker-approvals-m6-fix-20261117,不能改用更早的work计划。
|
||||
|
||||
## 修复内容与证据
|
||||
|
||||
- 同任务同动作意图不能通过换operation_id及自动日历幂等键重新申请;真正不同动作可另行待确认。
|
||||
- 确认卡显示具体资源路径,超长目标拒绝,不静默截断。
|
||||
- 申请/决定/消费/关闭等事件增加最小审计,和状态变更处于同一事务。旧审批的历史事件不伪造补录。
|
||||
- 保留与清理逻辑已实现,但本次候选显式HN_APPROVAL_RETENTION_ENABLED=false,不删除既有审批内容或记录。手机现场测试继续暂缓。
|
||||
|
||||
205项核心组合回归通过,包含本机独立PostgreSQL 17.7的4/16连接竞争、SQL审计报错回滚和不同ID同意图竞争,0跳过。升级脚本修改后71项部署回归通过,包含从已启用审批的父计划再次升级及回滚;两批存在重叠,不能合算为276个不同功能验收。来源记录见M6_SECURITY_REVIEW_POSTGRES_20261117.md。
|
||||
|
||||
2026-11-17准备时本机到10.77.81.160的443请求超时,不能据此判定ECS服务故障;部署由用户在现有ECS shell执行,脚本会先核对实际镜像和父计划。此文档不表示升级已发生。
|
||||
|
||||
## 新候选准备与切换
|
||||
|
||||
先按交付命令fetch测试分支并用固定提交创建独立worktree:/home/cnooc/ecs-relay-approvals-m6-fix-20261117。下面步骤在该worktree内执行:
|
||||
|
||||
```bash
|
||||
sudo python3 deployable/scripts/broker_deploy.py state \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py prepare-update \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-20261117 \
|
||||
--output /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 \
|
||||
--enable-remote-approvals \
|
||||
--callback-identity /home/cnooc/hainuo-approval-callback-20261117.json
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py build \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py activate \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 \
|
||||
--client-v2-ready
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py state \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117
|
||||
```
|
||||
|
||||
复用现有私有callback identity文件,不需再次输入/复制令牌。新目录若已存在停止,不自动删除、覆盖或重建。prepare/build不切换服务,activate仅定向重建原connector/router/longconn。各角色分别复用当前运行镜像作基础,不在线拉依赖,不启动第二个机器人连接。
|
||||
|
||||
保留数据库、材料、绑定和keyring;准备目录包含私有配置,禁止贴回或提交Git。启动会新增审计/保留元数据表,但清理开关为false,不做旧数据清理。首次启动时旧审批后续事件可能标为unknown类别,不据此编造历史created事件。
|
||||
|
||||
## 核对及回滚
|
||||
|
||||
输出应包含connector/router candidate running healthy、longconn candidate running及platform_connected observed=yes。SDK连接信号延迟时再次只读state,不重复activate。失败可能部分切换,先state查看,不自动删除数据或回退数据库。
|
||||
|
||||
回滚前停止进行中的审批任务,然后在本次新worktree执行:
|
||||
|
||||
```bash
|
||||
sudo python3 deployable/scripts/broker_deploy.py rollback \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117 \
|
||||
--stop-broker-use
|
||||
|
||||
sudo python3 deployable/scripts/broker_deploy.py state \
|
||||
--plan /home/cnooc/hainuo-broker-approvals-m6-fix-20261117
|
||||
```
|
||||
|
||||
仅恢复父计划三服务镜像和配置,不回退绑定、OAuth刷新数据或数据库,不移除新增表。此回滚应保留父计划已有审批功能,不回到最早未启用审批的环境。--stop-broker-use是操作者已停止任务的声明,脚本不会代停电脑或撤销已发生业务动作。
|
||||
@@ -0,0 +1,74 @@
|
||||
# M6安全复核与PostgreSQL并发验证
|
||||
|
||||
日期:2026-11-17。对象:feat/approval-foundation-local-20261117工作树中未提交的M6审计与审批增量,基于03d34f1。此处“独立复核”指从功能流程之外按对抗场景重新检查,不是另一位团队验收人的签字或第三方测评。
|
||||
|
||||
## 结论
|
||||
|
||||
复现并修复两类问题,三项初始负向检查均从失败转为通过。最终组合回归205项通过,0失败、0跳过。数据库相关用例在真实PostgreSQL 17.7执行,未再以缺测试地址跳过。
|
||||
|
||||
修改仅在本地源码,未提交/推送、未部署到ECS、未改变已安装客户端或清理开关。手机现场验收继续暂缓。当前已部署03d34f1不包含本轮修复,不能将本报告当作现网升级回执。
|
||||
|
||||
## 复核发现及修复
|
||||
|
||||
| 编号 | 初始可复现行为 | 影响与修复 | 回归结果 |
|
||||
| --- | --- | --- | --- |
|
||||
| R1 | 原审批拒绝后,换operation_id并随之更换客户端自动生成的日历idempotency_key,同一动作可再次申请成功并生成新审批。 | 没有证明能不经批准直接执行,但可绕开原拒绝反复发卡。新增同一可信任务内的动作意图校验,覆盖等待、拒绝、过期及已消费的原申请;更换操作ID不能抹掉原决定。 | pending、rejected、consumed下换ID均拒绝;同ID换参数拒绝;真正不同的新动作仍可待确认;PostgreSQL两连接同动作不同ID竞争只建一条。 |
|
||||
| R2 | 确认卡只显示路径最后一段,两个不同日历都显示events;长资源标识静默截断。 | 用户不能判断真正目标。改为显示完整资源相对路径,保留日历/文件/成员等对象标识。超过300字符的目标直接拒绝,不能截断隐藏差异。 | 两个不同日历文案可区分;过长目标拒绝。 |
|
||||
|
||||
R1保留完整operation_digest用于许可消费时逐字节核对。意图规范化仅去除已经核实为客户端按owner和operation_id生成的日历幂等键;不任意忽略业务参数。身份来源、请求正文、nonce、真正操作者及卡片校验均保留。
|
||||
|
||||
相同动作只能在同一个任务内复用原审批,不允许模型换ID重新发卡;用户明确发起的新任务是新的授权范围。每任务最多50个不同审批意图,超过预算停止。旧加密内容不可校验时拒绝新申请,而不是猜测放行。
|
||||
|
||||
## 复核边界
|
||||
|
||||
- 平台来源:只接所选长连接的内部认证通道;核对应用、租户、Verification Token、事件时效及回调类型。不把action.value中的身份或event.token更新凭据当来源证明。
|
||||
- 决定归属:操作者、私聊、原卡片、随机nonce、本人设备、绑定代次、原task及动作摘要都要匹配。错误应用/租户/用户/卡片、旧任务和旧绑定不能决定。
|
||||
- 单次许可:决定、消费和审计在数据库事务中执行,按本人锁串行化;已关闭审批不能复活。重投只返回已有事实,未知结果不重新执行。
|
||||
- 内容与日志:审计不保存正文或token;动作类别固定。清理默认关闭,不删除活跃或结果未知任务所需内容,保留防重放记录;清理不构成物理擦除承诺。
|
||||
- 恢复:停止控制取实际Agent,许可返回后再核对停止/绑定。许可consumed不宣称业务成功,网络中断不承诺撤销已有外部请求。
|
||||
|
||||
## 真实PostgreSQL验证
|
||||
|
||||
从官方EDB HTTPS地址获取便携PostgreSQL 17.7,解包到D盘测试目录,不安装Windows服务。每轮创建独立数据目录、随机密码、本机随机端口,仅监听127.0.0.1。测试数据库为合成专用hainuo_m6_test,测试表位于随机schema;没有使用任何生产DSN、应用密钥或真实业务记录。
|
||||
|
||||
数据库并发用例用Barrier同时放行独立连接,不是单连接顺序调用:
|
||||
|
||||
1. 两连接竞争允许/拒绝,只有一个最终决定;覆盖自然竞争、预先允许及预先拒绝三种情况。
|
||||
2. 分别4和16连接竞争同一许可;允许时恰好一个execute=true,拒绝时全部false。
|
||||
3. 重开数据库连接后再次消费仍为false,原决定的审计事件保持唯一。
|
||||
4. 用PostgreSQL真实触发器让consumed审计INSERT报错,确认前面的状态UPDATE随同事务回滚,之后移除合成故障再按正常规则竞争。
|
||||
5. 两连接同时以不同操作ID申请同一动作,只建立一个审批,另一个拒绝。
|
||||
6. 原有PostgreSQL OAuth/绑定代次回归真实执行,不再跳过。
|
||||
|
||||
以上为6个参数化审批用例及1个授权/绑定用例;其他198项包含合成平台接口、SQLite应用链路、安全负向、卡片、部署与身份发现回归,不把205项都声称为PostgreSQL端到端业务实测。
|
||||
|
||||
初轮针对性验证12项通过。完整回归曾出现1个旧清理测试失败:它在清空原动作后,仍在同任务中申请新动作,与“无法校验旧意图则拒绝”的新保护冲突。调整为清理前创建活跃审批来验证不误清,未放宽正式校验。最终完整重跑205项全部通过。
|
||||
|
||||
## 证据和复现
|
||||
|
||||
最终XML:build-support/m6-independent-review-pg-final-20261117.xml。包含205项结果及耗时,未通过数与跳过数均为0。前序针对性XML:build-support/m6-postgres-20261117.xml。成对客户端源码仍通过HAINUO_APPROVAL_PC_SOURCE定位,缺少时不能宣称联合通过。
|
||||
|
||||
本地临时数据库的启动/运行/停止由工作区outputs/install-1087b-20261117/run_isolated_pg_tests.py完成;运行回执pg-test-receipt.json不含密码。下载来源与ZIP SHA256在D盘postgres-runtime/runtime-receipt.json,临时数据目录保留供复查,明文临时密码文件已移除。
|
||||
|
||||
部署仓库deployable目录的复现选择器:
|
||||
|
||||
```text
|
||||
tests/test_approval_postgres.py
|
||||
tests/test_approval_security_review.py
|
||||
tests/test_approval_audit.py
|
||||
tests/test_approval_integration.py
|
||||
tests/test_task_delivery.py
|
||||
tests/test_task_cards.py
|
||||
tests/test_credential_api.py
|
||||
tests/test_broker_deploy.py
|
||||
tests/test_approval_identity.py
|
||||
```
|
||||
|
||||
PostgreSQL测试只读取明确指定的HN_TEST_DATABASE_URL。测试schema名称由代码固定前缀加随机十六进制组成,清理仅作用于该schema,不读取现网配置。最终运行后pg_ctl已停止该独立测试实例。
|
||||
|
||||
## 尚不能据此放行的事项
|
||||
|
||||
- 真实手机点击、私有化SDK三秒回调时限,以及ECS ARM64与现场PostgreSQL版本的整链路行为仍未验收;Windows本地PostgreSQL通过不代替这些事实。
|
||||
- 真实服务器在提交中途退出、平台业务回执真实丢失、数据库网络分区等仍需要独立环境故障注入;当前证明是账本事务、并发和合成失联行为。
|
||||
- 当前审批执行闸门覆盖Work API边界,不是任意execute_code/终端工具的沙箱。M2/M8仍需完成所有工具统一风险规则与外部材料指令隔离,不能宣称整个客户端已安全放行。
|
||||
- 团队仍需另一位具备权限的工程师复核补丁并验收;本轮没有合并master或部署修复。M6整体保持待验收。
|
||||
@@ -0,0 +1,21 @@
|
||||
# C 阶段服务端进度
|
||||
|
||||
2026-11-12,基线 2e0d598,分支 feat/remote-interaction-C-20261112。
|
||||
|
||||
首批实现:新增设备签名的 POST /work/v2/tasks/progress,持久记录状态及递增序号;旧 state 接口保持签名兼容;本人、设备、当前绑定、任务期限校验沿用既有授权。旧序号不覆盖新状态,重复状态不重复执行,同序号冲突返回 409,终态不回退。执行完成、结果接收、平台投递分别判断。
|
||||
|
||||
已实现状态卡片:原私聊消息下创建一张 interactive 卡片,PATCH 更新同一消息;5秒合并更新,每次扫描最多发送10次请求,单次扫描限100条最近一天内任务。卡片只含任务编号、执行状态与交付状态,不含正文和工具参数,不含远程批准按钮。任务编号与源消息从服务端账本取得。创建回执未知停止重建;已知消息ID的绝对状态更新失败可在后续周期更新到最新状态。卡片维护独立于结果投递任务,失败保留既有文字/附件交付。
|
||||
|
||||
尚未实现:流式正文、可信按钮回调与远程审批、持久下发队列和取消、完整故障恢复及多设备会话管理。当前状态表只存任务标识、枚举、序号和服务端时间,不存正文或凭据。
|
||||
|
||||
客户端配套分支同名,必须先升级 ECS 再升级客户端。当前未部署、未更改长连接订阅,也未新增平台权限;真实私有化平台卡片兼容性待后续探测和验收。
|
||||
|
||||
默认关闭卡片;测试部署使用 prepare-update --enable-task-cards,将 HN_WORK_TASK_CARDS_ENABLED=true 仅加到候选connector。回滚使用父计划原环境,绑定、密钥与存储不重建。此开关不代表已验证私有化平台支持 interactive 回复和消息 PATCH。旧客户端仍可用原 state 接口,但不能凭结果投递推断执行成功,缺失执行状态时卡片显示待核对。
|
||||
|
||||
## 验证记录
|
||||
|
||||
ECS卡片、签名状态、凭据和部署套件:125 passed,1 skipped(已有可选测试)。客户端任务账本/原生运行/契约:27 passed。新增联合源码测试:1 passed,真实客户端账本及轮询代码通过签名HTTP向ECS上报四次状态,创建一次卡片、更新三次,原生任务仅受理一次。平台响应和原生Agent使用合成适配;没有发送真实Work消息或操作业务数据,不能代替私有化平台真人验收。
|
||||
|
||||
已覆盖:乱序状态、终态回退、身份/解绑、5秒合并、创建回执丢失、更新失败恢复、跨worker占用、创建进程中断、过期任务、卡片故障不阻断文字结果、候选开关及回滚保留父环境。尚需上线验证平台卡片格式兼容、更新权限/限流及真实跨端显示。
|
||||
|
||||
断连超时补充:等待可信任务进度超过request_timeout(默认45秒)时记录独立的client_idle_timeout原因;卡片在后续5秒维护周期变红,显示“客户端失联,任务结果待核对”。硬性任务期限、一般unknown仍单独处理,不冒充客户端断连;已知执行终态不被失联覆盖。原因持久保存,后续经设备签名验证的当前序号进度可清除;不重新下发或执行任务。实际耗时受部署参数、卡片限流及平台可用性影响,45—50秒不是SLA。
|
||||
@@ -0,0 +1,111 @@
|
||||
"""Opt-in device-authenticated approval API, with no business execution route."""
|
||||
import asyncio
|
||||
from fastapi import APIRouter,HTTPException,Request
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import Field
|
||||
|
||||
from .approval_callback import ApprovalCallbackVerifier,decision_toast
|
||||
from .approval_cards import ApprovalCardSender,decision_response_card
|
||||
from .approval_policy import describe_action
|
||||
from .approval_store import ApprovalError
|
||||
from .common import no_cache,require_secret
|
||||
from .task_delivery import TaskRequest
|
||||
|
||||
|
||||
class CreateApproval(TaskRequest):
|
||||
operation_id:str=Field(pattern=r'^[A-Za-z0-9_-]{8,100}$')
|
||||
action:dict=Field(repr=False)
|
||||
ttl:int=Field(default=120,strict=True,ge=30,le=600)
|
||||
|
||||
|
||||
class ApprovalReference(TaskRequest):
|
||||
approval_id:str=Field(pattern=r'^apr_[a-f0-9]{64}$')
|
||||
operation_digest:str=Field(pattern=r'^[a-f0-9]{64}$')
|
||||
|
||||
|
||||
class ApprovalAPI:
|
||||
def __init__(self,auth,worker,platform,spawn):
|
||||
self.auth,self.worker,self.spawn=auth,worker,spawn
|
||||
self.sender=ApprovalCardSender(worker,platform)
|
||||
self.scheduled=set()
|
||||
s=auth.s
|
||||
self.verifier=ApprovalCallbackVerifier(environment=s.environment,app_id=s.feishu_app_id,
|
||||
tenant_key=s.feishu_tenant_key,verification_token=s.feishu_verification_token,
|
||||
internal_secret=s.internal_secret,encrypt_key=s.feishu_encrypt_key)
|
||||
self.router=APIRouter()
|
||||
for name in ('create','status','consume','cancel'):
|
||||
self.router.add_api_route('/work/v2/approvals/'+name,getattr(self,name),methods=['POST'])
|
||||
self.router.add_api_route('/internal/approval-callback',self.callback,methods=['POST'])
|
||||
|
||||
async def _call(self,method,*args):
|
||||
try:return await self.worker.call(method,*args)
|
||||
except ApprovalError:
|
||||
raise HTTPException(409,'approval_unavailable_or_unconfirmed')from None
|
||||
|
||||
def _schedule(self,kind,approval_id):
|
||||
key=(kind,approval_id)
|
||||
if key in self.scheduled or len(self.scheduled)>=32:return
|
||||
self.scheduled.add(key)
|
||||
async def deliver():
|
||||
try:
|
||||
if kind=='send':await self.sender.send(approval_id)
|
||||
else:await self.sender.update_closed(approval_id)
|
||||
finally:self.scheduled.discard(key)
|
||||
self.spawn(deliver())
|
||||
|
||||
async def create(self,body:CreateApproval,request:Request):
|
||||
await self.auth.authenticate(body,request)
|
||||
try:summary,impact=describe_action(body.action)
|
||||
except (ApprovalError,TypeError,ValueError,RecursionError):
|
||||
raise HTTPException(400,'unsupported_approval_action')from None
|
||||
result=await self._call('create',body.task_id,body.binding_owner,body.host_id,
|
||||
body.binding_generation,body.operation_id,body.action,summary,impact,body.ttl)
|
||||
self._schedule('send',result['approval_id'])
|
||||
return no_cache(JSONResponse(result))
|
||||
|
||||
async def _reference(self,method,body,request):
|
||||
await self.auth.authenticate(body,request)
|
||||
args=(body.approval_id,body.task_id,body.binding_owner,body.host_id,body.binding_generation)
|
||||
# Cancel requires the same request digest too, not just visibility of an ID.
|
||||
if method=='cancel':
|
||||
await self._call('status',*args,body.operation_digest)
|
||||
result=await self._call('cancel',*args)
|
||||
else:result=await self._call(method,*args,body.operation_digest)
|
||||
if method in {'consume','cancel'}:self._schedule('close',body.approval_id)
|
||||
return no_cache(JSONResponse(result))
|
||||
|
||||
async def status(self,body:ApprovalReference,request:Request):
|
||||
return await self._reference('status',body,request)
|
||||
|
||||
async def consume(self,body:ApprovalReference,request:Request):
|
||||
return await self._reference('consume',body,request)
|
||||
|
||||
async def cancel(self,body:ApprovalReference,request:Request):
|
||||
return await self._reference('cancel',body,request)
|
||||
|
||||
async def callback(self,request:Request):
|
||||
if self.auth.s.feishu_event_source!='longconn':raise HTTPException(404,'not_found')
|
||||
supplied=request.headers.get('X-Internal-Secret','')
|
||||
require_secret(supplied,self.auth.s.internal_secret)
|
||||
try:
|
||||
raw=await asyncio.wait_for(request.body(),timeout=.2)
|
||||
decision=self.verifier.from_sdk_channel(raw,supplied)
|
||||
result=await self.worker.call('decide',decision)
|
||||
except (ApprovalError,asyncio.TimeoutError):
|
||||
return no_cache(JSONResponse({'toast':{'type':'warning',
|
||||
'content':'确认未完成或结果待核对,请查看原任务状态,勿重复发起操作。'}}))
|
||||
self._schedule('close',decision.approval_id)
|
||||
# The callback response updates the clicked private card immediately;
|
||||
# background PATCH remains only for other views/list state convergence.
|
||||
response=decision_toast(result)
|
||||
if result.get('state')in {'approved','rejected','expired','cancelled','consumed'}:
|
||||
response['card']={'type':'raw','data':decision_response_card(result['state'])}
|
||||
return no_cache(JSONResponse(response))
|
||||
|
||||
async def maintain(self):
|
||||
await self.worker.call('close_unavailable')
|
||||
for approval_id in await self.worker.call('cards_to_send'):self._schedule('send',approval_id)
|
||||
for approval_id in await self.worker.call('cards_to_close'):self._schedule('close',approval_id)
|
||||
if self.auth.s.approval_retention_enabled:
|
||||
await self.worker.call('maintain_retention',self.auth.s.approval_payload_retention_days,
|
||||
self.auth.s.approval_audit_retention_days)
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Minimal approval audit; no action bodies, tokens, names or card text."""
|
||||
import hashlib
|
||||
from .crypto import canonical
|
||||
|
||||
EVENTS={'created','approved','rejected','consumed','expired','cancelled','card_bound',
|
||||
'card_send_unknown','payload_purged'}
|
||||
STATES={'pending','approved','rejected','consumed','expired','cancelled'}
|
||||
ACTION_CLASSES={'work_api_read','work_api_write','work_api_delete','unknown'}
|
||||
|
||||
|
||||
class ApprovalAudit:
|
||||
def __init__(self,db,clock):
|
||||
self.db,self.clock=db,clock
|
||||
with db:
|
||||
db.execute('''CREATE TABLE IF NOT EXISTS work_approval_audit_v1(
|
||||
event_key TEXT PRIMARY KEY,environment TEXT NOT NULL,app_id TEXT NOT NULL,
|
||||
approval_id TEXT NOT NULL,task_scope TEXT NOT NULL,actor_scope TEXT NOT NULL,
|
||||
device_scope TEXT NOT NULL,binding_generation BIGINT NOT NULL,
|
||||
operation_digest TEXT NOT NULL,event TEXT NOT NULL,state TEXT NOT NULL,action_class TEXT NOT NULL,
|
||||
created_at DOUBLE PRECISION NOT NULL)''')
|
||||
db.execute('''CREATE INDEX IF NOT EXISTS work_approval_audit_time_v1
|
||||
ON work_approval_audit_v1(environment,app_id,created_at)''')
|
||||
|
||||
def record(self,row,event,action_class=None):
|
||||
# Caller owns the approval transaction and owner lock. Failure rolls
|
||||
# back the associated decision/consume instead of losing its audit.
|
||||
if event not in EVENTS or row['state']not in STATES:raise ValueError('invalid_approval_audit')
|
||||
scope=[row['environment'],row['app_id']]
|
||||
if action_class is None:
|
||||
created=self.db.execute('''SELECT action_class FROM work_approval_audit_v1
|
||||
WHERE environment=? AND app_id=? AND approval_id=? AND event='created' ''',
|
||||
(*scope,row['approval_id'])).fetchone()
|
||||
action_class=created['action_class']if created else'unknown'
|
||||
if action_class not in ACTION_CLASSES:raise ValueError('invalid_approval_audit_class')
|
||||
def digest(kind,value):return hashlib.sha256(canonical([*scope,kind,value])).hexdigest()
|
||||
key=digest('audit',[row['approval_id'],event])
|
||||
self.db.execute('''INSERT INTO work_approval_audit_v1 VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
ON CONFLICT DO NOTHING''',(key,*scope,row['approval_id'],digest('task',row['task_id']),
|
||||
digest('actor',row['owner']),digest('device',row['host_id']),row['binding_generation'],
|
||||
row['operation_digest'],event,row['state'],action_class,self.clock()))
|
||||
@@ -0,0 +1,153 @@
|
||||
"""C2 callback verifier; HTTP registration is opt-in in approval_api.
|
||||
|
||||
Only the SDK worker's authenticated internal channel or a signed platform
|
||||
webhook may enter here. action.value is a reference, never actor identity.
|
||||
"""
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
|
||||
from cryptography.hazmat.primitives import padding
|
||||
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
||||
|
||||
from .approval_store import ApprovalError, VerifiedDecision
|
||||
from .crypto import canonical
|
||||
|
||||
|
||||
class CallbackError(ApprovalError):
|
||||
pass
|
||||
|
||||
|
||||
def _pairs(pairs):
|
||||
result = {}
|
||||
for key, value in pairs:
|
||||
if key in result:
|
||||
raise CallbackError('ambiguous_callback')
|
||||
result[key] = value
|
||||
return result
|
||||
|
||||
|
||||
def _json(raw):
|
||||
if not isinstance(raw, bytes) or not 0 < len(raw) <= 65536:
|
||||
raise CallbackError('invalid_callback_budget')
|
||||
try:
|
||||
data = json.loads(raw, object_pairs_hook=_pairs,
|
||||
parse_constant=lambda _: (_ for _ in ()).throw(ValueError()))
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError()
|
||||
return data
|
||||
except (ValueError, RecursionError, UnicodeError):
|
||||
raise CallbackError('invalid_callback_json') from None
|
||||
|
||||
|
||||
def _text(value, pattern=r'[A-Za-z0-9_-]{1,128}'):
|
||||
if not isinstance(value, str) or not re.fullmatch(pattern, value):
|
||||
raise CallbackError('invalid_callback_field')
|
||||
return value
|
||||
|
||||
|
||||
def _equal_secret(actual, expected):
|
||||
return (isinstance(actual, str) and isinstance(expected, str) and bool(expected)
|
||||
and hmac.compare_digest(actual.encode(), expected.encode()))
|
||||
|
||||
|
||||
class ApprovalCallbackVerifier:
|
||||
def __init__(self, *, environment, app_id, tenant_key, verification_token,
|
||||
internal_secret='', encrypt_key='', clock=time.time):
|
||||
self.environment = _text(environment)
|
||||
self.app_id = _text(app_id)
|
||||
self.tenant_key = _text(tenant_key)
|
||||
if not isinstance(verification_token, str) or not verification_token:
|
||||
raise CallbackError('callback_verifier_not_configured')
|
||||
self.verification_token = verification_token
|
||||
self.internal_secret = internal_secret
|
||||
self.encrypt_key = encrypt_key
|
||||
self.clock = clock
|
||||
|
||||
def from_sdk_channel(self, raw, supplied_secret):
|
||||
"""Future internal handler only; never expose as an unauthenticated API.
|
||||
|
||||
Caller must enforce selected event source, a bounded body and the
|
||||
private network boundary. The secret is not read from callback JSON.
|
||||
"""
|
||||
if len(self.internal_secret) < 32 or not _equal_secret(supplied_secret, self.internal_secret):
|
||||
raise CallbackError('unauthenticated_callback_channel')
|
||||
return self._decision(_json(raw))
|
||||
|
||||
def from_webhook(self, raw, *, timestamp, nonce, signature):
|
||||
"""Verify the original bytes before decrypting. No token-only fallback."""
|
||||
if not self.encrypt_key:
|
||||
raise CallbackError('callback_verifier_not_configured')
|
||||
_text(timestamp, r'[0-9]{1,12}')
|
||||
_text(nonce)
|
||||
_text(signature, r'[a-f0-9]{64}')
|
||||
if abs(self.clock() - int(timestamp)) > 300:
|
||||
raise CallbackError('stale_callback')
|
||||
if not isinstance(raw, bytes) or not 0 < len(raw) <= 65536:
|
||||
raise CallbackError('invalid_callback_budget')
|
||||
expected = hashlib.sha256((timestamp + nonce + self.encrypt_key).encode() + raw).hexdigest()
|
||||
if not hmac.compare_digest(expected, signature):
|
||||
raise CallbackError('unauthenticated_callback_channel')
|
||||
envelope = _json(raw)
|
||||
try:
|
||||
if set(envelope) != {'encrypt'} or not isinstance(envelope['encrypt'], str):
|
||||
raise ValueError()
|
||||
ciphertext = base64.b64decode(envelope['encrypt'], validate=True)
|
||||
if len(ciphertext) < 32 or len(ciphertext) % 16:
|
||||
raise ValueError()
|
||||
decryptor = Cipher(algorithms.AES(hashlib.sha256(self.encrypt_key.encode()).digest()),
|
||||
modes.CBC(ciphertext[:16])).decryptor()
|
||||
padded = decryptor.update(ciphertext[16:]) + decryptor.finalize()
|
||||
unpadder = padding.PKCS7(128).unpadder()
|
||||
plaintext = unpadder.update(padded) + unpadder.finalize()
|
||||
except (ValueError, TypeError):
|
||||
raise CallbackError('invalid_callback_ciphertext') from None
|
||||
return self._decision(_json(plaintext))
|
||||
|
||||
def _decision(self, data):
|
||||
header, event = data.get('header'), data.get('event')
|
||||
if data.get('schema') != '2.0' or not isinstance(header, dict) or not isinstance(event, dict):
|
||||
raise CallbackError('unsupported_callback')
|
||||
if (header.get('event_type') != 'card.action.trigger'
|
||||
or header.get('app_id') != self.app_id
|
||||
or header.get('tenant_key') != self.tenant_key
|
||||
or not _equal_secret(header.get('token'), self.verification_token)):
|
||||
raise CallbackError('callback_application_conflict')
|
||||
event_id = _text(header.get('event_id'))
|
||||
created = _text(header.get('create_time'), r'[0-9]{1,20}')
|
||||
if abs(self.clock() - int(created) / 1000000) > 300:
|
||||
raise CallbackError('stale_callback')
|
||||
operator, context, action = event.get('operator'), event.get('context'), event.get('action')
|
||||
if not all(isinstance(item, dict) for item in (operator, context, action)):
|
||||
raise CallbackError('invalid_callback_context')
|
||||
if (event.get('host') != 'im_message' or action.get('tag') != 'button'
|
||||
or operator.get('tenant_key') != self.tenant_key):
|
||||
raise CallbackError('unsupported_callback_context')
|
||||
value = action.get('value')
|
||||
if isinstance(value, str):
|
||||
value = _json(value.encode())
|
||||
if not isinstance(value, dict) or set(value) != {'kind', 'approval_id', 'nonce', 'choice'}:
|
||||
raise CallbackError('invalid_approval_reference')
|
||||
if value['kind'] != 'hainuo.approval.v1' or value['choice'] not in ('allow', 'deny'):
|
||||
raise CallbackError('invalid_approval_choice')
|
||||
fields = dict(environment=self.environment, app_id=self.app_id,
|
||||
actor_open_id=_text(operator.get('open_id'), r'ou_[A-Za-z0-9_-]{1,120}'),
|
||||
chat_id=_text(context.get('open_chat_id'), r'oc_[A-Za-z0-9_-]{1,120}'),
|
||||
card_message_id=_text(context.get('open_message_id'), r'om_[A-Za-z0-9_-]{1,120}'),
|
||||
approval_id=_text(value['approval_id'], r'apr_[a-f0-9]{64}'),
|
||||
nonce=_text(value['nonce'], r'[A-Za-z0-9_-]{43}'), choice=value['choice'])
|
||||
# No token, raw callback or user text is persisted in the receipt.
|
||||
digest = hashlib.sha256(canonical(fields)).hexdigest()
|
||||
return VerifiedDecision(**fields, event_id=event_id, event_digest=digest)
|
||||
|
||||
|
||||
def decision_toast(status):
|
||||
"""Bounded platform response; never starts a task or executes an action."""
|
||||
state = status.get('state') if isinstance(status, dict) else None
|
||||
messages = {'approved': '已允许本次操作,等待客户端继续。',
|
||||
'rejected': '已拒绝本次操作。', 'expired': '确认已过期,请重新发起。',
|
||||
'consumed': '本次确认已使用,请勿重复操作。', 'cancelled': '本次操作已取消。'}
|
||||
return {'toast': {'type': 'info', 'content': messages.get(state, '确认未完成,请核对任务状态。')}}
|
||||
@@ -0,0 +1,90 @@
|
||||
"""C2 approval presentation and delivery through the dedicated DB worker."""
|
||||
import asyncio
|
||||
import hashlib
|
||||
import math
|
||||
import re
|
||||
import time
|
||||
|
||||
from .approval_store import ApprovalError, _public_text
|
||||
|
||||
|
||||
def approval_card(material, clock=time.time):
|
||||
reference, nonce = material.get('approval_id'), material.get('nonce')
|
||||
if (not isinstance(reference, str) or not re.fullmatch(r'apr_[a-f0-9]{64}', reference)
|
||||
or not isinstance(nonce, str) or not re.fullmatch(r'[A-Za-z0-9_-]{43}', nonce)):
|
||||
raise ApprovalError('invalid_approval_reference')
|
||||
summary = _public_text(material.get('summary'), 200)
|
||||
impact = _public_text(material.get('impact'), 500)
|
||||
expiry = material.get('expires_at')
|
||||
if type(expiry) not in (int, float) or not math.isfinite(expiry) or expiry <= clock():
|
||||
raise ApprovalError('approval_task_expired')
|
||||
seconds = min(600, max(1, math.ceil(expiry - clock())))
|
||||
buttons = []
|
||||
for choice, label, style in (('allow', '允许本次', 'primary'), ('deny', '拒绝', 'default')):
|
||||
buttons.append({'tag': 'button', 'text': {'tag': 'plain_text', 'content': label},
|
||||
'type': style, 'value': {'kind': 'hainuo.approval.v1', 'approval_id': reference,
|
||||
'nonce': nonce, 'choice': choice}})
|
||||
return {'config': {'wide_screen_mode': True, 'enable_forward': False, 'update_multi': True},
|
||||
'header': {'template': 'orange', 'title': {'tag': 'plain_text', 'content': '海诺操作确认'}},
|
||||
'elements': [
|
||||
{'tag': 'div', 'text': {'tag': 'plain_text', 'content': '准备做什么:' + summary}},
|
||||
{'tag': 'div', 'text': {'tag': 'plain_text', 'content': '可能的影响:' + impact}},
|
||||
{'tag': 'div', 'text': {'tag': 'plain_text', 'content': '仅允许本次操作;约' + str(seconds) + '秒后失效。'}},
|
||||
{'tag': 'action', 'actions': buttons}]}
|
||||
|
||||
|
||||
class ApprovalCardSender:
|
||||
def __init__(self, approvals, platform):
|
||||
self.approvals, self.platform = approvals, platform
|
||||
|
||||
async def send(self, approval_id):
|
||||
material = await self.approvals.call('card_material',approval_id)
|
||||
# Prepare before claiming: invalid presentation must not consume an intent.
|
||||
card = approval_card(material, self.approvals.clock)
|
||||
token = await self.platform.tenant_token()
|
||||
if not await self.approvals.call('claim_card',approval_id):
|
||||
return {'state': 'already_sent_or_unconfirmed'}
|
||||
delivery_id = hashlib.sha256(('approval-card-v1\0' + approval_id).encode()).hexdigest()[:32]
|
||||
try:
|
||||
# Receipt is the actual reply card ID, never the source user message.
|
||||
message_id = await asyncio.wait_for(self.platform.reply_task_card(
|
||||
material['source_message_id'], card, delivery_id, token=token), timeout=10)
|
||||
await self.approvals.call('bind_card',approval_id, message_id)
|
||||
except BaseException:
|
||||
# Cancellation/timeout can occur after platform acceptance. Never
|
||||
# infer "not sent" and automatically issue another confirmation.
|
||||
await self.approvals.call('card_send_unknown',approval_id)
|
||||
raise
|
||||
return {'state': 'sent'}
|
||||
|
||||
async def update_closed(self,approval_id,refresh=False):
|
||||
receipt=await self.approvals.call('closed_card_receipt',approval_id,refresh)
|
||||
if not receipt:return {'state':'no_closed_card'}
|
||||
card=closed_approval_card(receipt['state'])
|
||||
token=await self.platform.tenant_token()
|
||||
# Updating the known original message is retryable; sending a new
|
||||
# confirmation card after a lost create receipt is not.
|
||||
await asyncio.wait_for(self.platform.update_task_card(
|
||||
receipt['message_id'],card,token=token),timeout=5)
|
||||
await self.approvals.call('mark_card_closed',approval_id,receipt['state'])
|
||||
return {'state':'updated'}
|
||||
|
||||
|
||||
def closed_approval_card(state):
|
||||
messages={'rejected':'你已拒绝本次操作。',
|
||||
'expired':'本次确认已过期,未获得新的执行许可;需要时请重新发起任务。',
|
||||
'cancelled':'本次确认已取消,不能再通过这张卡片批准。',
|
||||
'consumed':'本次许可已交给客户端;是否执行及执行结果请查看任务进度。'}
|
||||
if state not in messages:raise ApprovalError('approval_not_closed')
|
||||
return {'config':{'wide_screen_mode':True,'enable_forward':False,'update_multi':True},
|
||||
'header':{'template':'grey','title':{'tag':'plain_text','content':'海诺操作确认已结束'}},
|
||||
'elements':[{'tag':'div','text':{'tag':'plain_text','content':messages[state]}}]}
|
||||
|
||||
|
||||
def decision_response_card(state):
|
||||
if state=='approved':
|
||||
return {'config':{'wide_screen_mode':True,'enable_forward':False,'update_multi':True},
|
||||
'header':{'template':'blue','title':{'tag':'plain_text','content':'海诺操作已允许'}},
|
||||
'elements':[{'tag':'div','text':{'tag':'plain_text',
|
||||
'content':'你已允许本次操作,等待客户端继续。执行结果请查看任务进度。'}}]}
|
||||
return closed_approval_card(state)
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Describe exactly the request being approved; never trust model risk prose."""
|
||||
import json
|
||||
import re
|
||||
from .approval_store import ApprovalError
|
||||
|
||||
|
||||
def describe_action(action):
|
||||
if (not isinstance(action,dict)or set(action)!={'kind','method','path','params','body'}
|
||||
or action['kind']!='work-api-v1' or action['method']not in ('GET','POST','PUT','PATCH','DELETE')
|
||||
or not isinstance(action['path'],str)or len(action['path'])>1500
|
||||
or not re.fullmatch(r'/open-apis/[A-Za-z0-9_./@!:$-]+',action['path'])
|
||||
or '..'in action['path']or '//'in action['path']
|
||||
or not isinstance(action['params'],dict)or not isinstance(action['body'],dict)):
|
||||
raise ApprovalError('unsupported_approval_action')
|
||||
encoded=json.dumps(action,sort_keys=True,ensure_ascii=True,separators=(',',':'),allow_nan=False).encode()
|
||||
if len(encoded)>32000:raise ApprovalError('operation_budget_exceeded')
|
||||
parts=action['path'].split('/');namespace=parts[2]
|
||||
targets={'calendar':'日历','contact':'通讯录','search':'搜索','drive':'云盘',
|
||||
'docx':'文档','wiki':'知识库','bitable':'多维表格','task':'任务','sheets':'电子表格','vc':'会议'}
|
||||
if namespace not in targets:raise ApprovalError('unsupported_approval_action')
|
||||
method=action['method'];label=targets[namespace]
|
||||
verb={'GET':'查询','POST':'提交','PUT':'更新','PATCH':'修改','DELETE':'删除'}[method]
|
||||
subject=next((action['body'][key].strip()[:80]for key in ('summary','title','name')
|
||||
if isinstance(action['body'].get(key),str)and action['body'][key].strip()),'')
|
||||
if any(ord(char)<32 for char in subject):subject=''
|
||||
summary=verb+label+'中的指定数据'+(':'+subject if subject else'')
|
||||
resource='/'.join(parts[4:])
|
||||
if len(resource)>300:raise ApprovalError('approval_target_display_budget_exceeded')
|
||||
impact=('本次是查询,但该接口尚未列入自动放行范围。'if method=='GET'else
|
||||
'可能修改平台数据或触发通知;已经发生的操作不能保证自动撤销。')
|
||||
if method=='DELETE':impact='会删除指定数据或移除关联,可能影响其他人,且不一定能够恢复。'
|
||||
if re.search(r'/(permissions|members|collaborators|share)(/|$)',action['path']):
|
||||
impact='可能改变其他人的访问权限或成员关系,请核对授权范围。'
|
||||
return summary,impact+'\n目标范围:'+label+';目标标识:'+resource
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Docker-console repair of known CLOSED cards only. No public route or send."""
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
from .config import Settings
|
||||
from .credential_store import CredentialKeyring
|
||||
from .approval_worker import ApprovalWorker
|
||||
from .approval_cards import ApprovalCardSender
|
||||
from .platform import Feishu
|
||||
|
||||
|
||||
async def repair(task,apply=False):
|
||||
s=Settings(role='connector')
|
||||
if not s.work_remote_approvals_enabled:raise ValueError('approvals_disabled')
|
||||
worker=ApprovalWorker(s,CredentialKeyring.from_file(s.credential_keyring_file))
|
||||
await worker.start();platform=Feishu(s)
|
||||
try:
|
||||
# Dedicated worker serializes SQL. This lookup is constrained to a known
|
||||
# task plus fixed app/environment; no user SQL or arbitrary message ID.
|
||||
ids=await worker.call('closed_for_task',task)
|
||||
if not apply:return {'state':'preview','closed_cards':len(ids)}
|
||||
sender=ApprovalCardSender(worker,platform);updated=0
|
||||
for approval_id in ids:
|
||||
result=await sender.update_closed(approval_id,refresh=True)
|
||||
updated+=result['state']=='updated'
|
||||
return {'state':'updated','closed_cards':len(ids),'updated':updated,'new_cards_sent':0}
|
||||
finally:
|
||||
await platform.close();await worker.close()
|
||||
|
||||
|
||||
def main():
|
||||
parser=argparse.ArgumentParser();parser.add_argument('--task',required=True);parser.add_argument('--apply',action='store_true')
|
||||
args=parser.parse_args()
|
||||
if not re.fullmatch(r'msg_[a-f0-9]{64}',args.task):raise SystemExit('invalid task reference')
|
||||
try:print(json.dumps(asyncio.run(repair(args.task,args.apply))))
|
||||
except Exception as error:
|
||||
print(json.dumps({'state':'repair_unconfirmed','error_class':type(error).__name__}))
|
||||
raise SystemExit(1)from None
|
||||
|
||||
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,394 @@
|
||||
"""Offline foundation for C2: task-scoped approvals with one-time consumption.
|
||||
|
||||
No HTTP routes, event subscription, platform message or execution adapter is
|
||||
registered by this module. VerifiedDecision must only be constructed by a
|
||||
platform callback verifier; a browser/button payload alone is not evidence.
|
||||
"""
|
||||
from dataclasses import dataclass
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import re
|
||||
import secrets
|
||||
import time
|
||||
from .crypto import canonical
|
||||
from .approval_audit import ApprovalAudit
|
||||
|
||||
|
||||
class ApprovalError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerifiedDecision:
|
||||
environment: str
|
||||
app_id: str
|
||||
actor_open_id: str
|
||||
chat_id: str
|
||||
card_message_id: str
|
||||
approval_id: str
|
||||
nonce: str
|
||||
choice: str
|
||||
event_id: str
|
||||
event_digest: str
|
||||
|
||||
|
||||
def _public_text(value,limit):
|
||||
if not isinstance(value,str)or not value.strip()or len(value)>limit or any(ord(c)<32 and c!='\n'for c in value):
|
||||
raise ApprovalError('invalid_approval_text')
|
||||
return value
|
||||
|
||||
|
||||
class ApprovalStore:
|
||||
def __init__(self,state,keyring,clock=time.time,enforce_execution_state=False):
|
||||
self.state,self.db,self.keyring,self.clock=state,state.db,keyring,clock
|
||||
self.enforce_execution_state=enforce_execution_state
|
||||
self._sweep_after=''
|
||||
self._retention_after=''
|
||||
with self.db:
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_approvals_v3(
|
||||
approval_id TEXT PRIMARY KEY,environment TEXT NOT NULL,app_id TEXT NOT NULL,
|
||||
task_id TEXT NOT NULL,owner TEXT NOT NULL,host_id TEXT NOT NULL,
|
||||
binding_generation BIGINT NOT NULL,operation_id TEXT NOT NULL,
|
||||
operation_digest TEXT NOT NULL,nonce_digest TEXT NOT NULL,
|
||||
key_id TEXT NOT NULL,ciphertext TEXT NOT NULL,card_message_id TEXT NOT NULL,
|
||||
state TEXT NOT NULL,expires_at DOUBLE PRECISION NOT NULL,
|
||||
updated_at DOUBLE PRECISION NOT NULL,
|
||||
UNIQUE(environment,app_id,task_id,operation_id))''')
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_approval_callbacks_v1(
|
||||
environment TEXT NOT NULL,app_id TEXT NOT NULL,event_id TEXT NOT NULL,
|
||||
approval_id TEXT NOT NULL,event_digest TEXT NOT NULL,
|
||||
received_at DOUBLE PRECISION NOT NULL,
|
||||
PRIMARY KEY(environment,app_id,event_id))''')
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_approval_card_outbox_v1(
|
||||
approval_id TEXT PRIMARY KEY,state TEXT NOT NULL,
|
||||
updated_at DOUBLE PRECISION NOT NULL)''')
|
||||
self.db.execute('''CREATE INDEX IF NOT EXISTS work_approvals_expiry_v3
|
||||
ON work_approvals_v3(environment,app_id,state,expires_at)''')
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_approval_retention_v1(
|
||||
approval_id TEXT PRIMARY KEY,purged_at DOUBLE PRECISION NOT NULL)''')
|
||||
self.audit=ApprovalAudit(self.db,self.clock)
|
||||
|
||||
def _task(self,task_id,owner,host,generation):
|
||||
if type(generation)is not int or generation<1:
|
||||
raise ApprovalError('invalid_binding_generation')
|
||||
task=self.state.task(task_id)
|
||||
if (not self.state.task_current(task)or not task['chat_id']or not task['source_message_id']or
|
||||
(task['owner'],task['host_id'],task['binding_generation'])!=(owner,host,generation)or
|
||||
task['status']!='inflight'or self._execution_closed(task_id)):
|
||||
raise ApprovalError('approval_task_unavailable')
|
||||
return task
|
||||
|
||||
def _execution_closed(self,task_id):
|
||||
if not self.enforce_execution_state:return False
|
||||
row=self.db.execute('SELECT execution_state FROM work_execution_state_v3 WHERE task_id=?',(task_id,)).fetchone()
|
||||
return bool(row and row['execution_state']in {'completed','failed','cancelled','interrupted','unknown'})
|
||||
|
||||
def _aad(self,row):
|
||||
return canonical(['work-approval-v3',*[row[k]for k in ('environment','app_id','task_id',
|
||||
'owner','host_id','binding_generation','operation_id','operation_digest','approval_id')]])
|
||||
|
||||
@staticmethod
|
||||
def _intent(action,owner,operation_id):
|
||||
normalized=json.loads(json.dumps(action,sort_keys=True,ensure_ascii=True,allow_nan=False))
|
||||
# The calendar key is generated by our PC boundary from owner+op ID;
|
||||
# changing that generated key must not turn a denied action into a new
|
||||
# business intent. No arbitrary caller field is discarded otherwise.
|
||||
if (normalized.get('kind')=='work-api-v1'and normalized.get('method')=='POST'
|
||||
and isinstance(normalized.get('path'),str)
|
||||
and re.fullmatch(r'/open-apis/calendar/v4/calendars/[A-Za-z0-9_.@!:$-]+/events',normalized['path'])
|
||||
and isinstance(normalized.get('params'),dict)):
|
||||
expected=hashlib.sha256(canonical([owner,operation_id])).hexdigest()
|
||||
if normalized['params'].get('idempotency_key')==expected:
|
||||
normalized['params'].pop('idempotency_key')
|
||||
return hashlib.sha256(canonical(normalized)).hexdigest()
|
||||
|
||||
def _row(self,approval_id):
|
||||
if not isinstance(approval_id,str)or not re.fullmatch(r'apr_[a-f0-9]{64}',approval_id):
|
||||
raise ApprovalError('invalid_approval_reference')
|
||||
row=self.db.execute('SELECT * FROM work_approvals_v3 WHERE approval_id=? AND environment=? AND app_id=?',
|
||||
(approval_id,self.state.environment,self.state.app_id)).fetchone()
|
||||
if not row:raise ApprovalError('approval_not_found')
|
||||
return dict(row)
|
||||
|
||||
def _expire(self,row):
|
||||
if row['expires_at']<=self.clock()and row['state']in {'pending','approved'}:
|
||||
self.db.execute("UPDATE work_approvals_v3 SET state='expired',updated_at=? WHERE approval_id=? AND state IN ('pending','approved')",
|
||||
(self.clock(),row['approval_id']))
|
||||
row=self._row(row['approval_id'])
|
||||
self.audit.record(row,'expired')
|
||||
return row
|
||||
|
||||
@staticmethod
|
||||
def _status(row):
|
||||
return {k:row[k]for k in ('approval_id','state','expires_at','operation_digest')}
|
||||
|
||||
def create(self,task_id,owner,host,generation,operation_id,action,summary,impact,ttl=120):
|
||||
if not isinstance(operation_id,str)or not re.fullmatch(r'[A-Za-z0-9_-]{8,100}',operation_id):
|
||||
raise ApprovalError('invalid_operation_id')
|
||||
if type(ttl)is not int or not 30<=ttl<=600:raise ApprovalError('invalid_approval_lifetime')
|
||||
if not isinstance(action,dict)or not action:raise ApprovalError('invalid_operation')
|
||||
encoded=json.dumps(action,sort_keys=True,ensure_ascii=True,separators=(',',':'),allow_nan=False).encode()
|
||||
if len(encoded)>32000:raise ApprovalError('operation_budget_exceeded')
|
||||
summary=_public_text(summary,200);impact=_public_text(impact,500)
|
||||
digest=hashlib.sha256(encoded).hexdigest()
|
||||
approval_id='apr_'+hashlib.sha256(canonical([self.state.environment,self.state.app_id,task_id,operation_id])).hexdigest()
|
||||
with self.db:
|
||||
self.state._owner_lock(owner)
|
||||
task=self._task(task_id,owner,host,generation)
|
||||
old=self.db.execute('SELECT * FROM work_approvals_v3 WHERE approval_id=?',(approval_id,)).fetchone()
|
||||
if old:
|
||||
row=self._expire(dict(old))
|
||||
if (row['owner'],row['host_id'],row['binding_generation'],row['operation_digest'])!=(owner,host,generation,digest):
|
||||
raise ApprovalError('operation_identity_conflict')
|
||||
return self._status(row)
|
||||
# Scope to the SAME authenticated task. Legitimate subsequent user
|
||||
# tasks are independent; changing a model-chosen operation ID inside
|
||||
# this task cannot erase a rejection, expiry or consumed grant.
|
||||
intent=self._intent(action,owner,operation_id)
|
||||
previous=self.db.execute('''SELECT * FROM work_approvals_v3
|
||||
WHERE environment=? AND app_id=? AND task_id=? LIMIT 51''',
|
||||
(self.state.environment,self.state.app_id,task_id)).fetchall()
|
||||
if len(previous)>=50:raise ApprovalError('approval_task_budget_exceeded')
|
||||
for prior in previous:
|
||||
prior=dict(prior)
|
||||
if not prior['ciphertext']:raise ApprovalError('approval_intent_unavailable')
|
||||
original=json.loads(self.keyring.decrypt(prior['key_id'],prior['ciphertext'],self._aad(prior)))
|
||||
if hmac.compare_digest(intent,self._intent(original['action'],prior['owner'],prior['operation_id'])):
|
||||
raise ApprovalError('approval_intent_already_registered')
|
||||
nonce=secrets.token_urlsafe(32)
|
||||
row={'approval_id':approval_id,'environment':task['environment'],'app_id':task['app_id'],
|
||||
'task_id':task_id,'owner':owner,'host_id':host,'binding_generation':generation,
|
||||
'operation_id':operation_id,'operation_digest':digest}
|
||||
key,cipher=self.keyring.encrypt(canonical({'action':action,'summary':summary,'impact':impact,'nonce':nonce}),self._aad(row))
|
||||
expires=min(self.clock()+ttl,task['deadline'])
|
||||
if expires<=self.clock():raise ApprovalError('approval_task_expired')
|
||||
self.db.execute('INSERT INTO work_approvals_v3 VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
|
||||
(approval_id,row['environment'],row['app_id'],task_id,owner,host,generation,operation_id,digest,
|
||||
hashlib.sha256(nonce.encode()).hexdigest(),key,cipher,'','pending',expires,self.clock()))
|
||||
action_class=('work_api_delete'if action.get('method')=='DELETE'else
|
||||
'work_api_read'if action.get('method')=='GET'else'work_api_write')if action.get('kind')=='work-api-v1'else'unknown'
|
||||
self.audit.record(self._row(approval_id),'created',action_class)
|
||||
return self._status(self._row(approval_id))
|
||||
|
||||
def card_material(self,approval_id):
|
||||
"""Internal sender only: private task origin + fixed confirmation text."""
|
||||
with self.db:
|
||||
row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id))
|
||||
task=self._task(row['task_id'],row['owner'],row['host_id'],row['binding_generation'])
|
||||
if row['state']!='pending':raise ApprovalError('approval_not_pending')
|
||||
material=json.loads(self.keyring.decrypt(row['key_id'],row['ciphertext'],self._aad(row)))
|
||||
return {'source_message_id':task['source_message_id'],'chat_id':task['chat_id'],
|
||||
'approval_id':approval_id,'summary':material['summary'],'impact':material['impact'],
|
||||
'nonce':material['nonce'],'expires_at':row['expires_at']}
|
||||
|
||||
def bind_card(self,approval_id,message_id):
|
||||
if not isinstance(message_id,str)or not re.fullmatch(r'om_[A-Za-z0-9_-]{1,120}',message_id):
|
||||
raise ApprovalError('invalid_card_receipt')
|
||||
with self.db:
|
||||
row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id))
|
||||
task=self.state.task(row['task_id'])
|
||||
if row['state']in {'pending','approved'}and(not self.state.task_current(task)or task['status']!='inflight'or self._execution_closed(row['task_id'])):
|
||||
self.db.execute("UPDATE work_approvals_v3 SET state='cancelled',updated_at=? WHERE approval_id=?",(self.clock(),approval_id))
|
||||
self.audit.record(self._row(approval_id),'cancelled')
|
||||
if not self.db.execute('SELECT approval_id FROM work_approval_card_outbox_v1 WHERE approval_id=?',(approval_id,)).fetchone():
|
||||
raise ApprovalError('approval_send_intent_required')
|
||||
if row['card_message_id']not in ('',message_id):raise ApprovalError('approval_card_conflict')
|
||||
self.db.execute('UPDATE work_approvals_v3 SET card_message_id=?,updated_at=? WHERE approval_id=?',
|
||||
(message_id,self.clock(),approval_id))
|
||||
self.db.execute("UPDATE work_approval_card_outbox_v1 SET state='sent',updated_at=? WHERE approval_id=?",
|
||||
(self.clock(),approval_id))
|
||||
self.audit.record(self._row(approval_id),'card_bound')
|
||||
|
||||
def claim_card(self,approval_id):
|
||||
"""Persist send intent before network I/O; lost receipt is not resendable."""
|
||||
with self.db:
|
||||
row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id))
|
||||
self._task(row['task_id'],row['owner'],row['host_id'],row['binding_generation'])
|
||||
if row['state']!='pending':raise ApprovalError('approval_not_pending')
|
||||
if row['card_message_id']:return False
|
||||
previous=self.db.execute('SELECT state FROM work_approval_card_outbox_v1 WHERE approval_id=?',(approval_id,)).fetchone()
|
||||
if previous:return False
|
||||
self.db.execute("INSERT INTO work_approval_card_outbox_v1 VALUES(?,'sending',?)",(approval_id,self.clock()))
|
||||
return True
|
||||
|
||||
def card_send_unknown(self,approval_id):
|
||||
with self.db:
|
||||
changed=self.db.execute("UPDATE work_approval_card_outbox_v1 SET state='unknown',updated_at=? WHERE approval_id=? AND state='sending'",
|
||||
(self.clock(),approval_id)).rowcount
|
||||
if changed:self.audit.record(self._row(approval_id),'card_send_unknown')
|
||||
|
||||
def close_unavailable(self,limit=100):
|
||||
"""Future bounded maintenance: expiry or invalid task never grants consent."""
|
||||
if type(limit)is not int or not 1<=limit<=100:raise ApprovalError('invalid_approval_budget')
|
||||
rows=self.db.execute("SELECT approval_id FROM work_approvals_v3 WHERE environment=? AND app_id=? AND state IN ('pending','approved') AND approval_id>? ORDER BY approval_id LIMIT ?",
|
||||
(self.state.environment,self.state.app_id,self._sweep_after,limit)).fetchall()
|
||||
if not rows:self._sweep_after=''
|
||||
closed=[];started=time.monotonic()
|
||||
for candidate in rows:
|
||||
if time.monotonic()-started>=.5:break
|
||||
with self.db:
|
||||
row=self._row(candidate['approval_id']);self.state._owner_lock(row['owner']);row=self._row(row['approval_id'])
|
||||
if row['state']not in {'pending','approved'}:continue
|
||||
task=self.state.task(row['task_id'])
|
||||
expired=row['expires_at']<=self.clock()or bool(task and task['deadline']<=self.clock())
|
||||
invalid=(not self.state.task_current(task)or task['status']!='inflight'or self._execution_closed(row['task_id'])or
|
||||
(task['owner'],task['host_id'],task['binding_generation'])!=
|
||||
(row['owner'],row['host_id'],row['binding_generation']))
|
||||
if expired or invalid:
|
||||
state='expired'if expired else'cancelled'
|
||||
self.db.execute("UPDATE work_approvals_v3 SET state=?,updated_at=? WHERE approval_id=? AND state IN ('pending','approved')",
|
||||
(state,self.clock(),row['approval_id']))
|
||||
closed.append({'approval_id':row['approval_id'],'state':state})
|
||||
self.audit.record(self._row(row['approval_id']),state)
|
||||
self._sweep_after=candidate['approval_id']
|
||||
return closed
|
||||
|
||||
def closed_card_receipt(self,approval_id,refresh=False):
|
||||
"""Internal renderer only. Never return action/nonce or imply execution success."""
|
||||
if type(refresh)is not bool:raise ApprovalError('invalid_card_refresh')
|
||||
with self.db:
|
||||
row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(approval_id))
|
||||
if row['state']not in {'rejected','expired','cancelled','consumed'}or not row['card_message_id']:
|
||||
return None
|
||||
sent=self.db.execute('SELECT state FROM work_approval_card_outbox_v1 WHERE approval_id=?',(approval_id,)).fetchone()
|
||||
if sent and sent['state']=='closed'and not refresh:return None
|
||||
self.db.execute('UPDATE work_approval_card_outbox_v1 SET updated_at=? WHERE approval_id=?',
|
||||
(self.clock(),approval_id))
|
||||
return {'message_id':row['card_message_id'],'state':row['state']}
|
||||
|
||||
def mark_card_closed(self,approval_id,state):
|
||||
with self.db:
|
||||
row=self._row(approval_id);self.state._owner_lock(row['owner']);row=self._row(approval_id)
|
||||
if row['state']!=state or state not in {'rejected','expired','cancelled','consumed'}:
|
||||
raise ApprovalError('approval_state_changed')
|
||||
self.db.execute("UPDATE work_approval_card_outbox_v1 SET state='closed',updated_at=? WHERE approval_id=?",
|
||||
(self.clock(),approval_id))
|
||||
|
||||
def cards_to_send(self):
|
||||
rows=self.db.execute("""SELECT a.approval_id FROM work_approvals_v3 a
|
||||
WHERE a.environment=? AND a.app_id=? AND a.state='pending' AND a.expires_at>?
|
||||
AND a.card_message_id='' AND NOT EXISTS(SELECT 1 FROM work_approval_card_outbox_v1 o WHERE o.approval_id=a.approval_id)
|
||||
ORDER BY a.updated_at LIMIT 5""",(self.state.environment,self.state.app_id,self.clock())).fetchall()
|
||||
return [row['approval_id']for row in rows]
|
||||
|
||||
def cards_to_close(self):
|
||||
rows=self.db.execute("""SELECT a.approval_id FROM work_approvals_v3 a
|
||||
JOIN work_approval_card_outbox_v1 o ON o.approval_id=a.approval_id
|
||||
WHERE a.environment=? AND a.app_id=? AND a.state IN ('rejected','expired','cancelled','consumed')
|
||||
AND a.card_message_id<>'' AND o.state<>'closed' ORDER BY o.updated_at LIMIT 5""",
|
||||
(self.state.environment,self.state.app_id)).fetchall()
|
||||
return [row['approval_id']for row in rows]
|
||||
|
||||
def closed_for_task(self,task_id):
|
||||
if not isinstance(task_id,str)or not re.fullmatch(r'msg_[a-f0-9]{64}',task_id):
|
||||
raise ApprovalError('invalid_task_reference')
|
||||
rows=self.db.execute('''SELECT approval_id FROM work_approvals_v3 WHERE environment=? AND app_id=?
|
||||
AND task_id=? AND state IN ('rejected','expired','cancelled','consumed') AND card_message_id<>'' LIMIT 10''',
|
||||
(self.state.environment,self.state.app_id,task_id)).fetchall()
|
||||
return [row['approval_id']for row in rows]
|
||||
|
||||
def decide(self,decision):
|
||||
if not isinstance(decision,VerifiedDecision):raise ApprovalError('verified_callback_required')
|
||||
if decision.choice not in {'allow','deny'}:raise ApprovalError('invalid_approval_choice')
|
||||
if (not isinstance(decision.event_id,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',decision.event_id)
|
||||
or not isinstance(decision.event_digest,str)or not re.fullmatch(r'[a-f0-9]{64}',decision.event_digest)):
|
||||
raise ApprovalError('invalid_callback_receipt')
|
||||
fields={key:getattr(decision,key)for key in ('environment','app_id','actor_open_id',
|
||||
'chat_id','card_message_id','approval_id','nonce','choice')}
|
||||
if not hmac.compare_digest(hashlib.sha256(canonical(fields)).hexdigest(),decision.event_digest):
|
||||
raise ApprovalError('callback_receipt_conflict')
|
||||
with self.db:
|
||||
row=self._row(decision.approval_id);self.state._owner_lock(row['owner']);row=self._expire(self._row(decision.approval_id))
|
||||
task=self._task(row['task_id'],row['owner'],row['host_id'],row['binding_generation'])
|
||||
if (decision.environment,decision.app_id,decision.actor_open_id,decision.chat_id,decision.card_message_id)!=\
|
||||
(row['environment'],row['app_id'],row['owner'],task['chat_id'],row['card_message_id'])or not row['card_message_id']:
|
||||
raise ApprovalError('callback_identity_conflict')
|
||||
if not isinstance(decision.nonce,str)or len(decision.nonce)>128 or not hmac.compare_digest(hashlib.sha256(decision.nonce.encode()).hexdigest(),row['nonce_digest']):
|
||||
raise ApprovalError('invalid_approval_nonce')
|
||||
# Owner lock serializes duplicate clicks for this approval. Reused
|
||||
# event IDs for a different approval also fail the database PK.
|
||||
receipt=self.db.execute('SELECT approval_id,event_digest FROM work_approval_callbacks_v1 WHERE environment=? AND app_id=? AND event_id=?',
|
||||
(row['environment'],row['app_id'],decision.event_id)).fetchone()
|
||||
if receipt:
|
||||
if (receipt['approval_id'],receipt['event_digest'])!=(row['approval_id'],decision.event_digest):
|
||||
raise ApprovalError('callback_receipt_conflict')
|
||||
return self._status(row)
|
||||
target='approved'if decision.choice=='allow'else'rejected'
|
||||
if row['state']not in {'pending',target}:raise ApprovalError('approval_already_closed')
|
||||
self.db.execute('INSERT INTO work_approval_callbacks_v1 VALUES(?,?,?,?,?,?)',
|
||||
(row['environment'],row['app_id'],decision.event_id,row['approval_id'],decision.event_digest,self.clock()))
|
||||
self.db.execute("UPDATE work_approvals_v3 SET state=?,updated_at=? WHERE approval_id=? AND state='pending'",
|
||||
(target,self.clock(),row['approval_id']))
|
||||
self.audit.record(self._row(row['approval_id']),target)
|
||||
return self._status(self._row(row['approval_id']))
|
||||
|
||||
def status(self,approval_id,task_id,owner,host,generation,operation_digest):
|
||||
"""Future device-authenticated polling endpoint; no nonce/action returned."""
|
||||
with self.db:
|
||||
self.state._owner_lock(owner);row=self._expire(self._row(approval_id))
|
||||
self._task(task_id,owner,host,generation)
|
||||
if (row['task_id'],row['owner'],row['host_id'],row['binding_generation'],row['operation_digest'])!=\
|
||||
(task_id,owner,host,generation,operation_digest):raise ApprovalError('operation_identity_conflict')
|
||||
return self._status(row)
|
||||
|
||||
def consume(self,approval_id,task_id,owner,host,generation,operation_digest):
|
||||
with self.db:
|
||||
self.state._owner_lock(owner)
|
||||
row=self._expire(self._row(approval_id))
|
||||
self._task(task_id,owner,host,generation)
|
||||
if (row['task_id'],row['owner'],row['host_id'],row['binding_generation'],row['operation_digest'])!=(task_id,owner,host,generation,operation_digest):
|
||||
raise ApprovalError('operation_identity_conflict')
|
||||
if row['state']!='approved':return {**self._status(row),'execute':False}
|
||||
material=json.loads(self.keyring.decrypt(row['key_id'],row['ciphertext'],self._aad(row)))
|
||||
changed=self.db.execute("UPDATE work_approvals_v3 SET state='consumed',updated_at=? WHERE approval_id=? AND state='approved'",
|
||||
(self.clock(),approval_id)).rowcount
|
||||
if changed!=1:return {**self._status(self._row(approval_id)),'execute':False}
|
||||
self.audit.record(self._row(approval_id),'consumed')
|
||||
return {**self._status(self._row(approval_id)),'execute':True,'action':material['action']}
|
||||
|
||||
def cancel(self,approval_id,task_id,owner,host,generation):
|
||||
with self.db:
|
||||
self.state._owner_lock(owner);row=self._row(approval_id)
|
||||
self._task(task_id,owner,host,generation)
|
||||
if (row['task_id'],row['owner'],row['host_id'],row['binding_generation'])!=(task_id,owner,host,generation):raise ApprovalError('approval_identity_conflict')
|
||||
changed=self.db.execute("UPDATE work_approvals_v3 SET state='cancelled',updated_at=? WHERE approval_id=? AND state IN ('pending','approved')",
|
||||
(self.clock(),approval_id)).rowcount
|
||||
if changed:self.audit.record(self._row(approval_id),'cancelled')
|
||||
return self._status(self._row(approval_id))
|
||||
|
||||
def maintain_retention(self,payload_days=30,audit_days=90,limit=10):
|
||||
"""Prune CLOSED payloads/audit only. Durable tombstones fence replay."""
|
||||
if (type(payload_days)is not int or type(audit_days)is not int
|
||||
or not 1<=payload_days<=audit_days<=365 or type(limit)is not int or not 1<=limit<=20):
|
||||
raise ApprovalError('invalid_approval_retention')
|
||||
cutoff=self.clock()-payload_days*86400
|
||||
rows=self.db.execute('''SELECT approval_id FROM work_approvals_v3 a WHERE environment=? AND app_id=?
|
||||
AND state IN ('rejected','expired','cancelled','consumed') AND updated_at<?
|
||||
AND approval_id>?
|
||||
AND NOT EXISTS(SELECT 1 FROM work_approval_retention_v1 r WHERE r.approval_id=a.approval_id)
|
||||
ORDER BY approval_id LIMIT ?''',(self.state.environment,self.state.app_id,cutoff,self._retention_after,limit)).fetchall()
|
||||
if not rows:self._retention_after=''
|
||||
purged=0;started=time.monotonic()
|
||||
for item in rows:
|
||||
if time.monotonic()-started>.5:break
|
||||
self._retention_after=item['approval_id']
|
||||
with self.db:
|
||||
row=self._row(item['approval_id']);self.state._owner_lock(row['owner']);row=self._row(row['approval_id'])
|
||||
if row['state']not in {'rejected','expired','cancelled','consumed'}or row['updated_at']>=cutoff:continue
|
||||
if row['state']=='consumed':
|
||||
task=self.state.task(row['task_id'])
|
||||
if not task or task['status']in {'inflight','unknown'}:continue
|
||||
marked=self.db.execute('INSERT INTO work_approval_retention_v1 VALUES(?,?) ON CONFLICT DO NOTHING',
|
||||
(row['approval_id'],self.clock())).rowcount
|
||||
if not marked:continue
|
||||
self.db.execute("UPDATE work_approvals_v3 SET ciphertext='',key_id='',nonce_digest='' WHERE approval_id=?",(row['approval_id'],))
|
||||
self.audit.record(row,'payload_purged');purged+=1
|
||||
# Audit removal is bounded too; approval and operation tombstones stay.
|
||||
with self.db:
|
||||
removed=self.db.execute('''DELETE FROM work_approval_audit_v1 WHERE event_key IN (
|
||||
SELECT x.event_key FROM work_approval_audit_v1 x JOIN work_approval_retention_v1 r
|
||||
ON r.approval_id=x.approval_id WHERE x.environment=? AND x.app_id=? AND x.created_at<?
|
||||
ORDER BY x.created_at LIMIT ?)''',
|
||||
(self.state.environment,self.state.app_id,self.clock()-audit_days*86400,limit)).rowcount
|
||||
return {'payloads_purged':purged,'audit_events_removed':removed,'tombstones_preserved':True}
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Single bounded approval worker with its own database connection."""
|
||||
import asyncio
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import time
|
||||
|
||||
from .approval_store import ApprovalError, ApprovalStore
|
||||
|
||||
|
||||
class ApprovalWorker:
|
||||
def __init__(self,settings,keyring):
|
||||
self.settings,self.keyring,self.clock=settings,keyring,time.time
|
||||
self.executor=ThreadPoolExecutor(max_workers=1,thread_name_prefix='work-approval')
|
||||
self.slots=asyncio.Semaphore(8)
|
||||
self.store=None
|
||||
|
||||
def _start(self):
|
||||
from .store import Store
|
||||
from .binding_state import BindingState
|
||||
self.connection=Store(self.settings.data_dir/'metadata.sqlite',self.settings.database_url)
|
||||
if self.settings.database_url:
|
||||
self.connection.db.execute("SET lock_timeout TO '500ms'")
|
||||
self.connection.db.execute("SET statement_timeout TO '800ms'")
|
||||
else:self.connection.db.execute('PRAGMA busy_timeout=500')
|
||||
self.store=ApprovalStore(BindingState(self.connection,self.settings.environment,
|
||||
self.settings.feishu_app_id),self.keyring,enforce_execution_state=True)
|
||||
|
||||
async def start(self):
|
||||
await asyncio.get_running_loop().run_in_executor(self.executor,self._start)
|
||||
|
||||
def _invoke(self,deadline,method,args):
|
||||
if time.monotonic()>deadline or self.store is None:
|
||||
raise ApprovalError('approval_temporarily_unavailable')
|
||||
try:
|
||||
# Postgres may reconnect after a lost connection. Reapply this
|
||||
# worker's limits before opening a new approval transaction.
|
||||
if self.settings.database_url:
|
||||
db=self.connection.db
|
||||
if db.conn.closed or db.conn.broken:
|
||||
db.execute("SET lock_timeout TO '500ms'")
|
||||
db.execute("SET statement_timeout TO '800ms'")
|
||||
return getattr(self.store,method)(*args)
|
||||
except ApprovalError:raise
|
||||
except Exception:raise ApprovalError('approval_temporarily_unavailable')from None
|
||||
|
||||
async def call(self,method,*args):
|
||||
allowed={'create','status','consume','cancel','decide','card_material','claim_card',
|
||||
'bind_card','card_send_unknown','close_unavailable','closed_card_receipt',
|
||||
'mark_card_closed','cards_to_send','cards_to_close','maintain_retention','closed_for_task'}
|
||||
if method not in allowed:raise ApprovalError('unsupported_approval_operation')
|
||||
try:await asyncio.wait_for(self.slots.acquire(),timeout=.05)
|
||||
except asyncio.TimeoutError:raise ApprovalError('approval_temporarily_unavailable')from None
|
||||
future=asyncio.get_running_loop().run_in_executor(self.executor,self._invoke,
|
||||
time.monotonic()+1.2,method,args)
|
||||
# Keep the queue slot until the worker really stops, even when a caller
|
||||
# times out. A timeout is not proof that a database commit did not occur.
|
||||
def settled(completed):
|
||||
self.slots.release()
|
||||
if not completed.cancelled():completed.exception()
|
||||
future.add_done_callback(settled)
|
||||
try:return await asyncio.wait_for(asyncio.shield(future),timeout=1.5)
|
||||
except asyncio.TimeoutError:raise ApprovalError('approval_result_unconfirmed')from None
|
||||
|
||||
async def close(self):
|
||||
def finish():
|
||||
if self.store is not None:self.connection.close()
|
||||
self.store=None
|
||||
try:await asyncio.get_running_loop().run_in_executor(self.executor,finish)
|
||||
finally:self.executor.shutdown(wait=False,cancel_futures=True)
|
||||
@@ -41,6 +41,12 @@ class Settings(BaseSettings):
|
||||
max_connections: int = 1000
|
||||
max_result: int = 16000
|
||||
work_files_enabled: bool = False
|
||||
work_task_cards_enabled: bool = False
|
||||
work_remote_approvals_enabled: bool = False
|
||||
feishu_tenant_key: str = ''
|
||||
approval_retention_enabled: bool = False
|
||||
approval_payload_retention_days: int = 30
|
||||
approval_audit_retention_days: int = 90
|
||||
work_file_max_bytes: int = 10*1024*1024
|
||||
work_file_batch_limit: int = 20
|
||||
work_file_retention: int = 86400
|
||||
@@ -56,6 +62,16 @@ class Settings(BaseSettings):
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_runtime(self):
|
||||
if not 1<=self.approval_payload_retention_days<=self.approval_audit_retention_days<=365:
|
||||
raise ValueError('approval retention requires 1 <= payload days <= audit days <= 365')
|
||||
if self.approval_retention_enabled and not self.work_remote_approvals_enabled:
|
||||
raise ValueError('approval retention requires remote approvals')
|
||||
if self.work_remote_approvals_enabled:
|
||||
if (self.credential_mode!='broker'or self.feishu_event_source!='longconn'
|
||||
or not self.feishu_enabled or not self.feishu_verification_token
|
||||
or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',self.feishu_tenant_key)):
|
||||
raise ValueError('remote approvals require broker, longconn and explicit callback identity')
|
||||
if self.work_task_cards_enabled and self.credential_mode!='broker':raise ValueError('task cards require broker mode')
|
||||
if not 1024<=self.work_file_max_bytes<=100*1024*1024 or not 1<=self.work_file_batch_limit<=50 or not 300<=self.work_file_retention<=7*86400:
|
||||
raise ValueError('invalid work file deployment limits')
|
||||
if self.work_files_enabled and self.credential_mode!='broker':raise ValueError('work files require broker mode')
|
||||
|
||||
@@ -58,6 +58,10 @@ class Pending:
|
||||
progress: asyncio.Event = field(default_factory=asyncio.Event)
|
||||
|
||||
|
||||
class ClientIdleTimeout(asyncio.TimeoutError):
|
||||
"""No authenticated task progress arrived within the idle timeout."""
|
||||
|
||||
|
||||
class Broker:
|
||||
def __init__(self, settings, redis, store):
|
||||
self.s, self.r, self.store = settings, redis, store
|
||||
@@ -66,6 +70,7 @@ class Broker:
|
||||
self.worker_id = uuid.uuid4().hex
|
||||
self.lifecycle = None
|
||||
self.files = None
|
||||
self.task_cards = None
|
||||
|
||||
async def replies(self, pub):
|
||||
async for message in pub.listen():
|
||||
@@ -113,7 +118,8 @@ class Broker:
|
||||
if item.future in done:
|
||||
return item.future.result()
|
||||
if not done:
|
||||
raise asyncio.TimeoutError()
|
||||
if remaining<=self.s.request_timeout:raise asyncio.TimeoutError()
|
||||
raise ClientIdleTimeout()
|
||||
finally:
|
||||
progress.cancel()
|
||||
with contextlib.suppress(asyncio.CancelledError):
|
||||
@@ -212,10 +218,12 @@ class Broker:
|
||||
self.lifecycle.transition(turn,"completed","finished")
|
||||
self.store.audit(owner, "instruction", host, result.get("status","unknown") if task else "completed")
|
||||
return result
|
||||
except asyncio.TimeoutError:
|
||||
except asyncio.TimeoutError as error:
|
||||
self.store.finish(turn, "unknown")
|
||||
if task:
|
||||
self.lifecycle.transition(turn,"inflight","unknown")
|
||||
if isinstance(error,ClientIdleTimeout) and self.task_cards:
|
||||
self.task_cards.mark_connection_lost(turn)
|
||||
return {"status": "unknown", "turn_id": turn,
|
||||
"result": "等待结果超时,执行状态未知;不要直接重发有副作用的任务。"}
|
||||
finally:
|
||||
@@ -257,6 +265,8 @@ def create_app(settings=None, redis=None, platform=None):
|
||||
lifecycle = None
|
||||
from .notifications import router as notification_router
|
||||
background = set()
|
||||
approvals = None
|
||||
approval_worker = None
|
||||
|
||||
def spawn(coro):
|
||||
task = asyncio.create_task(coro)
|
||||
@@ -265,9 +275,16 @@ def create_app(settings=None, redis=None, platform=None):
|
||||
# Retrieve errors without displaying possibly sensitive exception text.
|
||||
task.add_done_callback(lambda t: t.exception() if not t.cancelled() else None)
|
||||
|
||||
if s.work_remote_approvals_enabled:
|
||||
from .approval_api import ApprovalAPI
|
||||
from .approval_worker import ApprovalWorker
|
||||
approval_worker=ApprovalWorker(s,credential_store.keyring)
|
||||
approvals=ApprovalAPI(oauth,approval_worker,f,spawn)
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app):
|
||||
await r.ping()
|
||||
if approval_worker:await approval_worker.start()
|
||||
if lifecycle:
|
||||
await lifecycle.maintain()
|
||||
await deliveries.recover_interrupted_sends()
|
||||
@@ -294,14 +311,20 @@ def create_app(settings=None, redis=None, platform=None):
|
||||
await deliveries.recover_interrupted_sends()
|
||||
if broker.files:await broker.files.flush()
|
||||
await deliveries.flush()
|
||||
if approvals:await approvals.maintain()
|
||||
except Exception:
|
||||
# No credentials or raw exception text in background logs.
|
||||
pass
|
||||
lifecycle_task = asyncio.create_task(maintenance()) if lifecycle else None
|
||||
async def card_maintenance():
|
||||
while True:
|
||||
await asyncio.sleep(5)
|
||||
with contextlib.suppress(Exception):await deliveries.cards.flush()
|
||||
cards_task=asyncio.create_task(card_maintenance()) if lifecycle and deliveries.cards else None
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
work = [reply_task,sweep_task,*background] + ([lifecycle_task] if lifecycle_task else [])
|
||||
work = [reply_task,sweep_task,*background] + ([lifecycle_task] if lifecycle_task else []) + ([cards_task] if cards_task else [])
|
||||
for task in work:
|
||||
task.cancel()
|
||||
await asyncio.gather(*work, return_exceptions=True)
|
||||
@@ -310,14 +333,19 @@ def create_app(settings=None, redis=None, platform=None):
|
||||
await r.delete("hn:task-worker:"+broker.worker_id)
|
||||
await pub.aclose()
|
||||
await f.close()
|
||||
if approval_worker:await approval_worker.close()
|
||||
store.close()
|
||||
if redis is None:
|
||||
await r.aclose()
|
||||
|
||||
app = FastAPI(title="Hainuo B1 connector", lifespan=lifespan, docs_url=None, redoc_url=None)
|
||||
app.add_middleware(BodyLimitMiddleware, max_body=s.max_body,
|
||||
path_limits={'/work/v2/files/upload':s.work_file_max_bytes}if broker.files else {})
|
||||
path_limits={**({'/work/v2/files/upload':s.work_file_max_bytes}if broker.files else {}),
|
||||
**({'/internal/approval-callback':65536,'/work/v2/approvals/create':49152}if approvals else {})})
|
||||
app.include_router(oauth.router)
|
||||
if approvals:
|
||||
app.include_router(approvals.router)
|
||||
app.state.approvals=approvals
|
||||
if lifecycle:
|
||||
app.include_router(deliveries.router)
|
||||
app.state.deliveries=deliveries
|
||||
|
||||
@@ -121,6 +121,7 @@ class CredentialAPI:
|
||||
raise HTTPException(503, "credential_temporarily_unavailable") from None
|
||||
self.require_binding(body.host_id, device["owner"], device["binding_generation"])
|
||||
return no_cache(JSONResponse({"credential_mode": "broker", "protocol_version": 2,
|
||||
"approval_protocol": "work-approval-v1" if self.s.work_remote_approvals_enabled else None,
|
||||
"binding_owner": device["owner"],
|
||||
"binding_generation": device["binding_generation"],
|
||||
"permission_profiles": sorted(self.s.credential_permission_profiles), **result}))
|
||||
|
||||
@@ -4,12 +4,31 @@ import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor,TimeoutError as FutureTimeout
|
||||
|
||||
import httpx
|
||||
import redis
|
||||
from .config import Settings
|
||||
|
||||
|
||||
def safe_card_callback_response(payload):
|
||||
"""Forward only code-owned terminal cards returned after verified decisions."""
|
||||
from .approval_cards import decision_response_card
|
||||
fallback={'toast':{'type':'warning','content':'确认结果待核对,请查看任务状态。'}}
|
||||
if not isinstance(payload,dict):return fallback
|
||||
toast=payload.get('toast')
|
||||
if (not isinstance(toast,dict)or toast.get('type')not in {'info','success','warning','error'}
|
||||
or not isinstance(toast.get('content'),str)or len(toast['content'])>200):return fallback
|
||||
response={'toast':{'type':toast['type'],'content':toast['content']}}
|
||||
card=payload.get('card')
|
||||
if card is not None:
|
||||
allowed=[{'type':'raw','data':decision_response_card(state)}for state in
|
||||
('approved','rejected','expired','cancelled','consumed')]
|
||||
if card not in allowed:return fallback
|
||||
response['card']=card
|
||||
return response
|
||||
|
||||
|
||||
def disable_proxy_environment():
|
||||
"""This dedicated SDK worker must connect directly, including discovery and WS."""
|
||||
for key in ("HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "http_proxy", "https_proxy", "all_proxy"):
|
||||
@@ -51,7 +70,48 @@ def main():
|
||||
raise RuntimeError("connector did not accept event")
|
||||
print(json.dumps({"event": "platform_event_accepted", "type": data.get("header", {}).get("event_type")}), flush=True)
|
||||
|
||||
handler = lark.EventDispatcherHandler.builder("", "").register_p2_im_message_receive_v1(received).build()
|
||||
builder = lark.EventDispatcherHandler.builder("", "").register_p2_im_message_receive_v1(received)
|
||||
callback_pool=None
|
||||
if s.work_remote_approvals_enabled:
|
||||
from lark_oapi.event.callback.model.p2_card_action_trigger import P2CardActionTriggerResponse
|
||||
callback_pool=ThreadPoolExecutor(max_workers=2,thread_name_prefix='work-card-callback')
|
||||
callback_slots=threading.BoundedSemaphore(2)
|
||||
def forward_card(raw):
|
||||
deadline=time.monotonic()+2.1
|
||||
# Own this client until the worker ends. The SDK response deadline
|
||||
# does not imply an in-flight authenticated decision was rolled back.
|
||||
with httpx.Client(base_url=url,trust_env=False,follow_redirects=False,
|
||||
timeout=httpx.Timeout(.8,connect=.2,pool=.1))as callback_http:
|
||||
with callback_http.stream('POST','/internal/approval-callback',content=raw,
|
||||
headers={'X-Internal-Secret':s.internal_secret,'Content-Type':'application/json'})as response:
|
||||
if response.status_code!=200:raise ValueError()
|
||||
body=bytearray()
|
||||
for chunk in response.iter_bytes():
|
||||
body.extend(chunk)
|
||||
if len(body)>4096 or time.monotonic()>deadline:raise ValueError()
|
||||
return json.loads(body)
|
||||
def decision(event):
|
||||
# Same authenticated SDK connection/lease, no second consumer.
|
||||
# Never log callback bodies or the card-update credential.
|
||||
fallback={'toast':{'type':'warning','content':'确认结果待核对,请查看原任务状态。'}}
|
||||
try:
|
||||
raw=lark.JSON.marshal(event).encode('utf-8')
|
||||
if len(raw)>65536:return P2CardActionTriggerResponse(fallback)
|
||||
if not callback_slots.acquire(blocking=False):return P2CardActionTriggerResponse(fallback)
|
||||
try:future=callback_pool.submit(forward_card,raw)
|
||||
except BaseException:
|
||||
callback_slots.release()
|
||||
raise
|
||||
def settled(completed):
|
||||
callback_slots.release()
|
||||
if not completed.cancelled():completed.exception()
|
||||
future.add_done_callback(settled)
|
||||
try:payload=future.result(timeout=2.3)
|
||||
except FutureTimeout:return P2CardActionTriggerResponse(fallback)
|
||||
return P2CardActionTriggerResponse(safe_card_callback_response(payload))
|
||||
except Exception:return P2CardActionTriggerResponse(fallback)
|
||||
builder=builder.register_p2_card_action_trigger(decision)
|
||||
handler=builder.build()
|
||||
class ObservableClient(lark.ws.Client):
|
||||
async def _connect(self):
|
||||
try:
|
||||
@@ -69,6 +129,7 @@ def main():
|
||||
finally:
|
||||
done.set()
|
||||
client.close()
|
||||
if callback_pool:callback_pool.shutdown(wait=False,cancel_futures=True)
|
||||
try:
|
||||
lease.release()
|
||||
except redis.exceptions.LockError:
|
||||
|
||||
@@ -122,6 +122,22 @@ class Feishu:
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
json={"msg_type": "text", "content": json.dumps({"text": text}, ensure_ascii=False)})
|
||||
|
||||
async def reply_task_card(self,message_id,card,delivery_id,*,token):
|
||||
import re
|
||||
if not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',message_id):raise PlatformError('invalid reply reference')
|
||||
data=await self._json('POST',f'/open-apis/im/v1/messages/{message_id}/reply',
|
||||
headers={'Authorization':'Bearer '+token},json={'msg_type':'interactive',
|
||||
'content':json.dumps(card,ensure_ascii=False),'uuid':delivery_id})
|
||||
result=data.get('data',{}).get('message_id')
|
||||
if not isinstance(result,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',result):raise PlatformError('card receipt invalid')
|
||||
return result
|
||||
|
||||
async def update_task_card(self,message_id,card,*,token):
|
||||
import re
|
||||
if not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',message_id):raise PlatformError('invalid card reference')
|
||||
await self._json('PATCH',f'/open-apis/im/v1/messages/{message_id}',
|
||||
headers={'Authorization':'Bearer '+token},json={'content':json.dumps(card,ensure_ascii=False)})
|
||||
|
||||
async def download_message_file(self,message_id,file_key,max_bytes):
|
||||
import re
|
||||
if not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',message_id)or not re.fullmatch(r'[A-Za-z0-9_-]{1,256}',file_key):raise PlatformError('invalid file reference')
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Durable private task status cards. No task body, credentials or approval buttons."""
|
||||
import hashlib
|
||||
import json
|
||||
import time
|
||||
|
||||
|
||||
EXECUTION={'queued':'排队中','running':'执行中','waiting_for_approval':'等待本人确认',
|
||||
'completed':'执行完成','failed':'执行失败','cancelled':'已取消','interrupted':'已中断','unknown':'状态待核对'}
|
||||
DELIVERY={None:'尚未接收结果','ready':'已接收结果,等待投递','sending':'正在投递结果',
|
||||
'replied':'结果已发送到当前会话','unknown':'结果投递待核对','discarded':'结果已停止交付'}
|
||||
|
||||
|
||||
def card_content(task_id,execution,delivery,files=None,connection_lost=False,remote_approvals=False):
|
||||
execution=execution if execution in EXECUTION else 'unknown'
|
||||
color='green' if execution=='completed' else 'red' if execution in {'failed','interrupted'} else 'orange' if execution in {'waiting_for_approval','unknown'} else 'blue'
|
||||
if execution=='cancelled':color='grey'
|
||||
label=EXECUTION[execution]
|
||||
if connection_lost and execution not in {'completed','failed','cancelled','interrupted'}:
|
||||
color='red';label='客户端失联,任务结果待核对'
|
||||
elements=[{'tag':'div','text':{'tag':'plain_text','content':'任务状态:'+label}},
|
||||
{'tag':'div','text':{'tag':'plain_text','content':'文字回复:'+DELIVERY.get(delivery,'状态待核对')}},
|
||||
{'tag':'note','elements':[{'tag':'plain_text','content':'任务编号:'+task_id}]}]
|
||||
if execution=='waiting_for_approval':
|
||||
hint=('请在本任务的操作确认卡片中核对动作和影响,选择允许本次或拒绝。'if remote_approvals else
|
||||
'请在客户端查看并确认具体操作。本卡片暂不提供远程批准。')
|
||||
elements.insert(1,{'tag':'div','text':{'tag':'plain_text','content':hint}})
|
||||
if execution=='completed' and delivery!='replied':
|
||||
elements.insert(1,{'tag':'div','text':{'tag':'plain_text','content':'任务已执行完成,结果交付请看下方状态。'}})
|
||||
return {'config':{'wide_screen_mode':True,'enable_forward':False},
|
||||
'header':{'template':color,'title':{'tag':'plain_text','content':'海诺任务进度'}},'elements':elements}
|
||||
|
||||
|
||||
class TaskCards:
|
||||
def __init__(self,state,platform,worker_id,clock=time.time,files_enabled=False,remote_approvals=False):
|
||||
self.state,self.db,self.platform,self.worker_id,self.clock=state,state.db,platform,worker_id,clock
|
||||
self.files_enabled=files_enabled
|
||||
self.remote_approvals=remote_approvals
|
||||
with self.db:
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_task_connection_v3(
|
||||
task_id TEXT PRIMARY KEY,reason TEXT NOT NULL)''')
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_task_cards_v3(
|
||||
task_id TEXT PRIMARY KEY,message_id TEXT NOT NULL,digest TEXT NOT NULL,
|
||||
status TEXT NOT NULL,worker_id TEXT NOT NULL,updated_at DOUBLE PRECISION NOT NULL)''')
|
||||
|
||||
def mark_connection_lost(self,task_id):
|
||||
with self.db:
|
||||
self.db.execute("INSERT INTO work_task_connection_v3 VALUES(?,'client_idle_timeout') ON CONFLICT DO NOTHING",(task_id,))
|
||||
|
||||
def clear_connection_lost(self,task_id):
|
||||
with self.db:self.db.execute('DELETE FROM work_task_connection_v3 WHERE task_id=?',(task_id,))
|
||||
|
||||
def valid(self,task):
|
||||
device=self.state.store.device(task['host_id']) if task else None
|
||||
return bool(device and not device['revoked'] and device['owner']==task['owner']
|
||||
and device['binding_generation']==task['binding_generation'] and task['source_message_id'] and task['chat_id'])
|
||||
|
||||
def snapshot(self,task):
|
||||
row=self.db.execute('SELECT execution_state FROM work_execution_state_v3 WHERE task_id=?',(task['turn_id'],)).fetchone()
|
||||
delivery=self.db.execute('SELECT status FROM work_result_outbox_v2 WHERE task_id=?',(task['turn_id'],)).fetchone()
|
||||
execution=row['execution_state'] if row else 'unknown'
|
||||
if task['status']=='unknown' and execution not in {'completed','failed','cancelled','interrupted'}:execution='unknown'
|
||||
if task['deadline']<=self.clock() and execution not in {'completed','failed','cancelled','interrupted'}:execution='unknown'
|
||||
lost=self.db.execute('SELECT reason FROM work_task_connection_v3 WHERE task_id=?',(task['turn_id'],)).fetchone()
|
||||
return card_content(task['turn_id'],execution,delivery['status'] if delivery else None,
|
||||
connection_lost=bool(lost),remote_approvals=self.remote_approvals)
|
||||
|
||||
async def flush(self):
|
||||
# Bound one maintenance pass; business result delivery remains independent.
|
||||
rows=self.db.execute('''SELECT * FROM work_tasks_v2 WHERE environment=? AND app_id=?
|
||||
AND source_message_id<>'' AND chat_id<>'' AND updated_at>? ORDER BY updated_at DESC LIMIT 100''',
|
||||
(self.state.environment,self.state.app_id,self.clock()-86400)).fetchall()
|
||||
sent=0
|
||||
for raw in rows:
|
||||
if sent>=10:break
|
||||
task=dict(raw)
|
||||
if not self.valid(task):continue
|
||||
card=self.snapshot(task)
|
||||
digest=hashlib.sha256(json.dumps(card,sort_keys=True,ensure_ascii=True).encode()).hexdigest()
|
||||
old=self.db.execute('SELECT * FROM work_task_cards_v3 WHERE task_id=?',(task['turn_id'],)).fetchone()
|
||||
if old and (old['status']=='unknown' or old['digest']==digest or self.clock()-old['updated_at']<5):continue
|
||||
if not old and task['deadline']<=self.clock():continue
|
||||
try:token=await self.platform.tenant_token()
|
||||
except Exception:continue
|
||||
with self.db:
|
||||
self.state._owner_lock(task['owner'])
|
||||
task=self.state.task(task['turn_id'])
|
||||
if not self.valid(task):continue
|
||||
old=self.db.execute('SELECT * FROM work_task_cards_v3 WHERE task_id=?',(task['turn_id'],)).fetchone()
|
||||
if old and old['status']=='sending':
|
||||
# A lost creation receipt cannot safely become another card.
|
||||
if self.clock()-old['updated_at']>=60:
|
||||
self.db.execute("UPDATE work_task_cards_v3 SET status=? WHERE task_id=?",
|
||||
('ready' if old['message_id'] else 'unknown',task['turn_id']))
|
||||
continue
|
||||
if old and (old['status']=='unknown' or old['digest']==digest or self.clock()-old['updated_at']<5):continue
|
||||
# Recompute after token/lock awaits so an older observation cannot overwrite new status.
|
||||
card=self.snapshot(task)
|
||||
digest=hashlib.sha256(json.dumps(card,sort_keys=True,ensure_ascii=True).encode()).hexdigest()
|
||||
message_id=old['message_id'] if old else ''
|
||||
if not old:
|
||||
self.db.execute('INSERT INTO work_task_cards_v3 VALUES(?,?,?,?,?,?)',
|
||||
(task['turn_id'],'','','sending',self.worker_id,self.clock()))
|
||||
else:
|
||||
self.db.execute("UPDATE work_task_cards_v3 SET status='sending',worker_id=?,updated_at=? WHERE task_id=?",
|
||||
(self.worker_id,self.clock(),task['turn_id']))
|
||||
try:
|
||||
sent+=1
|
||||
if message_id:await self.platform.update_task_card(message_id,card,token=token)
|
||||
else:
|
||||
delivery_id=hashlib.sha256(('task-card-v3\0'+self.state.environment+'\0'+self.state.app_id+'\0'+task['turn_id']).encode()).hexdigest()[:32]
|
||||
message_id=await self.platform.reply_task_card(task['source_message_id'],card,delivery_id,token=token)
|
||||
status='ready';saved_digest=digest
|
||||
except Exception:
|
||||
status='ready' if message_id else 'unknown';saved_digest=old['digest'] if old else ''
|
||||
with self.db:
|
||||
self.db.execute("UPDATE work_task_cards_v3 SET message_id=?,digest=?,status=?,updated_at=? WHERE task_id=? AND status='sending' AND worker_id=?",
|
||||
(message_id,saved_digest,status,self.clock(),task['turn_id'],self.worker_id))
|
||||
@@ -17,12 +17,20 @@ class ResultRequest(TaskRequest):
|
||||
result: str = Field(max_length=16000, repr=False)
|
||||
|
||||
|
||||
class StateRequest(TaskRequest):
|
||||
execution_state: str = Field(pattern=r'^(queued|running|waiting_for_approval|completed|failed|cancelled|interrupted|unknown)$')
|
||||
status_sequence: int = Field(ge=1, le=2147483647, strict=True)
|
||||
|
||||
|
||||
class TaskDelivery:
|
||||
def __init__(self, auth, state, broker, platform):
|
||||
self.auth,self.state,self.broker,self.platform=auth,state,broker,platform
|
||||
self.db=state.db
|
||||
self.keyring=auth.broker.store.keyring
|
||||
with self.db:
|
||||
self.db.execute('''CREATE TABLE IF NOT EXISTS work_execution_state_v3(
|
||||
task_id TEXT PRIMARY KEY,execution_state TEXT NOT NULL,
|
||||
status_sequence BIGINT NOT NULL,updated_at DOUBLE PRECISION NOT NULL)''')
|
||||
self.db.execute("""CREATE TABLE IF NOT EXISTS work_result_outbox_v2(
|
||||
task_id TEXT PRIMARY KEY,environment TEXT NOT NULL,app_id TEXT NOT NULL,
|
||||
owner TEXT NOT NULL,host_id TEXT NOT NULL,binding_generation BIGINT NOT NULL,
|
||||
@@ -30,7 +38,14 @@ class TaskDelivery:
|
||||
status TEXT NOT NULL,updated_at DOUBLE PRECISION NOT NULL)""")
|
||||
self.router=APIRouter()
|
||||
self.router.add_api_route('/work/v2/tasks/state',self.status,methods=['POST'])
|
||||
self.router.add_api_route('/work/v2/tasks/progress',self.progress,methods=['POST'])
|
||||
self.router.add_api_route('/work/v2/tasks/result',self.submit,methods=['POST'])
|
||||
self.cards=None
|
||||
if auth.s.work_task_cards_enabled:
|
||||
from .task_cards import TaskCards
|
||||
self.cards=TaskCards(state,platform,broker.worker_id,files_enabled=auth.s.work_files_enabled,
|
||||
remote_approvals=auth.s.work_remote_approvals_enabled)
|
||||
broker.task_cards=self.cards
|
||||
|
||||
def current(self, body):
|
||||
self.auth.require_binding(body.host_id,body.binding_owner,body.binding_generation)
|
||||
@@ -49,7 +64,11 @@ class TaskDelivery:
|
||||
def receipt(self,task):
|
||||
row=self.db.execute('SELECT status,digest FROM work_result_outbox_v2 WHERE task_id=?',
|
||||
(task['turn_id'],)).fetchone()
|
||||
execution=self.db.execute('SELECT execution_state,status_sequence FROM work_execution_state_v3 WHERE task_id=?',(task['turn_id'],)).fetchone()
|
||||
return {'task_id':task['turn_id'],'status':row['status'] if row else task['status'],
|
||||
'progress_protocol':'work-progress-v1',
|
||||
'execution_state':execution['execution_state'] if execution else 'unknown',
|
||||
'status_sequence':execution['status_sequence'] if execution else 0,
|
||||
'result_received':bool(row),'result_digest':row['digest'] if row else None,
|
||||
'deadline':task['deadline']}
|
||||
|
||||
@@ -57,6 +76,31 @@ class TaskDelivery:
|
||||
await self.auth.authenticate(body,request)
|
||||
task=self.current(body)
|
||||
pending=self.broker.pending.get(body.task_id)
|
||||
if pending and pending.host==body.host_id:pending.progress.set()
|
||||
return no_cache(JSONResponse(self.receipt(task)))
|
||||
|
||||
async def progress(self,body:StateRequest,request:Request):
|
||||
await self.auth.authenticate(body,request)
|
||||
with self.db:
|
||||
self.state._owner_lock(body.binding_owner)
|
||||
task=self.current(body)
|
||||
if body.execution_state is not None:
|
||||
old=self.db.execute('SELECT * FROM work_execution_state_v3 WHERE task_id=?',(body.task_id,)).fetchone()
|
||||
terminal={'completed','failed','cancelled','interrupted'}
|
||||
if old and body.status_sequence==old['status_sequence'] and body.execution_state!=old['execution_state']:
|
||||
raise HTTPException(409,'execution_state_conflict')
|
||||
if not old:
|
||||
self.db.execute('INSERT INTO work_execution_state_v3 VALUES(?,?,?,?)',
|
||||
(body.task_id,body.execution_state,body.status_sequence,time.time()))
|
||||
elif body.status_sequence>old['status_sequence']:
|
||||
if old['execution_state'] in terminal and body.execution_state!=old['execution_state']:
|
||||
raise HTTPException(409,'execution_already_terminal')
|
||||
self.db.execute('UPDATE work_execution_state_v3 SET execution_state=?,status_sequence=?,updated_at=? WHERE task_id=?',
|
||||
(body.execution_state,body.status_sequence,time.time(),body.task_id))
|
||||
pending=self.broker.pending.get(body.task_id)
|
||||
if self.cards:
|
||||
old=self.db.execute('SELECT status_sequence FROM work_execution_state_v3 WHERE task_id=?',(body.task_id,)).fetchone()
|
||||
if old and body.status_sequence>=old['status_sequence']:self.cards.clear_connection_lost(body.task_id)
|
||||
if pending and pending.host==body.host_id:
|
||||
pending.progress.set()
|
||||
return no_cache(JSONResponse(self.receipt(task)))
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Prepare a private callback identity file without printing the token."""
|
||||
import argparse
|
||||
import getpass
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
|
||||
# Runs inside the EXISTING connector, not a new SDK/consumer container. All
|
||||
# stored credentials stay there. SQL is read-only; never refresh or authorize.
|
||||
DISCOVERY_CODE = r"""
|
||||
import json,re,sqlite3,time
|
||||
from urllib.parse import urlsplit
|
||||
import httpx
|
||||
from relay.config import Settings
|
||||
from relay.credential_store import CredentialStore,CredentialKeyring
|
||||
from relay.store import Postgres
|
||||
|
||||
diag={'stage':'settings','matched_bindings':0,'expired_access':0,'credential_errors':0,
|
||||
'user_mismatch':0,'missing_tenant':0,'platform_rejected':0,'network_errors':0}
|
||||
|
||||
def discover():
|
||||
s=Settings(role='connector')
|
||||
target=urlsplit(s.feishu_api_base)
|
||||
if (target.scheme!='https' or not target.hostname or target.username or target.password
|
||||
or target.path not in ('','/') or target.query or target.fragment):
|
||||
raise ValueError()
|
||||
target.port
|
||||
if s.credential_mode!='broker' or not s.credential_keyring_file:raise ValueError()
|
||||
diag['stage']='database_connect'
|
||||
if s.database_url:
|
||||
db=Postgres(s.database_url)
|
||||
db.execute('SET default_transaction_read_only TO on')
|
||||
db.execute("SET statement_timeout TO '2000ms'")
|
||||
else:
|
||||
db=sqlite3.connect((s.data_dir/'metadata.sqlite').resolve().as_uri()+'?mode=ro',uri=True,timeout=2)
|
||||
db.row_factory=sqlite3.Row
|
||||
try:
|
||||
diag['stage']='credentials_query'
|
||||
rows=db.execute('''SELECT c.principal_id FROM work_credentials_v2 c
|
||||
JOIN devices d ON d.owner=c.principal_id
|
||||
WHERE c.environment=? AND c.app_id=? AND d.revoked=0
|
||||
ORDER BY c.updated_at DESC LIMIT 3''',(s.environment,s.feishu_app_id)).fetchall()
|
||||
diag['matched_bindings']=len(rows)
|
||||
# Reuse the existing getter WITHOUT running its schema constructor.
|
||||
diag['stage']='keyring'
|
||||
credentials=CredentialStore.__new__(CredentialStore)
|
||||
credentials.db=db;credentials.keyring=CredentialKeyring.from_file(s.credential_keyring_file)
|
||||
with httpx.Client(base_url=s.feishu_api_base.rstrip('/'),trust_env=False,
|
||||
follow_redirects=False,timeout=httpx.Timeout(3,connect=2))as client:
|
||||
for row in rows:
|
||||
try:
|
||||
diag['stage']='credential_read'
|
||||
value=credentials.get(s.environment,s.feishu_app_id,row['principal_id'])
|
||||
if not value or value.access_expires_at<=time.time()+10:
|
||||
diag['expired_access']+=1
|
||||
continue
|
||||
diag['stage']='user_info'
|
||||
with client.stream('GET','/open-apis/authen/v1/user_info',
|
||||
headers={'Authorization':'Bearer '+value.access_token})as response:
|
||||
diag['http_status']=response.status_code
|
||||
if response.status_code!=200:
|
||||
diag['platform_rejected']+=1
|
||||
continue
|
||||
body=bytearray()
|
||||
for chunk in response.iter_bytes():
|
||||
body.extend(chunk)
|
||||
if len(body)>65536:raise ValueError()
|
||||
data=json.loads(body)
|
||||
info=data.get('data',{})
|
||||
if type(data.get('code'))is int:diag['platform_code']=data['code']
|
||||
if data.get('code')!=0 or not isinstance(info,dict):
|
||||
diag['platform_rejected']+=1
|
||||
continue
|
||||
if info.get('open_id')!=row['principal_id']:
|
||||
diag['user_mismatch']+=1
|
||||
continue
|
||||
tenant=info.get('tenant_key')
|
||||
if not isinstance(tenant,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',tenant):
|
||||
diag['missing_tenant']+=1
|
||||
continue
|
||||
return {'status':'verified','tenant_key':tenant,'app_id':s.feishu_app_id,'environment':s.environment}
|
||||
except httpx.HTTPError as error:
|
||||
diag['network_errors']+=1
|
||||
diag['error_class']=type(error).__name__
|
||||
continue
|
||||
except Exception as error:
|
||||
diag['credential_errors']+=1
|
||||
diag['error_class']=type(error).__name__
|
||||
continue
|
||||
raise ValueError()
|
||||
finally:db.close()
|
||||
|
||||
try:print(json.dumps(discover()))
|
||||
except Exception as error:
|
||||
if not any(diag.get(k)for k in ('expired_access','credential_errors','user_mismatch','missing_tenant','platform_rejected','network_errors')):
|
||||
diag['error_class']=type(error).__name__
|
||||
print(json.dumps({'status':'tenant_discovery_unavailable','diagnostic':diag}))
|
||||
"""
|
||||
|
||||
|
||||
def safe_diagnostic(value):
|
||||
if not isinstance(value,dict):return {}
|
||||
result={}
|
||||
if value.get('stage')in {'settings','database_connect','credentials_query','keyring','credential_read','user_info'}:
|
||||
result['stage']=value['stage']
|
||||
if value.get('error_class')in {'ValidationError','OperationalError','ProgrammingError','UndefinedTable',
|
||||
'UndefinedColumn','CredentialError','PermissionError','FileNotFoundError','ReadOnlySqlTransaction',
|
||||
'ConnectError','ConnectTimeout','ReadTimeout','SSLError','ValueError','TypeError'}:
|
||||
result['error_class']=value['error_class']
|
||||
for key in ('matched_bindings','expired_access','credential_errors','user_mismatch','missing_tenant',
|
||||
'platform_rejected','network_errors','http_status','platform_code'):
|
||||
number=value.get(key)
|
||||
if type(number)is int and 0<=number<=2147483647:result[key]=number
|
||||
return result
|
||||
|
||||
|
||||
def discover_tenant(container,env):
|
||||
result=subprocess.run(['docker','exec','-i',container,'python','-'],input=DISCOVERY_CODE,
|
||||
stdout=subprocess.PIPE,stderr=subprocess.PIPE,universal_newlines=True,timeout=25)
|
||||
try:
|
||||
if result.returncode or len(result.stdout)>4096:
|
||||
print('TENANT_DIAGNOSTIC '+json.dumps({'stage':'container_execution','exit_code':result.returncode}))
|
||||
raise ValueError()
|
||||
value=json.loads(result.stdout)
|
||||
if isinstance(value,dict)and value.get('status')=='tenant_discovery_unavailable':
|
||||
print('TENANT_DIAGNOSTIC '+json.dumps(safe_diagnostic(value.get('diagnostic'))))
|
||||
raise ValueError()
|
||||
if (set(value)!={'status','tenant_key','app_id','environment'}or value['status']!='verified'
|
||||
or value['app_id']!=env.get('HN_FEISHU_APP_ID')
|
||||
or value['environment']!=env.get('HN_ENVIRONMENT','production')
|
||||
or not isinstance(value['tenant_key'],str)
|
||||
or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',value['tenant_key'])):
|
||||
raise ValueError()
|
||||
return value['tenant_key']
|
||||
except (ValueError,TypeError,KeyError):
|
||||
raise ValueError('tenant discovery unavailable; preserve current services and check Work authorization')from None
|
||||
|
||||
|
||||
def main():
|
||||
parser=argparse.ArgumentParser()
|
||||
parser.add_argument('--output')
|
||||
parser.add_argument('--diagnose',action='store_true',help='Read-only discovery check; no output file or token prompt.')
|
||||
parser.add_argument('--container',default='hainuo-relay-connector-1')
|
||||
args=parser.parse_args();path=Path(args.output)if args.output else None
|
||||
if not args.diagnose and(path is None or not path.is_absolute()or path.exists()or path.is_symlink()):
|
||||
raise ValueError('output must be a new absolute file')
|
||||
# Keep existing fields in process; never print docker Env or token values.
|
||||
result=subprocess.run(['docker','inspect',args.container,'--format','{{json .Config.Env}}'],
|
||||
stdout=subprocess.PIPE,stderr=subprocess.PIPE,universal_newlines=True)
|
||||
if result.returncode:raise ValueError('cannot inspect current connector')
|
||||
env=dict(item.split('=',1)for item in json.loads(result.stdout)if '='in item)
|
||||
tenant=discover_tenant(args.container,env)
|
||||
if args.diagnose:
|
||||
print('TENANT_DIAGNOSTIC '+json.dumps({'status':'verified',
|
||||
'verification_token_configured':bool(env.get('HN_FEISHU_VERIFICATION_TOKEN'))}))
|
||||
return
|
||||
configured=env.get('HN_FEISHU_TENANT_KEY')
|
||||
if configured and configured!=tenant:
|
||||
raise ValueError('configured tenant does not match verified platform identity')
|
||||
token=env.get('HN_FEISHU_VERIFICATION_TOKEN')or getpass.getpass('请输入应用“事件与回调”中的 Verification Token(输入不显示):')
|
||||
if (not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',tenant)or not 8<=len(token)<=512
|
||||
or any(ord(c)<33 or ord(c)>126 for c in token)):
|
||||
raise ValueError('invalid callback identity; no values displayed')
|
||||
fd=os.open(str(path),os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
|
||||
with os.fdopen(fd,'w',encoding='utf-8')as stream:
|
||||
json.dump({'tenant_key':tenant,'verification_token':token},stream)
|
||||
stream.write('\n')
|
||||
print('Tenant verified automatically from existing authorization; callback identity prepared privately. No services changed.')
|
||||
|
||||
|
||||
if __name__=='__main__':
|
||||
try:main()
|
||||
except (ValueError,OSError,EOFError,KeyboardInterrupt,subprocess.TimeoutExpired):
|
||||
raise SystemExit('STOP: callback identity not verified; current services preserved. Check Work authorization or platform connectivity; do not guess tenant_key.')from None
|
||||
@@ -180,14 +180,33 @@ def router_public_key(config):
|
||||
raise DeployError('router connector public key is not a verified read-only trust mount')
|
||||
|
||||
|
||||
def validate_overlay(base, merged, candidate, router_key_required=True):
|
||||
def targets(plan):
|
||||
return ('connector','router','longconn')if plan.get('remote_approvals_managed')else('connector','router')
|
||||
|
||||
|
||||
def target_tag(plan,name):
|
||||
return plan['longconn_candidate_image']if name=='longconn'else plan['candidate_image']
|
||||
|
||||
|
||||
def receipt_file(root,name):
|
||||
return root/('candidate-longconn.json'if name=='longconn'else'candidate.json')
|
||||
|
||||
|
||||
def validate_overlay(base, merged, candidate, router_key_required=True, approval_plan=None):
|
||||
old, new = base["services"], merged["services"]
|
||||
require_services(old)
|
||||
require_services(new)
|
||||
for name in set(old)-{"connector","router"}:
|
||||
managed=targets(approval_plan or {})
|
||||
for name in set(old)-set(managed):
|
||||
if old[name] != new[name]:
|
||||
raise DeployError("overlay changes an unrelated relay service: "+name)
|
||||
for name in ("connector","router"):
|
||||
for name in managed:
|
||||
if name=='longconn':
|
||||
expected_env=dict(old[name].get('environment',{}))
|
||||
expected_env.update((approval_plan or {})['longconn_approval_environment'])
|
||||
expected=dict(old[name]);expected.update(image=approval_plan['longconn_candidate_image'],environment=expected_env)
|
||||
if new[name]!=expected:raise DeployError('approval overlay changes unrelated longconn settings')
|
||||
continue
|
||||
for field in set(old[name]) | set(new[name]):
|
||||
if field not in {"environment","image","volumes"} and old[name].get(field) != new[name].get(field):
|
||||
raise DeployError("overlay changes non-migration service settings")
|
||||
@@ -196,7 +215,7 @@ def validate_overlay(base, merged, candidate, router_key_required=True):
|
||||
env = new[name]["environment"]
|
||||
for key, value in old[name].get("environment", {}).items():
|
||||
if key not in {"HN_CREDENTIAL_MODE","HN_ROUTER_ADMISSION_SECRET","HN_CREDENTIAL_KEYRING_FILE",
|
||||
"HN_CREDENTIAL_PERMISSION_PROFILES","HN_CONNECTOR_INTERNAL_URL","HN_FEISHU_APP_ID","HN_OAUTH_ENABLED","HN_WORK_FILES_ENABLED"}:
|
||||
"HN_CREDENTIAL_PERMISSION_PROFILES","HN_CONNECTOR_INTERNAL_URL","HN_FEISHU_APP_ID","HN_OAUTH_ENABLED","HN_WORK_FILES_ENABLED","HN_WORK_TASK_CARDS_ENABLED","HN_WORK_REMOTE_APPROVALS_ENABLED","HN_FEISHU_TENANT_KEY","HN_FEISHU_VERIFICATION_TOKEN","HN_APPROVAL_RETENTION_ENABLED"}:
|
||||
if env.get(key) != value:
|
||||
raise DeployError("existing environment changed: "+key)
|
||||
old_volumes = {v["target"]:v for v in old[name].get("volumes", [])}
|
||||
@@ -219,6 +238,14 @@ def validate_overlay(base, merged, candidate, router_key_required=True):
|
||||
# namespace rather than silently converting it to production. All modes still
|
||||
# require PostgreSQL and HTTPS/WSS for this existing-server migration.
|
||||
require_platform(env)
|
||||
approval_keys={'HN_WORK_REMOTE_APPROVALS_ENABLED','HN_FEISHU_TENANT_KEY','HN_FEISHU_VERIFICATION_TOKEN','HN_APPROVAL_RETENTION_ENABLED'}
|
||||
for name in ('connector','router'):
|
||||
before=old[name].get('environment',{});after=new[name]['environment']
|
||||
if name=='connector'and approval_plan and approval_plan.get('remote_approvals_managed'):
|
||||
wanted=approval_plan['connector_approval_environment']
|
||||
if any(after.get(k)!=v for k,v in wanted.items()):raise DeployError('approval identity mismatch')
|
||||
elif any(after.get(k)!=before.get(k)for k in approval_keys):
|
||||
raise DeployError('unreviewed approval environment change')
|
||||
for key in ("HN_ADMIN_SECRET","HN_INTERNAL_SECRET","HN_FEISHU_APP_SECRET"):
|
||||
if env["HN_ROUTER_ADMISSION_SECRET"] == env.get(key):
|
||||
raise DeployError("router admission credential must be independent")
|
||||
@@ -375,6 +402,33 @@ def prepare_update(args):
|
||||
raise DeployError('missing update source')
|
||||
if getattr(args,'enable_work_files',False)and not (source/'relay/work_files.py').is_file():
|
||||
raise DeployError('work files implementation missing from update source')
|
||||
if getattr(args,'enable_task_cards',False)and not (source/'relay/task_cards.py').is_file():
|
||||
raise DeployError('task cards implementation missing from update source')
|
||||
enable_approvals=bool(getattr(args,'enable_remote_approvals',False))
|
||||
if parent.get('remote_approvals_managed')and not enable_approvals:
|
||||
raise DeployError('approval parent requires explicit --enable-remote-approvals and callback identity on each update')
|
||||
approval_identity=None
|
||||
if enable_approvals:
|
||||
if any(not (source/'relay'/name).is_file()for name in ('approval_api.py','approval_worker.py','approval_callback.py')):
|
||||
raise DeployError('remote approval implementation missing')
|
||||
identity_path=Path(args.callback_identity)
|
||||
if identity_path.is_symlink()or not identity_path.is_file()or identity_path.stat().st_size>8192:
|
||||
raise DeployError('invalid private callback identity file')
|
||||
if os.name!='nt'and identity_path.stat().st_mode&0o077:
|
||||
raise DeployError('callback identity file must be private (chmod 600)')
|
||||
approval_identity=json.loads(identity_path.read_text(encoding='utf-8'))
|
||||
if (not isinstance(approval_identity,dict)or set(approval_identity)!={'tenant_key','verification_token'}
|
||||
or not isinstance(approval_identity['tenant_key'],str)
|
||||
or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',approval_identity['tenant_key'])
|
||||
or not isinstance(approval_identity['verification_token'],str)
|
||||
or not 8<=len(approval_identity['verification_token'])<=512
|
||||
or any(ord(c)<33 or ord(c)>126 for c in approval_identity['verification_token'])):
|
||||
raise DeployError('invalid callback identity fields; values withheld')
|
||||
live=run(compose(parent,root/'compose.broker.json')+['config','--format','json'],True)
|
||||
a=live['services']['connector']['environment'];b=live['services']['longconn']['environment']
|
||||
if (a.get('HN_FEISHU_APP_ID')!=b.get('HN_FEISHU_APP_ID')or a.get('HN_INTERNAL_SECRET')!=b.get('HN_INTERNAL_SECRET')
|
||||
or b.get('HN_FEISHU_EVENT_SOURCE')!='longconn'):
|
||||
raise DeployError('existing unique longconn does not match connector identity')
|
||||
output=output.resolve()
|
||||
output.mkdir(mode=0o700)
|
||||
os.chmod(output,0o700)
|
||||
@@ -392,9 +446,34 @@ def prepare_update(args):
|
||||
if mount['target']=='/run/hainuo-credentials/keyring.json':
|
||||
mount['source']=str(output/'credential-keyring.json')
|
||||
rollback=json.loads(json.dumps(overlay))
|
||||
if enable_approvals:
|
||||
plan['remote_approvals_managed']=True
|
||||
plan['longconn_candidate_image']='hainuo-relay:approval-longconn-'+plan_id
|
||||
# Keep separate runtime ancestry: the connector image may lack lark-oapi.
|
||||
plan['longconn_base_image']=roles['longconn']['image']
|
||||
shared={'HN_WORK_REMOTE_APPROVALS_ENABLED':'true',
|
||||
'HN_FEISHU_TENANT_KEY':approval_identity['tenant_key'],
|
||||
'HN_FEISHU_VERIFICATION_TOKEN':approval_identity['verification_token']}
|
||||
# Retention remains disabled for this candidate even if a deployment
|
||||
# environment contains an unrelated setting. Enabling needs review.
|
||||
plan['connector_approval_environment']={**shared,'HN_APPROVAL_RETENTION_ENABLED':'false'}
|
||||
plan['longconn_approval_environment']={**shared,'HN_CREDENTIAL_MODE':'broker'}
|
||||
overlay['services']['connector']['environment'].update(plan['connector_approval_environment'])
|
||||
overlay['services']['connector']['environment']['HN_WORK_TASK_CARDS_ENABLED']='true'
|
||||
plan['work_task_cards_enabled']=True
|
||||
original_long=live['services']['longconn']
|
||||
overlay['services']['longconn']={'image':plan['longconn_candidate_image'],
|
||||
'environment':plan['longconn_approval_environment']}
|
||||
defaults={'HN_WORK_REMOTE_APPROVALS_ENABLED':'false','HN_CREDENTIAL_MODE':'legacy',
|
||||
'HN_FEISHU_TENANT_KEY':'','HN_FEISHU_VERIFICATION_TOKEN':''}
|
||||
rollback['services']['longconn']={'image':roles['longconn']['image'],
|
||||
'environment':{k:original_long.get('environment',{}).get(k,defaults[k])for k in plan['longconn_approval_environment']}}
|
||||
if getattr(args,'enable_work_files',False):
|
||||
overlay['services']['connector']['environment']['HN_WORK_FILES_ENABLED']='true'
|
||||
plan['work_files_enabled']=True
|
||||
if getattr(args,'enable_task_cards',False):
|
||||
overlay['services']['connector']['environment']['HN_WORK_TASK_CARDS_ENABLED']='true'
|
||||
plan['work_task_cards_enabled']=True
|
||||
write_json(output/'compose.broker.json',overlay)
|
||||
for name in ('connector','router'):
|
||||
rollback['services'][name]['image']=roles[name]['image']
|
||||
@@ -469,14 +548,24 @@ def deployment_state(root,plan):
|
||||
state = entry.get("state",{})
|
||||
status = state.get("status") if state.get("status") in {'created','running','paused','restarting','removing','exited','dead'} else 'unknown'
|
||||
health = state.get("health") if state.get("health") in {'none','starting','healthy','unhealthy'} else 'unknown'
|
||||
image = 'candidate' if expected and entry['image']==expected else (
|
||||
role_expected=expected
|
||||
if name=='longconn'and plan.get('remote_approvals_managed')and receipt_file(root,name).is_file():
|
||||
role_expected=json.loads(receipt_file(root,name).read_text(encoding='utf-8')).get('image')
|
||||
image = 'candidate' if role_expected and entry['image']==role_expected else (
|
||||
'original' if entry['image']==plan['roles'][name]['image'] else 'different')
|
||||
numbers = {k:v for k,v in state.items() if k in {'exit_code','restart_count'} and type(v) is int}
|
||||
print(name+': image='+image+' status='+status+' health='+health+' '+json.dumps(numbers,sort_keys=True))
|
||||
if name in {'connector','router'}:
|
||||
if name in targets(plan):
|
||||
raw = run(['docker','logs','--tail','100',entry['id']],optional=True)
|
||||
if raw is not None:
|
||||
print(' startup signals: '+(', '.join(log_signals(raw)) or 'no recognized error class; raw logs withheld'))
|
||||
if name=='longconn':
|
||||
observed=False
|
||||
for line in raw.splitlines():
|
||||
try:record=json.loads(line)
|
||||
except ValueError:continue
|
||||
if isinstance(record,dict)and record.get('event')=='platform_connected':observed=True
|
||||
print(' platform_connected observed: '+('yes'if observed else'not yet; inspect again after startup'))
|
||||
print('STATE inspection only; no files, keys or containers changed. Healthy candidate services need no reactivation; inspect mixed/unhealthy states before further actions.')
|
||||
|
||||
|
||||
@@ -499,18 +588,19 @@ def check(root, plan, baseline=True, router_key_required=True):
|
||||
if digest(root/path) != expected:
|
||||
raise DeployError("prepared keyring or overlay changed; do not regenerate credential keys")
|
||||
for name, old in plan["roles"].items():
|
||||
current = service(plan["project"],name,include_stopped=not baseline and name in {"connector","router"})
|
||||
if (baseline or name not in {"connector","router"}) and current["image"] != old["image"]:
|
||||
current = service(plan["project"],name,include_stopped=not baseline and name in targets(plan))
|
||||
if (baseline or name not in targets(plan)) and current["image"] != old["image"]:
|
||||
raise DeployError("running service image changed since preparation: "+name+"; this plan may already be activated. Run state; do not rebuild or reactivate a healthy candidate")
|
||||
if not baseline and name in {"connector","router"}:
|
||||
if not baseline and name in targets(plan):
|
||||
allowed = {old['image']}
|
||||
if (root/'candidate.json').is_file():
|
||||
allowed.add(json.loads((root/'candidate.json').read_text(encoding='utf-8'))['image'])
|
||||
receipt=receipt_file(root,name)
|
||||
if receipt.is_file():
|
||||
allowed.add(json.loads(receipt.read_text(encoding='utf-8'))['image'])
|
||||
if current['image'] not in allowed:
|
||||
raise DeployError('unreviewed target image; selected parent plan does not match running '+name+'; use the latest successfully activated plan for a new update. No services changed')
|
||||
base = run(compose(plan)+["config","--format","json"],True)
|
||||
merged = run(compose(plan,root/"compose.broker.json")+["config","--format","json"],True)
|
||||
validate_overlay(base,merged,plan["candidate_image"],router_key_required=router_key_required)
|
||||
validate_overlay(base,merged,plan["candidate_image"],router_key_required=router_key_required,approval_plan=plan)
|
||||
# Require the current edge's explicit private-path block, before catch-all handling.
|
||||
mounts = base["services"]["edge"].get("volumes",[])
|
||||
caddy = next((v["source"] for v in mounts if v.get("target") == "/etc/caddy/Caddyfile" and v.get("type") == "bind"),None)
|
||||
@@ -536,8 +626,17 @@ def build(root,plan):
|
||||
image = run(["docker","image","inspect",plan["candidate_image"],"--format","{{.Id}}"])
|
||||
run(["docker","run","--rm","--network","none","--read-only","--entrypoint","python",
|
||||
plan["candidate_image"],"-c","import relay.credential_api, relay.binding_state, relay.device_lifecycle"])
|
||||
long_image=None
|
||||
if plan.get('remote_approvals_managed'):
|
||||
tag=plan['longconn_candidate_image']
|
||||
if run(['docker','image','inspect',tag],optional=True)is not None:
|
||||
raise DeployError('longconn candidate already exists; do not overwrite')
|
||||
run(['docker','build','--pull=false','--network','none','--build-arg',
|
||||
'BASE_IMAGE='+plan['longconn_base_image'],'-t',tag,str(root/'build')])
|
||||
long_image=run(['docker','image','inspect',tag,'--format','{{.Id}}'])
|
||||
runtime_check(root,plan)
|
||||
write_json(root/"candidate.json",{"image":image,"tag":plan["candidate_image"]})
|
||||
if long_image:write_json(root/'candidate-longconn.json',{'image':long_image,'tag':plan['longconn_candidate_image']})
|
||||
print("BUILT: "+plan["candidate_image"]+" (offline runtime reuse; not activated)")
|
||||
|
||||
|
||||
@@ -553,12 +652,17 @@ def runtime_check(root,plan):
|
||||
" s=Settings(role=data['role'])\n"
|
||||
" if data['role']=='connector': CredentialKeyring.from_file(s.credential_keyring_file)\n"
|
||||
" if data['role']=='router': load_public(s.connector_public_key)\n"
|
||||
" if data['role']=='connector' and s.work_remote_approvals_enabled: import relay.approval_api, relay.approval_worker\n"
|
||||
" if data['role']=='longconn':\n"
|
||||
" import lark_oapi as lark\n"
|
||||
" from lark_oapi.event.callback.model.p2_card_action_trigger import P2CardActionTriggerResponse\n"
|
||||
" lark.EventDispatcherHandler.builder('','').register_p2_card_action_trigger(lambda e:P2CardActionTriggerResponse({})).build()\n"
|
||||
"except Exception:\n sys.exit(1)\n")
|
||||
for role in ("connector","router"):
|
||||
for role in targets(plan):
|
||||
env = {key:str(value) for key,value in merged["services"][role]["environment"].items() if value is not None}
|
||||
command=["docker","run","--rm","-i","--network","none","--read-only","--entrypoint","python",
|
||||
"-v",str(root/"credential-keyring.json")+":/run/hainuo-credentials/keyring.json:ro",
|
||||
"-v",str(public_source)+":"+public_key+":ro",plan["candidate_image"],"-c",code]
|
||||
"-v",str(public_source)+":"+public_key+":ro",target_tag(plan,role),"-c",code]
|
||||
print('Runtime check: '+role+' configuration and required key files',flush=True)
|
||||
run(command,input_text=json.dumps({"role":role,"environment":env}))
|
||||
|
||||
@@ -609,12 +713,17 @@ def activate(root,plan,ready):
|
||||
image = run(["docker","image","inspect",plan["candidate_image"],"--format","{{.Id}}"])
|
||||
if built != {"image":image,"tag":plan["candidate_image"]}:
|
||||
raise DeployError("candidate image identity changed")
|
||||
if plan.get('remote_approvals_managed'):
|
||||
tag=plan['longconn_candidate_image']
|
||||
saved=json.loads((root/'candidate-longconn.json').read_text(encoding='utf-8'))
|
||||
actual=run(['docker','image','inspect',tag,'--format','{{.Id}}'])
|
||||
if saved!={'image':actual,'tag':tag}:raise DeployError('longconn candidate image identity changed')
|
||||
print('ACTIVATION step: isolated candidate configuration/keyring check (no service switch)',flush=True)
|
||||
runtime_check(root,plan)
|
||||
print('ACTIVATION step: recreate connector/router; failures may leave a partial switch, inspect state',flush=True)
|
||||
print('ACTIVATION step: recreate '+','.join(targets(plan))+'; failures may leave a partial switch, inspect state',flush=True)
|
||||
run(compose(plan,root/"compose.broker.json")+["up","-d","--no-deps","--no-build","--pull","never",
|
||||
"--wait","--wait-timeout","180","connector","router"])
|
||||
print("ACTIVATED: connector/router only. Verify real OAuth, v2 client, unbind and task routing.")
|
||||
"--wait","--wait-timeout","180",*targets(plan)])
|
||||
print('ACTIVATED: '+','.join(targets(plan))+' only. Verify real platform connection and task routing.')
|
||||
|
||||
|
||||
def container_permission_profiles(container):
|
||||
@@ -714,6 +823,9 @@ def main():
|
||||
update.add_argument("--source-root",default=str(Path(__file__).resolve().parents[1]))
|
||||
update.add_argument("--output",required=True)
|
||||
update.add_argument('--enable-work-files',action='store_true',help='Opt in after paired-client preparation; rollback retains the previous flag.')
|
||||
update.add_argument('--enable-task-cards',action='store_true',help='Opt in to paired-client status cards; rollback retains the previous flag.')
|
||||
update.add_argument('--enable-remote-approvals',action='store_true',help='Update the existing longconn too; needs private callback identity file.')
|
||||
update.add_argument('--callback-identity',help='Private JSON file with tenant_key and verification_token; never paste values into command arguments.')
|
||||
permissions=sub.add_parser('update-permissions')
|
||||
permissions.add_argument('--plan',required=True)
|
||||
permissions.add_argument('--profiles',required=True)
|
||||
@@ -758,7 +870,7 @@ def main():
|
||||
raise DeployError("rollback requires stopping broker clients; old refresh credentials are not restored")
|
||||
check(root,plan,baseline=False,router_key_required=False)
|
||||
run(compose(plan,root/"compose.rollback.json")+["up","-d","--no-deps","--no-build","--pull","never",
|
||||
"--wait","--wait-timeout","180","connector","router"])
|
||||
"--wait","--wait-timeout","180",*targets(plan)])
|
||||
print("ROLLED BACK containers only; preserve keyring/database. Client mode and reauthorization must be coordinated.")
|
||||
except (DeployError,OSError,ValueError,KeyError,TypeError):
|
||||
error = sys.exc_info()[1]
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
"""M6 audit and cleanup on temporary encrypted approval stores only."""
|
||||
import json
|
||||
import pytest
|
||||
from test_credential_api import env
|
||||
from test_approval_integration import approval_env,pair
|
||||
from relay.approval_store import ApprovalStore,ApprovalError
|
||||
|
||||
|
||||
async def test_audit_is_minimal_and_repeated_decisions_do_not_duplicate(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']();response,body=await p['click']()
|
||||
await job
|
||||
e=p['env'];await e['client'].post('/internal/approval-callback',json=body,
|
||||
headers={'X-Internal-Secret':e['settings'].internal_secret})
|
||||
events=[dict(r)for r in e['store'].db.execute('SELECT * FROM work_approval_audit_v1').fetchall()]
|
||||
assert {r['event']for r in events}>={'created','card_bound','approved','consumed'}
|
||||
assert sum(r['event']=='approved'for r in events)==1
|
||||
assert sum(r['event']=='consumed'for r in events)==1
|
||||
encoded=json.dumps(events)
|
||||
for secret in ('ou_a','oc_synthetic','om_source','合成测试','fake-access','verification-synthetic',row['ciphertext']):
|
||||
assert secret not in encoded
|
||||
assert all(len(r['actor_scope'])==64 and len(r['device_scope'])==64 for r in events)
|
||||
assert all(r['action_class']=='work_api_write'for r in events)
|
||||
|
||||
|
||||
async def test_closed_payload_purge_keeps_replay_tombstone_and_active_payload(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']();await p['click']('deny')
|
||||
from test_approval_integration import pc_errors
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
e=p['env'];now=__import__('time').time()
|
||||
with e['store'].db:e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=?',(now-31*86400,))
|
||||
api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now)
|
||||
active=api.create(('msg_'+'a'*64),'ou_a','a'*32,1,'operation-active',{'kind':'synthetic'},'合成动作','仅测试')
|
||||
with e['store'].db:e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=? WHERE approval_id=?',
|
||||
(now-32*86400,active['approval_id']))
|
||||
result=api.maintain_retention()
|
||||
assert result['payloads_purged']==1 and result['tombstones_preserved']
|
||||
saved=e['store'].db.execute('SELECT * FROM work_approvals_v3').fetchone()
|
||||
assert saved['state']=='rejected'and saved['ciphertext']==''and saved['nonce_digest']==''
|
||||
assert api.maintain_retention()['payloads_purged']==0
|
||||
status=api.status(row['approval_id'],('msg_'+'a'*64),'ou_a','a'*32,1,row['operation_digest'])
|
||||
assert status['state']=='rejected'
|
||||
assert api.consume(row['approval_id'],('msg_'+'a'*64),'ou_a','a'*32,1,row['operation_digest'])['execute']is False
|
||||
action=json.loads(e['api'].broker.store.keyring.decrypt(row['key_id'],row['ciphertext'],api._aad(row)))['action']
|
||||
assert api.create(('msg_'+'a'*64),'ou_a','a'*32,1,'operation-synthetic',action,'同动作','不得重新放行')['state']=='rejected'
|
||||
# A pre-existing pending approval is not selected even when timestamp is old.
|
||||
assert api.maintain_retention()['payloads_purged']==0
|
||||
assert e['store'].db.execute('SELECT ciphertext FROM work_approvals_v3 WHERE approval_id=?',
|
||||
(active['approval_id'],)).fetchone()['ciphertext']
|
||||
|
||||
|
||||
async def test_retention_is_environment_scoped_and_bounded(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']();await p['click']('deny')
|
||||
from test_approval_integration import pc_errors
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
e=p['env'];now=__import__('time').time()
|
||||
api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now)
|
||||
with e['store'].db:
|
||||
e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=?',(now-100*86400,))
|
||||
e['store'].db.execute('UPDATE work_approval_audit_v1 SET created_at=?',(now-100*86400,))
|
||||
result=api.maintain_retention(limit=1)
|
||||
assert result['audit_events_removed']==1
|
||||
assert e['store'].db.execute('SELECT COUNT(*) n FROM work_approvals_v3').fetchone()['n']==1
|
||||
assert e['store'].db.execute('SELECT COUNT(*) n FROM work_approval_callbacks_v1').fetchone()['n']==1
|
||||
for kwargs in ({'payload_days':0},{'audit_days':366},{'payload_days':91,'audit_days':90},{'limit':100}):
|
||||
with pytest.raises(ApprovalError):api.maintain_retention(**kwargs)
|
||||
|
||||
|
||||
async def test_audit_write_failure_rolls_back_consume_and_never_sends(pair,monkeypatch):
|
||||
from relay.approval_audit import ApprovalAudit
|
||||
from test_approval_integration import pc_errors
|
||||
original=ApprovalAudit.record
|
||||
def fail_consume(self,row,event,action_class=None):
|
||||
if event=='consumed':raise RuntimeError('synthetic audit storage failure')
|
||||
return original(self,row,event,action_class)
|
||||
monkeypatch.setattr(ApprovalAudit,'record',fail_consume)
|
||||
p=pair;job=p['launch']();await p['click']()
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
e=p['env']
|
||||
assert not p['calls']
|
||||
assert e['store'].db.execute("SELECT state FROM work_approvals_v3").fetchone()['state']!='consumed'
|
||||
assert not e['store'].db.execute("SELECT * FROM work_approval_audit_v1 WHERE event='consumed'").fetchall()
|
||||
|
||||
|
||||
async def test_retention_never_touches_another_environment(pair):
|
||||
from test_approval_integration import pc_errors
|
||||
p=pair;job=p['launch']();await p['click']('deny')
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
e=p['env'];now=__import__('time').time()
|
||||
with e['store'].db:e['store'].db.execute("UPDATE work_approvals_v3 SET environment='development',updated_at=?",(now-100*86400,))
|
||||
api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now)
|
||||
assert api.maintain_retention()['payloads_purged']==0
|
||||
assert e['store'].db.execute('SELECT ciphertext FROM work_approvals_v3').fetchone()['ciphertext']
|
||||
|
||||
|
||||
def test_retention_configuration_is_explicit_and_ordered():
|
||||
from relay.config import Settings
|
||||
from pydantic import ValidationError
|
||||
settings=Settings(role='test',environment='test')
|
||||
assert settings.approval_retention_enabled is False
|
||||
assert (settings.approval_payload_retention_days,settings.approval_audit_retention_days)==(30,90)
|
||||
for changes in ({'approval_retention_enabled':True},
|
||||
{'approval_payload_retention_days':91,'approval_audit_retention_days':90},
|
||||
{'approval_audit_retention_days':366}):
|
||||
with pytest.raises(ValidationError):Settings(role='test',environment='test',**changes)
|
||||
|
||||
|
||||
async def test_consumed_but_result_unknown_is_not_purged(pair):
|
||||
p=pair;job=p['launch']();await p['click']();await job
|
||||
e=p['env'];now=__import__('time').time()
|
||||
api=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring,clock=lambda:now)
|
||||
with e['store'].db:
|
||||
e['store'].db.execute('UPDATE work_approvals_v3 SET updated_at=?',(now-100*86400,))
|
||||
e['store'].db.execute("UPDATE work_tasks_v2 SET status='unknown'")
|
||||
assert api.maintain_retention()['payloads_purged']==0
|
||||
assert e['store'].db.execute('SELECT ciphertext FROM work_approvals_v3').fetchone()['ciphertext']
|
||||
with e['store'].db:e['store'].db.execute("UPDATE work_tasks_v2 SET status='completed'")
|
||||
# Bounded keyset scan wraps on the next maintenance pass.
|
||||
outcomes=[api.maintain_retention()['payloads_purged']for _ in range(2)]
|
||||
assert sum(outcomes)==1
|
||||
@@ -0,0 +1,60 @@
|
||||
import json
|
||||
import pytest
|
||||
|
||||
from test_credential_api import env
|
||||
from test_approval_integration import approval_env, pair, pc_errors
|
||||
from relay.approval_cards import decision_response_card
|
||||
from relay.longconn import safe_card_callback_response
|
||||
|
||||
|
||||
@pytest.mark.parametrize(('choice','state'),[('deny','rejected'),('allow','approved')])
|
||||
async def test_callback_returns_no_button_terminal_card_to_clicked_view(pair,choice,state):
|
||||
p=pair
|
||||
job=p['launch']()
|
||||
response,_=await p['click'](choice)
|
||||
assert response.status_code==200
|
||||
payload=response.json()
|
||||
assert payload['toast']['type']=='info'
|
||||
forwarded=safe_card_callback_response(payload)
|
||||
assert forwarded['card']==payload['card']
|
||||
assert forwarded['card']=={'type':'raw','data':decision_response_card(state)}
|
||||
card=forwarded['card']['data']
|
||||
assert card['config']['update_multi'] is True
|
||||
assert '"button"' not in json.dumps(card)
|
||||
if choice=='deny':
|
||||
with pytest.raises(pc_errors.ApprovalRejected):await job
|
||||
assert not p['calls']
|
||||
else:
|
||||
assert (await job)['status']=='completed'
|
||||
assert len(p['calls'])==1
|
||||
|
||||
|
||||
def test_callback_proxy_discards_untrusted_card_payload():
|
||||
result=safe_card_callback_response({'toast':{'type':'info','content':'ok'},
|
||||
'card':{'type':'raw','data':{'secret':'do-not-forward'}}})
|
||||
assert result=={'toast':{'type':'warning','content':'确认结果待核对,请查看任务状态。'}}
|
||||
assert 'do-not-forward' not in json.dumps(result)
|
||||
|
||||
|
||||
async def test_explicit_refresh_repairs_a_known_closed_card_without_new_message(pair):
|
||||
p=pair
|
||||
job=p['launch']();row=await p['card']();await p['click']('deny')
|
||||
with pytest.raises(pc_errors.ApprovalRejected):await job
|
||||
e=p['env'];sender=e['app'].state.approvals.sender
|
||||
await sender.update_closed(row['approval_id'])
|
||||
assert await sender.update_closed(row['approval_id'])=={'state':'no_closed_card'}
|
||||
assert await sender.update_closed(row['approval_id'],refresh=True)=={'state':'updated'}
|
||||
assert len(e['platform'].cards)==1
|
||||
assert e['store'].db.execute('SELECT state FROM work_approvals_v3').fetchone()['state']=='rejected'
|
||||
assert not p['calls']
|
||||
|
||||
|
||||
async def test_admin_repair_preview_is_scoped_and_never_sends_card(pair):
|
||||
p=pair
|
||||
job=p['launch']();row=await p['card']();await p['click']('deny')
|
||||
with pytest.raises(pc_errors.ApprovalRejected):await job
|
||||
api=p['env']['app'].state.approvals
|
||||
before=len(p['env']['platform'].cards)
|
||||
ids=await api.worker.call('closed_for_task',p['context']['task_id'])
|
||||
assert ids==[row['approval_id']]
|
||||
assert len(p['env']['platform'].cards)==before
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Automatic tenant lookup: real encrypted temp SQL, synthetic user-info API."""
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from test_credential_api import env,authorize
|
||||
|
||||
spec=importlib.util.spec_from_file_location('approval_identity_test',Path(__file__).resolve().parents[1]/'scripts/approval_identity.py')
|
||||
identity=importlib.util.module_from_spec(spec);spec.loader.exec_module(identity)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure',[None,'wrong_user','expired','permission','missing_tenant','redirect'])
|
||||
async def test_in_container_lookup_is_readonly_and_never_refreshes(env,monkeypatch,capsys,failure):
|
||||
await authorize(env)
|
||||
from relay import config
|
||||
monkeypatch.setattr(config,'Settings',lambda **kw:env['settings'])
|
||||
credentials=env['api'].broker.store
|
||||
if failure=='expired':
|
||||
old=credentials.get('test','cli_test','ou_a')
|
||||
credentials.save('test','cli_test','ou_a',expected_version=old.version,
|
||||
access_token='fake-expired-access',refresh_token='fake-refresh-private',
|
||||
access_expires_at=1,refresh_expires_at=9999999999,scopes=['read'])
|
||||
before='\n'.join(env['store'].db.iterdump());calls=[]
|
||||
def platform(request):
|
||||
calls.append(request)
|
||||
assert request.method=='GET'and request.url.path=='/open-apis/authen/v1/user_info'
|
||||
assert request.url.host=='work.example.test'
|
||||
if failure=='permission':return httpx.Response(403,json={'code':99991679})
|
||||
if failure=='redirect':return httpx.Response(302,headers={'Location':'https://unapproved.example.test'})
|
||||
data={'open_id':'ou_other'if failure=='wrong_user'else'ou_a',
|
||||
'tenant_key':'tenant_verified','name':'synthetic-private-user','email':'synthetic-private@example.test'}
|
||||
if failure=='missing_tenant':data.pop('tenant_key')
|
||||
return httpx.Response(200,json={'code':0,'data':data})
|
||||
original=httpx.Client
|
||||
monkeypatch.setattr(httpx,'Client',lambda **kwargs:original(**kwargs,transport=httpx.MockTransport(platform)))
|
||||
exec(identity.DISCOVERY_CODE,{})
|
||||
output=capsys.readouterr().out;result=json.loads(output)
|
||||
if failure is None:
|
||||
assert result=={'status':'verified','tenant_key':'tenant_verified','app_id':'cli_test','environment':'test'}
|
||||
else:
|
||||
assert result['status']=='tenant_discovery_unavailable'
|
||||
diagnostic=result['diagnostic']
|
||||
if failure=='expired':assert diagnostic['expired_access']==1
|
||||
elif failure=='wrong_user':assert diagnostic['user_mismatch']==1
|
||||
elif failure=='missing_tenant':assert diagnostic['missing_tenant']==1
|
||||
else:assert diagnostic['platform_rejected']==1
|
||||
assert len(calls)==(0 if failure=='expired'else 1)
|
||||
assert '\n'.join(env['store'].db.iterdump())==before
|
||||
assert env['platform'].refreshes==0
|
||||
assert all(secret not in output for secret in ('fake-access','fake-refresh','synthetic-private','email'))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('mutate',[{'app_id':'different'},{'environment':'production'},
|
||||
{'tenant_key':'invalid tenant'},{'status':'unavailable'},{'extra':'forbidden'}])
|
||||
def test_lookup_rejects_mismatched_container_metadata(monkeypatch,mutate):
|
||||
body={'status':'verified','tenant_key':'tenant_verified','app_id':'cli_test','environment':'test',**mutate}
|
||||
calls=[]
|
||||
def run(args,**kwargs):calls.append((args,kwargs));return SimpleNamespace(returncode=0,stdout=json.dumps(body))
|
||||
monkeypatch.setattr(identity.subprocess,'run',run)
|
||||
with pytest.raises(ValueError):identity.discover_tenant('synthetic-connector',{'HN_FEISHU_APP_ID':'cli_test','HN_ENVIRONMENT':'test'})
|
||||
assert calls[0][0]==['docker','exec','-i','synthetic-connector','python','-']
|
||||
|
||||
|
||||
def test_prepare_never_asks_user_for_tenant_or_prints_verification_token(tmp_path,monkeypatch,capsys):
|
||||
output=tmp_path/'callback.json'
|
||||
env=['HN_FEISHU_APP_ID=cli_test','HN_ENVIRONMENT=test','HN_FEISHU_VERIFICATION_TOKEN=synthetic-private-verify']
|
||||
monkeypatch.setattr(sys,'argv',
|
||||
['approval_identity.py','--output',str(output)])
|
||||
monkeypatch.setattr(identity.subprocess,'run',lambda *a,**kw:SimpleNamespace(returncode=0,stdout=json.dumps(env)))
|
||||
monkeypatch.setattr(identity,'discover_tenant',lambda *a:'tenant_verified')
|
||||
def forbidden(*a):raise AssertionError('No interactive identity fields should be requested')
|
||||
monkeypatch.setattr('builtins.input',forbidden)
|
||||
monkeypatch.setattr(identity.getpass,'getpass',forbidden)
|
||||
identity.main()
|
||||
assert json.loads(output.read_text())=={'tenant_key':'tenant_verified','verification_token':'synthetic-private-verify'}
|
||||
assert 'synthetic-private-verify'not in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_diagnostic_whitelist_drops_secrets_and_untrusted_strings():
|
||||
assert identity.safe_diagnostic({'stage':'user_info','http_status':403,'platform_code':99991679,
|
||||
'email':'private@example.test','access_token':'private-secret','error_class':'private-secret',
|
||||
'matched_bindings':True,'expired_access':-1})=={'stage':'user_info','http_status':403,'platform_code':99991679}
|
||||
|
||||
|
||||
def test_diagnose_does_not_prompt_or_write(tmp_path,monkeypatch,capsys):
|
||||
monkeypatch.setattr(sys,'argv',['approval_identity.py','--diagnose'])
|
||||
monkeypatch.setattr(identity.subprocess,'run',lambda *a,**kw:SimpleNamespace(returncode=0,stdout='[]'))
|
||||
monkeypatch.setattr(identity,'discover_tenant',lambda *a:'tenant_verified')
|
||||
def forbidden(*a):raise AssertionError('Diagnostic must not prompt')
|
||||
monkeypatch.setattr(identity.getpass,'getpass',forbidden)
|
||||
identity.main()
|
||||
assert '"status": "verified"'in capsys.readouterr().out
|
||||
assert not list(tmp_path.iterdir())
|
||||
|
||||
|
||||
def test_failed_discovery_reports_only_safe_stage_counts(monkeypatch,capsys):
|
||||
body={'status':'tenant_discovery_unavailable','diagnostic':{'stage':'user_info',
|
||||
'expired_access':2,'network_errors':0,'access_token':'do-not-print','name':'do-not-print'}}
|
||||
monkeypatch.setattr(identity.subprocess,'run',lambda *a,**kw:SimpleNamespace(returncode=0,stdout=json.dumps(body)))
|
||||
with pytest.raises(ValueError):identity.discover_tenant('connector',{})
|
||||
output=capsys.readouterr().out
|
||||
assert '"expired_access": 2'in output and 'do-not-print'not in output
|
||||
@@ -0,0 +1,350 @@
|
||||
"""Two-repo offline approval integration: real proofs/SQL, synthetic platform only."""
|
||||
import asyncio
|
||||
import importlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import secrets
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
|
||||
from test_credential_api import env,wire,authorize,Platform
|
||||
from relay.connector import create_app
|
||||
from relay.crypto import pem_private
|
||||
from relay.approval_store import ApprovalStore
|
||||
from relay.store import Store
|
||||
from relay.binding_state import BindingState
|
||||
from relay.approval_worker import ApprovalWorker
|
||||
|
||||
|
||||
PC_SERVICE=Path(os.environ.get('HAINUO_APPROVAL_PC_SOURCE',str(
|
||||
Path(__file__).resolve().parents[3]/'hainuo-build-phase-A-local-20261116')))/'integrations/work-relay/service/relay'
|
||||
if not (PC_SERVICE/'approval_transport.py').is_file():
|
||||
pytest.skip('paired PC source missing; set HAINUO_APPROVAL_PC_SOURCE to run joint acceptance',allow_module_level=True)
|
||||
# An explicit checkout selection keeps imports independent of cwd and installed
|
||||
# runtime. Missing paired sources are reported as skipped, never a joint pass.
|
||||
spec=importlib.util.spec_from_file_location('approval_pc',PC_SERVICE/'__init__.py',
|
||||
submodule_search_locations=[str(PC_SERVICE)])
|
||||
pc=importlib.util.module_from_spec(spec);sys.modules['approval_pc']=pc;spec.loader.exec_module(pc)
|
||||
pc_broker=importlib.import_module('approval_pc.broker_client')
|
||||
pc_gate=importlib.import_module('approval_pc.approval_transport')
|
||||
pc_config=importlib.import_module('approval_pc.release_config')
|
||||
pc_context=importlib.import_module('approval_pc.task_context')
|
||||
pc_errors=importlib.import_module('approval_pc.work_errors')
|
||||
|
||||
|
||||
class CardPlatform(Platform):
|
||||
def __init__(self):
|
||||
super().__init__();self.cards=[];self.updates=[];self.lose_card_receipt=False
|
||||
|
||||
async def reply_task_card(self,source,card,delivery_id,*,token):
|
||||
self.cards.append((source,card,delivery_id))
|
||||
if self.lose_card_receipt:raise ConnectionError('synthetic lost card receipt')
|
||||
return 'om_approval_'+str(len(self.cards))
|
||||
|
||||
async def update_task_card(self,message_id,card,*,token):
|
||||
self.updates.append((message_id,card))
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def approval_env(env):
|
||||
settings=env['settings'].model_copy(update=dict(work_remote_approvals_enabled=True,
|
||||
feishu_tenant_key='tenant_synthetic',feishu_verification_token='verification-synthetic'))
|
||||
platform=CardPlatform();app=create_app(settings,env['redis'],platform)
|
||||
async with app.router.lifespan_context(app):
|
||||
async with httpx.AsyncClient(transport=httpx.ASGITransport(app),base_url=settings.public_url)as client:
|
||||
current={**env,'settings':settings,'platform':platform,'app':app,'client':client,
|
||||
'store':app.state.store,'api':app.state.oauth}
|
||||
await authorize(current)
|
||||
yield current
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def pair(approval_env,tmp_path,monkeypatch):
|
||||
e=approval_env;directory=tmp_path/'pc';directory.mkdir()
|
||||
host='a'*32;device=e['store'].device(host)
|
||||
(directory/'server.pem').write_bytes(e['settings'].ca_cert.read_bytes())
|
||||
(directory/'device.key').write_bytes(pem_private(e['keys'][host]))
|
||||
(directory/'device.json').write_text(json.dumps({'host_id':host,'owner':'ou_a',
|
||||
'binding_generation':device['binding_generation']}))
|
||||
release=pc_config.validate_config(dict(public_url=e['settings'].public_url,credential_mode='broker',
|
||||
app_id='cli_test',environment='test',permission_profile='product-v1',
|
||||
work_api_base='https://work.example.test',remote_approvals=True))
|
||||
monkeypatch.setattr(pc_config,'load_config',lambda:release)
|
||||
monkeypatch.setattr(pc_broker,'load_config',lambda:release)
|
||||
loop=asyncio.get_running_loop();faults=set();calls=[];trace=[];stop=threading.Event()
|
||||
def forward(request):
|
||||
async def dispatch():
|
||||
response=await e['client'].request(request.method,request.url.path,
|
||||
headers=dict(request.headers),content=request.content)
|
||||
return httpx.Response(response.status_code,headers=response.headers,content=response.content,request=request)
|
||||
response=asyncio.run_coroutine_threadsafe(dispatch(),loop).result(timeout=10)
|
||||
trace.append((request.url.path,response.status_code))
|
||||
if request.url.path.endswith('/consume')and 'consume_receipt_lost'in faults:
|
||||
raise httpx.ReadTimeout('synthetic lost consume response')
|
||||
if request.url.path.endswith('/status')and '/approvals/'in request.url.path and 'poll_lost'in faults:
|
||||
raise httpx.ConnectError('synthetic offline')
|
||||
return response
|
||||
transport=httpx.MockTransport(forward)
|
||||
actual_transport=pc_gate.ApprovalTransport
|
||||
monkeypatch.setattr(pc_gate,'ApprovalTransport',lambda d,r:actual_transport(d,r,http_transport=transport))
|
||||
def platform(request):
|
||||
assert request.url.host=='work.example.test'
|
||||
assert request.headers['authorization'].startswith('Bearer fake-access')
|
||||
calls.append(request)
|
||||
if 'business_receipt_lost'in faults:raise httpx.ReadTimeout('synthetic lost business response')
|
||||
return httpx.Response(200,json={'code':0,'data':{'event':{'event_id':'synthetic-event'}}})
|
||||
client=pc_broker.BrokerWorkClient(directory,transport=transport,api_transport=httpx.MockTransport(platform))
|
||||
task=e['store'].lifecycle.create_task(('msg_'+'a'*64),'ou_a',host,device['binding_generation'],
|
||||
'synthetic-session',e['app'].state.broker.worker_id,time.time()+600,'oc_synthetic','om_source')
|
||||
context={k:task[k]for k in ('environment','app_id','owner','host_id','binding_generation',
|
||||
'conversation_id','chat_id','source_message_id')}
|
||||
context.update(task_id=task['turn_id'],channel='work-dm')
|
||||
marker=pc_context.CURRENT_TASK.set(context);control=pc_context.CURRENT_TASK_STOP.set(stop.is_set)
|
||||
jobs=[]
|
||||
def launch(operation='operation-synthetic'):
|
||||
nonlocal loop
|
||||
loop=asyncio.get_running_loop()
|
||||
job=asyncio.create_task(asyncio.to_thread(client.api,'POST',
|
||||
'/open-apis/calendar/v4/calendars/cal_synthetic/events',{},
|
||||
{'summary':'合成测试:不创建真实日程'},operation))
|
||||
jobs.append(job);return job
|
||||
async def card(bound=True):
|
||||
deadline=time.monotonic()+10
|
||||
while time.monotonic()<deadline:
|
||||
for job in jobs:
|
||||
if job.done():raise AssertionError(('task settled before card',job.result()))
|
||||
rows=e['store'].db.execute('SELECT * FROM work_approvals_v3').fetchall()
|
||||
if rows and e['platform'].cards and (not bound or rows[0]['card_message_id']):return dict(rows[0])
|
||||
await asyncio.sleep(.01)
|
||||
raise AssertionError(('synthetic approval card was not delivered/bound',trace,
|
||||
[(r['state'],bool(r['card_message_id']))for r in rows],len(e['platform'].cards)))
|
||||
async def click(choice='allow',actor='ou_a',event_id=None,mutate=None):
|
||||
row=await card();value=e['platform'].cards[0][1]['elements'][-1]['actions'][0]['value']
|
||||
body={'schema':'2.0','header':{'event_type':'card.action.trigger','app_id':'cli_test',
|
||||
'tenant_key':'tenant_synthetic','token':'verification-synthetic',
|
||||
'create_time':str(int(time.time()*1000000)),'event_id':event_id or secrets.token_hex(16)},
|
||||
'event':{'host':'im_message','operator':{'open_id':actor,'tenant_key':'tenant_synthetic'},
|
||||
'context':{'open_message_id':row['card_message_id'],'open_chat_id':'oc_synthetic'},
|
||||
'action':{'tag':'button','value':{**value,'choice':choice}}}}
|
||||
if mutate:mutate(body)
|
||||
response=await e['client'].post('/internal/approval-callback',json=body,
|
||||
headers={'X-Internal-Secret':e['settings'].internal_secret})
|
||||
return response,body
|
||||
try:yield dict(env=e,client=client,directory=directory,context=context,stop=stop,faults=faults,
|
||||
calls=calls,launch=launch,card=card,click=click,transport=transport,platform=platform)
|
||||
finally:
|
||||
stop.set()
|
||||
if jobs:await asyncio.wait_for(asyncio.gather(*jobs,return_exceptions=True),timeout=15)
|
||||
pc_context.CURRENT_TASK_STOP.reset(control);pc_context.CURRENT_TASK.reset(marker)
|
||||
client.close()
|
||||
|
||||
|
||||
async def test_allow_callback_replay_and_local_repeat_execute_once(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']()
|
||||
assert not p['calls'] and pc_gate.approval_pending(p['context'])
|
||||
response,body=await p['click']();assert response.status_code==200
|
||||
result=await asyncio.wait_for(job,10)
|
||||
assert result['status']=='completed'and len(p['calls'])==1
|
||||
replay=await p['env']['client'].post('/internal/approval-callback',json=body,
|
||||
headers={'X-Internal-Secret':p['env']['settings'].internal_secret})
|
||||
assert replay.status_code==200
|
||||
duplicate=await p['launch']()
|
||||
assert duplicate['status']=='duplicate'and len(p['calls'])==1
|
||||
assert not pc_gate.approval_pending(p['context'])
|
||||
|
||||
|
||||
async def test_deny_never_sends_or_reprompts_same_operation(pair):
|
||||
p=pair;job=p['launch']();await p['click']('deny')
|
||||
with pytest.raises(pc_errors.WorkError,match='拒绝'):await asyncio.wait_for(job,10)
|
||||
assert not p['calls']
|
||||
assert (await p['launch']())['previous_status']=='not_executed'
|
||||
assert len(p['env']['platform'].cards)==1
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault',['consume_receipt_lost','poll_lost','business_receipt_lost'])
|
||||
async def test_ambiguous_response_never_replays_after_reopen(pair,fault):
|
||||
p=pair;p['faults'].add(fault);job=p['launch']();await p['click']()
|
||||
if fault=='business_receipt_lost':
|
||||
assert (await asyncio.wait_for(job,10))['status']=='unknown'
|
||||
else:
|
||||
with pytest.raises(pc_errors.WorkError):await asyncio.wait_for(job,10)
|
||||
expected=1 if fault=='business_receipt_lost'else 0
|
||||
assert len(p['calls'])==expected
|
||||
reopened=pc_broker.BrokerWorkClient(p['directory'],transport=p['transport'],
|
||||
api_transport=httpx.MockTransport(p['platform']))
|
||||
try:
|
||||
result=await asyncio.to_thread(reopened.api,'POST',
|
||||
'/open-apis/calendar/v4/calendars/cal_synthetic/events',{},
|
||||
{'summary':'合成测试:不创建真实日程'},'operation-synthetic')
|
||||
assert result['status']=='duplicate'and len(p['calls'])==expected
|
||||
finally:reopened.close()
|
||||
if fault=='consume_receipt_lost':
|
||||
e=p['env'];connection=Store(e['settings'].data_dir/'metadata.sqlite')
|
||||
try:
|
||||
restored=ApprovalStore(BindingState(connection,'test','cli_test'),e['api'].broker.store.keyring)
|
||||
row=connection.db.execute('SELECT state FROM work_approvals_v3').fetchone()
|
||||
assert row['state']=='consumed'
|
||||
finally:connection.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure',['stop','rebind','terminal','expired'])
|
||||
async def test_stop_binding_terminal_and_expiry_do_not_execute(pair,failure):
|
||||
p=pair;job=p['launch']();row=await p['card']();e=p['env']
|
||||
if failure=='stop':p['stop'].set()
|
||||
elif failure=='rebind':
|
||||
value=json.loads((p['directory']/'device.json').read_text());value['binding_generation']+=1
|
||||
(p['directory']/'device.json').write_text(json.dumps(value))
|
||||
elif failure=='terminal':
|
||||
with e['store'].db:e['store'].db.execute('INSERT INTO work_execution_state_v3 VALUES(?,?,?,?)',
|
||||
(('msg_'+'a'*64),'failed',1,time.time()))
|
||||
else:
|
||||
with e['store'].db:e['store'].db.execute('UPDATE work_approvals_v3 SET expires_at=0')
|
||||
if failure in {'terminal','expired'}:await p['click']()
|
||||
try:
|
||||
result=await asyncio.wait_for(job,10)
|
||||
assert result['status']!='completed'
|
||||
except pc_errors.WorkError:pass
|
||||
assert not p['calls']
|
||||
|
||||
|
||||
async def test_other_operator_cannot_approve_but_owner_can(pair):
|
||||
p=pair;job=p['launch']();response,_=await p['click'](actor='ou_b')
|
||||
assert response.json()['toast']['type']=='warning'and not p['calls']
|
||||
assert p['env']['store'].db.execute('SELECT state FROM work_approvals_v3').fetchone()['state']=='pending'
|
||||
await p['click']();assert (await asyncio.wait_for(job,10))['status']=='completed'
|
||||
assert len(p['calls'])==1
|
||||
|
||||
|
||||
async def test_card_send_receipt_loss_never_sends_second_card(pair):
|
||||
p=pair;p['env']['platform'].lose_card_receipt=True;job=p['launch']();await p['card'](bound=False)
|
||||
deadline=time.monotonic()+5
|
||||
while time.monotonic()<deadline:
|
||||
row=p['env']['store'].db.execute('SELECT state FROM work_approval_card_outbox_v1').fetchone()
|
||||
if row and row['state']=='unknown':break
|
||||
await asyncio.sleep(.01)
|
||||
assert row['state']=='unknown'
|
||||
await p['env']['app'].state.approvals.maintain()
|
||||
await p['env']['app'].state.approvals.maintain()
|
||||
assert len(p['env']['platform'].cards)==1
|
||||
p['stop'].set()
|
||||
with pytest.raises(pc_errors.WorkError):await asyncio.wait_for(job,10)
|
||||
assert not p['calls']
|
||||
|
||||
|
||||
async def test_unsigned_rpc_proof_replay_and_other_task_are_rejected(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']();e=p['env'];path='/work/v2/approvals/status'
|
||||
fields=dict(task_id=('msg_'+'a'*64),approval_id=row['approval_id'],operation_digest=row['operation_digest'])
|
||||
unsigned=await e['client'].post(path,json=fields);assert unsigned.status_code==422
|
||||
signed=wire(e,path,**fields)
|
||||
assert (await e['client'].post(path,json=signed)).status_code==200
|
||||
assert (await e['client'].post(path,json=signed)).status_code==409
|
||||
other=await e['client'].post(path,json=wire(e,path,**{**fields,'task_id':'other-task'}))
|
||||
assert other.status_code==409 and not p['calls']
|
||||
p['stop'].set()
|
||||
with pytest.raises(pc_errors.WorkError):await asyncio.wait_for(job,10)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field',['app','tenant','token','timestamp','chat','card','nonce','value_identity'])
|
||||
async def test_forged_callback_cannot_approve(pair,field):
|
||||
p=pair;job=p['launch']();await p['card']()
|
||||
def mutate(body):
|
||||
if field=='app':body['header']['app_id']='another-app'
|
||||
elif field=='tenant':body['header']['tenant_key']='another-tenant'
|
||||
elif field=='token':body['header']['token']='wrong-verification'
|
||||
elif field=='timestamp':body['header']['create_time']='1'
|
||||
elif field=='chat':body['event']['context']['open_chat_id']='oc_other'
|
||||
elif field=='card':body['event']['context']['open_message_id']='om_other'
|
||||
elif field=='nonce':body['event']['action']['value']['nonce']='A'*43
|
||||
else:body['event']['action']['value']['actor_open_id']='ou_a'
|
||||
response,_=await p['click'](mutate=mutate)
|
||||
assert response.json()['toast']['type']=='warning'
|
||||
assert p['env']['store'].db.execute('SELECT state FROM work_approvals_v3').fetchone()['state']=='pending'
|
||||
assert not p['calls']
|
||||
p['stop'].set()
|
||||
with pytest.raises(pc_errors.WorkError):await asyncio.wait_for(job,10)
|
||||
|
||||
|
||||
async def test_conflicting_clicks_and_concurrent_consume_grant_at_most_once(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']()
|
||||
# Prevent PC consumption while testing two consumers of the same grant.
|
||||
response,_=await p['click']('allow')
|
||||
conflict,_=await p['click']('deny')
|
||||
assert response.json()['toast']['type']=='info'
|
||||
assert conflict.json()['toast']['type']=='warning'
|
||||
e=p['env'];path='/work/v2/approvals/consume'
|
||||
fields=dict(task_id=('msg_'+'a'*64),approval_id=row['approval_id'],operation_digest=row['operation_digest'])
|
||||
responses=await asyncio.gather(*(e['client'].post(path,json=wire(e,path,**fields))for _ in range(2)))
|
||||
assert sum(r.json().get('execute')is True for r in responses)<=1
|
||||
try:result=await asyncio.wait_for(job,10)
|
||||
except pc_errors.WorkError:result=None
|
||||
assert len(p['calls'])<=1
|
||||
assert sum(r.json().get('execute')is True for r in responses)+len(p['calls'])==1
|
||||
|
||||
|
||||
async def test_expiry_removes_buttons_and_update_receipt_loss_is_retryable(pair,monkeypatch):
|
||||
p=pair;job=p['launch']();row=await p['card']();e=p['env']
|
||||
with e['store'].db:e['store'].db.execute('UPDATE work_approvals_v3 SET expires_at=0')
|
||||
await e['app'].state.approvals.worker.call('close_unavailable')
|
||||
sender=e['app'].state.approvals.sender;original=e['platform'].update_task_card
|
||||
attempts=[]
|
||||
async def lost_once(*args,**kwargs):
|
||||
await original(*args,**kwargs);attempts.append(1)
|
||||
if len(attempts)==1:raise ConnectionError('synthetic update receipt loss')
|
||||
monkeypatch.setattr(e['platform'],'update_task_card',lost_once)
|
||||
with pytest.raises(ConnectionError):await sender.update_closed(row['approval_id'])
|
||||
assert await sender.update_closed(row['approval_id'])=={'state':'updated'}
|
||||
assert await sender.update_closed(row['approval_id'])=={'state':'no_closed_card'}
|
||||
assert len(e['platform'].cards)==1 and len(attempts)==2
|
||||
assert all('"button"'not in json.dumps(card)for _,card in e['platform'].updates)
|
||||
with pytest.raises(pc_errors.WorkError):await asyncio.wait_for(job,10)
|
||||
assert not p['calls']
|
||||
|
||||
|
||||
async def test_reviewed_read_bypasses_confirmation(pair):
|
||||
p=pair
|
||||
result=await asyncio.to_thread(p['client'].api,'GET','/open-apis/calendar/v4/calendars')
|
||||
assert result['status']=='completed'and len(p['calls'])==1
|
||||
assert not p['env']['platform'].cards
|
||||
assert not p['env']['store'].db.execute('SELECT * FROM work_approvals_v3').fetchall()
|
||||
|
||||
|
||||
async def test_default_off_does_not_register_approval_or_callback_routes(env):
|
||||
status=await env['client'].post('/work/v2/status',json=wire(env,'/work/v2/status'))
|
||||
assert status.status_code==200 and status.json()['approval_protocol']is None
|
||||
for path in ('/work/v2/approvals/create','/internal/approval-callback'):
|
||||
assert (await env['client'].post(path,json={})).status_code==404
|
||||
assert not env['store'].db.execute("SELECT name FROM sqlite_master WHERE name='work_approvals_v3'").fetchall()
|
||||
|
||||
|
||||
async def test_approval_worker_recreation_preserves_original_pending_card(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']();e=p['env']
|
||||
api=e['app'].state.approvals;original=api.worker
|
||||
restored=ApprovalWorker(e['settings'],e['api'].broker.store.keyring)
|
||||
await restored.start()
|
||||
api.worker=restored;api.sender.approvals=restored
|
||||
try:
|
||||
await api.maintain()
|
||||
assert len(e['platform'].cards)==1
|
||||
response,_=await p['click']()
|
||||
assert response.json()['toast']['type']=='info'
|
||||
assert (await asyncio.wait_for(job,10))['status']=='completed'
|
||||
assert len(p['calls'])==1
|
||||
saved=e['store'].db.execute('SELECT approval_id,state,card_message_id FROM work_approvals_v3').fetchone()
|
||||
assert (saved['approval_id'],saved['state'],saved['card_message_id'])==(
|
||||
row['approval_id'],'consumed',row['card_message_id'])
|
||||
finally:
|
||||
# Drain only this adapter's tracked card work before closing its
|
||||
# connection. App lifespan still owns the original worker shutdown.
|
||||
deadline=time.monotonic()+5
|
||||
while time.monotonic()<deadline:
|
||||
if not api.scheduled:break
|
||||
await asyncio.sleep(.01)
|
||||
api.worker=original;api.sender.approvals=original
|
||||
await restored.close()
|
||||
@@ -0,0 +1,109 @@
|
||||
"""Opt-in isolated-schema PostgreSQL race test; never uses production config."""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import secrets
|
||||
import time
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
import psycopg
|
||||
from psycopg.conninfo import make_conninfo
|
||||
from relay.store import Store
|
||||
from relay.binding_state import BindingState
|
||||
from relay.credential_store import CredentialKeyring
|
||||
from relay.approval_store import ApprovalStore,VerifiedDecision,ApprovalError
|
||||
from relay.crypto import canonical
|
||||
|
||||
pytestmark=pytest.mark.skipif(not os.environ.get('HN_TEST_DATABASE_URL'),reason='isolated PostgreSQL test URL not supplied')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('winner',['race','allow','deny'])
|
||||
@pytest.mark.parametrize('consumers',[4,16])
|
||||
def test_concurrent_postgres_decisions_consumption_and_reopen(tmp_path,winner,consumers):
|
||||
schema='approval_test_'+secrets.token_hex(10)
|
||||
target=os.environ['HN_TEST_DATABASE_URL']
|
||||
control=psycopg.connect(target,autocommit=True)
|
||||
control.execute('CREATE SCHEMA '+schema)
|
||||
dsn=make_conninfo(target,options='-c search_path='+schema,connect_timeout=5)
|
||||
keyring=CredentialKeyring('synthetic',{ 'synthetic':secrets.token_bytes(32)})
|
||||
connections=[]
|
||||
def connect():
|
||||
store=Store(tmp_path/'unused.sqlite',dsn)
|
||||
store.db.execute("SET lock_timeout TO '2s'")
|
||||
store.db.execute("SET statement_timeout TO '3s'")
|
||||
state=BindingState(store,'test','cli_pg_test')
|
||||
return store,state,ApprovalStore(state,keyring)
|
||||
try:
|
||||
store,state,approvals=connect();connections.append(store)
|
||||
with store.db:
|
||||
store.db.execute("INSERT INTO devices VALUES(?,?,?,?,?,?)",('a'*32,'synthetic-cert','synthetic-fingerprint','ou_pg',0,1))
|
||||
state.create_task('pg-task','ou_pg','a'*32,1,'session','worker',time.time()+300,'oc_pg','om_pg_source')
|
||||
action={'kind':'synthetic','value':'no real side effect'}
|
||||
item=approvals.create('pg-task','ou_pg','a'*32,1,'pg-operation',action,'合成并发确认','不执行真实业务')
|
||||
material=approvals.card_material(item['approval_id']);assert approvals.claim_card(item['approval_id'])
|
||||
approvals.bind_card(item['approval_id'],'om_pg_card')
|
||||
def make_decision(choice):
|
||||
fields=dict(environment='test',app_id='cli_pg_test',actor_open_id='ou_pg',chat_id='oc_pg',
|
||||
card_message_id='om_pg_card',approval_id=item['approval_id'],nonce=material['nonce'],choice=choice)
|
||||
return VerifiedDecision(**fields,event_id=secrets.token_hex(16),event_digest=hashlib.sha256(canonical(fields)).hexdigest())
|
||||
if winner!='race':approvals.decide(make_decision(winner))
|
||||
decision_gate=threading.Barrier(2,timeout=10)
|
||||
def decide(choice):
|
||||
connection,_,ledger=connect()
|
||||
try:
|
||||
decision_gate.wait()
|
||||
try:return ledger.decide(make_decision(choice))['state']
|
||||
except ApprovalError:return 'conflict'
|
||||
finally:connection.close()
|
||||
with ThreadPoolExecutor(max_workers=2)as executor:
|
||||
decisions=list(executor.map(decide,('allow','deny')))
|
||||
row=store.db.execute('SELECT state FROM work_approvals_v3').fetchone()
|
||||
assert row['state']in {'approved','rejected'}and decisions.count('conflict')==1
|
||||
if winner=='allow':
|
||||
# A real PostgreSQL statement error must roll back the earlier
|
||||
# consumed UPDATE in the same transaction, not merely hide a reply.
|
||||
store.db.execute("""CREATE FUNCTION fail_approval_audit() RETURNS trigger LANGUAGE plpgsql AS $$
|
||||
BEGIN IF NEW.event='consumed' THEN RAISE EXCEPTION 'synthetic audit storage failure';
|
||||
END IF; RETURN NEW; END $$""")
|
||||
store.db.execute('''CREATE TRIGGER synthetic_audit_failure BEFORE INSERT ON work_approval_audit_v1
|
||||
FOR EACH ROW EXECUTE FUNCTION fail_approval_audit()''')
|
||||
with pytest.raises(psycopg.Error):
|
||||
approvals.consume(item['approval_id'],'pg-task','ou_pg','a'*32,1,item['operation_digest'])
|
||||
assert store.db.execute('SELECT state FROM work_approvals_v3').fetchone()['state']=='approved'
|
||||
assert not store.db.execute("SELECT * FROM work_approval_audit_v1 WHERE event='consumed'").fetchall()
|
||||
store.db.execute('DROP TRIGGER synthetic_audit_failure ON work_approval_audit_v1')
|
||||
store.db.execute('DROP FUNCTION fail_approval_audit()')
|
||||
consume_gate=threading.Barrier(consumers,timeout=20)
|
||||
def consume(_):
|
||||
connection,_,ledger=connect()
|
||||
try:
|
||||
consume_gate.wait()
|
||||
return ledger.consume(item['approval_id'],'pg-task','ou_pg','a'*32,1,item['operation_digest'])['execute']
|
||||
finally:connection.close()
|
||||
with ThreadPoolExecutor(max_workers=consumers)as executor:executed=list(executor.map(consume,range(consumers)))
|
||||
assert sum(executed)==(1 if row['state']=='approved'else 0)
|
||||
restored,_,ledger=connect();connections.append(restored)
|
||||
assert ledger.consume(item['approval_id'],'pg-task','ou_pg','a'*32,1,item['operation_digest'])['execute']is False
|
||||
assert store.db.execute("SELECT COUNT(*) n FROM work_approval_audit_v1 WHERE event IN ('approved','rejected')").fetchone()['n']==1
|
||||
state.create_task('pg-intent-task','ou_pg','a'*32,1,'session','worker',time.time()+300,'oc_pg','om_pg_intent')
|
||||
intent_gate=threading.Barrier(2,timeout=10)
|
||||
def create_same_intent(operation):
|
||||
connection,_,ledger=connect()
|
||||
try:
|
||||
intent_gate.wait()
|
||||
try:
|
||||
ledger.create('pg-intent-task','ou_pg','a'*32,1,operation,action,'相同合成动作','不执行真实业务')
|
||||
return 'created'
|
||||
except ApprovalError:return 'conflict'
|
||||
finally:connection.close()
|
||||
with ThreadPoolExecutor(max_workers=2)as executor:
|
||||
outcomes=list(executor.map(create_same_intent,('pg-intent-one','pg-intent-two')))
|
||||
assert outcomes.count('created')==1 and outcomes.count('conflict')==1
|
||||
assert store.db.execute("SELECT COUNT(*) n FROM work_approvals_v3 WHERE task_id='pg-intent-task'").fetchone()['n']==1
|
||||
finally:
|
||||
for connection in connections:connection.close()
|
||||
# The unique generated test schema is the only deletion target.
|
||||
control.execute('DROP SCHEMA '+schema+' CASCADE');control.close()
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Adversarial M6 checks, no real platform operations."""
|
||||
import hashlib
|
||||
import json
|
||||
import pytest
|
||||
from test_credential_api import env,wire
|
||||
from test_approval_integration import approval_env,pair,pc_errors
|
||||
from relay.approval_store import ApprovalStore,ApprovalError
|
||||
from relay.approval_policy import describe_action
|
||||
from relay.crypto import canonical
|
||||
|
||||
|
||||
@pytest.mark.parametrize('choice',[None,'deny','allow'])
|
||||
async def test_same_intent_cannot_get_new_card_by_changing_operation_id(pair,choice):
|
||||
p=pair;job=p['launch']();row=await p['card']()
|
||||
if choice:
|
||||
await p['click'](choice)
|
||||
if choice=='deny':
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
else:await job
|
||||
e=p['env'];ledger=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring)
|
||||
action=json.loads(ledger.keyring.decrypt(row['key_id'],row['ciphertext'],ledger._aad(row)))['action']
|
||||
other='new-operation-id'
|
||||
action['params']['idempotency_key']=hashlib.sha256(canonical(['ou_a',other])).hexdigest()
|
||||
path='/work/v2/approvals/create'
|
||||
response=await e['client'].post(path,json=wire(e,path,task_id=('msg_'+'a'*64),operation_id=other,action=action,ttl=120))
|
||||
assert response.status_code==409
|
||||
assert e['store'].db.execute('SELECT COUNT(*) n FROM work_approvals_v3').fetchone()['n']==1
|
||||
assert len(p['calls'])==(1 if choice=='allow'else 0)
|
||||
if not choice:
|
||||
p['stop'].set()
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
|
||||
|
||||
def test_confirmation_identifies_actual_calendar_not_just_events_suffix():
|
||||
def action(calendar):return {'kind':'work-api-v1','method':'POST',
|
||||
'path':'/open-apis/calendar/v4/calendars/'+calendar+'/events','params':{},'body':{'summary':'同一标题'}}
|
||||
first=describe_action(action('calendar_a'))
|
||||
second=describe_action(action('calendar_b'))
|
||||
assert 'calendar_a'in first[1]and 'calendar_b'in second[1]and first!=second
|
||||
|
||||
|
||||
def test_long_target_is_rejected_instead_of_silently_truncated():
|
||||
with pytest.raises(ApprovalError):describe_action({'kind':'work-api-v1','method':'DELETE',
|
||||
'path':'/open-apis/drive/v1/files/'+'x'*400,'params':{},'body':{}})
|
||||
|
||||
|
||||
async def test_changed_action_same_id_rejected_but_distinct_new_action_can_confirm(pair):
|
||||
p=pair;job=p['launch']();row=await p['card']();await p['click']('deny')
|
||||
with pytest.raises(pc_errors.WorkError):await job
|
||||
e=p['env'];ledger=ApprovalStore(e['store'].lifecycle,e['api'].broker.store.keyring)
|
||||
action=json.loads(ledger.keyring.decrypt(row['key_id'],row['ciphertext'],ledger._aad(row)))['action']
|
||||
action['body']['summary']='真正不同的合成动作'
|
||||
path='/work/v2/approvals/create'
|
||||
same=await e['client'].post(path,json=wire(e,path,task_id=('msg_'+'a'*64),
|
||||
operation_id='operation-synthetic',action=action,ttl=120))
|
||||
assert same.status_code==409
|
||||
other='distinct-operation'
|
||||
action['params']['idempotency_key']=hashlib.sha256(canonical(['ou_a',other])).hexdigest()
|
||||
distinct=await e['client'].post(path,json=wire(e,path,task_id=('msg_'+'a'*64),
|
||||
operation_id=other,action=action,ttl=120))
|
||||
assert distinct.status_code==200 and distinct.json()['state']=='pending'
|
||||
assert not p['calls']
|
||||
@@ -658,3 +658,127 @@ def test_prepare_update_file_feature_is_explicit_and_rollback_restores_parent_fl
|
||||
assert current['services']['connector']['environment']['HN_WORK_FILES_ENABLED']=='true'
|
||||
assert rollback['services']['connector']['environment']==old['services']['connector']['environment']
|
||||
assert (output/'credential-keyring.json').read_bytes()==(root/'credential-keyring.json').read_bytes()
|
||||
|
||||
|
||||
def test_prepare_update_cards_are_opt_in_and_rollback_keeps_parent_environment(scenario):
|
||||
parent=prepared(scenario);root=scenario['output'];image='sha256:'+'a'*64
|
||||
for name in ('connector','router'):scenario['roles'][name]['image']=image
|
||||
deploy.write_json(root/'candidate.json',{'image':image,'tag':parent['candidate_image']})
|
||||
output=root.parent/'update-cards'
|
||||
deploy.prepare_update(argparse.Namespace(plan=str(root),source_root=str(ROOT),output=str(output),enable_task_cards=True))
|
||||
current=json.loads((output/'compose.broker.json').read_text())
|
||||
rollback=json.loads((output/'compose.rollback.json').read_text())
|
||||
old=json.loads((root/'compose.broker.json').read_text())
|
||||
assert current['services']['connector']['environment']['HN_WORK_TASK_CARDS_ENABLED']=='true'
|
||||
assert rollback['services']['connector']['environment']==old['services']['connector']['environment']
|
||||
assert (output/'credential-keyring.json').read_bytes()==(root/'credential-keyring.json').read_bytes()
|
||||
|
||||
|
||||
def approval_update(scenario,**changes):
|
||||
parent=prepared(scenario);root=scenario['output'];image='sha256:'+'a'*64
|
||||
for name in ('connector','router'):scenario['roles'][name]['image']=image
|
||||
deploy.write_json(root/'candidate.json',{'image':image,'tag':parent['candidate_image']})
|
||||
env=scenario['base']['services']['connector']['environment']
|
||||
scenario['base']['services']['longconn']['environment'].update(
|
||||
HN_FEISHU_APP_ID=env['HN_FEISHU_APP_ID'],HN_INTERNAL_SECRET=env['HN_INTERNAL_SECRET'],HN_FEISHU_EVENT_SOURCE='longconn')
|
||||
identity=root.parent/'callback-private.json'
|
||||
identity.write_text(json.dumps({'tenant_key':'tenant_test','verification_token':'synthetic-verify-token'}))
|
||||
identity.chmod(0o600)
|
||||
args=argparse.Namespace(plan=str(root),source_root=str(ROOT),output=str(root.parent/'approvals'),
|
||||
enable_remote_approvals=True,callback_identity=str(identity))
|
||||
for key,value in changes.items():setattr(args,key,value)
|
||||
return args
|
||||
|
||||
|
||||
def test_approval_update_preserves_runtime_ancestry_and_original_longconn_rollback(scenario):
|
||||
args=approval_update(scenario);deploy.prepare_update(args)
|
||||
root=Path(args.output);plan=deploy.load_plan(root)
|
||||
candidate=json.loads((root/'compose.broker.json').read_text());rollback=json.loads((root/'compose.rollback.json').read_text())
|
||||
assert deploy.targets(plan)==('connector','router','longconn')
|
||||
assert plan['longconn_base_image']==scenario['roles']['longconn']['image']
|
||||
assert candidate['services']['longconn']['image']!=candidate['services']['connector']['image']
|
||||
assert candidate['services']['longconn']['environment']['HN_CREDENTIAL_MODE']=='broker'
|
||||
assert candidate['services']['connector']['environment']['HN_APPROVAL_RETENTION_ENABLED']=='false'
|
||||
assert rollback['services']['longconn']['image']==scenario['roles']['longconn']['image']
|
||||
assert rollback['services']['longconn']['environment']['HN_CREDENTIAL_MODE']=='legacy'
|
||||
assert rollback['services']['longconn']['environment']['HN_WORK_REMOTE_APPROVALS_ENABLED']=='false'
|
||||
assert (root/'credential-keyring.json').read_bytes()==(Path(args.plan)/'credential-keyring.json').read_bytes()
|
||||
|
||||
|
||||
def test_second_approval_update_uses_active_parent_images_and_retains_off(scenario):
|
||||
args=approval_update(scenario);deploy.prepare_update(args)
|
||||
root=Path(args.output);parent=deploy.load_plan(root)
|
||||
connector='sha256:'+'c'*64;longconn='sha256:'+'d'*64
|
||||
for name in ('connector','router'):scenario['roles'][name]['image']=connector
|
||||
scenario['roles']['longconn']['image']=longconn
|
||||
deploy.write_json(root/'candidate.json',{'image':connector,'tag':parent['candidate_image']})
|
||||
deploy.write_json(root/'candidate-longconn.json',{'image':longconn,'tag':parent['longconn_candidate_image']})
|
||||
output=root.parent/'approvals-m6-fix'
|
||||
deploy.prepare_update(argparse.Namespace(plan=str(root),source_root=str(ROOT),output=str(output),
|
||||
enable_remote_approvals=True,callback_identity=args.callback_identity))
|
||||
result=deploy.load_plan(output)
|
||||
candidate=json.loads((output/'compose.broker.json').read_text())
|
||||
rollback=json.loads((output/'compose.rollback.json').read_text())
|
||||
assert result['base_image']==connector and result['longconn_base_image']==longconn
|
||||
assert candidate['services']['connector']['environment']['HN_APPROVAL_RETENTION_ENABLED']=='false'
|
||||
assert rollback['services']['connector']['image']==connector
|
||||
assert rollback['services']['longconn']['image']==longconn
|
||||
assert rollback['services']['longconn']['environment']['HN_WORK_REMOTE_APPROVALS_ENABLED']=='true'
|
||||
assert (output/'credential-keyring.json').read_bytes()==(root/'credential-keyring.json').read_bytes()
|
||||
|
||||
|
||||
def test_approval_runtime_checks_three_roles_and_sdk_without_network(scenario):
|
||||
args=approval_update(scenario);deploy.prepare_update(args);root=Path(args.output);plan=deploy.load_plan(root)
|
||||
scenario['calls'].clear();deploy.runtime_check(root,plan)
|
||||
probes=[c for c in scenario['calls']if c[:2]==['docker','run']]
|
||||
assert len(probes)==3
|
||||
assert all('--network'in c and c[c.index('--network')+1]=='none'for c in probes)
|
||||
assert any(plan['longconn_candidate_image']in c and 'register_p2_card_action_trigger'in c[-1]for c in probes)
|
||||
assert not any('synthetic-verify-token'in part for c in probes for part in c)
|
||||
|
||||
|
||||
def test_approval_activation_targets_original_three_services_only(scenario):
|
||||
args=approval_update(scenario);deploy.prepare_update(args);root=Path(args.output);plan=deploy.load_plan(root)
|
||||
image='sha256:'+'a'*64
|
||||
deploy.write_json(root/'candidate.json',{'image':image,'tag':plan['candidate_image']})
|
||||
deploy.write_json(root/'candidate-longconn.json',{'image':image,'tag':plan['longconn_candidate_image']})
|
||||
scenario['calls'].clear();deploy.activate(root,plan,True)
|
||||
up=[c for c in scenario['calls']if 'up'in c]
|
||||
assert len(up)==1 and up[0][-3:]==['connector','router','longconn']
|
||||
assert '--no-deps'in up[0]and '--pull'in up[0]
|
||||
|
||||
|
||||
def test_approval_overlay_rejects_longconn_command_or_network_change(scenario):
|
||||
args=approval_update(scenario);deploy.prepare_update(args);root=Path(args.output);plan=deploy.load_plan(root)
|
||||
base=deploy.run(deploy.compose(plan)+['config','--format','json'],True)
|
||||
merged=deploy.run(deploy.compose(plan,root/'compose.broker.json')+['config','--format','json'],True)
|
||||
merged['services']['longconn']['entrypoint']=['sh','-c','unreviewed']
|
||||
with pytest.raises(deploy.DeployError,match='unrelated longconn'):
|
||||
deploy.validate_overlay(base,merged,plan['candidate_image'],approval_plan=plan)
|
||||
|
||||
|
||||
def test_approval_update_rejects_different_app_or_missing_identity(scenario):
|
||||
args=approval_update(scenario)
|
||||
scenario['base']['services']['longconn']['environment']['HN_FEISHU_APP_ID']='other'
|
||||
with pytest.raises(deploy.DeployError,match='does not match'):deploy.prepare_update(args)
|
||||
assert not Path(args.output).exists()
|
||||
|
||||
|
||||
def test_approval_build_reuses_distinct_longconn_base_and_saves_both_receipts(scenario):
|
||||
args=approval_update(scenario);deploy.prepare_update(args);root=Path(args.output);plan=deploy.load_plan(root)
|
||||
scenario['calls'].clear();deploy.build(root,plan)
|
||||
commands=[c for c in scenario['calls']if c[:2]==['docker','build']]
|
||||
assert len(commands)==2
|
||||
assert 'BASE_IMAGE='+scenario['roles']['longconn']['image']in commands[1]
|
||||
assert plan['longconn_candidate_image']in commands[1]
|
||||
assert all('--network'in c and c[c.index('--network')+1]=='none'for c in commands)
|
||||
assert (root/'candidate.json').is_file()and(root/'candidate-longconn.json').is_file()
|
||||
|
||||
|
||||
def test_approval_cli_rollback_restores_original_longconn_too(scenario,monkeypatch):
|
||||
args=approval_update(scenario);deploy.prepare_update(args);root=Path(args.output)
|
||||
monkeypatch.setattr(deploy.sys,'argv',['broker_deploy.py','rollback','--plan',str(root),'--stop-broker-use'])
|
||||
scenario['calls'].clear();deploy.main()
|
||||
commands=[c for c in scenario['calls']if 'up'in c]
|
||||
assert len(commands)==1 and commands[0][-3:]==['connector','router','longconn']
|
||||
assert str(root/'compose.rollback.json')in commands[0]
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
"""Joint source test: encrypted PC journal -> signed ECS progress -> card projection."""
|
||||
import importlib
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import types
|
||||
import httpx
|
||||
import pytest
|
||||
from test_credential_api import env,wire
|
||||
from test_task_delivery import task
|
||||
from test_task_cards import setup_cards
|
||||
|
||||
|
||||
async def test_native_states_reach_signed_server_and_existing_card(env,tmp_path,monkeypatch):
|
||||
client=Path(__file__).resolve().parents[3]/'hainuo-build-remote-C-20261112/integrations/work-relay/service/relay'
|
||||
if not client.is_dir():pytest.skip('paired client source not present')
|
||||
package=types.ModuleType('joint_pc_relay');package.__path__=[str(client)]
|
||||
monkeypatch.setitem(sys.modules,'joint_pc_relay',package)
|
||||
journal_module=importlib.import_module('joint_pc_relay.task_journal')
|
||||
native=importlib.import_module('joint_pc_relay.hainuo1065')
|
||||
monkeypatch.setattr(journal_module,'protect',lambda data:b'test:'+data)
|
||||
monkeypatch.setattr(journal_module,'unprotect',lambda data:data[5:])
|
||||
clock=[0.0]
|
||||
# Patch only the native module clock, not shared time.monotonic used by event loop.
|
||||
monkeypatch.setattr(native,'time',types.SimpleNamespace(monotonic=lambda:clock[0],time=__import__('time').time))
|
||||
async def no_wait(_):clock[0]+=1
|
||||
monkeypatch.setattr(native,'asyncio',types.SimpleNamespace(sleep=no_wait))
|
||||
context={'task_id':'task-durable','conversation_id':'synthetic-conversation'}
|
||||
ledger=journal_module.TaskJournal(tmp_path);ledger.begin(context,'synthetic')
|
||||
task(env);cards,calls,card_clock=setup_cards(env,monkeypatch)
|
||||
states=iter(['running','waiting_for_approval','running','completed']);reported=[];posts=[]
|
||||
run_id='run_'+'a'*32
|
||||
def handler(request):
|
||||
if request.method=='POST':posts.append(request);return httpx.Response(202,json={'run_id':run_id})
|
||||
return httpx.Response(200,json={'run_id':run_id,'status':next(states),'output':'synthetic result'})
|
||||
config=tmp_path/'native.json';config.write_text(__import__('json').dumps({'url':'http://127.0.0.1:8643','api_key':'fake-'*10,'mode':'text-only'}))
|
||||
adapter=native.Hainuo1065Agent(config,transport=httpx.MockTransport(handler));adapter.model='synthetic'
|
||||
async def progress():
|
||||
saved=ledger.get(context);path='/work/v2/tasks/progress'
|
||||
response=await env['client'].post(path,json=wire(env,path,task_id=context['task_id'],
|
||||
execution_state=saved['execution_state'],status_sequence=saved['status_sequence']))
|
||||
assert response.status_code==200,response.text
|
||||
reported.append(response.json()['execution_state'])
|
||||
card_clock[0]+=6;await cards.flush()
|
||||
try:
|
||||
assert await adapter._execute_run('synthetic',{'X-Hermes-Session-Id':'synthetic-session'},progress,journal=ledger,context=context)=='synthetic result'
|
||||
assert reported==['running','waiting_for_approval','running','completed']
|
||||
assert ledger.get(context)['status_sequence']==4
|
||||
assert len(posts)==1
|
||||
assert [call[0] for call in calls]==['create','update','update','update']
|
||||
assert len(env['platform'].replies)==0 # Execution completion is not result delivery.
|
||||
finally:await adapter.close()
|
||||
@@ -0,0 +1,207 @@
|
||||
import asyncio
|
||||
import json
|
||||
import time
|
||||
import httpx
|
||||
import pytest
|
||||
from test_credential_api import env,wire
|
||||
from test_task_delivery import task,upload
|
||||
from relay.task_cards import TaskCards,card_content
|
||||
from relay.platform import Feishu,PlatformError
|
||||
|
||||
|
||||
def setup_cards(env,monkeypatch):
|
||||
calls=[];clock=[time.time()]
|
||||
async def create(message,card,delivery_id,*,token):
|
||||
calls.append(('create',message,card,delivery_id));return 'om_card'
|
||||
async def update(message,card,*,token):calls.append(('update',message,card))
|
||||
monkeypatch.setattr(env['platform'],'reply_task_card',create,raising=False)
|
||||
monkeypatch.setattr(env['platform'],'update_task_card',update,raising=False)
|
||||
return TaskCards(env['store'].lifecycle,env['platform'],'worker-test',clock=lambda:clock[0]),calls,clock
|
||||
|
||||
|
||||
async def report(env,state,sequence):
|
||||
path='/work/v2/tasks/progress'
|
||||
response=await env['client'].post(path,json=wire(env,path,task_id='task-durable',execution_state=state,status_sequence=sequence))
|
||||
assert response.status_code==200,response.text
|
||||
|
||||
|
||||
async def test_card_created_once_updates_in_order_and_delivery_separate(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
await report(env,'running',1);await cards.flush();await cards.flush()
|
||||
assert len(calls)==1 and calls[0][0]=='create' and calls[0][1]=='om-synthetic'
|
||||
await report(env,'waiting_for_approval',2)
|
||||
await cards.flush();assert len(calls)==1 # Update throttle.
|
||||
clock[0]+=6;await cards.flush()
|
||||
assert calls[-1][0]=='update' and '等待本人确认' in json.dumps(calls[-1],ensure_ascii=False)
|
||||
assert 'button' not in json.dumps(calls[-1])
|
||||
await report(env,'completed',3);clock[0]+=6;await cards.flush()
|
||||
text=json.dumps(calls[-1],ensure_ascii=False)
|
||||
assert '执行完成' in text and '尚未接收结果' in text and '结果已发送' not in text
|
||||
await upload(env);await env['app'].state.deliveries.flush()
|
||||
clock[0]+=6;await cards.flush()
|
||||
assert '结果已发送到当前会话' in json.dumps(calls[-1],ensure_ascii=False)
|
||||
assert len([c for c in calls if c[0]=='create'])==1
|
||||
|
||||
|
||||
async def test_unknown_create_receipt_is_durable_and_never_recreated(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
async def lost(*args,**kwargs):calls.append(('lost',));raise TimeoutError()
|
||||
monkeypatch.setattr(env['platform'],'reply_task_card',lost)
|
||||
await report(env,'running',1);await cards.flush()
|
||||
clock[0]+=65
|
||||
reopened=TaskCards(env['store'].lifecycle,env['platform'],'new-worker',clock=lambda:clock[0])
|
||||
await report(env,'completed',2);await reopened.flush()
|
||||
assert len(calls)==1
|
||||
assert env['store'].db.execute('SELECT status FROM work_task_cards_v3').fetchone()['status']=='unknown'
|
||||
await upload(env);await env['app'].state.deliveries.flush()
|
||||
assert len(env['platform'].replies)==1 # Card failure never suppresses result text.
|
||||
|
||||
|
||||
async def test_failed_update_retries_same_card_with_newest_state(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
await report(env,'running',1);await cards.flush()
|
||||
original=env['platform'].update_task_card
|
||||
async def lost(*args,**kwargs):calls.append(('lost-update',args[0]));raise TimeoutError()
|
||||
monkeypatch.setattr(env['platform'],'update_task_card',lost)
|
||||
await report(env,'waiting_for_approval',2);clock[0]+=6;await cards.flush()
|
||||
monkeypatch.setattr(env['platform'],'update_task_card',original)
|
||||
await report(env,'failed',3);clock[0]+=6;await cards.flush()
|
||||
assert calls[-1][0:2]==('update','om_card')
|
||||
assert '执行失败' in json.dumps(calls[-1],ensure_ascii=False)
|
||||
assert len([c for c in calls if c[0]=='create'])==1
|
||||
|
||||
|
||||
async def test_unbind_during_token_preflight_prevents_card_send(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
async def token():env['store'].unbind('ou_a');return 'fake'
|
||||
monkeypatch.setattr(env['platform'],'tenant_token',token)
|
||||
await cards.flush();assert not calls
|
||||
|
||||
|
||||
async def test_two_workers_and_stale_snapshot_cannot_send_old_state(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
await report(env,'running',1)
|
||||
paused=asyncio.Event();resume=asyncio.Event()
|
||||
original=env['platform'].reply_task_card
|
||||
async def create(*args,**kwargs):paused.set();await resume.wait();return await original(*args,**kwargs)
|
||||
monkeypatch.setattr(env['platform'],'reply_task_card',create)
|
||||
first=asyncio.create_task(cards.flush());await paused.wait()
|
||||
other=TaskCards(env['store'].lifecycle,env['platform'],'worker-other',clock=lambda:clock[0])
|
||||
await other.flush();assert not calls
|
||||
resume.set();await first
|
||||
assert len(calls)==1
|
||||
await report(env,'waiting_for_approval',2)
|
||||
async def token():
|
||||
# A new authenticated status arrives while card worker awaits preflight.
|
||||
await report(env,'completed',3);return 'fake'
|
||||
monkeypatch.setattr(env['platform'],'tenant_token',token)
|
||||
clock[0]+=6;await other.flush()
|
||||
assert '执行完成' in json.dumps(calls[-1],ensure_ascii=False)
|
||||
|
||||
|
||||
async def test_stale_creation_claim_is_unknown_but_stale_update_can_recover(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
db=env['store'].db
|
||||
with db:db.execute('INSERT INTO work_task_cards_v3 VALUES(?,?,?,?,?,?)',('task-durable','','','sending','dead',clock[0]-61))
|
||||
await cards.flush();await cards.flush();assert not calls
|
||||
assert db.execute('SELECT status FROM work_task_cards_v3').fetchone()['status']=='unknown'
|
||||
with db:db.execute("UPDATE work_task_cards_v3 SET message_id='om_card',status='sending',updated_at=?",(clock[0]-61,))
|
||||
await cards.flush();await cards.flush()
|
||||
assert calls[0][0:2]==('update','om_card')
|
||||
|
||||
|
||||
def test_card_projection_never_exposes_task_body_or_markdown_controls():
|
||||
data=card_content('synthetic-task','waiting_for_approval',None)
|
||||
text=json.dumps(data,ensure_ascii=False)
|
||||
assert 'button' not in text and 'markdown' not in text
|
||||
assert data['config']['enable_forward'] is False
|
||||
|
||||
|
||||
def test_text_delivery_never_claims_attachments_delivered():
|
||||
text=json.dumps(card_content('synthetic','completed','replied',[]),ensure_ascii=False)
|
||||
assert '文字回复:' in text and '附件交付' not in text and '未收到登记' not in text
|
||||
mixed=json.dumps(card_content('synthetic','completed','replied',['sent','ready','unknown']),ensure_ascii=False)
|
||||
assert '附件交付' not in mixed and '待投递1件' not in mixed
|
||||
|
||||
|
||||
@pytest.mark.parametrize('state,color',[
|
||||
('running','blue'),('waiting_for_approval','orange'),('unknown','orange'),
|
||||
('completed','green'),('failed','red'),('interrupted','red')])
|
||||
def test_execution_color_is_independent_of_text_delivery(state,color):
|
||||
card=card_content('synthetic',state,'replied',[])
|
||||
assert card['header']['template']==color
|
||||
if state=='interrupted':
|
||||
assert '已中断' in json.dumps(card,ensure_ascii=False)
|
||||
|
||||
|
||||
async def test_card_uses_task_bound_source_and_skips_foreign_and_expired(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
with env['store'].db:
|
||||
env['store'].db.execute('UPDATE work_tasks_v2 SET deadline=?',(clock[0]-1,))
|
||||
await cards.flush();assert not calls
|
||||
with env['store'].db:
|
||||
env['store'].db.execute("UPDATE work_tasks_v2 SET deadline=?,binding_generation=999",(clock[0]+300,))
|
||||
await cards.flush();assert not calls
|
||||
|
||||
|
||||
async def test_timeout_projects_unknown_without_claiming_failure(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
await report(env,'running',1);await cards.flush()
|
||||
with env['store'].db:env['store'].db.execute("UPDATE work_tasks_v2 SET status='unknown'")
|
||||
clock[0]+=6;await cards.flush()
|
||||
text=json.dumps(calls[-1],ensure_ascii=False)
|
||||
assert '状态待核对' in text and '执行失败' not in text
|
||||
|
||||
|
||||
async def test_lost_client_turns_red_persists_and_recovers_on_authenticated_progress(env,monkeypatch):
|
||||
task(env);cards,calls,clock=setup_cards(env,monkeypatch)
|
||||
api=env['app'].state.deliveries;api.cards=cards
|
||||
await report(env,'running',1);await cards.flush()
|
||||
cards.mark_connection_lost('task-durable')
|
||||
with env['store'].db:env['store'].db.execute("UPDATE work_tasks_v2 SET status='unknown'")
|
||||
clock[0]+=6;await cards.flush()
|
||||
card=calls[-1][2]
|
||||
assert card['header']['template']=='red'
|
||||
text=json.dumps(card,ensure_ascii=False)
|
||||
assert '客户端失联,任务结果待核对' in text and '执行失败' not in text
|
||||
reopened=TaskCards(env['store'].lifecycle,env['platform'],'worker-new',clock=lambda:clock[0])
|
||||
assert reopened.snapshot(env['store'].lifecycle.task('task-durable'))['header']['template']=='red'
|
||||
await report(env,'completed',2);clock[0]+=6;await cards.flush()
|
||||
assert calls[-1][2]['header']['template']=='green'
|
||||
assert not env['store'].db.execute('SELECT * FROM work_task_connection_v3').fetchall()
|
||||
|
||||
|
||||
async def test_idle_timeout_is_distinct_from_hard_tracking_limit():
|
||||
from types import SimpleNamespace
|
||||
from relay.connector import Broker,Pending,ClientIdleTimeout
|
||||
broker=object.__new__(Broker)
|
||||
broker.s=SimpleNamespace(request_timeout=.01,task_tracking_timeout=1)
|
||||
item=Pending(asyncio.get_running_loop().create_future(),None,None,'synthetic','synthetic')
|
||||
with pytest.raises(ClientIdleTimeout):await broker.wait_tracked(item)
|
||||
broker.s.task_tracking_timeout=0
|
||||
with pytest.raises(asyncio.TimeoutError)as error:await broker.wait_tracked(item)
|
||||
assert not isinstance(error.value,ClientIdleTimeout)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('state',['completed','failed','cancelled','interrupted'])
|
||||
def test_lost_connection_never_overwrites_known_terminal_state(state):
|
||||
card=card_content('synthetic',state,None,connection_lost=True)
|
||||
assert '客户端失联' not in json.dumps(card,ensure_ascii=False)
|
||||
|
||||
|
||||
async def test_platform_card_wire_format_and_receipt_validation(env):
|
||||
seen=[]
|
||||
def handler(request):
|
||||
seen.append(request)
|
||||
return httpx.Response(200,json={'code':0,'data':{'message_id':'om_receipt'}})
|
||||
platform=Feishu(env['settings'],transport=httpx.MockTransport(handler))
|
||||
card=card_content('synthetic','running',None)
|
||||
try:
|
||||
assert await platform.reply_task_card('om_source',card,'a'*32,token='fake')=='om_receipt'
|
||||
await platform.update_task_card('om_receipt',card,token='fake')
|
||||
body=json.loads(seen[0].content)
|
||||
assert body['msg_type']=='interactive' and json.loads(body['content'])==card
|
||||
assert seen[1].method=='PATCH' and seen[1].url.path.endswith('/om_receipt')
|
||||
with pytest.raises(PlatformError):await platform.reply_task_card('../other',card,'a'*32,token='fake')
|
||||
assert len(seen)==2
|
||||
finally:await platform.close()
|
||||
@@ -16,6 +16,40 @@ async def upload(env,turn='task-durable',text='synthetic-private-result'):
|
||||
return await env['client'].post(path,json=wire(env,path,task_id=turn,result=text))
|
||||
|
||||
|
||||
async def test_execution_status_order_replay_terminal_and_delivery_are_separate(env):
|
||||
task(env)
|
||||
path='/work/v2/tasks/progress'
|
||||
async def report(state,sequence,turn='task-durable'):
|
||||
return await env['client'].post(path,json=wire(env,path,task_id=turn,
|
||||
execution_state=state,status_sequence=sequence))
|
||||
assert (await report('running',1)).json()['execution_state']=='running'
|
||||
waiting=await report('waiting_for_approval',2)
|
||||
assert waiting.json()['status_sequence']==2
|
||||
assert (await report('running',1)).json()['execution_state']=='waiting_for_approval'
|
||||
assert (await report('waiting_for_approval',2)).status_code==200
|
||||
assert (await report('running',2)).status_code==409
|
||||
completed=await report('completed',3)
|
||||
assert completed.json()['execution_state']=='completed'
|
||||
assert not completed.json()['result_received']
|
||||
assert (await report('running',4)).status_code==409
|
||||
await upload(env)
|
||||
assert (await report('completed',3)).json()['result_received']
|
||||
|
||||
|
||||
async def test_execution_state_auth_binding_and_parameter_validation(env):
|
||||
task(env)
|
||||
path='/work/v2/tasks/progress'
|
||||
missing=await env['client'].post(path,json=wire(env,path,task_id='task-durable',execution_state='running'))
|
||||
assert missing.status_code==422
|
||||
task(env,'foreign-task','ou_b','b'*32)
|
||||
foreign=await env['client'].post(path,json=wire(env,path,task_id='foreign-task',execution_state='running',status_sequence=1))
|
||||
assert foreign.status_code==403
|
||||
env['store'].unbind('ou_a')
|
||||
revoked=await env['client'].post(path,json=wire(env,path,task_id='task-durable',execution_state='running',status_sequence=1))
|
||||
assert revoked.status_code==403
|
||||
assert not env['store'].db.execute('SELECT * FROM work_execution_state_v3').fetchall()
|
||||
|
||||
|
||||
async def test_result_is_encrypted_durable_and_delivered_once_after_worker_loss(env):
|
||||
task(env)
|
||||
api=env['app'].state.deliveries
|
||||
|
||||
Reference in New Issue
Block a user