3 Commits
16 changed files with 688 additions and 12 deletions
+8
View File
@@ -15,3 +15,11 @@ HN_LISTEN_ADDRESS=0.0.0.0
HN_HTTPS_PORT=443
COMPOSE_PROJECT_NAME=hainuo-relay
HN_IMAGE=hainuo-relay:deploy-20261027
# Opt-in paired-client Work files; deployment defaults, not measured product promises.
HN_WORK_FILES_ENABLED=false
HN_WORK_FILE_MAX_BYTES=10485760
HN_WORK_FILE_BATCH_LIMIT=20
HN_WORK_FILE_RETENTION=86400
HN_WORK_FILE_OWNER_QUOTA_BYTES=536870912
HN_WORK_FILE_GLOBAL_QUOTA_BYTES=2147483648
+27
View File
@@ -0,0 +1,27 @@
# Work materials / artifact delivery (test deployment accepted)
Branch feat/work-files-20261110. Requires paired PC feat/client-foundation-20261110 and broker mode. Off by default. Existing users/keys/permissions are not migrated by this source change.
Flow: platform-verified private file messages stage encrypted materials; user sends all files, waits for receipts, then sends task instructions. The next task in that exact owner/device/binding/chat claims them. Device capability work-files-v1 is signed in its handshake; older PCs are told to upgrade instead of receiving a text-only task that silently omits files. In-progress downloads block dispatch; failed inputs remain explicit per-file records.
Separate /work/v2/files/list, /download, /upload, /state routes use current device signatures and task binding. Upload metadata is signed and supplied in X-Hainuo-File-Request; bytes have exact length/SHA256 and independent path size limits. No caller URL/recipient or arbitrary source message is accepted. Downloads are verified on PC. Output manifests are restricted to the current task outputs, not arbitrary local paths.
All material/output bytes are AES-GCM encrypted using the existing server keyring with scoped AAD. Server stores ciphertext in DB, so backups, disk capacity and retention matter. File content is cleared after confirmed send, expiry or binding discard. Metadata persists for idempotency. Per-owner/global quotas serialize checks across workers.
Output stages: ready -> uploading -> uploaded -> sending -> sent. Failure after a mutating platform call is unknown and never automatically replayed. Client lost upload receipt first queries original file state before re-upload; ECS received is not platform sent, and sent is not user read. Stale uploading/sending states become unknown. Uploading to Work and replying to the original source_message_id are separate durable steps.
Deployment defaults (configurable, not measured product commitments): 10MiB/file, 20/task, 24h retention, 512MiB/owner and 2GiB/global encrypted payload allowance. Encryption/base64 and DB backups require extra capacity. Verify corporate reverse proxy limits too. Formatted documents are not executed; macro/embedded/external active-content handling remains PC policy.
Platform adapters use GET im/v1/messages/{message_id}/resources/{file_key}?type=file; POST im/v1/files multipart; POST im/v1/messages/{message_id}/reply file content. Scope and private-platform compatibility must be checked against internal docs and real synthetic test files. These are bot message resource APIs, distinct from user-token docx media download.
Tests: synthetic platform events, encrypted routing over loopback WebSocket, signed PC requests to actual FastAPI routes, 3 Word/Excel/PDF materials, real local Office conversion, lost HTTP receipt recovery, one original-message file reply, cross-user/binding checks, conflicting IDs, quotas, body limits and invalid names. Mock platform acceptance does not prove real Work functionality.
Existing ECS parent confirmed by user: /home/cnooc/hainuo-broker-update-20261105-recovery, connector image hainuo-relay:broker-update-30585d6d28a18521, digest sha256:a32e13a92d2191ad0619a8f34e4703fd3e3ce8e44ca784f2b0d092cc49476b02. Before deployment verify router also matches parent, source branch availability and inactive tasks. prepare-update --enable-work-files creates a new plan with the flag; rollback keeps the parent's previous flag and existing key material. Do not run from the initial prepared plan or regenerate keyring. No production deployment performed.
## Live test acceptance, 2026-11-10
Candidate 79c15ee was activated on the test ECS using /home/cnooc/hainuo-broker-update-20261110-files, image hainuo-relay:broker-update-f6b1859625e89b5f. Connector/router reported healthy. This supersedes the historical pre-deployment status above. Future prepare-update operations must use this newly activated plan as parent; retain bindings and key material.
A real private Work task received Word, Excel and PDF materials, verified their hashes on the updated PC, and returned two generated files to the original conversation. Both remote file states were sent; text was replied. User screenshots confirmed both outputs opened in WPS: department totals 120 and 80, formula checks/chart consistent, and the report disclosed missing metadata and the 19.5 difference between source totals 200 and 219.5.
Acceptance is scoped to this task. Full installer release, network/restart faults, partial platform failures, and real multi-user/device acceptance remain pending. No master merge or new client installer is implied.
+6
View File
@@ -82,6 +82,12 @@ services:
HN_FEISHU_OAUTH_SCOPES: ${HN_FEISHU_OAUTH_SCOPES:-}
HN_ENROLLMENT_SIGNER_URL: http://enrollment-signer:8000
HN_REQUEST_TIMEOUT: ${HN_REQUEST_TIMEOUT:-180}
HN_WORK_FILES_ENABLED: ${HN_WORK_FILES_ENABLED:-false}
HN_WORK_FILE_MAX_BYTES: ${HN_WORK_FILE_MAX_BYTES:-10485760}
HN_WORK_FILE_BATCH_LIMIT: ${HN_WORK_FILE_BATCH_LIMIT:-20}
HN_WORK_FILE_RETENTION: ${HN_WORK_FILE_RETENTION:-86400}
HN_WORK_FILE_OWNER_QUOTA_BYTES: ${HN_WORK_FILE_OWNER_QUOTA_BYTES:-536870912}
HN_WORK_FILE_GLOBAL_QUOTA_BYTES: ${HN_WORK_FILE_GLOBAL_QUOTA_BYTES:-2147483648}
HTTP_PROXY: ''
HTTPS_PROXY: ''
ALL_PROXY: ''
+14 -2
View File
@@ -23,19 +23,31 @@ async def rate_limit(redis, bucket: str, limit=60, seconds=60):
class BodyLimitMiddleware:
def __init__(self, app, max_body=262144):
def __init__(self, app, max_body=262144, path_limits=None):
self.app, self.max_body = app, max_body
self.path_limits=path_limits or {}
async def __call__(self, scope, receive, send):
if scope["type"] != "http":
return await self.app(scope, receive, send)
if scope.get('path')in self.path_limits:
# Authenticate upload metadata before buffering its binary body.
class FileBodyTooLarge(Exception):pass
size=0
async def bounded():
nonlocal size
message=await receive();size+=len(message.get('body',b''))
if size>self.path_limits[scope['path']]:raise FileBodyTooLarge()
return message
try:return await self.app(scope,bounded,send)
except FileBodyTooLarge:return await JSONResponse({'detail':'body too large'},status_code=413)(scope,receive,send)
messages, size = [], 0
while True:
message = await receive()
if message["type"] == "http.disconnect":
return
size += len(message.get("body", b""))
if size > self.max_body:
if size > self.path_limits.get(scope.get('path'),self.max_body):
return await JSONResponse({"detail": "body too large"}, status_code=413)(scope, receive, send)
messages.append(message)
if not message.get("more_body", False):
+11
View File
@@ -40,6 +40,12 @@ class Settings(BaseSettings):
max_devices: int = 1000
max_connections: int = 1000
max_result: int = 16000
work_files_enabled: bool = False
work_file_max_bytes: int = 10*1024*1024
work_file_batch_limit: int = 20
work_file_retention: int = 86400
work_file_owner_quota_bytes: int = 512*1024*1024
work_file_global_quota_bytes: int = 2*1024*1024*1024
allow_insecure_local: bool = False
enrollment_signer_url: str = ""
credential_mode: str = "legacy"
@@ -50,6 +56,11 @@ class Settings(BaseSettings):
@model_validator(mode="after")
def validate_runtime(self):
if not 1024<=self.work_file_max_bytes<=100*1024*1024 or not 1<=self.work_file_batch_limit<=50 or not 300<=self.work_file_retention<=7*86400:
raise ValueError('invalid work file deployment limits')
if self.work_files_enabled and self.credential_mode!='broker':raise ValueError('work files require broker mode')
if not self.work_file_max_bytes<=self.work_file_owner_quota_bytes<=self.work_file_global_quota_bytes:
raise ValueError('invalid work file storage quotas')
if not 30 <= self.task_tracking_timeout <= 3600:
raise ValueError("task tracking timeout must be 30-3600 seconds")
if self.credential_mode not in {"legacy", "broker"}:
+37 -4
View File
@@ -65,6 +65,7 @@ class Broker:
self.pending: dict[str, Pending] = {}
self.worker_id = uuid.uuid4().hex
self.lifecycle = None
self.files = None
async def replies(self, pub):
async for message in pub.listen():
@@ -144,6 +145,11 @@ class Broker:
except Exception:
self.store.audit(owner, "dispatch", host, "identity_rejected")
raise HTTPException(409, "device identity rejected") from None
file_capable=bool(self.files and proof.get('work_files')=='work-files-v1' and chat_id and source_message_id)
if self.files and self.files.pending_count(owner,host,device['binding_generation'],chat_id,True):
return {'status':'materials_receiving','turn_id':turn,'result':'材料仍在接收,尚未执行任务。请等附件接收回执后再发送任务要求。'}
if self.files and self.files.pending_count(owner,host,device['binding_generation'],chat_id) and not file_capable:
return {'status':'unavailable','turn_id':turn,'result':'材料已保留,但当前客户端尚未支持附件处理;请升级配套客户端后再发任务要求。'}
if not self.store.begin_turn(turn, owner, host):
return {"status": "duplicate", "turn_id": turn, "result": "该请求已接收,不会重复执行。"}
task = None
@@ -154,6 +160,12 @@ class Broker:
except ValueError:
self.store.finish(turn,"unbound")
return {"status":"unbound","turn_id":turn,"result":"绑定已变化,任务未下发。"}
if task and file_capable:
try:self.files.claim(task)
except ValueError:
self.store.finish(turn,'not_dispatched')
self.lifecycle.transition(turn,'inflight','completed')
return {'status':'not_dispatched','turn_id':turn,'result':'材料接收或设备绑定状态发生变化,尚未执行任务;请等附件回执后核对。'}
private = x25519.X25519PrivateKey.generate()
future = asyncio.get_running_loop().create_future()
self.pending[turn] = Pending(future, private, public, host, session)
@@ -161,6 +173,8 @@ class Broker:
"text": text, "read_only": True, "channel": "im",
"reply_key": raw_public(private.public_key())}
if task:
if file_capable:
payload['file_transfer']='work-files-v1'
if source_message_id:
payload['result_delivery']='durable-http-v1'
payload["task_context"] = {key:task[key] for key in ("environment","app_id","owner","host_id","binding_generation",
@@ -234,6 +248,9 @@ def create_app(settings=None, redis=None, platform=None):
lifecycle = DeviceLifecycle(store.lifecycle,r,broker)
from .task_delivery import TaskDelivery
deliveries = TaskDelivery(oauth,store.lifecycle,broker,f)
if s.work_files_enabled:
from .work_files import WorkFiles
broker.files=WorkFiles(s,oauth,store.lifecycle,f)
else:
oauth = OAuth(s, r, f)
oauth.store = store
@@ -275,6 +292,7 @@ def create_app(settings=None, redis=None, platform=None):
try:
await lifecycle.maintain()
await deliveries.recover_interrupted_sends()
if broker.files:await broker.files.flush()
await deliveries.flush()
except Exception:
# No credentials or raw exception text in background logs.
@@ -297,11 +315,15 @@ def create_app(settings=None, redis=None, platform=None):
await r.aclose()
app = FastAPI(title="Hainuo B1 connector", lifespan=lifespan, docs_url=None, redoc_url=None)
app.add_middleware(BodyLimitMiddleware, max_body=s.max_body)
app.add_middleware(BodyLimitMiddleware, max_body=s.max_body,
path_limits={'/work/v2/files/upload':s.work_file_max_bytes}if broker.files else {})
app.include_router(oauth.router)
if lifecycle:
app.include_router(deliveries.router)
app.state.deliveries=deliveries
if broker.files:
app.include_router(broker.files.router)
app.state.files=broker.files
app.include_router(notification_router(s, r, store, f, oauth))
from .web_session import WebSessions
web_sessions = WebSessions(s, r, store, broker, f)
@@ -528,7 +550,8 @@ def create_app(settings=None, redis=None, platform=None):
message, sender = event.get("message", {}), event.get("sender", {})
owner = sender.get("sender_id", {}).get("open_id")
message_id = message.get("message_id", "")
if (message.get("chat_type") != "p2p" or message.get("message_type") != "text"
kind=message.get('message_type')
if (message.get("chat_type") != "p2p" or kind not in ({'text','file'}if broker.files else {'text'})
or sender.get("sender_type") != "user" or not isinstance(owner,str)
or not 0 < len(owner) <= 128 or any(ord(c) < 32 for c in owner)
or not re.fullmatch(r"[A-Za-z0-9_-]{1,128}", message_id)):
@@ -539,13 +562,23 @@ def create_app(settings=None, redis=None, platform=None):
or any(ord(c) < 32 for c in message["chat_id"])):
return
try:
text = json.loads(message["content"])["text"].strip()
if not text or len(text) > 8000:
content=json.loads(message['content'])
text = content['text'].strip() if kind=='text' else ''
if kind=='text' and (not text or len(text) > 8000):
return
# Persistent receipt BEFORE any side effect. Restarts do not execute a delivery twice.
event_turn = "event_" + hashlib.sha256(message_id.encode()).hexdigest()
if not store.begin_turn(event_turn, owner, "event"):
return
if kind=='file':
try:
await broker.files.receive(owner,message['chat_id'],message_id,content)
result='材料已接收。可以继续发送其他材料,最后发出本次任务要求;暂未执行模型任务。'
except Exception:
result='这件附件未接收成功,请核对文件格式、大小、资源权限或设备绑定;其他已接收材料仍保留。'
try:await f.reply(message_id,result)
finally:store.finish(event_turn,'attachment_processed')
return
if text == "/绑定":
result = await make_binding_code(owner)
elif text == "/解绑":
+33
View File
@@ -122,6 +122,39 @@ class Feishu:
headers={"Authorization": f"Bearer {token}"},
json={"msg_type": "text", "content": json.dumps({"text": text}, ensure_ascii=False)})
async def download_message_file(self,message_id,file_key,max_bytes):
import re
if not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',message_id)or not re.fullmatch(r'[A-Za-z0-9_-]{1,256}',file_key):raise PlatformError('invalid file reference')
token=await self.tenant_token()
try:
async with self.http.stream('GET',f'/open-apis/im/v1/messages/{message_id}/resources/{file_key}',
params={'type':'file'},headers={'Authorization':'Bearer '+token},timeout=60)as response:
if response.status_code!=200 or response.headers.get('content-type','').lower().startswith(('application/json','text/html')):raise PlatformError('message file download rejected')
raw=bytearray()
async for chunk in response.aiter_bytes():
raw.extend(chunk)
if len(raw)>max_bytes:raise PlatformError('message file size limit')
if not raw:raise PlatformError('empty message file')
return bytes(raw)
except httpx.HTTPError:raise PlatformError('message file download failed')from None
async def upload_file(self,name,content,mime,*,token):
data=await self._json('POST','/open-apis/im/v1/files',headers={'Authorization':'Bearer '+token},
data={'file_type':'stream','file_name':name},files={'file':(name,content,mime)},timeout=60)
key=data.get('data',{}).get('file_key')
import re
if not isinstance(key,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,256}',key):raise PlatformError('file upload receipt invalid')
return key
async def reply_file(self,message_id,file_key,delivery_id,*,token):
import re
if not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',message_id):raise PlatformError('invalid reply reference')
data=await self._json('POST',f'/open-apis/im/v1/messages/{message_id}/reply',
headers={'Authorization':'Bearer '+token},json={'msg_type':'file','content':json.dumps({'file_key':file_key}), 'uuid':delivery_id})
message=data.get('data',{}).get('message_id')
if not isinstance(message,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,128}',message):raise PlatformError('file message receipt invalid')
return message
async def send(self, owner, msg_type, content, delivery_id, *, token=None):
token = token or await self.tenant_token()
data = await self._json('POST', '/open-apis/im/v1/messages',
+1 -1
View File
@@ -1,4 +1,4 @@
"""Connector-owned persistent metadata. Never stores message/file/token bodies."""
"""Core routing metadata; adjunct services store payloads only as authenticated ciphertext."""
import sqlite3
import time
import psycopg
+229
View File
@@ -0,0 +1,229 @@
"""Opt-in, device/task-bound encrypted materials and artifact outbox."""
import base64
import hashlib
import json
import re
import time
from pathlib import PurePosixPath
from fastapi import APIRouter,HTTPException,Request
from fastapi.responses import JSONResponse,Response
from pydantic import Field
from .common import no_cache
from .crypto import canonical
from .task_delivery import TaskRequest
FORMATS={'.docx':'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
'.xlsx':'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
'.pptx':'application/vnd.openxmlformats-officedocument.presentationml.presentation',
'.pdf':'application/pdf','.txt':'text/plain','.csv':'text/csv','.md':'text/markdown'}
def file_name(value):
if (not isinstance(value,str)or not 1<=len(value)<=200 or value!=PurePosixPath(value).name
or any(c in value for c in '\\/:\r\n\x00')or any(ord(c)<32 for c in value)
or value.endswith((' ','.'))or PurePosixPath(value).suffix.lower()not in FORMATS):
raise ValueError('unsupported_file_name_or_format')
if value.split('.')[0].upper()in {'CON','PRN','AUX','NUL',*('COM'+str(i)for i in range(1,10)),*('LPT'+str(i)for i in range(1,10))}:raise ValueError('unsupported_file_name_or_format')
return value
class FileRequest(TaskRequest):
file_id:str=Field(pattern=r'^[A-Za-z0-9_-]{8,100}$')
class UploadRequest(TaskRequest):
artifact_id:str=Field(pattern=r'^[A-Za-z0-9_-]{8,100}$')
name:str=Field(min_length=1,max_length=200)
sha256:str=Field(pattern=r'^[a-f0-9]{64}$')
size:int=Field(strict=True,ge=1)
class WorkFiles:
def __init__(self,settings,auth,state,platform):
self.s,self.auth,self.state,self.platform=settings,auth,state,platform
self.db=state.db;self.keyring=auth.broker.store.keyring
self.environment,self.app_id=state.environment,state.app_id
with self.db:
self.db.execute('CREATE TABLE IF NOT EXISTS work_file_quota_v1(environment TEXT NOT NULL,app_id TEXT NOT NULL,epoch BIGINT NOT NULL,PRIMARY KEY(environment,app_id))')
self.db.execute('''CREATE TABLE IF NOT EXISTS work_files_v1(
file_id TEXT PRIMARY KEY,environment TEXT NOT NULL,app_id TEXT NOT NULL,
owner TEXT NOT NULL,host_id TEXT NOT NULL,binding_generation BIGINT NOT NULL,
chat_id TEXT NOT NULL,task_id TEXT NOT NULL,source_message_id TEXT NOT NULL,
direction TEXT NOT NULL,name TEXT NOT NULL,size BIGINT NOT NULL,sha256 TEXT NOT NULL,
key_id TEXT NOT NULL,ciphertext TEXT NOT NULL,status TEXT NOT NULL,
platform_key TEXT NOT NULL,platform_message TEXT NOT NULL,
updated DOUBLE PRECISION NOT NULL,expires DOUBLE PRECISION NOT NULL)''')
self.router=APIRouter()
self.router.add_api_route('/work/v2/files/list',self.list_files,methods=['POST'])
self.router.add_api_route('/work/v2/files/download',self.download,methods=['POST'])
self.router.add_api_route('/work/v2/files/upload',self.upload,methods=['POST'])
self.router.add_api_route('/work/v2/files/state',self.file_state,methods=['POST'])
def _aad(self,row):
return canonical(['work-files-v1',*[row[k]for k in ('environment','app_id','owner','host_id','binding_generation',
'chat_id','source_message_id','direction','file_id','sha256','size','name')]])
def _id(self,kind,*parts):return kind+'_'+hashlib.sha256(canonical([self.environment,self.app_id,*parts])).hexdigest()
def reserve(self,owner,size):
# Called inside the owner's transaction. Serialize global quota checks
# across PostgreSQL workers as well as the SQLite test implementation.
scope=(self.environment,self.app_id)
self.db.execute('INSERT INTO work_file_quota_v1 VALUES(?,?,0) ON CONFLICT DO NOTHING',scope)
self.db.execute('UPDATE work_file_quota_v1 SET epoch=epoch+0 WHERE environment=? AND app_id=?',scope)
total=self.db.execute("SELECT COALESCE(SUM(size),0) AS n FROM work_files_v1 WHERE environment=? AND app_id=? AND ciphertext!='' AND expires>?",(*scope,time.time())).fetchone()['n']
own=self.db.execute("SELECT COALESCE(SUM(size),0) AS n FROM work_files_v1 WHERE environment=? AND app_id=? AND owner=? AND ciphertext!='' AND expires>?",(*scope,owner,time.time())).fetchone()['n']
if total+size>self.s.work_file_global_quota_bytes or own+size>self.s.work_file_owner_quota_bytes:raise HTTPException(413,'file_storage_quota')
def _insert(self,*,file_id,owner,host,generation,chat,task,message,direction,name,raw,status,reserved_size=None):
row=dict(file_id=file_id,environment=self.environment,app_id=self.app_id,owner=owner,
host_id=host,binding_generation=generation,chat_id=chat,task_id=task,source_message_id=message,
direction=direction,name=name,size=len(raw)if reserved_size is None else reserved_size,sha256=hashlib.sha256(raw).hexdigest())
key,cipher=self.keyring.encrypt(raw,self._aad(row))
self.db.execute('INSERT INTO work_files_v1 VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT DO NOTHING',
(*[row[k]for k in ('file_id','environment','app_id','owner','host_id','binding_generation','chat_id','task_id',
'source_message_id','direction','name','size','sha256')],key,cipher,status,'','',time.time(),time.time()+self.s.work_file_retention))
return dict(self.db.execute('SELECT * FROM work_files_v1 WHERE file_id=?',(file_id,)).fetchone())
async def receive(self,owner,chat,message,content):
"""Only called after platform-event validation and persistent event receipt."""
device=self.state.store.by_owner(owner)
if not device:raise ValueError('bound_device_required')
name=file_name(content.get('file_name'));key=content.get('file_key')
if not isinstance(key,str)or not re.fullmatch(r'[A-Za-z0-9_-]{1,256}',key):raise ValueError('invalid_platform_file_reference')
identity=(owner,device['host_id'],device['binding_generation'])
fid=self._id('in',*identity,message)
with self.db:
self.state._owner_lock(owner)
self.auth.require_binding(device['host_id'],owner,device['binding_generation'])
if self.pending_count(owner,device['host_id'],device['binding_generation'],chat)>=self.s.work_file_batch_limit:raise ValueError('pending_material_count_limit')
old=self.db.execute('SELECT * FROM work_files_v1 WHERE file_id=?',(fid,)).fetchone()
if old:return self.metadata(dict(old))
self.reserve(owner,self.s.work_file_max_bytes)
self._insert(file_id=fid,owner=owner,host=device['host_id'],generation=device['binding_generation'],
chat=chat,task='',message=message,direction='input',name=name,raw=b'',status='receiving',reserved_size=self.s.work_file_max_bytes)
try:
raw=await self.platform.download_message_file(message,key,self.s.work_file_max_bytes)
if not raw or len(raw)>self.s.work_file_max_bytes:raise ValueError('file_size_limit')
with self.db:
self.state._owner_lock(owner)
self.auth.require_binding(device['host_id'],owner,device['binding_generation'])
row=dict(self.db.execute('SELECT * FROM work_files_v1 WHERE file_id=?',(fid,)).fetchone())
if row['status']!='receiving':raise ValueError('material_state_changed')
row.update(size=len(raw),sha256=hashlib.sha256(raw).hexdigest())
key_id,cipher=self.keyring.encrypt(raw,self._aad(row))
self.db.execute("UPDATE work_files_v1 SET size=?,sha256=?,key_id=?,ciphertext=?,status='staged',updated=? WHERE file_id=? AND status='receiving'",(len(raw),row['sha256'],key_id,cipher,time.time(),fid))
row['status']='staged'
return self.metadata(row)
except Exception:
with self.db:self.db.execute("UPDATE work_files_v1 SET status='failed',size=0,ciphertext='',updated=? WHERE file_id=? AND status='receiving'",(time.time(),fid))
raise
def pending_count(self,owner,host,generation,chat,receiving_only=False):
statuses=('receiving',)if receiving_only else('staged','receiving','failed')
placeholders=','.join('?'for _ in statuses)
return self.db.execute("SELECT COUNT(*) AS n FROM work_files_v1 WHERE environment=? AND app_id=? AND owner=? AND host_id=? AND binding_generation=? AND chat_id=? AND status IN ("+placeholders+") AND expires>?",(self.environment,self.app_id,owner,host,generation,chat,*statuses,time.time())).fetchone()['n']
def claim(self,task):
with self.db:
self.state._owner_lock(task['owner'])
if not self.state.task_current(task):raise ValueError('stale_task')
if self.pending_count(task['owner'],task['host_id'],task['binding_generation'],task['chat_id'],True):raise ValueError('materials_still_receiving')
self.db.execute("UPDATE work_files_v1 SET task_id=?,status=CASE WHEN status='failed' THEN 'failed_assigned' ELSE 'assigned' END,updated=? WHERE environment=? AND app_id=? AND owner=? AND host_id=? AND binding_generation=? AND chat_id=? AND direction='input' AND status IN ('staged','failed') AND expires>?",(task['turn_id'],time.time(),self.environment,self.app_id,task['owner'],task['host_id'],task['binding_generation'],task['chat_id'],time.time()))
def current(self,body):
self.auth.require_binding(body.host_id,body.binding_owner,body.binding_generation)
task=self.state.task(body.task_id)
if (not self.state.task_current(task)or not task['source_message_id']or
(task['owner'],task['host_id'],task['binding_generation'])!=(body.binding_owner,body.host_id,body.binding_generation)
or task['status']in {'unbound','revoked','result_discarded','offline'}):raise HTTPException(403,'task_not_recoverable')
return task
def metadata(self,row):
return {k:row[k]for k in ('file_id','direction','name','size','sha256','status')}
async def list_files(self,body:TaskRequest,request:Request):
await self.auth.authenticate(body,request);self.current(body)
rows=self.db.execute("SELECT * FROM work_files_v1 WHERE environment=? AND app_id=? AND task_id=? AND expires>? ORDER BY file_id",(self.environment,self.app_id,body.task_id,time.time())).fetchall()
return no_cache(JSONResponse({'protocol':'work-files-v1','files':[self.metadata(dict(r))for r in rows],
'max_bytes':self.s.work_file_max_bytes,'batch_limit':self.s.work_file_batch_limit}))
def selected(self,body):
task=self.current(body)
row=self.db.execute('SELECT * FROM work_files_v1 WHERE file_id=? AND task_id=? AND environment=? AND app_id=?',
(body.file_id,body.task_id,self.environment,self.app_id)).fetchone()
if not row or row['expires']<=time.time():raise HTTPException(404,'file_not_available')
if (row['owner'],row['host_id'],row['binding_generation'])!=(task['owner'],task['host_id'],task['binding_generation']):raise HTTPException(403,'file_not_available')
return dict(row)
async def download(self,body:FileRequest,request:Request):
await self.auth.authenticate(body,request);row=self.selected(body)
if row['direction']!='input' or row['status']!='assigned':raise HTTPException(403,'input_file_required')
raw=self.keyring.decrypt(row['key_id'],row['ciphertext'],self._aad(row))
if len(raw)!=row['size']or hashlib.sha256(raw).hexdigest()!=row['sha256']:raise HTTPException(409,'file_integrity_failed')
self.current(body)
return no_cache(Response(raw,media_type='application/octet-stream',headers={'X-Hainuo-File-SHA256':row['sha256']}))
async def upload(self,request:Request):
encoded=request.headers.get('X-Hainuo-File-Request','')
if not encoded or len(encoded)>24000:raise HTTPException(422,'file_metadata_required')
try:body=UploadRequest.model_validate(json.loads(base64.b64decode(encoded,altchars=b'-_',validate=True)))
except Exception:raise HTTPException(422,'invalid_file_metadata')from None
await self.auth.authenticate(body,request);task=self.current(body)
try:name=file_name(body.name)
except ValueError:raise HTTPException(422,'unsupported_file_name_or_format')from None
if body.size>self.s.work_file_max_bytes:raise HTTPException(413,'file_size_limit')
raw=await request.body()
if len(raw)!=body.size or hashlib.sha256(raw).hexdigest()!=body.sha256:raise HTTPException(409,'file_integrity_failed')
fid=self._id('out',task['turn_id'],body.artifact_id)
with self.db:
self.state._owner_lock(task['owner']);self.current(body)
old=self.db.execute('SELECT * FROM work_files_v1 WHERE file_id=?',(fid,)).fetchone()
if old and (old['expires']<=time.time()or old['status']in {'expired','discarded'}):raise HTTPException(410,'file_expired')
if not old:
count=self.db.execute("SELECT COUNT(*) AS n FROM work_files_v1 WHERE task_id=? AND direction='output'",(body.task_id,)).fetchone()['n']
if count>=self.s.work_file_batch_limit:raise HTTPException(413,'artifact_count_limit')
self.reserve(task['owner'],len(raw))
row=self._insert(file_id=fid,owner=task['owner'],host=task['host_id'],generation=task['binding_generation'],
chat=task['chat_id'],task=task['turn_id'],message=task['source_message_id'],direction='output',name=name,raw=raw,status='ready')
if (row['sha256'],row['size'],row['name'])!=(body.sha256,body.size,name):raise HTTPException(409,'artifact_conflict')
return no_cache(JSONResponse({'received':True,**self.metadata(row)}))
async def file_state(self,body:FileRequest,request:Request):
await self.auth.authenticate(body,request)
return no_cache(JSONResponse(self.metadata(self.selected(body))))
async def flush(self):
now=time.time()
with self.db:
self.db.execute("UPDATE work_files_v1 SET status='expired',ciphertext='',platform_key='' WHERE environment=? AND app_id=? AND expires<? AND ciphertext!=''",(self.environment,self.app_id,now))
self.db.execute("UPDATE work_files_v1 SET status='failed',size=0,ciphertext='',updated=? WHERE environment=? AND app_id=? AND status='receiving' AND updated<?",(now,self.environment,self.app_id,now-120))
# Interrupted mutating platform calls remain ambiguous, never replayed.
self.db.execute("UPDATE work_files_v1 SET status='unknown',updated=? WHERE environment=? AND app_id=? AND status IN ('uploading','sending') AND updated<?",(now,self.environment,self.app_id,now-120))
rows=self.db.execute("SELECT * FROM work_files_v1 WHERE environment=? AND app_id=? AND direction='output' AND status IN ('ready','uploaded') AND expires>? ORDER BY updated LIMIT 10",(self.environment,self.app_id,now)).fetchall()
for raw_row in rows:
row=dict(raw_row);task=self.state.task(row['task_id'])
if not self.state.task_current(task):
with self.db:self.db.execute("UPDATE work_files_v1 SET status='discarded',ciphertext='',platform_key='' WHERE file_id=?",(row['file_id'],))
continue
try:token=await self.platform.tenant_token()
except Exception:continue
with self.db:
self.state._owner_lock(row['owner'])
if not self.state.task_current(self.state.task(row['task_id'])):continue
working='uploading'if row['status']=='ready'else'sending'
changed=self.db.execute('UPDATE work_files_v1 SET status=?,updated=? WHERE file_id=? AND status=?',(working,time.time(),row['file_id'],row['status'])).rowcount
if changed!=1:continue
try:
if working=='uploading':
content=self.keyring.decrypt(row['key_id'],row['ciphertext'],self._aad(row))
if hashlib.sha256(content).hexdigest()!=row['sha256']:raise ValueError()
key=await self.platform.upload_file(row['name'],content,FORMATS[PurePosixPath(row['name']).suffix.lower()],token=token)
with self.db:self.db.execute("UPDATE work_files_v1 SET status='uploaded',platform_key=?,updated=? WHERE file_id=? AND status='uploading'",(key,time.time(),row['file_id']))
else:
response=await self.platform.reply_file(task['source_message_id'],row['platform_key'],hashlib.sha256(row['file_id'].encode()).hexdigest()[:32],token=token)
with self.db:self.db.execute("UPDATE work_files_v1 SET status='sent',platform_message=?,ciphertext='',updated=? WHERE file_id=? AND status='sending'",(response,time.time(),row['file_id']))
except Exception:
with self.db:self.db.execute("UPDATE work_files_v1 SET status='unknown',updated=? WHERE file_id=? AND status=?",(time.time(),row['file_id'],working))
+8 -2
View File
@@ -196,7 +196,7 @@ def validate_overlay(base, merged, candidate, router_key_required=True):
env = new[name]["environment"]
for key, value in old[name].get("environment", {}).items():
if key not in {"HN_CREDENTIAL_MODE","HN_ROUTER_ADMISSION_SECRET","HN_CREDENTIAL_KEYRING_FILE",
"HN_CREDENTIAL_PERMISSION_PROFILES","HN_CONNECTOR_INTERNAL_URL","HN_FEISHU_APP_ID","HN_OAUTH_ENABLED"}:
"HN_CREDENTIAL_PERMISSION_PROFILES","HN_CONNECTOR_INTERNAL_URL","HN_FEISHU_APP_ID","HN_OAUTH_ENABLED","HN_WORK_FILES_ENABLED"}:
if env.get(key) != value:
raise DeployError("existing environment changed: "+key)
old_volumes = {v["target"]:v for v in old[name].get("volumes", [])}
@@ -373,6 +373,8 @@ def prepare_update(args):
raise DeployError('update output must be a new absolute directory outside the live deployment')
if not (source/'relay/connector.py').is_file():
raise DeployError('missing update source')
if getattr(args,'enable_work_files',False)and not (source/'relay/work_files.py').is_file():
raise DeployError('work files implementation missing from update source')
output=output.resolve()
output.mkdir(mode=0o700)
os.chmod(output,0o700)
@@ -389,8 +391,11 @@ def prepare_update(args):
for mount in overlay['services']['connector']['volumes']:
if mount['target']=='/run/hainuo-credentials/keyring.json':
mount['source']=str(output/'credential-keyring.json')
write_json(output/'compose.broker.json',overlay)
rollback=json.loads(json.dumps(overlay))
if getattr(args,'enable_work_files',False):
overlay['services']['connector']['environment']['HN_WORK_FILES_ENABLED']='true'
plan['work_files_enabled']=True
write_json(output/'compose.broker.json',overlay)
for name in ('connector','router'):
rollback['services'][name]['image']=roles[name]['image']
# An update rollback must retain broker env/key mounts, never silently
@@ -708,6 +713,7 @@ def main():
update.add_argument("--plan",required=True)
update.add_argument("--source-root",default=str(Path(__file__).resolve().parents[1]))
update.add_argument("--output",required=True)
update.add_argument('--enable-work-files',action='store_true',help='Opt in after paired-client preparation; rollback retains the previous flag.')
permissions=sub.add_parser('update-permissions')
permissions.add_argument('--plan',required=True)
permissions.add_argument('--profiles',required=True)
+15
View File
@@ -643,3 +643,18 @@ def test_private_approved_profile_excludes_pending_review_scopes():
approved=set(json.loads((ROOT/'examples/work-product-profiles.approved.json').read_text())['current-config-v1'])
assert all_scopes-approved=={'contact:department.base:readonly','contact:department.organize:readonly','task:tasklist:write'}
assert len(approved)==20 and {'drive:drive','wiki:wiki','bitable:app','task:task:write'}<=approved
def test_prepare_update_file_feature_is_explicit_and_rollback_restores_parent_flag(scenario):
parent=prepared(scenario);root=scenario['output'];image='sha256:'+'a'*64
for name in ('connector','router'):scenario['roles'][name]['image']=image
deploy.write_json(root/'candidate.json',{'image':image,'tag':parent['candidate_image']})
output=root.parent/'update-files'
args=argparse.Namespace(plan=str(root),source_root=str(ROOT),output=str(output),enable_work_files=True)
deploy.prepare_update(args)
current=json.loads((output/'compose.broker.json').read_text())
rollback=json.loads((output/'compose.rollback.json').read_text())
old=json.loads((root/'compose.broker.json').read_text())
assert current['services']['connector']['environment']['HN_WORK_FILES_ENABLED']=='true'
assert rollback['services']['connector']['environment']==old['services']['connector']['environment']
assert (output/'credential-keyring.json').read_bytes()==(root/'credential-keyring.json').read_bytes()
+2 -2
View File
@@ -74,7 +74,7 @@ class Platform:
@pytest_asyncio.fixture
async def env(tmp_path):
async def env(tmp_path,request):
ca_key, ca = issue_ca()
ca_path = tmp_path / "ca.pem"
ca_path.write_bytes(ca.public_bytes(serialization.Encoding.PEM))
@@ -89,7 +89,7 @@ async def env(tmp_path):
feishu_app_id="cli_test", feishu_app_secret="fake-app-secret", credential_mode="broker",
credential_keyring_file=keyring, credential_permission_profiles={"product-v1": ["read"], "product-v2": ["read", "write"]},
public_url="https://relay.example.test", feishu_api_base="https://work.example.test",
feishu_authorize_url="https://work.example.test/open-apis/authen/v1/index")
feishu_authorize_url="https://work.example.test/open-apis/authen/v1/index",work_files_enabled=bool(getattr(request,'param',False)))
redis = fakeredis.aioredis.FakeRedis(server=fakeredis.FakeServer(), decode_responses=True)
platform = Platform()
app = create_app(settings, redis, platform)
+2 -1
View File
@@ -52,7 +52,7 @@ async def routed(env, tmp_path):
await asyncio.wait_for(server_task,5)
async def device(env, host="a"*32, *, generation=None, legacy=False):
async def device(env, host="a"*32, *, generation=None, legacy=False, work_files=False):
ws = await connect(env["url"],proxy=None)
env["sessions"].append(ws)
challenge = json.loads(await ws.recv())["challenge"]
@@ -65,6 +65,7 @@ async def device(env, host="a"*32, *, generation=None, legacy=False):
plain.update(protocol_version=2,environment=env["settings"].environment,
app_id=env["settings"].feishu_app_id,binding_owner=row["owner"] or "ou_a",
binding_generation=row["binding_generation"] if generation is None else generation)
if work_files:plain['work_files']='work-files-v1'
await ws.send(json.dumps({"host_id":host,"cert":row["cert"],"proof":sign(plain,env["keys"][host])}))
accepted = json.loads(await asyncio.wait_for(ws.recv(),3))
assert accepted["type"] == "connected"
+154
View File
@@ -0,0 +1,154 @@
import asyncio
import base64
import hashlib
import json
import time
from pathlib import Path
import pytest
from test_credential_api import env,wire
from relay.work_files import WorkFiles
pytestmark=pytest.mark.parametrize('env',[True],indirect=True)
def create_task(env,name='task-files',owner='ou_a',host='a'*32,chat='chat-files'):
generation=env['store'].device(host)['binding_generation']
return env['store'].lifecycle.create_task(name,owner,host,generation,'session','worker',time.time()+600,chat,'om-original')
def fake_platform(env,monkeypatch):
files={};calls=[]
async def download(message,key,limit):calls.append(('download',message,key));return b'PRIVATE_MATERIAL_'+key.encode()
async def upload(name,raw,mime,*,token):
calls.append(('upload',name));key='file_'+hashlib.sha256(raw).hexdigest();files[key]=raw;return key
async def reply(message,key,delivery,*,token):calls.append(('reply_file',message,key,delivery));return 'om-file-reply'
monkeypatch.setattr(env['platform'],'download_message_file',download,raising=False)
monkeypatch.setattr(env['platform'],'upload_file',upload,raising=False)
monkeypatch.setattr(env['platform'],'reply_file',reply,raising=False)
return files,calls
async def upload(env,task='task-files',raw=b'PRIVATE_GENERATED_RESULT',artifact='artifact-01',**overrides):
path='/work/v2/files/upload'
data=dict(task_id=task,artifact_id=artifact,name='result.txt',sha256=hashlib.sha256(raw).hexdigest(),size=len(raw));data.update(overrides)
body=wire(env,path,**data)
return await env['client'].post(path,content=raw,headers={'X-Hainuo-File-Request':base64.urlsafe_b64encode(json.dumps(body).encode()).decode()})
async def test_material_event_is_staged_without_model_and_duplicate_does_not_redownload(env,monkeypatch):
_,calls=fake_platform(env,monkeypatch)
event={'header':{'event_type':'im.message.receive_v1','app_id':'cli_test'},'event':{
'sender':{'sender_type':'user','sender_id':{'open_id':'ou_a'}},
'message':{'chat_type':'p2p','chat_id':'chat-files','message_type':'file','message_id':'om-input',
'content':json.dumps({'file_name':'source.txt','file_key':'file-source'})}}}
for _ in range(2):
assert (await env['client'].post('/internal/events',json=event,headers={'X-Internal-Secret':env['settings'].internal_secret})).status_code==200
await asyncio.sleep(.02)
assert len([x for x in calls if x[0]=='download'])==1
row=env['store'].db.execute('SELECT * FROM work_files_v1').fetchone()
assert row['status']=='staged' and row['task_id']==''
assert 'PRIVATE_MATERIAL'not in str(dict(row))
assert env['store'].db.execute('SELECT COUNT(*) AS n FROM work_tasks_v2').fetchone()['n']==0
assert len(env['platform'].replies)==1
async def test_claim_download_and_cross_owner_rejection(env,monkeypatch):
fake_platform(env,monkeypatch);api=env['app'].state.files
incoming=await api.receive('ou_a','chat-files','om-input',{'file_name':'source.txt','file_key':'file-source'})
task=create_task(env);api.claim(task)
path='/work/v2/files/list';listed=await env['client'].post(path,json=wire(env,path,task_id='task-files'))
assert listed.json()['files'][0]['file_id']==incoming['file_id']
path='/work/v2/files/download';body=wire(env,path,task_id='task-files',file_id=incoming['file_id'])
received=await env['client'].post(path,json=body)
assert received.content==b'PRIVATE_MATERIAL_file-source'
assert received.headers['X-Hainuo-File-SHA256']==hashlib.sha256(received.content).hexdigest()
bad=await env['client'].post(path,json=wire(env,path,host='b'*32,task_id='task-files',file_id=incoming['file_id']))
assert bad.status_code==403
assert (await env['client'].post(path,json=body)).status_code==409
async def test_artifact_handoff_encrypted_idempotent_and_replies_to_original_only(env,monkeypatch):
files,calls=fake_platform(env,monkeypatch);create_task(env)
first=await upload(env);assert first.status_code==200 and first.json()['received']
assert (await upload(env)).json()['file_id']==first.json()['file_id']
row=env['store'].db.execute("SELECT * FROM work_files_v1 WHERE direction='output'").fetchone()
assert 'PRIVATE_GENERATED_RESULT'not in str(dict(row))
assert (await upload(env,raw=b'changed')).status_code==409
api=WorkFiles(env['settings'],env['api'],env['store'].lifecycle,env['platform'])
await api.flush();await api.flush();await api.flush()
assert len([x for x in calls if x[0]=='upload'])==1
replies=[x for x in calls if x[0]=='reply_file'];assert len(replies)==1 and replies[0][1]=='om-original'
assert files[replies[0][2]]==b'PRIVATE_GENERATED_RESULT'
assert env['store'].db.execute("SELECT status,ciphertext FROM work_files_v1").fetchone()['status']=='sent'
async def test_bad_digest_filename_size_and_stale_binding_are_rejected(env,monkeypatch):
fake_platform(env,monkeypatch);create_task(env)
assert (await upload(env,sha256='0'*64)).status_code==409
assert (await upload(env,name='../secret.txt')).status_code==422
assert (await upload(env,size=env['settings'].work_file_max_bytes+1)).status_code==413
env['store'].unbind('ou_a')
assert (await upload(env)).status_code==403
assert env['store'].db.execute('SELECT COUNT(*) AS n FROM work_files_v1').fetchone()['n']==0
async def test_ambiguous_platform_upload_or_send_never_repeats(env,monkeypatch):
_,calls=fake_platform(env,monkeypatch);create_task(env);assert (await upload(env)).status_code==200
async def uncertain(*a,**kw):calls.append(('uncertain',));raise ConnectionError('PRIVATE_PLATFORM_ERROR')
monkeypatch.setattr(env['platform'],'upload_file',uncertain)
api=env['app'].state.files;await api.flush();await api.flush()
assert len([x for x in calls if x[0]=='uncertain'])==1
assert env['store'].db.execute('SELECT status FROM work_files_v1').fetchone()['status']=='unknown'
async def test_unbind_before_delivery_discards_without_platform_upload(env,monkeypatch):
_,calls=fake_platform(env,monkeypatch);create_task(env);await upload(env)
env['store'].unbind('ou_a');await env['app'].state.files.flush()
assert not calls
row=env['store'].db.execute('SELECT status,ciphertext FROM work_files_v1').fetchone()
assert row['status']=='discarded'and row['ciphertext']==''
async def test_file_body_limit_is_separate_from_json_limit(env,monkeypatch):
fake_platform(env,monkeypatch);create_task(env)
raw=b'x'*(env['settings'].max_body+1)
assert (await upload(env,raw=raw)).status_code==200
response=await env['client'].post('/work/v2/files/list',content=b'x'*(env['settings'].max_body+1))
assert response.status_code==413
async def test_receiving_material_blocks_dispatch_and_failed_input_is_explicit(env,monkeypatch):
fake_platform(env,monkeypatch);api=env['app'].state.files;entered=asyncio.Event();resume=asyncio.Event()
async def slow(*args):entered.set();await resume.wait();raise ConnectionError('synthetic download failed')
monkeypatch.setattr(env['platform'],'download_message_file',slow)
receiving=asyncio.create_task(api.receive('ou_a','chat-files','om-slow',{'file_name':'source.txt','file_key':'file-slow'}))
await entered.wait()
assert api.pending_count('ou_a','a'*32,env['store'].device('a'*32)['binding_generation'],'chat-files',True)==1
task=create_task(env)
with pytest.raises(ValueError,match='receiving'):api.claim(task)
resume.set()
with pytest.raises(ConnectionError):await receiving
api.claim(task)
row=env['store'].db.execute('SELECT status,task_id,ciphertext FROM work_files_v1').fetchone()
assert row['status']=='failed_assigned'and row['task_id']=='task-files'and row['ciphertext']==''
async def test_owner_quota_blocks_storage_without_upload_or_download(env,monkeypatch):
_,calls=fake_platform(env,monkeypatch);api=env['app'].state.files;create_task(env)
monkeypatch.setattr(env['settings'],'work_file_owner_quota_bytes',10)
response=await upload(env)
assert response.status_code==413 and not calls
from fastapi import HTTPException
with pytest.raises(HTTPException)as error:await api.receive('ou_a','chat-files','om-quota',{'file_name':'source.txt','file_key':'file-source'})
assert error.value.status_code==413 and not calls
async def test_lost_platform_message_receipt_is_unknown_without_repeat(env,monkeypatch):
_,calls=fake_platform(env,monkeypatch);create_task(env);await upload(env)
api=env['app'].state.files;await api.flush()
async def unknown(*args,**kw):calls.append(('reply_unknown',));raise ConnectionError('lost message receipt')
monkeypatch.setattr(env['platform'],'reply_file',unknown)
await api.flush();await api.flush()
assert len([x for x in calls if x[0]=='reply_unknown'])==1
assert env['store'].db.execute('SELECT status FROM work_files_v1').fetchone()['status']=='unknown'
@@ -0,0 +1,92 @@
"""Optional cross-checkout test using actual PC transport/signatures, no real Work."""
import asyncio
import importlib.util
import importlib
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import time
from types import SimpleNamespace
import httpx
import pytest
from test_credential_api import env
from test_work_files import create_task,fake_platform
from relay.crypto import pem_private
pytestmark=pytest.mark.parametrize('env',[True],indirect=True)
async def test_three_materials_native_output_lost_receipt_and_original_file_reply(env,tmp_path,monkeypatch):
source=os.environ.get('HN_PC_FILES_SOURCE')
if not source:pytest.skip('explicit PC source checkout required')
source=Path(source);package=source/'integrations/work-relay/service/relay'
spec=importlib.util.spec_from_file_location('_pc_work_file_chain',package/'__init__.py',submodule_search_locations=[str(package)])
module=importlib.util.module_from_spec(spec);sys.modules[spec.name]=module;spec.loader.exec_module(module)
files_module=importlib.import_module(spec.name+'.work_files')
credentials=importlib.import_module(spec.name+'.credential_provider')
office_spec=importlib.util.spec_from_file_location('_office_chain',source/'capability-slim/runtime/office_workflows.py')
office=importlib.util.module_from_spec(office_spec);office_spec.loader.exec_module(office)
remote_files,platform_calls=fake_platform(env,monkeypatch)
samples=os.environ.get('HN_PC_SAMPLE_DIR');native_python=os.environ.get('HN_OFFICE_PYTHON')
names=['材料.docx','表格.xlsx','参考.pdf']if samples else['one.txt','two.csv','three.md']
if samples:
paths=[Path(samples)/n for n in ['报告模板.docx','材料一.xlsx','文本材料.pdf']]
raw_materials=[p.read_bytes()for p in paths]
else:raw_materials=[b'synthetic material 1',b'column,value\nx,1',b'# source']
mapping={f'file-input-{i}':content for i,content in enumerate(raw_materials)}
async def download(message,key,limit):return mapping[key]
monkeypatch.setattr(env['platform'],'download_message_file',download)
api=env['app'].state.files
for i,name in enumerate(names):await api.receive('ou_a','chat-files',f'om-input-{i}',{'file_name':name,'file_key':f'file-input-{i}'})
task=create_task(env);api.claim(task)
context={k:task[k]for k in ['environment','app_id','owner','host_id','binding_generation','chat_id','source_message_id','conversation_id']}
context.update(task_id='task-files',channel='work-dm')
device=tmp_path/'pc-device';device.mkdir();(device/'device.json').write_text(json.dumps({'host_id':'a'*32,'owner':'ou_a','binding_generation':task['binding_generation']}))
(device/'device.key').write_bytes(pem_private(env['keys']['a'*32]))
loop=asyncio.get_running_loop();requests=[];lost=[False]
def forward(request):
requests.append(request.url.path)
future=asyncio.run_coroutine_threadsafe(env['client'].request(request.method,request.url.path,
content=request.read(),headers=dict(request.headers)),loop)
response=future.result(20)
if request.url.path.endswith('/upload')and not lost[0]:
lost[0]=True;raise httpx.ReadError('synthetic lost upload receipt')
return httpx.Response(response.status_code,headers=dict(response.headers),content=response.content)
transport=httpx.Client(base_url='https://relay.example.test',transport=httpx.MockTransport(forward))
provider=object.__new__(credentials.CredentialProvider)
provider.directory=device;provider.host='a'*32;provider.owner='ou_a';provider.wall=time.time;provider.http=transport
provider.release={'environment':'test','app_id':'cli_test'};provider.cache_key=('synthetic',)
client=SimpleNamespace(provider=provider,http=transport)
home=tmp_path/'pc-home'
try:
staged=await asyncio.to_thread(files_module.stage_materials,client,context,home)
assert len(staged['materials'])==3 and all(m['status']=='ready'for m in staged['materials'])
assert sorted(Path(m['path']).read_bytes()for m in staged['materials'])==sorted(raw_materials)
root=files_module.task_workspace(context,home)
if native_python and samples:
doc=next(m['path']for m in staged['materials']if m['name'].endswith('.docx'))
output=root/'outputs/converted.pdf'
request={'action':'export_pdf','arguments':{'source':doc,'output':str(output)}}
completed=await asyncio.to_thread(subprocess.run,[native_python,'-X','utf8',str(source/'capability-slim/runtime/office_workflows.py')],
input=json.dumps(request).encode(),capture_output=True,timeout=120)
assert completed.returncode==0,completed.stderr
assert json.loads(completed.stdout)['status']=='completed',completed.stdout
assert output.read_bytes().startswith(b'%PDF')
else:
output=root/'outputs/result.txt';output.write_text('synthetic final output')
office.register_artifacts(root,[output])
first=await asyncio.to_thread(files_module.deliver_artifacts,client,context,home)
assert first['unknown']==1
second=await asyncio.to_thread(files_module.deliver_artifacts,client,context,home)
assert second['received']==1 and second['unknown']==0
assert requests.count('/work/v2/files/upload')==1 # Read state instead of resending bytes.
await api.flush();await api.flush();await api.flush()
sent=[x for x in platform_calls if x[0]=='reply_file']
assert len(sent)==1 and sent[0][1]=='om-original'
assert remote_files[sent[0][2]]==output.read_bytes()
assert len([x for x in platform_calls if x[0]=='upload'])==1
finally:transport.close()
@@ -0,0 +1,49 @@
import asyncio
import hashlib
import json
import pytest
from test_credential_api import env
from test_device_lifecycle import routed,device,instruction,reply,send_event,event,wait_for_reply
from test_work_files import fake_platform
pytestmark=pytest.mark.parametrize('env',[True],indirect=True)
def file_event(env,message_id='om-file',chat='chat-a',owner='ou_a'):
value=event(env,message_id,'',chat=chat,owner=owner)
value['event']['message'].update(message_type='file',content=json.dumps({'file_name':'source.txt','file_key':'file-source'}))
return value
async def test_file_then_instruction_uses_encrypted_routing_and_capability_gate(routed,monkeypatch):
env=routed;_,calls=fake_platform(env,monkeypatch)
connection=await device(env,work_files=True)
await send_event(env,file_event(env));await wait_for_reply(env['platform'],1)
await send_event(env,event(env,'om-instruction','请处理刚才的材料'))
payload=await instruction(env,connection)
assert payload['file_transfer']=='work-files-v1'
task_id='msg_'+hashlib.sha256(b'om-instruction').hexdigest()
assert payload['task_context']['task_id']==task_id
rows=env['store'].db.execute('SELECT task_id,status FROM work_files_v1').fetchall()
assert len(rows)==1 and rows[0]['task_id']==task_id and rows[0]['status']=='assigned'
await reply(env,connection,payload,'synthetic result')
await wait_for_reply(env['platform'],2)
assert len([x for x in calls if x[0]=='download'])==1
async def test_old_client_does_not_receive_a_task_that_silently_omits_materials(routed,monkeypatch):
env=routed;fake_platform(env,monkeypatch);connection=await device(env)
await send_event(env,file_event(env));await wait_for_reply(env['platform'],1)
await send_event(env,event(env,'om-instruction','请处理材料'));await wait_for_reply(env['platform'],2)
assert '升级' in env['platform'].replies[-1][1]
assert env['store'].db.execute('SELECT status FROM work_files_v1').fetchone()['status']=='staged'
assert env['store'].db.execute('SELECT COUNT(*) AS n FROM work_tasks_v2').fetchone()['n']==0
async def test_invalid_upload_metadata_is_rejected_before_binary_body_is_read(env):
consumed=[]
async def content():
consumed.append(True);yield b'not read'
response=await env['client'].post('/work/v2/files/upload',content=content())
assert response.status_code==422 and not consumed