+ lHj{&\a0t$Rt^RIHt^RIt^RIt^RIt^RIt^RIt^RI t ^RI t ^RI t ^RI H t ^RIHtHtHtHtHt^RIHtHt^RIHt^RIHt^RIHtHt^R IHtH t ]PB!]"4t#Rs$R ]%R &] PL!4t'^s(R ]%R &] PL!4t)/t*R]%R&/t+R]%R&/t,R]%R&Rt-^t.^2t/Rt0Rt1^t2Rt3Rt4] PL!4t5Rt6Rt7] PL!4t8]9!4t:R]%R&]9!4t;R]%R&/tR t?R!t@] PL!4tARsBR"]%R#&] P!4tD0RmtERtFRtGRtHRtIR$tJR%R&ltKR'R(ltLR)tMR*R+ltNR,R-ltOR.R/ltPR0R1ltQR2R3ltRR4R5ltSR6R7ltTR8R9ltUR:R;ltVRR?lltWR@R/RARBlltXR@R/RCRDlltYRERFltZRGRHlt[RIRJlt\RKRLlt]RMRNlt^RORPlt_RQRRlt`RSRTltaRURVltbRWRXltcRYRZltdR[R\lteR]R^ltfR_R`ltgRaRblth]i!0Rm4tjRtkRcRdltlRRf^R@R/RgRhllltmRiRjltnRkRlltoRmRnltpRoRpltqRRqRrlltrRsRtltsRuRvlttRwRxltuRyRzltvR{R|ltwR}R~ltxRRltyRRRRRR/RRlltzRRRReRReRRR].RR/RRllt{RRRReRReRRR].RRRRRRRRRR/ RRllt|RRlt}RRlt~RRltRRltRRltRRltRRltRRltRRltRRltRRltRRltRRRlltRRRlltRReRR/RRlltRRltRRltRRltR#)a Async (background) delegation registry behind ``delegate_task(background=true)``. The parent dispatches a subagent on a module-level daemon executor and returns a handle immediately. On completion a ``type="async_delegation"`` event (self-contained task-source block) is pushed onto the SHARED ``process_registry.completion_queue`` the CLI/gateway drain while idle, so results surface as a NEW turn (never mid-turn) and inherit its de-dup and crash-recovery wiring. Only the async lifecycle lives here; the child run is an injected ``runner``.) annotationsN)ThreadPoolExecutor)AnyCallableDictListOptional)get_hermes_homehermes_home_key)DaemonThreadPoolExecutor)propagate_context_to_thread) worker_ledgerrequire_ledger_owner)redact_child_resultredact_completion_eventzOptional[ThreadPoolExecutor] _executorint_executor_max_workerszdict[str, dict[str, Any]]_recordszdict[Any, frozenset]_refusedzdict[tuple, set] _refused_by session_keyparent_session_idigr@g>@gN@set_offered_startup_replayedzdict[str, float]_last_orphan_sweepg |@g@g^@zOptional[threading.Thread]_monitor_threadc$\4R, #)zstate.db)r ;/home/ubuntu/.hermes/hermes-agent/tools/async_delegation.py_db_pathr"ks  z ))r cV^8dQhRR/#)returnsqlite3.Connectionr)formats"r! __annotate__r(os$r c ^RIHpHpV!4'd V!R4h^RIHp^RIHp^RIHp\4pV!VP4V!VRR7V!VRR R R \R 7pV!V4V#) )WorkerPersistenceErroris_worker_process$worker_delegation_ledger_unavailable)open_db)mkdir_under_hermes_home)_secure_state_db_filesT) create_mainzstate.db (async_delegation)i'FN)db_labelbusy_timeout_mswal row_factory initialize) agent.runtime_session_storer+r,hermes_cli.sqlite_utilr.hermes_constantsr/ hermes_stater0r"parent_initialize_schema)r+r,r.r/r0pathconns r!_connectr?osjU$%KLL.93 :DDKK(4T2 4"?QW$;M OD4 Kr c V^8dQhRRRR/#)r$r>r&r%Noner)r's"r!r(r(s ! !/ !D !r c>^RIHp^RIHpV!V4V!V4R#)r*)apply_durability_barriers)reconcile_state_schemaN)hermes_state_repairrChermes_state_schemarD)r>rCrDs& r!r<r<s=:d#4 r c.^RIHpV!\44#)r* transaction)r8rIr?rHs r! _transactionrJs2 xz ""r cV^8dQhRR/#r$r%dict[str, Any]r)r's"r!r(r(sr c^RIHp\Uu/uF(q!RVP4 2R4;p'gK&WbK* up#uupi \d/u#i;i)zSnapshot scope_id/user_id/user_name on the PARENT thread (the daemon worker has no contextvars) so a restart-replayed completion can rebuild a SessionSource. Best-effort: empty values are omitted.get_session_envHERMES_SESSION_)gateway.session_contextrP _ROUTING_KEYSupper Exception)rPkvs r!_capture_routing_originrYs^;+im_WXW^W^W`VaEbdf5g0g0gmiii  s&A#AAAA AAc V^8dQhRRRR/#)r$recordrMr%rAr)r's"r!r(r(snr c\4;p'dVPV4#\P!4p^RIHpV!\ P !44p.R O\OUu/uFpWP9gK WPPV4bK pp\ P!4VR&\;_uu_4\4;_uu_4pVPRVR,VPRR4VPRR4VPR 4VR ,V\ P !4V\P !V4VPR R43 4RRR4RRR4\#4R# \dRpEL'i;iuupi \dLi;i +'giLN;i +'giLY;i) r*)get_process_start_timeN owner_cwda~INSERT OR REPLACE INTO async_delegations (delegation_id, origin_session, origin_ui_session_id, parent_session_id, state, dispatched_at, updated_at, delivery_state, delivery_attempts, owner_pid, owner_started_at, task_json, origin_session_id) VALUES (?, ?, ?, ?, 'running', ?, ?, 'pending', 0, ?, ?, ?, ?) delegation_idrrRorigin_ui_session_idr dispatched_atorigin_session_id) goalgoalscontexttoolsetsrolemodelis_batch task_indexestask_transcripts)r dispatchtimegateway.statusr]osgetpidrVrTgetgetcwdOSError_DB_LOCKrJexecutejsondumps_prune_durable_records)r[remotenowr]owner_started_atkey task_payloadr>s& r!_persist_dispatchr~sq v v&& ))+C 91"))+> MLMMC = ZZ_M $&IIK [! <>>T Q O $fjj&CVZZPfhjEk ZZ+ ,f_.EsBIIKYi ZZ %vzz2Er'J L  M")       !>sT!E2'F4F F 2F/BF F/2 FF FF F, 'F// F? cV^8dQhRR/#r$r%rAr)r's"r!r(r(sBBBr c n\4\P!4\, p\;_uu_4\ 4;_uu_4pVP RV34VP R4P 4^,pV\8dVP RV\, 34VP R4P 4^,pV\8dVP RV\, 34RRR4RRR4R# +'giL;i +'giR#;i)zBBound terminal history, preferring delivered records for deletion.zQDELETE FROM async_delegations WHERE delivery_state='delivered' AND updated_at < ?zRSELECT COUNT(*) FROM async_delegations WHERE state NOT IN ('running','finalizing')aXDELETE FROM async_delegations WHERE delegation_id IN ( SELECT delegation_id FROM async_delegations WHERE state NOT IN ('running','finalizing') ORDER BY CASE delivery_state WHEN 'delivered' THEN 0 ELSE 1 END, updated_at ASC LIMIT ? )z~SELECT COUNT(*) FROM async_delegations WHERE state NOT IN ('running','finalizing') AND delivery_state='pending'aDELETE FROM async_delegations WHERE delegation_id IN ( SELECT delegation_id FROM async_delegations WHERE state NOT IN ('running','finalizing') AND delivery_state='pending' ORDER BY updated_at ASC LIMIT ? )N) rrm_DURABLE_RETENTION_SECONDSrtrJrufetchone_MAX_RETAINED_COMPLETED_MAX_DURABLE_PENDING)cutoffr>terminal_count pending_counts r!rxrxs YY[5 5F <>>T _bhaj l `bbjbjblmnp 3 3 LL )+BBD  F  &[\\d\d\fghj / / LL(*>>@  B">>s$D# B2D >D# D D## D4 c$V^8dQhRRRRRR/#)r$eventrMresultr%rAr)r's"r!r(r(s& M M~ M~ M$ Mr c \V4\V4r\4;p'dVPW4#\P!4p\ ;_uu_4\ 4;_uu_4pVPRVPRR4VPRV4V\P!V4\P!V4VR,34RRR4RRR4R# +'giL;i +'giR#;i)zUPDATE async_delegations SET state=?, completed_at=?, updated_at=?, event_json=?, result_json=?, delivery_state='pending' WHERE delegation_id=?status completed completed_atr_N) rrr completermrtrJrurqrvrw)rrryrzr>s&& r!_persist_completionrs+E24G4O6 v u-- ))+C <>>T (YYx -uyy/Ms ZZ  6 2E/4J L M">>s%C</A(C) C<) C9 4C<< D c$V^8dQhRRRRRR/#)r$r_strentryrMr%rAr)r's"r!r(r(s&IISIIDIr c \V4p\4;p'dVPW4#\;_uu_4\ 4;_uu_4pVP RV34P 4pVfRRR4RRR4R#\\P!V^,;'gR4;'g/4pVPR4;'g.Uu.uF+qfPR4VPR48wgK)VNK- ppVPV4VP R\P!RVRR/4\P!4V34RRR4RRR4R#uupi +'giL;i +'giR#;i \d,\PR YPR4RR 7R#i;i) anDurably record ONE finished child of a still-running multi-child unit on the unit's own row, so a crash before the unit joins loses only the children that had not finished. Stored in ``result_json`` (overwritten by the real result at finalize); ``recover_abandoned_delegations`` replays it. Best-effort: a failed write costs recovery fidelity, never the live result.zSSELECT result_json FROM async_delegations WHERE delegation_id=? AND state='running'N{}results task_indexzbUPDATE async_delegations SET result_json=?, updated_at=? WHERE delegation_id=? AND state='running'partialTz7Async delegation %s: could not record finished child %sexc_info)rr childrtrJrurrvloadsrqappendrwrmrVloggerwarning)r_rryr>rowrrrs&& r!record_unit_childrst  &E v ||M11 I X|~~,,t -/119 { &XX *$**SV^^t*D*J*JKG")++i"8">">B">q">Q%% BUY^YbYbcoYpBpqq">Gq NN5 ! LL}**i)T%JKTYY[Zgh j&XX r &~XXX IPR_ajajkwaxDH IIsF#F'E< 8FF# $E< / E< =E< E< !&E7 E7 AE< %F-F#7E< < F F F F# F##2GGc(V^8dQhRRRRRRRR/#) r$taskrM result_json Optional[str]errorrr%zOptional[list[dict[str, Any]]]r)r's"r!r(r(s0yy^y-yPSyXvyr c \P!T;'gR4;'g/pVPR4'd/VPR4'dVPR4'gR#VR,Uu/uF4p\VPR4\4'gK*VR,VbK6 ppVPR4;'g7\ \ \VPR4;'g.444pVUu.uF%quPV4;'g RVR R R RR V/NK' up#uupiuupi) zaPer-task results for an abandoned unit: recorded children as they finished, the rest ``unknown``.rrirrNrrjrdrunknownsummaryr)rvrrq isinstancerlistrangelen)rrrrrrecordedindexesis&&& r!_recovered_resultsrsjj,,-33G HHZ W[[%;%; I@V@V,3I,>g,>q*QUUS_M`beBf",",>Hghh~&SS$uS'9J9P9Pb5Q/R*SGpw xpwklLLO f f a9iQUW^`ef fpw xxh xs='D:)D: D?(D?cV^8dQhRR/#)r$r%z$Optional[Callable[[Any, Any], bool]]r)r's"r!r(r( s  = r c\aaa^RIHoHoHoRTTT3RllpT# \dR#i;i)z``alive(owner_pid, owner_started_at)`` over the shared drift-tolerant start-time comparator, or None when the liveness probes cannot be imported.) _pid_existsr]start_time_fingerprints_matchNcV^8dQhRR/#)r$r%boolr)r's"r!r(%_owner_liveness..__annotate__snntnr c<\V4;'dHS!\V44;'d/VRJ;'g#S!TS!\V44;'g^4#N)rr)pidstartedrr]rs&&r!alive_owner_liveness..alives`Cynn[S2nn tO l l0recover_abandoned_delegations../s!X&755?i;W11&7s( (z2Delegation owner exited before the unit finished; /zJ child results were recorded and are included below, the rest are unknown.last_known_statusrk)git_state_hinttranscript_tailsrr^rtypeasync_delegationr_rr`rbrRrrcrdrerfrgrhrirrrrrrarzUPDATE async_delegations SET state='unknown', completed_at=?, updated_at=?, event_json=?, result_json=?, delivery_state='pending' WHERE delegation_id=?)rrrmrtrJrufetchallrvrrsumrrq%tools.async_delegation_recovery_hintsrrrrTrrrw)rrz recoveredr>rowsrr_r origin_ui parent_idrarr task_json origin_sidr last_staterrrecovered_resultsdone diagnosticsrrtailshintrWrrs r!recover_abandoned_delegationsrs  E }YY[! <>>T||QRS[RZR\ CMP JM mRYfp@JS""::i//40DbE 24e L  X&7XXMdVSTUXYjUkTlm__. u F IT F6GI01B F  F(5D}! :1d1g:}D FE 9dGU7^WWbW;L)%67RTWF3E:R scK:AK' *AK' "K' *#K' K' *B5K' K"=K" K' 'A)K' K:"K' ' K7 2K:: L cV^8dQhRR/#rr)r's"r!r(r(NsLLSLr c ^RIHpV!4'd^#\4P4'g^#\ 4\ P !4p\ ;_uu_4\4;_uu_4pVPR4P4p\W4WRR7uuRRR4uuRRR4# +'giM;iRRR4R# +'giR#;i)uEnqueue durable pending completions as fresh turns after process start. Restored events are stamped ``restored=True`` in memory only: they came from a PREVIOUS process, so drains without an ownership filter must leave them for a consumer that can prove ownership. Rows older than ``_MAX_COMPLETION_REPLAY_AGE_S`` are terminally dropped instead of replaying a turn nobody is waiting on. Every restored event is stamped ``restored=True`` (in-memory only — the stamp is added after the durable payload is deserialized and is never persisted). Restored events originate from a *previous* process, so no consumer in THIS process implicitly owns them: drain paths that run without an ownership filter (the legacy single-session behavior) must leave them queued for a consumer that can positively prove ownership, otherwise a brand-new session adopts a dead session's delegation results seconds after boot (#64484). )r,zSELECT delegation_id, event_json, completed_at, dispatched_at FROM async_delegations WHERE state != 'running' AND delivery_state='pending' AND event_json IS NOT NULL ORDER BY completed_at, delegation_idT)startupN) r7r,r"existsrrmrtrJrur_replay_pending) target_queuer,rzr>rs& r!restore_undelivered_completionsrNs> :    !# ))+C <>>T||789A  t>>s$#C6-B8 # C8 C C C& rFc$V^8dQhRRRRRR/#)r$rzfloatrrr%rr)r's"r!r(r(ms"##5#d#WZ#r c\\44^reVEF}wrxrT ;'gT p V 'dWW;, \8dFVPRW734\P RWsV , R, \R, 4Kq\ ;_uu_4WW3\9d RRR4KRRR4\P!V4p \V \4'd \V 4p \ ;_uu_4V'd\PV4\V \4'dRV R&V\9dRV R&RRR4VPV 4\ ;_uu_4\PWW34RRR4V^, pEK V# +'giL;i +'giLq;i +'giLF;i)a)Put each pending ``(delegation_id, event_json, completed_at, dispatched_at)`` row on ``target_queue`` stamped ``restored``, or terminally drop it past ``_MAX_COMPLETION_REPLAY_AGE_S``. Records the offer so the orphan sweep skips the row until the copy is handed back (``return_completion_offer``).zUPDATE async_delegations SET delivery_state='dropped', delivery_claim=NULL, delivery_claimed_at=NULL, updated_at=? WHERE delegation_id=? AND delivery_state='pending'z|Async delegation %s: pending completion is %.1fh old (cap %.1fh); terminally dropping the replay (result remains queryable).g @NTrestoredstartup_replay)r r _MAX_COMPLETION_REPLAY_AGE_Srurr _orphan_lockrrvrrdictrraddput) r>rrrzrhomerr_payloadrra age_basisevts &&&&$ r!rrmsk%_%67(?C;  11M #/-II LLILOJ^ ` NNe(?f*DFbekFk m  \$0 \0 jj! c4 )#.C \!%%m4#t$$"&J $55,0C()   \ LL$. /A ;@D< O'\\\s%F  G rzc V^8dQhRRRR/#r$rzOptional[float]r%rr)r's"r!r(r(s*A*A_*APS*Ar c \4pVe\4P4'g^#\4Vf\P!4MTp\ \ 44p\;_uu_4\UUu0uFwrEWC8XgK VkK pppRRR4\;_uu_4\4;_uu_4pVPRV\, V\, 34P4p.p VFwwrZrrpVX9gV!W4'dKT;'g^\8d1VPRW34\ P#RV\4KdV P%WZW34Ky \'WyW4uuRRR4uuRRR4#uuppi +'giEL;i +'giM;iRRR4R# +'giR#;i)aOffer this home's completions whose owner died after THIS process started (#97202). Startup replay (``restore_undelivered_completions``) covers owners that died before the process started; this covers the rest while it runs. Abandoned in-flight rows are first classified by ``recover_abandoned_delegations``. A terminal row qualifies when it is pending with an event, idle past ``_ORPHAN_STALE_S``, not under a live delivery claim, and its owner fails the shared liveness check. A row is offered once per live in-memory copy: a consumer that discards the copy with the row still pending hands it back for the next sweep. The consumer's ``claim_completion_delivery`` stays the atomic cross-process gate, so two processes offering one row never both deliver it. Rows past the delivery budget or the replay age converge to ``dropped``. Reads the current profile's ledger: callers bind the owning profile first.NaSELECT delegation_id, event_json, completed_at, dispatched_at, owner_pid, owner_started_at, delivery_attempts FROM async_delegations WHERE state NOT IN ('running','finalizing') AND delivery_state='pending' AND event_json IS NOT NULL AND updated_at < ? AND (delivery_claim IS NULL OR delivery_claimed_at < ?) ORDER BY completed_at, delegation_idzUPDATE async_delegations SET delivery_state='dropped', delivery_claim=NULL, delivery_claimed_at=NULL, updated_at=? WHERE delegation_id=? AND delivery_state='pending'nAsync delegation %s exhausted its %d delivery attempts; marking terminally dropped (result remains queryable).)rr"rrrmr r rrrtrJru_ORPHAN_STALE_S_CLAIM_LEASE_Sr_MAX_DELIVERY_ATTEMPTSrrrr)rrzrrr|r_offeredr>rorphansrrrarrattemptss&$ r!sweep_orphaned_completionsrsv  E }HJ--//!#$))+#C ?, -D ;CS8%7Ss{==8S  <>>T||7 :=9NPSVdPd8e g hpgogq [_ WMLx'5+>+> A"88 MORNbd X,.DF NNMLP Q\`tl@-">T !>sP5 F? F F F-GA$F2 %AF2  GF F/ 2 G =G G c V^8dQhRRRR/#rr)r's"r!r(r(s?VYr c\\44pVf\P!4MTp\;_uu_4\ P V4pVeW, \8d RRR4^#V\ V&RRR4\V4# +'giL;i \d\PRRR7^#i;i)z``sweep_orphaned_completions`` at most once per ``ORPHAN_SWEEP_INTERVAL_S`` per home (``now`` is monotonic), for delivery loops that tick far more often. Never raises into the loop.Nz&Orphaned async delegation sweep failedTr) r r rm monotonicrrrqORPHAN_SWEEP_INTERVAL_SrrVrdebug)rrzrlasts&$ r! maybe_sweep_orphaned_completionsrs ?, -D!k$.. sC !%%d+   -D D $'4  ),77   = Ms#,B3 B B# B #"C C c$V^8dQhRRRRRR/#)r$sqlrparamstupler%rr)r's"r!r(r(s!77#7u77r c  \;_uu_4\4;_uu_4pVPW4P^8HuuRRR4uuRRR4# +'giM;iRRR4R# +'giR#;i)z?Run one UPDATE on the ledger; True iff exactly one row changed.N)rtrJrurowcount)r r r>s&& r!_update_deliveryrs? <>>T||C(11Q6">>>s"A2A A2 A% A22 B c V^8dQhRRRR/#r$r_rr%rr)r's"r!r(r(s``S`T`r c\\4\P!4p\RWV34#)zDAtomically acknowledge successful injection of a durable completion.zUPDATE async_delegations SET delivery_state='delivered', delivered_at=?, updated_at=? WHERE delegation_id=? AND delivery_state!='delivered')rrmr)r_rzs& r!mark_completion_deliveredrs1 ))+C  DFIP]E^ ``r c$V^8dQhRRRRRR/#r$r_rclaim_idr%rr)r's"r!r(r(s!!!S!C!D!r c \4;p'dVPW4#\P!4p\;_uu_4\ 4;_uu_4pVP RV34P 4pVfRRR4RRR4R#VP RWW0V\, 34pVP^8HuuRRR4uuRRR4# +'giM;iRRR4R# +'giR#;i)zBClaim one pending completion across competing consumers/processes.zBSELECT delivery_state FROM async_delegations WHERE delegation_id=?NTaUPDATE async_delegations SET delivery_claim=?, delivery_claimed_at=?, delivery_attempts=delivery_attempts+1, updated_at=? WHERE delegation_id=? AND delivery_state='pending' AND (delivery_claim IS NULL OR delivery_claimed_at < ?)) r claimrmrtrJrurrr)r_rryrzr>rcurs&& r!claim_completion_deliveryrs v ||M44 ))+C <>>Tll PS`Rbddldldn ; " llLCn0D E G ||q ">>>s0C3'C C3.C  C3 C& !C33 D c V^8dQhRRRR/#)r$rrMr%rr)r's"r!r(r(sZZ^ZZr cnVPR4R8H;'d\VPR44#)aAn early per-task notice for a batch that is still running. It shares the batch's ``delegation_id`` but is NOT the durable completion: it must never claim, acknowledge or dedup against the final result's row (independent review reproduced exactly that loss).rrtask_failure_notice)rqr)rs&r!is_interim_delegation_eventrs/ 776?0 0 Y YT#''BW:X5YYr c$V^8dQhRRRRRR/#)r$rrMconsumerrr%rr)r's"r!r(r(s&TTnTT Tr cR\V4'dR#VPR4R8Xd%\VPR4;'gR4MRpV'gR#V R\P!4 R\ P !4P 2p\W#4'dV#R#)zYClaim a durable delegation event; non-durable events (and interim notices) need no token.rRrrr_:N) rrqrrorpuuiduuid4hexr)rr r_rs&& r!claim_event_deliveryr&s"3'';>776?N`;`C066B7fhM 1RYY[M4::<+;+;*<=H0II8StSr c$V^8dQhRRRRRR/#rr)r's"r!r(r(s!!!s!c!d!r c 6\4;p'dVPW4#\P!4p\;_uu_4\ 4;_uu_4pVP RW0V\ 34pVP^8Xd/\PRV\ 4RRR4RRR4R#VP RW0V34pVP^8HuuRRR4uuRRR4# +'giM;iRRR4R# +'giR#;i)zRelease a failed delivery claim so another consumer may retry. Attempts are counted at claim time; once the budget is exhausted the row converges to terminal ``dropped`` (only pending rows replay on restart).a UPDATE async_delegations SET delivery_state='dropped', delivery_claim=NULL, delivery_claimed_at=NULL, updated_at=? WHERE delegation_id=? AND delivery_state='pending' AND delivery_claim=? AND delivery_attempts>=?rNTzUPDATE async_delegations SET delivery_claim=NULL, delivery_claimed_at=NULL, updated_at=? WHERE delegation_id=? AND delivery_state='pending' AND delivery_claim=?) r releasermrtrJrurrrr)r_rryrzr>cappedrs&& r!release_completion_deliveryr+s v ~~m66 ))+C <>>TB+A B D ??a  NNT(*@ B"ll),/x*HJ||q ">>>s1DAC* !D3"C*  D* C: 5D D c$V^8dQhRRRRRR/#rr)r's"r!r(r(s!00S0C0D0r c\4;p'dVPW4#\R\P!4W34#)zOReturn an unadmitted completion to pending without spending a delivery attempt.zUPDATE async_delegations SET delivery_claim=NULL, delivery_claimed_at=NULL, delivery_attempts=MAX(0, delivery_attempts-1), updated_at=? WHERE delegation_id=? AND delivery_state='pending' AND delivery_claim=?)r deferrrmr_rrys&& r!defer_completion_deliveryr0sA v ||M44 V m.  00r c$V^8dQhRRRRRR/#rr)r's"r!r(r((s& N NC N3 N4 Nr c\4;p'dVPW4#\R\P!4W34#)u,Terminally drop a claimed completion whose target is permanently gone (the spawning session ended at an explicit user boundary such as /new or reset). ``dropped`` — not ``delivered`` — keeps the ack honest; not ``pending`` keeps restart recovery from replaying it into a fail-closed drop forever.zUPDATE async_delegations SET delivery_state='dropped', updated_at=?, delivery_claim=NULL, delivery_claimed_at=NULL WHERE delegation_id=? AND delivery_state='pending' AND delivery_claim=?)r droprrmr/s&& r!drop_completion_deliveryr4(sD  v {{=33 %(,yy{M&L  NNr c$V^8dQhRRRRRR/#rr)r's"r!r(r(6s& K K Ks Kt Kr c\4;p'dVPW4#\P!4p\RW3W34#)z;Acknowledge acceptance for the consumer holding this claim.aUPDATE async_delegations SET delivery_state='delivered', delivered_at=?, updated_at=?, delivery_claim=NULL, delivery_claimed_at=NULL WHERE delegation_id=? AND delivery_state='pending' AND delivery_claim=?)r ackrmr)r_rryrzs&& r!complete_completion_deliveryr86sG v zz-22 ))+C %(+&I  KKr c V^8dQhRRRR/#rr)r's"r!r(r(BsCCsCTCr c \P!4p\;_uu_4\4;_uu_4pVPRW \34pVP ^8Xd/\ PRV\4RRR4RRR4R#VPRW 34pVP ^8wdRRR4RRR4R#VPRV34P4p\;_uu_4\PV4RRR4\W6W4^8HuuRRR4uuRRR4# +'giL3;i +'giM;iRRR4R# +'giR#;i)aXOffer an acknowledged completion again when its receiving turn could not confirm the owner. The row returns to pending and its event goes back on ``target_queue``, so the next delivery re-runs the pre-flight. A row past the delivery budget converges to ``dropped`` instead, which bounds the loop; a row without a stored event is left as it is.zUPDATE async_delegations SET delivery_state='dropped', updated_at=? WHERE delegation_id=? AND delivery_state='delivered' AND delivery_attempts>=?rNFzUPDATE async_delegations SET delivery_state='pending', delivered_at=NULL, updated_at=? WHERE delegation_id=? AND delivery_state='delivered' AND event_json IS NOT NULLzySELECT delegation_id, event_json, completed_at, dispatched_at FROM async_delegations WHERE delegation_id=?) rmrtrJrurrrrrrrdiscardr)r_rrzr> exhaustedreopenedrs&& r!requeue_admitted_completionr>Bs ))+C <>>TLL"` !7 8:     " NNT(*@ B"<<!b "    !"||?AN@PRRZRZR\ \  % %m 4t<=B'">"\#">sSE&AE =E&%E 4E&/E 5D6 E ! E&6 EE E E&& E7 c$V^8dQhRRRRRR/#r$rrMrrr%rAr)r's"r!r(r(_s&AAA3A4Ar c&\\W4R#r)_event_deliveryr8rrs&&r!complete_event_deliveryrD_s 0#@r c$V^8dQhRRRRRR/#r@r)r's"r!r(r(cs!!!!#!$!r c<\\W4\V4R#)zRelease a failed claim for a consumer that discards its copy (the TUI poller): the row is pending again, so it must stay eligible for the orphan sweep.N)rBr+return_completion_offerrCs&&r!release_event_deliveryrHcs/?C r c V^8dQhRRRR/#)r$rrMr%rAr)r's"r!r(r(js Y Y YD Yr c VPR4R8Xd%\VPR4;'gR4MRpV'd\V4'dR#\;_uu_4\P \Uu0uFq"^,V8XgKVkK up4RRR4R#uupi +'giR#;i)apHand an offered completion back to the orphan sweep after its in-memory copy was discarded while the durable row stays pending, e.g. a TUI session that cannot prove it owns the event drops it (every session poller drains one process-wide queue). The next sweep may offer the row again. Delegation ids are unique across profiles, so this clears the offer in every home.rrr_rRN)rqrrrrdifference_update)rr_r|s& r!rGrGjs B) B) B.)B.. B? c$V^8dQhRRRRRR/#r@r)r's"r!r(r(vs!::^:s:t:r cV'dFVPR4R8Xd.T!\VPR4;'gR4V4R#R#R#)rrr_rRN)rqr)fnrrs&&&r!rBrBvs<CGGFO'99 3sww'--2 .9:xr c V^8dQhRRRR/#)r$r_rr%zOptional[dict[str, Any]]r)r's"r!r(r({sHH#H2JHr c\4;p'dCVPV4pVe,VPR4e/VCR\VR,4/C#V#\;_uu_4\ 4;_uu_4pVP RV34P4pRRR4RRR4XfR#RTRV^,RV^,RV^,RV^,RV^,'d'\\P!V^,44MRRV^,R V^,R V^,;'gR / # +'giL;i +'giL;i) NrzSELECT origin_session, state, dispatched_at, completed_at, result_json, delivery_state, delivery_attempts, origin_session_id FROM async_delegations WHERE delegation_id=?r_origin_sessionstaterardelivery_statedelivery_attemptsrbrR) r readrqrrtrJrurrvr)r_rysavedr>rs& r!get_durable_delegationrW{s1 v  M*  8!4!@LeLX':5?'KL L <>>Tll?BO@PRS[RZR\ " ;4H(8#a&'3q6SbdghidjAUXYZU[U[*=djjQ>P*Qaegwy|}~ySV%8#a&,,B%HH ">s$#E6"D2 E2 E =E E c V^8dQhRRRR/#)r$rawrr%rMr)r's"r!r(r(s44m44r cV'd\P!V4M/p\T\4'dT#/# \d/u#i;ir)rvr ValueErrorrr)rYvalues& r! _json_objectr]sG#& 3Bud++533  s<< A  A rRlimitc ,V^8dQhRRRRRRRRRR /#) r$r`rrr^rrzrr%list[dict[str, Any]]r)r's"r!r(r(s4(((7:(KN(Zi((r cRV3RV33UUu.uFwrEV'gKWE3NK pppV'g.#VeTM\P!4\, pRPRV44p\;_uu_4\ 4;_uu_4p V P RV R2.RV4OVNVN54P 4p RRR4RRR4.p X EFewrrpp\V4\\V44pp\VPR 4\4'd!VR ,'dVPR 4MVPR 4;'gR .p\\VPR 4;'g\\V44V44p\VPR 4\4'd VR ,MVPR4'd.MV.pV'g9\!V 4P#4\$9dR^RVPR4/.pVEF p\!VPR4;'g T ;'gR 4P#4pV\$9dKM\VPR4\&4'dVPR4M^pVPR4;'gVPR4pT P)RT RTRTR \!VPVV^,4;'gR 4RV'd \!V4MRRVRV/4EK EKh V RV#uuppi +'giEL;i +'giEL;i)a=Recently failed async delegation tasks owned by a session, newest first. The live roster forgets a child once it ends and does not survive a renderer reload, so a failed delegation had nowhere to show (#97202). This reads the durable row instead: one entry per failed task (a batch unit that "completed" can still carry failed tasks) with ``delegation_id``, ``task_index``, ``goal``, ``status``, ``error``, ``dispatched_at`` and ``completed_at``. Either selector claims a row: the UI session id at dispatch, or the spawner's durable session id (survives a reload re-mint).r`rNz OR c30"TF wrV R2xK R#5i)z=?Nr)rcol_s& r!r1failed_delegations_for_session..s?Y63se2JYszSELECT delegation_id, state, dispatched_at, completed_at, task_json, result_json FROM async_delegations WHERE (zt) AND state NOT IN ('running','finalizing') AND completed_at >= ? ORDER BY completed_at DESC LIMIT ?c3*"TF wrVxK R#5irr)rrdvals& r!rres+vqssrdrcrRrjrrirrrr_rar)rm_FAILURE_SURFACE_WINDOW_SjoinrtrJrurr]rrrqrrziprrrlower_FAILED_TASK_STATESrr)r`rr^rzrcrg selectorsr owner_sqlr>rfailedr_rRrarrrrrrdgoal_fortasksrrindexrs&&$$ r!failed_delegations_for_sessionrss !569LN_8`-bj-beh#-bIj  _c$))+9R RF ?Y??I <>>T||!{#3 6 <++ ;V ;U ; =>FXZ " $&FUYQ m9k#I.0CLQ\D]0^f%/0A4%H%HTRY]]!aeaiaijpaqawawuw`xDHH^4IIc%j8I5QR%/ 90Et%L%Ly!X\X`X`akXlXlRTsyrzU))+/BB"Aw 70CDEEE8,;;;;<BBDF00/9%))L:QSV/W/WEIIl+]^EIIg&==&**W*=E MM eXvHLLa9??R@'Y^3u:dh N O VZ" &5>=j ">>s. MM=M 3M M  M  M  M1 c V^8dQhRRRR/#)r$ max_workersrr%rr)r's"r!r(r(s  s '9 r c\;_uu_4\f\VRR7sVsMV\8dV\nVs\uuRRR4# +'giR#;i)zLazily create (or grow in place, never shrink) the shared daemon executor. Raising ``_max_workers`` is enough: the next ``submit`` spawns threads up to the new cap.Nzasync-delegate)ruthread_name_prefix)_executor_lockrr r _max_workers)rus&r! _get_executorrzsH   0[]mnI$/ ! 0 0%0I "$/ ! s 6A A! cV^8dQhRR/#rr)r's"r!r(r(sTTcTr c\;_uu_4\R\P444uuRRR4# +'giR#;i)zfNumber of live async delegation UNITS (one per completion message: a task group or an ungrouped task).c3\"TF"qPR4\9gK^xK$ R#5irNrq _LIVE_STATESrs& r!ractive_count..s!S/55?l3R11/, ,N _records_lockrrvaluesrr r! active_countrs( Shoo/SS  $> A cV^8dQhRR/#rr)r's"r!r(r(sXX3Xr c\;_uu_4\R\P444uuRRR4# +'giR#;i)uNumber of running child subagents (a batch of N contributes N; a batch with no goal list counts 1) — the truthful observability figure, unlike slots.c3L"TFqPR4R9gKVPR4'df\VPR4\\34'd;VR,'d,\ VPR4;'g VR,4M^xK R#5i)rrirdrjN>running finalizing)rqrrr rrs& r!r$active_task_count..sX'%%/=V*V euuZ  Zgu %N%NSTU\S]S] n%337 4cd e&sB$B$9B$/B$ B$Nrrr r!active_task_countrs6 X__&XX rc(V^8dQhRRRRRRRR/#)r$rrr`rr%rr)r's"r!r(r(s0 [ [C [s [_b [gk [r c aRV3RV3RV33UUu.uFwrEV'gKWE3NK pppV'g.#\;_uu_4\P4Uau.uF_oSPR4V9gK\;QJdV3RlV4F 'gK RM RM!V3RlV44'gK]SNKa upuuRRR4#uuppiuupi +'giR#;i) u>Records in ``statuses`` owned by a session: any non-empty selector claims the record — ``origin_ui_session_id`` (TUI tab), ``session_key`` (routing key at dispatch), or ``parent_session_id`` (spawner's durable id — the right one for gateway chats, whose session_key survives ``/new`` while the session id rotates).r`rrrc3v<"TF.wr\SPV4;'gR4V8HxK0 R#5i)rRN)rrq)rfieldwantedrs& r!r#_session_records..s-YymeAEE%L..B/69ys#99TFN)rrrrqany)statusesrr`rrrrmrs&&&& `r!_session_recordsrs !56 8T /072=72]U5;!%72I=  #??,[,ah80KCYyYCCCYyYY,[  = [ s@ C C CC8 CCC:CCC C) c(V^8dQhRRRRRRRR/#)r$rrr`rr%rr)r's"r!r(r(s0ffcfcfcffptfr c6\\\WV44#)zOWhether a session still owns any live (running/stalling/finalizing) delegation.)rrr)rr`rs&&&r!has_live_for_sessionrs  {Rcd eer cV^8dQhRR/#r$r%rr)r's"r!r(r(s++C+r cTR\P!4PR, 2#)deleg_:NN)r#r$r%rr r!_new_delegation_idrs! DJJL$$R() **r cV^8dQhRR/#rr)r's"r!r(r(s   r cD\P4UUu.uF$wrVPR4\9gK!W3NK& pppVP RR7VR\ ^\ V4\, 4Fwr\PVR4K R#uuppi)zDrop the oldest completed records beyond the cap. Caller holds ``_records_lock``. ``stalling``/``finalizing`` are still live: evicting one makes the late runner return hit ``_finalize``'s missing-record path and silently drop a real result.rcV^,PR4;'g#V^,PR4;'g^#)rrar)kvs&r!)_prune_completed_locked..s2"Q%))N";"^"^r!uyy?Y"^"^]^"^r )r|N) ritemsrqrsortmaxrrpop)ridrrrds r!_prune_completed_lockedrs)1(8`(8fcAEE(OS_<_#(8I` NN^N_Nc!S^6M%MNO S$Pas BBcV^8dQhRR/#rr)r's"r!r(r(s  C r c^RIHpV!RR4R8HpV'dV!RR4;'gR#R# \dR#i;i)aRaw session id of the ORIGINATING api_server request, or ``""``. ``HERMES_SESSION_ID`` is unsafe here: building the child agent calls ``set_current_session_id(child.session_id)`` just before dispatch, so the wake would self-post into the subagent's own session. The request-scoped ``HERMES_SESSION_CHAT_ID`` (raw X-Hermes-Session-Id on api_server) survives child construction; on push platforms chat_id is a chat, not a session => ``""``.rOHERMES_SESSION_PLATFORMrR api_serverHERMES_SESSION_CHAT_ID)rSrPrV)rPis_apis r!_current_origin_session_idrsL ; !:B?<OHN 8"=CCVTVV s1111 AAc$V^8dQhRRRRRR/#r$rrdurationrr%rMr)r's"r!r(r(s&nnnn.nr c RRRRRVR^RV/#)rrrN api_callsduration_secondsrrrs&&r! _single_crashrs gy$ QPbdl mmr c$V^8dQhRRRRRR/#rr)r's"r!r(r(s&OOOuOOr cR.RVRV/#)rrtotal_duration_secondsrrs&&r! _batch_crashrs r7E+CX NNr c V^8dQhRRRR/#)r$combinedrMr%rr)r's"r!r(r(sooNosor caVPR4;'g.pRoV'dC\;QJdV3RlV4F 'dK RM RM!V3RlV44'dR#R#)zCBatch status: completed unless every child errored/was interrupted.rrc3J<"TFqPR4S9xK R#5ir~r)rroks& r!r _batch_status..s+]}!EE(O2,E}s #FTr)rsuccess)rqall)r child_resultsrs& @r! _batch_statusrsLLL+11rM !B#+]}+]+]}+](](]7ncnnr cV^8dQhRR/#rLr)r's"r!r(r("s,,>,r c ^RIHpVP4;_uu_4pV'gRRRR/uuRRR4#\R/VBuuRRR4# +'giR#;i)r* retirementrrejectedrz*backend is retiring; reconnect to continueNr)hermes_cli.backend_retirementrwork_dispatch_admitted)kwargsradmitteds, r! _dispatchr"sH8   hj'3_`  "+F+    sA  A  A slot_keyrjrkc(hV^8dQhRRRRRRRRRRR RR RR RR RR RRRRRRRRRRRRRRRRRRRRR/#)r$r_rrcrdOptional[list[str]]rerrfrgrhrrrunnerCallable[[], dict[str, Any]]r`rb interrupt_fnOptional[Callable[[], None]]max_async_childrenr progress_fnOptional[Callable[[], tuple]]capacity_errorrrjOptional[list[int]]rkOptional[dict[str, str]]r%rMr)r's"r!r(r(+sXDXDXD!$XD-@XDKXXD!XD),XD5BXDQTXD%XD/KXDcfXD XD+G XD]` XD / XD AD XD P] XD & XD/XDXDr caa aaaa a!a"\4VRJpV'dRMRo V'd\MRoV'd\M\o\P!4oRSRV/V'dR\ V4/M/CRTR V'd \ V4MRR VR VR VR V RV RV/C\ 4CRRRSRRRV /CV'dRR/M/CRTRT;'gS/CV'dR\V4/M/CVeR\ V4/M/CR\P!4RRRSRR/Co!\;_uu_4\;QJd#V!3Rl\4F 'gK RM R M!V!3Rl\44'dRR!R"R#/uuRRR4#\P4Uu0uFApVPR4\ 9gKVPR4;'g VR,kKC ppS!R,V9d!\#V4V 8dRR!R"V/uuRRR4#S!\S&\%R$\P444pRRR4\'S!4\)\+V X44pR%VVVVV V 3R&llp^R'IHo"S"P14VP3\5V44pVP7V"3R(l4Te \G4RR,RS/#uupi +'giL;i \8dpS"P;4\;_uu_4\P=SR4RRR4M +'giM;i\4;p'dTP?S4Mm\@;_uu_4\C4;_uu_4pTPER)S34RRR4M +'giM;iRRR4M +'giM;iRR!R"R*S R+T 2/uRp?#Rp?ii;i)-aShared dispatch core for single (``goals is None``) and batch units. Capacity check + record insert happen under ONE lock hold so concurrent dispatches can't both pass the check and exceed the cap. At capacity the dispatch is REJECTED (never queued) so a runaway model can't pile up unbounded background work. ``slot_key`` names the pool slot the unit occupies (default: its own id); the units of one delegate_task call share the first unit's id so splitting a call into per-group completions never consumes more capacity than the call did.N batchrRc8VPR4;'gR#)rrr)rs&r!r$_dispatch_admitted..=sx9W9WK9Wr r_rcrdrerfrgrhrr`rbrrrrarrriTrrrkrj_context_progress_token _progress_ts_interrupted_atc3F<"TFqSV,3\9xK R#5ir)r)rrr[s& r!r%_dispatch_admitted..RsQ.ve}%4.s!Frrz?This session was closed (exit or /new): no new background work.c3\"TF"qPR4\9gK^xK$ R#5ir~rrs& r!rrXs!Y$5qxL9X$5rcV^8dQhRR/#rr)r's"r!r((_dispatch_admitted..__annotate__^s55T5r c></pRp\;_uu_4\PS4pVe'VPR\P!4R7RRR4S !4;'g/pS!V4p\SW4R# +'giL8;i \ djp\ PRS R2S4S!\T4P RT 2\\P!4S, ^44pRp?LRp?ii;i \SY4i;i)rNT)_startedrAsync delegationz %s crashed: ) rrrqupdatermrVr exceptionr__name__round _finalize) rrrecexcclassify crash_resultr_ralabelrs r!_worker#_dispatch_admitted.._worker^s!# ],,}-C Dtyy{ C   5X^^Ff%F mV 4] i   /wkBM R!T#Y%7%7$83%"@% VcHcefBghF i mV 4s7ABB B  D "ADDD  DDrc$<SP4#r)r))rdrs&r!rrvs:+=+=+?r z3DELETE FROM async_delegations WHERE delegation_id=?z#Failed to schedule async delegationr dispatched)$r rrrrmrrYr contextvars copy_contextrr_FENCED_FIELDSrrrq_ACTIVE_STATESrrr~rzrrracquiresubmitr add_done_callbackrVr)r unscheduledrtrJru_ensure_stale_monitor)#r_rcrdrerfrgrhrrrr`rbrrrrrrjrkrir active_slots live_unitsexecutorrfuturerryr>rrrarr[rs#d$$$$$$$$d$$$$$$$$$ @@@@@@r!rr+sOD H HbE (}/WH#+<LIIKMYS['4;9OacY7J(XfVZ\cej{$:B  #)YHOO$5YY  fS!3Z@AH55"9 e!>T RUbTde*>>>*g1TUZT[[]^a]b/cdd es&$K !K/K?KK 6K K %K -K/K! K K ! O$,O L*! O* L: 5:O/N;N  N; N/*N;2 O; O OO$O$r`rbrrrcPV^8dQhRRRRRRRRRRR RR RR R R RRRRRRRRRRR/#)r$rcrrerrfrrgrhrrrrr`rbrrrrrrr%rMr)r's"r!r(r(s (4GOR[h)6Gc8;Pl If   r c \4p \R/RV bRVbRRbRVbRVbRVbRVbR VbR VbR VbR VbR V bRV bRV bRV bRRV R2bpVR,R8Xd3\PRY;'gRT;'gRR,4V#)aSpawn ``runner`` on the daemon executor and return a handle immediately. ``session_key``/``parent_session_id`` are captured on the parent thread (the worker carries no contextvars) and route the completion back to the spawning session. ``progress_fn() -> (token, in_tool)`` enables stale monitoring; omitted = unmonitored. Returns ``{"status": "dispatched", "delegation_id"}`` or ``{"status": "rejected", "error"}``.r_rcrdNrerfrgrhrrrr`rbrrrr#Async delegation capacity reached (z running). Wait for one to finish (its result will re-enter the chat), or run this task synchronously (background=false). Raise delegation.max_concurrent_children in config.yaml to allow more concurrent background subagents.rrz3Dispatched async delegation %s (session_key=%s): %srR:NPNr)rrrinfo)rcrerfrgrhrrrr`rbrrrr_handles$$$$$$$$$$$$$ r!dispatch_async_delegationrs '(M  w# w*. w6: wDK w w $ w,1 w?J w, w5; w2 wFW w " w 7I w Wb w22D1EFu u wFh<' I!#9#9'DJJB;L N Mr r_c$`V^8dQhRRRRRRRRR RR RR RR R RRRRRRRRRRRRRRRRRRRR/#)r$rdz list[str]rerrfrrgrrhrrrrr`rbrrrrr_rrrrjrrkrr%rMr)r's"r!r(r(s!!!"/!;N!VY!bo!!)6!Gc!!8;!Pl! !KX ! / ! BO ! & !/!!r cT ;'g \4p VeVUu.uF pVV,NK upM \V4p\V4pV^8Xd V^,M!V R2RPRV44,p\ R/RV bRVbRVbRVbRVbR VbR VbR VbR VbR VbRVbRV bRV bRV bRV bRVbRVbRVbRRV R2bpVR,R8Xd#\ P RT TT;'gR4V#uupi)uDispatch a fan-out unit (a whole batch, or one ``group`` of a delegate_task call) as ONE background unit: ``runner`` runs its tasks and returns the combined ``{"results": [...], "total_duration_seconds": N}`` dict. The unit occupies ONE async slot — or joins the slot named by ``slot_key`` (in-unit parallelism is bounded separately) — and produces a SINGLE completion event carrying per-task ``results``.z parallel subagents: z; c32"TF qR,xK R#5i):N(NNr)rgs& r!r2dispatch_async_delegation_batch..sXtisde[^Y_Y_issr_rcrdrerfrgrhrrrr`rbrrrrrjrkrrz running). Wait for one to finish (its result will re-enter the chat), or raise delegation.max_concurrent_children in config.yaml to allow more concurrent background units.rrzADispatched async delegation batch %s (%d task(s), session_key=%s)rr)rrrrirrr)rdrerfrgrhrrrr`rbrrr_rrrjrkr unit_goalsn combined_goalrs$$$$$$$$$$$$$$$$$ r!dispatch_async_delegation_batchr s"99%7%9M5A5ML1Lq%((L1SWX]S^J JA%&!VJqMA36K1LtyyXtisXtOt1tM  G# G*7 G?D GNU G G $ G,1 G?J G, G5; G2 GFW G " G 7I G Wb G mu G " G 5E G22D1EFE E GFh<' W!1k&<& M#2sC2c(V^8dQhRRRRRRRR/#)r$r_rrrrr%rAr)r's"r!r(r(s(""S"#"s"t"r c&\;_uu_4\PV4pVeVPR4\9d RRR4R#RVR&\P!4VR&RVR&RVR&\ V4pRRR4\ X\V4'd V!V4MTV4\;_uu_4V\9dV\V,R&\4RRR4R# +'giLt;i +'giR#;i)aAtomically claim terminal delivery, push the completion event, then mark ``status``. ``result`` is a dict or a callable receiving the record snapshot (stall path). The record stays active ("finalizing") until durable persistence and queue publication finish; otherwise process shutdown can kill this daemon worker after status flips but before SQLite commits. A second call for the same id (late runner return after a forced stall) is a no-op.Nrrrrr) rrrqrrmr_push_completion_eventcallabler)r_rrr[snapshots&&& r!rrs m, >VZZ1G  (x!%~!%~ $}< 8&9I9IVH%5vW]^  H $06H] #H -!   s5C,2C,<&C?, C< ? D c(V^8dQhRRRRRRRR/#)r$r[rMrrrr%rAr)r's"r!r(r(s04J4J>4J>4JSV4J[_4Jr c\VRR7p\VPR44pV'dRMRp^RIHpTPR 4;'g\P!4pTPR 4;'g\P!4pT'dtRR RTPR4;'g.RTPR4RTPR4RTPR4/TPR4e RTR,/M/Cp MXRTPR4RTPR4RTPR^4RTPR\Y, ^44/p RRR TPR 4RTPRR4RTPRR4RTPRR4RTPR4RTPRR4/T'dRTPR4/M/CRTPR4RTPR4R TPR 4R!T'dTPR!4M)TPR!4;'gTPR!4R"T/CT CR TR T/CT'd/MR#TPR#4/C\U u/uF$qPT 4'gKYT ,bK& up C\U u/uFqT9gK YT ,bK up Cp \Y4TPPT 4R # \ d6p\ PRT R2TPR 4T4R p?R #R p?ii;iuup iuup i \ d5p\ PRT R$2TPR 4T4R p?LR p?ii;i \ d6p\ PRT R%2TPR 4T4R p?R #R p?ii;i)&aPush a type='async_delegation' event onto the shared completion queue. Batch records (``is_batch``) carry the per-task ``results`` list (plus live transcript paths, the full-fidelity record of each child's run) instead of a single summary. Best-effort: failure must not crash the worker, but it WOULD mean a silently-lost result, so we log loudly. The queued event is the parent's in-process copy and honours ``security.redact_secrets: false``; ``_persist_completion`` re-scrubs with force for the durable row (and the worker IPC).FforcerirrRprocess_registryrz@ %s finished but process_registry import failed; result lost: %sr_NrarTrlive_transcriptsrrgrouprrrrrrr`rbrrcrdrerfrgrhr exit_reasonzo %s: durable completion write failed; delivering in-memory only (a restart may report this unit as unknown): %sz8 %s: failed to enqueue completion event; result lost: %s)rrrqtools.process_registryr+rVrrrmrrT_STALL_META_KEYSrcompletion_queuer) r[rrrirr+rrarrrWrs &&& r!r#r#s!u 5FFJJz*+H HbE; JJ/>>499;M::n-<<L iI)>)D)D"  += >T[I\ $fjj1I&JW.4ZZ-@-L)RT W vzz),gvzz'7JKY_YcYcdoqrYs  +=u\Eacd?e fh "OVZZ5Pvzz-4 +A2 FVZZ(;R@VZZ(;< 62& CNVGVZZ5H+I[] C 6::i(*fjj6LfV\V`V`agVhG$vzz'7J7a7afjjY`Na& C$ C&5m^Ua C2 vzz-/HI C"/ @A**Q-<1Qi< @ C"2 A!1A&[<1QiVZZ1G  <  ; F" F$94 FAKT FS\^e F F(1E7 F x||M26 F  -CR H F X\\*=rB F X\\*=> F  VR( F+28<<3H FKTU]UaUabkUl F HLL, F/5hll66J FMTU]UaUabiUj F ) F-hll?.K.Z.Ztyy{\jlplululw F$1 D=aLLO>1qk>= D FCq))--c2-   kmz A E q []jppqsM5G G/G1H . H H% G. 1 H<HH% I0I  IcV^8dQhRR/#rr)r's"r!r(r(:s   t  r c@\;_uu_4\e&\P4'd RRR4R#\P 4\ P !\RRR7s\P4RRR4R# +'giR#;i)zStart (once) the stale-delegation monitor thread. One daemon thread serves every dispatch; it exits when no monitorable records remain and is restarted by the next dispatch with a ``progress_fn``.Nzasync-delegate-stale-monitorT)targetnamedaemon) _monitor_lockris_alive _monitor_stopclear threadingThread_stale_monitor_loopstartrr r!r r :sg   &?+C+C+E+E   #**&-KTXZ s"B B AB  B cV^8dQhRR/#)r$rzrr)r's"r!r(r(Hs#-#-U#-r c *..Rr2p\P4EFOpVPR4pVR8XdGRpYPR4;'gT, \8dVP VR,4KbVPR4pVR8wgVfKRpVPR 4'gKV!4wrxWtPR 48wdVP WpR 7KYPR 4;'gT, p V'd\M\p W8gEK VP RV\V ^4V \V4R 7VP VR,W34EKR WV3# \ dTPR 4RrLi;i)aOne monitor pass over ``_records``; caller holds ``_records_lock``. Returns ``(stalled, expired, any_monitorable)``: newly-stalling ``(delegation_id, quiet_for, in_tool)`` tuples, stalling ids past the grace window, and whether anything is left to monitor.FrstallingTrr_rrrr)rrr)rr_stall_quiet_seconds_stall_threshold_seconds_stall_in_tool) rrrq_STALL_GRACE_SECONDSrrVr_STALE_IN_TOOL_SECONDS_STALE_IDLE_SECONDSrr) rzstalledexpiredany_monitorabler[rrtokenin_tool quiet_forr^s & r!_sweep_stale_lockedrRHst)+BoG//#H% Z "Ojj!23::s;?SSvo67 jj/ Y +"5 zz*%%  B(]NE JJ01 1 MM%M B ::n5<<= *1&7J   MM!3US\^_M`).tG}  N NNF?3YH I;$< _ ,, B$ZZ(9:E7 Bs6 E22FFc V^8dQhRRRR/#)r$msgrr%rr)r's"r!r(r(ns  S D r c\V4'gR#V!4R# \d&p\P!T.TOTN5!Rp?R#Rp?ii;i)zLInvoke an ``interrupt_fn``; True on success, else debug-log ``msg`` (+ exc).FTN)r$rVrr)rNrTargsrs&&* r!_call_interruptrWnsD B<<   S%4%%s AA  AcV^8dQhRR/#rr)r's"r!r(r(zsTr cH\P\4'EgH\P!4p\;_uu_4\ V4wrpRRR4XFtwrEp\ PRWEV\4\;_uu_4\PV4;'g/PR4pRRR4\XRV4Kv XFp\;_uu_4\PV4;'g/PR4;'g\P!4pRRR4XP\WD3RlR4K X'dEKeR#R# +'giEL ;i +'giL;i +'giLe;i)aGSweep running delegations for stalled progress. A changed progress token refreshes the record's timestamp; a frozen token past the idle/in-tool threshold marks the record ``stalling`` and calls ``interrupt_fn``; a ``stalling`` record still unreturned after the grace window is force-finalized with a terminal ``stalled`` event.Nu`Async delegation %s made no progress for %.0fs (in_tool=%s) — interrupting; grace window %.0fsrz.Async delegation %s stall interrupt failed: %src\W4#r)_stalled_result)rds&&r!r%_stale_monitor_loop..s ?[\Kbr rL)r=wait_STALE_CHECK_INTERVALrmrrRrrrIrrqrWrrrunr) rzrLrMrNr_rQrPrNctxs r!rArAzs"   !677iik ]0CC0H -Go18 -Mg NNO(W>R Tll=177R<<^L B PR_ ` 29%M||M288b==jIgg[MeMeMg GGI}.bdm n% !8 ]] s*E* /E>(5FF* E; > F  F! c V^8dQhRRRR/#)r$ event_recordrMr%rr)r's"r!r(r(s < SUGg( 2; ;,u/>:>> jmE, jh =R[ j_i jjr c$V^8dQhRRRRRR/#)r$rOrrzrr%zOptional[list]r)r's"r!r(r(s!5^r c \V4p.pTFp\T\\34'd\ T4^8gTP R4KBRT^,RT^,/p\ T4^8dT\T^,\ \34'd1\\RT\T^,4, 4^4TR&TP T4K T# \dR#i;i)zParse a progress token into per-child activity dicts (best-effort): delegate_tool emits one ``(api_call_count, current_tool, last_activity_ts)`` tuple per child; foreign token shapes degrade to ``None`` entries.Nr current_toolgseconds_since_activity) r TypeErrorrr rrrrrr)rOrzpartsoutpartrs&& r!_children_activity_from_tokenr}sU +-C4$//CIN JJt  !,d1g~tAw O t9>ja3,??.3CS5a>=Q4RTU.VE* + 5 J s C!! C0/C0cV^8dQhRR/#)r$r%r`r)r's"r!r(r(s&& 4&r c \P!4p/p\;_uu_4.p\P4EFpVP 4UUu/uF)wrEVR 9gKVP R4'dK'WEbK+ pppVP R4pV\9dfVP R4'd\WR,, ^4VR&\VP R44'dVR,WR,&VR 9d8\F-wrVP V4fKVP V4Wi&K/ VPV4EK RRR4XFTpVP VP R44p V fK)V !4wr\Y4p T eYR&\T 4TR &KV V#uuppi +'giLr;i \dKi;i) a9Snapshot of async delegations (running + recently completed) without callables or private monitor bookkeeping; adds computed live fields for UIs (``seconds_since_progress``, ``children_activity``/``in_tool`` sampled from ``progress_fn``) and stall context once tripped. Safe to call from any thread. See #51690. rrdrrseconds_since_progressr_Nchildren_activityrP>rr)rErL)rmrrrr startswithrqrrr$_STALL_FIELD_MAPrrVr}r)rzsamplersrrrWrXitemrsrcdstrNrOrPactivitys r!list_async_delegationsrs ))+C$&H "A%&WWYuYTQ!;Z2ZDcdcocopsctDADYDuUU8_F'55((5:3>AR;RTU5VD12AEE-01134]3CH/000 0HCuuSz-$%EE#J !1 LL # " \\$((?3 4 :  TNE1<  (0$ %w-Y L5v .   sH.G F? $F? =F? 5G9A0G.-G G?G G  G'&G'c ,V^8dQhRRRRRRRRRR/#) r$targetsr`callerrreasonrTr%rr)r's"r!r(r(s2 4c3UX]`r cna\V3RlV44pV'd\PW4V4V#)zSCall ``interrupt_fn`` on each record; log ``msg`` once; returns how many succeeded.c 3|<"TF1p\VPR4RSVPR44xK3 R#5i)rz%s: %s interrupt failed: %sr_N)rWrq)rrrs& r!r%_interrupt_records..s=A n-/LfVWV[V[\kVlmms9<)rrr)rrrrTcounts&f&& r!_interrupt_recordsrs2  E  C' Lr c V^8dQhRRRR/#)r$rrr%rr)r's"r!r(r(skk#kskr c\;_uu_4\P4Uu.uF qPR4\9gKVNK" ppRRR4\ XRVR4#uupi +'giL#;i)zSignal every running async delegation to stop (``/stop``, shutdown). Returns how many. The child still emits a completion event (status='interrupted') via the normal finalize path.rN interrupt_allz'Interrupted %d async delegation(s) (%s))rrrrqrr)rrrs& r!rrsZ &oo/U/55?n3T11/U  g@i jjV s"A)A$A$ A)$A)) A9 c ,V^8dQhRRRRRRRRRR/#)r$rrr`rrr%rr)r's"r!r(r(s;ppp14pNQp`cppr c@\\WV4p\VRVR4#)zSignal running async delegations owned by ONE ending session to stop (any selector matches, see ``_session_records``). Returns how many.interrupt_for_sessionz:Interrupted %d async delegation(s) for ending session (%s))rrr)rr`rrrs&&&& r!rrs- ~{RcdG (&2n ppr parent_session_idsc$V^8dQhRRRRRR/#)r$rOrrrr%rAr)r's"r!r(r( s" ? ?c ?3 ?W[ ?r c`V'dRV30M \4pVPRV44\;_uu_4\V4\ V4\ V&VF0p\ PV\44PV4K2 RRR4R# +'giR#;i)zRefuse new dispatches for ONE closing session (any selector matches) until ``allow_for_session``. Set before ``interrupt_for_session``: a dispatch registered first is interrupted, a later one refused.rc3<"TFq'gK RV3xK R#5i)rNr)rr;s& r!r%refuse_for_session..sZ=O6SY.%v.=Os  N) rrr_unfence frozensetrr setdefaultr)rOrrfenceselectors&$$ r!refuse_for_sessionr sw/:m[ ) *suE LLZ=OZZ #E*H  " "8SU 3 7 7 > s AB B- c V^8dQhRRRR/#r$rOrr%rAr)r's"r!r(r(sSTr cr\;_uu_4\V4RRR4R# +'giR#;i)zYLift ONE session's refusal: a viewer attached again, or the owner retired its live state.N)rr)rOs&r!allow_for_sessionrs  s % 6 c V^8dQhRRRR/#rr)r's"r!r(r(s&&C&D&r c\PVR4F2p\V,pVPV4V'dK+\VK4 R#)z7Drop *token*'s refusal; caller holds ``_records_lock``.Nr)rrrr;)rOrtokenss& r!rrs:LL+X&uvH% ,r cV^8dQhRR/#rr)r's"r!r(r('s##$#r c \;_uu_4\e\PRR7Rs^sRRR4\P 4\ ;_uu_4\RupsRRR4Xe)VP4'dVP^R7\;_uu_4\P4\P4\P4RRR4\;_uu_4\ P4\"P4\$P4RRR4R# +'giEL;i +'giL;i +'giL;i +'giR#;i)z@Test-only: clear all state and tear down the executor + monitor.NF)r^)timeout)rxrshutdownrr=rr;rr<rirrr>rrrrrr)threads r!_reset_for_testsr's     E  * !   "14  foo// A    !  "     s/#E E(=E);=E< E  E& ) E9 < F )rri: gA>rrEr)rrE)scope_iduser_id user_name)rnrorprr))rFrn)rGro)rHrs>rrofailurerLrr interruptedg@)rRrR)rRrRrR)r)rRrRrR session_endr)__conditional_annotations____doc__ __future__rrrvloggingrosqlite3r?rmr#concurrent.futuresrtypingrrrrrr9r r tools.daemon_poolr tools.thread_contextr tools.async_delegation_workerr rtools.delegation_redactionrr getLoggerrrr__annotations__Lockrxrrrrrr_DEFAULT_MAX_ASYNC_CHILDRENrrrrrrrtrrrrrrrr_rKrJrIr;rEventr=rrrTr0rr"r?r<rJrYr~rxrrrrrrrrrrrrrr&r+r0r4r8r>rDrHrGrBrWrrlrhr]rsrzrrrrrrrrrrrrrr rr#r5r rRrWrArkr[r}rrrrrrrr)rs@r!rs h#   166=6<MS   8 $ +/ '.!s &( #("$ # " "5- + >>  ~~ #3')$) .2+2! 4 (5 sr *, ! # <B4 MI.y 1hL>#U#L*At*AZT"7 `!$ZT!20 N KC:A! Y: H" no'4(QS(lp((X T X [f +  nOo,XD `d XD )- XD26XDv9=!#>@os: im 8!9=!!#!>@!os!: ![_ ! 26 ! RV ! )- !26!J",4Jn"qL  #-L 0 <k((&Rkp ? ?PR ? &#r