+ jHjsO8a0t$Rt^RIHt^RIt^RIt^RIt^RIt^RIH t H t ^RI H t H t ]P!]4tRtRRltRR ltR R ltR R ltRRlt] !RR44t]P0!RRR7tR]R&RRRlltRRltRRRRRR/tRRltR R!ltR"R#lt R$R%lt!/]!!R&R&R'R(R)R*4C]!!R+R,R-R.R/R0R1R2R3R4R5R6R7R8R9R:R;RR?R@RARBRCRDRERF4C]!!RGRHRIRJRKRLRMRNRORPRQRRRSRTRURVRWRXRYRZR[R\R]R^R_4C]!!R`RaRbRcRdReRfRgRhRiRjRk4 C]!!RlRlRmRnRoRp4C]!!RqRqRrRsRtRuRvRw4C]!!R!C]!!RxRyRzR{R|R}R~RRR4 C]!!R!C]!!RRRRRRRRRRRRRRR4C]!!RRRRRRRRRRRR4 Ct"R]R&]#!0Rm4t$]PJ!R4t&]PJ!R4t']PJ!R4t(Rt)^t*RRlt+]#!^^04t,RRlt-RRlt.RRR/RRlllt/] !RR44t0RRlt1RRlt2/]3PiRR4C]3PiRR4C]3PiRR4CRRRRRRRRRRRR/Ct5RRlt6RRlt7RRlt8Rt9RRlt:RRlt;RRltRRRRRR/t?RRlt@RRltARRltBR#)aAgent-harness accuracy shared metrics: file-edit matching, loop guards, recovery after a tool error, terminal command outcomes and bad model replies. These tune the agent loop itself: which fuzzy-match strategies earn their keep, how often the loop guards fire, whether a model recovers after a failed tool call, which kinds of commands fail, and which models return empty / refused / truncated replies. Producers pass RAW values; every builder maps them onto closed enums. Nothing here reads command text, file paths, tool arguments or reply text beyond the structural checks Hermes already makes (think-block stripping for "visible text"). Hermes-owned agent loops (the background self-improvement review and the curator fork) are not user work and never count. Recording is a no-op unless shared metrics are enabled for the owning profile, and never raises into the caller. Stdlib-only at import time: tool modules import this on their hot path. ) annotationsN) dataclassfield)AnyCallablebackground_reviewcV^8dQhRR/#returnbool)formats"T/home/ubuntu/.hermes/hermes-agent/hermes_cli/observability/shared_metrics_harness.py __annotate__rs$cB^RIHpV!4# \dR#i;i)enabledF)relay_shared_metricsr Exceptionrs r_enabledrs$1y   c$V^8dQhRRRRRR/#)r mark_attrstrdatadict[str, str]r Noner )rs"rrr's&QQsQ.QTQrc^RIHp^RIHpV!\ W 4V4R# \ d\ PRTRR7R#i;i)r)shared_metrics_contract)record_process_markzShared-metrics %s not recordedTexc_infoN)r!rr"getattrrloggerdebug)rrcontractr"s&& r_recordr*'sAQ9=GH8$? Q 5y4 PQs"#A A cV^8dQhRR/#r r )rs"rrr1strcB^RIHpV!4# \dR#i;i)is_background_reviewF)tools.skill_provenancer/rr.s r_background_tool_callr11s%?#%% rc V^8dQhRRRR/#r agentrr r r )rs"rrr:sddsdtdrcBVRJ;'d\VRR4\8g#)zIThe review/curator forks tag themselves with the background write origin.N_memory_write_origin)r&_BACKGROUND_ORIGINr4s&r _user_agentr9:s%   c c0F!MQc!ccrc V^8dQhRRRR/#)r r4rr rr )rs"rrr?sXX#X.XrcL^RIHpV!\VRR4\VRR44#)r) model_routeproviderNmodel)shared_metrics_modelr<r&)r4r<s& r_router@?s%1 wuj$7QU9V WWrcT]tRt^Gt$Rt]!]R7tR]R&]!]R7t R]R&Rt R#) _EditProbezMWhat the fuzzy matcher saw during one edit tool call (several hunks for V4A).default_factory list[str] strategiesmissesr N) __name__ __module__ __qualname____firstlineno____doc__rlistrF__annotations__rG__static_attributes__r rrrBrBGs#W!$7J 7d3FI3rrBshared_metrics_edit_probe)defaultz)contextvars.ContextVar[_EditProbe | None] _EDIT_PROBEc$V^8dQhRRRRRR/#)r strategy str | Nonemissr rr )rs"rrrTs! " "j " "d "rc\P4pVfR#V'dVPPV4R#V'dVPPV4R#R#)zCalled by the fuzzy matcher: the strategy that landed a replacement, or why none did (``no_match`` / ``ambiguous``). A no-op outside a metered edit tool call.N)rRgetrFappendrG)rTrVprobes&& rnote_edit_matchr[TsI OO E } )  D! rcV^8dQhRR/#)r r zdict[str, int]r )rs"rrr`sIIIrcd^RIHp\V4UUUu/uF wpwr#W!bK uppp#uupppi)r-) STRATEGIES)tools.fuzzy_matchr^ enumerate)r^indexname_fns r_strategy_rankrd`s-,2;J2G H2G.E;DDK2G HH Hs+replacepatchv4a whole_filec ,V^8dQhRRRRRRRRRR /#) r toolrmoderresultrZrBr rr )rs"rrrjs0cc*Q_rcfa ^RIHpHp\V4pVP R4RJg'V'd9RV9d2VP R4'gVP R4RJdRMRpM*RVP 9dRpMR VP 9dR pMR pR pVR8Xd8VP 'd&\4o \VP V 3R lR 7p\P \T;'gR4P4P4R4p RW9dTMR RW9dT MRRVRV/#)r)FILE_EDIT_MODESFILE_EDIT_STRATEGIESsuccessTerror no_changealready_appliedapplied ambiguousno_matchfailednonec(<SPVR4#)r)rX)rbranks&r"file_edit_fields..{s$((4:Lr)keyr%rematch_strategyrkoutcomerj) r!rnro_parsedrXrGrFrdmax _EDIT_MODESrstriplower) rjrkrlrZrnrorrrT mode_valuer{s $$$$ @rfile_edit_fieldsrjsN 6?D xx d"t 0EdhhW^N_N_'+xx '<'D#)  $ u|| #H) 0 0 0u''-LMTZZR!6!6!8!>!>!@)LJh&F(Fj; 7  rc(V^8dQhRRRRRRRR/#)r rjrrkcallzCallable[[], str]r r )rs"rrrs)3c1Bsrc \4'd\4'dV!4#\4p\P V4pV!4p\P V4\ R\WWSR74V# \P T4i;i \d\PRRR7T#i;i)aRun one ``patch`` / ``write_file`` tool call and count how its edit landed; returns the call's result unchanged. The matcher reports into a context-local probe, so strategies from unrelated callers (skill patches, V4A validation of another call) never mix in.FILE_EDIT_MARK)rjrkrlrZz%Shared-metrics file edit not recordedTr#) rr1rBrRsetresetr*rrr'r()rjrkrrZtokenrls&&& rrecord_file_editrs ::.00v LE OOE "E!% M "2PV"de M %  M %B>B"B>=B>c V^8dQhRRRR/#)r rlrr zdict[str, Any]r )rs"rrrs22C2N2rc\V\4'dV#\V\\34'd\P !V4MRp\T\4'dT#/# \ \3d/u#i;iN) isinstancedictrbytesjsonloads TypeError ValueError)rlrs& rrrsm&$ %/e %E%Etzz&!4dD))41r1 z " s4A((A>=A>c$V^8dQhRRRRRR/#)r kindrnamesr rr )rs"rrrs!&&&c&n&rc,\PW4#r)rfromkeys)rrs&*r_kindsrs == %%rgitghhubzgit-lfstigpackage_managerpippip3uvuvxpoetrypipxcondamamba micromambanpmpnpmyarnaptzapt-getbrewdnfyumpacmanapkgembundlecomposernixsnapchocowingetportbuildmakecmakeninjacargogogradlegradlewmvnbazelmesongcczg++ccclangzclang++rustcjavactscdotnetmsbuildsconsswift xcodebuildzig test_runnerpytestzpy.testtoxnoxjestvitestmocharspecphpunitctest playwrightpythonpython3ipythonpypypypy3nodenpxdenobunzts-nodetsxnodejsshellbashshzshdashfishkshpwsh powershellcmdnetworkcurlwgetsshscprsyncpingncdignslookuphttptelnetsftpftp traceroute containerdockerpodmankubectlzdocker-composehelmnerdctlminikuberk9s apptainer singularityr_COMMAND_KINDSz\s*([^\s;|&()<>]+)z[A-Za-z_][A-Za-z0-9_]*=zpython\d+(\.\d+)?c V^8dQhRRRR/#)r commandrr rr )rs"rrrs##rc|\V\4'd VR\MRp^p\\^,4Fp\ P W4pVfR#VP4pVP^4PR4p\P V4'g1VP4\9gVPR4'dKVPR^4R,P4pV\9d\V,u#\ P#V4'dRu#Ru# R#) zA closed kind from the command's first program word (after env assignments and wrappers like ``sudo``). The word itself is never exported.Nr%otherz'"-/rrz)rr_COMMAND_HEAD_CHARSrange_MAX_PREFIX_TOKENS _TOKEN_REmatchendgroupr_ASSIGNMENT_REr_COMMAND_WRAPPERS startswithrsplitr _PYTHON_RE fullmatch)rtextposition_rwordrbs& r command_kindr(s-7w,D,D7'' ("DH %) */ =99;{{1~##E*    % %9J)Jdoo^aNbNb {{3"2&,,. > !!$' '%//55xB7B+ rc V^8dQhRRRR/#)r rlrr rUr )rs"rrrsKKSKZKrcJ\V\4'gR#VPR4RJdR#VPR4RJdR#VPR4p\V\4'd\V\4'dR#V^8XdR#V^8g V\ 9dR#R #) zHow one foreground command ended, from the backend's raw result dict. Hermes' own deadline and interrupt set flags on the result, so a command's own ``exit 124`` reads ``nonzero``.Nhermes_timed_outTtimeouthermes_interruptedkilled returncodeoknonzero)rrrXintr _KILLED_STATUSES)rlcodes& rterminal_outcomer5s fd # # zz$%- zz&'4/ ::l #D dC JtT$:$: qyax4+;#;8JJrc(V^8dQhRRRRRRRR/#)r rrbackendrrr rr )rs"rrrs(cCNrc^RIHp\T;'gR4P4P 4pRWC9dTMRR\ V4RV/#)r)TERMINAL_BACKENDSr%r7rr(r)r!r9rrrr()rr7rr9 backend_values$$$ rterminal_outcome_fieldsr;sL: 2&,,.446MM$F=G W-7 rrc ,V^8dQhRRRRRRRRRR/#) r rrr7rlrrUr rr )rs"rrrs8TTST3TTXbTnrTrc >\4'd\4'dR#^RIHpVP 4'dR#T;'g \ V4pVfR#\ R\WVR74R# \d\PRRR7R#i;i)aCount one foreground terminal command that reached an exit status (or Hermes' deadline). Commands handed to the background, refused by a guard, or whose backend failed never ran to an exit status and are not counted here (``hermes.execution_backend.count`` covers them).N) _UNMETEREDTERMINAL_OUTCOME_MARK)rr7rz,Shared-metrics terminal outcome not recordedTr#) rr1shared_metrics_loopr>rXr5r*r;rr'r()rr7rlrr>resolveds&&&$ rrecord_terminal_outcomerBs Tzz244 3 >>   66.v6   ')@ks)tu T Cd STs( A6A6A6 A6A66"BBcp]tRtRt$Rt]!]R7tR]R&]!]R7t R]R&]!] R7t R]R &R t R #) _TurnStatei$zUPer-turn harness bookkeeping; ``agent._harness_metrics_turn`` is reset at turn start.rCzlist[tuple[str, bool]]round_outcomesrEpending_errorszset[tuple[str, str]] guard_signalsr N) rHrIrJrKrLrrMrErNrFrrGrOr rrrDrD$s4_-24-HN*H %d ;NI;*/*DM'DrrDc V^8dQhRRRR/#)r r4rr rDr )rs"rrr-sszrcl\VRR4p\V\4'g\4pWnV#)_harness_metrics_turnN)r&rrDrJ)r4states& r _turn_staterL-s0 E2D 9E eZ ( ( &+# Lrc V^8dQhRRRR/#r3r )rs"rrr5s--3-4-rc<\V4;'d \4#r)r9rr8s&r_activerO5s u  , ,(*,r exact_failureidempotent_no_progresssame_tool_failureidentical_call_streak_haltidentical_call_streakidentical_cycle_haltidentical_cycleloop_web_search_capweb_search_caploop_subagent_cap subagent_capc(V^8dQhRRRRRRRR/#)r r4rsignalrdetectorr rr )rs"rrrEs.ZZZcZSZTZrc\V4pW3VP9dR#VPPW34\R/\ V4CRVRV/C4R#)NLOOP_GUARD_MARKr]r\)rLrGaddr*r@)r4r\r]rKs&&& r _record_guardraEsX  EU000 V./ X&-XXxQWXYrc(V^8dQhRRRRRRRR/#)r r4ractionr4r rr )rs"rrrMs. N NS N# NS NT NrcL\P\T;'gR44pRRRRRR/P\T;'gR44pVeVe\V4'gR#\ WV4R# \ d\ PRR R 7R#i;i) zA tool-loop guardrail verdict: ``warn`` (the call ran, the model was told it is repeating) is ``repeated_tool_call``; ``block``/``halt`` (the harness stopped it) is ``loop_detected``. Once per turn per detector and signal.r%warnrepeated_tool_callblock loop_detectedhaltNz&Shared-metrics loop guard not recordedTr#)_GUARD_DETECTORSrXrrOrarr'r()r4rcr4r]r\s&&& rrecord_guardrail_decisionrkMsN#''DJJB8.&Rabffgjkqkwkwuwgxy  v~WU^^ eX. N = MNs"A=)A= $A=/ A=="B#"B#c(V^8dQhRRRRRRRR/#)r r4rdecision stall_kindrUr rr )rs"rrr[s)==S=C=Z=TX=rc \VRR4R8Xd\VR\VRR44V'd\VRV4R#R#)zThe warnings one tool result carried: a guardrail ``warn`` verdict and/or an identical-call stall notice (``stall_kind`` names its detector).rcNrer4)r&rk)r4rmrns&&&rrecord_guardrail_warningsrp[s>x4(F2!%641PQ!%<rc V^8dQhRRRR/#)r rbrr rr )rs"rrris11S1S1rc"^RIHpV!RV/4#)r)tool_metric_name tool_name)r!rs)rbrss& r _tool_nameruis9 [$/ 00rc(V^8dQhRRRRRRRR/#)r r4rrtis_errorr r rr )rs"rrros.PPPPtPPrc\V4'dE\V4PP\ T;'gR4\ V434R#R# \ d\PRRR7R#i;i)zPEvery committed tool result of the current round, in the model's emission order.r%z(Shared-metrics tool outcome not observedTr#N) rOrLrErYrr rr'r()r4rtrws&&&robserve_tool_outcomeryosfP 5>>   - - 4 4c)//r6JDQYN5[ \  P ?$ OPsA?c 0V^8dQhRRRRRRRRRRR R /#) r r4rrouterrjr next_tool next_outcomer rr )rs"rrrxs8Cccadimrc D\R/VCRVRVR\V4/C4R#)TOOL_RECOVERY_MARKr}r|rjN)r*ru)r4r{rjr|r}s&&&&&r_record_recoveryrxs2 # #{IvzZ^O_#rc V^8dQhRRRR/#)r r4rr rr )rs"rrr~sQQSQTQrca\V4'gR#\V4pVP.uq!nV'gR#VP'di\ V4pVPFMo\ V3RlV4R4pVfV^,^,MTp\ YSVfRMRV'dRMR4KO VUUu.uFwreV'gKVNK uppVnR#uuppi \d\PRRR 7R#i;i) zResolve the previous round's failed calls against this round's first calls, then carry this round's failures forward. The next call to the same tool is the retry; with none, the model switched tools and its first call is what it tried instead.Nc3><"TFwrVS8XgKVxK R#5irr ).0rbrwrjs& r $finish_tool_round..sQ DDLVVs   differentsamerqrpz)Shared-metrics tool recovery not recordedTr#) rOrLrErFr@nextrrr'r()r4rKoutcomesr{rrwrbrjs& @rfinish_tool_roundr~sQu~~ E").)=)=r&&     5ME,,QQSWX+/<!QT $t| \bQXhq- :BLVLL Q @4 PQs@C%&C%C%AC%,C%= CC C%C%%"D  D c 0V^8dQhRRRRRRRRRRRR/#) r r4rturn_exit_reasonfinal_response interruptedrwr rr )rs"rrrsALLsLcL3LX[LehLmqLrc&\V4'gR#\V4p\T;'gR4pVP\4'd\ VRR4VP .uqunV'gR#VPR4;'d*\V4;'dV'*;'dV'*p\V4p VFp \Y T RV'dRMR4K R# \d\PR R R 7R#i;i) uTurn end: the iteration cap, and failed calls the model never followed with another call — ``no_tool_call`` when it answered in text, ``gave_up`` when the turn ended without a reply from it (halted, budget spent, interrupted, errored).Nr% iteration_capiteration_budgetztext_response(rx no_tool_callgave_upz$Shared-metrics turn end not recordedTr#) rOrLrr _BUDGET_EXIT_PREFIXESrarFr r@rrr'r() r4rrrrwrKreasonpendingansweredr{rjs &&&$$ r finish_turnrsLu~~ E"%++,   2 3 3 %2D E(-(<( envexecnicesudotimenohupstdbufbuiltinr)repeated_exact_failure_blockrepeated_exact_failure_warning)idempotent_no_progress_blockidempotent_no_progress_warning)same_tool_failure_haltsame_tool_failure_warning)budget_exhaustedzmax_iterations_reached()C__conditional_annotations__rL __future__r contextvarsrloggingre dataclassesrrtypingrr getLoggerrHr'r7rr*r1r9r@rB ContextVarrRrNr[rdrrrrrr frozensetrcompilerrr"rrr(r3r5r;rBrDrLrOrrrjrarkrprruryrrrrrrr)rs@rr"s #  (   8 $(Qd X 44 4:E9O9O: 6 "I)We\<P 6&2&'" UE4 59'" 5&$xRY[g vvuixQVX]_gE67Hf'" '7D(IuV]_f udGY%S\^eu'" xE5&(GU\^g'"  XxIvwG!'"" VVUFE9eXN#'"$ %'". WfdE665&,X]^/'"0 1'"> 665%&$z[ack|?'"F Xx4DfiYcek {MG'"'Rno JJ, - 67 ZZ, - ,c3Z(K"TeiTT* EE E-  mmVXgh mmVXpq mmKM`a!"9;RTk-/@BS+-@. Z N=H 1 P Q0L4#Ix9K\[mn PX Qr