+ jHjVfRt^RIHt^RIHt^RIHt^RIt^RIt^RI t ^RI t ^RI t ^RI t ^RI t ^RIt^RIt^RIt^RIt^RIHt^RIHt^RIHt^RIHtHt]P8!]4tR tR t Rt!Rt"R t#R R lt$RRlt%RRlt&RRlt'RRlt(]!^R7RRl4t)!RR4t*R#)uSupervision of one llama-server in router mode. The router process is ours (restart with backoff on crash); router children are its problem — child failures surface via GET /models exit_code, never auto-retried here. Learned on real hardware: health-200 is NOT readiness — every readiness claim requires a touch generation (temp-0, expected token, generous budget, reasoning_content scanned); always dial 127.0.0.1 — resolving localhost adds ~2s per request on Windows via IPv6 fallback. ) annotations)suppress) lru_cacheN)Path runtimes_root) child_query)server_child_env spawn_serverz3Reply with exactly one word: the capital of France.parisiHcV^8dQhRR/#)returnr)formats"H/home/ubuntu/.hermes/hermes-agent/hermes_cli/local_runtime/supervisor.py __annotate__r-s++D+c$\4R, #)zEndpoint state for other Hermes processes (provider resolution routes llamacpp-alias requests at the managed server from this).z server.jsonrrrr state_pathr-s ?] **rcV^8dQhRR/#r rNoner)rs"rrr3s  $ rc|\\4;_uu_4V!4RRR4R# +'giR#;i)zABest-effort call; a child that vanished mid-walk is not an error.N)r Exception)fns&r_quietr3s! )       s* ; cV^8dQhRR/#r rintr)rs"rrr9s""C"rc\P!4;_uu_4pVPR4VP4^,uuRRR4# +'giR#;i) 127.0.0.1N)r!)socketbind getsocknamess r _free_portr(9s6 A  }}q! s (A A" cV^8dQhRR/#rr)rs"rrr?s  c rc \P!4;_uu_4pVPR\34\uuRRR4# +'giR#;i \d(\P R\4\ 4u#i;i)zThe stable default port, or an ephemeral one only when something else already listens there (a leftover managed server would have been cleaned up by stop()).r!Nurport %d busy; managed llama-server falling back to an ephemeral port — resumed sessions follow the live endpoint)r#r$ _DEFAULT_PORTOSErrorloggerwarningr(r&s r _stable_portr/?sd ]]__ FFK/ 0 ___  ABO Q| s-AA A A AA/B  B cV^8dQhRR/#r rstrr)rs"rrrMsrc\4R, p\\4;_uu_4VPRR7P 4p\ V4^8d VuuRRR4#RRR4\ P!^4pVPPRRR7VPVRR7V# +'giLY;i \d"p\PRT4Rp?T#Rp?ii;i) uOne key for the life of the install, persisted beside the runtimes. Endpoint identity must survive restarts as a UNIT — sessions persist base_url + api_key, so a per-boot key strands every resumed session on HTTP 401 exactly as a per-boot port would on connection errors. z.api_keyz utf-8-sig)encodingNTparentsexist_okutf-8zJcould not persist api key (%s); sessions will need a re-pick after restart) rrr, read_textstriplensecrets token_urlsafeparentmkdir write_textr-r.)key_pathexistingkeyexcs r_stable_api_keyrEMs+H '  %%{%;AAC x=B         #C7dT:C'2 J   7125 7 7 J7s#2B80C 8 C C7C22C7)maxsizec V^8dQhRRRR/#)r executablerrztuple[str, ...]r)rs"rrrds;;;;rc \P!\V4R.RRRRR^\VP4R7pVPVP ,pRV9dR#RV9dR #R #) zRSelect the loading option supported by this engine, including older pinned builds.z--helpTr8replace)capture_outputtextr4errorschecktimeoutcwd --load-modez --direct-io)rQdio)z-dior) subprocessrunr2r>stdoutstderr)rHresult help_texts& r_direct_io_argsrYcsn^^S_h7!% QU$&C 0A0A,BDF  -I !%%%29::rc ]tRt^otRtR9tR^RRRRRRRR/RR llt]R R l4tR R lt R:RR/RRlllt R:RRllt RRlt R;RRllt RRltRRltRRltRRlt]R R!/R"R#ll4tR$R%ltRR.R/lltR?R0R1lltR=R2R3lltR>R4R5lltR>R6R7lltR8tR#)@LlamaServerSupervisorzEOwn one llama-server router process for the life of a Hermes session. models_maxportN extra_argslog_path preset_pathc4V^8dQhRRRRRRRRRR R R R R /#) r binaryr models_dirr\rr]z int | Noner^zlist[str] | Noner_z Path | Noner`r)rs"rr"LlamaServerSupervisor.__annotate__xsF..t.. .,6.-.'.* .rc <\V4Vn\V4VnW0nT;'g \ 4Vn\ 4Vn\T;'g.4Vn T;'g%VPPR, R, Vn Wpn RVn RVnRVn^VnRVn\$P&!4Vn\$P*!4VnRVnRVnRVn/VnRVnR#)logszllama-server.logNF)rrbrcr\r/r]rEapi_keylistr^r>r_r`proc_job primary_model _restarts _stopping threadingEvent _stop_eventRLock_lifecycle_lock_state _watchdog _log_handle _idle_since _refit_usable)selfrbrcr\r]r^r_r`s&&&$$$$$r__init__LlamaServerSupervisor.__init__xs6l z*$**LN &( z//R0 ZZT__%;%;f%DGY%Y &-1  )-$??,(0#' 26-/)-rcV^8dQhRR/#r1r)rs"rrrds22#2rc "RVP R2#)http://127.0.0.1:z/v1r]rxs&rbase_urlLlamaServerSupervisor.base_urls"499+S11rc V^8dQhRRRR/#)r router2rr)rs"rrrds66#6#6rc $RVP V 2#)r}r~)rxrs&&r_urlLlamaServerSupervisor._urls"499+eW55r json_typeTc(V^8dQhRRRRRRRR/#) r rr2body dict | None timeout_srrboolr)rs"rrrds)>>3>k>S> >rc 0RRVP 2/pV'dRVR&\PPVP V4TVe%\ P !V4P4MRR7p\PPWcR7#) AuthorizationzBearer zapplication/jsonz Content-TypeN)headersdatarO) rgurllibrequestRequestrjsondumpsencodeurlopen)rxrrrrrreqs&&&&$ r_openLlamaServerSupervisor._opens"gdll^$<= &8GN #nn$$TYYu%5wGKGW$**T*:*A*A*C]a%c~~%%c%==rc(V^8dQhRRRRRRRR/#) r rr2rrrrrdictr)rs"rrrds)22c222UY2rc VPWV4;_uu_4pVP4pV'd\P!V4M/uuRRR4# +'giR#;iN)rreadrloads)rxrrrrraws&&&& r_requestLlamaServerSupervisor._requests> ZZY / /1&&(C&)4::c?r0 / / /s 0A A' cV^8dQhRR/#rr)rs"rrrds&&&rc dVPp\V4RRR\VP4RVPR\VP4RRRRR .\ V4OpVP 'd@VP P4'd VR \VP 4., pMVR \VP4., pW P, pVPPPR R R 7VPe \VPP4\!VPRRRR7Vn VPP#RV R24VPP%4\'W P\(P*\VP4\-\.P04R7wVnVn\6P9RVP2P:VP4VP=4R#)z--hostr!z--portz --api-keyz --models-maxz--models-autoloadz --metricsz--slotsz--no-uiz--jinjaz--models-presetz --models-dirTr5Nar8rJ)r4rMz # spawn:  )rUrVrPenvz*llama-server router spawned pid=%s port=%s)rbr2r]rgr\rYr`existsrcr^r_r>r?rurcloseopenwriteflushr rSSTDOUTr osenvironrirjr-infopid _write_state)rxexecmds& r_spawnLlamaServerSupervisor._spawnskk H k c$))n  C0     S ! "     0 0 7 7 9 9 %s4+;+;'<= =C NC$89 9C  ""4$"?    ' 4##)) * sWYW SE45  +C8H8HQ[QbQb14SZZFVWYWaWaFb d 49 @$))--QUQZQZ[ rc V^8dQhRRRR/#r rrrrr)rs"rrrdssTrc nVP;_uu_4RVnVPP4VP 4RRR4VP V4\ P!VPRRR7Vn VPP4R# +'giLe;i)FNTzllamacpp-supervisor)targetdaemonname) rrrmrpclearr _wait_healthrnThread_watchrtstart)rxrs&&rrLlamaServerSupervisor.startsy  ! ! !"DN    " " $ KKM" )$"))TPef  " !s 2B$$ B4 cV^8dQhRR/#rr)rs"rrrds99d9rc *^RIp^RIp^RIHp^RIHpVP VPP4pRVPRVPRVPRVP4RV!VP4R VP4R VP!4R VP 4P4R V!VP!44/ Vn\4p^R IHpV!VP$4V!W`PRR7R#)r"N)get_process_start_time)atomic_json_writerrgr create_time start_timerH owner_pidowner_create_timeowner_start_time)mkdir_under_hermes_homei)mode)rpsutilgateway.statusrutilsrProcessrirrrgrrgetpidrsrhermes_constantsrr>)rxrrrrripathrs& rr"LlamaServerSupervisor._write_states9+~~diimm,!4==)T\\dhh t7G7G7I#%;DHH%E#TXXZbiik*FNN,<,H,H,J)+A"))++N P |< ,$ %8rc V^8dQhRRRR/#rr)rs"rrrds b bc bd brc  \P!4V,p\P!4V8EdGVPP4'd \ R4hVP 'dMVP P 4e1\ RVP P RVP R24h\\PP\\4;_uu_4\PP!VP#R4^R7;_uu_4pVP$^8XdRRR4RRR4R#RRR4RRR4\P&!^4EKa\RV RVP R24h +'giLM;i +'giLX;i) zllama-server startup cancelledNzllama-server exited rc=z during startup (log: )z/healthrzllama-server not healthy after zs (log: )time monotonicrpis_set RuntimeErrorripoll returncoder_rrerrorURLErrorr, TimeoutErrorrrrstatussleep)rxrdeadliners&& rr"LlamaServerSupervisor._wait_healths@>>#i/nn)&&(("#CDDyyyTYY^^-9"%>&,,//,GG^^++DIIi,@!+LLPQxx3MHG&MH JJqM,>?&-! *) E >DD EsBF3F (F32 F *F3=F3 F0 +F33 G >GG cV^8dQhRR/#rr)rs"rrrds((d(rc lVP;_uu_4RVnVPP4VP'd8VPP 4fVP VP4VPe"VPP4RVnTP'd"TPP4RTn RRR4R# TPe#TPP4RTnii;i +'giR#;i)TN) rrrmrpsetrir_terminate_treerjrrurs&rstopLlamaServerSupervisor.stops  ! ! !!DN    " %999!1!9((399(IIOO% $DI  &&(#' " !99(IIOO% $DI)" ! !s*"D"A C-AD"!D"-2DD"" D3 verified_rootFc$V^8dQhRRRRRR/#)r rizsubprocess.Popenrrrrr)rs"rrrd0s(DD.D$DSWDrc  Ja.p\P3p\\4;_uu_4^RIpW4P3, pV'dEVP RR7pVP 4'gVPVP4hMQ\VP4;_uu_4VPVP4P RR7pRRR4RRR4VFo\SP4K VP4VP^R7VFo\V3Rl4K R# +'giLt;i +'giL;i TdTP4LYi;i TFo\T3Rl4K i;i)a-Terminate the router AND its model children. Each child holds gigabytes of VRAM; terminating only the router (TerminateProcess on Windows does no cleanup) orphans them with the weights still resident. Enumerate children FIRST (the parent must be alive to walk them), terminate all, escalate to kill. NT) recursivercT<SP4;'dSP4#r) is_runningkill)childsr7LlamaServerSupervisor._terminate_tree..Osu//1BBejjlBr)rSTimeoutExpiredr ImportErrorrchildrenr NoSuchProcessrErrorrr terminaterr)rirrtimeoutsrrs&$ @rr%LlamaServerSupervisor._terminate_tree/s;--/ k " "  ..0 0H==4=8(( ..txx88)fll++%~~dhh7@@4@PH,# D!u'" NN   " %"BC",+# "$   "BC"sTAE49E-,E E*.FE+ E E E( +FFFFF"cV^8dQhRR/#rr)rs"rrrdQsrc ZVPe$VPP4RVnR#\PR8XdR#^RIp\ VP 4pTPeTPPMRpTPRR.4Fp\TPTP4;_uu_4TPPR4;'g^pTPPR4T8wg1TPT8Xg T'd#TP!T4'd RRR4K\"P%RTP4TP'4RRR4K R# \dR#i;i +'giEK;i)a)Kill model children orphaned by a router crash, before respawn. A dead parent can't be walked, so match by identity: any process running OUR llama-server binary whose parent is gone is an orphan of a previous router. Its VRAM must come back before the new router loads models next to the ghosts. Nwin32rppidz*reaping orphaned llama-server child pid=%s)rjrsysplatformrr2rbrrir process_iterrr AccessDeniedrget pid_existsr-r.r)rxrrown_pidprs& rr-LlamaServerSupervisor._reap_orphaned_childrenQs8 99 IIOO DI  <<7 "   dkk"C$(99#8$))--d$$eV_5A&..0C0CDDvvzz&)..QFFJJu%,0@ V%6%6t%<%< ED KQUUS ED6   EDDs1 FAF&F=F0F FF F* c V^8dQhRRRR/#)r rrrrr)rs"rrrdpsXXXTXrc VPRVR7PR.4Uu/uF-pVR,VPR/4PRR4bK/ up#uupi)z*{model_id: status_value} from GET /models.z/modelsrridrvalueunknown)rr)rxrms&& rmodelsLlamaServerSupervisor.modelspsiyIFJJ6SUVXVA$x,00)DDVX XXs3Ac$V^8dQhRRRRRR/#)r model_idr2rrrrr)rs"rrrdus&PP3P3PPrc 2VPRRV/VR7R#)z /models/loadmodelrNr)rxr"rs&&&r load_model LlamaServerSupervisor.load_modelus nw&9Y OrcV^8dQhRR/#rr)rs"rrrdxs888rc ,VPR^ R7R#)zHave the router re-read the preset file (GET /models?reload=1). It applies new launch flags to models that aren't loaded and unloads any loaded model whose flags changed, so callers rewrite the file only while nothing is loaded.z/models?reload=1rNr%rs&rreload_presets$LlamaServerSupervisor.reload_presetsxs (B 7rc V^8dQhRRRR/#)r r"r2rrr)rs"rrrd~s  S T rc TVPRRV/^xR7\P!4^,p\P!4V8dJVP4P V4.\ ORN59dR#\P!R4KcR# \ dR#i;i)uFree the child's VRAM now (POST /models/unload; bogus name -> 400). Momentary: never touches primary_model — the declaration is durable, an eject is not.z/models/unloadr$r unloadingNg333333?)rrrrr _RESIDENTrr)rxr"rs&& r unload_model"LlamaServerSupervisor.unload_model~s &((;s K>>#b(nn) ;;=$$X.6O 6O;6OOP JJsO *  s-B B'&B'c V^8dQhRRRR/#)r nowz float | Nonerz list[str]r)rs"rrrds!!l!i!rc  Vf\P!4MTp.pVP4pTP 4EFBwrET\ 9dTP PTR4K/TPT4pTfB\PRT\YP PYA4, 44KTRJdTP PTR4KTP PYA4pY, TP8dKTPT4TP PTR4TP!T4\PRT\Y, 44EKE T# \dTu#i;i \d#p\P#RYH4Rp?EKRp?ii;i)a6Unload models idle past IDLE_UNLOAD_S; returns their ids. Idle = no busy slots and no queued work, tracked per model across calls; a model seen busy resets its clock. A failed telemetry probe is neither idle nor busy: the clock is kept, so a flaky probe cannot pin a resident model (and its VRAM) indefinitely.Nz7idle probe for %s failed; keeping idle clock (idle %ds)Fzidle-unloaded %s (idle %ds)zidle unload of %s failed: %s)rrrritemsr/rvpop _probe_idler-rrr setdefault IDLE_UNLOAD_Sr0appendr.) rxr3unloadedstatusesr"rprobe first_idlerDs && r sweep_idle LlamaServerSupervisor.sweep_idles #&+dnn3  {{}H!) 0 HY&  $$Xt4$$X.E} UW_&6&6&:&:8&I IJL~  $$Xt4))44XCJ$"4"44 N!!(+  $$Xt4) 98SIYEZ['!1,1 O , N=xMM Ns*FA$F FF G&GGc$V^8dQhRRRRRR/#r r"r2rrrrr)rs"rrrds!  s s T rc  VPRRVRRRR\/.RRR ^/VR 7pVR ,^,R ,pVPR4;'gR R,VPR4;'gR ,p\VP 49# \ d"p\ PRY4Rp?R#Rp?ii;i)uThe readiness proof. Generous budget + reasoning_content scan — small token budgets false-fail reasoning models, which spend their first tokens thinking.z/v1/chat/completionsr$messagesroleusercontent max_tokensi temperaturerchoicesmessage reasoning_contentz"touch generation failed for %s: %sNF)r TOUCH_PROMPTr TOUCH_EXPECTlowerrr-r.)rxr"rrespmsgblobrDs&&& rtouch_generate$LlamaServerSupervisor.touch_generates ==!7:L0Y/Zc=!:5@I!KDy/!$Y/CGGI&,,"3sww?R7S7Y7YWYZD4::</ /  NN? O s$AB B7B C"B>>Cc$V^8dQhRRRRRR/#rBr)rs"rrrds!--3-3--rc VP4PV4pVf\RV R24hV\9dVP WR7VP V4#)z=Load if needed, then prove readiness with a touch generation.zmodel z not present in models dirr)rrKeyErrorr/r&rU)rxr"rrs&&& rr(LlamaServerSupervisor.ensure_model_readysY""8, >VH:-GHI I  " OOHO :""8,,rc V^8dQhRRRR/#)r r" str | Nonerrr)rs"rrrds22 2d2rc (VPV4RJ#)zNo processing requests and no busy slots. Router quirk: /slots and /metrics are per-child and require ?model= (bare calls 400). With ``model_id`` checks that one child; without, every loaded child.T)r7)rxr"s&&ris_idleLlamaServerSupervisor.is_idles)T11rc V^8dQhRRRR/#)r r"r\rz bool | Noner)rs"rrrdsJ+rc VeV.M>VP4P4UUu.uFwr#V\9gKVNK upppVEFpVPR\ V4 24p\;QJdRT4F 'gK RM RM !RT44'dR#TPR\ T4 2^ RR7;_uu_4pTP4P4p RRR4X P4FEp T PR 4'gK\T P!4R ,4R 8wgKCR# EK R#uuppi \ P PdpTPR8XdRp?EKKhRp?ii;i +'giL;i \"dR#i;i) uDTri-state idle probe for the sweeper: True = confirmed idle, False = confirmed busy, None = the probe itself failed. The sweeper must never mistake a dead probe for activity — that resets the idle clock and pins the model's VRAM. A model that isn't loaded is idle: the router answers 400 for it (the reads don't autoload).Nz/slots?ic3B"TFqPR4xK R#5i) is_processingN)r).0r's& r 4LlamaServerSupervisor._probe_idle..s=u!uu_--usTFz /metrics?)rrzllamacpp:requests_processingg)rr5r/rrrr HTTPErrorcodeanyrrdecode splitlines startswithfloatsplitr) rxr"rrloadedmidslotsrDrrLlines && rr7!LlamaServerSupervisor._probe_idlesd  $,$8xj/3{{}/B/B/D\/D)!R[H[AA/D\  MMGK4D3E*FGE 3=u=333=u=== ZZ)K,<+= >"X]Z^^bc668??,D_ OO-D(FGG %djjl2&6 73 >$.!]||--xx3  _^  s(F1EE F1E, F17F1 F1&F1)(F1F0/F1$"F1 F1F1F1F=FF1FFF1 F. )F11 G?G)rvrjrrrurwrlrsrprmrtrgrbr^r_rcr\r]r`rkrii)N)x)ru)iXr)i,)__name__ __module__ __qualname____firstlineno____doc__r9rypropertyrrrrrrrrrr staticmethodrrrr&r*r0r?rUrr^r7__static_attributes__rrrr[r[osO M.#$.9=.04.*..-1 .@226>#'>>2&P9& bE:( DDDB>X P8 !F -2 rr[)<)rpready)+r{ __future__r contextlibr functoolsrrloggingrr<r#rSrrnr urllib.errorrurllib.requestpathlibr!hermes_cli.local_runtime.binariesr!hermes_cli.local_runtime.endpointr"hermes_cli.local_runtime.processesr r getLoggerrwr-rOrPrr/r+rrr(r/rErYr[rrrrs#     ;9M   8 $D  #   +  " , 2;;wwr