+ jHjZRt^RIHt^RIHtHt^RIt^RIt^RIt^RI H t ^RI H t ^RI HtHt]P !]4tRsRsRsRR ltR R ltR R ltRRltRR/RRlltRRltRRltRRltRRltRRltRRlt R9R R!llt!R:R"R/R#R$lllt"R;R%R&llt#]$!0RR/R0ll4t+R?R1R2llt,R3R4lt-R5t.R6t/^xt0^t1R7R8lt2R#)@arBootstrap for the managed runtime: config -> installed binaries -> running supervised server. One public call, ``ensure_local_runtime(config)``, safe at any session start: disabled or already-running -> no-op; enabled -> serve the installed build under a supervisor. Kept import-light: callers gate on config before importing so disabled sessions never pay the import. ) annotations)contextmanagersuppressN)Path) runtimes_root) SPLIT_PART_REmodel_id_from_stemFcV^8dQhRR/#returnNone)formats"G/home/ubuntu/.hermes/hermes-agent/hermes_cli/local_runtime/bootstrap.py __annotate__rs  t cX\'g^RIpVP\4RsR#R#)a3The process that boots the server stops it on a clean exit, whatever surface it is (the desktop backend also stops it in its shutdown handler; the second call is a no-op). Processes that only adopted another's server never stop it. A hard kill still leaves an orphan, which the next boot adopts or replaces.NT) _EXIT_HOOKEDatexitregistershutdown_local_runtime)rs r _stop_at_exitrs# <./ rcV^8dQhRR/#)r r str | Noner)rs"rrr)s H HJ Hrc^RIHp^RIHpV!4pVe(VP R4'dRVR,,#V!4pVR9dV#R#)zBest-effort GPU vendor for backend selection. Preserve NVIDIA's detailed name when nvidia-smi is available, then use the platform host facts so Intel and AMD do not silently demote ``auto`` to CPU. )_cached_nvidia_gpu_query) gpu_classNgpu_nameznvidia >amdintelnvidia)!hermes_cli.local_runtime.hardwarerhermes_platform.host.factsrget)rrquerydetecteds r_detect_gpu_vendorr')sQ K4 $ &E UYYz225,,,{H#==8G4GrcV^8dQhRR/#r r rr)rs"rrr9s00D0rc*^RIHpV!4R, #)zMachine-scoped, deliberately NOT profile-scoped: a 20 GB GGUF is a machine asset, and every profile shares the one managed server that serves it (same rule as runtimes_root()).get_default_hermes_rootmodels)hermes_constantsr,r+s r models_dirr/9s9 " $x //rcV^8dQhRR/#r)r)rs"rrrAs##D#rc$\4R, #)uNon-model companion files (mmproj projectors, spec-decode drafts). A subdirectory so the router's model listing — and staged_models() — never mistakes an asset for a servable model.assets)r/rrr assets_dirr3As <( ""rrequire_completeTc$V^8dQhRRRRRR/#)r r/rr4boolr list[Path]r)rs"rrrGs!$TZrc aaa\VPR44pVUu0uFq3PkK upo.pVEF p\P!VP4oSfVP V4K;SP ^4R8wdKSVPRSP4\SP ^44uopV'di\;QJd2VVV3Rl\^V^,44F 'dK RM' RM#!VVV3Rl\^V^,444'gKVP V4EK V#uupi)uServable GGUFs in a directory: single files, plus split GGUFs once by their first part. With ``require_complete`` a split counts only when EVERY part is on disk — a mid-download split is not servable and must not surface anywhere as a model.*.ggufN00001c3`<"TF#pS RVR RSP^4 R2S9xK% R#5i)-05dz-of-z.ggufN)group).0imnamesstems& r staged_in..Vs<'E0C1+/q3tAGGAJstartintallrange) r/r4filespouttotalrArBrCs &$ @@@r staged_inrSGs :??8, -E" #UVVU #E C    ( 9 JJqM  771: ff[qwwy)3qwwqz? e33'E05a0C'E333'E05a0C'E$E$E JJqM J $sEcV^8dQhRR/#r r r7r)rs"rrr\s))Z)rc \^RIHpHpV!4R, pVP4'g.#.p\ VP 44EFZpVR, pVP PR4'g%VP4'dV!V4'gKTV\43VR, \433Fwrg\ VPR44F~pWxP , p V P4'd\PRW4KCVPRRR 7\P !W4TP'T 4K K VR, V3F3p \)\$4;_uu_4V P+4R R R 4K5 EK] V'd)\P-R \/V4\44V# \"dEK\$d$p \PR YT 4R p ? EK6R p ? ii;i +'giK;i) a Move GGUFs left in the old per-profile ``/models`` layout (and its assets/) into the machine-scoped dirs, so everything downstream keeps reading one directory. ``os.rename`` only: within one filesystem it is instant even for a 20 GB model, while ``shutil.move`` silently degrades to a copy across devices. A cross-device profile dir is left in place with a warning rather than copying tens of GB at session start. A name that already exists in the destination is left alone (check-then-rename: the only window is two processes adopting two profiles' same-named file at once, and a same name is the same catalog variant). Two processes racing on one file are harmless: the loser's rename finds the source gone and skips it. Returns the new paths of the moved files.)r,named_profile_has_identityprofilesr-.r2r9z,legacy model %s not moved: %s already existsTparentsexist_okz#legacy model %s not moved to %s: %sNz%moved %d legacy model file(s) into %s)r.r,rWis_dirrFiterdirrH startswithr/r3rGexistsloggerwarningmkdirosrenameFileNotFoundErrorOSErrorrJrrmdirinfolen) r,rW profiles_rootmovedhomeoldsrc_dirdest_dirsrcdestexcemptieds radopt_legacy_modelsru\sU+- :M    ! ! E},,./Xo 99   $ $CJJLL@Z[_@`@` #& "5h 7U!V Ggll845((*;;==NN#QSV]NN4$N?IIc(  T"6"Wh,G'"" #"-'0,  ;SZV L)NN#H#Y\] #""s0 )GH  H *H 3H 4H H  H+ cV^8dQhRR/#rUr)rs"rrrs##z#rc(\\44#)zWServable staged models (continuation parts, incomplete splits and assets/ never count).)rSr/rrr staged_modelsrxs Z\ ""rcV^8dQhRR/#)r r z list[str]r)rs"rrrsAA)Arcf\4Uu.uFp\VP4NK up#uupiN)rxrrC)rPs rstaged_model_idsr|s&0= @1 qvv & @@ @s.cV^8dQhRR/#r r r6r)rs"rrrs   rca\\4;_uu_4^RIHpV!4o\;QJd'V3Rl\ 44F 'gK RM RM!V3Rl\ 444uuRRR4# +'giR#;i)u|True when a staged model has no section in the preset INI — it would autoload with stock fit instead of a policy decision.)read_preset_decisionsc3<"TFcpVS9;'gRSV,P'*;'d2SV,P;'g/PR4'*xKe R#5i)modelN)refusalkeysr$)r?midknowns& rrD!_presets_stale..s]20se#llE#J,>,>(>(k(kc H]H][]GbGbcjGkCkl0sA.A.A.A.TFN)r Exception hermes_cli.local_runtime.presetsranyr|)rrs @r_presets_stalers_ )  J%'s2.02sss2.022     s5A;A;; B cV^8dQhRR/#r r)rs"rrrs``D`rc^RIHpHpHpHpHp^RIHpV!4pV!V4'd?V!V4'g(\PRVPR44R#V!V4pMV!V4pVfR#VPVRR7R# \d-p\PRTPT4Rp?R#Rp?ii;i) urStop the server the state file points at before this process boots a replacement, but only an orphan (recorded owner gone): a live owner's watchdog would respawn its router, so that one is left running and the replacement boots beside it, as it always has. The process comes from the identity-guarded reader, never a bare PID — the endpoint dict callers hold has none.)_owner_is_dead is_modernlegacy_recorded_process read_staterecorded_processLlamaServerSupervisorz@llama-server pid=%s belongs to a live Hermes process; leaving itpidNT) verified_rootz6could not stop the incumbent llama-server (pid=%s): %s)!hermes_cli.local_runtime.recoveryrrrrr#hermes_cli.local_runtime.supervisorrrarir$_terminate_treerrbr) rrrrrrstateprocrss r_stop_state_serverrs J LEe$$ KKZ\a\e\efk\l m &&u- |`--d$-G `OQUQYQY[^__`s7B C!B>>CcV^8dQhRR/#r~r)rs"rrrstrc^RIHp\f^RIHpV!4pVfR#\ 4M \ 4\V!4RR7RJ# \d"p\PRT4Rp?R#Rp?ii;i)aRestart the managed server so it rescans the models directory. The router's model list is SPAWN-ONLY: a GGUF added after start is invisible to GET /models and 400s on completion, so anything that changes the staged set while the server runs must bounce it.) load_configN_state_endpointFT)forcez local runtime refresh failed: %s) hermes_cli.configr _SUPERVISOR!hermes_cli.local_runtime.endpointrrrensure_local_runtimerrarb)rrrrss rrefresh_local_runtimerse 1   I#%E}  " $#KM>dJJ 93?sA (A A8A33A8c(V^8dQhRRRRRRRR/#)r mdirr configuredrL engine_tagrr r)rs"rrrs)t*X[rc^RIHp^RIHpV!RR7pVP'd \ V4MRpV!WWVR7pYq8wd\PRYq4T# \ d#p\PRY4TuRp?#Rp?ii;i) uResidency cap to hand the router: derived from the hardware budget, ``models_max`` as a ceiling. A cap of "four" on a card that holds one model is how a second child ends up paged (WDDM) and silently slow — llama.cpp evicts its LRU before an incoming load only when the cap says the card is full. A probe miss or an unpriceable model keeps the configured count: this must never block a boot. On Windows unified memory, free commit bounds the count too.  probe_budget)admitted_residency_countTplanningN)liverz4residency cap probe failed (%s); using models_max=%sz@residency cap: %s resident model(s) on this card (models_max=%s)) r"rrruma_launch_budgetrrarbri) rrrrrcapacityrcaprss &&& r_admitted_models_maxrs BM.+3<<<~h'T&tzYcd  V % J Ms_s"&AA B *BB B rc ,V^8dQhRRRRRRRRRR /#) r rr preset_path residentsrLrrr z Path | Noner)rs"rrrs1$$D$t$$",$8C$rc ^RIHp^RIHpV!RR7pV!WV\ V4W#R7F7pVP 'gK\ PRVP 4K9 V# \dUpTP4'd\ PRT4TuRp?#\ PR T4Rp?R#Rp?ii;i) uWrite the launch-policy INI for every staged model; returns the path to hand the router. Priced against CAPACITY, not live free VRAM: this runs while the outgoing server instance may still hold the card (restart, refresh after a download), and its memory is freed before the new instance loads anything. Pricing against live-free once pinned a fitting model's weights to CPU. The window is then narrowed to what fits beside other programs' GPU memory (``hardware.launch_budget``), which never moves weights to the CPU. Degradation ladder on failure: a STALE policy still beats no policy — stock fit (f16 KV at max context, no placement) is the silent-busy-wait failure on Windows. Keep serving with the previous INI when one exists; only a first boot with no INI at all falls to stock fit. ``residents`` is the router's model count, which splits the prompt-cache room on Windows unified memory. r)generate_presetsTrrrrz"model refused by physics check: %supreset generation failed (%s); serving with the PREVIOUS launch policies — models staged since the last successful generation run unpoliced until this is fixedNzWpreset generation failed (%s) and no previous policy file exists; router runs stock fit) r"rrrrrrarbrr`error) rrrrrrrentryrss &&&$ r_generate_presetsrs"?A.%dkW_H`09RE}}}CU]]SR        LL/14 5  ABE G s)0A)%A)) C4-C!C'CCcV^8dQhRR/#)r own_bytesrLr)rs"rrrsrc^RIHpV!WR7# \d"p\P RT4Rp?R#Rp?ii;i)zS``hardware.launch_budget``, or None when the probe fails: a boot never waits on it.) launch_budget)rz>free GPU memory probe failed (%s); launch windows use capacityN)r"rrrarb)rrrrss&& rrrs8?X;; WY\]s =8=cV^8dQhRR/#r~r)rs"rrr"s&&t&rc ^RIHp^RIHpHpVP pVe(VP 4'd\V4'dR#V!RR7p\V4pVfR#VPpVe(\VPV, 4\8dR#T!T!\4YVVP\e\P MRR74pVP";_uu_4VP$e.VP$P'4f\V4'd RRR4R#VPVn WP)RR 78Xd RRR4R#^R IHp V !WHR VP. R 2R R7VP14RRR4\2P5RVPR, 4R# +'giL9;i)aRe-plan launch windows against free GPU memory while no model is loaded. Boot plans every window once, but models load later (on demand, after an idle unload), and by then other programs may hold more or less of the card. Returns True when the router was given new presets. A request that starts a load between the last status check and the reload is unloaded by the router and fails once; that gap is a few milliseconds, and only when a window changes. r) plan_presetsrender_presetsNFTrrz utf-8-sig)encoding)atomic_write_textrY_) tmp_prefixmodez9launch windows re-planned for %.1f GiB of free GPU memoryi@)r"rrrrrr`_any_holds_memoryr _refit_usableabsusable_vram_bytes_REFIT_STEP_BYTESr/ models_max_SERVING_ENGINEtag_lifecycle_lockrpoll read_textutilsrrHreload_presetsrari) suprrrpathrrlasttextrs & rrefit_idle_presetsr"so?M ??D |4;;==,=c,B,BT*H ( #D |   D C 6 6 =>ARR ,z|XTWTbTbIXId/2E2Ejnp qD    88 sxx}}:>OPS>T>T  !22 >>;>7 7   ,$1TYYKq1AN   KKK&&'24   s)=G0(G"/G G cV^8dQhRR/#r~r)rs"rrrPsWWdWrc\;QJd:RVP^R7P44F 'gK R# R#!RVP^R7P444#)c32"TF q\9xK R#5ir{) _HOLDS_MEMORY)r?statuss& rrD$_any_holds_memory..QsV5U6&5Us) timeout_sTF)rr-values)rs&rrrPsS 3VSZZ!Z5L5S5S5UV33V3V3VSZZ!Z5L5S5S5UV VVrc V^8dQhRRRR/#)r fdrLr r6r)rs"rrrTs#$rc\PR8Xd^RIp\P!V4P^8Xd\P !VR4\P !V^\P4VPWP^4R#^RI pVPWPVP,4R# \dR#i;i \dR#i;i)z=Non-blocking exclusive attempt; portable across fcntl/msvcrt.ntNTF)rdrHmsvcrtfstatst_sizewritelseekSEEK_SETlockingLK_NBLCKrgfcntlflockLOCK_EXLOCK_NBrrrs& r_try_lock_boot_fdrTs ww$ 88B<  1 $ HHR  Q $  NN2 2   KKMMEMM9 :    s$=C ,C  CC C/.C/c V^8dQhRRRR/#)r rrLr r r)rs"rrrks ' ' ' 'rc\PR8XdI^RIp\P!V^\P4VP WP ^4R#^RIpVPWP4R#)rN) rdrHrrrrLK_UNLCKrrLOCK_UNrs& r_unlock_boot_fdrksI ww$ Q $r??A. B &rcV^8dQhRR/#)r rfloatr)rs"rrrxsrc #"\4R, pVPPRRR7\P!V\P \P ,R4p\P!4T,p\T4'gJ\P!4T8d\PRT4M\P!R4KZRx\\4;_uu_4\T4RRR4\P !T4R# \d&p\PRT4RxRp?R#Rp?ii;i +'giL[;i \\4;_uu_4\T4RRR4i +'gii;i;i \P !T4i;i5i) uSerialize the state-check-then-spawn sequence across every Hermes process on this machine — the ``_SUPERVISOR`` singleton above only rules out a race within ONE process. Two profiles booting in the same second each see no ``server.json`` yet and each spawn a router on the stable port (#116682); an OS-held lock makes the second caller wait for the first to publish its state file, so it re-checks and adopts instead of spawning a duplicate. Bounded, not indefinite: never hang session start dead if the lock is somehow stuck, and never raise into session start: an unwritable runtimes dir (or a foreign-owned lock file) proceeds unlocked with a warning, like the contention timeout.z boot.lockTrZrz1boot lock unavailable (%s); proceeding without itNz5boot lock contended past %.0fs; proceeding without itg?)rparentrcrdopenO_CREATO_RDWRrgrarbtime monotonicrsleeprrclose)rrrrsdeadlines& r_cross_process_boot_lockr ws' ?[ (D $6 WWT2:: 15 9 >>#i/#B''~~8+VXab JJsO $ '""##  # JCP #"'""##"""  sGAD,.A6F5%E2)F5 E F5G, E7EGEG E/ *F52F2 F  F2 F/ *F22F55G  Gc$V^8dQhRRRRRR/#)r configdictrr6r z object | Noner)rs"rrrs&MMMdM}MrchT;'g/PR4;'g/pV'gVPR4'gR#\e\#\4V'g(\ 4'g\PR4R#^RIH p\4;_uu_4V!4pVeP\4'g!\PR4RRR4R#\PR4\4^R I Hp^R IHpV!VPR R 44pVf!\PR 4RRR4R#Vs\%4p V P'RRR7\)V \+VPR^44VP,4p \/V \14R, V VP,R7p T!VP2YT \+VPR^44;'gRR7p V P54T s\=4\PRT P>TP@TP,4\CT 4T uuRRR4# \d"p\P RT4Rp?EL,Rp?ii;i \6dF\9\64;_uu_4T P;4RRR4h +'gih;ii;i \6d,pRs\P RT4Rp?RRR4R#Rp?ii;i +'giR#;i)uIdempotent boot of the managed runtime. Returns the supervisor (or None when disabled/unavailable). Never raises into a session start — failures log and return None; chat falls back to configured providers. local_runtimeenabledNz legacy model adoption failed: %sz7local runtime enabled but no models staged; not bootingrz6managed llama-server already running (another process)zfrunning server's presets predate the staged models; replacing it so every model launches with a policy)installed_enginerbackendautozKlocal runtime enabled but no PM engine installed; use the Local Models paneTrZrz presets.ini)rport)rrrz1managed llama-server up at %s (backend=%s tag=%s)z%managed local runtime unavailable: %s)"r$rrurgrarbrxrirrr rr!hermes_cli.local_runtime.binariesrrrrr/rcrrLrrrbinaryrKrrstoprbase_urlr_start_idle_sweeper) rrsectionrsrrrrenginerrrrs && rrrs ||  177RG Y//@  MNB ! # #!  !## TU $ # KKM N  #  J Q%gkk)V&DEF~ ij# $ #$%O._loops^^% hh"sxx}}'> JJ~ &~~*,>!^^- @NN$ G"3'(?" !@LL!93??@ G @#FF Gs0B8' C&8 C#CC#& D1D  DTzlocal-runtime-idle-sweep)targetdaemonrH) threadingThreadrK)rr1r-sf rrrs/ GE$5OPVVXrr{))>readyloadedloadingsleepingi)g@`@)F)3__doc__ __future__r contextlibrrloggingrdrpathlibrrrhermes_cli.local_runtime.ggufrr getLogger__name__rarrrrr'r/r3rSrurxr|rrrrrr frozensetrrrrrrr rrr"r$r)r(rrrrrBs #/ ;K   8 $    H 0# D*)X# A `>.0$/3$$NDE &VW. 'BM`  Yr