+ iHj/g a0t$Rt^RIHt^RIt^RIt^RIt^RIt^RIt^RI H t ^RI H t ^RI Ht^RIHtHtHtHtHtHtHt]P.!]4t]!0R`m4tRatR P;R]44t]P>!R],R,]P@4t!]P>!R],R,]P@4t"]P>!R]PF4]P>!R43t$RRlt%RRlt&RRlt'/](PSRbR4C](PSRcR4Ct*RRlt+RR lt,R!R"lt-R#R$lt.R%R&lt/R'R(lt0R)R*lt1R+R,/R-R.llt2R/R0lt3R1R2lt4R3R4lt5RdR5R6llt6R7R8lt7R9R:lt8R;R<lt9R=R>lt:R?R@lt;RA]93RB]:3RC];33tR+R,/RHRIllt?/](PSRfRJ4C](PSRgRK4Ct@.RhORQ]@P44ORiNRjNRkN5tBRR]=RS&RTRUltC]!0Rlm4tDRVRWltERRMR RMRRLR RNR ROR RP/tFReRXRYlltGRZR[ltHR\R]ltIReR^R_lltJ.RmOtKR#)naRRouting helpers for inbound user-attached images. ``native`` attaches images as OpenAI-style ``image_url`` parts; ``text`` runs ``vision_analyze`` up-front and prepends the lossy description (right for non-vision models). :func:`decide_image_input_mode` picks once per turn from ``agent.image_input_mode`` (``auto`` | ``native`` | ``text``): in ``auto`` an explicit ``auxiliary.vision`` backend forces ``text`` even for vision-capable main models (``native`` is the absolute override); else ``supports_vision`` (config override or catalog) decides. ``vision_analyze`` stays a tool regardless. ) annotationsN)suppress)BytesIO)Path)AnyCallableDictIterableListOptionalTuple.png.jpg.jpeg.gif.webp.bmp|c#B"TFqPR4xK R#5i).N)lstrip).0es& 8/home/ubuntu/.hermes/hermes-agent/agent/image_routing.py r!sA[hhsmm[zB(?\"']+?\.(?:z)(?:\?[^\s<>\"']*)?z```[^\n]*\n.*?```z `[^`\n]+`c$V^8dQhRRRRRR/#)patternz re.Patterntextstrreturn Iterable[str])formats"r __annotate__r%3s&ii:iSi]ica\UUu.uF9q"PV4F"q3P4VP43NK$ K; uppoV3RlVPV44#uuppi)zWYield ``pattern`` matches whose start is not inside a fenced block or inline backticks.c3<a"TFXo\;QJdV3RlS4F 'gK RM RM!V3RlS44'dKESP^4xKZ R#5i)c3p<"TF+wrVSP4u;8*;'dV8MuxK- R#5iN)start)rsrms& rr2_matches_outside_code...6s.AgafY]YZ!qwwyBTBTSTBTBTafs36TFN)anygroup)rr-spanss&@rr(_matches_outside_code..6s> h 61ccAgafAgcccAgafAg>gJAGGAJJ 6sA$A$A$ A$)_CODE_SPAN_RESfinditerr+end)rrpr-r1s&& @r_matches_outside_coder73sS+9 T>a::dCSaggi !CS !> TE h 0 0 6 hh Us?A)c V^8dQhRRRR/#)r candidater r! Optional[str]r#)r$s"rr%r%9s  c m r&c\PPV4p\PPV4'd \PP V4#R# \ dR#i;i)ubNormalized path when it is a regular file; None otherwise (incl. OSError on pathological input). The return value is the OS-canonical spelling of the real file (same norm class as ``str(Path(...))``), so callers can compare it against actual paths — ``~`` expansion alone would leave the textual ``/`` separators of the source text in place on Windows.N)ospath expanduserisfilenormpathOSError)r9expandeds& r_existing_filerC9s[ ww!!),H-/WW^^H-E-Erww)O4O s$A'A'%A'' A65A6c V^8dQhRRRR/#)rrr r!ztuple[list[str], list[str]]r#)r$s"rr%r%Es ) )S )%@ )r&c 6\V\4'd V'g..3#\PR\ \ \ \V4444p\PR\ \V444p\V4\V43#)u*Scan free-form text for image references → ``(local_paths, urls)``, each ordered and deduplicated. Local paths must exist as files; URLs are not validated (the provider fetches them). Code spans are skipped so pasted snippets aren't live attachments (mirrors ``BaseAdapter.extract_local_files``).c38"TFq'gK VxK R#5ir*r#)rr6s& rr%extract_image_refs..Ls Ya]^Y  c3B"TFqPR4xK R#5i)z .,;:!?)]>N)rstriprus& rrrGPsc8b1+..8br) isinstancer dictfromkeysmaprCr7_LOCAL_IMAGE_PATH_RE _IMAGE_URL_RElist)r local_pathsurlss& rextract_image_refsrVEs{ dC 2v -- ~'<=QSW'XY K ==c8Mm]a8bc cD  d4j ((r&TFc V^8dQhRRRR/#)rrawrr!Optional[bool]r#)r$s"rr%r%Zs S S S Sr&c\V\4'dV#\V\4'dVR9d \V4#R#\V\4'd2\P VP 4P44#R#)a4Strict boolean coercion for capability overrides: real bools, 0/1 and YAML boolean tokens only; anything else is None so the caller falls through to models.dev. ``bool("false")`` is True, so a quoted ``supports_vision: "false"`` would otherwise silently enable native routing on a model that can't handle it.N))rMboolintr _BOOL_TOKENSgetstriplowerrXs&r_coerce_capability_boolrdZsi #t #s6MtCy3t34>sC4H4H<  CIIK--/ 0RdRr&c V^8dQhRRRR/#)rrXrr!dict[str, Any]r#)r$s"rr%r%fs0000r&c6\V\4'dV#/#r*)rMrNrcs&r_dict_or_emptyrhfsS$''3/R/r&c V^8dQhRRRR/#rrXrr!r r#)r$s"rr%r%js""C"C"r&cH\T;'gR4P4#)r rarcs&r _clean_strrnjs syyb>   !!r&c V^8dQhRRRR/#)rkeyr r!rr#)r$s"rr%r%nsssr&cl^RIHpV!V4pVR8XdV#\V4# \dR#i;i)zNContext-local credential source or stripped runtime text; "" when unavailable._runtime_main_valueapi_keyrm)agent.auxiliary_clientrsrn Exception)rprsvalues& r _runtime_mainrxns=>#C(y(u?j.?? s$ $ 33c$V^8dQhRRRRRR/#)r models_cfgrmodelr r!rYr#)r$s"rr%r%ys&^^^C^N^r&c\\V4PV44p\VPRVPR444#)zNPer-model ``supports_vision`` (or ``vision`` alias) from a ``models`` mapping.supports_visionvision)rhr`rd)rzr{ per_models&& r_model_supports_vision_overriderys=~j9==eDEI "9==1BIMMRZD[#\ ]]r&c$V^8dQhRRRRRR/#)rcfgrfnamesr"r!zIterable[dict[str, Any]]r#)r$s"rr%r%s'WW.WWKcWr&c#ra"\V4pRV4FoV3RlV4RjxL K R#L 5i)zYield legacy ``custom_providers`` entries matching ``names`` (case-insensitive); ``names`` is the outer loop so list order cannot let a persisted default shadow the live route.c3\"TF"qP4P4xK$ R#5ir*)rarb)rns& rr+_custom_provider_entries..s4e779??$$es*,c3<"TF6p\VPR44P4S8XgK2VxK8 R#5i)nameN)rnr`rb)rrwanteds& rrrs0Vw!*QUU6]*C*I*I*Kv*UAAws /A AN)_custom_provider_list)rrentriesrs&& @r_custom_provider_entriesrs2$C(G4e4VwVVV5Vs '75 7c V^8dQhRRRR/#)rrrfr!zlist[dict[str, Any]]r#)r$s"rr%r%sTT~T2FTr&cVPR4p\V\4'd+VUu.uFp\V\4'gKVNK up#.#uupi)zSDict entries of the legacy ``custom_providers`` list (empty when absent/malformed).custom_providers)r`rMrSrN)rrXrs& rrrsF ''$ %C6@d6K6Ks 2s!jD1AAs 2SQSS 2s A Arequested_providerrmc ,V^8dQhRRRRRRRRRR/#) rrOptional[dict[str, Any]]providerr r{rr!rYr#)r$s"rr%r%sA u u ! u u  u  u  ur&c Na\V\4'gR#\VPR44p\ VPR44pVeV#.p\ RW1\ VPR4434F`pVPV4VPR4'gK-V\R4R'gKCVPV\R4R4Kb \\PV44p\VPR44pVU u.uF,p \VPV 44PR4NK. p p T \W4U u.uFqPR4NK up , p \RV3R lV 44R4#uup iuup i) uResolve user-declared vision capability from config.yaml; None when unset. First hit wins: ``model.supports_vision`` → ``providers.

.models.`` → legacy ``custom_providers[].models.``. Named custom providers are rewritten to ``provider="custom"`` at runtime while config keeps the user's name under ``model.provider``, so the requested, runtime and config identities are all tried, plus the bare ```` of any ``custom:``. Nr{r}rcustom: providersmodelsc30"TF qfKVxK R#5ir*r#rvs& rr,_supports_vision_override..sm[q[s c3<<"TFp\VS4xK R#5ir*)r)rr-r{s& rrrs[PZ1`` → ``providers..`` → ``custom_providers[].``, ```` covering the provider and ``model.provider`` in both bare and ``custom:``-prefixed forms.rt)materialize_probe_api_keyrmr{Nrr:rrc38"TFq'gK VxK R#5ir*r#rs& rr+_resolve_inference_value..sKEqErHc3X<"TFp\VPS44xK! R#5ir*)rnr`)rrrps& rrrs!EWZc 33Ws'*)rxagent.command_token_sourcercallablerMrNrhr`rnsetraddrbrsplitrPrr)rrrprruntimerrrwrr6rloweredrrrs&&f$ r_resolve_inference_valuersC G iH G  ,W5 5+G4:g&& c4 swww/0I y}}S) *E  eE D8Z j0I%JK L !   (<(-_resolve_inference_base_url..s%]\\i=;T;Z;Z;\.\\r&r)rnrbr)rrrs&&@r_resolve_inference_base_urlrs-8$**,I # z\ r&c$V^8dQhRRRRRR/#rr#)r$s"rr%r%s(YY$<YYPSYr&c \WRRR7#)aBest-effort API key, resolved like :func:`_resolve_inference_base_url` so it matches the base URL actually probed; otherwise the local server-type probe hits a keyed remote endpoint without Authorization and sprays 401s on every image turn. Mirrors :func:`_resolve_inference_base_url`'s resolution order (runtime value, then ``model.api_key``, then the providers blocks) so the key matches the base URL actually being probed. See #89863. rtcR#)Tr#)rs&rr,_resolve_inference_api_key..sSWr&r)r)rrs&&r_resolve_inference_api_keyrs $C9 XXr&c(V^8dQhRRRRRRRR/#)rrr rrtr!r]r#)r$s"rr%r%s)#sTXr&c\V4P4R8XdR#V'gR#^RIHpHp\ V!V44;'d V!WR7R8H# \ dR#i;i)aTrue when the active provider likely fronts a local Ollama server. Fingerprint probing is only valid for LOCAL endpoints: remote OpenAI-compatible APIs (sglang, vLLM) expose Ollama-compat routes that can misidentify, and probing them without an api_key returns 401 on every leg.ollamaTF)detect_local_server_typeis_local_endpointrt)rnrbagent.model_metadatarrr]rv)rrrtrrs&&& r_should_probe_ollama_visionrsd (!!#x/ T%h/0tt5Mh5hlt5tt sA A A('A(c V^8dQhRRRR/#rjr#)r$s"rr%r%s44c4c4r&c\V\4'dVP4P4MRpV\9dV#R#)zHNormalize a config value into one of the valid modes (default ``auto``).rmauto)rMr rarb _VALID_MODES)rXmodes& r _coerce_moders6",S#"6"6399;   BD<'43V3r&c V^8dQhRRRR/#)rrrr!r]r#)r$s"rr%r%s  '? D r&c\\\V4PR44PR44p\V4;'d\VPR44P 4R9;'dG\VPR44'*;'d \VPR44'*'*#)uTrue when the user configured a specific ``auxiliary.vision`` backend — the de-facto image route in ``auto`` mode even when the main model has native vision. ``auto``/empty provider with no model and no base_url is not explicit. auxiliaryr~rr{r>rmr)rhr`r]rnrb)rr~s& r_explicit_aux_vision_overridersN>#+>+B+B;+OPTTU]^ _F <  6::j)*002lB 3 36::g./ / 3 36::j12 2r&c(V^8dQhRRRRRRRR/#rrr r{rrr!rYr#)r$s"rr%r%s0EESEE;SEXfEr&cn^RIHpHpT!T\W 4;'gR4pV'd V!V4#R#)aManaged local runtime verdict: the server receiving the image is the authority on whether it can see (its /props reports modalities). Cloud catalogs have never heard of a local GGUF, so without this every local model reads as text-only and screenshots detour to a cloud auxiliary.)is_managed_providermanaged_model_supports_visionrmN)%hermes_cli.local_runtime.capabilitiesrrr)rr{rrrmanageds&&& r_probe_managed_runtimers6 i!(,G,V,\,\Z\]G3: ( /DDr&c(V^8dQhRRRRRRRR/#rr#)r$s"rr%r%s*:::C:6N:Sa:r&c^RIHpT;'gRP4P4R8Xd^RIHpV!V4pV!WRR7pVfR#VP #)umodels.dev catalog verdict. ``allow_network=True`` on purpose: this runs only when an image needs routing, and the text-only-main guard depends on catalog data — a cold cache returning "unknown" would reintroduce attempting the call. The fetch is cached (4h TTL) and backoff-limited.)get_model_capabilitiesrmz openai-codex)"strip_codex_context_variant_suffixT) allow_networkN)agent.models_devrrarbrrr})rr{rrrcapss&&& r_probe_models_devrsZ 8 B%%'>9 L259 !( FD<49T%9%99r&c(V^8dQhRRRRRRRR/#rr#)r$s"rr%r%4s/ J JC J J2J J~ Jr&c\W 4pV'g!\V4P4R8XdRp\W 4p\ WVR7'gR#^RIHpV!WVR7#)z[Ollama ``/api/show`` verdict for local endpoints (see :func:`_should_probe_ollama_vision`).rzhttp://localhost:11434/v1rN)query_ollama_supports_vision)rrnrbrrrr)rr{rrrtrs&&& r _probe_ollamar4sS*39H  8,224@.(7G &x7 K KA ' IIr&zmanaged-runtime caps lookupz caps lookupzollama vision probez5tuple[tuple[str, Callable[..., Optional[bool]]], ...]_VISION_PROBESc ,V^8dQhRRRRRRRRRR/#) rrr r{rrrr!rYr#)r$s"rr%r%Js:*** * "*  *  *r&c V'gZ\R4P4\V4P48Xd%\R4\V48Xd \R4p\W WR7pVeV#V'd V'gR#\FwrVV!WV4pTfKTu# R# \ d#p\ PRYPY4Rp?KERp?ii;i)uWReturn True/False if vision capability can be resolved, None if unknown. Order: config ``supports_vision`` override → :data:`_VISION_PROBES` (managed local runtime → models.dev catalog → Ollama probe). A plugin's per-model ``ProviderProfile.model_capabilities`` reaches this through the catalog probe. The provider-wide ``ProviderProfile.supports_vision`` flag is deliberately NOT consulted: it declares that tool-result messages may carry images (``tools.vision_tools``), not that every model on the provider can see user attachments — relays such as ``router`` set it for mixed catalogs. rr{r)rNu)image_routing: %s failed for %s:%s — %s)rxrbrnrrrvloggerdebug) rr{rroverridelabelprobeverdictexcs &&&$ r_lookup_supports_visionrJs,  * % + + -H1E1K1K1M M ' "j&7 7*+?@(eH 5&  HS1G  N'    LLDeW\ b  s B** C5CCc ,V^8dQhRRRRRRRRRR/#)rrr r{rrrr!r#)r$s"rr%r%wsAbbb b "b  b  br&c\\\V4PR44PR44pVR8wdV#\V4'dR#V'dRV/M/p\ WV3/VBRJdR#R#)zReturn ``"native"`` or ``"text"`` for the given turn (``cfg`` None behaves as auto; ``requested_provider`` is the identity before runtime canonicalization).agentimage_input_moderrrTnative)rrhr`rr)rr{rrmode_cfgextras&&&$ rdecide_image_input_moderws{N>#+>+B+B7+KLPPQcdeH6$S)):L !#5 6RTE.xMuMQUU8a[aar&z image/avifz image/heic image/png image/jpeg image/gif image/webp image/bmpc#6"TFwrR^V33V3xK R#5i)N)r sftypr#)rbrandmimes& rrrs"Q.s%Ij(#s3SX~&#-js&)FT:NiNLPLARGBr)PILr ImportErrorrinforrv pillow_heifregister_heif_opener pillow_avifopenrrconvertsavegetvalue)rXrr'r)imbufrs& r_transcode_to_pngr0s )  ((*  )     ZZ % %ww==ZZ')C GGCG 6<<> & %   =      & % %  RTWXs_CDD!"D?AD+ D? DD D  D( + D< 6D?<D?? E+ E&&E+c$V^8dQhRRRRRR/#)rr=rrXrr!r r#)r$s"rr%r%s&@@d@@C@r&c"Ve \V4MRpT;'g'\P!\V44^,pV'dVP R4'dV#\ P VPP4R4#)uYImage MIME for *path*: magic bytes (authoritative) → ``mimetypes`` → suffix → jpeg.Nzimage/r) r mimetypes guess_typer r _SUFFIX_MIMESr`suffixrb)r=rXsniffedr s&& r _guess_mimer8sj-0_$S)$G  8 8i**3t95a8D ))   T[[..0, ??r&cV^8dQhRR/#)rr! frozensetr#)r$s"rr%r%s ( ( (r&c^RIHp^RIHpHpT!\ V!R4;'gR4\ V!R4;'gR44'dV#\# \ d \#i;i)uProvider-accepted MIME set for the current main runtime. The managed local server decodes fewer formats (no WebP — and a WebP part fails SILENTLY: the model confabulates a description), so its narrower set transcodes those here.rr)ACCEPTED_IMAGE_MIMESrrrmr)rursrr<rr rv_UNIVERSALLY_SUPPORTED_MIMES)rsr<rs r_accepted_mimesr>sp >c s#6z#B#H#HbI3ObcmOnOtOtrtKu v v' ' w ('   '' s"#AAAA A-,A-c V^8dQhRRRR/#)rr=rr!r:r#)r$s"rr%r%sIIDI]Ir&cJ^RIHpV!\V44TP4p\YR7pT\49dL\T4;pf\P RY4R#\PRTPT4TRrCR T R \P!T4PR 4 2# \d"p\P RY4Rp?R#Rp?i\ dLi;i \ d"p\P RY4Rp?R#Rp?ii;i) u2Encode a local image as a base64 data URL at native size (the agent retry loop shrinks on rejection, so lenient providers pay no silent quality tax); MIMEs outside the accepted set are transcoded to PNG. None when unreadable, blocked by the read guard, or untranscodable — the caller reports ``skipped``.)raise_if_read_blockedz6image_routing: blocked local image attachment %s -- %sNu'image_routing: failed to read %s — %srczimage_routing: %s is %s which is not accepted by the active provider and could not be transcoded to PNG; skipping this attachment.zIimage_routing: transcoded %s (%s) -> image/png for provider compatibilityrzdata:z;base64,ascii)agent.file_safetyrAr ValueErrorrwarningrv read_bytesr8r>r0r&rbase64 b64encodedecode)r=rArrXr  transcodeds& r_file_to_data_urlrKs  ;c$i( oo t %D ?$$+C0 0J 9 NNPQU  _aeajajlpq T 4&!1!1#!6!=!=g!F G HH) OQU[     @$Ls:B;C6; C3C"" C3/C32C36 D"DD"c(V^8dQhRRRRRRRR/#) r user_textr image_pathsz list[str] image_urlszOptional[list[str]]r!z&tuple[list[dict[str, Any]], list[str]]r#)r$s"rr%r%s6LLLL$L, Lr&c .p.pVFp\V4pVP4'd"VP4'd \V4MRpV'dVP VRV R234KhVP \ V44K TRT;'g.4Uu.uFq'gK VRV R23NK up, pT;'gRP 4p V'gV 'd RRRV /.V3#.V3#T ;'gR R 2R PR V44,p VUU u.uF wrRR R RV//NK p pp RRRV /.V OV3#uupiuup pi)uBuild an OpenAI-style ``content`` list for a user turn. Local paths become base64 ``data:`` URLs; remote URLs pass through verbatim. When any image attaches, one text part combines the caption (or a neutral default) with a ``[Image attached at: ]`` / ``[Image attached: ]`` hint per image — a string handle for tools taking an image path/URL, mirroring ``Runner._enrich_message_with_vision``. Returns ``(content_parts, skipped)``; ``skipped`` holds unreadable local paths (URLs are never skipped). Nz[Image attached at: ]c3T"TFq;'gRP4xK R#5i)rmN)rarKs& rr-build_native_content_parts..,s 8eTdq''r9J9JTds ((z[Image attached: rmtyperzWhat do you see in this image?z  c3*"TF wrVxK R#5ir*r#)rrhs& rrrS1sQi`hX\XYRS`hs image_urlurl)rexistsis_filerKrr rajoin) rMrNrOskippedattachedraw_pathr6data_urlrLr combined_textr image_partss &&& rbuild_native_content_partsrcseG&(H N+,88::!))++$Q'4  OOX)=hZq'IJ K NN3x= )   8eT^TdTdbdTd8ek8e1ij.!(1-.8ekkH OO " " $D 48&&&$/07JJb7JJ???@E Qi`hQiHiiMNVWhdaFKuajAhKW VV] 3 Bk BG KKl Xs# E1 E.E>rrr) r rrrrrz.tiffz.tifz.heic)trueyeson1)falsenor0rlr*)savifsavis)sheicsheixshevcshevxsmif1smsf1sheimsheis))))r[sPNG  r)))r[sr)))r[sGIF87ar)))r[sGIF89ar)))r[sRIFF)sWEBPr)))r[sBMr)))r[sII* image/tiff)))r[sMM*rl)))r[sz image/x-icon>rrrr)rcrrV)L__conditional_annotations____doc__ __future__rrGloggingr3r<re contextlibriorpathlibrtypingrrrr r r r getLogger__name__rr:r _IMAGE_EXTSr\_IMAGE_EXT_PATTERNcompile IGNORECASErQrRDOTALLr3r7rCrVrNrOr_rdrhrnrxrrrrrrrrrrrrrr__annotations__rr _FTYP_BRANDSitemsr rr=r0r5r8r>rKrc__all__)rms@rrs #  GGG   8 $34 [ XXA[AA zzI  MM   !$669OOQSQ^Q^ **1299=rzz,?WXi  ) mm.5 mm/7 S0"^ WT u ! uF&QRY$4 E:. J #$:;%&M*IE* ! **Zb ! b8 mm& 5 mm\^jk  A RL