+ lHjaRt0tRt^RIt^RIt^RIt^RIHt^RIHt^RI t ^RI t ^RI t ^RI H t ^RIHtHtHtHtHtHt^RIHt^RIt^RIHtRs]^kRs]^kRR lt^R IHt^R IH t ^R I!H"t"^R I#H$t%H&t'H(t)H*t+H,t-H.t/^RI0H1t1H2t2H3t3H4t4H5t5H6t6H7t7] Pp!]94t:] !RRR7t;RR/RRllt4t?]?fRt?Rt@RRltARRltB]B!4tC]!]CRR7tDR tER!R"ltFR#R$ltGR%tHR&R'ltIR(R)ltJR*R+ltKR,tLR-R.ltMRR/R0lltNRR1R2lltORtPR3tQRtRRtSR4R5ltTR6R7ltUR8tVR]RRRR93R:R;lltW]X!0Rm4tY]X!0Rm4tZRR<R=llt[R>R?lt\R@RAlt]RBRClt^RDRElt_RRFRGllt`RHRIlta!RJRK]b4tc!RLRM]4tdRNROlteRPRQltfRRRSltgRRTRUllthRVR/RWRXlltiRYRZltjRtk]kR[3RR3tl]k]l^,^,3R]SR^33tmR_R`]l3RaRb]m3/tnRcRdltoRReRflltpRgRhltqRRiRjlltrRkRlltsRmRnltt^RoIuHvtvHwtwRpRqRrRsRtRuRvRwR\RuRxRrRy/RzRuRxRrR{/R|RuR}R~RuR/R^R^RrR//RR\Rz.//txRRltyRRltz]vP!RqR]x]z]tRRR7RRRRRRRRRRRRRR/t|Rt}Rt~RRltRRltRRRlltRRltRRRlltRpRRrRRtRuRvRwR]RuRxRrR/RzRuRxRrR//RR]Rz.//tRRlt]vP!RRa]]]sRRR7R#)a[Vision tools: ``vision_analyze`` (image) and ``video_analyze``. Images resolve through :mod:`tools.image_source`, are normalized to a provider-supported format (:mod:`tools.vision_tools_image_prep`), then either attach natively to a vision-capable main model (multimodal tool-result envelope) or are described by the auxiliary vision LLM router. N)ThreadPoolExecutor)BytesIO)Path)Any AwaitableCallableDict NamedTupleOptional)urlparse) make_user_dircV^8dQhRR/#)returnN)formats"7/home/ubuntu/.hermes/hermes-agent/tools/vision_tools.py __annotate__rsiiic\e \f=^RIHp\;'g VPs\;'g VPsR#R#)N)auxiliary_client)async_call_llmextract_content_or_reasoningagentr)_auxs r_load_auxiliary_clientrs>!=!E2'>>4+>+>'C'h'htGhGh$"Fr)get_hermes_dir) DebugSession)check_website_access)native_turn_duplicate record_embed release_embedrepeat_refusalresolve_repeat_capresolve_embed_target_bytes)&_VISION_MAX_VALIDATED_AGGREGATE_PIXELS!_VISION_MAX_VALIDATED_FRAME_COUNT_crop_image_region_determine_mime_type_image_exceeds_dimension_normalize_to_supported_image _validate_raster_image_decodable vision_toolsVISION_TOOLS_DEBUG)env_vardefaultc$V^8dQhR\/#)rkeysstr)rs"rrr>s#rc^^RIHpHpV!V!4R.VO5RV/# \dTu#i;i)zQ``auxiliary.`` from config.yaml; ``default`` when config is unavailable.)cfg_get load_config auxiliaryr/)hermes_cli.configr5r6 Exception)r/r1r5r6s$* r_cfg_auxiliaryr:>s7:{}kJDJ'JJ s  ,,c0V^8dQhR\R\/#)rr.keyr2)rs"rrrGs N N# NC NrcaaVV3RlpT!\P!VR4P4;'gR4pVeV#V!\RV44#)uEnv var → ``auxiliary.vision.`` → None. Values that fail ``cast`` or fall below ``minimum`` are skipped in favor of the next source (a cap can never be disabled by a bad value).cd<S!V4pSeTS8dT#R# \\3dR#i;iN) TypeError ValueError)rawvalcastminimums& r_accept%_read_vision_setting.._acceptJsA s)CosATA:&  s //Nvision)osgetenvstripr:)r.r<rDrErFrCs&&ff r_read_vision_settingrMGsLB "))GR(..088D 9C/3Mw~h/L'MMrHERMES_VISION_DOWNLOAD_TIMEOUTdownload_timeoutg>@c$V^8dQhR\/#rrint)rs"rrrcs++3+rc\^\\P!^444# \\ 3d-\^\P !4;'g^4u#i;i)zKUsable CPU count (``sched_getaffinity`` honors cpuset pinning), at least 1.)maxlenrJsched_getaffinityAttributeErrorOSError cpu_countrrr_detect_host_cpusr[csS+1c"..q1233 G $+1blln))**+s(+2A(A('A(c$V^8dQhR\/#rQrR)rs"rrrks&&S&rcP\RR\^R7pT;'g \4#)udHERMES_VISION_MAX_CONCURRENCY → ``auxiliary.vision.max_concurrency`` → host cores (< 1 ignored).HERMES_VISION_MAX_CONCURRENCYmax_concurrency)rE)rMrSr[)rCs r_resolve_vision_cpu_workersr`ks( >@QSV`a bC  % %#%%rz vision-encode) max_workersthread_name_prefixc"^RIp\P!4pVP\VP !V.VO5/VB4GRjxL #L5i)zZRun a sync encode/resize callable on the bounded vision CPU executor (never the LLM call).N) functoolsasyncioget_running_looprun_in_executor_vision_cpu_executorpartial)fnargskwargsrdloops&*, r_run_encode_on_cpu_executorrnxsG  # # %D%%&:IPARTRR7hi;ii;iRRR4GRjxL M +GRjxL 'giM;iRRR4GRjxL M +GRjxL 'giM;iTu# \8dpTp Y^, 8gT'gS\CT4'gB\>PERY;^,\#T4R,RR7T'ghRp?EK^T ^,,p\>PGRY;^,T\#T4R,4T'g\>PGRT4\HPJ!T4GRjxL Rp?EKRp?ii;i5i)uSSRF-safe streaming download with exponential backoff (2s/4s/8s). ``retry_all=False`` (images) retries only :func:`_is_retryable_download_error` errors — a 404/403 never succeeds on retry.)atomic_replace)create_ssrf_safe_async_clientTrNmessager)rfollow_redirects event_hooksGETz User-AgentAccept)headerszcontent-lengthz too large (z bytes, max ).z.tmpr missing_okz%Could not delete partial download: %sexc_infoz*%s download failed after %s attempt(s): %sNdNz&%s download failed (attempt %s/%s): %s:N2NzRetrying in %ss... _download_z3 exited retry loop without attempting (max_retries=)(utilsrrzrparentrrangerrrstream_DOWNLOAD_USER_AGENTraise_for_statusrgetrSrAr3rp with_namenameuuiduuid4hexr aiter_bytesrVrr9unlinkrYrrrr~warningresleep RuntimeErrorlower)rprrrrrrrrr last_errorattemptblockedclientrcl declared_sizetmp_destination bytes_writtenfchunke wait_times&&&$$$$$ r_download_mediars %>TD9J%> +*3/G%gi&8995$ZRfQgDhs\3GSY$Z'))+%%))*:;)/1CGtM!,1J$ }L|\e[ffg%hii.s8<A I (P!2L"5L8 I K I K II I6  I)$I6 5K 6 J? J J? #J;8J? :J;;J? ?K  L K LK-K K-&K-(L0 L";K><L"L L  L L  L"P!" P-PA PP!A6P P PP!PP!cHV^8dQhR\R\R\R\/#)r image_urlrrr)r3rrS)rs"rrrSs/aaSata#aVZarc R"\WVRR\\RR7GRjxL #L5i)zCDownload an image with SSRF protection and error-class-aware retry.Imagezimage/*,*/*;q=0.8FrrrrrN)r_VISION_MAX_DOWNLOAD_BYTES_VISION_DOWNLOAD_TIMEOUT)rrrs&&&r_download_imagerSs4  $7,6NZ_a aa as '%'cRV^8dQhR\R\\,R\/#)r image_path mime_typerrr r3)rs"rrr[s+]]$]8C=]TW]rcT;'g \V4pRV R\P!VP44P R4 2#)zK``data:;base64,...`` for a file (MIME from extension when not given).data:;base64,ascii)r(base64 b64encode read_bytesdecode)rrmimes&& r_image_to_base64_data_urlr[sI  8 8,Z8D 4&!1!1*2G2G2I!J!Q!QRY!Z [ \\ri c0V^8dQhR\R\/#r}r)rs"rrrxsaa adarca\V4P4o\;QJd*V3Rl\R,4F 'gK R# R#!V3Rl\R,44#)z;Detect if an API error is related to image or payload size.c3,<"TF qS9xK R#5ir?r.0hinterr_strs& r '_is_image_size_error..{s`+_4w+_TF)rinvalid_request)r3rany_SIZE_ERROR_HINTS)r~r s&@r_is_image_size_errorrxsL%j G 3`+.__annotate__sVV#Vrc2<S;'g\SSR7#)r5)r)data_urlrrsr_raw&_resize_image_for_vision.._rawsUU4Z9UUru;Pillow not installed — cannot auto-resize oversized imagezcImage file is %.1f MB (estimated base64 %.1f MB, limit %.1f MB, max_dimension=%s), auto-resizing... image/pngz)Pillow cannot open image for resizing: %sRGBc4V^8dQhR\R\RR/#)rwhrNrR)rs"rrr7s&kkkkkrcl<Se/W3S8wd%SPS^,S^,WR7R#R#R#)N)rrrr)update)r?r@ orig_dimsr2s&&r _record_scale/_resize_image_for_vision.._record_scales8  aVy%8    ! )A,Z[  j&9 rg?zResized to %dx%d (attempt %d)rqualityrrrz4Auto-resized image fits: %.1f MB (quality=%s, %dx%d)z=Auto-resize could not fit image under %.1f MB (best: %.1f MB))PNGr<)JPEGz image/jpeg>Lr=r?)UFr)statst_sizer)rrVr.rinfor(rr9rconvertwidthheightrrUrSresizeLANCZOSrsaverrgetvaluerr)rrr0r1r2r3 file_size estimated_b64r:ris_png pil_formatout_mimeimgr quality_steps prev_dims candidaterDrnew_wnew_hdims_okqbufr9rCsff&&f& @@r_resize_image_for_visionres(!))I]q(3.MH(!&>z=&Y&Y,Z9M x=, ,OVV % 'E } QRv  KKu[)=K+H K0-A;;/ ; K ^ ^T^P^F39/?UJjj$ chhl2kk% %GSUW:X5E{syy1}C b399n =>C"aC R#**_ =>Cu~***eU^U]];CI KK7w O4'VV3syy#**+E+VA)C HHS U UaiiQR^ Uz&2B2B3<<>2R2Y2YZa2b1cdI9~!11gg R Nk:Asyy#**Vcii4  .v  NNR#{3S^{5SU#))SZZ( U  ?Ev s P11 Q#<QQ#Q#c<V^8dQhR\R\R\/#rprovidermodelrrq)rs"rrr s!  # c 4 rcD^RIHpV!W4# \dR#i;i)uvHard veto: the provider's ``ProviderProfile`` declares ``supports_vision_tool_messages=False`` — images are accepted in user messages but list-type tool-result content is rejected with 400 (xiaomi/MiMo "text is not set"). ``supports_vision`` alone must not override this, or the multimodal tool-result envelope 400s every turn and the image never enters context (#89981). ))routed_model_rejects_vision_tool_messagesF) providersrkr9)rhrirks&& r_profile_rejects_tool_mediarm s'G8II s  c<V^8dQhR\R\R\/#rgrq)rs"rrrs!c#$rc:a\V\4'dVP4P4MRpV'd\ W!4'dR#V\ 9dR#V\ 9dq\V\4'dVP4P4MRo\;QJdV3RlR4F 'gK R# R#!V3RlR44#^RIH pV!V4pVRJ;'d\VP4# \dR#i;i)aWhether provider+model accepts image content inside a tool-result message. Unknown providers are False (caller falls back to aux-LLM text) unless their ``ProviderProfile`` declares ``supports_vision``; ``supports_vision_tool_messages=False`` is a hard veto.rHFTc3,<"TF qS9xK R#5ir?r)rtagms& rr 2_supports_media_in_tool_results..'sV'U!8'Ur )get_provider_profileN)zgemini-3z gemini-pro-3zgemini-flash-3) rwr3rLrrm_TOOL_RESULT_MEDIA_PROVIDERS_GEMINI_PROVIDERSrrlrtrrsupports_visionr9)rhriprtprofilerrs&& @r_supports_media_in_tool_resultsrzs%/x$=$= 2A +A55 (( %/s%;%;EKKM   !sV'UVssVsVsV'UVVV2&q)d"DDtG,C,C'DD sD 5D DDc V^8dQhR\R\R\\\\3,,R\/#rrhricfgrr3r rrrr)rs"rrr0s= A A# Ac AcSVh@X A]a Arcp\W4'dR#\W4'dR#^RIHpV!WV4RJ#)uOne gate for both native lanes — the ``vision_analyze`` fast path and the ``computer_use`` capture route (#115248): the profile's ``supports_vision_tool_messages=False`` veto first, then either the provider's tool results are known to carry media or the per-model capability lookup (config override → catalog incl. the profile's ``model_capabilities`` → local probes) attests the model as vision-capable.FT)_lookup_supports_vision)rmrzagent.image_routingr)rhrir}rs&&& r_accepts_tool_result_imagesr0s5 #833&x77; "8C 8D @@rc V^8dQhR\R\R\\\\3,,R\/#r|r~)rs"rrr=s={{{S{xSRUX?W{\`{rcN^RIHpV!WV4R8H;'d \WV4#)aTHE gate for every tool that can hand the main model pixels (``vision_analyze``, browser and ``computer_use`` screenshots, MCP ``ImageContent``): image routing resolves to ``native`` (``agent.image_input_mode``, an explicit ``auxiliary.vision`` backend, the catalog) AND the route accepts images inside tool results. One predicate, so the lane never depends on which tool produced the image.)decide_image_input_modenative)rrr)rhrir}rs&&& r_native_tool_result_imagesr=s+ < "8C 8H D z zIdemvyIzzrc$V^8dQhR\/#rQr)rs"rrrGsTrc^RIHpHp^RIHp\ V!4V!4V!44# \ d"p\PRT4Rp?R#Rp?ii;i)uT:func:`_native_tool_result_images` for the active main model; any failure → False.)_read_main_provider_read_main_model)r6z(Native vision fast-path check failed: %sNF) agent.auxiliary_clientrrr8r6rr9rr)rrr6rs r#_should_use_native_vision_fast_pathrGsJP1)*=*?AQASU`Ubcc  ?Es), AAAcV^8dQhR\R\R\R\R\\,R\\\3,/#)rrquestionimage_data_urlimage_size_bytes scale_noter)r3rSr rr)rs"rrrRsRggg!g36gJMg g #s(^grc Rp\V\4'd1VP4'dVRVP4 2, pV'd VRV 2, pRRRRRRV/RR R R V//.R R VR , R R2RR VR,RVRR//#)zMultimodal tool-result envelope. The text part is intentionally minimal (the model already has the question); ``text_summary`` is the fallback for providers without multimodal tool results.ulImage loaded into your context — you can see it natively now. Use your built-in vision to answer the user.z Question: z Note: _multimodalTcontenttypetextrrp text_summaryz,Image attached natively for the main model (.1fz# KB). Answer using built-in vision.metaNN size_bytes native_vision)rwr3rL)rrrrr text_parts&&&&& r _build_native_vision_tool_resultrRs 7(C  X^^%5%5~hnn&6%788 z*.. t VVY / [+~/F GI :;Kd;RSV:WX, ,io|=M`de ggrc>V^8dQhR\\,RR/#)rrrN)r r)rs"rrrjs(4.Trc\VeVPRR7R#R# \dR#i;i)NTr)rr9)rs&r_unlink_quietlyrjs4   KK4K (   s  ++c]tRtRtRtRtR#)_ImagePrepErrorirz=Raised by :func:`_prepare_image`; the message is user-facing.rN)__name__ __module__ __qualname____firstlineno____doc____static_attributes__rrrrrrsGrrc0a]tRtRtoRtV3RltRtVtR#)_PreparedImageivzHTemp image ready to encode; ``path`` is owned by the caller (delete it).cZ<V^8dQh/S[;R&S[S[,;R&S[;R&S[;R&#)rrrrr)rr r3rSr)r __classdict__s"rr_PreparedImage.__annotate__vs9 J 3-O   rrN)rrrrr__annotate_func__r__classdictcell__)rs@rrrvsRrrc V^8dQhR\R\\,R\\,R\R\/#)rrtask_idregionvalidate_decoder)r3r listrrr)rs"rrr~s<(?(?(?%c](?46 6D   0$ FFF$,MM3x}}3ErkD070A0AB_ae0l*l'x .!("K"K.KL L  " D !"D,,J !<0$13Y"[[L%l33  9@9J9J"D[:J4J 0L,</%h&G&G2GHH D !%1<ARARATA\A\ D $j >>?Z 1c#h'S01G+m[ 4J  s H GGGAH 9G;:&H !H?G> H!;HH:H; H/H6H7HH/H H GG8G33G88H >HHHHH c0V^8dQhR\R\/#)rrrr2)rs"rrrsOOsOsOrcNR\V4R, R R\R, R R2#)z2Image too large for vision API: base64 payload is r MB (limit .0fzs MB) even after resizing. Run `hermes pm repair` to restore Pillow for auto-resize, or compress the image manually.rG)rV_MAX_BASE64_BYTES)rs&r_too_large_messagers= ->15AA+I>N!i(G"$q(H)-HH)+2   mU;D H 9 %   HMM * I 7+IX\]]H ; %x}}  OO 9:6 $hmm5IKK ~ !3 3z#3!3CWdh$jjN>"%66!"4^"DeT H 9 %   HMM *  ) $/>%00()<)<h>R>R>Z>ZVZ[] H 9 %   HMM * G^ 7c#h6 6< H 9 %   HMM * E 7OKj I;SA23%8%HH H 9 %   HMM * I H 9 %   HMM * s,M6AM60K M6&M6III)K K2K 8K9K K +K,-K M6!M6)&M6 K ?K  K $M64&M6I J=(J8=J=>K M6&M68J==K K K L9%L49L9:L<>M6&M64L99L<<M3 &M33M6 min_timeoutc V^8dQhR\R\\,R\R\\,R\/#)rmessagesridefault_timeoutrr)rr r3rr)rs"rrrs933t3HSM3E3"*5/3=A3rcRTRrT\R/R7;'g/pVPR4e,\\VR,4Vf \R4MT4pVPR4e\VR,4pRRRTRTRT/T'dR T/C#/C# \dL&i;i) z``async_call_llm`` kwargs with ``auxiliary.vision.timeout`` / ``.temperature`` from config. Local vision models (llama.cpp, ollama) can take well over 30s, hence generous defaults (temperature 0.1); ``min_timeout`` lets video enforce a floor.g?rI)r/rz-inf temperaturetaskrri)r:rrUrr9)rrirrrr _vision_cfgs&&&$ r_aux_call_kwargsrs +C[ $Xr:@@b ??9 % 1% I 67+J]vcnoG ??= ) 5 M :;K Hj(M;PY[b 3#( 33.0 33    sA:B B&%B&cHV^8dQhR\R\R\R\/#)r user_prompt part_typer9r)r3r)rs"rrrs.ddddddrc &RRRRRRV/RWRV//./.#)zJSingle user message: text + one ``image_url``/``video_url`` data-URL part.roleuserrrrrpr)rrr9s&&&r_media_messagesrs< VY - uV^N_/`)b c ddrzkInsufficient credits or payment required. Please top up your API provider account and try again. Error: {e}r video_urlzbThe video is too large for the API. Try compressing or trimming the video (max ~50 MB). Error: {e}imagevision_analyze_toolvideovideo_analyze_toolcHV^8dQhR\R\R\R\/#)rkindsourcerr)r3r)rs"rrr-s.mm3mm#mmrc x\V4^8dVR,R,MTpRV R2VRVRV/RRR R R ^R W0 R 2^/#)rrz... parameters_urlrrir~NrFanalysis_length model_used _size_bytes)rV)rrrriprompts&&&& r_debug_call_datar-sa*-k*:S*@[  &kF$t}fmVWeTy%):A|UV\\gThjk mmrc0V^8dQhR\R\/#)r call_kwargs empty_logrr3)rs"rrr4s    rc"\4Vf\R/VBGRjxL p\V4pV'g2\P V4\\R/VBGRjxL 4pV#LJL 5i)zAux vision LLM analysis text, retrying once on empty content (reasoning-only response). ``response``: an already-made first call (the image path handles size-error retries itself).Nr)rrrrr)rrranalysiss&&& r_call_vision_llmr4sa'6+66+H5H y!/n6S{6S0ST O 71Ts!A.A*?A.A, A.,A.c V^8dQhR\R\R\R\\,R\\\\.\ \ ,3,R\/#)rrrrristager)r3r rrrrtuple)rs"rrrAs^4h4h 4h4h),4h5=c]4h S$%y'77 84h=@4hrc aaa"\V,wop\V\4'gVe \V4MRp\WW#4o.pRVV3Rllp^RIHpV!4'd`\ RRR7VFLp V 'gK V P4'gK%V P4\PR V4KN #\PR WR,4\PRVR,4V!VSV4GRjxL wrV 'd \V 4M^p \PRVP4V 4T ;'gRV R2p RR RV 'd RV RV 2MT /pV 'dWR&SP!R V R7V!V4VFLp V 'gK V P4'gK%V P4\PR V4KN # \d%p \PR T R R 7Rp ? EKRp ? ii;iEL \d$p \PR T R R 7Rp ? KRp ? ii;i \EdpRT RT: 2p\P#RTR R 7\T4P%4o\'T3RlT4RT R24pTP)YR7p TSR &T!RRR TRT /4uRp?TF}p T 'gK T P4'gK%T P4\PR T4KN \d$p \PR T R R 7Rp ? KwRp ? ii;i #Rp?ii;i TF}p T 'gK T P4'gK%T P4\PR T4KN \d$p \PR T R R 7Rp ? KwRp ? ii;i i;i5i)!udAux-LLM analysis skeleton for image/video: interrupt check → ``stage`` → JSON result. ``stage(user_prompt, debug_call_data, temp_paths)`` returns ``(analysis, scale_note)``; temp files it appends to ``temp_paths`` are deleted in ``finally``. Returns JSON ``{"success": bool, "analysis": str}`` (``analysis`` carries the error explanation on failure). NrHc0V^8dQhR\R\/#)rresultrr)rs"rr#_run_analysis..__annotate__Ps@@t@@rc<\PSS4\P4\P!V^RR7#)rF)indent ensure_ascii)_debuglog_callrUjsondumps)rdebug_call_data tool_names&rfinish_run_analysis..finishPs- ?3 zz&??rrrFrzCleaned up temporary %s filez#Could not delete temporary file: %sTrzAnalyzing %s: %s:N<NzUser prompt: %srz%%s analysis completed (%s characters)z-There was a problem with the request and the z could not be analyzed.rr[z] r)rr zError analyzing z: z%sc3<"TFKwr\;QJdV3RlV4F 'gK RM RM!V3RlV44'gKGVxKM R#5i)c3,<"TF qS9xK R#5ir?rrs& rr *_run_analysis...is/REDEr TFN)r)rhintstplr s& rr  _run_analysis..is2 S5ZUCC/RE/RCCC/RE/R,RSS5s&AA Az" could not be analyzed. Error: {e})rrir~)_ANALYSIS_KINDSrwr3rrrrexistsrrrr9rrOrV capitalizerBr~rnextr)rrrrirrules temp_pathsr'rr cleanup_errorrrr rr error_msgtemplater%r r&s&&&&& @@@r _run_analysisr:AsX't,Iu k3 ' '*5*Ac+&r &t[HOJ@@ h2   mU;.Dt hKKMLL!?F -  &Sk: %{4'89%*;%TT+3#h- ;T__=NP_`ll!NtfTklT:z:,b /K_gh #-< t_Mf~Dt hKKMLL!?F  !hNN#H-bfNggh3 U2!hNN#H-bfNggh!  T&tfBqe4  T9t 4a&,,. S5 S;D6Ae fh??Q?4#, y%)ZRSSDt hKKMLL!?F hNN#H-bfNggh  TDt hKKMLL!?F hNN#H-bfNggh s;AO2I5 O2O2&G/O2AIH!AII+I>I O2+O2&H$*O2/ H:HO2HO2!I$ I/I O2 IO2 M'!A8M"M'M* O2-O2&L.,O2. M9MO2MO2"M''M** O/:O/&N;9O/; O) O$ O/$O) )O//O2c V^8dQhR\R\R\R,R\\,R\\,R\/#)rrrriNrrr)r3r r)rs"rrrxsP&N&N&N!$&N-04Z&N c]&N+3D>&NEH&Nrc\aaaa"RVVVV3Rllp\RSVSV4GRjxL #L5i)zDescribe an image (URL, local path, data: URL) with the auxiliary vision LLM. ``user_prompt`` is pre-formatted by the caller. Temp images live under $HERMES_HOME/cache/vision/.cHV^8dQhR\R\R\R\/#rrr%r6rr3rrr)rs"rr)vision_analyze_tool..__annotate__}s. ^ ^C ^$ ^D ^U ^rc<"\S SS RR7GRjxL pVPVP4\P RVP R, 4\P R4\ \VPVPR7GRjxL p\P R\V4R, 4/p\V4\8d<\W54GRjxL p\V4\8d\\V44hVP VR&\VR V4p\P R 4\VS R 4p\!4\#R/VBGRjxL p\+VRV4GRjxL p T \-T;'gRVP.;'gR43#ELELLLM \$dp \'T 4'd\T4\(8gh\P R \T4R, \(R, 4\Y54GRjxL pYF^,R ,^,R ,R&\#R/TBGRjxL pRp ? LRp ? ii;iL5i)FrNzImage ready (%.1f KB)rzConverting image to base64...r5z#Image converted to base64 (%.1f KB)rrz%Processing image with vision model...g^@zYAPI rejected image (%.1f MB, likely too large); auto-resizing to ~%.0f MB and retrying...rrpz0Vision LLM returned empty content, retrying oncerrG)rrrrrOrrnrrrVrrrArrrrrr9r_RESIZE_TARGET_BYTESrr(r)rr%r6rrrrrr_api_errrrrirrs&&& rr"vision_analyze_tool..stage}s' 7FTYZZ(--( +X-@-@4-GH 34: %x}}  OO 93~;NQU;UV ~ !2 2#3H#JJN>"%66 !3N!CDD.6.A.A*+"6;G ;<&x>   ;+:k::H* KXWW*;+>+>$@T@T@\@\X\]]]?[ O K; ;(22s>7JMa7a KKkN#{35I[5Y [$4H#JJJN.F</F>3I8I6I8I8, I87I8:I8<F>> I3 A&I./H203I.#I&$I.)I8.I33I8rNr:)rrrirrrsf&fff rrrxs. ^ ^Bw ;uM MM Ms ,*,c$V^8dQhR\/#rQr)rs"rrrs  $ rca^RIHpHoV!4;_uu_4\;QJd#V3Rl/RR/34F 'gK RM RM!V3Rl/RR/344uuRRR4# +'giR#;i)uNTrue when ``call_llm(task="vision")`` could resolve a client. Mirrors its fallback chain: explicit ``auxiliary.vision.provider``, then auto (main provider → openrouter → nous) — without the auto step the tool would vanish whenever the explicit name was unresolvable. Probe mode skips real SDK client construction. See #31179. aux_probe_moderesolve_vision_provider_clientc3F<"TFpS!R/VB^,RJxK R#5i)Nrr)rkwrJs& rr +check_video_requirements..s( JdB * 0R 0 34 ?Jds!rhautoTFN)rrIrJrrHs @rcheck_video_requirementsrPseV   s KMPZ\bOcJd sss KMPZ\bOcJd      s$A&A&& A7 c$V^8dQhR\/#rQr)rs"rrrsOO4Orc:\4;'g \4#)unImage gate (``vision_analyze``, ``browser_vision``): an aux vision client OR the native fast path. Both handlers attach pixels straight to a vision-capable main model, so a main model on a provider the aux resolver does not know (OAuth, local vLLM) must not hide a working tool (#47149). ``video_analyze`` keeps the aux-only gate — its handler has no native path. )rrPrrrcheck_vision_requirementsrSs / 0 N N4L4NNr)registryrrvision_analyze descriptionuLoad an image into the conversation so you can see it. Call it any time the user references an image — then answer from what you see.r robject propertiesstringz>Image URL (http/https), local file path, or data: URL to load.rz)Your question or request about the image.rarrayitemsintegerminItemsmaxItemsuOptional [x1, y1, x2, y2] crop in ORIGINAL-image pixel coordinates, applied before any downscaling — the crop keeps full resolution. Load the full image first, then re-call with a region to zoom into small text or fine detail.requiredcRV^8dQhR\R\R\\,/#)rsectionsenv_varsr)rr r3)rs"rrrs* V VE VU Vx} VrcVFSp\VR4pV'gK\V4P4'd\V4P4u#M \RRV44R4#)z~First non-empty ``auxiliary.
.model`` from config.yaml, else the first non-empty env var (legacy override), else None.ric38"TFq'gK VxK R#5ir?r)rvs& rr (_configured_aux_model..sNHqAHs  c3l"TF*p\P!VR4P4xK, R#5i)rHN)rJrKrL)rrs& rr rfs'Hx!RYYq"-3355xs24N)r:r3rLr4)rarbsection_vmodels&& r_configured_aux_modelrjsd '2 77|!!##7|))++   NHxHNPT UUrc^V^8dQhR\\\3,R\R\/#rrkrMr)rr3r)rs"rrrs.ddtCH~dSdSdrc^"VPRR4VPRR4VPR4rCpVPR4p\4'd+\PR4\ W#WTR7GRjxL #RV 2p\ R R 4p\ W&WuVR7GRjxL #L+L5i) rrHrrrz vision_analyze: native fast path)rrNz]Fully describe and explain everything about this image, then answer the following question: )rI)AUXILIARY_VISION_MODEL)rrrrOrrjr)rkrMrrrr full_promptris&, r_handle_vision_analyzerps"&((;";TXXjRT=UW[W_W_`hWiIffYG+,, 67+I``` ""* - "+/J KE$YU\bc cca ds$A;B-=B)>&B-$B+%B-+B-rITu👁️)rtoolsetschemahandlercheck_fnis_asyncemoji.mp4 video/mp4z.webmz video/webmz.movz video/movz.aviz.mkvz.mpegz video/mpegz.mpgcFV^8dQhR\R\\,/#)r video_pathrr)rs"rrr s<<<#k llrcRV^8dQhR\R\\,R\/#)rrzrrr)rs"rrr)s+]]$]8C=]TW]rcT;'g\V4;'gRpRV R\P!VP44P R4 2#)rxrrr)r~rrrr)rzrrs&& r_video_to_base64_data_urlr)sQ  J J/ ; J J{D 4&!1!1*2G2G2I!J!Q!QRY!Z [ \\rc^V^8dQhR\R\\,R\R\/#)rrrr6r)r3r rr)rs"rrr.s41d1d1dhsm1dQU1dZ^1drc "^RIHpHpHpHpVP R4p\ \PPV44pT;'gRP4P4p \V 4;'dV PR4'*p V!4'gV 'd\PRV4\ V4P P4p V \"9d\%\'V 44hV!W!VR7RR7GRjxL p \)R R 4pTR \*P,!4 T 2, p\/YP04TP3T4T#VP54'd0^R IHpV!\;V44\PR V4V#\=V4GRjxL 'd\?V4pV'd\AVR,4h\)R R 4pVR\*P,!4 R2, pVP3V4\CW^RR\DRRR7GRjxL V#\%R4hELE Tdp \%RT 24T hRp ? ii;iLL45i)uvLocal video path for a terminal-backend path, local file, or HTTP(S) URL. Only files created here are appended to ``temp_paths`` — never user-provided paths. Terminal-backend reads use the shared media resolver with ``permitted=("video",)`` — the exact pipeline vision_analyze uses (media-cache host reads, bounded in-sandbox exec-read, credential-read guard, 50MB cap).)rr_is_local_terminal_backendrzfile://rHz-Reading video source via terminal backend: %sr) permittedNz,Could not read video from terminal backend: z cache/videotemp_video_filesterminal_video_)raise_if_read_blockedzUsing local video file: %sr temp_video_rwVideozvideo/*,*/*;q=0.8gN@TrzKInvalid video source. Provide an HTTP/HTTPS URL or a valid local file path.)rurvr)r)#rrrrr removeprefixrrJr expanduserrLrrrrxrrOr}r|rArrrrrrris_fileagent.file_safetyrr3r{rrr_MAX_VIDEO_BASE64_BYTES)rrr6rrrrr local_pathlowered path_liker}rrrrrrs&&& r_materialize_videor.s5  # #I .Fbgg((01JB%%'--/GW ZZg&8&89Y&Z"ZI % ' 'I CYOf$$**, * *6v>? ? \1>':jRRH%]4FGODJJL>&BBT==1$ ;c*o. 0)< &y 111&y1 !')"45 5 %]4FGK ~T::$ QG*I99I>>JJc xV^8dQhR\R\R\R,R\\,R\/#)rrrriNrr)r3r )rs"rrrbsDNNN!$N-04ZNIQRUNbeNrcXaaa"RVVV3Rllp\RSVSV4GRjxL #L5i)zEAnalyze a video via multimodal LLM. Returns JSON {success, analysis}.cHV^8dQhR\R\R\R\/#r>r?)rs"rr(video_analyze_tool..__annotate__es(C$DUrcd<"\S S V4GRjxL pVP4PpVR, p\P RV4\ V4pV'g\ \VP44hV\8d\PRV4\W6R7p\V4\8d0\ R\V4R, R R\R, R R24hWAR &\VR V4p\VS R R R 7p \!V R 4GRjxL p V R3#ELL 5i)NzVideo ready (%.1f MB)u,Video is %.1f MB — may be slow or rejectedr5z+Video too large for API: base64 payload is rrrz+ MB). Compress or trim the video and retry.video_size_bytesrgf@)rz#Empty video response, retrying oncerG)rrMrNrrOr~rArr}_VIDEO_SIZE_WARN_BYTESrrrVrrrr)rr%r6temp_video_pathr video_size_mb detected_mimevideo_data_urlrrrrirrs&&& rr!video_analyze_tool..stagees) 29gz RR*//199(K8  +];/@ 67M7MNO O 4 4 NNI= Y2?\ ~ !8 8=c.>QU`>abe=fg1[A#FG89: :/?*+"6;G&x5Q )+7\]]~'S$^s"D0D+D D0"D.# D0.D0rNrE)rrrirrsf&ff rrrbs+*w ;uM MM Ms *(* video_analyzeu?Analyze a video from a URL or local file path using a multimodal AI model. Sends the video to a video-capable model (e.g. Gemini) for understanding. Use this for video files — for images, use vision_analyze instead. Supports mp4, webm, mov, avi, mkv, mpeg formats. Note: large videos (>20 MB) may be slow; max ~50 MB.z5Video URL (http/https) or local file path to analyze.zpYour specific question about the video. The AI will describe what happens in the video and answer your question.ctV^8dQhR\\\3,R\R\\,/#rl)rr3rr)rs"rrrs2XXS#XXcXinXrc VPRR4VPRR4r2RV 2p\RR4p\W$WQPR4R7#)rrHrzFully describe and explain everything happening in this video, including visual content, motion, audio cues, text overlays, and scene transitions. Then answer the following question: rr)rrI)AUXILIARY_VIDEO_MODELrn)rrjr)rkrMrrroris&, r_handle_video_analyzersZ((;3TXXj"5Mx ??Gj J ""57j kE ieVVIEV WWru🎬chV^8dQh/^\9d \;R&^\9d \;R&#)rrr)__conditional_annotations__r)rs"rrrs0./0)(c(1rr?i )i@iP)z too largepayload413content_too_largerequest_too_largeexceedsz size limit> nousclaudeopenaivertexbedrock anthropic openrouter openai-chat azure-openai openai-codex google-vertexanthropic-directanthropic-vertex>geminigoogle google-geminigoogle-vertex-gemini)rH)NN)402 insufficientzpayment requiredcreditsbilling))does not supportznot support imagecontent_policy multimodalunrecognized request argumentz image inputzY{model} does not support vision or our request was not accepted by the server. Error: {e}))r rzThe vision API rejected the image. This can happen when the image is in an unsupported format, corrupted, or still too large after auto-resize. Try a smaller JPEG/PNG and retry. Error: {e}))rznot support videorrrz video inputrzThe model does not support video analysis or the request was rejected. Ensure you're using a video-capable model (e.g. google/gemini-2.5-flash). Error: {e})NNN)rrrrer#concurrent.futuresriorloggingrJrpathlibrtypingrrrrr r urllib.parser rhermes_dir_aclr rrrhermes_constantsrtools.debug_helpersrtools.website_policyr!tools.vision_tools_history_budgetrrr rr!rr"rr#rr$rtools.vision_tools_image_prepr%r&r'r(r)r*r+ getLoggerrrr!r:rMrrrr[r`_VISION_CPU_WORKERSrhrnr{rrrrrrrrrrrrBrrr(r.re frozensetrurvrmrzrrrrrrArrrrrrrr_BILLING_HINTS_IMAGE_ERROR_RULES_VIDEO_ERROR_RULESr1rrr:rrPrStools.registryrTrVISION_ANALYZE_SCHEMArjrpregisterr|rrr~rrrrVIDEO_ANALYZE_SCHEMArr)rs@rrsW 1 GG! ($((i,,5&&&   8 $ n.B Ct N00PRdfkl##.+& 23)# d(9  9x*ZMm`a]%' a .(*KO6J=A:>16 V| )*  [\ ( A{g0HjHZ(?VO:+z3483$dS67+ '*1-.3 *+  #%7 8 "$6 79 m 4hn&NR $O0   _ J &),AA  . [*-3(V Vd"    " &   K \ K K K \ L+)< m] 1dhN8 O ?  V  R   [*- 4X     ! %  r