+ lHj&Rt^RIt^RIt^RIHtHtHt]P!R4tRt Rt RRut t RR lt R R ltR R ltRRltRRltRRltRRltRtRRltRRltRRltR#)aTruncate-and-store pipeline for web_extract (no LLM). Pages at or under the char budget are returned whole; larger pages become a head+tail window plus a footer that says how much is shown, where the full text is stored (cache/web) and the exact read_file call that pages the omitted middle. Inline base64 images become ``[IMAGE: alt]`` placeholders. Logs under the origin (tools.web_tools) logger. N)AnyListOptionalztools.web_toolsi:iii c0V^8dQhR\R\/#valuereturnrint)formats"=/home/ubuntu/.hermes/hermes-agent/tools/web_tools_truncate.py __annotate__rsHHSHSHcP\\\\V4\44#)zClamp to [2k, 500k] (below 2k the truncation footer dominates; a config typo must not blow up context); raises TypeError/ValueError for non-numeric input.)max_CHAR_LIMIT_FLOORminr _CHAR_LIMIT_CEILINGrs&r _clamp_char_limitrs  #c%j2E"F GGrc0V^8dQhR\R\/#rr )r s"r rr!s**S*S*rchVf\#\V4# \\3d \u#i;i)zV``_clamp_char_limit(value)``; ``None`` or non-numeric input falls back to the default.)DEFAULT_EXTRACT_CHAR_LIMITr TypeError ValueErrorrs&r _clamp_or_defaultr!s8*-2])X@QRW@XX z "*))*s  11c$V^8dQhR\/#)rr )r )r s"r rr)sKKKrcL^RIHp\V!4PR44#)zE``web.extract_char_limit`` clamped to a sane range, else the default._load_web_configextract_char_limit)tools.web_toolsr rgetrs r _get_extract_char_limitr$)s 0 -/334HI JJrc0V^8dQhR\R\/#rtextr str)r s"r rr/s N N N NrcRRlp\P!RW4p\P!RRV4p\P!RRV4#)a#Replace inline base64 image blobs (token bombs) with ``[IMAGE: alt]`` placeholders: markdown images (alt kept), parenthesised blobs, and bare ``data:image/...;base64,`` payloads. Real http(s) markdown image links are left untouched so the agent can ``web_extract`` / ``vision_analyze`` them.cZV^8dQhR\P\,R\/#)rmr )reMatchr))r s"r r4convert_base64_images_to_links..__annotate__3s&[[BHHSM[c[rctVPR4;'gRP4;p'dRV R2#R#)altz[IMAGE: ][IMAGE])groupstrip)r,r1s& r _md_repl0convert_base64_images_to_links.._md_repl3s:-.WWU^-A-Ar,H,H,J%JS%J#a ZQZZrzD!\[(?P[^\]]*)\]\(\s*data:image/[^;]+;base64,[A-Za-z0-9+/=\s]+\)z0\(\s*data:image/[^;]+;base64,[A-Za-z0-9+/=\s]+\)r4z'data:image/[^;]+;base64,[A-Za-z0-9+/=]+)r-sub)r'r7outs& r convert_base64_images_to_linksr;/sD[ &&XZb iC &&DiQT UC 66rKr@rA cache_dirpathrHexcs && r _store_full_textr[;s35"; < t4jo.aszz'?R0S0]0]0_`c0d/eehii w</ /44545J14MN'l1%%WYG < TETJ4y  GRsCC C7C22C7c jV^8dQhR\R\R\R\\\3,/#)rr>r= char_limitr )r)r tuplebool)r s"r rrVs2*=*=3*=S*=c*=eCQUIFV*=rc \V4V8:dVR3#\VR,4pW#, pVRVW)RreVPR4;qsR,8dVRVp^VPR4;pu;8:dVR,8dMM Wg^,Rp\ W4pV'd^RIHp V !V4pRRR\V4R R \V4R R \V4R R 2.p V'd/VPR4^,p V R V 2RV RV R2., p MV PR4V PR4VR,V,p V R,RPV 4,R3#)aReturn (model_text, was_truncated). Pages over ``char_limit`` become a ~75% head / ~25% tail window cut on line boundaries, plus a footer saying how much is shown, where the full text is stored, and the read_file call that pages the omitted middle. Deterministic.Fg?N g?)to_agent_visible_cache_pathr2zShowing rGz chars (head) + z chars (tail) of z total clean characters.zFull text saved to: z,To read the omitted middle: read_file path="z " offset=zS limit=200 (the file is the complete page; raise/lower offset to page through it).zwFull text could not be stored; re-run web_extract on a more specific URL or use browser_navigate for the complete page.u+ [... middle omitted — see footer ...] Tu=──────── [TRUNCATED] ────────uW─────────────────────────────) rRr rfindfindr[tools.credential_filesrbcountappendjoin) r>r=r] head_budget tail_budgetheadtailnl stored_pathrb footer_linesmiddle_start_line model_texts &&& r _truncate_with_footerrrVs 7|z!~j4'(K*K+& (>$jj"33CRy499T? "R7kC&77FG}"30K G1+>  3 3t9Q-/D !}='l1 5 7L  JJt,q0";- 0:;-H'()6 7    J  #IIDPJ  tyy6 6 <VP V4'gKWVu# R#)aMagic-byte type name when a fetched body is a raw binary file, else ``""``. Backends return the body as text with NUL bytes dropped, so signatures are compared NUL-stripped on both sides. Only the file tools' own signature table; HTML/markdown/JSON never start with one.)_MAGIC_SIGNATURES:N Nzlatin-1ignorerr2)tools.file_operationsrxrPreplace startswith_UNAMBIGUOUS_BINARY_KINDS)r'rxrkprefixnamesigs& r _binary_payload_kindrss8 9  Ix 0 8 8# FD) nnWc* 34??#<==$//RUBVBVK* rcVV^8dQhR\\,R\R\RR/#)rresultsr]debug_call_datar N)listdictr )r s"r rrs3AAtDzAsATAVZArc VEF]pVPRR4pVPRR4;'gVPRR4pVPR4'g V'gKc\V4pV'd1RVR&RV R2VR&\PRWF\ V44K\ V4p\ WtV4wrWR&V 'dsVR ;;,^, uu&VR ,PRVR \ V4R \ V4/4\PR V\ V4\ V44EK=\PRV\ V44EK` R#)zIn place: replace each successful entry's content with its base64-cleaned, budgeted text; per-page truncation metrics go into ``debug_call_data``.r=r2 raw_contentr>errorzURL returned binary content (z), not a page. Download it with the terminal (curl -L -o) and use read_file (SQLite/Office/PDF auto-extract) or terminal utilities on the file.z)%s (binary payload: %s, %d chars dropped)pages_truncatedtruncation_metrics original_size sent_sizez%s (truncated %d -> %d chars)z%s (%d chars, whole)N)r#rrVinforRr;rrrg) rr]rresultr=r binary_kindcleanrq truncateds &&& r _truncate_resultsrs:jj#jj3PPvzz)R7P ::g  k *;7 !#F9 / }=uu 7O KKCSWZ[fWg h .{; 5e* M &y  - .! 3 . 0 1 8 8_c%j+s:W  KK7c%j#j/ Z KK.SZ @3rc\V^8dQhR\\,R\\,/#)rrr )rr)r s"r rrs"  4: $t* rc VUu.uF`pRVPRR4RVPRR4RVPRR4RVPR4/RV9d RVR,/M/CNKb up#uupi)zOKeep only url/title/content/error per entry (+ blocked_by_policy when present).r=r2titler>rblocked_by_policy)r#)rrs& r _trim_resultsrs   A 155#WaeeGR.@)QUUS\^`Ma QUU7^ ATWX@X#Q':%;<^`    sA&A.)SQLiteZIPgzipbzip2xzz7-ZipELFzMach-OPNGJPEGGIFTIFFFLACOgg)__doc__loggingr-typingrrr getLoggerrVrrSrrrrr$r;r[rrrurrrrrr rs &&   , - #")-w&&H *K N6*=Zb L A> r