+ iHjK Rt^RIt^RIHt^RIHtHtHt^RIH t ]!RR7!RR44t R R lt R R lt RBR Rllt RCRRlltRBRRlltRRltRDRRlltRRlt]!0REm4t]!R04t]!R04tRRltRRltR t]!0RFm4tR!R"ltR#R$R%R$/R&R'lltR(RR)R/R*R+lltR,R-ltR.R/ltR0R1lt R2R3lt!R4R5lt"R6R7lt#R8R9R:RR;RRR?llt$RGR@RAllt%R#)HuJAnthropic prompt caching strategy — pure functions, no AIAgent dependency. Default layout: 4 cache_control breakpoints — the static system prefix, the end of the system prompt, and the last 2 non-system messages (without a static prefix: one system breakpoint plus the last 3 messages). All markers share one TTL (5m or 1h). N) dataclass)AnyDictList)find_stable_prefixT)frozenc0a]tRt^toRtV3RltRtVtR#)PromptCachePlanzARequest-local message and tool sections with their cache markers.c~<V^8dQh/S[S[S[S[3,,;R&S[S[S[S[3,,;R&#)messagestoolslistdictstrr)format __classdict__s"9/home/ubuntu/.hermes/hermes-agent/agent/prompt_caching.py __annotate__PromptCachePlan.__annotate__s94S>""  S#X  N)__name__ __module__ __qualname____firstlineno____doc____annotate_func____static_attributes____classdictcell__)rs@rr r sKrr cXV^8dQhR\R,R\R,R\/#)r providerNbase_urlreturnrbool)rs"rrrs0 S St SsUYz S^b Src^RIHpT!T;'gRP4P4T;'gR4'*#)aYWhether the envelope-layout route honors part-level markers on role:tool. OpenRouter/Nous Portal relocate a part-level ``cache_control`` onto the ``tool_result`` block; LiteLLM-style proxies copy parts verbatim, so it lands at ``tool_result.content[0]`` (non-retryable 400). There, role:tool carries no part markers and the budget reallocates. )_is_litellm_route)agent.agent_runtime_helpersr(striplower)r"r#r(s&& r*envelope_tool_part_cache_markers_supportedr-s5> (..b!7!7!9!?!?!A8>>rR RRrcV^8dQhR\\\\3,,R,R\\\\3,,R,R\/#)r r Nr r$rrrrr&)rs"rrr#sO [ [$tCH~"6"= [d4PSUXPX>FZ]aFa [fj [rcV'dR#T;'g.Uu.uFp\V\4'gKVNK pp\;QJdRV4F 'gK RM RM !RV44'dR#VUu.uFq"PR4NK ppVP R4^8*;'d:\;QJdRV4F 'gK RM RM !RV44'*#uupiuupi)aTrue for a request no later call can read back from cache: no tools, no assistant/tool turns, at most one user turn (compression summaries, titles, MoA synthesis). A marker there only buys a cache write (1.25x input at 5m, 2x at 1h) that nothing ever reads. A caller that already decorated the request (a MoA advisor's first iteration, whose prefix later iterations replay) decided it will reuse it, so that decision wins over the shape.Fc3p"TF,pRV9;'g\VPR44xK. R#5i) cache_controlcontentN)_has_part_markerget).0ms& r &is_one_shot_request..,s/ Tt!?a  E E#3AEE)4D#E Ets6#6Troleuserc3*"TF qR9xK R#5i) assistantN)r=toolr)r6rs& rr8r9/s/ZTYq5J0JTYs) isinstanceranyr5count)r r r7msgsroless&& ris_one_shot_requestrE#s  ~~2~ =~!At) *sC.C.>C3cJV^8dQhR\R\R,R\/#)r text cache_markerNr$rr)rs"rrr2s%St trc"RRRV/pVeWR&V#)typerGr2r)rGrHparts&& r _text_partrM2s#&&$/D ,_ KrFc LV^8dQhR\R\R\R\RR/#)r msgrHnative_anthropictool_part_markersr$Nrr&)rs"rrr9s144T444+/4;?4rcVPRR4pVPR4pVR8XdV'g V'gR#VR8Xd V'g VeVR8XdVR9d V'dWR&R#R#\V\4'd%\W4;'g \ WQ4.VR&R#\V\ 4'd7V'd-\VR,\ 4'dWR,R&R#R#R#R#) zFAdd cache_control to a single message, handling all format variations.r:r)r3r>Nr2)r>r=)r5r@r_scaffold_partsrMrr)rOrHrPrQr:r3s&&&& r_apply_cache_markerrV9s 7762 Dggi G v~.7H +7b=--6F#/ 7G GS ! !);bb 7@a?bI GT " "w:gbk43P3P'3 O$4Qw "rcXV^8dQhR\R\R,R\R,/#)r rOrHNr$)rr)rs"rrrPs.mmmTD[mD4KmrcVPR4pVPR4R8Xd"\V\4'd \V4MRpVfR#\ W14\ V\ V4R4.#)z``[scaffold, tail]`` parts of a string user message with a registered scaffold, the marker (if any) on the scaffold; ``find_stable_prefix`` keeps the tail non-blank.r3r:r;N)r5r@rrrMlen)rOrHr3prefixs&& rrUrUPskggi G,/GGFOv,E*U\^aJbJb  (hlF>4l 6(H*U\]`ag]h]iUjJk'llrc`V^8dQhR\\\\3,,RR/#)r r r$Nr)rs"rrrXs' 4 4T#s(^(< 4 4rc\V4F<wr\V\4'gK\V4;p'gK2/VCRV/CW&K> R#)a Send every registered scaffold message as [scaffold, tail] parts, marked or not (in place). Provider caches key on content, and two text parts are different content from one string: a skill message split while it held a breakpoint must keep the same parts after the rolling marker window moves past it, or the next request re-writes the whole scaffold (#81867). Each hit refreshes the scaffold's registry LRU slot, so a scaffold still in history stays registered and the failover stripper can still recognise its split. r3N) enumerater@rrU)r irOpartss& r_split_unmarked_scaffoldsr`XsGH% c4 s/C&Ce&C&C3S3)U3HK&rcHV^8dQhR\R\R\R\/#)r rOrPrQr$rR)rs"rrrfs/ B B4 B4 BD B\` BrcV'dR#VPR4R8Xd V'gR#VPR4p\V\4'd V'd\VR,\4#\V\4;'dVR8g#)aPTrue if a marker on this message is actually honored by the provider. Native Anthropic honors every message; the envelope layout only honors markers inside content parts (empty content wastes a breakpoint) and ``tool_part_markers=False`` excludes role:tool too (400). Must agree with :func:`_apply_cache_marker` (marks the LAST part). Tr:r>Fr3r)rT)r5r@rrr)rOrPrQr3s&&& r_can_carry_markerrcfs wwv& ):ggi G,6w,E,E':gbk4 (BWabiknWoXBXBt{@BuBBrcRV^8dQhR\R\\\3,/#)r ttlr$rI)rs"rrrus&XXsXtCH~Xrc$VR8XdRRRR/#RR/#)zCBuild a cache_control marker dict for the given TTL ('5m' or '1h').1hrK ephemeralrer)res&r _build_markerrius!14FK -W6;BWWr opencode-goz minimax-m2.5c0V^8dQhR\R\/#r modelr$r)rs"rrrs<V^8dQhR\R,R\/#)r sourceNr$rn)rs"rrrs"UUcDjUSUrclT;'gRP4P4\9dR#R#)zThe tier ``auto`` resolves to for a session source: ``5m`` for machine-paced sources, ``1h`` for everything a human types into (cli, tui, desktop, messaging platforms).r)5mrg)r+r,MACHINE_PACED_SOURCES)rzs&rauto_cache_ttl_for_sourcer~s0LLb'')//15JJ4TPTTrrmr)r"cVV^8dQhR\R,R\R\R\/#)r reNrmr"r$rn)rs"rrrs3 ` `S4Z `3 `s `TW `rcVR8wd T;'gR#T;'gRP4pV\9d\V4\9dR#R#\ V4'g V\ 9dR#R#)uClamp a requested cache TTL to what the destination route supports (``None`` → ``5m``). Qwen/Alibaba routes drop ``1h`` (→ ``5m``) except on ``MEASURED_1H_PROVIDERS`` minus ``NO_1H_TIER_MODELS``; that check runs BEFORE the generic Qwen clamp, which would swallow it. rgr|r))r,MEASURED_1H_PROVIDERSrsNO_1H_TIER_MODELSrwALIBABA_FAMILY_PROVIDERS)rermr"provider_lowers&$$ reffective_cache_ttlrsj  d{{{dnn"++-N.."5)->>tHDH ''>=U+U4_[__r mark_suffixfallback_to_wholeczV^8dQhR\R\R\R,R\R\R\R\/#) r messagerHstatic_system_prefixNrPrrr$)rrr&int)rs"rrrsJ   !% =@4Z  )- IM   rcVPR4p\V\4'dV'd\V\4'dlVPV4'dUV\ V4RpVP 4'd0\ W!4\ Yt'dTMR4.VR&V'd^#^#M V'g^#\WVR7^#)uMark the static system prefix (and optionally the full prompt); returns markers applied. The stored system prompt stays one string, split only in the request. ``mark_suffix=False`` is the tool-cache-plan layout (suffix budget spent on the tools array); ``fallback_to_whole= False`` marks nothing when the split is impossible. When the prompt IS the prefix the whole message is one block — never an empty text block (400). r3NrP)r5r@r startswithrYr+rMrV)rrHrrPrrr3suffixs&&&$$$ r_apply_system_cache_markersrskk)$G&,,1E*U\^aJbJbgngygy{OhPhP1234 <<>>",-A"P",V[\VZ"["]GI #1 * * @PQ rc0V^8dQhR\R\/#)r r3r$)rr&)rs"rrrsuucudurc\V\4;'d5\;QJdRV4F 'gK R# R#!RV44#)c3\"TF"p\V\4;'dRV9xK$ R#5ir2Nr@rr6rLs& rr8#_has_part_marker..s+,tlsdhZd-C-_-_[_H_-_lss,,TF)r@rrA)r3s&rr4r4s< gt $ t t,tls,ttt,tls,t)ttrcV^8dQhR\\\\3,,R\\\\3,,/#)r api_messagesr$r)rs"rrrs7&&T#s(^0D&dSVX[S[nI]&rc xVEF%p\V\4'gKVPRR4VPR4p\V\4'gKWVPR4pVR8H;'dK\ V4^8H;'d5\ ;QJdRV4F 'dK RM RM !RV44pT;'d!RV^,9;'d RV^,9p\V4'deVUUUu.uFOp\V\4'd5RV9d.VP4UUu/uFwrxVR8wgKWxbK uppMTNKQ uppp;q!R&T;'d5\ ;QJdR V4F 'dK RM RM !R V44p V 'gEKR PR V44p T;'g'T;'d\V 4V^,R ,8Hp \ V4^8Xg#VR 8Xd\ V4^8Xg V 'gEK!WR&EK( V#uuppiuupppi)aRemove ``cache_control`` markers and undo decoration-produced list shapes (in place). Used before re-decorating after a mid-turn failover. Flattening to a string is restricted to the exact shapes :func:`apply_anthropic_cache_control` produces from string content (single text part, two-part system split, two-part skill split) so the ``""``-join is byte-exact. Marker removal is copy-on-write on part dicts: parts can alias caller-held lists and stripping must never rewrite the stored transcript. r2Nr3r:r;c3B"TFp\V\4xK R#5iNr)r6ps& rr80strip_anthropic_cache_control..s@f^eYZAtATAT^esFTc3"TFp\V\4;'dcVPRR4R8H;'dF\VPR4\4;'d\ VP 44RR08*xK R#5i)rKrGN)r@rr5rsetkeysrs& rr8rs+ gn_c tT " [ [txx'?6'I [ [488F+S1 [ [69$))+6F6SYJZ6Z [gnsB B &B "'B r)c32"TF qR,xK R#5i)rGNrrs& rr8rs:'$f'srGsystem) r@rpopr5rrYallr4itemsjoinr) rrOr3r: user_pairfirst_part_markedrLkvplain_text_partsjoined skill_splits & rstrip_anthropic_cache_controlrs#t$$  &'')$'4(( wwvFNffs7|q'8ffSS@f^e@fSSS@f^e@f=f &mm/WQZ*GmmOcjklcmLm G $ $]d(\cTX:dD))o.E#'**,G,$!!2F,GKOP\c( G)n#  ss+ gn+ sss+ gn+ (  :'::(kkI,j,j:LV:TX_`aXbciXj:j w<1 !1c'la6GKK# N9: H(s5H5 8 H/H/ H5 /H5 cV^8dQhR\\\\3,,R\\\\3,,/#)r r r$r)rs"rrrs4  d4S>&: tDcN?S rc\V4Fiwr\V\4'gKRV9g(\VPR4\4'gKK\ \V4.4^,W&Kk V#)aRe-canonicalise planner output before re-planning (in place on the list, copy-on-write per message). Every list-content message is stripped, not just marked ones: an UNMARKED skill split (outside the previous request's marker window) left as two parts would get the new breakpoint on its volatile tail instead of the scaffold. A shallow copy suffices because the strip is copy-on-write on content parts. r2r3)r]r@rr5rr)r r^rOs& r_strip_prior_decorationrs^H% c4 o&< 377S\K]_c@d@d7c DQGHK& OrcV^8dQhR\\\\3,,R,R\\\\3,,/#)r r Nr$r)rs"rrrs>d4S>.BT.IdSWX[]`X`SaNbrc\P!T;'g.4pVF-p\V\4'gKVP RR4K/ V#)zBReturn copied tools without request-local Anthropic cache markers.r2N)copydeepcopyr@rr)r cleanedr>s& r"strip_anthropic_tool_cache_controlrsBmmEKKR(G dD ! ! HH_d + NrcV^8dQhR\\\\3,,R\\\\3,,R\/#)r r r r$)rrrrr)rs"rrrsApp4S#X#7pT#s(^@TpY\prc VUUu.uFSp\V\4'gK\VPR4\4'gKBVR,Fq3NK KU ppp\ R.VOVOVO44#uuppi)z=Count the wire-visible cache markers in a request-local plan.r3c3h"TF(p\V\4'gKRV9gK$^xK* R#5irr)r6items& rr8'_count_cache_markers..s(o8TJtT6J"ae"jnorjs"rcaRV3Rllp.p^pV\S48EdZSV,p\V\4'dVPR4R8Xd V^, pKQVPR4pVR8XdpVPR4'dYV!V^,4pWt^,8d9\ SV^, ,V4'dVP V^, 4TpKVR8Xd V!V4pKVR8H;'dV^,\S48;'g"VR8H;'dVPR 4R 9pV'g#\ WQ4'dVP V4V^, pEKjV#) z