+ jHj/vRt^RIt^RIt^RIt^RIt^RIt^RIHtHtH t H t H t H t H t ^RIHt^RIHt^RIHtHtHtHtHtHtHtHtHtHtHtHtH t H!t!^RI"H#t#H$t$]PJ!R4t&RR lt'R t(]PR!R ]PT!](4 R 24t+],!0R3m4t-]-R 0,t.]PR!R4t/]PR!R4t0Rt1Rt2Rt3Rt4RRRR/t5Rt6R4t7RRlt8RRlt9RRlt:RR lt;R!R"ltR'R(lt?R)R*lt@R+R,ltAR-RR.R/R/R0lltB!R1R24tCR#)5zFull-text / trigram / CJK message search and FTS maintenance for SessionDB. Plain mixin for ``hermes_state.SessionDB`` (no ``__init__``/state of its own). Must never import hermes_state (cycle); shared constants live in hermes_state_common. N)AnyCallable CollectionDictListOptionalTuple)describe_skill_invocation)is_malformed_db_error)FTS_CJK_STALE_KEYFTS_SQL FTS_STALE_KEYFTS_STORAGE_VERSIONFTS_TOOL_CONTENT_PREFIX_CHARSFTS_TRIGRAM_EXCLUDED_SOURCESFTS_TRIGRAM_SQLMAX_FTS5_QUERY_CHARSSCHEMA_VERSION_FTS_CJK_TRIGGERS escape_likefts_rebuild_admissionfts_trigram_session_sqlrouted_sessions_setting)fts_layout_currentfts_layout_drift hermes_statec$V^8dQhR\/#return)float)formats"8/home/ubuntu/.hermes/hermes-agent/hermes_state_search.py __annotate__r#sc\RR4pVe\V4P4R8XdR#\V4# \\ 3dR#i;i)zU``sessions.search_slow_ms`` for the served profile (default 1000; 0 logs every call).search_slow_msHERMES_SEARCH_SLOW_MS@@)rstrstripr TypeError ValueError)values r"_search_slow_msr/sS #$46M NE#e**:*:*<*BvTe T z "s#= =AAz+{}():"^@/#&|~[]<>,;!?$=\'[]NEARz "[^"]+"|\S+z"[^"]*"zLm.timestamp, m.tool_name, s.source, s.model, s.started_at AS session_startedzCsubstr(m.content, max(1, instr(m.content, ?) - 40), 120) AS snippetz`(m.content LIKE ? ESCAPE '\' OR m.tool_name LIKE ? ESCAPE '\' OR m.tool_calls LIKE ? ESCAPE '\')z(COALESCE(m.content, '') LIKE ? ESCAPE '\' OR COALESCE(m.tool_name, '') LIKE ? ESCAPE '\' OR COALESCE(m.tool_calls, '') LIKE ? ESCAPE '\')newestzORDER BY m.timestamp DESC, rankoldestzORDER BY m.timestamp ASC, ranka%WITH target AS ( SELECT session_id, timestamp, id FROM messages WHERE id IN ({ids}) ) SELECT t.id AS match_id, m.role, m.content FROM target t JOIN messages m ON m.id IN ( t.id, (SELECT p.id FROM messages p WHERE p.session_id = t.session_id AND (p.timestamp, p.id) < (t.timestamp, t.id) ORDER BY p.timestamp DESC, p.id DESC LIMIT 1), (SELECT n.id FROM messages n WHERE n.session_id = t.session_id AND (n.timestamp, n.id) > (t.timestamp, t.id) ORDER BY n.timestamp, n.id LIMIT 1) ) ORDER BY t.id, m.timestamp, m.idcZV^8dQhR\R\\P,/#)rkeyr)r*rsqlite3Row)r!s"r"r#r#Rs'YYY'++!6Yr$cDVPRV34P4#)z9Point-read one ``state_meta`` row (``None`` when absent).z*SELECT value FROM state_meta WHERE key = ?)executefetchone)connr6s&&r" _meta_rowr=Rs <<Dsf M V V XXr$c(V^8dQhR\RR/#)rkeysrNr*)r!s"r"r#r#Ws__c_d_r$c^VPRRPRV44 R2V4R#)z%DELETE FROM state_meta WHERE key IN (,c3&"TFpRxK R#5i?N.0_s& r" _delete_meta..XsATt!#t)N)r:join)r<r?s&*r" _delete_metarOWs,LL8ATtAT9T8UUVWY]^r$c0V^8dQhR\R\/#)rcpr)intbool)r!s"r"r#r#[s9999r$ca\;QJd#V3Rl\4F 'gK R# R#!V3Rl\44#)c3T<"TFwrVSu;8*;'dV8*MuxK R#5iNrF)rHlohirQs& r"rJ_is_cjk..\s8K&"rR~~2~~Ks%(TF)any _CJK_RANGES)rQsfr"_is_cjkr\[s, 38K83383838K8 88r$cFV^8dQhR\R\\,/#r raw_queryrr*list)r!s"r"r#r#_s"MMCMDIMr$cVP4Uu.uFqP4\9gKVNK! up#uupirV)splitupper_FTS_OPERATORS)r_ts& r"_non_operator_tokensrg_s0 ( L(!GGI^,KAA( LL Ls;;c0V^8dQhR\R\/#r^r@)r!s"r"r#r#csr$cNRPRVP444#)zYQuote each non-operator token (neutralising FTS5 special characters), keeping AND/OR/NOT. c3"TF?qP4\9dTMRVPRR4,R,xKA R#5i)"z""N)rdrereplace)rHtoks& r"rJ$_quote_fts_tokens..es;`qY\yy{n,# C8N2NQT2TT`qsAA )rNrc)r_s&r"_quote_fts_tokensrpcs) 88`i`o`o`q r$cFV^8dQhR\R\\,/#)rtermrr`)r!s"r"r#r#js++s+tCy+r$c0R\V4 R2.^,#)z;One ``%term%`` bind per column of ``_LIKE_ANY_COLUMN_SQL``.%) _escape_like)rrs&r" _like_paramsrvjs T"#1 % & **r$c0V^8dQhR\R\/#r^r@)r!s"r"r#r#os++C+C+r$c(RV9dV#.pVP4FYpVP4\9dVPV4K/TPVP R4;'gT4K[ RP V4;'gT#)uDrop the trailing prefix wildcard callers append for ASCII ("nimb" -> "nimb*"). None of the CJK routes can honour that star: the bigram and trigram routes quote every token before MATCH (so ``*`` matches a literal asterisk) and LIKE has no ``*`` wildcard at all (only ``%``/``_``). Left in place, every CJK search arriving from the web/desktop search box — which appends the star to each unquoted token so partial English words match — searches for a term ending in a literal ``*`` and returns nothing (#90636). Only TRAILING stars go: a star written inside a quoted phrase is the user's own text, and a token that is ALL stars keeps its original form so it cannot degrade to a match-everything empty term. *rj)rcrdreappendrstriprN)r_strippedtokens& r"_strip_cjk_wildcardsr~osx )H" ;;=N * OOE " OOELL-66 7 # 88H  * **r$c0V^8dQhR\R\/#)rdecodedr)rr*)r!s"r"r#r#s773737r$cn\V\4'dVUu.uFEp\V\4'gKVPR4R8XgK3VPRR4NKG ppRP RV44P 4;'gR#\V\ 4'dV#R#uupi)zVMultimodal part list -> joined text (or the placeholder); str passes through; else ''.typetextr(rjc38"TFq'gK VxK R#5irVrFrHrfs& r"rJ _flatten_text..s.5aA5s  z[multimodal content]) isinstanceradictgetrNr+r*)rppartss& r" _flatten_textrs'4  ,3gGqz!T7J"quuU[}`fOf"vr"Ggxx.5..446PP:PP #..76B6hsB2B2B2c4V^8dQhR\R\RR/#)rnamer.rN)r*r)r!s"r"r#r#s!>>>C>D>r$c\V\4'g\V\4'g\V R24hV^8:d\ V R24hR#)z must be an integerz must be greater than zeroN)rrSrRr,r-)rr.s&&r" _positive_intrsM%j&<&<4& 3455 zD6!;<==r$c vV^8dQhR\R\R\\,R\R\R\/#)r snippet_sqlfrom_sqlwhereorder_by limit_sqlrr`)r!s"r"r#r#s=   C  3  tCy  TW  dg  lo  r$c RRV R\ RV RRPV4 RV RV R2 #)zIResult-row SELECT shared by the FTS and LIKE routes (SQL text is pinned).zC SELECT m.id, m.session_id, m.role, z, z FROM zF JOIN sessions s ON s.id = m.session_id WHERE  AND z z )_SEARCH_SELECT_TAILrN)rrrrrs&&&&&r"_search_select_sqlrsX=!'()<<&'( J K   r$after_ts before_tscBV^8dQhR\\,R\R\R\\\,,R\\\,,R\\\,,R\\,R\\,R R / #) rrparamsinclude_inactive source_filterexclude_sources role_filterrrrN)rar*rSrrR)r!s"r"r#r#s&& 9&"&9=&NVW[\_W`Na&d3i(&7?S 7J&sm&08}&IM&r$cV'gVPR4VPR4Ve=VPRRPRV44 R24VPV4Ve=VPRRPR V44 R24VPV4V'd=VPR RPR V44 R24VPV4Ve,VPR 4VP\V44Ve.VPR 4VP\V44R#R#)atAppend the visibility/source/role/session-start predicates every search route shares. Live rows (active=1) AND compaction-archived rows (compacted=1) are discoverable; only rewind/undo rows (active=0, compacted=0) are hidden. ``after_ts``/``before_ts`` bound ``sessions.started_at`` (inclusive / exclusive) inside the query so LIMIT cannot be filled by out-of-window hits.z!(m.active = 1 OR m.compacted = 1)z(COALESCE(m.display_kind, '') <> 'hidden'Nz s.source IN (rBc3&"TFpRxK R#5irDrFrGs& r"rJ)_search_filter_clauses..s-I=ac=rLrMzs.source NOT IN (c3&"TFpRxK R#5irDrFrGs& r"rJrs1O!#rLz m.role IN (c3&"TFpRxK R#5irDrFrGs& r"rJrs+EACrLzs.started_at >= ?zs.started_at < ?)rzrNextendrR)rrrrrrrrs&&$$$$$$r"_search_filter_clausesrs  89 LL;<  }SXX-I=-I%I$J!LM m$" (1O1O)O(PPQRS o& {388+E+E#E"FaHI k" () c(m$ '( c)n%r$ca]tRt^toRtRt]V3RlRl4tV3RlRltV3RlRlt V3RlR lt V3R lR lt R t R t R]!R4 R2tR] R2tR]!R4 2tV3RlRltV3RlRltV3RlRltV3RlRltV3RlRltRR/V3RlR lltV3R!lR"ltV3R#lR$ltV3R%lR&ltV3R'lR(ltV3R)lR*lt]V3R+lR,l4t]V3R-lR.l4t R/R/V3R0lR1llt!V3R2lR3lt"V3R4lR5lt#V3R6lR7lt$V3R8lR9lt%V3R:lR;lt&V3R<lR=lt'R>R?R@RA/V3RBlRCllt(RV3RDlREllt)RV3RFlRGllt*]V3RHlRIl4t+]V3RJlRKl4t,]V3RLlRMl4t-]V3RNlROl4t.]V3RPlRQl4t/]V3RRlRSl4t0]V3RTlRUl4t1V3RVlRWlt2V3RXlRYlt3]V3RZlR[l4t4R\R?R]R?/V3R^lR_llt5V3R`lRalt6]V3RblRcl4t7V3RdlRelt8V3RflRglt9RV3RhlRillt:RV3RjlRkllt;RV3RllRmlltRV3RrlRsllt?V3RtlRult@V3RvlRwltAV3RxlRyltBV3RzlR{ltCR|R?/V3R}lR~lltDRtEVtFR?#)SessionSearchMixinu>See module docstring — mixin for SessionDB (Search cluster).cj<V^8dQhRS[S[S[,,RS[S[S[R3,,/#)rfieldsr.)rrr*tuple)r! __classdict__s"r"r#SessionSearchMixin.__annotate__s: a aHZ_,E a(SXY\^aYaSbJc ar$c aVfR#\V\4'd \R4h\V4oSP VP 4pV'd'\ RRP\V44 24h\;QJd!.V3RlVP 4FNK 5#!V3RlVP 44#)z=Validate and canonically order an optional result projection.Nz?search fields must be a collection of field names, not a stringz unknown search result field(s): , c38<"TFqS9gK VxK R#5irVrF)rHfield requesteds& r"rJ.s`(IuV_M_UU(Is ) rr*r,set difference_SEARCH_MESSAGE_RESULT_FIELDSr-rNsortedr)clsrunknownrs&& @r"_search_message_fields)SessionSearchMixin._search_message_fieldss > fc " "]^ ^K &&s'H'HI ? &QX/@Z?[\] ]u`(I(I`u`u`(I(I```r$c<V^8dQhRR/#rrNrF)r!rs"r"r#rs Q QD Qr$c VP'gR#VPVPR7R# \d"p\P RT4Rp?R#Rp?ii;i)uOne bounded FTS5 merge pass that never fails the already-committed write (a caller must never replay an ambiguous, possibly-durable write — even on the bare SystemError CPython's sqlite3 layer can raise under cross-thread errmsg scrambling).N) max_pagesz-FTS incremental merge failed after commit: %s) _fts_enabled_merge_fts_incrementally_FTS_MERGE_MAX_PAGES_PER_INDEX Exceptionloggerwarning)selfexcs& r"_try_incremental_merge_fts-SessionSearchMixin._try_incremental_merge_ftssV    Q  ) )D4W4W ) X Q NNJC P P  Qs4 A AA cF<V^8dQhRS[S[S[S[3,,/#rrrr*r)r!rs"r"r#rs!33HT#s(^$<3r$c $VPR4#)zDeferred-rebuild progress ``{"pending", "total", "indexed", "percent"}`` or None. Reads via the pooled reader (not get_meta/self._lock) so search never blocks on the writer. fts_rebuild_rebuild_statusrs&r"fts_rebuild_status%SessionSearchMixin.fts_rebuild_statuss##M22r$cF<V^8dQhRS[S[S[S[3,,/#rr)r!rs"r"r#rs!77c3h(@7r$c $VPR4#)z:CJK-index backfill progress, or None when none is pending.fts_cjk_rebuildrrs&r"fts_cjk_rebuild_status)SessionSearchMixin.fts_cjk_rebuild_statuss##$566r$cL<V^8dQhRS[RS[S[S[S[3,,/#)rprefixr)r*rrr)r!rs"r"r#rs,xxcxhtCH~.Fxr$cVPRV R2V R234pVUu/uFq3R,VR,bK ppVPV R24pVe\V4^8:dR#\V4\VPV R24;'g^4rvRRRVR VR \^d\^dV,V, 44/#uupi) z5SELECT key, value FROM state_meta WHERE key IN (?, ?) _high_water _progressr6r.NpendingTtotalindexedpercent) _read_allrrRmin)rrrowsrmeta high_waterrprogresss&& r"r"SessionSearchMixin._rebuild_statuss~~U"(5&7KLN.23d%!G*$d3XX 45  ZA!5j/3txx6()8L/M/R/RQR+Sx4%HiQTUXZ]^adl^lot^tZuQvww 4sCzINSERT INTO {table}(rowid, content, tool_name, tool_calls) SELECT m.id, m.content, m.tool_name, m.tool_calls FROM messages m WHERE m.id > ? AND m.id <= ? {extra}AND NOT EXISTS (SELECT 1 FROM {table}_docsize d WHERE d.id = m.id)a4INSERT INTO messages_fts(rowid, content, tool_name, tool_calls) SELECT m.id, CASE WHEN m.role = 'tool' THEN substr(COALESCE(m.content, ''), 1, ?) ELSE m.content END, m.tool_name, m.tool_calls FROM messages m WHERE m.id > ? AND m.id <= ? AND NOT EXISTS (SELECT 1 FROM messages_fts_docsize d WHERE d.id = m.id)zINSERT INTO messages_fts_trigram(rowid, content, tool_name) SELECT m.id, m.content, m.tool_name FROM messages m JOIN sessions s ON s.id = m.session_id WHERE m.id > ? AND m.id <= ? AND m.role <> 'tool' AND szP AND NOT EXISTS (SELECT 1 FROM messages_fts_trigram_docsize d WHERE d.id = m.id)zINSERT INTO {table}(rowid, content, tool_name, tool_calls) SELECT id, CASE WHEN role = 'tool' THEN substr(COALESCE(content, ''), 1, zW) ELSE content END, tool_name, tool_calls FROM messages WHERE id > ? AND id <= ?{extra}c<V^8dQhRR/#rrF)r!rs"r"r#rs O OT Or$c VPR3.pVP'dVPVPR34VP RV4\ P R4R#)a9Finalize the deferred rebuild: boundary sweep + clear markers. The sweep is cheap insurance against a write that slipped between high_water capture and trigger activation. The trigram half is gated on ``_trigram_available``: without the tokenizer/table an unconditional INSERT raises and aborts the whole rebuild.TFru7Deferred FTS rebuild complete — all messages indexed.N)_BASE_BOUNDARY_SWEEP_SQL_trigram_availablerz_TRIGRAM_BOUNDARY_SWEEP_SQL_rebuild_finishrinfo)rsweepss& r"_fts_rebuild_finish&SessionSearchMixin._fts_rebuild_finishsW 00$78  " " " MM4;;UC D ]F3 MNr$c<V^8dQhRR/#rrF)r!rs"r"r#r*sOOOr$c VPPRRR7pVPRVR3.4RVn\P R4R#) z?Boundary sweep + clear the cjk markers; index becomes servable.messages_fts_cjkzAND m.role <> 'tool' tableextrarFTu7CJK FTS index backfill complete — serving CJK search.N)_BOUNDARY_SWEEP_SQLr!r_fts_cjk_availablerr)rsweeps& r"_fts_cjk_rebuild_finish*SessionSearchMixin._fts_cjk_rebuild_finish*sL((//6HPg/h .%0@A"& MNr$cP<V^8dQhRS[RS[S[S[S[3,,RR/#)rr sweep_sqlsrN)r*rarrS)r!rs"r"r#r1s1 ! !c !tE#t)._do4stx{%;s [ [$ [r$c VP'gR#VPPRRR7.pVP'dVP VP 4VP RVRVPRR7#) zBackfill one chunk of the deferred FTS rebuild; True while work remains. Chunks are claimed atomically inside the write transaction, so concurrent processes interleave instead of duplicating rows.F messages_ftsr(rrz)FTS rebuild chunk failed (will retry): %sT)fail_msgfinishfinish_when_empty)r_CHUNK_INSERT_SQLr!rrz_TRIGRAM_CHUNK_INSERT_SQL _rebuild_stepr)rinsertss& r"fts_rebuild_step#SessionSearchMixin.fts_rebuild_step>s|   ))00~R0PQ  " " " NN499 :!!-Cn)-)A)AUY"[ [r$c <V^8dQhRS[/#rr)r!rs"r"r#rJs\\d\r$c VP'dVP'gR#VPPRRR7pVP RV.VP RR7#)z=Backfill one chunk of the CJK index. True while work remains.Frz AND role <> 'tool'rrz-CJK FTS rebuild chunk failed (will retry): %s)rr)r_fts_cjk_loadedrr!rr)rinserts& r"fts_cjk_rebuild_step'SessionSearchMixin.fts_cjk_rebuild_stepJse   (<(<(<''..5GOd.e!!"3fXdFbFb+Z"\ \r$rFc H<V^8dQhRS[RS[S[,RS[RS[RS[/#)rr insert_sqlsrrr)r*rarS)r!rs"r"r#rRs6''C'd3i'c')-':>'r$c aaa a VPS R24pVfR#\V4o VPo V V VV3RlpVPV4pTRJdCTPS4p T'dS ^8:gT eT R,T R,8dT!4R#\T4# \P d!p \ PY94Rp ? R#Rp ? ii;i)zShared chunk engine for the base and CJK deferred backfills. ``finish_when_empty`` finalizes a high_water <= 0 marker (empty messages table) instead of leaving it pending.rNFc <\VS R24pVfR#\V^,4pVS8dR#\VS,S4pSFpVPWBV34K VPR\ V4S R234VS8#)rFz-UPDATE state_meta SET value = ? WHERE key = ?)r=rRrr:r*) r<rowrrdrchunkrr#rs & r"r -SessionSearchMixin._rebuild_step.._do\sDVHI"67C{3q6{H:%5(*5E" SU"34# LLH3u:Z`YaajWkJl m:% %r$Trr) get_metarR_FTS_REBUILD_CHUNK_ROWSr r7OperationalErrorrdebugrrS) rrr#rrrhigh_water_rawr morerstatusr'rs &ff$$$ @@r"r SessionSearchMixin._rebuild_stepRs&'=>  !( ,, & &" &&s+D 5=))&1F!jAo"vi'8F7O'KDz''  LL ' sB$$C9CCc <V^8dQhRS[/#rr)r!rs"r"r#r{s>>$>r$c  aaaSP4;_uu_4pVPRSPPRR4R,34P 4Uu.uF q"^,NK ppRRR4X'gR#V^,oVVV3RlpR RV3R lllo\ SP V44#uupi +'giLV;i \Pd"p\PR T4Rp?R #Rp?ii;i) uTear down one chunk of a demoted v22 FTS shadow table (a PLAIN table now); True while work remains. INTEGER single-column-key tables drain with a high-water marker so each chunk's scan is bounded (restarting the scan was O(n²)); compound-key tables keep the chunked ``LIMIT`` delete — they are small by construction. Single-column-key trash tables (the common shape — FTS shadow tables carry a rowid/integer PK) are drained with a high-water marker mirroring :meth:`fts_rebuild_step`: each chunk deletes only rows after the previously-drained key, so the per-chunk scan is bounded instead of re-scanning from the start of the table every chunk (O(n²) total on large trash tables, #79324). zNSELECT name FROM sqlite_master WHERE type = 'table' AND name LIKE ? ESCAPE '\'rIz\_rtNFc ~<VPRS R24Uu.uF<q^,^8gKV^,V^,;'gRP43NK> ppV'dRPRV44MRp\V4^8XdV^,^,R8XdRS R2p\ W4pVe\ V^,4M^pVPR V R S R V R V R S P 2 V34P4pV'g S !W4#VR,^,pVPRS R V RV R2Wh34p V P^8dS PV\V4VR7R#VPRS RV RV R S R S P R2 4p V P^8Xd S !V4#R#uupi)zPRAGMA table_info(rMr(rc3*"TF wrVxK R#5irVrF)rHr_typs& r"rJKSessionSearchMixin._fts_teardown_trash_step.._do..s;7ZTD7srowidINTEGER fts_teardown_rzSELECT z FROM z WHERE z > ? ORDER BY z LIMIT z DELETE FROM z > ? AND z <= ?cursorTz WHERE (z ) IN (SELECT ) r:rdrNlenr=rRr*fetchallrowcountset_metar*) r<rpk_infor6 marker_keyr&r upper_rowsrdcur_droprtbls & r"r 8SessionSearchMixin._fts_teardown_trash_step.._dos=A\\L^_b^ccdJe=fs=flmjnqrjr3!qtzzr0023=fGs?F$));7;;GC7|q WQZ]i%? -SE; 1,/OSQ[ !\\cU&WSEuGTXTpTpSqrM(*" 22"2q)ll\#gcU)C5PU#VYcXkl<.__annotate__s  HSM T r$cz<VPRS 24Ve \W4\PRS4R#)uADrained — the DROP is cheap now. True: re-check for more trash.zDROP TABLE IF EXISTS z"Old FTS shadow table %s torn down.T)r:rOrr)r<rBrFs&&r"rE:SessionSearchMixin._fts_teardown_trash_step.._drops7 LL06 7%T. KKrSr r7r+rr,)rr<rtrashr rrErFsf @@r"_fts_teardown_trash_step+SessionSearchMixin._fts_teardown_trash_step{s^^  #'<<a''//U;cAC$hj$$aqTT$E Ah ><   ++C01 1_ b''  LLKS Q s6AC!B=2C#C=C C D *DD c<V^8dQhRR/#rrF)r!rs"r"r#rs000r$c VP'gR#RpVPV4'dVP4R#R#)zFrom-scratch rebuild of a stale cjk index (triggers were dropped, gap extent unknown): drop table + triggers, clear the breadcrumb, recreate (fresh backfill markers).Nc\V\4fR#\FpVPRV 24K VPR4VPR4\ V\RR4R#)NFzDROP TRIGGER IF EXISTS z%DROP TABLE IF EXISTS messages_fts_cjkz(DROP VIEW IF EXISTS messages_fts_cjk_srcfts_cjk_rebuild_high_waterfts_cjk_rebuild_progressT)r=r rr:rO)r<trigs& r"r 7SessionSearchMixin._fts_cjk_reset_if_stale.._dos_019) 6tf=>* LL@ A LLC D 02NPj kr$)rr _ensure_cjk_schema_committedrr s& r"_fts_cjk_reset_if_stale*SessionSearchMixin._fts_cjk_reset_if_stales<###     s # #  - - / $r$c<V^8dQhRR/#rrF)r!rs"r"r#rs  d r$cVP;_uu_4VPVP4VPP4RRR4R# +'giR#;irV)_lock_ensure_fts_cjk_schema_conncommitrs&r"rX/SessionSearchMixin._ensure_cjk_schema_committeds9 ZZZ  ' ' 3 JJ   ZZZs 6A A& c <V^8dQhRS[/#rr)r!rs"r"r#rs  t r$c VPR4P4^,'gR#VPR4P4^,'*# \PdR#i;i)aTrue when the base FTS table indexes nothing while ``messages`` has rows (post-demote empty-index shape). Caller holds ``self._lock``. docsize is the authoritative "is this rowid indexed" surface; EXISTS not COUNT(*) because this runs on every writable open.z%SELECT EXISTS(SELECT 1 FROM messages)Fz1SELECT EXISTS(SELECT 1 FROM messages_fts_docsize))r:r;r7r+rr<s&&r"'_fts_external_index_empty_with_messages:SessionSearchMixin._fts_external_index_empty_with_messagessc << GHQQSTUVV||$WXaacdeff f''  s+A*AA43A4c<V^8dQhRR/#rrF)r!rs"r"r#rs D D Dr$c \VR4f?VPV4'gVPV4VPRRVR7R#R#)ahigh_water without progress: fts_rebuild_step reads missing progress as "done by another process" and optimize would no-op then stamp. Reset to known-empty, re-seed. Truncation goes through FTS5 ``'delete-all'`` (a plain DELETE is O(rows) and corrupts the index when indexed rows diverged from ``messages``); the backfill worker replays without an anti-join, so it needs a known-empty index. A missing docsize table counts as empty.fts_rebuild_progressN0r:)r=_fts_index_known_empty_reset_fts_index_to_emptyr@res&&r"_reseed_missing_progress+SessionSearchMixin._reseed_missing_progresssI T1 2 :..t44..t4 MM0#dM C ;r$c <V^8dQhRS[/#rr)r!rs"r"r#rsr$c \VPR4P4^,4^8H# \PdR#i;i)zZTrue when the base external-content index holds no rows (a missing table counts as empty).z)SELECT COUNT(*) FROM messages_fts_docsizeT)rRr:r;r7r+)r<s&r"rl)SessionSearchMixin._fts_index_known_emptysI t||$OPYY[\]^_cdd d''  s14A  A c<V^8dQhRR/#rrF)r!rs"r"r#rsOO4Or$c  RFPp\P!\P4;_uu_4VP RV RV R24RRR4KR R# +'giKg;i)zTruncate the v23 external-content tables via FTS5 ``'delete-all'`` (O(1); a plain DELETE is O(rows) and corrupts the index when indexed rows diverged from ``messages``). INSERT INTO (z) VALUES('delete-all')N)rmessages_fts_trigram) contextlibsuppressr7r+r:)r<rFs& r"rm,SessionSearchMixin._reset_fts_index_to_emptysS> |C5#6LMN?><>>>s A A* forcec&<V^8dQhRS[RS[/#)rr{r)rSrR)r!rs"r"r#rs    r$c <\VR4pVe,V'g$VPV4\V^,4#VPR4P 4^,pVP R\ V4VR7VP RRVR7\V4#)zWrite ``fts_rebuild_high_water`` / ``fts_rebuild_progress`` for a full backfill; returns the high-water id. Without ``force`` an existing high_water only gets a missing progress key repaired. Caller holds the write transaction.fts_rebuild_high_waterz)SELECT COALESCE(MAX(id), 0) FROM messagesr:rjrk)r=rnrRr:r;r@r*)rr<r{ existing_hwrs&&$ r"_seed_fts_rebuild_markers,SessionSearchMixin._seed_fts_rebuild_markerss &>?  "5  ) )$ /{1~& & \\E F O O QRS T .B E ,c$ ?2wr$c<V^8dQhRR/#rrF)r!rs"r"r#rs!!d!r$c 6aV3RlpSPV4R#)ukHeal interrupted demote/backfill bookkeeping before optimize runs: orphan high_water gets progress re-seeded; an empty external index with messages and no markers gets a full backfill claim. Never invents markers on a still-legacy inline DB — optimize would then skip demote and INSERT against the inline table forever. Covers two post-#65798 failure classes: c<\VR4eSPV4R#SPV4'dR#SPV4'd"\ VR4SP VRR7R#R#)r~Nfts_storage_versionTr{)r=rn_db_has_legacy_inline_ftsrfrOr)r<rs&r"r ._dosk78D--d3--d33;;DAAT#89..t4.@Br$Nr rYsf r"_repair_optimize_bookkeeping/SessionSearchMixin._repair_optimize_bookkeepings A C r$c <V^8dQhRS[/#rr)r!rs"r"r#rsr$c VP'dVP'dR#VP4;_uu_4pVPV4;'g\ VR4RJ;'goVP ;'d1\ VR4RJ;'gE\ V\ 4RJ;'g+VPV4;'gVPV4uuRRR4# +'giR#;i)a6True when `optimize_fts_storage()` has work: legacy inline FTS or a v23 trigram still carrying ``tool_calls`` (``_db_needs_fts_storage_upgrade``), an interrupted optimize (markers/trash), a CJK backfill on this tokenizer-capable host, or an empty external index without markers. False when FTS5 is unavailable.Fr~NrT) r read_onlyrL_db_needs_fts_storage_upgrader=rr _has_fts_trashrfres& r"fts_optimize_available)SessionSearchMixin.fts_optimize_availables    DNNN ^^  2248FFT#;rrO)r<r&rrs& r"_stage>SessionSearchMixin._demote_legacy_fts_to_trash.._stage4s  # #D ) LLG H||?hj 89 p ;<<<L (* CLL<Ax7PQTUVQWPX!YZ //D/AB 7 8Ir$z@failed to create v23 messages_fts during optimize-storage demote)rRr _ensure_v23_fts_tables)rrrsf r"_demote_legacy_fts_to_trash.SessionSearchMixin._demote_legacy_fts_to_trash.s3  >$$V, - ##$fg r$c$<V^8dQhRS[RR/#)rfailure_messagerNr@)r!rs"r"r#rXs   c  d  r$c |VP;_uu_4VPVPR\4pVPVPR\4p\ V4VnV'g\P!V4hVPP4RRR4R# +'giR#;i)zEnsure the v23 base + trigram tables under the lock (IF NOT EXISTS, cheap); raise *failure_message* without the base table (the backfill loop would retry forever).rrwN) r^_ensure_fts_schemar`r rrSrr7r+ra)rrbase_ok trigram_oks&& r"r)SessionSearchMixin._ensure_v23_fts_tablesXs|ZZZ--djj.'RG00=SUdeJ&*:&6D #..?? JJ    ZZZs B B** B; c <V^8dQhRS[/#rr)r!rs"r"r#rcs$r$c VP;_uu_4VPPR4RRR4RpVP;_uu_4VPPR4RRR4V# +'giLM;i \Pd#p\ P RT4RpRp?LRp?ii;i +'giT#;i \d"p\ PRT4Rp?T#Rp?ii;i)zPhase 3: reclaim freed pages to the OS. False when VACUUM failed (usually no free disk for its temp copy; a later VACUUM reclaims).VACUUMNTz$VACUUM after FTS optimize failed: %sFzPRAGMA wal_checkpoint(PASSIVE)z9WAL checkpoint (PASSIVE) after optimize VACUUM failed: %s) r^r`r:r7r+rrrr,)r vacuum_okrs& r"_optimize_vacuum#SessionSearchMixin._optimize_vacuumcs  ""8,I [  ""#CD %''  NNA3 GI  [ LLTVY Z Z [sbB A7 B CC-C7 B B CB<<C C CC D#C??Dc0<V^8dQhRS[S[,/#r)rr*)r!rs"r"r#r{s r$c x\VR4eR#VPV4'dR#VPV4'dR#\V4pV'g\ V4'dR#V'd"VP R\ \4VR7\VR4VPR \\34R#) aPhase 4 (inside the write transaction, so a concurrent writer cannot race a stamp past incomplete work): stamp the FTS layout (source of truth for "optimized"), clear the "available" flag, advance a lagging schema_version. Returns a refusal reason or None. Refuses while optimize work remains; an empty base index against non-empty messages also refuses (settling there meant permanent search-index loss).r~Nbackfill_incompleteteardown_incomplete layout_driftrr:rz7UPDATE schema_version SET version = ? WHERE version < ?) r=rrfrrr@r*rrOr:r)rr<currents&& r"_optimize_settle#SessionSearchMixin._optimize_settle{s T3 4 @(   t $ $(  7 7 = =($T*+D11!  MM/5H1IRVM WT34 NQ_aoPpqr$ progress_cbNvacuumTc~<V^8dQhRS[S[S[S[S[3,.R3,,RS[RS[S[S[3,/#)rrNrr)rrrr*rrS)r!rs"r"r#rsLM3M3&xc3h0@$0F'GHM3Y]M3 c3hM3r$c faaa SP'gRRRR/#SP'dRRRR/#SP4SP;_uu_4SP SP 4pRRR4SP R4RJpX'dV'gSP4M!V'dV'gSPR4SP4SP'dSP4R VV3R llo R V V3R llpS !R 4V!R SP4S !R 4V!R SP4S !R4V!RSP4SP4;_uu_4p\!VR4RJpSP#V4pSP%V4p RRR4X'gX'g X 'd5V'g X 'dRMRp \&P)RWXX 4RRRV RR/#Rp V'dS !R4SP+4p SP-SP.4p V e\&P)RV 4RRRV RV /#S !R4\&P1R\24RRRV /# +'giEL!;i +'giL;i)aYRepair an older FTS layout into the current v23 shape, foreground and to completion: legacy-v22 inline -> external-content, or a v23 ``messages_fts_trigram`` that still stores ``tool_calls``. Re-running resumes. ``progress_cb`` receives {"phase", "percent", "indexed", "total"}. A missing trigram tokenizer is not fatal (CJK falls back to LIKE).okFreasonfts5_unavailablerNr~z?failed to re-create v23 messages_fts on optimize-storage resumec(V^8dQhR\RR/#rphaserNr@)r!s"r"r#=SessionSearchMixin.optimize_fts_storage..__annotate__s d d d dr$c <SfR#SP4;'gSP4pS!RTRV'd VR,M^dRV'd VR,M^RV'd VR,M^/4R#)Nrrrr)rr)rstrrs& r"_emit6SessionSearchMixin.optimize_fts_storage.._emitse"((*KKd.I.I.KB %RByMS"RByMQXZG`ac dr$c(V^8dQhR\RR/#rr@)r!s"r"r#rs w w# w wr$c <\P!4pV!4'gR#S!V4\P!\SP\P!4V, SP ,44K)uRun *step* to completion; the inter-chunk sleep is the single place the duty cycle is enforced — back-to-back BEGIN IMMEDIATE chunks starve a live gateway/CLI out of its lock retries.N)time monotonicsleepmax_FTS_REBUILD_MIN_PAUSE_FTS_REBUILD_DUTY_FACTOR)rstep_t0rrs&& r"_drive7SessionSearchMixin.optimize_fts_storage.._drivesWnn&vve  3t::T^^=MPS=SW[WtWt > >>>%;7 7 ))+ ZZZ$($F$Ftzz$R !-- 89E   , , . 2  ' '(i j $$&      - - / d d w w jz4001 jz4445 jz4889 ^^  %d,DETQM--d3KFFtLK K;.;{*PeF NNm!+{ L%6:tD D  (O--/I%%d&;&;<   NNI7 S%7J J J f  EGZ[dJ 22ZZT sJ 1J  J J0 cx<V^8dQhRS[RS[RS[RS[RS[S[S[R3,,RS[S[S[3,/#)r session_idaround_message_idwindowbookend keep_roles.r)r*rRrrrr)r!rs"r"r#rsS, , , 25, ?B, QT, U38_-, JNsTWx., r$c ^aaaaaa\S^4oSPSW#R7pVR,pV'g R.R^R^R.R./#TpVeJ\V4p VU u.uF2qPR4V8XgV PR 4V 9gK0V NK4 pp .p .p S^8dVfR MR R P R V44 R2oVf.M \ V4oSP 4;_uu_4oRVVVVV3Rllp V !RV^,R,R4p \ \V !RVR,R,R444p RRR4RV3RllpRTRVR,RVR,RV Uu.uF q!V4NK upRV Uu.uF q!V4NK up/#uup i +'giLb;iuupiuupi)aAnchored window (``get_messages_around``) plus session bookends, so one call yields the goal and the resolution of a long session. ``window`` is filtered to ``keep_roles`` (None disables) EXCEPT the anchor; ``bookend_start`` / ``bookend_end`` are the first/last ``bookend`` non-empty-content messages with ids strictly outside the window (empty when it overlaps the head/tail). Empty result when the anchor isn't in the session.)rrmessages_beforemessages_after bookend_start bookend_endNidroler(z AND role IN (rBc3&"TFpRxK R#5irDrFrGs& r"rJ7SessionSearchMixin.get_anchored_view..sQj_iZ[RU_irLrMc<V^8dQhR\R\R\/#)rop boundary_idorderr*rR)r!s"r"r#:SessionSearchMixin.get_anchored_view..__annotate__s!!!!3!s!r$c f<SPRV RS RV R2SV.SOSN54P4#)z3SELECT * FROM messages WHERE session_id = ? AND id z ?z% AND length(content) > 0 ORDER BY id z LIMIT ?)r:r>)rrrrr< role_clause role_paramsrs&&&r"_bookend6SessionSearchMixin.get_anchored_view.._bookendsS<<779d"[MJ'',gX7$[H;HH  hj !r$<ASC>DESCcFV^8dQhR\\\3,/#r)rr*r)r!s"r"r#rs h hT#s(^ hr$c,<SPVRRR7#)get_anchored_viewF) warn_context summary_flag)_row_to_message_dict)r&rs&r"_hydrate6SessionSearchMixin.get_anchored_view.._hydrates,,S?Raf,g gr$r<)rget_messages_aroundrrrNrarLreversed)rrrrrr primitive window_rowsfiltered_windowkeep_setmbookend_start_rowsbookend_end_rowsrrrr<rrsff&&f& @@@r"r$SessionSearchMixin.get_anchored_viewsgq/,,Z9J,Z ) b"3Q8H!_^`boqst t%  !:H*5w+QtHY9Y]^]b]bci]jnv]vqq+Ow(*&( Q; * 2".Qj_iQjIjHkkl8mK * 2"Z8HK!!T!!&.c;q>$3G%O"#'#{2t?TV\1](^#_ " h h o'8)DU:V i(89 3EF3Eahqk3EF 1AB1AAHQK1AB   +x "!$GBs%-F  F (AF"F%9F* F" c X<V^8dQhRS[RS[RS[RS[S[S[S[3,,/#)rrlimitrrr*rRrSrarr)r!rs"r"r#rs?###&)#BF#SWX\]`be]eXfSg#r$c  V'dRMRpRpVP4;_uu_4pVPRV V R2V\V4^,^,34P4pRRR4^RIHp.p XFp \ V 4\V48dV #VPV R,4p VPV 4'dKP\V \4'd\V 4;'gT p M \V 4p R PV P44p \ V 4^P8dV R ,R ,p V PR V R ,R V R ,RV /4K V # +'giEL ;i)uThe *limit* most-recent real user turns, newest first, as ``{id, timestamp, preview}`` (80 chars, whitespace collapsed); used by /rewind and ``/undo [N]``. Bookkeeping rows (``display_kind`` set) are excluded. Legacy compaction handoffs are role='user' rows with NO display_kind — invisible to SQL — so fetch with headroom and drop them in the decode loop; otherwise ``/undo N`` pairs an in-memory count that excludes handoffs with a DB pick that includes them.r(z AND active = 1zJ AND (display_kind IS NULL OR display_kind = '' OR display_kind = 'steer')zRSELECT id, timestamp, content FROM messages WHERE session_id = ? AND role = 'user'z ORDER BY id DESC LIMIT ?N)ContextCompressorcontentrj:NMNz...r timestamppreview)rLr:rRr>agent.context_compressorr r=_decode_content_is_context_summary_contentrr*r rrNrcrz) rrrr active_clausedisplay_clauser<rr resultr&rrs &&&& r"list_recent_user_messages,SessionSearchMixin.list_recent_user_messagessW/4E f ^^  <<d /.!12++SZ!^a/0  hj  ?')C6{c%j( **3y>:G <.__annotate__Ds" 8 8RXXc] 8s 8r$cp<SPVP^44R\S4^, R2#)Q)rzgroupr=)r _quoted_partss&r"_hold6SessionSearchMixin._sanitize_fts5_query.._holdDs3   ,3}-12$7 7r$rlrjrtz\*+ryz(^|\s)\*z\1z(?i)^(AND|OR|NOT)\b\s*r(z(?i)\s+(AND|OR|NOT)\s*$z\b(\w+(?:[._-]\w+)+)\bz"\1"r"r#) r_QUOTED_PHRASE_REsubrm_FTS5_SPECIAL_REr _contains_cjkrr+ enumerate)rr& sanitizediquotedr%s& @r"_sanitize_fts5_query'SessionSearchMixin._sanitize_fts5_query8s' ++,!  8 8&))%7??SI %((i8 ) $6$D$DY$O$O!))#s3IFF63 2 FF;y9 FF4b)//:KL FF5r9??;LM FF4gyI "=1IA!))E!D/6BI2  r$c&<V^8dQhRS[RS[/#rrrr*rS)r!rs"r"r#r^s44C4D4r$cj\;QJdRV4F 'gK R# R#!RV44#)c3J"TFp\\V44xK R#5irVr\ordrHchs& r"rJ3SessionSearchMixin._contains_cjk.._s3d73r7##ds!#TF)rZrs&r"r+ SessionSearchMixin._contains_cjk]s+s3d3ss3s3s3d333r$c&<V^8dQhRS[RS[/#r3r)r!rs"r"r#rbs9999r$c&\RV44#)c3^"TF#p\\V44'gK^xK% R#5i)Nr7r9s& r"rJ0SessionSearchMixin._count_cjk..cs8tws2w'711ts- -)sumr<s&r" _count_cjkSessionSearchMixin._count_cjkas8t888r$c&<V^8dQhRS[RS[/#rr4)r!rs"r"r#rfs    r$c ^pVF4p\\V44'd V^, pK(V^8XdR#^pK6 V^8H#)uTrue when any maximal CJK run is a single char: the cjk-bigram index stores unigrams only for isolated chars, so such a term can't match inside longer runs — keep LIKE.Tr7)rrunr:s& r"_has_lone_cjk_run$SessionSearchMixin._has_lone_cjk_runesEBs2wq!8 axr$c6<V^8dQhRS[RS[S[,/#r)r*r)r!rs"r"r#rts ? ? ?# ?r$c .p\PV4F7pVP4pVR9dR#VR8wgK&VPV4K9 \ V4^8dRP V4#R#)aOThe sanitized implicit-AND query rewritten as an any-term OR query, or ``None`` when relaxation does not apply: fewer than two searchable units (a single term cannot relax) or explicit ``OR``/``NOT`` (the caller expressed exact semantics). Quoted phrases stay whole units: ``"docker networking" tls`` -> ``"docker networking" OR tls``.NAND OR >ORNOT)_LIKE_TOKEN_REfindallrdrzr=rN)runits raw_tokenrds& r"_or_relaxed_query$SessionSearchMixin._or_relaxed_queryssh '//6IOO%E %~ Y' 7 &)Z1_v{{5!>$>r$c&<V^8dQhRS[RS[/#rr4)r!rs"r"r#rsAAAAr$c \VPR4P44p\V4;'d5\;QJdRV4F 'dK R# R#!RV44#)zTrue when every non-operator token is >=3 chars: a shorter token produces no trigrams, and with FTS5's implicit AND one such token empties the whole MATCH.rlc3>"TFp\V4^8xK R#5iN)r=rs& r"rJ>SessionSearchMixin._trigram_eligible_tokens..s#@ACFaKsFT)rgr+rSall)rtokenss& r"_trigram_eligible_tokens+SessionSearchMixin._trigram_eligible_tokenssU&ekk#&6&<&<&>?F|@@#@#@@@#@#@ @@r$c&<V^8dQhRS[RS[/#r^r4)r!rs"r"r#rskkSkTkr$c a\;QJd(V3Rl\V44F 'gK R# R#!V3Rl\V444#)uTrue when any non-operator CJK token has fewer than 3 CJK chars — the trigram tokenizer needs >=3 per token, so such a query must take the LIKE route.c3~<"TF2pSPV4'gKSPV4^8xK4 R#5irZ)r+rD)rHrfrs& r"rJ:SessionSearchMixin._has_short_cjk_token..s3j2QQUXUfUfghUi(3>>!$q(2Qs==TF)rZrg)rr_sf&r"_has_short_cjk_token'SessionSearchMixin._has_short_cjk_tokens:sj2Fy2Qjssjsjsj2Fy2Qjjjr$c&<V^8dQhRS[RS[/#r^r4)r!rs"r"r#rs--3-4-r$c VPV4^8;'d+VPV4'*;'d VP#)uPer-token CJK length gate for the trigram index: ``广西 OR 桂林 OR 漓江`` has 6 CJK chars total but 2 per token, so trigram returns 0.)rDrer)rr_s&&r"_trigram_route_ok$SessionSearchMixin._trigram_route_oksH *a/,,8Q8QR[8\4\,,++ -r$c&<V^8dQhRS[RS[/#rr@)r!rs"r"r#rs33r$c VP'dR#TPT;'gR4pV'gR#VPV4'gR#VPR4P4pVP'dVP V4'gR#VP V4'dR#R# \dR #i;i) z>Best-effort name of the routing path a query takes (log-only).like_scan_fts_staler(emptyfts5rlfts_cjktrigram like_scanr) _fts_staler0r+r+rrIrir)rrr-raws&& r"_describe_search_path(SessionSearchMixin._describe_search_paths ,11%++2>I%%i00//#&,,.C&&&t/E/Ec/J/J $ 6 6s ; ;9 L L  s>B9B9B9B90B9B9B95B97B99 CCcT<V^8dQhRS[RS[RS[RS[RS[RS[S[S[3,/#)rr match_query order_by_sqlroffsetr)r*rRrra)r!rs"r"r#rsC  c  3 RU _b %*39%5 r$c V R2.pV.p\Wg3/VBVPW4.4\RV R2V RV R2WbR4pW3#)zHMATCH query + params against one FTS5 index joined to messages/sessions.z MATCH ?zsnippet(z), -1, '>>>', '<<<', '...', 40) AS snippetz' JOIN messages m ON m.id = z.rowidLIMIT ? OFFSET ?)rrr) rrxryrrzfiltersrrrs &&&$$, r"_fts_match_sql!SessionSearchMixin._fts_match_sqlsl7(#$#}u88 uo& ugF Gg=eWF KUbt {r$ fail_openoperational_debugc<V^8dQhRS[RS[RS[RS[S[,RS[S[,RS[S[S[S[S[3,,,/#)rrrxryrrr)r*rrarr)r!rs"r"r#rs\3cYabeYf'/}JRSWX\]`be]eXfSgJhr$c VP!WV3/VBwrxVPWx4U u.uFp \V 4NK up #uup i \Pd$T'd\ P TRR7R#\Pd>p TeTPT 4'gh\ PRYJ4Rp ? R#Rp ? ii;i)uRun one MATCH against *table*; ``None`` when the query cannot execute (tokenizer / syntax) so the caller falls back. *fail_open* names the index for the substring-capable routes: a corruption-class ``DatabaseError`` there detaches the derived indexes (``_enter_fts_fail_open``) and answers from canonical rows — a live search never runs the unbounded rebuild. Other ``DatabaseError``s propagate.T)exc_infoNz[%s FTS search hit a corruption error (%s); detached FTS and falling back to canonical LIKE.) r~rrr7r+rr, DatabaseError_enter_fts_fail_openr) rrrxryrrkwargsrrr&rs &&&&$$, r" _match_rowsSessionSearchMixin._match_rowss))%lUfU  )-)DE)D#DI)DE EE''   . >$$  (A(A#(F(F NNm   s?AAAAC$C>CCC -C  Cc n<V^8dQhRS[S[,RS[RS[RS[RS[S[S[S[3,,/#)rrrrrr)rar*rr)r!rs"r"r#rsOBBS B4BcBVYB^bcghkmphpcq^rBr$c \\RWV4pVPWR4Uu.uFp\V4NK up#uupi)zDCanonical-table LIKE scan; ``params[0]`` is the snippet anchor term.z messages m)r_LIKE_SNIPPET_SQLrr)rrrrrrr&s&&&$$ r" _like_rowsSessionSearchMixin._like_rowss; !2L%S\]%)^^C%@A%@cS %@AAAs=c`<V^8dQhRS[RS[S[S[S[,S[S[,3,/#r)r*rrarr)r!rs"r"r#rs4#B#B3#B5d3iRU9V3W#Br$c Z..pRp\PV4FpVP4pVR8Xd%VR,'dVP.4RpK>VR9dKGVR8XdRpKRVP R4P R4P 4pV'gKVR,PWR34RpK .p.pRpVFp V 'd;\ ;QJdRV 4F 'gK RM RM !RV 44'gKG.p V FWwr[T PV 'd R \ 2M\ 4VP\V44VeKKV 'dKUTpKY VPR R PV 4 R 24K R PV4Wx3#)zCompile the supported FTS boolean subset into LIKE predicates: terms within an OR group are ANDed (FTS5's implicit conjunction) and ``NOT`` negates the next term.FrOrPTrlryNc34"TFwrV'*xK R#5irVrF)rHrInegateds& r"rJASessionSearchMixin._compile_like_boolean_query..s#H%JAKK%szNOT rvrrMrNr<>rMr2) rQrRrdrzr+rZ_LIKE_COALESCED_COLUMN_SQLrrvrN) rgroups negate_nextrToperatorrrcompiled_groupsr snippet_termr$clausesrs & r"_compile_like_boolean_query.SessionSearchMixin._compile_like_boolean_querysw13t '//6I (H4"::MM"%# ?*5 " ??3'--c288:Dtr !!4"56# 7"&(&* E#H%#H#H%#H H H!#G!& g&@%ABSmn l401'#'L "'  " "Qw||G'<&=Q#? @{{?+VAAr$c n<V^8dQhRS[RS[RS[RS[S[,RS[S[S[S[3,,/#)rrrrzsortr)r*rRrrarr)r!rs"r"r#rsR u u u$' u14 uSessionSearchMixin._finalize_search_matches..sAXx!#xrL)idsmatch_idrr NN)ranger=_CONTEXT_WINDOW_SQLr!rNrLr:rar>rzrrrrpop) rrrstartbatchmatchcontextsrr<rr&rs &&& r"_finalize_search_matches+SessionSearchMixin._finalize_search_matchess  I$>q#g,4eck29>?$KO?"-44AXxAX9X4YC))T#||Ch@IIK*# Z188= $#E.(0||E$K'D,F'D$S[)]4K_K_`cdm`nKo=pqu=vw'D,Fi(#5&E IIi &  $gnogn^cYSX.+U|+YgnGo+@*)!"!H",F%.+-i(. ZosrF& AF>*F+9/F>.G > G GGG*)G*-G2c0<V^8dQhRS[RS[S[,R,RS[S[,R,RS[S[,R,RS[RS[RS[R,R S[R S[S[S[,,R S[S[,R S[S[,R S[S[S[S[3,,/ #rrrNrrrrzrrrrrrr*rarRrSrrrr)r!rs"r"r#r.s***)-cT)9*SWX[S\_cSc*#Y%*58*GJ*VY\`V`**19C0I*3- *4'dVP@!RV3/VB;'gTpV'gHVPB'd6VPEV4'dVP@!RV3/VB;'gTpV'gRV'gJVP'g8VPGV4pVe#VP@!R V3RRRR/VB;'gTpVP WR7#uupi \(P*d.u#\(P,d=pTP/T4'ghTP!T3RTRTRT/T BpRp?ELRp?ii;iuupi \(P*d"p\8P;RT4Rp?ELRp?ii;i)a2FTS5 search across session messages (keywords, ``"phrases"``, AND/OR/NOT, ``prefix*``). Returns snippet + session metadata + 1-message context per hit; ``fields`` selects a projection. ``sort``: None = BM25 rank; "newest"/"oldest" = timestamp then rank (the CJK LIKE fallback ignores it). Rewound rows (``active=0, compacted=0``) are excluded by default; compaction-archived rows ARE included; ``include_inactive`` = every row. ``after_ts``/``before_ts`` bound ``sessions.started_at`` on every route (FTS5, CJK, trigram, LIKE fallback, unindexed-gap supplement).rrrrrrtoolrrzr)rNz ORDER BY rankryc32"TF q\9xK R#5irV)r)rHsrcs& r"rJ;SessionSearchMixin._search_messages_impl..ls'eWdPS/K(KWdsTFrrc3F<"TFqR,S9gKVxK R#5i)rNrF)rHrseen_idss& r"rJrsQ+Q49Pqq+s! !z$Unindexed-gap supplement skipped: %srlrrwrz OR-relaxedrz1OR-relaxed FTS retry failed; keeping empty resultrF)$rr+r0rrrrrsr _FTS_ORDER_BYrrr*rrSrZr+ _search_cjkr~rr7r+rrrr=_search_unindexed_gaprrr,rprrrr_rU)rrrrrrrzrrrrrrr}rryroutewants_unindexed_rowsis_cjkrrr&r gap_matchesrfb_queryrelaxedrs&&&&&&&&&&&& @r"r(SessionSearchMixin._search_messages_implEs.33F; EKKMMI))%0I(8'6 (? 6[099%rurU[rbfrjqrG000V V %%' ???99%rurU[rbfrjqrG000V V   I$((DRUAVAV););)=\`bqr V,VeVFVgV!%[ 1 K Kf 6K g g   e eCC'eWd'eCCC'eWd'e$e ##E* &&u.BEJG--neMuMKC w04sF0KL0K490KL"  " " $ 0S\E5I J"88G @T`X_` -45WAdGGW5Q+QQvtK/@/@V{EZ(S)9)?)?)ABH&&&**+=xQ5Q\\U\t6664;X;XY^;_;_**+A8UuU``Y`vdooo,,U3G"**>74l4=p4-24??7>,,W,RRuM++  (( w0055==ev5vY_vfjvnuv w"6++ J CSII Js`>O!O&O! 'Q2Q QO!!Q :Q  Q 1QQ  QR'RRc n<V^8dQhRS[RS[RS[S[S[3,RS[S[S[S[3,,/#)rrrrr)r*rSrrra)r!rs"r"r#rsKcccDccSVhc\`aefiknfnao\pcr$c \V4PR4P4p\V4pVP'd>V'g6VP V4'gVP !RV3RRRR/VBpVeV#VP V4'd%V'gVP !RV3RR/VBpVeV#\V4;'gV.pVUU u.uFp\V4FqNK K p pp R R P\.\V4,4 R 2.p RU u/uF qW<,bK p p \W3/V BVPW^,.V OVR ,NVR ,NRRR7#uup piuup i)uCJK routing: the unicode61 table splits CJK into single characters (false positives, missed phrases). cjk-bigram serves every shape except queries wanting rows the substring indexes exclude (role='tool', cron/subagent sources) and LONE 1-char CJK runs (bigrams only exist for runs >=2 — LIKE is broader); then trigram (>=3 CJK chars per token); then a LIKE substring scan with one clause per non-operator token so "广西 OR 桂林 OR 漓江" matches each term.rlrrz CJK-bigramrz;messages_fts_cjk query failed; falling back to trigram/LIKErwTrigramrvrNrMrrzORDER BY m.timestamp DESCr|rr)r~r+rprrIrrirgrvrN_LIKE_ANY_COLUMN_SQLr=rr)rrrrr_rxr non_op_tokensrnr like_params like_wherekr}s&&&& r"rSessionSearchMixin._search_cjks)/55c:@@B ' 2  " " "+?H^H^_hHiHi&&"Kj;Gj"_jchjG"  ! !) , ,5I&&'={iV_ichiG",Y7FFI; *7S-3cARAQARQ- S&++';&.s/ lJkQ!3!3C!8!>!>!@!@Jks:<zm.id > ? AND m.id <= ?rrc3H"TFp\V4Fq"xK K R#5irV)rv)rHrrrs& r"rJrs!=nED[ghl[mVWa[maEs "rzLIMIT ?r) rrQrRrd_LIKE_SKIP_TOKENSrr=rr) rrrr}r/rntermsrrs &&&, r"r(SessionSearchMixin._search_unindexed_gaps((* >UaZI l.J`J`ajJk lB lIIK/@@ lBI)R.B-Cc%j-PRy)6'?o=nE=nou88uQx&@&&@%&@Kf)24 4Bs C C&Cc<V^8dQhRS[RS[RS[RS[R,RS[S[,R,RS[S[,R,RS[S[S[S[3,,/#) rrrinclude_archivedsourceNsourcesrrr )r!rs"r"r#rso222!$2=A2RUX\R\2cT!2;?9t;K2W[\`adfiai\jWk2r$c  Na a T;'gRP4P4o S 'dV^8:d.#VPWEV\V^,V4^VRS R7pRV 3Rllo \ \ V4V 3RlR7pVRVU U u.uFwrV NK up p #uup p i) zSearch surfaced sessions by exact/prefix/substring session id. Also matches ``_lineage_root_id`` so an old compression root id resolves to the live continuation.r(T)rrrrrzrorder_by_last_activeid_querycRV^8dQhR\\\3,R\/#)rr&r)rr*rrR)r!s"r"r#>SessionSearchMixin.search_sessions_by_id..__annotate__s& U UtCH~ U# Ur$c<\VPR4;'gR4\VPR4;'gR43Uu.uFq'gK VP4NK pp\;QJdV3RlV4F 'gK RM RM!V3RlV44'd^#\;QJdV3RlV4F 'gK RM RM!V3RlV44'd^#^#uupi)rr(_lineage_root_idc3,<"TF qS8HxK R#5irVrFrHr.needles& r"rJJSessionSearchMixin.search_sessions_by_id..score..s; uF? sTFc3D<"TFqPS4xK R#5irV) startswithrs& r"rJrsM*,,V44*s )r*rrrZ)r&v normalizedrs& r"score7SessionSearchMixin.search_sessions_by_id..scores.1#''$-2E2E2.FCGGTfLgLmLmkmHn-ou-ost)!'')-oJus; ;sss; ;;;M*MM*MMM1 TST Tvs C6C6c4<S!V^,4V^,3#)rArF)itemrs&r":SessionSearchMixin.search_sessions_by_id..stAwQUVWQX@Yr$)r6N)r+rlist_sessions_richrrr,) rrrrrrr candidatesrankedrIr&rrs &&&&&&& @@r"search_sessions_by_id(SessionSearchMixin.search_sessions_by_ids ++2$$&,,.!I,,OSVW\_`W`bgSh'7d]c-e  U U  *-3YZ"(%.1..111s B!c&<V^8dQhRS[RS[/#)rrrr4)r!rs"r"r#rscdr$c ~VPPRV R24R# \PdR#i;i)uTrue if an FTS5 virtual table is queryable ("no such table" and "vtable constructor failed" — missing tokenizer / mid-teardown — both count as not).zSELECT 1 FROM z LIMIT 0TF)r`r:r7r)rrs&&r"_fts_table_exists$SessionSearchMixin._fts_table_existss=  JJ  vX> ?$$  s #<<c0<V^8dQhRS[S[,/#r)rar*)r!rs"r"r#rsPPT#YPr$c tVPUu.uFqPV4'gKVNK up#uupi)z3Queryable FTS tables (caller holds ``self._lock``).) _FTS_TABLESr)rrFs& r"_present_fts_tables&SessionSearchMixin._present_fts_tabless/#//O/3I3I#3N/OOOs55c <V^8dQhRS[/#rr)r!rs"r"r#rscr$c  VP4^pVP;_uu_4VP4VPfVP RR7VP 4F/pVPP RV RV R24V^, pK1 RRR4V# \Pd"p\PRY#4Rp?KnRp?ii;i +'giT#;i)aMerge fragmented FTS5 segments into one per index (``'optimize'``). Pure maintenance: changes neither results nor ``snippet()`` output, only layout and speed; VACUUM then returns the freed pages. Returns the number optimized. A quarantined handle never issues ``'optimize'``: it rewrites index segments in place and would compound structural damage (or a split WAL generation) instead of leaving it diagnosable.Nwriterrurvz) VALUES('optimize')zFTS optimize failed for %s: %s) _raise_if_db_corruptr^_raise_if_db_replacedr`_reopen_after_close_lockedrr:r7rrr)r optimizedrFrs& r" optimize_ftsSessionSearchMixin.optimize_ftss !!# ZZZ  & & (zz!///@//1OJJ&&cU!C5@T'UVNI2 ,,ONN#CSNNOZs6AC,+B%C%C :C CC C C/ c <V^8dQhRS[/#rr)r!rs"r"r#rs,,S,r$c  hVP4^p\VP4;_uu_4pV'g!\P R4RRR4^#VP ;_uu_4VP 4VPfVPRR7VP4FIpVPPRV RV R24VPP4V^, pKK RRR4RRR4V# \PdipTPP4\T4'd\P!RY44Rp?K\P R Y44Rp?KRp?ii;i +'giL;i +'giT#;i) u)Rebuild FTS5 indexes from ``messages`` (``'rebuild'``) — the recovery for a corrupt index that rejects writes while reads succeed. Two processes rebuilding one state.db concurrently corrupted production DBs, so this admits through ``fts_rebuild_admission`` and FAILS CLOSED, returning 0 on deferral (callers treat 0 as "no progress" and use the stale-FTS breadcrumb path). Returns indexes rebuilt. Uses the FTS5 ``'rebuild'`` command, which rewrites the internal b-tree segments from the content rows. Unlike ``optimize_fts`` (which merges existing segments), ``rebuild`` discards and recreates the index data entirely — the more destructive of the two, so it is quarantined the same way. See #50502. A full structural rebuild must never run concurrently in two processes sharing one state.db — that interleaving has structurally corrupted the database in production (PR #93200) — so this admits through the cross-process ``fts_rebuild_admission`` authority and FAILS CLOSED: if another process holds the rebuild lock beyond the bounded wait, this call defers (returns 0) rather than racing it. Callers already treat 0 as "rebuild made no progress" and fall back to the stale-FTS breadcrumb path, which retries in-process from the gateway housekeeping tick (``retry_deferred_fts_recovery``) and at next startup. z]Deferred in-place FTS rebuild: another process holds the rebuild authority for this state.db.Nrrrurvz) VALUES('rebuild')zFTS rebuild failed with a corruption-class error for %s: %s; run 'hermes sessions repair' (or 'hermes doctor --fix') to rebuild the index offlinezFTS rebuild failed for %s: %s)rrdb_pathrrr^rr`rrr:rar7rrollbackr error)rrebuiltadmittedrFrs& r" rebuild_ftsSessionSearchMixin.rebuild_ftssV& !!# "4<< 0 0Hsu 1 0 **,::%33G3D335C V **\#auDW+XY ))+1  6  1.#00V ++-055"LL!w #** #NN+JCUUV 1 0.sbF F -AF 2AD 7F ;F  F "AF #F )F ?F F F F F  F1 max_commandsc<<V^8dQhRS[RS[S[,RS[/#)rrr*r)rRr)r!rs"r"r#rDs(SQT adr$c  T\RV4Vf VPp\RV4^pVP;_uu_4VP4FpVP'g#VP P RV RV R24\V4FkpVP PpVP P RV RV R2V34V^, pVP PV, ^8gKjK K RVnRRR4V# +'giT#;i) aRun bounded FTS5 ``'merge'`` commands against each present index. A positive merge rank stops after ~that many output pages, so each command holds the write lock for milliseconds regardless of index size (``'optimize'`` takes 9-18 s per index on a 10 GB DB). ``usermerge`` is lowered to its minimum of 2 (persisted in ``%_config``, once per instance) so a merge acts on ANY level with >= 2 segments; at the default 4 a fragmented index cannot converge. Up to *max_commands* per index, stopping on the no-progress signal ``total_changes`` delta < 2 (the INSERT itself is 1). Each command is its own implicit transaction, so processes interleave mid-pass. Missing tables are skipped (optimize_fts_storage drops + backfills them live); other SQLite errors propagate. Returns commands executed.rNr*rurvz, rank) VALUES('usermerge', 2)z, rank) VALUES('merge', ?)T) r_FTS_MERGE_COMMANDS_PER_PASSr^r_fts_usermerge_floor_appliedr`r:r total_changes)rrr*executedrFrIbefores&$$ r"r+SessionSearchMixin._merge_fts_incrementallyDs k9-  <>@00&    D DOO u  !!&(((T   0.M3KOM3`dM3M3b, , \##N"!"!H4499   ? ?AA kk -- &  im7;,BB #B#BJ u u Z Z>**.hShSTcc:44"220PP(,,\X\r$r>rOrMrP))iNi)i4iM)iiߦ)i0i?0)i@0i0)i0i0)ii)Dr?rxloggingrr7rtypingrrrrrrragent.skill_commandsr hermes_state_errorsr hermes_state_commonr r r rrrrrrrrrurrrhermes_state_ftsrr getLoggerrr/_FTS5_SPECIAL_CHARScompileescaper* frozensetrerrQr(rrrrrrr[r=rOr\rgrprvr~rrrrrrFr$r"rOsY   III:5 B   > *5::")),?"@!ACD/0"fX-N+JJz*eYi5   &#&@D&<]]r$