߲jSpdZddlmZddlZddlZddlmZddlmZm Z m Z m Z m Z ddl mZmZmZmZmZddlmZej,dZed Zd Zd Zd>d Zd?d Zd@dAdZdBdZdddfddeedzfddedfddedfddedfddedefddedfd defd!ded"fd#ded$fd%ded&efd'ded(ej@fd)ded*fd+ded,fd-ded.fd/ded0efd1deedzfd2deed3zfd4deedzfd5deed3zfd6ded7fd8ded9ffZ!e"d:e!Dhd;zZ#Gd<d=Z$y)CzQRetention pruning, stale-session archiving and VACUUM policy mixin for SessionDB.) annotationsN)Path)AnyDictListOptionalTuple)AUTO_VACUUM_MIN_FREELIST_RATIO _id_chunks _placeholders_sql_session_last_active escape_like)report_startup_progress hermes_statesz<(COALESCE(s.input_tokens, 0) + COALESCE(s.output_tokens, 0))z4COALESCE(s.actual_cost_usd, s.estimated_cost_usd, 0)c<dt|jdS)N%) _escape_likelower)values =/home/ubuntu/.hermes/hermes-agent/hermes_state_maintenance.py_likers |EKKM*+1 --c@ddlm}||\}}|gt|fS)Nr)_cwd_prefix_clause)hermes_state_sessionsrlist)rrclauseparamss r_cwd_prefix_filterr s$8'.NFF 8T&\ !!rcfdS)Nc(gr |gfS|gfSN)vrconvs rz_one.."s!vhDa 89a 89rr$)rr&s``r_oner(!s 99rcR |r|t|z SdS#ttf$rYywxYw)zKAge of a state_meta timestamp; None when unset or corrupt (= no prior run).N)float TypeError ValueError)nowraws r_seconds_sincer/%s4#&sU3Z0D0 z "s&&last_active_beforenotnonec tdzdg||gfS)N < ?zG(COALESCE(s.end_reason, '') != 'startup_orphan_reap' OR s.ended_at < ?))_LAST_ACTIVE_SQL)r%s rr'r'1s! F " R T A1rlast_active_afterz >= ?started_beforezs.started_at < ? started_afterzs.started_at >= ?sourcetruthyz s.source = ? title_likez.LOWER(COALESCE(s.title, '')) LIKE ? ESCAPE '\' end_reasonzs.end_reason = ? cwd_prefix min_messageszs.message_count >= ? max_messageszs.message_count <= ? model_likez.LOWER(COALESCE(s.model, '')) LIKE ? ESCAPE '\'providerz+LOWER(COALESCE(s.billing_provider, '')) = ?user_idz s.user_id = ?chat_idz s.chat_id = ? chat_typezs.chat_type = ? branch_likez3LOWER(COALESCE(s.git_branch, '')) LIKE ? ESCAPE '\' min_tokens max_tokensz <= ?min_costmax_costmin_tool_callsz#COALESCE(s.tool_call_count, 0) >= ?max_tool_callsz#COALESCE(s.tool_call_count, 0) <= ?c#(K|] \}}}| ywr#r$).0name_s r rOKsFq!Fs>archivedinclude_pinnedlineage_tips_onlyc$eZdZdZdddZddZddddddd  d d Zedddd  d!d Zd"d Z d# d$dZ d# d%dZ d# d%dZ ddd&dZ d' d(dZd)dZd*dZd+dZd,dZd-dZdddddef d.dZy)/SessionMaintenanceMixinzKRetention pruning, stale-session archiving and VACUUM policy for SessionDB.Nctjdz fd}j|xsg}|r|ndD]}j||t|S)zCRemove empty TUI ghost sessions (no messages, no title, >24hr old).Qc|jdfjDcgc]}|d }}t|D]!}|jdt|d|#|rj ||Scc}w)NaZ SELECT id FROM sessions WHERE source = 'tui' AND title IS NULL AND ended_at IS NOT NULL AND started_at < ? AND NOT EXISTS ( SELECT 1 FROM messages WHERE messages.session_id = sessions.id ) r"DELETE FROM sessions WHERE id IN ())executefetchallr r #_delete_unreferenced_system_prompts)connridschunkcutoffselfs r_doz?SessionMaintenanceMixin.prune_empty_ghost_sessions.._doTs!% / "%HJ (A1Q4 (C ($C b A-PUBVAWWXY[`a b88>J (s A7r$)time_execute_write_remove_session_fileslen)rb sessions_dirrc removed_idssidras` @rprune_empty_ghost_sessionsz2SessionMaintenanceMixin.prune_empty_ghost_sessionsQs]u$  ))#.4" ".;B :C  & &|S 9 :;rc jddlm}ddlm} |j||fddddd|y#||f$rYywxYw)zTrue when a live turn lease / compression lock protects ``sid``; expired or dead-holder guards are reclaimed and fenced as a side effect.r)!SessionCompressionInProgressError)SessionTurnLeaseLostErrorNT)compression_lock_holderturn_lease_holderreject_active_turn_leasereject_active_compression_lockF)rrmhermes_state_errorsrn_check_transcript_write_guards)rbr]rjkwargsrmrns r_write_guards_rejectz,SessionMaintenanceMixin._write_guards_rejectis^ CA  /D / /c ^374)-d ^V\ ^ 23LM  s (22)tuidesktopsubagentr$FT)sources exclude_idsexclude_pinnedheartbeat_staleness_seconds!heartbeat_ownership_grace_secondsrespect_gateway_heartbeatscftd|D}|dks|sgS||} } | r| dkDs|dz} | | dk\s| } tjx} |z } |rdnd} dtdd}d }|r |d z }| | z | f}d t|d | d |g|| | |fd}j |xsgS)uClose session rows orphaned by a dead gateway process (its in-process disconnect grace timer died with it, leaving ``ended_at IS NULL`` forever). Rows of ``sources`` whose ``started_at`` AND canonical last activity are both older than ``max_idle_seconds`` get ``end_reason='startup_orphan_reap'`` (the ``started_at`` predicate protects fresh compression/branch children whose copied activity is old). Only pass sources whose lifecycle the caller owns — never messaging platforms like ``telegram`` (ending those triggers a routing loop). ``exclude_ids`` spares rows this process still holds. Non-destructive: messages kept, row resumable, first-reason-wins. With ``respect_gateway_heartbeats`` a row is reaped only when no live backend (heartbeat within ``heartbeat_staleness_seconds``, default ``2 * max_idle_seconds``) could own it: B owns S if ``B.started_at <= S.started_at + grace`` (default = staleness) — grace covers a migrating backend whose sessions predate its first heartbeat, bounded so a PID-reuse respawn cannot protect rows forever. Disable the gate only for state.db-owned sources. SELECT, live-lease validation and UPDATE run in one ``BEGIN IMMEDIATE`` transaction; active leases/locks spare the row, expired guards are removed so their owner is fenced. See #65194. ``exclude_pinned`` is intended for broad automatic sweeps; pinned rows remain explicitly recoverable. See #60609. c3&K|] }|s| ywr#r$)rLrs rrOzBSessionMaintenanceMixin.sweep_orphaned_sessions..s-11Q-srz AND COALESCE(pinned, 0) = 0zstarted_at < ? AND sessionsr3r$z| AND NOT EXISTS (SELECT 1 FROM gateway_heartbeats h WHERE h.last_heartbeat >= ? AND h.started_at <= sessions.started_at + ?)z AND source IN (rY AND cj|jdj}Dchc]}|st|}}d|DDcgc]}||vr j||s|}}|sgS|jdt |dt j g||Scc}wcc}w)Nz.SELECT id FROM sessions WHERE ended_at IS NULLc38K|]}t|dyw)idN)str)rLrows rrOzOSessionMaintenanceMixin.sweep_orphaned_sessions.._do..s&F#s3t9~&FszRUPDATE sessions SET ended_at = ?, end_reason = 'startup_orphan_reap' WHERE id IN (z) AND ended_at IS NULL)rZr[rrvr rd) r]rowsxexcludedrjvictimsr{ scope_params scope_sqlrbs rrcz._dos<<"PQZP[ \ ,..6hj (391qA9H9&F&F\sX-d6O6OPTVY6Z\G\ LL!!.w!7 88Nyk[6w66 8N:\sB+ B+ B0)tuplerdr r re)rbmax_idle_secondsrzr{r|r}r~rsrcs hb_stalenesshb_gracer-ra pin_scopeorphan_predicateheartbeat_paramsrcrrs` ` @@rsweep_orphaned_sessionsz/SessionMaintenanceMixin.sweep_orphaned_sessionsvs 2--- q I!<>_h !1+a/L$Q#H$#(886D2" 01I*1U0VVZ[.0 % "g h  #l 2H= &}T':&;1YKuM]L^_ AAvAvA0@A  ""3'-2-r)rPrQrRc t|tz }|rtdt|ddg}|r|j dg}t D]Y\}}} |j |} |dk(r|  n t| s-| | \} } |j| |j| [t|tr|j dt||s|j ddj||fS) uShared WHERE clause for bulk prune/archive selection (alias ``s``): ``_PRUNE_FILTERS`` AND together, only ended sessions are ever candidates, ``archived`` is tri-state (None = both), ``*_like`` are case-insensitive substrings, the rest exact. ``lineage_tips_only`` (bulk archive) drops compression ancestors: they are archived with their tip, never on their own age — matching an old ancestor would fan out over the lineage and hide its OPEN, recently active tip (#115489).zQSessionMaintenanceMixin._prune_filter_where() got an unexpected keyword argument rs.ended_at IS NOT NULLz+COALESCE(s.end_reason, '') <> 'compression'r1z s.archived = zCOALESCE(s.pinned, 0) = 0r) set_PRUNE_FILTER_NAMESr+sortedappend_PRUNE_FILTERSgetboolextend isinstanceintjoin) rPrQrRfiltersunknownclausesrrMappliesbuildr new_clauses new_paramss r_prune_filter_wherez+SessionMaintenanceMixin._prune_filter_wheresg,!44 006w0B/EGH H+,  NNH I$2 * D'5KK%E&-&:!U */,' Z{+ j)  * h % NN]3x=/: ; NN6 7||G$f,,rc|S|jdB|jd1|dkrtd|dtj|dzz |d<|jdd|i|S) zRTranslate the legacy age window into the shared activity filter, then build WHERE.r0r6rz"older_than_days must be >= 0, got zO: a negative retention builds a future cutoff that matches every ended session.rVr8r$)rr,rdr)rbolder_than_daysr8rs r _prune_wherez$SessionMaintenanceMixin._prune_wheres  'GKK8L,M,UKK 019" 88KLYYZZ-1IIK?U;R,SG( )'t''AvAAArc |j|||\}}|jdtd|d|Dcgc] }t|c}Scc}w)zDry-run: sessions a matching prune/archive would touch, oldest first (``older_than_days`` = inactivity threshold: freshest of ``last_activity_at`` / latest message / ``started_at``).zRSELECT s.id, s.source, s.title, s.model, s.started_at, z~ AS last_active, s.ended_at, s.message_count, s.archived FROM sessions s WHERE z? ORDER BY last_active ASC, s.started_at ASC)r _read_allr4dict)rbrr8rwhererrs rlist_prune_candidatesz-SessionMaintenanceMixin.list_prune_candidatessq))/67K v%)^^,-.++0'2? BDJ &KLcS L LLsA c p|j|||\}}t|jd||dS)zNCount-only :meth:`list_prune_candidates` (CLI reports spared pinned sessions).&SELECT COUNT(*) FROM sessions s WHERE r)rr _read_one)rbrr8rrrs rcount_prune_matchesz+SessionMaintenanceMixin.count_prune_matchessA))/67K v4>>$J5'"RTZ[\]^__rc |j|||\}}d}|j|s tdd|t|d}t |j d||dS)zZCount open sessions a matching prune skips (``ended_at`` guard inverted); visibility-only.rz0prune filter lost its ended-session safety guardzs.ended_at IS NULLNrr)r startswith RuntimeErrorrgrr)rbrr8rrr ended_guard open_wheres rcount_open_prune_matchesz0SessionMaintenanceMixin.count_open_prune_matchessz))/67K v.  ,QR R)%K0@0A*B)CD 4>>$J:,"WY_`abcddr)r|c||dkrytjt|dzz }|rdnd}|jd|dtd|j|f}|D]}|j |ddt |S) uArchive every session untouched for ``idle_days`` (freshest of ``last_activity_at`` / latest message / ``started_at``); may archive unended sessions. ``archived = 0`` makes repeats no-ops; only lineage tips (``end_reason <> 'compression'``) are candidates — a stale tip archives its chain via :meth:`set_session_archived`, so an old compressed-away root with a recent continuation is never matched. The hidden canonical Bot Chat (same predicate as :meth:`set_session_pinned`) is exempt: only a deliberate archive may retire it, since archiving releases its registry title to the next Bot open.r@zAND s.pinned = 0rz SELECT s.id FROM sessions s WHERE s.archived = 0 AND COALESCE(s.end_reason, '') <> 'compression' zd AND NOT (COALESCE(s.hidden, 0) <> 0 AND COALESCE(s.title, '') = ?) AND z7 < ? ORDER BY s.started_at ASC T)rdr*rr4CANONICAL_BOT_CHAT_TITLEset_session_archivedrg)rb idle_daysr|ra pin_clauserrs rarchive_stale_sessionsz.SessionMaintenanceMixin.archive_stale_sessionss   A uY/'99+9'r ~~l#$%  //8 : 4C  % %c!fd 3 44yrZc  j|||\ g  fd}j|} D]}j|||S)aDelete ended sessions inactive for ``older_than_days`` (an explicit ``started_before`` / ``last_active_before`` overrides it; None = no implicit bound) matching the filters. Children outside the window are orphaned (parent NULLed), not cascade-deleted. With *sessions_dir*, transcript files are removed outside the DB transaction. ``exclude_active_write_guards`` (automatic maintenance) skips rows under a live turn lease or compression lock while expired/dead holders are reclaimed and fenced.c |jd  }|jDchc]}|d }}r'||Dchc]} j||dr|c}z}|syt|D]`}t |}|jd|d||jd|d||jd |d|j |b j |t|Scc}wcc}w) Nz"SELECT s.id FROM sessions s WHERE rT)allow_closed_compression_parentrzIUPDATE sessions SET parent_session_id = NULL WHERE parent_session_id IN (rYz*DELETE FROM messages WHERE session_id IN (rX)rZr[rvr r rr\rg) r]cursorr session_idsrjr`phexclude_active_write_guardsrirbr where_paramss rrcz3SessionMaintenanceMixin.prune_sessions.._dos\\$Fug"NP\]F060AB3t9BKB*{ o#'#<#C$8 C!CC!$C.c ddlm}m}ddddd} ||dkrtj d|d| d<| S||j } | d| d<| S t j } t| |jd } | | |d zkrd| d<| || Std d |j||dx| d<} td d |jt|dz|jdd}t|| d<t| |jd}|duxs||dzk\}|r| dkDr|r|jx| d<}ddlm}m}||j ||j |z}|rHt|| d<tj'dt|dj)d|ddDnc|||kDr? td d |j+d| d<|j-dt/| ntj'd!|d"z|d"z|j-d t/| |s| dkDr)tj3d#t|| || drd$nd%|| | S#t0$r }tj d |Yd}~yd}~wwxYw#t0$r.}tj d&|t/|| d'<Yd}~hd}~wwxYw#|| wxYw)(aIdempotent startup auto-maintenance (never raises): prune inactive sessions, reap stale open state-owned rows, optional VACUUM. Runs at most once per ``min_interval_hours``; VACUUM has its own ``min_vacuum_interval_days`` throttle and also requires ``freelist_count / page_count`` > ``min_vacuum_freelist_ratio`` so a small prune on a dense multi-GB database never triggers a full rewrite. Stale-open reconciliation: cron/kanban/subagent/one-shot CLI rows never set ``ended_at`` when their process dies and prune only deletes ended rows, so after pruning, open rows from :attr:`_AUTO_PRUNE_STALE_OPEN_SOURCES` older than ``retention_days`` are closed (``startup_orphan_reap``); they stay resumable and age from their close. Returns ``{"skipped", "pruned", "closed", "vacuumed"}`` plus ``"freelist_ratio"`` when a VACUUM was considered and ``"error"`` on failure. Records the last run timestamp in state_meta so subsequent calls within ``min_interval_hours`` no-op. Designed to be called once at startup from long-lived entrypoints (CLI, gateway, cron scheduler). See #54189. When *sessions_dir* is provided, on-disk transcript files (``.json`` / ``.jsonl`` / ``request_dump_*``) for pruned sessions are removed as part of the same sweep (issue #3015). Messaging and UI sources are never touched here. See #54189. r)_release_auto_maintenance_lock"_try_acquire_auto_maintenance_lockF)skippedprunedclosedvacuumedNzstate.db auto-maintenance skipped: sessions.retention_days=%r is outside the allowed range (a whole number of days >= 0); set sessions.auto_prune: false to disable pruningTrlast_auto_pruneig @state_db_auto_prune)phase)rrhrrstate_db_auto_sweepr)rrzr|rr last_vacuumrVfreelist_ratio)foreign_state_db_holdersin_process_state_db_holders)excludevacuum_skipped_holderszestate.db auto-maintenance: skipping VACUUM, %d other process(es) hold the store or a WAL sidecar (%s)z, c30K|]\}}|d|yw):Nr$)rLpidtargets rrOzFSessionMaintenanceMixin.maybe_auto_prune_and_vacuum..s/akc63%q0A/asstate_db_auto_vacuumrzstate.db VACUUM failed: %sz_state.db auto-maintenance: skipping VACUUM, only %.1f%% of pages reclaimable (threshold %.0f%%)gY@zgstate.db auto-maintenance: closed %d stale open session(s), pruned %d session(s) inactive for %d days%sz + VACUUMrz$state.db auto-maintenance failed: %serror)hermes_state_repairrrrrdb_pathrdr/get_metarrrr*_AUTO_PRUNE_STALE_OPEN_SOURCESrgrhermes_state_holdersr r rrrset_metarrinfo)rbretention_daysmin_interval_hoursrrhmin_vacuum_interval_daysmin_vacuum_freelist_ratiorrresultmaintenance_lockr- since_prunerr since_vacuum vacuum_dueratior r holdersrs rmaybe_auto_prune_and_vacuumz3SessionMaintenanceMixin.maybe_auto_prune_and_vacuumss/. k-2a1Z_!`  !^a%7 NNi !%F9 M=dllK  # $F9 ME =))+C(dmmLL"R!&0IE0QS MM+SX 6! JKPVWaPbhjl ++; < %%J'CSIIJ ' NNA3 G!#hF7O ' ++; W\6;-/3-K[--: B L,@ L`*-` e/2eRV4SW`e%3Y]%(B A ]( <%'"UY'+R+Il!lr=sW" 33<   > *+C0L B ." :9')T*:W*D%EFy$'9":;i&9!:; xn-.8T"SUZ[\8T"4568/0Y%; <=Y%; <=8T"SUZ[\4 MsyyYZ$/0$/0(D!234Hd#Y[`ab9d;#89:9d;#89:DW!456DW!456y$'L"MNy$'L"MN58 F~FFI||QQr