߲jGZdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl mZddlmZddlmZddlmZmZmZmZddlmZerdd lmZej8d Zdd Z dd Zd d Z dZ!d dZ"d!dZ#d"dZ$d#dZ%d$dZ&d$dZ'd$dZ(d$dZ) d%dZ*d&dZ+d'dZ, d( d)dZ-d*dZ. d( d+dZ/ddl0m1Z2y),aCron pre-run script execution: timeouts, Windows venv bootstrap, process-tree termination, and the claim-heartbeat thread that keeps a long script's run claim alive. Split out of ``cron.scheduler``. Import names from this module directly (``cron.scheduler`` only imports the few it calls itself). Origin-resident helpers and sibling split modules are reached late-bound (``_sched`` / module refs at the bottom) so monkeypatching the defining module works. ) annotationsN)cron_env_setting)_ensure_cron_dir)Path)AnyCallableOptional TYPE_CHECKING)windows_hide_flags)_CancelEventLikezcron.schedulerc<tt|}|dkDr|SdS)zD``int(float(raw))`` when > 0, else None; raises on unparsable input.rN)intfloat)rawtimeouts :/home/ubuntu/.hermes/hermes-agent/cron/scheduler_script.py _positive_intr#s %*oGk7+t+ct|j}|r ||}||S t j xsi}t|tr|jdini}|j|}| ||}||Sy#t$rtj d||YwxYw#t$r!} tjd|| Yd} ~ yd} ~ wwxYw)uShared env → ``cron.`` resolution. ``parse`` returns the value or None to keep looking; a parse error on the env var WARNs, on config DEBUGs. None when neither yields.Nz#Invalid %s=%r; using config/defaultcronz!Failed to load %s from config: %s) rstrip Exceptionloggerwarning_sched load_config isinstancedictgetdebug) env_var config_keyparselabel env_valuevaluecfgcron_cfg configuredexcs r_timeout_from_env_or_configr+)s!)//1I V)$E  ! F  "(b*4S$*?37762&R\\*-  !*%E    V NN@'9 U V F 8%EE Fs) BAB- B*)B*- C6CCc4tjtjk7r ttj}||S tddtd}|tjS|S#t$r't j dtjYUwxYw)zOResolve cron pre-run script timeout from module/env/config with a safe default.z H.6-=6 ) )K8K ( NNN&& ( (sA''-BBi,c:tddtd}|tS|S)zPer-attachment media-send timeout: HERMES_CRON_MEDIA_SEND_TIMEOUT env, then ``cron.media_send_timeout_seconds``, then 300s (long TTS audio can exceed a 30s window).HERMES_CRON_MEDIA_SEND_TIMEOUTmedia_send_timeout_secondszcron media-send timeout)r+r_DEFAULT_MEDIA_SEND_TIMEOUTr1s r_get_media_send_timeoutr8Ws,+(*F !#H+3*: &HHrc2tddtd}|dS|S)aBound on run_job's SessionDB init: HERMES_CRON_SESSION_DB_TIMEOUT env, then ``cron.session_db_timeout_seconds`` (in DEFAULT_CONFIG), then 10s. Unlike sibling timeouts, 0 is meaningful (unlimited, debugging opt-in), so values pass through untouched.HERMES_CRON_SESSION_DB_TIMEOUTsession_db_timeout_secondszcron.session_db_timeout_secondsg$@)r+rr7s r_get_session_db_timeoutr<`s++(*F)+H#411rc |dz jdj}d|DDcic]2\}}|jj |j4c}}S#t$ricYSwxYwcc}}w)Nz pyvenv.cfgutf-8)encodingc3JK|]}d|vs|jddyw)=N)split).0rs r z+_read_windows_pyvenv_cfg..qs!L399S!,Ls ##) read_text splitlinesOSErrorrlower)venv_dirlineskeyr&s r_read_windows_pyvenv_cfgrMjsL(33W3EPPR ML  C U[[]*    s#A,7A=, A:9A:cDtjdk7r|ifStj|}|jj}i}|j j dk(r#|jd}|jr|}t|}|jdd}|dz dz }d|vr|rtj|dz }|jr|jr|}t||d <ttjtjjd t|g} tj jd d} | r| j#| tj$j'| |d <t||fS) aHidden, output-capable Python invocation for Windows cron scripts. ``pythonw.exe`` loses captured output; uv venv launchers can re-exec the base console python and flash a window even with CREATE_NO_WINDOW, so run the base python directly with venv paths overlaid in env.win32z pythonw.exez python.exehomeLib site-packagesuv VIRTUAL_ENVrB PYTHONPATH)sysplatformrrparentnamerI with_nameexistsrMrstr__file__resolveparentsosenvironappendpathsepjoin) python_exe interpreterrJ env_overlaysiblingr'rP site_packages base_pythonpythonpath_entriesexisting_pythonpaths r_windows_cron_python_invocationrnusg ||w2~++j)K!!((H"$K=0'' 5 >> !K "8 ,C 7762 Du$6M s{tkk$',6    M$8$8$:%K),XK &FKK)113;;A>?]AS"U "$**..r"B ""))*=>(* 8J(KK % { [ ((rc|jytjdk7r t|n; t j ddt |jddgddtd  |jd y#ttjf$r|jY?wxYw#tj$r%|j|jd YywxYw) zBBest-effort hard stop of a cron script and every child it spawned.NrOtaskkillz/PIDz/Tz/FT F)capture_outputr creationflagscheck?r) pollrWrX_terminate_process_group subprocessrunr]pidr rHTimeoutExpiredkillwait)procs r_terminate_cron_script_processrs yy{ ||w &  NNVS]D$?PT^`02% A  # 223  IIK   $ $  # s#:B-B/)B,+B,/5C'&C'c  tj|j}tj|tj tjtj5|jdddd tj|dtjt ttf5tj|ttdtj dddy#t ttf$rYywxYw#1swYxYw#t tf$rYywxYw#1swYyxYw)zPOSIX: TERM the script's process group, then KILL if ANY member survived (a survivor holds the pipe write ends open and the caller's communicate() would block on EOF forever).NrurvrSIGKILL)ragetpgidr{killpgsignalSIGTERMProcessLookupErrorPermissionErrorrH contextlibsuppressryr|r~getattr)r process_groups rrxrxs 488,  -0   Z66 7 #  -#   0/7K LM -FNN!KLMM  9  (MMs<AC;(DD!>4D6;DDD!D32D36D?c,jyddd fd}tdd}t|tr|dkr |d|S ddlm} ||ry |d |y#t $r |d d cYSwxYw#t $r|d |d cYSwxYw)zHTerminate a script tree, then fall back to the local process-group path.NFexc_infocTtj|dzg|d|ity)Nz+; falling back to process-group terminationr)rrr)reasonrargsrs rfallbackz-_terminate_cron_script_tree..fallbacks3 B B ^EI ^T\ ^&t,rr{rz-Cron script tree-kill received invalid pid %r)kill_process_treez,agent.deadline.kill_process_tree unavailableTz'Cron script tree-kill failed for pid %sz3Cron script tree-kill reported no signal for pid %s)rr]rboolreturnNone)rwrrragent.deadlinerr)rrr{rs` r_terminate_cron_script_treers yy{6;- $t $C c3 3!8GMMW 5W S !  " BCH WFQUVVW WA3QUVVWs$A#A<#A98A9<BBchtjtj5|j d dddy#1swYnxYwtjt 5|j dddn #1swYnxYw|j|jfD]B}tjt 5||jddd8#1swYAxYwtjtj5|jddddy#1swYyxYw)zReap a terminated script without blocking forever: a surviving descendant can hold the pipe write ends open, so bound the drain and abandon the pipes (output is not needed).g@rvN) rrryr| communicaterHr}stdoutstderrcloser~)rstreams r_drain_script_pipesrs   Z66 7 %   W % ;; ,   ) !       Z66 7 # s/AA &BB CC% D((D1ctj|jdddz dz }|jstj d|||gSdt |d}|d||gS) axBootstrap a cron script under the base interpreter with ``.pth`` support. Overlay mode puts the venv on ``PYTHONPATH``, but ``.pth`` files are only processed by ``site.addsitedir()``, so editable installs would be invisible; bootstrap via addsitedir + ``runpy.run_path`` (keeps ``__file__``/``sys.path[0]`` semantics). Plain invocation if the venv is unresolvable.rUrQrRrSz}Windows cron script: venv site-packages %s not found; running without .pth processing (editable installs may be unimportable)z,import os, runpy, site, sys;site.addsitedir(z);script = sys.argv[1];sys.argv = [script] + sys.argv[2:];sys.path.insert(0, os.path.dirname(os.path.abspath(script)));runpy.run_path(script, run_name='__main__')z-c)rrris_dirrrr])rfrh script_pathrj bootstraps r_windows_cron_bootstrap_argvrs KK  r BCeKo]M    ! N  K(( }-016 6 i 55rc@tjdz }t||j}dt |vrdd|fS tj |j }|jr|jn||z j} |j||js dd|d |d fS|jsdd |fS|dfS#tttf$r dd|fcYSwxYw#t$r dd|d|fcYSwxYw) uValidate a job script path; ``(path, None)`` or ``(None, error)``. Scripts MUST resolve inside HERMES_HOME/scripts/ (relative, absolute and ``~`` paths are all validated — path traversal / absolute-path injection); contract of lifecycle_guard._expand_candidate_path.scriptsNz*Blocked: script path contains a NUL byte: z5Blocked: script path is not a valid filesystem path: z=Blocked: script path resolves outside the scripts directory (z): zScript not found: z<. Cron scripts are looked up only in this profile's folder (zr); if the job was copied from another profile, copy the script there too, or edit the job with `hermes cron edit`.zScript path is not a file: )r_get_hermes_homerr_r]r expanduser ValueError RuntimeErrorrH is_absolute relative_tor\is_file)r scripts_dirscripts_dir_resolvedrpaths r_resolve_script_pathrsg))+i7K[!&..0[!!A+QQQ]kk+&113 OO-3;;=K#4E3N3N3PD  -. ;;= '$%&B C  <<>24&999 :/  g .]L[O\\\]  $%S 9   s$#C&D&DDDDc|jjdvr_tjdxs#tj j drdnd}|did|jdfS|t|gidfSttj\}}|rt||t||dfS|t|g|dfS)u``(argv, env_overlay, error)`` for a validated script. Interpreter by extension — the shebang is deliberately NOT honoured (small, auditable surface): ``.sh``/``.bash`` → bash, else ``sys.executable`` (Windows uv-venv overlay gets the .pth bootstrap).>.sh.bashbashz /bin/bashNzCannot run .sh/.bash script z{: bash not found on PATH. On Windows, install Git for Windows (which ships Git Bash) or rewrite the script as Python (.py).) suffixrIshutilwhichrarisfilerZr]rnrW executabler)r_bashrfrhs r _script_argvr/s {{.. V$^{8SY] =.tyym<99  s4y!2t++=cnnMJ +J SYOQ\^bbb D "K 55rc t|\}}|d|fSt}t|\}}}|d|fS ddlm}ddd} t j dk(r tttd dzd dd } |d } | j|tj|ftjtjd|xst|j| d | } tj |z} |'|j#rt%| t'| y| tj z } | dkr t%| t'| dd|d|fS | j)t+d| \}} |xsdj/}|xsdj/} ddlm}||}||}| j:dk7rOd| j:g}|r|j=d||r|j=d|ddj?|fSd|fS#tj,$rY>wxYw#t4$r$}t6j9d|dx}}Yd}~d}~wwxYw#t4$r}dd|fcYd}~Sd}~wwxYw)uExecute a cron job's script and return ``(success, output)``; on failure *output* is the error message for the LLM to report. Env goes through ``build_subprocess_env`` (SECURITY.md §2.3). ``workdir`` sets the subprocess cwd only; the Python process cwd is NEVER mutated (an ``os.chdir()`` would leak into concurrent gateway sessions). Args: script_path: Path to the script. Relative paths are resolved against HERMES_HOME/scripts/. Absolute and ~-prefixed paths are also validated to ensure they stay within the scripts dir. workdir: Optional absolute path to use as the script's cwd. When set, the subprocess runs in this directory instead of the scripts-dir parent. See #69396. NFr)build_subprocess_envTreplace)start_new_sessionerrorsrOCREATE_NEW_PROCESS_GROUPr>)rsr?r)strip_launch_profile)rrtextcwdenv)Fz5Script cancelled because cron fire ownership was lostzScript timed out after zs: g?rvrQ)redact_sensitive_textz/Failed to redact sensitive text from output: %sz[REDACTED - redaction failed]zScript exited with code zstderr: zstdout:  zScript execution failed: ) rr2rtools.environments.localrrWrXr rryupdatePopenPIPEr]rYtime monotonicis_setrrrminr|r agent.redactrrrr returncodercre)rworkdir cancel_eventrerrscript_timeoutargvrhr popen_kwargsrrdeadline remaining stdout_raw stderr_rawrrrepartsr*s r_run_job_scriptrCs%[1ID# |cz(*N)$/D+s |czQ8A"&(  <<7 "!3!5*&@!D"E $#%L#= ;  F#t+3t{{+F8DF>>#n4'L,?,?,A+D1#D)U 4>>#33IA~+D1#D) 77Gs4&QQQ )-)9)9#c9BU)9)V& J"))+"))+ > :*62F*62F ??a //@ABE y12 y12$))E** *V|-,,   > NNLa P= =FV > 81#7778syC'I%;I%H9)I%#H59AI%I%H2.I%1H22I%5 I">II%I""I%% I?.I:4I?:I?ctjtjj|f|d} |j |S#t $r |YywxYw)zStart ``loop_fn`` on a daemon thread inside a copy of the current context (multiplexed profile ContextVars). On failure calls ``fail_log()`` inside the except (traceback intact) and returns None.T)targetrrZdaemonN) threadingThread contextvars copy_contextrzstartr)loop_fnrZfail_logthreads r_start_heartbeat_threadrs^  '')--WJTRVXF  M  sA AAcx |jd}|jd}t|trt|jdxsdnd t|tr|jddk(r st |||St|jdxsdt j d fd }t|d fd }|t |||S t ||| j|jd S# j|jd wxYw)uxRun a cron script while heartbeating its owned one-shot claim. A long script can outlive the stale-claim TTL; without a heartbeat another scheduler would re-dispatch the one-shot. Recurring/unclaimed runs have no durable claim → no thread. The owner is captured from the dispatched job, never re-read, so a stale runner cannot extend a replacement owner's claim.schedule run_claimbyrQkindonce)rridcjtjs9 tjjtjs8yy#t$rt j ddYEwxYw)N)expected_ownerz+Job '%s': script run_claim heartbeat failedTr)r~r_RUN_CLAIM_HEARTBEAT_SECONDSheartbeat_run_claimrrr )job_idownerstopsr_heartbeat_loopz=_run_job_script_with_claim_heartbeat.._heartbeat_loopsf))F??@ c**6%H))F??@ c JF]a b csA!A?>A?zcron-script-claim-heartbeatc4tjddS)Nz4Job '%s': could not start script run_claim heartbeatTr)rr )rsrz6_run_job_script_with_claim_heartbeat..s BFUY[rrurv)rr) rrrr]rrEventrsetre) jobrrrrclaimrheartbeat_threadrrrs @@@r$_run_job_script_with_claim_heartbeatrswwz"H GGK E*4UD*AC $%2 &rE x &8<<+?6+Ie{G,WW $" %F ?? Dc/6 [ {G,WW+{G,W  c*  c*s % D$D9) scheduler)rz Optional[int])r!r]r"r]r#zCallable[[Any], Any]r$r])rr)rr)rJrrdict[str, str])rfr]rztuple[str, dict[str, str]])rzsubprocess.Popenrr)rfr]rhrrr]rz list[str])rr]rz$tuple[Optional[Path], Optional[str]])rrrz9tuple[Optional[list[str]], dict[str, str], Optional[str]])NN)rr]r Optional[str]rOptional[_CancelEventLike]rtuple[bool, str])rZr]rzOptional[threading.Thread]) rrrr]rrrrrr )3__doc__ __future__rrrloggingrarrryrWrrcron.env_settingsr cron.jobsrpathlibrtypingrrr r hermes_cli._subprocess_compatr cron.schedulerr getLoggerrrr+r2r6r8r<rMrnrrxrrrrrrrrrrrrrrse#   .&99</   + ,,  !*>GJ2L$"I2)D(M$I6  66"06?B6GP62+\6*04/3g8g8,g8,g8g8T ;?/3%+ %+%+*7%+,%+%+T%r