߲j^XdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddlmZddlmZddlZddlmZddlmZej.eZe j4Ze j8ZdZdZee e gdfZ!ee"e#e e fgdfZ$eegdfZ%ee"e gdfZ&ege"e#e e ffZ'd Z(d Z)d Z*d Z+d Z,dZ-de.fdZ/de fdZ0de de1fdZ2d%dedzde.fdZ3d&de de"e#e e ffdZ4de de fdZ5de6e fdZ7de"e de fdZ8de"e de fdZ9de"e#e e fde"e fd Z:d!e de fd"Z;Gd#d$Z=A>c<ttjdS)zTemp prefix embedding the owning PID so the stale sweep can tell a hard-killed process's leftovers from another live gateway's in-flight transfer without guessing from mtime (a staging dir's mtime does not move while content streams into it).-)_SYNC_BACK_TEMP_PREFIXosgetpidr_sync_back_temp_prefixr$Ns% %biik]! 44r#namec|ttdjddd}|jsyt j t |S)zWhether the process that created a sync-back temp entry may still be running. Names without a PID count as alive: the age cutoff applies.NrrT)lenrsplitisdigitpsutil pid_existsr)r%pid_parts r_temp_entry_owner_aliver.UsMC./0177Q?BH       S] ++r#temp_dirc|xsttj}tjtz }d} t |j td}|D]} |jrt|jr|jj|k\rH|j!rt#j$|n|j'|dz }tjd||S#t$rtjd|YywxYw#t$rtjd|YwxYw)aoRemove sync-back tars and staging dirs left behind by a hard-killed process. Only entries carrying this module's prefix are touched: at once when their owner PID is dead, otherwise only past ``_SYNC_BACK_STALE_SECONDS``. Returns the number of entries removed; a permission error or a race with another sync-back must not prevent the current one. r*z0sync_back: could not scan temporary directory %sr'z+sync_back: removed stale temporary entry %sz4sync_back: could not remove stale temporary entry %s)rtempfile gettempdirtime_SYNC_BACK_STALE_SECONDSlistglobrOSErrorrdebug is_symlinkr.r%lstatst_mtimeis_dirshutilrmtreeunlink)r/ directorycutoffremoved candidates candidates r_cleanup_stale_sync_back_temprF^s'7D!4!4!67I YY[3 3FG)..,B+C1)EFG  \  \##%&y~~69??;L;U;UY_;_! i(  " qLG LLF R \ N#  GS \ LLOQZ [ \s1!D'D%82D%+AD%D"!D"%EEcontainer_basec ddlm}m}m}|Dcgc]}|d|dj d|df}}|g||||Dcgc] }|d|dfc}z }|Scc}wcc}w)zEnumerate all (host_path, remote_path) pairs to sync to a remote. Credential paths are remapped from the hardcoded /root/.hermes to *container_base* (remote home may differ).r)get_credential_file_mountsiter_cache_filesiter_skills_files host_pathcontainer_path /root/.hermesr')rG)tools.credential_filesrIrJrKreplace)rGrIrJrKentryfiless riter_sync_filesrSsgf01 3  { U#34<<_n^_`a 3E 3 H(GH'~FHI  { U#345IIE L 3Is "A(A-rLc tt|jjS#t$r%tt|jcYSwxYw)zVCanonical string form of a host path (``resolve()`` falling back to ``expanduser()``).)strr expanduserresolver8)rLs r_resolve_host_path_strrXsO14 ?--/779:: 14 ?--/001s/2+A A c ddlm}|}|Dchc]1}t |t r|j drt|d3c}S#t$r tcYSwxYwcc}w)zBReturn credential files that are upload-only for remote sandboxes.r)rIrL)rOrI Exceptionset isinstancedictrrX)rImountsrQs r_credential_host_pathsr_smE+-  S z%6599[;Q u[12 SS u  Ss A 6A% A"!A" remote_pathsc8ddjd|DzS)zz$quoted_rm_command..sD!u{{1~D!join)r`s rquoted_rm_commandrns chhD|DD DDr#dirsc8ddjd|DzS)z>Build a shell ``mkdir -p`` command for a batch of directories.z mkdir -p rbc3FK|]}tj|ywrdre)rhds rrjz'quoted_mkdir_command..s!?Q%++a.!?rkrl)ros rquoted_mkdir_commandrss !?$!?? ??r#rRc pt|Dchc]\}}tj|c}}Scc}}w)zCExtract sorted unique parent directories from (host, remote) pairs.)sorted posixpathdirname)rR_remotes runique_parent_dirsrzs* eDF9$$V,D EEDs2 pathctj}t|d5tfddD]}|j | ddd|j S#1swY|j SxYw)z$Return hex SHA-256 digest of a file.rbc&jdS)Ni)read)fsrz_sha256_file..s!&&-r#r#N)hashlibsha256openiterupdate hexdigest)r{hchunkrs @r _sha256_filersiA dD Q/5 E HHUO  ;;= ;;=s &A!!A9ceZdZdZeddfdedededede dzde dzf d Z d d d e d dfdZ d d d e d dfdZdeeeefd eeeeefeeeeeffeeffdZdeeeefdeed dfdZd&dedzd dfdZd&dedzd dfdZded dfdZded dfdZd'dZdededeeeefdeed ef d Zd&dedeeeefdzd edzfd!Zd&dd"dedeeeefdzdeedzd edzfd#Ze d$edeed e fd%Z!y)(FileSyncManageraTracks local file changes and syncs to a remote environment. Backends supply transport callbacks (upload, delete) and a file-source callable; the manager handles mtime-based change detection, deletion tracking, rate limiting, and transactional state.N get_files_fn upload_fn delete_fn sync_intervalbulk_upload_fnbulk_download_fnc||_||_||_||_||_t j |_i|_i|_ t|_ d|_ ||_ y)Ng) _get_files_fn _upload_fn_bulk_upload_fn_bulk_download_fn _delete_fn threadingLock_transaction_lock _synced_files_pushed_hashesr[_upload_only_host_paths_last_sync_time_sync_interval)selfrrrrrrs r__init__zFileSyncManager.__init__sc*#-!1#!*!1;=.014$&)+r#Fforcerr cj|j5|j|dddy#1swYyxYw)a1Run a sync cycle: upload changed files, delete removed files. Rate-limited to once per ``sync_interval`` unless *force* or ``HERMES_FORCE_FILE_SYNC=1``. Transactional: state is committed only if ALL operations succeed; on failure it rolls back so the next cycle retries everything.rN)r_sync_transaction)rrs rsynczFileSyncManager.syncs3  # # 0  " " " / 0 0 0)2cT|sHtjjts%t |j z |j kry|j}|jjt|j|\}}}|s|st |_yt|j}t|j} tj d5}g} i} t#|D]h\} \} } tj$j'|t)| }t+j,| |t/|| | <| j1|| fj|j3| |ddd|jj |D]0}|j5|d|jj5|d2||_ t |_y#1swYpxYw#t6$r.}||_ ||_t8j;d|Yd}~yd}~wwxYw)z:Execute one sync cycle while holding the per-manager lock.Nzhermes-sync-push-prefixz-file_sync: sync failed, rolled back state: %s)r environr_FORCE_SYNC_ENV _monotonicrrrrrr_ _plan_syncr]rrr2TemporaryDirectory enumerater{rmrUr>copy2rappend_pushpoprZrr)rr current_files to_upload new_files to_delete prev_files prev_hashesstaging staged_files pushed_hashesindexrL remote_path staged_pathriexcs rrz!FileSyncManager._sync_transactionsJJNN?3 t333d6I6II **,  $$++,B,DE*.//-*H' 9i#- ``(to_upload, new_synced_state, to_delete)``.)r]rrrr) rrrrrLrfile_keyrxrycurrent_remote_pathsrirs rrzFileSyncManager._plan_syncs,. ++, &3 . "I{&y1H4#5#5#9#9+#F(#R   i5 6%-Ik "  . 9FF91fFF $ 2 2T1a?S6SQT T)Y.. GTs+ B B#B#rrc|rtjdt||rtjdt||r=|j1|j|tjdt|n3|D].\}}|j ||tjd||0|r(|j |tjd|yy)zCRun the transport calls for one cycle (bulk upload when available).zfile_sync: uploading %d file(s)z+file_sync: deleting %d stale remote file(s)Nz#file_sync: bulk-uploaded %d file(s)zfile_sync: uploaded %s -> %szfile_sync: deleted %s)rr9r(rrr)rrrrLrs rrzFileSyncManager._pushs  LL:C N K  LLFI W --9   + LL>I O*3 U& ; ;7 ;Y T U  OOI & LL0) < r# hermes_homecj|j5|j|dddy#1swYyxYw)aPull remote changes back to the host: download the remote ``.hermes/`` as a tar and apply only files whose SHA-256 differs from what was pushed. SIGINT is deferred until complete; concurrent gateway sandboxes are serialized via a file lock.)rN)r_sync_back_transaction)rrs r sync_backzFileSyncManager.sync_back*s7 # # A  ' 'K ' @ A A Arc|jy|js"|jstj dy|xs t dz }|j jddd}ttD]} |j|ytjdt|y#t$rG}|}|tdz kr/t|}tjd|dz||t|Yd}~d}~wwxYw) zLExecute sync-back (with retries) against a stable snapshot of manager state.Nu+sync_back: no prior push state — skippingz .sync.lockT)parentsexist_okr'z2sync_back: attempt %d failed (%s), retrying in %dsz%sync_back: all %d attempts failed: %s)rrrrr9rparentmkdirrange_SYNC_BACK_MAX_RETRIES_sync_back_oncerZ_SYNC_BACK_BACKOFFr_sleep)rr lock_pathlast_excattemptrdelays rrz&FileSyncManager._sync_back_transaction1s  ! ! ) ""4+=+= LLF G  5O$5E td;%)34 "G "$$Y/ " >@VX`a "3a77.w7ENN#WY`cdYdfikpq5M  "s?B// C?8=C::C?rchtjtju}gd}|rLtjtj }fd}tjtj | |j ||rN|Ktjtj |r$tjtj yyyy#|rN|Ktjtj |r$tjtj wwwwxYw)z>Single sync-back attempt with SIGINT protection and file lock.NcVj||ftjdy)Nz/sync_back: SIGINT deferred until sync completes)rrr9)signumframedeferred_sigints r _defer_sigintz6FileSyncManager._sync_back_once.._defer_sigintXs"&&7 NOr#)rcurrent_thread main_threadsignal getsignalSIGINT_sync_back_locked raise_signal)rron_main_threadoriginal_handlerrrs @rrzFileSyncManager._sync_back_onceMs#113y7L7L7NN(* %// >  P MM&-- 7 7  " "9 -"2"> fmm-=>"'' 6 ##?~~"2"> fmm-=>"'' 6 ##?~s =CAD1ct|jyt|dd} tj|tj|j tj|tj |jy#t tf$rY"wxYw# tj|tj n#t tf$rYnwxYw|jwxYw)z;Sync-back under file lock (serializes concurrent gateways).Nwzutf-8)encoding) fcntl_sync_back_implrflockLOCK_EXLOCK_UNr8IOErrorclose)rrlock_fds rrz!FileSyncManager._sync_back_lockedhs =  " y#8  KK /  "  GU]]3 MMOW%    GU]]3W%   MMOsA4B&$BB#"B#&C3($C  C3 CC3CC3c |j td t|j}t t jtd\}}tj| |jt| tjj|}t}||kDr4t j#d||t$ tj&|yt j(t5}t+j,|5}|j/|d ddd|j0t3z}d} tj4|D]~\} } } | D]s} tjj7| | }d ttjj9||j;z}| |j=||||z } u| rt j?d | nt jAd ddd tj&|y#t$rg}YwxYw#t$rd}YwxYw#t$rYywxYw#1swY9xYw#1swY`xYw#t$rYywxYw# tj&|w#t$rYwwxYwxYw) z1Download, diff, and apply remote changes to host.Nz/_sync_back_impl called without bulk_download_fnz.tar)rsuffixru]sync_back: remote tar is %d bytes (cap %d, override with terminal.%s) — skipping extractionrdata)filter/z%sync_back: applied %d changed file(s)z%sync_back: no remote changes detected)!r RuntimeErrorr6rrZrFr2mkstempr$r rrr{getsizer8rrrrr@rtarfiler extractallrr_walkrmrelpathas_posix_apply_staged_fileinfor9)r file_mappingfdtar_pathtar_size max_bytesrtar upload_onlyapplieddirpath _dirnames filenamesfname staged_filers rrzFileSyncManager._sync_back_implys  ! ! )PQ Q  2 2 45L &' ''/E/GPVW H  $   " "4> 2 77??84-.I)#si)AC(  (#',,4J4LM JQX\\(+;sNN76N:;#::=S=UU 57WWW5Ep1GY !*p&(ggll7E&B &)Dg1V,W,`,`,b&b 4#:#:; Uacn#oo ppKK GQLL!HI! J$  (#[ L "  :  ';; J J(    (#  sH=4JI.+JI!1JI=%I09C%I=J'J = I  I  IJIJ! I-,I-0I: 5I==JJ JJJ?J0/J?0 J<9J?;J<<J?rrrupload_only_host_pathscN|jj|}|t||k(ry|j||}|-|j |||}|t j d|y|j||rt j d|y|Ctjj|r$t||k7rt jd|tjtjj|dtj||y) aCopy one extracted remote file onto the host if it changed since push. Returns 1 if applied, 0 if skipped (unchanged, unmapped, or an upload-only credential). A host file modified since push is overwritten with the remote version (last-write-wins) with a warning.rrz(sync_back: skipping %s (no host mapping)z2sync_back: skipping upload-only credential file %suwsync_back: conflict on %s — host modified since push, remote also changed. Applying remote version (last-write-wins).T)rr')rrr_resolve_host_path_infer_host_pathrr9_is_upload_only_host_pathr r{existsrmakedirsrwr>r)rrrrr pushed_hashrLs rrz"FileSyncManager._apply_staged_files ))--k:  "|K'@K'O++KF  --k<`v-wI  GU  ) ))5K L LLM{ [  "rww~~i'@\R[E\`kEk NN^  BGGOOI.> [),r#c6tfd|xsgDdS)zAFind the host path for a known remote path from the file mapping.c34K|]\}}|k(s |ywrdr")rhhostryrs rrjz5FileSyncManager._resolve_host_path..sZldFFkDYTZs N)next)rrrs ` rrz"FileSyncManager._resolve_host_pathsZl.@bZ\`aar#rc|xs t}|xsgD]s\}}|j||rtj|}|j |dzsCt t |j|t|dzdz cSy)zInfer a host path for a new remote file by matching path prefixes: an existing remote->host pair whose parent directory prefixes *remote_path* gets the same substitution (``/root/.hermes/skills/b.md`` -> ``~/.hermes/skills/b.md``).rr'N) r[rrvrw startswithrUrrr()rrrrrry remote_dirs rrz FileSyncManager._infer_host_paths "8!@35(.B RLD&--d4JK"**62J%%j3&674:,,{3z?Q;N;O/PPQQ  R r#rLct||vSrd)rX)rLrs rrz)FileSyncManager._is_upload_only_host_paths%i04JJJr#rd)r N)"__name__ __module__ __qualname____doc___SYNC_INTERVAL_SECONDS GetFilesFnUploadFnDeleteFnfloat BulkUploadFnBulkDownloadFnrboolrrr6tuplerUr]rrrrrrrrrr[rrr staticmethodrr"r#rrrseT 6.226, ,, ,  , %t+ ,)4/,(%*0T0d027*Q$*Q4*QX/!%S/2/ tE#s(O$d3eSj0A+A&BDIM N/ =tE#s(O4=c=t=$ATD[ADAb$+bb877$764D"7r-0@DU3PS8_@Uorsvow >bcbeCQTHoAVY]A]bilosisb CG C tE#s(O?TW[?[ 14SD LORVJ KSK#c(KW[KKr#rrd)rN)=r#rloggingr rvrfr>rrr2rr4r ImportErrorpathlibrtypingrr+hermes_constantsrtools.environments.baser getLoggerr rsleepr monotonicrr$rrUr&r6r,r)r*r'r%rrrrrr5rrr$r+r.rFrSrXr[r_rnrsrzrrr"r#rr7s-    ,3   8 $  ^^ * S#J$ %eCHo./56 4&$,' T#YK% & b$uS#X// 0 -0,#  c  55,#,$,D4K3B C d5c?>S  1c1c1 SC SEDIE#E @tCy@S@ Fd5c?3FS F ssjKjKK EsEE E