߲jk$bdZddlZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z ddl mZmZmZddlmZddlmZdd lmZmZej0eZd efd Zd efd Zd efd Zd efdZ ded dfdZ!d efdZ"d efdZ#ejHZ%ddeded efdZ&GddeZ'y)zSingularity/Apptainer persistent container environment. Security-hardened with --containall, --no-home, capability dropping. Supports resource limits and optional persistence via writable overlay dirs that survive sessions. N)Path)Optionalget_hermes_home)BaseEnvironment_load_json_store_save_json_store) _popen_bash)sanitize_task_id_for_path) bash_argv run_capturereturnctdz S)Nzsingularity_snapshots.jsonrC/home/ubuntu/.hermes/hermes-agent/tools/environments/singularity.py_snapshot_storers  ; ;;rcXdD]}tj|s|cStd)z/Locate the apptainer or singularity CLI binary.) apptainer singularityzNeither 'apptainer' nor 'singularity' was found in PATH. Install Apptainer (https://apptainer.org/docs/admin/main/installation.html) or Singularity and ensure the CLI is available.)shutilwhich RuntimeError)exes r_find_singularity_executablers6+ << J  : ;;rcHt} t|dgd}|j dk7r;|jjd d }td|d |j d ||S#t$rtd|dtj $rtd|dwxYw) z?Preflight check: resolve the executable and verify it responds.version timeoutz"Singularity backend selected but 'z' could not be executed.'z version' timed out.rNz version' failed (exit code z): ) rr FileNotFoundErrorr subprocessTimeoutExpired returncodestderrstrip)rresultr's r_ensure_singularity_availabler*)s & (C:c9-r: A$$&t,Qse#?@Q@Q?RRUV\U]^__ J _?uD\]^^  $ $:Qse#7899:s A'':B!c(ttSN)rrrrr_load_snapshotsr-8s O- ..rdatac,tt|yr,)r r)r.s r_save_snapshotsr0<s_&-rctjd}|r t|}nddlm}|dz }td}|j ritj |tjrE|tjddz dz }|jd d tjd ||jd d |S) z\``TERMINAL_SCRATCH_DIR`` override, else a writable ``/scratch`` (HPC), else the sandbox dir.TERMINAL_SCRATCH_DIRr)get_sandbox_dirrz/scratchUSERhermesz hermes-agentTparentsexist_okz Using /scratch for sandboxes: %s) osgetenvrtools.environments.baser3existsaccessW_OKmkdirloggerinfo)custom_scratch scratch_pathr3scratchs r_get_scratch_dirrE@sYY56NN+ ;&(=8 z" >>  '277 ;"RYYvx%@@>QL   td  ; KK:L Itd3 rctjd}|r t|n tdz }|j dd|S)NAPPTAINER_CACHEDIRz .apptainerTr6)r9r:rrEr?) cache_dir cache_paths r_get_apptainer_cache_dirrJQs@ ./I$-i3C3E 3TJTD1 rimage executablec|jdrt|js|jds|S|j ddj ddj dd}t }||dz }|jr t |St5|jrt |cdddStjdtjd |tjd ||d z }|jd d ddl m }|dd}t ||d<t ||d< t|dt ||gd|}|jdk7rCtjdtjd|j dd|cdddStjdt |cdddS#t"j$$rCtjd|jr|j'|cYcdddSt($r+} tjd| |cYd} ~ cdddSd} ~ wwxYw#1swYyxYw)zVBuild (once, cached) a SIF from a ``docker://`` URL; falls back to the URL on failure.z.sifz docker:///-:Nz&Building SIF image (one-time setup)...z Source: %sz Target: %stmpTr6r)build_subprocess_envF) scrub_secretsinherit_profile_homeAPPTAINER_TMPDIRrGbuildiX)r envz/SIF build failed, falling back to docker:// URLz Error: %sizSIF image built successfullyz2SIF build timed out, falling back to docker:// URLz2SIF build error: %s, falling back to docker:// URL)endswithrr< startswithreplacerJstr_sif_build_lockr@rAr?tools.environments.localrSr r&warningr'r$r%unlink Exception) rKrL image_namerHsif_pathtmp_dirrSrXr)es r_get_or_build_sifrf[s v4;#5#5#7@P@PQ\@] {B/77SAII#sSJ(*Ij\..H8}   ?? x=    <= NE* NH-e# dT 2 B"US"%g, $' N !  *gs8}e!LVY_bcF  A%PQ}fmmDS.AB-  . KK6 7x=1  2((  NNO P !L;  <  NNOQR SLA  < =  sQ#I/BI/A#G$;G$$A I,.I/9I,I'I,I/'I,,I//I8ceZdZdZdZ ddededededed ed ed effd Z d Z dddddededededzde jf dZ dZxZS)SingularityEnvironmentaHardened Singularity/Apptainer container with resource limits and persistence. Spawn-per-call: every execute() spawns a fresh ``apptainer exec ... bash -c`` process. Session snapshot preserves env vars across calls; CWD persists via in-band stdout markers. TFrKcwdr cpumemorydiskpersistent_filesystemtask_idc t |||t|_t ||j|_dt jjdd|_ d|_ ||_ ||_ d|_ ||_||_|jr>t!dz dt#|z |_ |jj%dd|j'|j)y) N)rir hermes_ Fzhermes-overlayszoverlay-Tr6)super__init__r*rLrfrKuuiduuid4hex instance_id_instance_started _persistent_task_id _overlay_dir_cpu_memoryrEr r?_start_instance init_session) selfrKrir rjrkrlrmrn __class__s rrszSingularityEnvironment.__init__s S'279&udoo> $TZZ\%5%5cr%:$;<!&0 ,0    !"%668D]^eDfCg9hh      # #D4 # @  rc|jddddg}|jr3|jr'|jdt |jgn|j d ddlm}m}g||D]!}|jd |d d |d d g# |jdkDr |jd|jdg|jdkDr&|jdt |jg|jt |j|jg t!|d}|j(dk7rt'd|j*d|_tj/d|j|jy#t$r!}tjd|Yd}~!d}~wwxYw#t"j$$r t'dwxYw)Ninstancestartz --containallz --no-homez --overlayz--writable-tmpfsr)get_credential_file_mountsget_skills_directory_mountz--bind host_pathrQcontainer_pathz:roz8Singularity: could not load credential/skills mounts: %sz--memoryMz--cpusxrzInstance start timed outzFailed to start instance: Tz/Singularity instance %s started (persistent=%s))rLryr{extendr\appendtools.credential_filesrrrar@debugr}r|rKrwr r$r%rr&r'rxrA)rcmdrrentryrer)s rr~z&SingularityEnvironment._start_instances G^[Q    1 1 JJ S):):%;< = JJ) * X eW57W:T:VW ] H{);(U=VVY&Z[\ ] <4D4D, 'rrh)r)(rloggingr9rr$ threadingrtpathlibrtypingrhermes_constantsrr;rrr tools.environments.base_outputr tools.environments.path_utilsr tools.environments.remote_commonr r getLoggerrr@rr\rr*dictr-r0rErJLockr]rfrhrrrrs   ,WW6CC   8 $<< ;c; s //.$.4.$"$!).."+S+c+C+\S'_S'r