߲jS9 UdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl m Z mZddlmZmZddlmZddlmZddlmZddlmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%dd l&m'Z'm(Z(m)Z)dd l*m+Z+e jXe-Z.e/ej`Z1d Z2e3e4d <d e5de/fdZ6d ee5de7de/fdZ8Gdde Z9hdZ:dZ;dZde/de/fdZ?de/de/de@e/fdZAGdde(e+e9ZBdd lmCZCdd!lmDZDdd"lmEZEdd#lmFZFdd$lmGZGdd%lHmIZIdd&lHmJZJddlKZKddlLZLd'ZMd'ZNeFe1ZOeGe1ZPd(d)d*d+d,d-d.d/d0d1d2d3d4 ZQd5ZRy)6aFile operations (read, write, patch, search) over any terminal backend. Every operation is a shell command run through the backend's ``execute()``, so one implementation serves every environment (local, docker, ssh, modal, ...). Companions: ``file_operations_common`` (result dataclasses, text helpers), ``file_operations_lint`` (LintMixin), ``file_operations_search`` (SearchMixin). N)ABCabstractmethod)OptionalDict)Path)has_binary_extension)get_write_denied_error) ExecuteResult PatchResult ReadResult SearchResult WriteResult _UTF8_BOM_detect_line_ending_has_bom_normalize_line_endings _strip_bom_strip_terminal_fence_leaksnormalize_read_paginationnormalize_search_pagination)LINTERS_INPROC LintMixin_FAIL_CLOSED_INPROC_EXTS) SearchMixin))sPNG  zPNG image data)szJPEG image data)sGIF87aGIF image data)sGIF89ar)sRIFFz$RIFF container (WAV/AVI/WebP family))s%PDF-z PDF document)sPKz$ZIP archive (also docx/xlsx/jar/apk))sPKzZIP archive (empty))szgzip compressed data)sBZhzbzip2 compressed data)s7zXZzxz compressed data)s7z'z 7-Zip archive)sELFzELF executable)sMZzWindows PE executable)szMach-O executable (64-bit))sz$Mach-O universal binary / Java class)sSQLite format 3zSQLite database)sOggSz Ogg container)sfLaCz FLAC audio)sID3zMP3 audio (ID3 tag))sz$ISO media container (MP4/MOV family))sBMzBMP image data)sII*zTIFF image data (little-endian))sMM*zTIFF image data (big-endian)_MAGIC_SIGNATURESsamplereturnc~tD]4\}}|j|s|jdr |dddk7r2|cSy)zBest-effort human name for binary content from its magic bytes; never raises. The ISO-media entry additionally requires ``ftyp`` at offset 4 (three leading NULs alone are too weak a signature).z ISO mediasftypzunknown binary)r startswith)rprefixnames :/home/ubuntu/.hermes/hermes-agent/tools/file_operations.pyidentify_binary_bytesr&GsK*    V ${+q w0FK   file_sizecxt|xsd}|dk\r |dz dd}n|dk\r |dz dd}n|d}d|d |d S) zOne-line binary-file refusal naming the TYPE ("PNG image data, 4.1 KB"), so the model gets what-is-this in one read instead of hunting for tools it may lack.r'z.1fz MBiz KBz bytesz Binary file (, u) — cannot display as text.)r&)rr(kindsizes r%describe_binary_filer.Ssl !3 /DK{+C04 d d"3's+F# 4&4&(E FFr'c>eZdZdZeddedededefdZededefdZ edded ed e ede fd Z e dded ed ede def dZededefdZedede fdZededede fdZe d dededede ededededededefdZy)!FileOperationsz@Abstract interface for file operations across terminal backends.pathoffsetlimitrcy)z$Read a file with pagination support.N)selfr1r2r3s r% read_filezFileOperations.read_filecr'cy)zFWhole file as a plain string: no pagination, line numbers or clamping.Nr5r6r1s r% read_file_rawzFileOperations.read_file_rawgr8r'Ncontent pre_contentcy)z8Write content to a file, creating directories as needed.Nr5)r6r1r<r=s r% write_filezFileOperations.write_filekr8r' old_string new_string replace_allcy)z,Replace text in a file using fuzzy matching.Nr5)r6r1r@rArBs r% patch_replacezFileOperations.patch_replaceor8r' patch_contentcy)zApply a V4A format patch.Nr5)r6rEs r% patch_v4azFileOperations.patch_v4atr8r'cy)z>Delete a file. Returns WriteResult with .error set on failure.Nr5r:s r% delete_filezFileOperations.delete_filexr8r'srcdstcy)zCMove/rename a file. Returns WriteResult with .error set on failure.Nr5)r6rJrKs r% move_filezFileOperations.move_file|r8r'patterntarget file_glob output_modecontextorderc y)zSearch for content or files.Nr5) r6rNr1rOrPr3r2rQrRrSs r%searchzFileOperations.searchr8r'NF.r<N2rr<r discovery)__name__ __module__ __qualname____doc__rstrintr r7r;rrr?boolr rDrGrIrMr rUr5r'r%r0r0`sJ3c33333*33U#U*UUGsGSGx}GXcGG*/;#;3;C;#';4?;;(s({((MM MMRSRsR{RRBKOP<='+c++C+"3-+7:+IL++69++-9++r'r0>.bmp.gif.ico.jpg.png.jpeg.webp__hermes_not_regular____hermes_missing__ __HERMES_RF_ __HERMES_WF_r#c6|tjddS)aPer-call separator line for a compound shell probe. 128 random bits make a collision with file content negligible; the underscores keep the token outside the base64 alphabet, so a sentinel leaking into a sample segment fails base64 validation instead of decoding into bytes.__)secrets token_hex)r#s r% _new_sentinelrvs! Xg''+,B //r'outputsentinelc*|j|dzS)a Split compound-probe stdout on its sentinel lines. Every producer (``wc``, ``base64``, ``cut``) newline-terminates or prints nothing, so the separator is always ``sentinel + " "`` on its own line; the text after the final sentinel is the status segment. split)rwrxs r%_split_segmentsr}s <<4 ((r'creZdZdZdmdefdZ dndededededef d Zd ede fd Z d edefd Z d ededefdZ dedefdZ dod edefdZededeefdZedede fdZdmd edede fdZd ede fdZdpdededefdZd edefdZdedefd Zdedefd!Zd ededd"fd#Zdmd ed$eede fd%Zd&ed'ed(edefd)Zed edefd*Zd edeeeffd+Z dqd ed,edefd-Z!d edee eeffd.Z"d/Z#d0Z$d ed1ed2ed3edd4f d5Z%drd ed1ed2edefd6Z&de fd7Z'd ed1ed2edefd8Z(ed3edefd9Z)d ed1ed:ed;edZ+d ed1ed2ed3ed?eedef d@Z,d ed1ed2edefdAZ-dBed1ed:edCed3edDee defdEZ.dFZ/d edeefdGZ0d edefdHZ1d edefdIZ2dmd edJeedefdKZ3d ede4fdLZ5dMedNede4fdOZ6e7jpdPZ9d ededee4fdQZ:ed edRededee4fdSZ;d edd edYedeee ee4ffdZZ?dmd eded$eede4fd[Z@d eded\ed]ed^ed_eedeAfd`ZBd ed'edeeAfdaZC dsd ed\ed]edbe deAf dcZDddedeAfdeZE dtdfed edgedheed2ed1ediedjedkedeFfdlZGy)uShellFileOperationsaFile operations over any terminal backend exposing ``execute(command, cwd)`` returning ``{"output": str, "returncode": int}``. cwd rule: every ``_exec`` prefers the LIVE ``env.cwd`` so a ``cd`` run via the terminal tool is picked up immediately; the init-time ``self.cwd`` is only a fallback for envs that don't track cwd (using it for every call once made patches "succeed" with a plausible diff while landing in the wrong directory). Ncwdc||_|xs+t|ddxstt|ddddxsd|_i|_i|_i|_y)Nrconfig/)envgetattrr_command_cache_rg_resolution_cache_rg_modified_capability)r6 terminal_envrs r%__init__zShellFileOperations.__init__sbV',t<V7<4@%NVRU  0246!AC$r'commandtimeout stdin_datarci}|r||d<|||d<|xs%t|jddxs |j}|jj|fd|i|}|j dd}|j dd} d} |d k(rJd | vrFdd lm} t|jd d} | r | | rd nd} d|d| d| jx} } |j dr|dk(rd}t| || S)uRun ``command`` on the backend. cwd: explicit arg → live ``env.cwd`` → init-time ``self.cwd``. ``stdin_data`` is piped (bypasses ARG_MAX).rNrr returncoderrw~zcd: )_is_container_backendenv_typezI; for container backends use a path inside the container, e.g. /workspacezworking directory zU does not exist on the active terminal backend (check terminal.cwd or the session cwdz). stdin_errorrW)stdout exit_code cwd_error) rrrexecutegettools.terminal_tool_configrstripr )r6rrrrkwargs effective_cwdresultrrwrrrhints r%_execzShellFileOperations._execs$  'F9   !#-F< Iwtxx=I !!!'G}GGJJ|Q/ Hb)   & 0 HtxxT:H$9($C`IK %]$56AAEc&,,.IY[ \I ::m $aIFi9UUr'cmdc|dk(r|j|duS||jvrL|jd|d}|jry|jj dk(|j|<|j|S)zCheck if a command exists in the environment (cached); rg goes through the resolver so a mid-session install becomes visible.rgNz command -v z >/dev/null 2>&1 && echo 'yes'Fyes)_resolve_commandrrrrr)r6rrs r% _has_commandz ShellFileOperations._has_commands $;((-T9 9 d)) )ZZ+cU2P QRF'-}}':':'<'ED   $""3''r'r1cJ|jd|j|dS)uG``cat`` the file with stderr silenced (missing file → non-zero exit).cat 2>/dev/nullr_escape_shell_argr:s r%_catzShellFileOperations._cats&zzD!7!7!= >lKLLr'nbytescP|jd|d|j|dS)Nhead -c  rr)r6r1rs r%_headzShellFileOperations._heads,zzHVHAd.D.DT.J-K<XYYr'snippetc|jd|j|}|jdk7r5d|jxsdvr#|jd|j|}|S)zRun ``snippet`` via the backend's ``python3``, retrying with ``python`` when only that name exists (Windows / older systems).z python3 -c rpython3rz python -c )rrrr)r6rrs r%_run_python_snippetz'ShellFileOperations._run_python_snippetsik$*@*@*I)JKL   q Y6==3FB%GZZ*T-C-CG-L,M NOF r'lengthc|jd|d|j|d}|jdk7ry|j|jS)a>First ``length`` raw bytes, base64-wrapped so they survive the terminal transport (which decodes stdout with ``errors="replace"`` and manufactures U+FFFD for every undecodable byte, including a multibyte char cut in half by ``head -c``). None when no clean base64 came back (no ``base64`` binary); callers then fall back to the text heuristic. Wrapping the sample in base64 lets the original bytes survive the transport, so binary detection can happen at the byte layer where it is well-defined (#80308 and friends). rrz 2>/dev/null | base64rN)rrr_decode_base64_sampler)r6r1rrs r%_sample_file_bytesz&ShellFileOperations._sample_file_bytessThvha0F0Ft0L/MMbcd   q ))&--88r'textcdjt|j}|sytjd|sy t j |dS#tjtf$rYywxYw)uDecode one ``head -c N | base64`` sample. Whitespace-joins the whole text first (``base64`` wraps at 76 columns), so callers hand over exactly one segment; anything else fails validation → None (legacy text heuristic).rr'z[A-Za-z0-9+/]+={0,2}NTvalidate) joinrr|re fullmatchbase64 b64decodebinasciiError ValueError)rencodeds r%rz)ShellFileOperations._decode_base64_sample sl ''5d;AACD||3W= ##Gd; ; +  sAA76A7rc|syd|vry |jdy#t$rX}|jt|dz k\r3 |d|jjdYd}~y#t$rYnwxYwYd}~yd}~wwxYw)uByte-layer binary detection: text iff valid UTF-8, allowing one incomplete multibyte sequence at the very end (artifact of the byte-boundary cut). NUL bytes or mid-stream invalid UTF-8 stay read-only so a read→edit→write round-trip never rewrites undecodable bytes as U+FFFD; a file that legitimately CONTAINS U+FFFD is valid UTF-8 and reads as text. See #80308. FTutf-8N)decodeUnicodeDecodeErrorstartlen)rexcs r%_is_likely_binary_bytesz+ShellFileOperations._is_likely_binary_bytess f   MM' "! yyCK!O+;SYY'..w7 ) s2 A=A8A$$ A0-A8/A00A88A=content_samplect|ry|r8d|ddvrytd|ddD}|tt|dz dkDSy)zILegacy text-layer binary check: extension, else >30% non-printable chars.Tu�Nc3HK|]}t|dks|dvsdyw) z rWN)ord).0cs r% z8ShellFileOperations._is_likely_binary..>s#faCFRKTU]eTefs """g333333?F)rsumminr)r6r1r non_printables r%_is_likely_binaryz%ShellFileOperations._is_likely_binary5sZ  % >%400f>%4+@ffM 3s>':D#AADH Hr'cntjj|djtvS)NrW)osr1splitextlowerIMAGE_EXTENSIONSr:s r% _is_imagezShellFileOperations._is_imageBs+ww%a(..04DDDr'r< start_linecddlm}||jdr|dd}djfdt |j d|DS)zPrefix each line with a compact ``|`` gutter, clamping long lines. Not fixed-width: padding cost ~16% more tokens per line for no accuracy gain in A/B, while dropping numbers regressed line-referencing.rget_max_line_lengthrzNc3^K|]$\}}|dt|kr|n|ddz&yw)|Nz... [truncated]r)rilinemax_line_lengths r%rz8ShellFileOperations._add_line_numbers..RsFM4c3t974TBR?=SVg=gh iMs*-)r)tools.tool_output_limitsrendswithr enumerater|)r6r<rrrs @r%_add_line_numbersz%ShellFileOperations._add_line_numbersEs^ A-/   D !crlGyyM$W]]4%8 KMM Mr'c^|r|jds|S|jd}|jdk(r|jj r|jj }|dk(r|S|jdr||ddzS|dd}|j d}|dk\r|d|n|}|rt jd|ri|jd |}|jdk(rF|jj r,|jj |dt|zdzS|S) uExpand ``~`` / ``~user`` via the backend's shell (its HOME, not the host's). Must run BEFORE shell escaping — ~ doesn't expand in quotes.~z echo $HOMErz~/rWNrz[a-zA-Z0-9._-]+zecho ~) r"rrrrfindrrr)r6r1rhomerest slash_idxusername expand_results r% _expand_pathz ShellFileOperations._expand_pathVs4??3/KL)   q V]]%8%8%:==&&(Ds{ t$d12h&8D #I+4>tJY'tHBLL);XF $ VH:+> ?  **a/M4H4H4N4N4P(//557$q3x=?P?Q:RRR r'argcJddlm}d||jddzdzS)a5Single-quote ``arg`` for the shell. On Windows, native drive paths and mixed MSYS leftovers are first rewritten to the Git Bash ``/c/Users/x`` form via the env-layer ``_bash_safe_path`` (bash eats backslashes; MSYS mangles drive paths), so shell file ops and the terminal ``cd`` agree.r)_bash_safe_path''"'"')tools.environments.localrreplace)r6rrs r%rz%ShellFileOperations._escape_shell_argms) =_S)11#yAACGGr'czddlm}m}|r|r||jdd}d|jddzdzS)uBQuote a path for a NATIVE Windows binary (rg, node, git ...): those don't understand the MSYS ``/c/...`` form and Hermes disables MSYS argument conversion, so nothing translates it back (→ ``os error 3``). ``C:/Users/x`` is accepted by every layer. Identical to ``_escape_shell_arg`` off Windows.r) _IS_WINDOWS_msys_to_windows_path\rrr)rrrr)r6rrrs r%_escape_native_tool_argz+ShellFileOperations._escape_native_tool_argus? P 3',44T3?CS[[i00366r'r c|j|}|jtjj|xsd}|jd}d|d|d|zdz}|j ||S)u7Write ``content`` atomically: stdin → temp file in the SAME directory → ``mv -f`` (same-FS rename; cross-device ``mv`` is copy+unlink, NOT atomic). ``mkdir -p`` folded in. Exit 0 = swap happened; non-zero = original intact. Symlink targets are resolved first (replacing the link would orphan the target) and the temp dir recomputed from the RESOLVED target. Existing target: mode copied via ``stat`` (GNU ``-c%a`` / BSD ``-f%Lp``) + ``chmod`` (``chmod --reference`` is GNU-only). New target: ``chmod "=rw"`` AFTER cat gives umask-default perms instead of mktemp's 0600 — not ``$(umask)`` arithmetic (zsh parses leading-zero constants as decimal), quoted so zsh doesn't =word-expand. ``trap ... EXIT`` removes the temp on every failure. r\z.hermes-tmp.XXXXXXz set -e; d=z; t=z; if [ -L "$t" ]; then rt="$(readlink -f "$t" 2>/dev/null || realpath "$t" 2>/dev/null || true)"; [ -n "$rt" ] && { t="$rt"; d="$(dirname "$t")"; }; fi; mkdir -p "$d"; tmp="$(mktemp -p "$d" a 2>/dev/null || mktemp "$d/.hermes-tmp.$$.XXXXXX" 2>/dev/null || { tmp="$d/.hermes-tmp.$$"; : > "$tmp" && echo "$tmp"; })"; [ -n "$tmp" ] || { echo "atomic write: could not create temp file" >&2; exit 1; }; trap 'rm -f \"$tmp\"' EXIT; if [ -e "$t" ]; then m="$(stat -c%a "$t" 2>/dev/null || stat -f%Lp "$t" 2>/dev/null || true)"; [ -n "$m" ] && chmod "$m" "$tmp" 2>/dev/null || true; fi; cat > "$tmp"; if [ ! -e "$t" ]; then chmod "=rw" "$tmp" 2>/dev/null || true; fi; mv -f "$tmp" "$t"; trap - EXIT)r)rrr1dirnamer)r6r1r<q_pathq_parenttmplscripts r% _atomic_writez!ShellFileOperations._atomic_writes''-))"''//$*?*F3G%%&:;   $vh'%! %,(,- ,,/-$  Lzz&Wz55r'r=cr|j|d}|jdk(xrt|jS)uWhether the on-disk file starts with a UTF-8 BOM. ALWAYS probes disk: ``pre_content`` usually comes from ``read_file_raw``, which strips BOMs, so trusting it would silently drop the marker on rewrite. Missing → False.rr)rrrr)r6r1r= head_results r% _file_has_bomz!ShellFileOperations._file_has_boms5jjq) $$)Jh{7I7I.JJr' old_content new_contentfilenamec djtj|jd|jdd|d|S)NrT)keependsza/zb/)fromfiletofile)rdifflib unified_diff splitlines)r6rrrs r% _unified_diffz!ShellFileOperations._unified_diffsSwww++  " "D " 1;3I3ISW3I3X(_r(_>? ?r'c"td|dS)z5Error for a path that exists but would block if read.z Cannot read 'z`': not a regular file (directory, FIFO, socket, or device). Reading it could block indefinitely.errorr )r1s r%_not_regular_errorz&ShellFileOperations._not_regular_errors&D6"G GI Ir'c `|j|}|jd|d|d|dtdtd }t |j j }|tk(ry|tk(ry|jd k7rd |jxsd fS t|d fS#t$rYy wxYw) a0Byte size of a REGULAR file: ``(file_size, status)`` with status ``"ok"``, ``"missing"``, ``"not_regular"``, ``"bad_size"`` (unparseable ``wc``), ``"env_unavailable"``, or the named working-directory error when the exec wrapper itself failed (``_env_unavailable_error`` surfaces it verbatim). ``wc -c <`` on a writer-less FIFO/socket//dev/zero blocks forever and a name-based blocklist can't cover a FIFO (a file TYPE at any path); ``[ -f ]`` is a stat (symlinks followed) so it answers without touching content.if [ -f  ]; then wc -c < z 2>/dev/null; elif [ -e  ]; then echo ; else echo ; fi)rmissing)r not_regularrenv_unavailableok)rbad_size) rrNOT_REGULAR_SENTINELMISSING_SENTINELrrrrrrdr)r6r1r stat_result stat_outputs r%_probe_regular_filez'ShellFileOperations._probe_regular_files$$T*jjse,SE2^,@+AB)*$ 01 2+2D2DEKKM * * . .#  A %k++@/@@ @ !{#T) ) !  !s B!! B-,B-statuscD|dk7r t|Std|dS)Nr*r1Terminal environment unavailable: could not stat C (the sandbox may still be starting or was removed). Retry shortly.r )r6r1r2s r%_env_unavailable_errorz*ShellFileOperations._env_unavailable_errors; & &F+ +#TUYTZ[f"fh hr'c|j|}|"t|}|xs|j||fSt|j |dj }|j ||dfS)u``(is_binary, sample_bytes)`` — byte-layer detection when the transport allows (base64 sample), else the legacy text heuristic (sample is None).Nr)rrrrrrr)r6r1 sample_bytes ext_binary sample_outputs r%_detect_binaryz"ShellFileOperations._detect_binarysr..t4  #-d3JK!=!=l!K\Y Y3DJJtT4J4Q4QR %%dM:D@@r'iir2r3r(zOptional[ReadResult]c t|s||jkDryd|dt|dt|d|jd|jd }|j |}t |j xsd}|jd }|jd k7s|d kry||td zdj} tjd | vr| jd d d n| } | d } t| d} t| jdd} ||zd z }| |kD}d| j'dg}|r!|j)d|d zd|d|d| d d dlm}|t/fd| jd D}t1|j3| || ||dj5||rdSdS#t t"t$f$rYywxYw)zRead ``path`` as UTF-16 transcoded to UTF-8, or None (caller falls back to the binary-file error). Skips known-binary extensions and files over 10 MiB. ``path`` must already be expanded.Nzimport sys, json, os p = z offset = z limit = z MAX = z SAMPLE = af try: size = os.path.getsize(p) if size > MAX: print('HERMES_UTF16:NO'); sys.exit(0) with open(p, 'rb') as f: data = f.read() sample = data[:SAMPLE] enc = None if sample[:2] == b'\xfe\xff': enc = 'utf-16-be' elif sample[:2] == b'\xff\xfe': enc = 'utf-16-le' else: odd = sum(1 for i in range(1, len(sample), 2) if sample[i] == 0) even = sum(1 for i in range(0, len(sample), 2) if sample[i] == 0) if even == 0 and odd >= 2: enc = 'utf-16-le' elif odd == 0 and even >= 2: enc = 'utf-16-be' if enc is None: print('HERMES_UTF16:NO'); sys.exit(0) text = data.decode(enc, 'replace') if text[:1] == '\ufeff': text = text[1:] text = text.replace('\r\n', '\n') lines = text.split('\n') total = len(lines) sel = lines[offset - 1: offset - 1 + limit] out = {'total_lines': total, 'encoding': enc, 'content': '\n'.join(sel)} print('HERMES_UTF16:OK') print(json.dumps(out, ensure_ascii=True)) except Exception: print('HERMES_UTF16:NO'); sys.exit(0) rzHERMES_UTF16:OKrrzrWr< total_linesencodingzutf-16zTranscoded from zP to UTF-8 for display. Text edits via patch/write_file would re-encode as UTF-8. Use offset= to continue reading (showing - of  lines)rc3:K|]}t|kDywrYrrrrs r%rz6ShellFileOperations._try_read_utf16..JsZdc$i/9ZrTr<r=r( truncatedrtruncated_lines)r_UTF16_MAX_BYTESrd_UTF16_SAMPLE_BYTESrrrrrrrjsonloadsr|rcrrKeyError TypeErrorupperappendrranyr rr)r6r1r2r3r(rrrmarkerpayloaddatar<r=r>end_linerH hint_partsrrIrs @r%_try_read_utf16z#ShellFileOperations._try_read_utf16s8  %T5J5J)J (F }%5zl#**+,001"2: ' : R))'2,V]]-@bA./   q FQJ#&7"889:@@B ::47?gmmD!4Q7PWXD9oGd=12K488J9:HE>A%(* ()9(:;RRS    hl^,"81XJd;-wH I A-/ZgmmTXFYZZ**7F;9388J;O$3D? ?:>? ?Hi0  sAGG&%G&c N|j|}t||\}}|jr|j|||S|j |s t |r|j |||Sddlm}d|zdz}||zdz }tt}|j|j|||||}|jxsd} || vrt| j} | t k(r|j#|||S| t$k(r|j'|St(j+d||j,t/| |j |||St1| |} |j,dk7st/| dk7r>t(j+d||j,t/| |j |||S| \} } }}}}t|j3} t5|dt5|d}} t5t| j}|dk(r|j;| nd }||j=|}nNt(j+d ||t|j?|d j}|jA||}|r|jC|||||S|dk7rtEdt|St|} t5t|j}t|j}|dvr|dk(nd }|jG||||||S#t6t8f$r0t(j+d ||d d |j |||cYSwxYw#t8$rd}YgwxYw#t8$rd}YwxYw)aERead a file with pagination, binary detection, and line numbers. ``offset`` is 1-indexed; ``limit`` is clamped by ``normalize_read_pagination``. One shell round-trip answers every question the read needs (existence, size, binary sample, page, line count, trailing newline; see ``_read_probe_cmd``). An unparseable reply falls back to ``_read_file_sequential`` (one probe per question), so an exotic shell can never do worse than before. On a local POSIX environment the read never touches the shell (``_read_file_native``). rrr rWrzmread_file: compound probe reply for %s has no sentinel (exit %s, %d chars); falling back to sequential probesznread_file: compound probe for %s returned exit %s with %d segments (want 6); falling back to sequential probesz]read_file: compound probe for %s has unparseable status %r; falling back to sequential probesiNznread_file: no usable base64 sample for %s (base64 exit %s); paying one extra round-trip for the text heuristicrFailed to read file: r)01r]r2rVr=r(file_ends_with_newline)$rr_native_read_enabled_read_file_nativerr_read_file_sequentialrrrv_READ_SENTINEL_PREFIXr_read_probe_cmdrrrr._read_file_missingr-r!loggerdebugrrr}r|rd IndexErrorrrrrr_read_binary_filer _assemble_read_result)r6r1r2r3rline_clamp_bytesrVrxproberwrSsegmentssize_seg sample_segpage_segwc_segtail_seg status_segr2 sample_rcread_rcr(r8 is_binaryr: read_outputr= tail_flagr_s r%r7zShellFileOperations.read_filePs  &1&%@   $ $ &))$> > >>$ #7#=--dFEB B@244q8E>A% !67 4//fhHXZbcd# 6 !08>>@F))..tVUCC--..t44 LLIeoos6{ 4--dFEB B"684 ??a 3x=A#5 LLGeoos8} 6--dFEB BGOD*h*,Z8>>@ C!$VAYVAYwI 7AGGIJI BKat11*=UY  #44\BI LLEFJI W8 48N8U8UVM..t]CI ))$y,W W a<&;*5Lc!1RV)) {8N*P PEJ' C LL459:cd;K M--dFEB B  C I , K s6,M "N%"N CB }}RZZ(**40 0JJ >>$ ..y9 9@'))A-E>A% {  !& CdD!# %R1$7 !=!=f!E11$yRXY # %# %  GGG,E MM%( %bc I($u{{5'99 ',RSzU':  E C'"ZZs3"(F">h">6&3t9u+< $c#T9J2K(L L+/L!""4y50 $c#b#TAR:S2T(U U KKd e(;<({', #q( !  1f!'# %L Ff88 KKd e+ ,1#((4.2G2GXa2G2bc ++ {;DI$6$,PIIryy"**bnnbnn^d^k^k^mn M"#56 @**4? ? C--dFEB B C(# %# %H C--dFEB B CsV*M8 O )AN=- O 7D0N=(N=+O 8"N:N:9N:=OO O)(O)c tdd|dS)NTzImage file detected. Automatically redirected to vision_analyze tool. Use vision_analyze with this file path to inspect the image contents.)is_imagervr(rr )r(s r%rz*ShellFileOperations._image_redirect_resultsTYXZ Zr'rVrkrxc|j|}d|}djgd|d|d|d|d|d|d |d |d |d |d |d|d|d|d|dtdtdS)uOne shell command answering every question ``read_file`` asks: six segments each closed by a ``sentinel`` line — byte size, base64 of the first 1000 bytes, the ``sed | cut`` page, ``wc -l``, whether the last byte is a newline, then the base64 and page pipeline statuses. Probes run only inside ``[ -f ]`` (stat-not-open, like ``_probe_regular_file``) so a FIFO/device never reaches ``head``/``sed``. A missing path echoes ``MISSING_SENTINEL`` (a compound command only reports its last status). Every stage silences stderr: the local backend merges stderr into stdout and a stray diagnostic would land inside a segment. The byte clamp is ``4 * max_line_length + 1``; see ``_read_file_sequential``.zecho rr#r$z 2>/dev/null; z; head -c 1000 z, 2>/dev/null | base64 2>/dev/null; __hs=$?; z ; sed -n ',p' z 2>/dev/null | cut -b1-z 2>/dev/null; __hr=$?; z ; wc -l < z ; tail -c 1 z 2>/dev/null | wc -l; z ; echo "$__hs $__hr"; elif [ -e r%r&r')rrr-r.)r6r1r2rVrkrxrmarks r%rdz#ShellFileOperations._read_probe_cmds$$T*xj! 0 0h 0se 0 0e 0) 0*. 00 05 0 L 0MQF 0S 0h 0  0!)z 0*- 0.1E 02 0++ 0,C 0DH& 0I 0 e 0 * 0 +/ 0 0 0  0 4 0 596 0 : 0 0, 0-A+A 0B 0** 0+/ 0 1r'c|j|}|J|j|||}d|d|d}|jr|d|j|_|S||_|S|j|S)u*Not-found recovery shared by every read path. Unicode-equivalent spellings (NFC/NFD, confusable spaces/quotes) render identically, so the model can never discover the byte mismatch by retyping — retrying is the tool's job. No equivalent spelling → suggest similar files.)r2r3zNote: 'zD' not found byte-for-byte; resolved to the unicode-equivalent file 'zM' (invisible encoding difference: NFC/NFD or special space/quote characters).r)_unicode_variant_matchr7r_suggest_similar_files)r6r1r2r3variantrnotes r%rez&ShellFileOperations._read_file_missing0s --d3  ^^GF%^HF$ 007y9  6<[[TF!FKK=1FKMGKFKM**400r'r8cb|j||||}||Std|t||S)aBBinary branch shared by every read path: UTF-16 text (Notepad, PowerShell ``>``) trips the binary guard; transcode it, else refuse with the type name. UTF-16 rescue (ported from MoonshotAI/kimi-code#2647): the terminal env decodes stdout as UTF-8 with errors="replace", so a UTF-16 text file (Windows Notepad .txt, PowerShell `>` redirects) arrives mangled with U+FFFD and trips the binary guard. Probe the raw bytes via the backend's Python and transcode to UTF-8 when a BOM or the zero-byte parity heuristic identifies UTF-16. Trvr(r)rXr r.)r6r1r2r3r(r8 utf16_results r%riz%ShellFileOperations._read_binary_fileAsD++D&%K  # i&|Y?A Ar'c &|j|\}}|dk(r|j|||S|dk(r|j|S|dvr|j||S|j |r|j |S|j |\}}|r|j|||||Sddlm }d|zdz} ||zdz } |jd|d | d |j|d | } | jdk7rtd | j St| j} |jd|j|} t!t| jj#}d}|| kDsj| j'drY|jd|j|d}|jdk(r&t|jj#dk7}|j)| || |||S#t$$rd}YwxYw)zOne-probe-per-call read: the pre-compound form, kept as fallback for image / known-binary extensions and unparseable compound replies. ``path`` is already expanded and ``offset``/``limit`` normalized.r(r)r+r,rrr rWzsed -n 'rrz | cut -b1-r[rzwc -l < Nrzz tail -c 1 z | wc -lr\r^)r1rer!r6rrr;rirrrrrr rrrdrrrrj)r6r1r2r3r(r2rvr8rrkrV read_resultrw wc_resultr=r_ tail_results r%rbz)ShellFileOperations._read_file_sequentialRsH!44T: 6 Y **4? ? ] "**40 0 + +..tV< < >>$ ..y9 9"&"5"5d"; < ))$y,W W A244q8E>A%jjvhazT-C-CD-I,J*+ -.   A %&;K*>t*D**z$2H2H2N1Ox%XYK$$))D[EWEW)X)^)^)`dg)g&)) {8N*P P K s,H HHrwr=r_c  |dkDr |dur|dz }|dk(rt|\}}||kD}d} |rd|dzd|d|d|d } |s|jd r |dur|dd }|dk(rtd ddd S||cxkDrdkDrnntd ||d|d|d|dSddlm} | t fd|j d D} t|j|||||| | rdSdS)aFTurn a raw ``sed | cut`` page into the final ``ReadResult``. Shared by every read path so the BOM strip, pagination hint, ``cut`` newline-artifact fix and the ambiguous-silence guards never drift apart. ``file_ends_with_newline`` is None when the caller could not tell (artifact left alone, as before).rFrWNr?r@rArBrCrzrrzFile is empty (0 bytes).)r<r=r(rz Note: offset z is beyond the end of the file (z$ lines total). Retry with offset <= r\rc3:K|]}t|kDywrYrrEs r%rz.s^dc$i/9^rFTrG)rrr rrrRr|r) r6rwr2rVr=r(r__rHrrrIrs @r%rjz)ShellFileOperations._assemble_read_resultsX q=3u< 1 K Q;' 4NK(*   A.LVHTUV^U__cdocppwxD[11$7ba1Kef f K #! # y#F8,#}$H"m1&( ( A-/^kFWFWX\F]^^**;?[94$3D? ?:>? ?r'))u r) r)u’r)u‘rcltjj|xsd}tjj|}|sydtdtffd }||}j dj |d}|jdk7s|jjsyt|jjDcgc]}|r||k7r |||k(r|}}t|d k(r1|dk7sd |vr#tjj||dS|dSycc}w) uOn-disk spelling of a file whose name is unicode-equivalent to ``path`` (NFC/NFD, confusable spaces/quotes). Returns the entry only when EXACTLY one matches — several candidates = homoglyph collision, guessing would read the wrong file.r\Nr$rctjd|}jD]\}}|j||}|S)NNFC) unicodedata normalize _CONFUSABLESr)r$outrJrKr6s r%_canonz:ShellFileOperations._unicode_variant_match.._canonsC''t4C -- ,Skk#s+ ,Jr'ls -1 rrrWr)rr1r basenamercrrrrrrrrr) r6r1dir_pathrrrO ls_resultentry candidatess ` r%rz*ShellFileOperations._unicode_variant_matchs, 77??4(/C77##D)    !JJ(>(>x(H'IVW   ! #9+;+;+A+A+C:9;K;KLWWYH(*ve}/F H H z?a /dev/null | head -50rzdZF<(g?Ng?r]c|d S)Nrr5)xs r%z.s 1Q4%r')keyFile not found: )r similar_files)rr1r rrrrrrrrr|r"rsetmaxrSequenceMatcherratiorQrsortr )r6r1rrbasename_no_extext lower_namerscoredflfscorecommonrfps r%rz*ShellFileOperations._suggest_similar_filesst77??4(/C77##D)''**84Q7==?ggx(+113^^% JJ(>(>x(H'II`ab    ! # (8(8(>(>(@%%++-33D9 FWWY#EWW%%a(+113FE]]:.*2G2G2KE2%E:%#b'A+ERWW--a0399;sB _s2w6F6{c#j/3r7&Cc&II "A:'"9"9$ B"O"U"U"W[^"^E19MM5"'',,x*C"DE/ F0   ("24& 9Z`acbcZdIeQVQRTV"IeffIes K# c\|j|}|j|\}}|dk(r|j|S|dk(r|j|S|dvr|j ||S|j |rt dd|S|j|\}}|rt d|t||S|jd|j|}|jdk7rt d |j Stt|j\}}t || S) zCWhole file as a plain string (no pagination/line numbers/clamping).r(r)rT)rrvr(rrrr[r)r<r()rr1rr!r6rr r;r.rrrrrr) r6r1r(r2rvr8 cat_result raw_contentrs r%r;z!ShellFileOperations.read_file_raws3  & 44T: 6 Y ..t4 4 ] "**40 0 + +..tV< < >>$ ttyQ Q"&"5"5d"; <  I]^jluIvw wZZ$t'='=d'C&D EF   1 $&;J > ] "**40 0 + +..tV< < Z &EdV$LM M  Y%:#+Ia=8I)TUVWXZ Z**y)?)?)E(FGH    !&B7>>BR$ST T''5gnnEKKMN Y   Wt 4IQUVVFOO112 Y&PQUPV$WX X YsD::2E/.E/c|j|}t|d}|r t|Sd|d}|j|}|jdk7r2td|d|j xsd j xsd StS) zDelete a single file (directories rejected) via the backend's ``python -c`` so one code path works on local/docker/ssh AND Windows shells (no ``rm``).Deleteverbrz-import shutil, pathlib, sys p = pathlib.Path(a\) recursive = False try: if p.is_dir() and not p.is_symlink(): if recursive: shutil.rmtree(p) else: print('is a directory: ' + str(p), file=sys.stderr); sys.exit(2) else: p.unlink() except FileNotFoundError: pass except Exception as exc: print(str(exc), file=sys.stderr); sys.exit(1) rzFailed to delete : rz unknown error)rr rrrrr)r6r1deniedrrs r%rIzShellFileOperations.delete_file(s  &'8< V, , $x (B B  ))'2   q '8b&--BUSUA\A\A^Aqbq@r%st t}r'rJrKcl|j|}|j|}||fD]}t|d}|st|cS|jd|j |d|j |}|j dk7rtd|d|d |j StS) NMoverrzmv rrzFailed to move z -> r)rr rrrrr)r6rJrKprrs r%rMzShellFileOperations.move_fileFs$$s 1A+AF;F"00 1c$"8"8"=!>a@V@VWZ@[?\]^   q se4uBv}}o%VW W}r'z[\ud800-\udc7f\udd00-\udfff]c|jj|}|r"td|d|jddSy)zRefuse content with a lone surrogate BEFORE any subprocess: letting it reach the pipe spawns a child that hangs or truncates the target via empty-stdin ``cat``. A regex scan needs no encode.Refusing to write 'z0': content contains a lone surrogate character (rzH) that cannot be encoded as UTF-8. The file was NOT created or modified.rN)_LONE_SURROGATE_RErUrgroup)r6r1r<ms r%_reject_unencodablez'ShellFileOperations._reject_unencodableXsV  # # * *7 3 %dV,(() ~6JJL Lr'rc |tvrtj|nd}|y||\}}|s|dk(rytd|d|d|dS)aGFail-closed pre-write gate for ``_FAIL_CLOSED_INPROC_EXTS`` (JSON/YAML/TOML): a structured-format write that doesn't parse is a corrupt write, so refuse before any bytes touch disk. Checked against the RAW content, before the BOM/CRLF shims (post-shim linting would false-positive on a BOM-marked file).N__SKIP__rz': candidate content fails z syntax validation (zC). The file was NOT created or modified. Fix the content and retry.r)rrrr)r1rr<linterr+errs r%_fail_closed_syntax_errorz-ShellFileOperations._fail_closed_syntax_errordsr -03K,K##C(QU >/C  "!$'Be'u-B BD Dr'bodyc |j|}|dk(rd|d}n|dk(rd|d}nd}d|d|d |d |d td S) aOne shell command for the on-disk questions ``write_file`` asks. Two segments closed by a ``sentinel`` line: base64 of the first three bytes (BOM detection at the byte layer, same on-disk truth as ``_file_has_bom``), then ``body``: ``"cat"`` for the full text when pre-content is wanted, ``"sample"`` for the 4 KB line-ending sample, or None for nothing. Gated on ``[ -f ]`` so a FIFO/device never reaches ``head``/``cat``; a missing path echoes ``MISSING_SENTINEL``.catrrrz head -c 4096 :r#z ]; then head -c 3 z( 2>/dev/null | base64 2>/dev/null; echo z; r&r')rr.)r6r1rxrrbody_cmds r%_write_probe_cmdz$ShellFileOperations._write_probe_cmdusz$$T* 5=cU,/H X &se<8HHseEhZrj)*$ 0 1r'want_prec|r|d}n|sd}nd}tt}|j|j|||}|jxsd}||vrrt |j tk(r|r t|nd}d||fStjd||jt||j|||St||} |jdk7st| dk7r>tjd ||jt| |j|||S| \} } |j| } | )tjd ||j!||} n$| j#t%j&d } |dk(r| r| }|r t|}n|dk(r| r t| }nd}| ||fS) a``(has_bom, pre_content, original_line_ending)`` for ``path`` in ONE round-trip (replaces ``cat`` when pre-content is wanted, a ``head -c 4096`` line-ending sample and a ``head -c 3`` BOM check). Semantics unchanged: pre-content is read only when wanted and not supplied; the line ending comes from pre-content when there is any, else from the sample; the BOM always comes from disk. An unparseable reply falls back to the separate probes.NrrrFzowrite_file: pre-write probe reply for %s has no sentinel (exit %s, %d chars); falling back to sequential probesrrzpwrite_file: pre-write probe for %s returned exit %s with %d segments (want 2); falling back to sequential probeszWwrite_file: no usable base64 head for %s; paying one extra round-trip for the BOM prober)rv_WRITE_SENTINEL_PREFIXrrrrrr.rrfrgrr_probe_write_target_sequentialr}rrr"rencode)r6r1r=r body_moderxrlrwendingrmhead_segr head_byteshas_boms r%_probe_write_targetz'ShellFileOperations._probe_write_targets  +',I II !78 400xKL# 6 !*6288:>NN=H,[9dk611 LLIeoos6{ 466t[(S S"684 ??a 3x=A#5 LLGeoos8} 666t[(S S!$//9   LL/04 6(({;G ++I,<,G 8G,')G ##D)BGGOOD12  w0AB ))$8  ! !Q &'=l>Q>Q=R%ST T)-)B)B4)W&,  # ,,T{Y`,a *.   +"5"5"99$Kfm9nvrvOm,88<<> > KDTFK  +G[*gV VG))   sA AAc t|j|}|jdk7rtd|St|j\}}|j ddj dd}|j ddj dd}||k7r(td|dt |d t |d Sy ) afRe-read ``path`` and confirm the intended bytes landed; error result or None. Catches silent persistence failures (FS oddities, races, truncated pipe). Line endings are normalized first (Windows text-mode ``open()`` writes LF as CRLF) and the re-read's BOM stripped (``new_content`` is the BOM-less string we matched against).rz2Post-write verification failed: could not re-read rr*rz r#z5: on-disk content differs from intended write (wrote z chars, read back zb chars after normalizing line endings). The patch did not persist. Re-read the file and try again.N)rrr rrrr)r6r1r verify_resultbomlessron_diskintendeds r%_verify_patch_persistedz+ShellFileOperations._verify_patch_persisted8s  $  " "a ''YZ^Y_%`a a 4 45 //&$/77dC&&vt4<\] ]"((  , <5K Z[6:2 [)U K1$((w JP[]bc c*'2 1+{KKt[kR   '@ASAS@T%UV V33D+F  # ,,TwU`,a t11';M_c^d*5$$&4(88 : :r'rEc\ddlm}m}||\}}|rtd|S|||S)zyApply a V4A format patch (``*** Begin Patch`` / ``*** Update File:`` / ``@@ hint @@`` hunks / ``*** End Patch``).r)parse_v4a_patchapply_v4a_operationszFailed to parse patch: r)tools.patch_parserrWrXr )r6rErWrX operations parse_errors r%rGzShellFileOperations.patch_v4ats: M"1-"@ K '>{m%LM M#J55r'rNrOrPrQrRrSc @t||\}}|dk(r| dvrtd| dS|j|}|j|} | j} | j rt| j Sd| vrd| vrtd|d Sd| vr.|j ||||||||| } | | S|j|S|j||||||||| } |j|}|r/| js#d jd |D}d |d | _ | S)uSearch for content (regex, ``target="content"``) or files (glob, ``target="files"``). ``output_mode``: "content", "files_only" or "count"; ``context``: lines of context around matches; ``order``: file-search ordering — fast "discovery" or exact "modified" time.files>modifiedr^zInvalid file search order z%; expected 'discovery' or 'modified'.rexists not_foundr4r5r+c3DK|]}|jddyw)rrNr{)ritems r%rz-ShellFileOperations.search..sK 3 3Ks z[Skipped macOS protected folders during broad search to avoid an unattended privacy prompt: z@. Search a protected folder directly when access is intentional.) rr r_path_exists_proberr_try_multi_path_search_path_not_found_result_dispatch_search_macos_search_exclusionsrrwarning)r6rNr1rOrPr3r2rQrRrSrl exists_probemultir exclusionsr8s r%rUzShellFileOperations.searchst4FEB  W .G!G3E9=446 6  &''-|| ??eoo6 6 < 'K|,K)Z[_Z`al(ln n , &//vy%gW\^E  ..t4 4&&wfiPV'2GUD2248 flliiK KKG118 :>> N r'rY)NNN)r)rW)r*rVrZr[)Hr_r`rarbrcrrdr rrerrrrr staticmethodrrrrrrrrrrrrrr r!tupler1r6r;rJrKrXr7r`rarrdrerirbrjrrrr;rrrIrMrcompilerrr rrrr(r?r rHrOrDrGr rUr5r'r%rrsA D# DCG $VSVsVCVV)6V@ ( ( (MMMZ#ZsZ}Z3= 9s 9C 9 C HUO  $6 c 3 $ EcEdEMM#McM".HSHSH7373766#666666pK#KHSMKTK??3?#?RU?IIII !!c3h!8h3hhT^h A3A5x1F+GA$(K?CK?K?SK?#&K?+AK?ZYPcYP3YP3YP*YPv Bd BVcV3VsVzVpZ#Z*ZZ1C111*-19<1AD121s1C11 1"AcA3AsA%(A8@ALVA"0P#0Ps0P30P:0Pd)?)?)?PS)?+.)?;>)?6>tn)?IS)?XL38C=6#g3#g:#gJD#D*D.WCWHSMWZW4 < S s {  $$CD  c h{>S DD#DDQ\H]DD 1S1C1x}1QT1(7,7,(3-7,SW7,!&tXc]HSM'I!J7,rJ3JXc]J^bJ,1$ xPS}2T,UJ UuXVZ^]efq]rMrGs&:bs:bS:bx}:bXc:b|*S*3*C*%(*7:*CKC=*U`**Cch{F[,+0%:#%:3%:C%:#'%:4?%:N6s6{6CLOP<='%c%%C%"3-%7:%IL%%69%%-9%r'r)Any)ClassVar)List)build_write_denied_paths)build_write_denied_prefixes) dataclass)fieldrX)tools.file_operations_commonDEFAULT_READ_LIMIT)rvDEFAULT_READ_OFFSET)rvDEFAULT_SEARCH_LIMIT)rvDEFAULT_SEARCH_OFFSET)ztools.file_operations_lintLINTERS)rv LintResult)ztools.transcription_common MAX_FILE_SIZE)zagent.search_policySEARCH_PRUNE_DIR_NAMES)rv SearchMatch)agent.file_safetyrr)rrs)tools interrupt) rwrxryrzr{r|r}r~rrrrstool_interruptctj|}|tdtd|ddl}ddlm}|t|g|t|j|d|dS)Nzmodule z has no attribute r) warn_oncerW) _PLUGIN_COMPAT_LAZYrAttributeErrorr_ importlibhermes_cli.plugin_compatrr import_module)r$rOrrs r% __getattr__rse $ $T *F ~wxl2DTHMNN2 h&v& 9**6!95vay AAr')SrbrrrrrrrrLloggingrtrabcrrtypingrrpathlibrtools.binary_extensionsragent.file_safetyr tools.file_operations_commonr r r r rrrrrrrrrtools.file_operations_lintrrrtools.file_operations_searchr getLoggerr_rfrcr_HOMErrm__annotations__rr&rdr.r0rr-r.rcrrvlistr}rrorprqrrrs dataclassesrtru posixpath threading MAX_LINESMAX_LINE_LENGTHWRITE_DENIED_PATHSWRITE_DENIED_PREFIXESrrr5r'r%rs  #!84YYYY[Z4   8 $ IDIIK5< % C  G% GS GS G%+S%+VN0 (&'0#0#0)C)3)49)z)[.zB(69!  -e43E:QRTV8@DOB Q#W,  Br'