߲j'#dZddlmZddlZddlZddlmZmZddlm Z m Z ddl m Z m Z mZmZmZehdZddZe Gd d Ze Gd d Ze Gd dZGddeZy)zAbstract backend interface for computer use. Any implementation (cua-driver over MCP, pyautogui, noop, future Linux/Windows) returns the shapes below. All methods are synchronous; async is handled inside the backend implementation if needed.) annotationsN)ABCabstractmethod) dataclassfield)AnyDictListOptionalTuple> c |jdr@t|dk\r2tjd|dd\}}t |t |fS|jdr!t|dkDrd}|dzt|kr||d k7r|d z }||d z|dz}}|d k(r,|t|kr|||d z}}|d k(r|t|kr|d vrb|d k(s|dzt|kDr ytj |||dzd }|dks||zt|kDr y|t vrC|dk\r>tj ||dz|dzd tj ||dz|dzd fS||z }|dzt|kry)z(width, height) for PNG / JPEG bytes, or None when unreadable. PNG: IHDR. JPEG: walk segments (skipping 0xFF fill bytes) to the first SOF marker; stop at SOS. Used by the tool layer's provider min-size guard.sPNG  z>IIs >bigN) startswithlenstructunpackint from_bytes_JPEG_SOF_MARKERS)rawwidthheightimarker segment_lens ?/home/ubuntu/.hermes/hermes-agent/tools/computer_use/backend.pyimage_dimensions_from_bytesr7s ~~*+CB eSBZ8 v5z3v;&& ~~k"s3x!| !ec#h1v~QAE AEAFD.QS\FAED.QS\%~QS!1  ..Qq1uu=KQ!k/CH"< **{a/?~~c!a%A&6>sSTWXSXYZ]^Y^O_af@ggg  A!!ec#h" ceZdZUdZded<ded<dZded<dZd ed <dZded <d Zded <d Z ded<e e Z ded<dZ ded<y) UIElementz/One interactable element on the current screen.r-indexstrrolelabel)rrrrzTuple[int, int, int, int]boundsapprpid window_iddefault_factoryDict[str, Any] attributesN Optional[str] element_token)__name__ __module__ __qualname____doc____annotations__r?r@rArBrCrdictrGrIr8r6r:r:,sZ9 J IE3O(4F %4CMCLIs!&t!rA window_titler png_bytes_lenimage_mime_typenoteax_max_elements)rJrKrLrMrNrTrlistrUrArVrWrXrYrZrPr8r6rRrR>srP I J K!G]! %d ;Ho;CML#M3&*O])D#NOSr8rRceZdZUdZded<ded<dZded<dZd ed <ee Z d ed <dZ ded<dZ ded<dZ ded<dZ ded<dZded<dZded<dZded<y) ActionResultuhResult of any action (click / type / scroll / drag / key / wait). ``ok`` is tool/transport success only — NOT the semantic verdict; read ``effect`` / ``escalation`` (cua-driver's structured verdict) to pick the next rung of the verify → escalate ladder. Structured fields are optional and additive: an older driver that omits ``structuredContent`` leaves them ``None``, behavior unchanged. Beyond the transport-level ``ok`` flag, this carries cua-driver's structured action verdict so the model can follow the documented verify → escalate ladder (NousResearch/hermes-agent#67052). boolokr<actionr>messageNzOptional[CaptureResult]capturerDrFmetazOptional[bool]verifiedrHeffectzOptional[Dict[str, Any]] escalationpathdegraded delivery_modecode)rJrKrLrMrNrarbrrOrcrdrerfrgrhrirjrPr8r6r]r]Xs~ H KGS'+G $+ 6D.6#Hn# FM +/J(/D-#Hn##'M='D-r8r]c eZdZdZeddZeddZeddZe d ddZedddddddd d  dd Z edddddddd d  dd Z eddddddd d ddZ edd d d dZ edd dd!dZ ed"dZd"dZed#d$dZed%d&dZd'dZy)(ComputerUseBackendaLifecycle: `start()` before first use, `stop()` at shutdown. Pointer/keyboard actions take ``delivery_mode`` (background (default) | foreground) and ``bring_to_front``; ``button`` is left | right | middle; ``modifiers`` a list of key names. ``element`` args are 1-based SOM indices from a prior capture. `direction` is up | down | left | right and `amount` is wheel ticks; `keys` is a combo such as 'cmd+s', 'ctrl+alt+t', 'return'.cyNrPselfs r6startzComputerUseBackend.startzs r8cyrnrPros r6stopzComputerUseBackend.stop}sr8cyrnrPros r6 is_availablezComputerUseBackend.is_availables$'r8NcyrnrP)rprSrArBrCs r6rbzComputerUseBackend.capturesCFr8leftr!F)elementxybutton click_count modifiersribring_to_frontcyrnrP) rprxryrzr{r|r}rir~s r6clickzComputerUseBackend.clicks cfr8) from_element to_elementfrom_xyto_xyr{r}rir~cyrnrP) rprrrrr{r}rir~s r6dragzComputerUseBackend.drags ber8r()amountrxryrzr}rir~cyrnrP) rp directionrrxryrzr}rir~s r6scrollzComputerUseBackend.scrolls dgr8)rir~cyrnrP)rptextrir~s r6 type_textzComputerUseBackend.type_textsADr8cyrnrP)rpkeysrir~s r6keyzComputerUseBackend.keystwr8cyrnrPros r6 list_appszComputerUseBackend.list_appss14r8cgS)zVisible native windows with PID and window identifiers. Optional compatibility hook: backends that predate window discovery stay instantiable and report none.rPros r6 list_windowszComputerUseBackend.list_windowss  r8cyrnrP)rprA raise_windows r6 focus_appzComputerUseBackend.focus_appsORr8cyrnrP)rpvaluerxs r6 set_valuezComputerUseBackend.set_valuesTWr8c ztjtdt|dt ddd|ddS) Ngg>@Twaitzwaited z.2fs)r_r`ra)timesleepmaxminr])rpsecondss r6rzComputerUseBackend.waits7 3sC./0tFggc]RSrsA# #(33lm: (( ("  2  4;V;Vr8