߲jf;dZddlmZddlZddlZddlZddlmZmZm Z m Z ddl m Z m ZmZmZmZddlmZmZmZej.eZdZdZd Zej:d ej<Zdd Z dd Z!dd Z"Gdde Z#ddiZ$dZ%y)uxAI Web Search — search-only provider backed by Grok's server-side ``web_search`` tool on the Responses API (https://docs.x.ai/developers/tools/web-search); Grok is asked for structured JSON so rows match every other Hermes web provider. Config: ``web.backend: "xai"``; optional ``web.xai``: ``model`` (default grok-build-0.1), ``allowed_domains`` / ``excluded_domains`` (max 5, mutually exclusive), ``timeout`` (default 90s). Auth: Grok OAuth via ``hermes auth``, else XAI_API_KEY. ) annotationsN)AnyDictListOptional)BaseWebSearchProvider search_fail search_ok setup_schema title_hit)has_xai_credentialshermes_xai_user_agentresolve_xai_http_credentialszgrok-build-0.1Zz \{[\s\S]*\}c ddlm}|}dD]%}t|tr|j |nd}'t|tr|SiS#t $r"}t jd|icYd}~Sd}~wwxYw)z7Read ``web.xai`` from config.yaml (returns {} on miss).r) load_config)webxaiNz!Could not load web.xai config: %s)hermes_cli.configr isinstancedictget Exceptionloggerdebug)rcfgkeyexcs =/home/ubuntu/.hermes/hermes-agent/plugins/web/xai/provider.py_load_xai_web_configr!sr1m! BC",S$"7#''#,TC B d+s33  8#> s$AA A A8A3-A83A8ct|trG|Dcgc]4}t|ts|js%|j6c}dtSgScc}wN)rliststrstrip_MAX_DOMAIN_FILTERS)valueitems r _coerce_domain_listr*)sYoyzBFpGU UTjs.C DJJL UVjWj kOMOO UsAAAcB ||S#ttf$r|cYSwxYwr#) TypeError ValueError)castr(defaults r _coercer0-s*E{ z "s  ceZdZdZdZdZddZdddZeddZ eddZ eddZ e dd Z edd Zedd Zedd Zdd Zy)XAIWebSearchProvideruSends a structured prompt with ``tools=[{"type": "web_search"}]`` and parses the JSON Grok returns; falls back to message annotations, then ``citations``. Trust model: Grok *generates* the URLs/titles/descriptions and is steerable by the query text — validate before fetching.rxAI Web Search (Grok)ctS)zCheap probe (env var OR auth-store tokens). Deliberately NOT ``resolve_xai_http_credentials``: must never refresh tokens or take the auth-store lock, since this runs on every ``hermes tools`` repaint.)r selfs r is_availablez!XAIWebSearchProvider.is_available<s #$$c  ddlm}|r tdS t }t |j dxsdj}t |j dxsdjjd}|s td Std ttt|d d }t}t|j d t r|d jndxst}|j!|} | tdS|d|j#||dg| gdgd} ddl} t(j+d|||||j-|| |tt.|j dt0t0|j ddk(\} } | r| St| t2r| j dnd}t|t2rL|j dxs|j dxsd}t(j5d|td|St7|j9| |S#t$rY?wxYw#t&$rtdcYSwxYw) Nr)is_interrupted Interruptedapi_keybase_urlzhttps://api.x.ai/v1/z_No xAI credentials found. Run `hermes auth` to sign in with xAI Grok OAuth, or set XAI_API_KEY.rdmodelzZweb.xai.allowed_domains and web.xai.excluded_domains cannot both be set (xAI restriction).user)rolecontentno_inline_citations)rBinputtoolsincludez4httpx is not installed (required for xAI web search)z0xAI web search via %s: '%s' (limit=%d, model=%s)timeoutproviderz xai-oauth) is_oauth_patherrormessagecodez unknown errorz*xAI web search returned error envelope: %szxAI returned an error: limit)tools.interruptr:_failrrr%rr&rstripmaxminr0intr!r DEFAULT_MODEL_web_search_tool _build_prompthttpx ImportErrorrinfo_post_responsesfloatDEFAULT_TIMEOUTrwarningr _extract_results)r6queryrQr:credsr<r>rrBweb_search_toolpayloadr[datarM api_errorerr_msgs r searchzXAIWebSearchProvider.searchBsg  6]++ -.eii *0b1779uyy,E0EFLLNUUVYZz{ {As73q1378"$)3CGGG4Dc)JW##%PRdWd//4  "uv v,1fY]YkYklqsxYy=z<{HWGXf{e|#} Q   FRWY^`ef** gwswwy/7Z\k(l 99Z0K?+ e L*4D$)?DHHW%T i &mmI.Z)--2GZ?G NNG Q27)<= =..t5.ABBI   & QOP P Qs#II III54I5c dDcic]}|t|j|}}|jDcic] \}}|s || }}}t|dk(ry|rd|dSddiScc}wcc}}w)z``web_search`` tool spec with optional domain filters; None when both allowed and excluded are set (xAI rejects the combination).)allowed_domainsexcluded_domainsN web_search)typefiltersrp)r*ritemslen)rkrqvs r rYz%XAIWebSearchProvider._web_search_toolms@gg!1)#''!*55gg$+MMO9DAqq1a499 w<1 =D 9`6S_J`` h9s"A) A.A.c ddl}ddtd}dfd }d}tdD],} |j|d||| }|j n| dtdfS |j!dfS#|j $r} | j | j jnd} | d k(r5| dk(r0|r.tjd |x} r| d| c|d <Yd} ~ | j | j jdd nd} n#t$rd} YnwxYwtjd| | dtd| d| jfcYd} ~ cSd} ~ w|j$r2} tjd| dtd| fcYd} ~ cSd} ~ wwxYw#t$r-} tjd| dtdfcYd} ~ Sd} ~ wwxYw)ufPOST ``/responses`` → ``(parsed_json, None)`` or ``(None, failure_envelope)``. Two attempts: on a first-call 401 with OAuth creds, force-refresh once and retry (opaque tokens the resolver can't pre-check; mid-window revocation/rotation). XAI_API_KEY creds can't be refreshed, so they skip the retry rather than burn quota. rNzBearer zapplication/json) Authorizationz Content-Typez User-Agentc ttdjdxsdj}|k7r|SdS#t$r }t j d|Yd}~yd}~wwxYw)zeNew bearer after a 401, or "" when refresh fails / returns the same token (retry would be pointless).T) force_refresh api_key_hintr<r=z1xAI web search OAuth refresh after 401 failed: %sN)r%rrr&rrra)r refresh_excr<s r _refreshed_keyz._refreshed_keysj 6TX_`ddenousuv||~!Wns4"4 RT_` s=AA A, A''A,rnz /responses)headersjsonrJizQxAI web search got 401 on first attempt; forcing OAuth refresh and retrying once.rwi,r=zxAI web search HTTP %d: %szxAI web search returned HTTP z: z xAI web search request error: %szCould not reach xAI: z#xAI web search produced no responsezxAI web search bad JSON: %sz/Could not parse xAI Responses API reply as JSON)returnr%)r[rrangepostraise_for_statusHTTPStatusErrorresponse status_coderr]textrrarSrT RequestErrorr~)r>rfr<rJrLr[r}r|respattemptrstatusnew_keybodys ` r r^z$XAIWebSearchProvider._post_responsesws &-gY$7I[lAlCD Qx BG BzzXJj"97QXbizj%%'  B* <DEE E R99;$ $'(( ^58\\5M11STS=W\mKK st"0"22w2PQQ Q Rsr(A;)F(;F% AE!)'DE! DE!D:E!F%!F%3%F F% F%( G1"GGGcd|d|S)zAsk for a JSON *object* (cheap to match with ``_JSON_BLOCK_RE``) and forbid prose/fences/inline citations to keep the payload parseable.u<Use the web_search tool to find current information for the query below, then respond with ONLY a single JSON object — no prose, no markdown fences, no inline citation links — matching this exact schema: {"results": [{"title": "string", "url": "string", "description": "1-2 sentence summary"}]} Return at most zy results, ordered by relevance, with absolute https:// URLs. If no usable results exist, return {"results": []}. Query: )rcrQs r rZz"XAIWebSearchProvider._build_prompts"  $W%W   r8c j|\}}tdfd|DDd}|s'|r'j|dj|x}r|S|j dxsg}t |t r[t|dDcgc]A\}}t |ts|js(tdt|d|dzCc}}SgScc}}w) a1Rows in order of preference: (1) the JSON object in ``output_text`` blocks, (2) ``url_citation`` annotations paired with surrounding text, (3) the raw ``citations`` list. (2) only short-circuits when it yields rows, so future annotation types don't mask real data in ``citations``.c3&K|] }|s| ywr#r).0ps r z8XAIWebSearchProvider._extract_results..sjQhiqjsc3DK|]}j|yw)rPN)_try_parse_json_results)rbclsrQs r rz8XAIWebSearchProvider._extract_results..s""dST3#>#>q#>#N"ds N rP citationsr=r@) _collect_output_textnext_results_from_annotationsjoinrrr$ enumerater%r&_row) r response_datarQ text_blocksrparsedrius ` ` r rbz%XAIWebSearchProvider._extract_resultss $'#;#;M#J [j"dXc"djlpq k1N1N{\`\e\efq\rz1N2A(Av(AM!%%k28b ~HIRTX~Y )FUBS8Ty1XbcdfiXjopovovoxRQQU+y a_a aysC!.C!?C!c |jd}t|tr|ngDcgc]u}t|trc|jddk(rOt|jdtr0|dD](}t|tr|jddk(r|*w}}}|Dcgc];}t|jdts#|dj s7|d=}}|Dcgc]?}t|jdts#|dD]}t|ts|A}}}||fScc}}wcc}wcc}}w)zC(text_blocks, annotations) from ``response.output`` message chunks.outputrprNrE output_textrr)rrr$rr%r&) rrr)chunkchunkscrars r rz)XAIWebSearchProvider._collect_output_textsV""8,$.fd#; $%$((6*:i*GJW[W_W_`iWjlpLqi  %&599V+< +M    +1iQJquuV}c4RWXY_W`WfWfWhqyi i"(EQJquu]7KT,R\]^k\lEWXpz{|CqDqEqE EK'' jEs*A:D<*$EE#E1$EE/Ectj|}|g|r&|jd|k7r|jdgngzD]} tj|}t|tr|jdnd}t|tsNg}|d|D]}t|tr)t|jddjnd}|sA|jtt|jddj|t|jddjt|dz|s|cSy#tj t f$rY5wxYw) zParse a JSON object with a ``results`` array out of ``text``; None when absent. Whole string first, then the regex-matched block (reasoning models prefix narration).rresultsNurlr=title descriptionr@)_JSON_BLOCK_RErjgroupr~loadsJSONDecodeErrorr-rrrr$r%r&appendrrs) rrQmatch candidaterr normalizedrowrs r rz,XAIWebSearchProvider._try_parse_json_resultssq%%d+uQSWAWEKKN#3]_` "I I.0:&$/Gfjj+TGgt,/1Jv R9CC9Nc#''%,-335TV%%d3swww/C+D+J+J+LcSVWZW^W^_lnpWqSrSxSxSz|AK}LOP}P'QR  R !! " ((*5  s EE:9E:c t}g}|D]6}|jddk(r)t|jddjnd}|r||vrJ|j |d}|jd|jd} }t |t rnt | t r^d|cxkr| cxkrt|krDnnA|td|dz |j}t|dkDr|d d j}|jtd||t|d zt||k\s6|S|S) zFallback rows from ``url_citation`` annotations: URL plus ~200 chars of preceding text as the description (the annotation title is just a number).rp url_citationrr= start_index end_indexri8Nr@) setrr%r&addrrWrsrUrr) r joined_textrQseenrannrrstartends r rz.XAIWebSearchProvider._results_from_annotationss/(* C58WWV_5V#cggeR()//1\^C#+ HHSMK/1E3E%%*S#*>1CgPSCgWZ[fWgCg)#a*=eDJJL {#c)"-de"4":":"r%rfrr<r%rJr_rLrrz$tuple[Any, Optional[Dict[str, Any]]])rcr%rQrWrr%)rrrQrWrList[Dict[str, Any]])rrrz&tuple[List[str], List[Dict[str, Any]]])rr%rQrWrzOptional[List[Dict[str, Any]]])rrrr%rQrWrrrr)__name__ __module__ __qualname____doc__NAME DISPLAY_NAMEr7rj staticmethodrYr^rZ classmethodrbrrrrrr8r r2r24se D*L% )CVaa.R.R`     a a ( (,* r8r2WebSearchProvider)zagent.web_search_providerrctj|}|tdtd|ddl}ddlm}|t|g|t|j|d|dS)Nzmodule z has no attribute r) warn_oncer@) _PLUGIN_COMPAT_LAZYrAttributeErrorr importlibhermes_cli.plugin_compatrgetattr import_module)nametargetrrs r __getattr__rse $ $T *F ~wxl2DTHMNN2 h&v& 9**6!95vay AAr8r)r(rrz List[str])r(rr/rrr)&r __future__rr~loggingretypingrrrrplugins.web._commonrr rSr r r rtools.xai_httpr rr getLoggerrrrXr`r'compile MULTILINErr!r*r0r2rrrr8r rs#  ,,wwcc   8 $ NBLL9 OP 0P tK Br8