\eiymdZddlZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z ddl mZmZmZmZddlmZmZddlmZGd d Zy) z Dirty Arbiter Process Asyncio-based arbiter that manages the dirty worker pool and routes requests from HTTP workers to available dirty workers. N)util)get_app_workers_attributeparse_dirty_app_spec) DirtyErrorDirtyNoWorkersAvailableErrorDirtyTimeoutErrorDirtyWorkerError) DirtyProtocolmake_error_response) DirtyWorkerc <eZdZdZdj Dcgc]}t t d|zc}}}}ZdZd"dZ dZ dZ d Z d Z d Zd Zd ZdZdZdZdZdZdZdZdZd#dZdZdZdZdZdZdZdZ dZ!dZ"d$d Z#d!Z$ycc}}}}w)% DirtyArbitera4 Dirty arbiter that manages the dirty worker pool. The arbiter runs an asyncio event loop and handles: - Spawning and managing dirty worker processes - Accepting connections from HTTP workers - Routing requests to available dirty workers - Monitoring worker health via heartbeat z HUP QUIT INT TERM USR1 USR2 CHLDzSIG%sNc ||_||_d|_tj|_||_tjd|_ |xs*tjj|jd|_ i|_ i|_i|_i|_i|_d|_d|_d|_d|_d|_i|_i|_i|_i|_i|_g|_|j;y)z Initialize the dirty arbiter. Args: cfg: Gunicorn config log: Logger socket_path: Path to the arbiter's Unix socket pidfile: Well-known PID file location for orphan detection Nzgunicorn-dirty-)prefixz arbiter.sockrT)cfglogpidosgetpidppidpidfiletempfilemkdtemptmpdirpathjoin socket_pathworkersworker_socketsworker_connections worker_queuesworker_consumers_worker_rr_index worker_agealive_server_loop_pending_requests app_specsapp_worker_mapworker_app_map_app_rr_indices_pending_respawns_parse_app_specs)selfrrrrs L/var/www/ceduoda/envi/lib/python3.12/site-packages/gunicorn/dirty/arbiter.py__init__zDirtyArbiter.__init__5sIIK  &&.?@ & "'',, KK+   "$ " !   !#  !!# c,|jjD]H}t|\}}| t|}|||d|j|<t|j|<Jy#t$r'}|j j d||Yd}~Xd}~wwxYw)a Parse all app specifications from config. Populates self.app_specs with parsed information about each app, including the import path and worker count limits. Worker count priority: 1. Config override (e.g., "module:Class:2") - highest priority 2. Class attribute (e.g., workers = 2 on the class) 3. None (all workers) - default Nz,Could not read workers attribute from %s: %s) import_path worker_count original_spec) r dirty_appsrr Exceptionrwarningr+setr,)r1specr6r7es r2r0zDirtyArbiter._parse_app_specsgsHH'' 5D(s r2r]zDirtyArbiter.runs99;  8$((C << D$,,,+GGCM*+ ""4(  ?+ ! KK) *    '++ D  !?CC D!       sT D#%DD#ED D ED;;E E EEEE$c|jD]&}tj|tj(tjtj|jtjtj |jtjtj |jtjtj|jtjtj|jtjtj|jy)zSet up signal handlers.N) SIGNALSsignalSIG_DFLSIGTERM_signal_handlerSIGQUITSIGINTSIGHUPSIGUSR1SIGCHLD)r1sigs r2rZzDirtyArbiter.init_signalss<< /C MM#v~~ . /  fnnd&:&:; fnnd&:&:; fmmT%9%9: fmmT%9%9: fnnd&:&:; fnnd&:&:;r4c|tjk(r+jrjjfdy|tjk(rj j y|tjk(r+jrjjfdyd_jr&jjjyy)zHandle signals.cJtjjSN)r\ create_task_handle_sigchldr1sr2z.DirtyArbiter._signal_handler..sG//0D0D0FGr4NcJtjjSrp)r\rqreloadrssr2rtz.DirtyArbiter._signal_handler..sG// >r4F) rdrlr)call_soon_threadsaferkr reopen_filesrjr' _shutdown)r1rmframes` r2rgzDirtyArbiter._signal_handlers &.. zz //G  &.. HH ! ! #  &-- zz //>  :: JJ + +DNN ; r4cR|jr|jjyy)zInitiate async shutdown.N)r(closerss r2ryzDirtyArbiter._shutdown s << LL    r4cKtj|_tjj |j rtj|j tj|j|j d{|_ tj|j d|jjd|j |jd{tj|j!} |j4d{|jj#d{dddd{|j' |d{|j)d{y777o7O7A#1d{7swYQxYw#tj$$rYiwxYw7V#tj$$rYiwxYw7Y#|j' |d{7n#tj$$rYnwxYw|j)d{7wxYww)z/Main async loop - start server, manage workers.)rNizDirty arbiter listening on %s)r\get_running_loopr)rrexistsrunlinkstart_unix_server handle_clientr(chmodrrTmanage_workersrq_worker_monitor serve_foreverCancelledErrorcancelstop)r1 monitor_tasks r2r^zDirtyArbiter._run_asyncs--/  77>>$** + IId&& '%66   !!   !!5)  5t7G7GH!!###**4+?+?+AB  || 3 3ll00222 3 3    ! """))+  9  $  32 3 3 3 3%%    #))       ! """))  ))+  s-BIFA"I5F6'IF9.F/F92F$F F$ F9 F"!F9%I6G;G<GIG-IIF9 F$"F9$F6*F- +F62F99G G/GG/GG*'I)G**I/IH H H  I H# I"H##I9H<:IIcK|jrtjdd{tj|j k7r3|j jdd|_|jy|jd{|jd{|jryy77-7w)z1Periodically check worker health and manage pool.g?Nz+Parent changed, shutting down dirty arbiterF) r'r\sleeprgetppidrrr;rymurder_workersrrss r2rzDirtyArbiter._worker_monitor8sjj--$ $ $zz|tyy(  !NO"  %%' ' '%%' ' 'jj $ ( 's:%CB?A+CCC+C,C=CCCczK|j|jr|jd{yy7w)z#Handle SIGCHLD - reap dead workers.N) reap_workersr'rrss r2rrzDirtyArbiter._handle_sigchldGs3  ::%%' ' '  's 0;9;ctK|jjd |jrE tj|d{}|j||d{|jrE|j |jd{y7U#t j $rYAwxYw7V#t$r&}|jjd|Yd}~qd}~wwxYw7T#t$rYywxYw#|j |jd{7w#t$rYwwxYwxYww)z Handle a connection from an HTTP worker. Routes requests to available dirty workers and returns responses. Supports both regular responses and streaming (chunk-based) responses. z&New client connection from HTTP workerNzClient connection error: %s) rdebugr'r read_message_asyncr\IncompleteReadError route_requestr:errorr| wait_closed)r1readerwritermessager>s r2rzDirtyArbiter.handle_clientNs ?@ **$1$D$DV$LLG ((&999** LLN ((***M22: = HHNN8! < < = +   LLN ((***  sD8 B7BBB B7B5B70D8C+C)C+D8BB2/B71B22B77 C&C!C:!C&&C:)C++ C74D86C77D8:D5 D&D" D&%D5& D2/D51D22D55D8cK|jdd}|jd}|j|d{}|h|js td}n%|r|jr t |}n td}t ||}tj||d{y||jvr|j|d{|j|}tjj} |j||| fd{ | d{y777a77 #t$rC} t |t!d| |}tj||d{7Yd} ~ yd} ~ wwxYww)a Route a request to an available dirty worker via queue. Each worker has a dedicated queue and consumer task. Requests are submitted to the queue and processed sequentially by the consumer. For streaming responses, messages (chunks) are forwarded directly to the client_writer as they arrive from the worker. Args: request: Request message dict client_writer: StreamWriter to send responses to client idunknownrLNzNo dirty workers availablezRequest failed:  worker_id)rB_get_available_workerr rr+rr r write_message_asyncr#_start_worker_consumerr\r~ create_futureputr:r ) r1request client_writer request_idrLrKrresponsequeuefuturer>s r2rzDirtyArbiter.route_requestisp[[y1 ;;z* 55h??  <<"#?@dnn4X>"#?@*:u=H33M8L L L  T// /--j9 9 9"":.))+99;ii-8999 MLL5@ M : :  M* #3A3!7:NH 33M8L L L  Ms8FD)A(F#D+$'F D- A FD/FD3#D1$D3(F+F-F/F1D33 E?<3E:/E20E:5F:E??FcKtjj<fd}tj|}|j<yw)z3Start a consumer task for a worker's request queue.cKjry jd{\}}} j||d{|js|j dj jrxyy7e7G#t $r+}|js|j |Yd}~Sd}~wwxYw#jwxYw#tj$rYywxYwwrp) r'rB_execute_on_workerdone set_resultr: set_exception task_doner\r)rrrr>rr1rKs r2consumerz5DirtyArbiter._start_worker_consumer..consumers**;@99;5F2G]F *"55& &{{}"--d3 )**5F %4%{{}"0034)--s C3CB CBB %B*C: C3C3 C B C!B=8C=CCCCC0-C3/C00C3N)r\Queuer#rqr$)r1rKrtaskrs`` @r2rz#DirtyArbiter._start_worker_consumersK ).:& $""8:.,0j)sAAcK|jdd} |j|d{\}}tj||d{ t j tj ||jjd{}|jd} | tjk(rtj||d{| tjk(rtj||d{y| tjtjfvrtj||d{y|j j#d| 7B7$7#tj$rLt|td|jj}tj||d{7YywxYw777#t$$rq} |j j'd|| |j)|t|t+d | | }tj||d{7Yd} ~ yd} ~ wwxYww) a Execute request on a specific worker (called by consumer). Handles both regular responses and streaming (chunk-based) responses. For streaming, chunk and end messages are forwarded directly to the client_writer as they arrive from the worker. rrN)timeoutzWorker timeouttypez$Unknown message type from worker: %sz Error executing on worker %s: %szWorker communication failed: r)rB_get_worker_connectionr rr\wait_forrr dirty_timeout TimeoutErrorr r MSG_TYPE_CHUNK MSG_TYPE_ENDMSG_TYPE_RESPONSEMSG_TYPE_ERRORrr;r:r_close_worker_connectionr ) r1rKrrrrrrrmsg_typer>s r2rzDirtyArbiter._execute_on_workers%[[y1 0 M#'#>#>z#JJNFF33FGD D D $+$4$4%88@ $ 6 6%G#;;v.};;;';;M7SSS}999';;M7SSS ? ? - < < >>';;M7SSS  !GRC K D ++2")*:DHH +v~ %Is$BC C,C ?CC C c~||jvr/|jj|\}}|jyy)zClose connection to a worker.N)r"rOr|)r1rK_readerrs r2rz%DirtyArbiter._close_worker_connection2s8 00 0"5599*EOGV LLN 1r4c|Kjsyjj}jrmtj|krUj }|nBt jdd{jrtj|krUtj|kDrtjjfd}j|tjt jdd{tj|kDr~yy77!w)z%Maintain the number of dirty workers.Nrc6j|jSrp)r age)pr1s r2rtz-DirtyArbiter.manage_workers..Ks4<<?+>+>r4)key) r'r dirty_workersrAr spawn_workerr\rminr kill_workerrdrf)r1 num_workersresult oldest_pids` r2rzDirtyArbiter.manage_workers8szz hh,, jjS.<&&(F~--$ $ $ jjS.<$,,+-T\\..0!>@J   Z 8--$ $ $ $,,+- % %s1A3D<6D87(D< A8D<D:D<6D<:D<c|jr|jjd}n|j}|s|jj dy|xj dz c_t jj|jd|j d}t|j |j||j|j|}t j}|dk7rr||_ ||j|<||j|<|j!|||jj#|||jj%d|||St j&|_ t)j*d |jj,d |j/t1j2dy#t4$rt6$r^|jj9d |j:st1j2|j<t1j2d YywxYw) aW Spawn a new dirty worker. Worker app assignment follows these priorities: 1. If there are pending respawns (from dead workers), use those apps 2. Otherwise, determine apps for a new worker based on allocation Returns: Worker PID in parent process, or None if no apps need workers rz)No apps need more workers, skipping spawnNrzworker-z.sock)rrrDrrrz,Spawned dirty worker (pid: %s) with apps: %szdirty-worker []z!Exception in dirty worker process)r/rOrFrr;r&rrrrr rrforkr r!rMdirty_post_forkrTrrr[ proc_name init_processsysexit SystemExitr: exceptionbootedWORKER_BOOT_ERROR)r1rDrworkerrs r2rzDirtyArbiter.spawn_workerOs  ! !..2215I557I HH  H I 1ggll KK74??"359 #  ggi !8FJ &DLL '2D   $  & &sI 6 HH $ $T6 2 HHMMHy *JYY[     0B0B/C1E F    ! HHQK    HH  B C==//0 HHRL  sAGA-IIc tj||y#t$r=}|jtjk(r|j |Yd}~yYd}~yd}~wwxYw)zKill a worker by PID.N)rkillOSErrorerrnoESRCH_cleanup_worker)r1rrmr>s r2rzDirtyArbiter.kill_workersK * GGC  *ww%++%$$S))& *s A.AAc|j|||jvr*|j|j|j|=|jj |d||j vr5t |j |}|r|jj||j||jj |d}|r|jj|||jj |d}|r7tjj!|r tj"|yyy#t$$rYywxYw)z Clean up after a worker exits. Saves the dead worker's app list to pending respawns so the replacement worker gets the same apps. N)rr$rr#rOr-rHr/rCrQr rdirty_worker_exitr!rrrrr)r1r dead_appsrrs r2rzDirtyArbiter._cleanup_workers. %%c* $'' '  ! !# & - - /%%c* sD) $%% %T0056I&&--i8 $!!#t,  HH & &tV 4))--c48 277>>+6  +&7;  s2E EEc,K|jjsyt|jj D]\}} t j |jjz |jjkrN |jsD|jjd|d|_ |j|tj |j|tj"y#ttf$rYwxYww)z!Kill workers that have timed out.NzDIRTY WORKER TIMEOUT (pid:%s)T)rrrHr r@time monotonictmp last_updater ValueErrorabortedrcriticalrrdSIGABRTSIGKILL)r1rrs r2rzDirtyArbiter.murder_workerssxx%%  2 2 45 6KC >>#fjj&<&<&>>$((BXBXXY >>!!"A3G!%  fnn5  fnn5 6Z(  s,ADAC? A5D?DDDDc\ tjdtj\}}|syd}tj|rtj|}nGtj |r2tj |}|jjd||||jk(r|jjd||j||jjd|#t$r(}|jtjk7rYd}~yd}~wwxYw)zReap dead worker processes.rNz)Dirty worker (pid:%s) killed by signal %sz$Dirty worker failed to boot (pid:%s)zDirty worker exited (pid:%s))rwaitpidWNOHANG WIFEXITED WEXITSTATUS WIFSIGNALEDWTERMSIGrr;rrrrTrrECHILD)r1wpidstatusexitcodermr>s r2rzDirtyArbiter.reap_workerss !zz"bjj9 f<<'!~~f5H^^F+++f-CHH$$%P%)30t555HHNN#I4P$$T* >$** +  $**+  ZZ , 8 "'',,t{{A67 8 HHT[[ !  7B%       s6EE BE% EE E"!E"% E10E1)NNrp)T)%__name__ __module__ __qualname____doc__splitgetattrrdrcrr3r0rFrMrQr]rZrgryr^rrrrrrrrrrrrrrrrrvrr`).0xrrds0000r2rr$s2779;;wvw{+;G0 d 5D6:$B"!: <<8! %N ((62Mh14:Mx(7T* %.?B*"H6&2 2.C};sB r)rr\rrrdrrrgunicornrapprrerrorsrrr r protocolr r rr rr4r2rsI   @  A CA Cr4