i<ddlZddlZGddeZGddZGddZGdd ZGd d ZGd d ZGddZ GddZ y)NceZdZfdZxZS)RequestExceededExceptioncP||_||_d|d|}t| |y)aError when requested amount exceeds what is allowed The request that raised this error should be retried after waiting the time specified by ``retry_time``. :type requested_amt: int :param requested_amt: The originally requested byte amount :type retry_time: float :param retry_time: The length in time to wait to retry for the requested amount zRequest amount z) exceeded the amount available. Retry in N) requested_amt retry_timesuper__init__)selfrrmsg __class__s k/var/www/acinvestment.org/public_html/aci/backend/venv/lib/python3.12/site-packages/s3transfer/bandwidth.pyr z!RequestExceededException.__init__s6+$ .WXbWcd )__name__ __module__ __qualname__r __classcell__)r s@r rrs rrceZdZdZy) RequestTokenzDA token to pass as an identifier when consuming from the LeakyBucketN)rrr__doc__rr rr%sNrrceZdZdZdZy) TimeUtilsc*tjS)zgGet the current time back :rtype: float :returns: The current time in seconds )timer s r rzTimeUtils.time,s yy{rc,tj|S)zwSleep for a designated time :type value: float :param value: The time to sleep for in seconds )rsleep)r values r rzTimeUtils.sleep4s zz%  rN)rrrrrrrr rr+s !rrceZdZddZ ddZy)BandwidthLimiterNcD||_||_|t|_yy)aLimits bandwidth for shared S3 transfers :type leaky_bucket: LeakyBucket :param leaky_bucket: The leaky bucket to use limit bandwidth :type time_utils: TimeUtils :param time_utils: Time utility to use for interacting with time. N) _leaky_bucket _time_utilsr)r leaky_bucket time_utilss r r zBandwidthLimiter.__init__>s)*%  ({D  rcnt||j||j}|s|j|S)aWraps a fileobj in a bandwidth limited stream wrapper :type fileobj: file-like obj :param fileobj: The file-like obj to wrap :type transfer_coordinator: s3transfer.futures.TransferCoordinator param transfer_coordinator: The coordinator for the general transfer that the wrapped stream is a part of :type enabled: boolean :param enabled: Whether bandwidth limiting should be enabled to start )BandwidthLimitedStreamr"r#disable_bandwidth_limiting)r fileobjtransfer_coordinatorenabledstreams r get_bandwith_limited_streamz,BandwidthLimiter.get_bandwith_limited_streamLs:( T'')=t?O?O   - - / rN)T)rrrr r-rrr r r =s +6:rr c\eZdZ ddZdZdZdZdZdZdZ dd Z d Z d Z d Z d Zy)r'Nc||_||_||_||_|t |_d|_t |_d|_||_ y)a[Limits bandwidth for reads on a wrapped stream :type fileobj: file-like object :param fileobj: The file like object to wrap :type leaky_bucket: LeakyBucket :param leaky_bucket: The leaky bucket to use to throttle reads on the stream :type transfer_coordinator: s3transfer.futures.TransferCoordinator param transfer_coordinator: The coordinator for the general transfer that the wrapped stream is a part of :type time_utils: TimeUtils :param time_utils: The time utility to use for interacting with time NTr) _fileobjr"_transfer_coordinatorr#r_bandwidth_limiting_enabledr_request_token _bytes_seen_bytes_threshold)r r)r$r*r%bytes_thresholds r r zBandwidthLimitedStream.__init__dsU0  )%9"%  ({D +/(*n /rcd|_y)z0Enable bandwidth limiting on reads to the streamTNr3rs r enable_bandwidth_limitingz0BandwidthLimitedStream.enable_bandwidth_limitings +/(rcd|_y)z1Disable bandwidth limiting on reads to the streamFNr9rs r r(z1BandwidthLimitedStream.disable_bandwidth_limitings +0(rc8|js|jj|S|xj|z c_|j|jkr|jj|S|j |jj|S)zhRead a specified amount Reads will only be throttled if bandwidth limiting is enabled. )r3r1readr5r6_consume_through_leaky_bucket)r amounts r r=zBandwidthLimitedStream.reads //==%%f- - F"   d33 3==%%f- - **,}}!!&))rcr|jjs9 |jj|j|j d|_y|jj#t $r/}|jj|jYd}~nd}~wwxYw|jjsi)Nr) r2 exceptionr"consumer5r4rr#rr)r es r r>z4BandwidthLimitedStream._consume_through_leaky_buckets ,,66 5""**$$d&9&9$% ,,66 6, 5  &&q||44 5,,66s7A&& B/%BBc$|jy)z6Signal that data being read is being transferred to S3N)r:rs r signal_transferringz*BandwidthLimitedStream.signal_transferrings &&(rc$|jy)z:Signal that data being read is not being transferred to S3N)r(rs r signal_not_transferringz.BandwidthLimitedStream.signal_not_transferrings '')rc<|jj||yr.)r1seek)r wherewhences r rIzBandwidthLimitedStream.seeks 5&)rc6|jjSr.)r1tellrs r rMzBandwidthLimitedStream.tells}}!!##rc|jr|jr|j|jj yr.)r3r5r>r1closers r rOzBandwidthLimitedStream.closes1  + +0@0@  . . 0 rc|Sr.rrs r __enter__z BandwidthLimitedStream.__enter__s rc$|jyr.)rO)r argskwargss r __exit__zBandwidthLimitedStream.__exit__s  r)Ni)r)rrrr r:r(r=r>rErGrIrMrOrQrUrrr r'r'csG " !0F01*(7$)**$rr'c8eZdZ ddZdZdZdZdZdZy) LeakyBucketNct||_||_|t|_t j |_||_|t|_||_ |t|_ yy)a9A leaky bucket abstraction to limit bandwidth consumption :type rate: int :type rate: The maximum rate to allow. This rate is in terms of bytes per second. :type time_utils: TimeUtils :param time_utils: The time utility to use for interacting with time :type rate_tracker: BandwidthRateTracker :param rate_tracker: Tracks bandwidth consumption :type consumption_scheduler: ConsumptionScheduler :param consumption_scheduler: Schedules consumption retries when necessary N) float _max_rater#r threadingLock_lock _rate_trackerBandwidthRateTracker_consumption_schedulerConsumptionScheduler)r max_rater% rate_trackerconsumption_schedulers r r zLeakyBucket.__init__sl.x%  ({D ^^% )  !5!7D &;# (*>*@D ' )rcl|j5|jj}|jj |r|j |||cdddS|j ||r|j|||n|j||cdddS dddy#1swYyxYw)acConsume an a requested amount :type amt: int :param amt: The amount of bytes to request to consume :type request_token: RequestToken :param request_token: The token associated to the consumption request that is used to identify the request. So if a RequestExceededException is raised the token should be used in subsequent retry consume() request. :raises RequestExceededException: If the consumption amount would exceed the maximum allocated bandwidth :rtype: int :returns: The amount consumed N) r]r#rr` is_scheduled,_release_requested_amt_for_scheduled_request_projected_to_exceed_max_rate!_raise_request_exceeded_exception_release_requested_amtr amt request_tokentime_nows r rBzLeakyBucket.consumes$ZZ B'',,.H**77 FHH B B 33CB66223A B B B B BsAB*7B**B3cX|jj||}||jkDSr.)r^get_projected_raterZ)r rlrnprojected_rates r rhz)LeakyBucket._projected_to_exceed_max_rates)++>>sHM..rc\|jj||j||Sr.)r`process_scheduled_consumptionrjrks r rgz8LeakyBucket._release_requested_amt_for_scheduled_requests0 ##AA  **399rc|t|jz }|jj|||}t ||)N)rr)rYrZr`schedule_consumptionr)r rlrmrnallocated_timers r riz-LeakyBucket._raise_request_exceeded_exception#sFuT^^4400EE  '*  rc>|jj|||Sr.)r^record_consumption_rate)r rlrns r rjz"LeakyBucket._release_requested_amt,s 223A r)NNN) rrrr rBrhrgrirjrrr rWrWs." !AFB>/: rrWc$eZdZdZdZdZdZy)rac i|_d|_y)z*Schedules when to consume a desired amountrN) _tokens_to_scheduled_consumption _total_waitrs r r zConsumptionScheduler.__init__2s02-rc||jvS)zIndicates if a consumption request has been scheduled :type token: RequestToken :param token: The token associated to the consumption request that is used to identify the request. )r{)r tokens r rfz!ConsumptionScheduler.is_scheduled7s====rc||xj|z c_|j|d|j|<|jS)aSchedules a wait time to be able to consume an amount :type amt: int :param amt: The amount of bytes scheduled to be consumed :type token: RequestToken :param token: The token associated to the consumption request that is used to identify the request. :type time_to_consume: float :param time_to_consume: The desired time it should take for that specific request amount to be consumed in regardless of previously scheduled consumption requests :rtype: float :returns: The amount of time to wait for the specific request before actually consuming the specified amount. ) wait_durationtime_to_consume)r|r{)r rlr~rs r ruz)ConsumptionScheduler.schedule_consumption@sA& O+!--.8 --e4rc||jj|}t|j|dz d|_y)zProcesses a scheduled consumption request that has completed :type token: RequestToken :param token: The token associated to the consumption request that is used to identify the request. rrN)r{popmaxr|)r r~scheduled_retrys r rsz2ConsumptionScheduler.process_scheduled_consumptionZs=??CCEJ   /@A A1 rN)rrrr rfrursrrr rara1s > 4  rrac<eZdZddZedZdZdZdZdZ y) r_c.||_d|_d|_y)aTracks the rate of bandwidth consumption :type a: float :param a: The constant to use in calculating the exponentional moving average of the bandwidth rate. Specifically it is used in the following calculation: current_rate = alpha * new_rate + (1 - alpha) * current_rate This value of this constant should be between 0 and 1. N)_alpha _last_time _current_rate)r alphas r r zBandwidthRateTracker.__init__hs !rc4|jy|jS)zmThe current transfer rate :rtype: float :returns: The current tracked transfer rate )rrrs r current_ratez!BandwidthRateTracker.current_ratexs ?? "!!!rc@|jy|j||S)aZGet the projected rate using a provided amount and time :type amt: int :param amt: The proposed amount to consume :type time_at_consumption: float :param time_at_consumption: The proposed time to consume at :rtype: float :returns: The consumption rate if that amt and time were consumed r)r*_calculate_exponential_moving_average_rater rltime_at_consumptions r rpz'BandwidthRateTracker.get_projected_rates* ?? ">> $  rcv|j||_d|_y|j|||_||_y)aRecord the consumption rate based off amount and time point :type amt: int :param amt: The amount that got consumed :type time_at_consumption: float :param time_at_consumption: The time at which the amount was consumed Nr)rrrrs r rxz,BandwidthRateTracker.record_consumption_ratesB ?? "1DO!$D  !LL $ .rcJ||jz }|dkr tdS||z S)Nrinf)rrY)r rlr time_deltas r _calculate_ratez$BandwidthRateTracker._calculate_rates.(4??: ? < j!!rc~|j||}|j|zd|jz |jzzS)N)rrr)r rlrnew_rates r rz?BandwidthRateTracker._calculate_exponential_moving_average_rates>''-@A{{X%T[[Drsk y(  !!$##LnnbZZz3 3 lNONOr