Repository navigation
refactor!: Update default timeout tiers on non-storage resource clients - #664
Merged
Merged
Conversation
Previously only storage clients (dataset, key_value_store, request_queue) had per-endpoint timeout tiers. All other resource clients defaulted to 'long' (360s) for every method, even simple metadata CRUD operations. This assigns appropriate tiers based on operation complexity: - short (5s): simple get/update/delete on actors, tasks, schedules, webhooks, users, env vars, versions, and storage collection get_or_create - medium (30s): list operations, create operations, triggering runs/builds (start, build, metamorph, resurrect, reboot), schedule get_log, webhook test - no_timeout: actor.call() and task.call() which wait indefinitely for run completion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vdusek
added
adhoc
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #664 +/- ##
==========================================
+ Coverage 94.83% 94.90% +0.06%
==========================================
Files 45 45
Lines 4340 4340
==========================================
+ Hits 4116 4119 +3
+ Misses 224 221 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
vdusek
changed the title
vdusek
changed the title
Pijukatel
reviewed
Mar 10, 2026
janbuchar
removed their request for review
March 10, 2026 12:14
Contributor
|
Seems like a legit change, but I don't think I can provide any useful feedback beyond that of the other reviewers |
tobice
approved these changes
Mar 10, 2026
tobice
left a comment
tobice
left a comment
Contributor
There was a problem hiding this comment.
I skimmed the PR description — the tiering approach makes sense to me. So if you applied it consistently across the endpoints, I see no problem in that (and don't feel like reviewing them one by one — unless you really want me to 😁 ).
If you want the timeouts to be really accurate, here is an idea: You could try pulling actual request duration from Grafana for each endpoint and set the timeout based on that.
Btw just looking at it and the 99th percentile is usually ~250 ms. So even the shortest timeout is pretty comfortable.
Pijukatel
approved these changes
Mar 10, 2026
vdusek
changed the title
vdusek
added a commit
that referenced
this pull request
Apr 19, 2026
Move #664 (default timeout tier retuning) back under Refactor — it adjusts internal defaults rather than introducing a new feature. Move #636 (Python 3.10 drop) to Miscellaneous Tasks to match the precedent set by v2.0.0 (3.10 bump) and v1.0.0 (3.7 drop). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vdusek
changed the title
This was referenced Sep 4, 2026
vdusek
added a commit
to apify/apify-client-js
that referenced
this pull request
Sep 13, 2026
### Issue - Closes #1029 - This is a port of the following: - apify/apify-client-python#653 - apify/apify-client-python#664 - apify/apify-client-python#962 ### Description The client already chose a timeout per method internally, but the only public knob was the global `timeoutSecs`, so a call that legitimately takes longer had no way to ask for more time. The tiers are now public, following the Python client, and every method that sends a request takes a `timeoutSecs` option. | Tier | Default | Covers | | --- | --- | --- | | `short` | 5 s | metadata reads and writes | | `medium` | 30 s | listing, batch and trigger calls | | `long` | 360 s | downloads, uploads and streaming | | `noTimeout` | none | polling that waits for a job to finish | Every method keeps the tier its Python counterpart has. `timeoutMaxSecs` caps a single attempt and bounds the growth across retries, which doubles from the requested value (T, 2T, 4T, 8T). The six `waitForFinish` methods get the API's one-minute hold on top of their tier, so the client doesn't abort a request the API is still holding and start a second run on retry. `timeoutSecs` takes a tier name, a number of seconds, or `'noTimeout'`. The unit lives in the option name rather than in the value, the way `waitSecs` and `timeoutMaxSecs` already do. The last commit also drops the leading underscore from non-public members, as agreed in review. Where the underscore was covering a collision with a public member, the helper got a new name: `_url` to `buildUrl`, `_params` to `buildParams`, `_get` to `getResource`, `_list` to `listResources`, `_waitForFinish` to `waitForJobFinish`, `_batchAddRequests` to `addRequestBatch`, and `ApifyClient._options` to `subClientOptions`. `src/apify_api_error.ts` and `src/statistics.ts` are left for #1057. ## Testing `test/client_timeouts.test.ts` covers the tier defaults, per-call overrides, the `waitForFinish` hold, `runTimeoutSecs` and the `requestQueue(id, { timeoutSecs })` cap. `test/http_client.test.ts` covers the cap warning and the retry growth. `test/apify_api_error.test.ts` guards `clientMethod`, which reads the renamed helpers off the stack. ## Breaking changes - `timeoutSecs` on the constructor is replaced by `timeoutShortSecs`, `timeoutMediumSecs`, `timeoutLongSecs` and `timeoutMaxSecs`. - Outside the storage clients, methods used the global 360 s and now use `short` or `medium`, so a call that waited out a slow API can fail sooner. - The run timeout of `ActorClient.start()` / `call()`, `TaskClient.start()` / `call()` and `RunClient.resurrect()` is now `runTimeoutSecs`. TypeScript reports a leftover `timeout` at compile time, and the option schemas are strict, so JavaScript gets an `ArgumentValidationError` instead of a silently dropped run limit. - `client.httpClient.call()` takes `timeoutSecs` in seconds, where the axios field it replaces took milliseconds. A direct call that passed `timeout: 30000` becomes `timeoutSecs: 30`. - Methods without options gained an options parameter, and methods that take a payload gained a second one. - The `protected` helpers of `ApiClient`, `ResourceClient` and `ResourceCollectionClient` are renamed, which reaches anyone subclassing them. `KeyValueStoreClient.setRecord()` keeps its `timeoutSecs`, and a plain number still means what it did in v2, so Crawlee's `setValue()` needs no paired PR. The option now takes a tier name or `'noTimeout'` as well. The same goes for `client.requestQueue(id, { timeoutSecs })`, which caps the tier of every request that queue client sends. *✍️ Drafted by Claude Code*
B4nan
pushed a commit
to apify/apify-client-js
that referenced
this pull request
Sep 30, 2026
### Issue - Closes #1029 - This is a port of the following: - apify/apify-client-python#653 - apify/apify-client-python#664 - apify/apify-client-python#962 ### Description The client already chose a timeout per method internally, but the only public knob was the global `timeoutSecs`, so a call that legitimately takes longer had no way to ask for more time. The tiers are now public, following the Python client, and every method that sends a request takes a `timeoutSecs` option. | Tier | Default | Covers | | --- | --- | --- | | `short` | 5 s | metadata reads and writes | | `medium` | 30 s | listing, batch and trigger calls | | `long` | 360 s | downloads, uploads and streaming | | `noTimeout` | none | polling that waits for a job to finish | Every method keeps the tier its Python counterpart has. `timeoutMaxSecs` caps a single attempt and bounds the growth across retries, which doubles from the requested value (T, 2T, 4T, 8T). The six `waitForFinish` methods get the API's one-minute hold on top of their tier, so the client doesn't abort a request the API is still holding and start a second run on retry. `timeoutSecs` takes a tier name, a number of seconds, or `'noTimeout'`. The unit lives in the option name rather than in the value, the way `waitSecs` and `timeoutMaxSecs` already do. The last commit also drops the leading underscore from non-public members, as agreed in review. Where the underscore was covering a collision with a public member, the helper got a new name: `_url` to `buildUrl`, `_params` to `buildParams`, `_get` to `getResource`, `_list` to `listResources`, `_waitForFinish` to `waitForJobFinish`, `_batchAddRequests` to `addRequestBatch`, and `ApifyClient._options` to `subClientOptions`. `src/apify_api_error.ts` and `src/statistics.ts` are left for #1057. ## Testing `test/client_timeouts.test.ts` covers the tier defaults, per-call overrides, the `waitForFinish` hold, `runTimeoutSecs` and the `requestQueue(id, { timeoutSecs })` cap. `test/http_client.test.ts` covers the cap warning and the retry growth. `test/apify_api_error.test.ts` guards `clientMethod`, which reads the renamed helpers off the stack. ## Breaking changes - `timeoutSecs` on the constructor is replaced by `timeoutShortSecs`, `timeoutMediumSecs`, `timeoutLongSecs` and `timeoutMaxSecs`. - Outside the storage clients, methods used the global 360 s and now use `short` or `medium`, so a call that waited out a slow API can fail sooner. - The run timeout of `ActorClient.start()` / `call()`, `TaskClient.start()` / `call()` and `RunClient.resurrect()` is now `runTimeoutSecs`. TypeScript reports a leftover `timeout` at compile time, and the option schemas are strict, so JavaScript gets an `ArgumentValidationError` instead of a silently dropped run limit. - `client.httpClient.call()` takes `timeoutSecs` in seconds, where the axios field it replaces took milliseconds. A direct call that passed `timeout: 30000` becomes `timeoutSecs: 30`. - Methods without options gained an options parameter, and methods that take a payload gained a second one. - The `protected` helpers of `ApiClient`, `ResourceClient` and `ResourceCollectionClient` are renamed, which reaches anyone subclassing them. `KeyValueStoreClient.setRecord()` keeps its `timeoutSecs`, and a plain number still means what it did in v2, so Crawlee's `setValue()` needs no paired PR. The option now takes a tier name or `'noTimeout'` as well. The same goes for `client.requestQueue(id, { timeoutSecs })`, which caps the tier of every request that queue client sends. *✍️ Drafted by Claude Code*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Update default timeout tiers on individual resource client methods.
Tier philosophy
short(5s)medium(30s)long(360s)no_timeoutwait_for_finish,call, streaming logs)Current state
Only storage clients (Dataset, KVS, RQ) were tiered in PR #653. Everything else defaults to
'long'— meaning even a simpleactor.get()has a 360s timeout.Changes description
ActorClient
get()update()delete()start()call()build()default_build()last_run()validate_input()ActorCollectionClient
list()create()ActorVersionClient
get()update()delete()ActorVersionCollectionClient
list()create()ActorEnvVarClient
get()update()delete()ActorEnvVarCollectionClient
list()create()BuildClient
get()delete()abort()get_open_api_definition()wait_for_finish()BuildCollectionClient
list()RunClient
get()update()delete()abort()wait_for_finish()metamorph()resurrect()reboot()get_streamed_log()charge()get_status_message_watcher()RunCollectionClient
list()TaskClient
get()update()delete()start()call()get_input()update_input()last_run()TaskCollectionClient
list()create()ScheduleClient
get()update()delete()get_log()ScheduleCollectionClient
list()create()WebhookClient
get()update()delete()test()WebhookCollectionClient
list()create()WebhookDispatchClient
get()WebhookDispatchCollectionClient
list()StoreCollectionClient
list()LogClient
get()get_as_bytes()stream()UserClient
get()monthly_usage()limits()update_limits()Storage Collection Clients (currently all
long)DatasetCollectionClient.list()DatasetCollectionClient.get_or_create()KeyValueStoreCollectionClient.list()KeyValueStoreCollectionClient.get_or_create()RequestQueueCollectionClient.list()RequestQueueCollectionClient.get_or_create()Summary of Changes
long→short(simple CRUD on non-storage resources)long→medium(list operations, creates, trigger operations)long→no_timeout(actor.call(),task.call())long(log operations, streaming, data-heavy ops)Test plan