Repository navigation
SK-3075: Document the new Detect methods for the skyvault and flowvault SDKs - #287
Open
promptless[bot] wants to merge 30 commits into
Open
promptless[bot] wants to merge 30 commits into
promptless[bot] wants to merge 30 commits into
Conversation
Implement Detect V2 (deidentify_string, deidentify_file, get_run, reidentify_string, reidentify_file, check_guardrails) in the shared `common` package and re-export it from both flowvault and skyvault. common: - detect models, controller, request/response mappers, run polling, file-output writer, and validations under common/vault/detect - V2 detect enums under common/utils/enums/detect.py; V2 detect messages in common SkyflowMessages - behaviour ported from the Java flowvault SDK @ flowvault-release/26.10.1 (exponential poll backoff 1..32, submit-only status None, minimal client-side validation, auto-write of processed files) flowvault: - wire DetectController via make_skyflow_class(detect_cls=...) - build the common detect REST client on VaultClient and expose get_strings_v2_api / get_files_v2_api / get_guardrails_v2_api - re-export skyflow.vault.detect and the V2 enums - detect_api samples, README section, and unit tests skyvault: - replace the removed V1 detect API with V2 (same method/class names) - migrate the hand-written layer to the regenerated client (drop V1 prefix, Skyflow -> SkyflowAuth) and update tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lines - common/skyvault generated rest version.py: guard metadata.version() with try/except PackageNotFoundError (matches flowvault), so importing the package never fails when the distribution metadata is absent. - Refresh flowvault/skyvault public-API contract baselines for the Detect V2 surface (flowvault adds the V2 enums; skyvault replaces V1 detect with V2). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The common package CI runs tests with `python -m unittest` (no pytest installed), so rewrite common/tests/vault/detect/test_detect.py as a unittest.TestCase (self.assertRaises, tempfile instead of the pytest tmp_path fixture). Runs green under both unittest and pytest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![promptless[bot]](https://avatars.githubusercontent.com/in/979718?s=60&v=4){kind=small}
Author
|
I noticed that some CI checks failed for this PR. I'm investigating whether the failures are caused by this suggestion. If they're unrelated or pre-existing, I'll leave this suggestion unchanged and create a separate suggestion if a standalone docs fix is needed. |
promptless
Bot
changed the title
…e and ReidentifySourceType
Mirror the Java Detect V2 contract change: the single DataSourceType enum is split
into DeidentifySourceType (deidentify_file data_source and the output_type of
DeidentifyFileResponse / GetRunResponse) and ReidentifySourceType (reidentify_file
data_source and ReidentifyFileResponse output_type).
Like Java, the two source enums are defined per SDK rather than in common:
- skyvault DeidentifySourceType: BASE64, SKYFLOW_ID, PRESIGNED_URL
- flowvault DeidentifySourceType: BASE64, PRESIGNED_URL (no skyflow id)
- ReidentifySourceType: BASE64, PRESIGNED_URL in both (re-identification never
accepts a skyflow id)
common stays SDK-agnostic: the controller/mappers resolve the response output_type
through the per-SDK VaultClient (get_deidentify_source_type / get_reidentify_source_type),
and the file-output writer compares the output type by value. Updates both SDK enum
re-exports, the reidentify_file sample, and the unit/re-export tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"# DetectController — Detect V2" produced an awkward anchor (#detectcontroller--detect-v2) and didn't match the cross-SDK convention; the Go flowvault README uses "Detect". Renamed to "# Detect" (anchor #detect); the section intro already states it is Detect V2 and how to get the controller. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3076 add v1-deprecation warning * SK-3076 fix ci-check
The 865a4f5 regeneration renamed generated REST client methods but the consuming code was not fully updated, breaking bearer-token auth (every e2e failed with "Failed to generate bearer token") and the skyvault data-plane: - common/service_account/_utils.py: call get_bearer_token() (renamed from authentication_service_get_auth_token). - common/utils/_skyflow_messages.py: resolve the host SDK version via an importlib.metadata fallback so messages/metrics report the real version instead of 0.0.0. - skyvault/skyflow/service_account/__init__.py: re-export from common.service_account (skyvault's generated authentication client was removed by the regen), matching flowvault. - skyvault/skyflow/vault/controller/_vault.py: rewire record/token/query calls to the new generated method names; move file upload to the files client (upload_file_v2). - skyvault/skyflow/vault/client/client.py: add get_files_api(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…entifySourceType split The detect DataSourceType enum was split into per-SDK DeidentifySourceType and ReidentifySourceType, but the committed griffe baselines were not regenerated, so Contract Tests failed on public API drift. Regenerate both skyvault and flowvault baselines to record the intended enum change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…d-detect-v2-skyvault-flowvault
…d common auth Mirror the controller/auth migration in the unit tests so they exercise the methods the code now calls: - test__vault.py: rename the mocked generated methods to the new names (batch_operation/insert_records/update_record/delete_records/get_records/ execute_query/detokenize/tokenize), point file-upload at the files client (get_files_api + upload_file_v2), and drop byot from the batch_operation expectation (batch_operation no longer takes it; BYOT rides on per-record tokens). - service_account/test__utils.py and utils/test__utils.py: the public skyflow.service_account now routes through common, so patch common.service_account._utils.* and assert get_bearer_token on the mocked auth client instead of the removed authentication_service_get_auth_token. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror skyflow-java's SK-3175 change. RedactionLevel was one common class carrying all four wire fields for every vault type. Replace it with an abstract RedactionRule base in common (exposing the four wire accessors) and a per-SDK RedactionLevel: - skyvault RedactionLevel carries entity_name + redaction_type - flowvault RedactionLevel carries token_group_name + redaction_pattern The reidentify mapper reads the wire fields through the rule's accessors, so each vault type only sends the fields its API supports. Refreshed the skyvault API contract baseline for the new class. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
common.vault.detect no longer exports a concrete RedactionLevel (it exports the RedactionRule base), so the reidentify mapping test uses a small concrete RedactionRule double instead of importing RedactionLevel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The skyvault sdist omitted common/, so rebuilding a wheel from it (the release smoke test) failed looking for a sibling common/ that only exists in the repo checkout. Mirror flowvault: vendor common/ into the sdist via a CustomSdist command plus graft common, and fall back to the vendored copy when the sibling checkout is absent, so sdist-to-wheel builds and installs cleanly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
UploadFileRequestV2.file now carries format:binary in the spec, so the
regenerated upload_file_v2 sends file as a multipart part
(files={"file": file}) instead of a string form field, fixing the 500
"Expected primitive type, got <class 'list'>" from data-plane file upload.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The files-client regen also rewrote generated/rest/version.py to look up
metadata.version("skyflow.generated.rest"), a distribution that is never
installed (the module ships inside the "skyflow" dist), so it raised
PackageNotFoundError. unittest discover enumerates the package __all__
(which includes __version__), tripping the lazy import and failing
test/run-tests. Restore the metadata.version("skyflow") lookup with the
0.0.0 fallback. Durable fix belongs in the fern generator config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
No code change (empty commit). The previous push updated the PR head and the release branch to the same SHA in one step, so GitHub created only the push-triggered release run and never fired the pull_request PR CI. This empty commit fires pull_request:synchronize so test/run-tests re-runs on the restored version.py. The published 2.1.3.dev0+deaa724 wheel is built from the identical code and stays valid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirrors the Java migration doc as a repo reference: method and class mapping, request field mapping, per-method V1/V2 examples, file behaviour differences, and a migration checklist. Confirmed against the V2 detect implementation and the V1 API on main. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ples Rewrite the skyvault README Detect section from the removed V1 API to V2 (method table, configuration contract, Detect configuration reference, file run status and output handling) and replace the V1 detect samples with V2 equivalents. Mirror the same V2 Detect structure into the flowvault README with flowvault semantics (token-group destination, token_group_name / redaction_pattern) and enhance the flowvault samples to cover inline configuration, submit-only / poll / presigned, and the failure branches. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-skyvault-flowvault' into promptless/SK-3075-skyvault-detect-v2-docs # Conflicts: # skyvault/README.md # skyvault/samples/README.md # skyvault/samples/detect_api/check_guardrails.py # skyvault/samples/detect_api/deidentify_file.py # skyvault/samples/detect_api/deidentify_string.py # skyvault/samples/detect_api/get_run.py # skyvault/samples/detect_api/reidentify_file.py # skyvault/samples/detect_api/reidentify_string.py
…ult API contract Cross-SDK parity review flagged four Detect V2 gaps in the Python flowvault path versus the Java reference. All fixes live in shared common/ code and so apply to both vaults. - deidentify_file source rule now counts exactly one of file_path, file, value; a missing source and multiple sources raise distinct errors (adds the missing-source message). - Shared deidentify/reidentify-file validation messages are vault-neutral and no longer name SKYFLOW_ID, which flowvault does not offer. - With neither configuration_id nor configuration set, a configuration carrying only the vault id is sent, matching the Java behaviour. - processed_file_extension resolves case-insensitively. Also add skyflow.vault.detect to the flowvault public API contract and regenerate both api-report baselines; skyvault is unchanged, flowvault gains the detect surface. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ustomType The generated wire types already carry these fields; surface them on the public Detect config so callers can control VAULT_TOKEN tokenization, matching the Java design doc. - Add the IncludePrefix enum (YES / NO) and re-export it from both vaults' skyflow.utils.enums. - Entity gains include_prefix and prefix_override. - CustomType gains deidentification_type, destination, include_prefix and prefix_override. - The request mapper sends all of them when set; omitted fields fall through to the server defaults. - Regenerate both api-report baselines (the new public surface is the IncludePrefix enum; Entity / CustomType are re-export aliases). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-skyvault-flowvault' into promptless/SK-3075-skyvault-detect-v2-docs
# Conflicts: # skyvault/README.md # skyvault/samples/README.md # skyvault/samples/detect_api/check_guardrails.py # skyvault/samples/detect_api/deidentify_file.py # skyvault/samples/detect_api/deidentify_string.py # skyvault/samples/detect_api/get_run.py # skyvault/samples/detect_api/reidentify_file.py # skyvault/samples/detect_api/reidentify_string.py
promptless
Bot
changed the title
promptless
Bot
changed the base branch from
saileshwar/SK-3075-add-detect-v2-skyvault-flowvault
to
flowvault-release/26.10.0
November 6, 2026 12:04
promptless
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Open in Promptless
Why
PR #286 (SK-3075) adds the new Detect methods to both Python SDKs and merged into
flowvault-release/26.10.0. The skyvault docs still described the removed Detect methods and classes, and the final source commit (5831909) added public Detect options that neither README documents. The flowvault README also listed a file source that the flowvault SDK doesn't have.Goal
Document the new Detect methods for skyvault and flowvault:
skyvault/README.md: adds a note that skyflow 3.0.0 replaces the previous Detect methods and points to the migration guide (docs/migrate_detect.md), sample links for each Detect method, and the changes below.docs/api_reference.md: replaces the previous Detect request, response, helper, and enum entries with the new models and enums, includingPollOptions,DeidentifySourceType, andReidentifySourceType. Adds theIncludePrefixenum, theinclude_prefixandprefix_overridefields onEntity, and the newCustomTypefields (deidentification_type,destination,include_prefix,prefix_override). Lists where flowvault's Detect classes differ:RedactionLevelfields, noSKYFLOW_IDsource, and token-group destinations.skyvault/README.mdandflowvault/README.md: add a "Token prefixes" passage forinclude_prefixandprefix_override, list the newCustomTypefields, and explain thatPollOptions.wait_timecaps the combined wait between status checks rather than elapsed time.flowvault/README.md: removesSKYFLOW_IDfrom thedeidentify_filesources, because flowvault'sDeidentifySourceTypehas onlyBASE64andPRESIGNED_URL. The API reference link now points readers to the flowvault differences.skyvault/samples/detect_api/: replaces the five previous samples with six new samples and updatesskyvault/samples/README.md.This PR now targets
flowvault-release/26.10.0, the branch PR #286 merged into, and includes that branch.The skyvault README note and
skyvault/samples/README.mdno longer use the "Detect V2" label, following the nomenclature change in #290.Before merging
docs/migrate_detect_v1_to_v2.mdtodocs/migrate_detect.md, and this PR links to the new name. Merge SK-3289: address Detect docs and sample parity feedback (Python) #290 first, or the link is broken until it merges.flowvault/README.mdandskyvault/README.md, including theEntityandCustomTypetable rows and theSKYFLOW_IDfile source. Whichever PR merges second gets conflicts there. Both PRs make the same factual corrections, so keep one version of each row and passage rather than both.Testing
common/vault/detect/_models.py,common/utils/enums/detect.py,common/vault/detect/_polling.py, and the generatedDetectEntityandDetectCustomTypestypes at5831909.Trigger Events