Skip to content

feat: support secure Hubble monitoring targets - #3096

Merged
imbajin merged 15 commits into
apache:masterfrom
hugegraph:cdx/hubble-monitoring-pd-secret-redaction
Jul 21, 2026
Merged

imbajin merged 15 commits into
apache:masterfrom
hugegraph:cdx/hubble-monitoring-pd-secret-redaction

Conversation

@imbajin

@imbajin imbajin commented Jul 19, 2026 •

Copy link
Copy Markdown
Member

Purpose

Provide the minimum Server/PD compatibility and security boundary needed by Hubble native
monitoring and GraphSpace administration. Matching Hubble work:
apache/hugegraph-toolchain#743

Core before → after

Area Before After
Store monitoring A Store registration address could become an HTTP monitoring target even if it was a gRPC address or malformed. Publish only a validated host + rest.port REST target (IPv4, hostname, or bracketed IPv6).
Hubble Gremlin query The two different /gremlin endpoints were easy to confuse. Hubble logs in once and calls HugeGraph REST /gremlin with its server-side token.
Direct Gremlin Server access Its token boundary was unclear. The separate TinkerPop Gremlin Server HTTP endpoint does not accept HugeGraph tokens in this PR; existing account-password authentication remains.
GraphSpace authorization Group, target, membership, and grant operations lacked a complete GraphSpace boundary. Scoped operations validate GraphSpace, role, and referenced-resource ownership; cross-space references are rejected.
Secrets and auth metadata PD config text could expose secretKey; generated role grants were editable; token cache/log handling was broad. Redact secretKey and tokens, hide generated grants, and evict only the logged-out token's role-cache entry.
Graph configuration Local graph limits could be copied too broadly into PD configs. Propagate only explicit safe cache/buffer limits, without overwriting supplied values.
Hubble browser
  │  log in once
  ▼
HugeGraph REST API `/gremlin` ── token ──> execute query                 ✅

Direct client ──────────────────> TinkerPop Gremlin Server HTTP `/gremlin`
                                    └─ HugeGraph token unsupported here  ✅

Scope

This PR does not add PD/Store credential verification, a token issuer, token support for the
separate TinkerPop Gremlin Server endpoint, alerting, notifications, or rolling-upgrade support.

Verification — head f3e0ceff7

  • Server auth/scoping: 35 passed; PD Store discovery: 1 passed.
  • Affected root reactors passed Checkstyle and RAT; git diff --check passed.
  • Independent read-only review found no blocking correctness or security issue.

Conceptual overview

Validate monitoring host and rest.port, distinguish HugeGraph REST Gremlin token authentication from TinkerPop credentials, and reject cross-GraphSpace references

Conceptual diagram; not runtime test evidence.

imbajin added 13 commits July 14, 2026 14:09
- exclude the PD secret key from generated toString output
- preserve secret access for authentication behavior
- cover the logging boundary in the PD core test suite
- derive the REST authority from the registered host and rest.port label
- share IPv4 and IPv6 validation with service discovery
- preserve the legacy discovery fallback for missing labels
- cover additive DTO output and malformed address handling
- propagate explicit schema and serializer limits to PD graph configs
- preserve values supplied by PD or callers with putIfAbsent
- exclude aliases, implicit defaults, and unrelated sensitive settings
- cover propagation and compatibility in the server unit suite
- move the address helper into the PD common module
- keep clean reactor tests independent from the repackaged service jar
- retain IPv4, IPv6, port, and malformed input coverage
- rely on the real REST deployment check for additive DTO output
- normalize legacy scoped target metadata
- cascade scoped group relationship cleanup
- apply scoped filtering before limits
- add auth isolation regression coverage
Copilot AI review requested due to automatic review settings July 19, 2026 05:21
@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. api Changes of API feature New feature tests Add or improve test cases labels Jul 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds the minimal Server/PD compatibility and security hardening needed for Hubble-native monitoring and GraphSpace-scoped administration in HugeGraph, including scoped auth metadata enforcement and safer authentication/logging behavior.

Changes:

  • Add GraphSpace-scoped authorization surfaces (scoped group API + scoped AuthManager overloads) and enforce cross-GraphSpace isolation for targets/belongs/accesses.
  • Improve Server authentication compatibility: support Gremlin HTTP Bearer auth, standards-correct Basic parsing, and prevent Bearer token leakage in logs.
  • Improve PD/Store monitoring compatibility: redact PD secret key in toString() output and provide a shared, validated Store REST address derivation.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/UnitTestSuite.java Registers new/updated unit tests covering auth + config behaviors introduced by this PR.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/GraphManagerConfigTest.java Adds unit coverage for graph-local cache/buffer config propagation rules.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/auth/HugeGraphAuthProxyTest.java Adds regression coverage to ensure Bearer tokens are not logged on auth failures.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/api/auth/LoginAPITest.java Adds regression tests to ensure login/logout/verify flows don’t log Bearer tokens.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/meta/managers/AuthMetaManagerTest.java Tests legacy GraphSpace restoration and mismatch rejection for target metadata in meta storage.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/core/AuthTest.java Expands core auth tests for scoped target metadata, schema upgrade, and scoped overload behavior.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/WsAndHttpBasicAuthHandlerTest.java Adds coverage for Gremlin HTTP auth: Bearer support, Basic parsing correctness, and rejection paths.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/StandardAuthManagerV2Test.java Tests core scoped-graphspace validation and scoped group relation deletion behavior.
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/auth/GraphSpaceGroupAPITest.java Adds API-level unit tests for scoped group behavior (filtering, payload handling, permissions).
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/auth/GraphSpaceAuthPayloadTest.java Adds API-level tests ensuring scoped payloads enforce path graphspace and reject foreign entities.
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/meta/managers/AuthMetaManager.java Ensures targets loaded from meta storage are graphspace-scoped consistently (legacy restore + mismatch rejection).
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/StandardAuthManagerV2.java Introduces graphspace-scoped overloads, scoped group naming/validation, and token-safe logging behavior.
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/StandardAuthManager.java Adds scoped overload support for standalone auth manager, enforcing single-graphspace constraints.
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/HugeTarget.java Adds GraphSpace + description metadata, map/vertex schema upgrade support, and serialization updates.
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/AuthManager.java Extends AuthManager API with scoped overload defaults to support GraphSpace-based flows.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/core/GraphManager.java Propagates explicitly configured local cache/buffer limits into attached graph configs (without overriding incoming).
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/WsAndHttpBasicAuthHandler.java Adds Gremlin HTTP Bearer auth support, fixes Basic parsing to standard Base64, and preserves authenticated identity.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeGraphAuthProxy.java Removes Bearer token value from token-validation error logs.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/auth/TargetAPI.java Adds scoped target create/list/get/update/delete flows, including graphspace enforcement and optional URL/description handling.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/auth/LoginAPI.java Sanitizes login/logout/verify debug logs to avoid printing Authorization headers.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/auth/GraphSpaceGroupAPI.java Adds scoped group CRUD API with server-generated scoped names and graphspace permission checks.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/auth/BelongAPI.java Implements graphspace-scoped belong CRUD/list operations with filtering and reference validation hooks.
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/auth/AccessAPI.java Implements graphspace-scoped access CRUD/list operations with target/group validation to prevent cross-space references.
hugegraph-pd/hg-pd-test/src/main/java/org/apache/hugegraph/pd/util/StoreRestAddressUtilTest.java Adds unit tests for validated Store REST address derivation (IPv4/IPv6/URI + invalid cases).
hugegraph-pd/hg-pd-test/src/main/java/org/apache/hugegraph/pd/rest/PDRestSuiteTest.java Wires Store REST address utility tests into PD REST suite.
hugegraph-pd/hg-pd-test/src/main/java/org/apache/hugegraph/pd/core/PDCoreSuiteTest.java Adds PDConfig redaction test into PD core suite.
hugegraph-pd/hg-pd-test/src/main/java/org/apache/hugegraph/pd/core/PDConfigTest.java Verifies PDConfig.toString() does not expose secretKey.
hugegraph-pd/hg-pd-service/src/main/java/org/apache/hugegraph/pd/service/SDConfigService.java Uses shared Store REST address derivation with a gRPC-address fallback when invalid/missing.
hugegraph-pd/hg-pd-service/src/main/java/org/apache/hugegraph/pd/rest/StoreAPI.java Exposes derived restAddress in store statistics output for monitoring/discovery.
hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/config/PDConfig.java Redacts secretKey from generated config/log output via Lombok @ToString.Exclude.
hugegraph-pd/hg-pd-common/src/main/java/org/apache/hugegraph/pd/util/StoreRestAddressUtil.java Adds shared, validated Store REST address construction logic used by PD service & API.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov

codecov Bot commented Jul 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 18.10700% with 398 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.77%. Comparing base (a802912) to head (5b082f8).

Files with missing lines Patch % Lines
.../apache/hugegraph/api/auth/GraphSpaceGroupAPI.java 0.00% 104 Missing ⚠️
...org/apache/hugegraph/auth/StandardAuthManager.java 0.00% 61 Missing ⚠️
.../java/org/apache/hugegraph/api/auth/AccessAPI.java 0.00% 51 Missing ⚠️
...g/apache/hugegraph/auth/StandardAuthManagerV2.java 41.33% 42 Missing and 2 partials ⚠️
.../java/org/apache/hugegraph/api/auth/BelongAPI.java 0.00% 41 Missing ⚠️
.../java/org/apache/hugegraph/api/auth/TargetAPI.java 0.00% 38 Missing ⚠️
...in/java/org/apache/hugegraph/auth/AuthManager.java 0.00% 20 Missing ⚠️
...ache/hugegraph/auth/WsAndHttpBasicAuthHandler.java 46.42% 12 Missing and 3 partials ⚠️
...ain/java/org/apache/hugegraph/auth/HugeTarget.java 66.66% 8 Missing and 5 partials ⚠️
...n/java/org/apache/hugegraph/core/GraphManager.java 11.11% 6 Missing and 2 partials ⚠️
... and 2 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3096      +/-   ##
============================================
- Coverage     37.16%   32.77%   -4.40%     
- Complexity      338      498     +160     
============================================
  Files           806      819      +13     
  Lines         69093    70648    +1555     
  Branches       9114     9361     +247     
============================================
- Hits          25679    23154    -2525     
- Misses        40664    44915    +4251     
+ Partials       2750     2579     -171     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The scoped auth API has proxy-boundary, standalone-mode, and graph-space isolation regressions, and monitoring discovery can emit invalid HTTP targets. Evidence: six independent static review lanes plus latest-head GitHub checks.


Map<String, Object> map = new HashMap<>();

map.put(Hidden.unHide(P.GRAPHSPACE), this.graphSpace);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ New nodes now persist graphspace (and optionally target_description) in shared target metadata, but the previous HugeTarget.property() throws AssertionError for both unknown keys. During a rolling upgrade, an older server reading a target written by a new server can therefore fail target loading and dependent authorization. Please version or feature-gate the new persisted format, avoid writing fields that old readers cannot accept where the scope can be derived from the metadata namespace, or explicitly block mixed-version operation; add an old-reader/new-writer compatibility test.

imbajin added 2 commits July 19, 2026 16:52
- delegate graphspace auth operations through the proxy
- invalidate cached token roles on logout
- hide built-in access metadata from business APIs
- require valid Store REST discovery addresses
- keep Gremlin HTTP on Basic authentication only
- call Store REST address utility directly from discovery
- remove the service test hidden by the repackaged jar
- retain Store address validation in the existing utility suite
@imbajin
imbajin merged commit 89b648a into apache:master Jul 21, 2026
18 of 19 checks passed
Adarsh-Me added a commit to Adarsh-Me/hugegraph-doc that referenced this pull request Sep 26, 2026
GroupAPI is mounted at /auth/groups on every version, and the
/graphspaces/{graphspace}/auth/groups form only exists from
apache/hugegraph#3096 onward, so on the 1.7.0 release the prefixed group
URL 404s while the page presents it as the 1.7.0+ spelling. Adds the
exception to the version notice and to the group section, in both
languages.

Refs: apache/hugegraph#3019
@imbajin
imbajin deleted the cdx/hubble-monitoring-pd-secret-redaction branch October 10, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Changes of API feature New feature size:XXL This PR changes 1000+ lines, ignoring generated files. tests Add or improve test cases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants