Skip to content

feat(transport): support CDR across Python and native SDK transports - #4722

Draft
TomCC7 wants to merge 27 commits into
cc/feat/cdr-unit-external-messagesfrom
cc/feat/cdr-unit-transports
Draft

TomCC7 wants to merge 27 commits into
cc/feat/cdr-unit-external-messagesfrom
cc/feat/cdr-unit-transports

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Why do we need this?

Generated message classes need CDR ports across the Python and native SDKs, typed WebRTC, and optional ROS. This complete transport unit replaces #4649, #4708, and #4712.

What is this feature?

Adds opt-in CDR over raw LCM and Zenoh, exported native SDK sources, and a three-language coordinator example. WebRTC frames retain their declared generated schema. Shared decoding rejects invalid encapsulation. The optional ROS bridge uses ROS CDR serialization.

Dependency and base: #4721 external authoring, which depends on #4366 built-in messages. Tests, examples, docs, and workflow checks stay in this unit. Default factory cutover, recording, converter, and teleop consumer changes remain separate. #4366 now includes current main through a normal merge. This branch includes that reconciled dependency.

Validation: 74 source tests, 66 installed-wheel tests, the installed Python/C++/Rust coordinator example, and 155 Rust SDK tests passed. The wheel built successfully with complete web assets. Generic native source preparation, explicit TF codecs, exact source timestamps, and shared pointcloud layout extraction are included in this SDK unit. Installed imports were checked against the wheel path. ROS tests mock ROS interfaces. No hardware or CI validation is claimed.

How to manually run this

Install this branch's matching artifacts with message-build and the native SDK dependencies. Follow the installed three-language module example. Run demo_modules.py --help, then use its interactive coordinator flow to exchange generated values through Python, C++, and Rust ports.

Which issue(s) does this PR close?

No issue closure. Consolidation replacement for #4649, #4708, and #4712.

Checklist

  • I have read and approved the CLA.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 254cb8b Previous: cb50a9d Ratio
peak memory 1627.5 MB 1541.547 MB 1.06
peak threads 360 threads 355 threads 1.01
network (transport) 2722.084 MB 2722.791 MB 1.00
disk write 1.672 MB 1.695 MB 0.99
instructions 122.819 G 114.949 G 1.07

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
6703 3 6700 54
View the top 3 failed test(s) by shortest run time
dimos.codebase_checks.test_docs_branding::test_docs_use_current_branding
Stack Traces | 0.033s run time
def test_docs_use_current_branding() -> None:
        """Fail if any file under docs/ spells the brand "DimOS" instead of "dimOS"."""
        hits = find_old_branding()
        if hits:
            listing = "\n".join(
                f"  - {p.relative_to(DIMOS_PROJECT_ROOT)}:{lineno}: {line.strip()}"
                for p, lineno, line in hits
            )
>           raise AssertionError(f'Found "DimOS" in docs/:\n{listing}\n\nThe brand is spelled "dimOS".')
E           AssertionError: Found "DimOS" in docs/:
E             - docs/development/message-source-packages.md:62: `rclpy.serialization` to convert through CDR. Native DimOS LCM/Zenoh consumers
E             - docs/development/message-source-packages.md:74: ROS workspace for a ROS node to consume them. DimOS code generation produces
E             - docs/development/message-source-packages.md:75: the DimOS language artifacts and complete schemas; it does not run `colcon`,
E           
E           The brand is spelled "dimOS".

hits       = [(PosixPath('.../docs/development/message-source-packages.md'), 62, '`rclpy.serialization` ...ment/message-source-packages.md'), 75, 'the DimOS language artifacts and complete schemas; it does not run `colcon`,')]
listing    = '  - docs/development/message-source-packages.md:62: `rclpy.serialization` to convert through CDR. Native DimOS LCM/Ze...evelopment/message-source-packages.md:75: the DimOS language artifacts and complete schemas; it does not run `colcon`,'

dimos/codebase_checks/test_docs_branding.py:43: AssertionError
dimos.codebase_checks.test_get_logger::test_no_get_logger
Stack Traces | 0.645s run time
def test_no_get_logger():
        """
        Fail if any file uses `= logging.getLogger` outside the whitelist.
        """
        violations = find_get_logger_usages()
        if violations:
            report_lines = [
                f"Found {len(violations)} forbidden use(s) of `logging.getLogger`. "
                "Use `setup_logger` instead:",
                "",
                "    from dimos.utils.logging_config import setup_logger",
                "",
                "    logger = setup_logger()",
                "",
                "If the usage is legitimate (e.g. standalone script, logging "
                "infrastructure, or third-party logger suppression), add it to the "
                "WHITELIST in dimos/codebase_checks/test_get_logger.py.",
                "",
            ]
            for path, lineno, text in violations:
                report_lines.append(f"  {path}:{lineno}: {text.strip()}")
>           raise AssertionError("\n".join(report_lines))
E           AssertionError: Found 1 forbidden use(s) of `logging.getLogger`. Use `setup_logger` instead:
E           
E               from dimos.utils.logging_config import setup_logger
E           
E               logger = setup_logger()
E           
E           If the usage is legitimate (e.g. standalone script, logging infrastructure, or third-party logger suppression), add it to the WHITELIST in dimos/codebase_checks/test_get_logger.py.
E           
E             .../pubsub/impl/rospubsub.py:49: logger = logging.getLogger(__name__)

lineno     = 49
path       = '.../pubsub/impl/rospubsub.py'
report_lines = ['Found 1 forbidden use(s) of `logging.getLogger`. Use `setup_logger` instead:', '', '    from dimos.utils.logging_config import setup_logger', '', '    logger = setup_logger()', '', ...]
text       = 'logger = logging.getLogger(__name__)'
violations = [('.../pubsub/impl/rospubsub.py', 49, 'logger = logging.getLogger(__name__)')]

dimos/codebase_checks/test_get_logger.py:128: AssertionError
dimos.codebase_checks.test_inline_heavy_imports::test_heavy_imports_are_inline
Stack Traces | 2.1s run time
def test_heavy_imports_are_inline() -> None:
        """Fail if any file imports cv2/open3d/rerun at module level."""
        hits = find_eager_heavy_imports()
        if hits:
            listing = "\n".join(
                f"  - dimos/{f}:{line}: `{module}`"
                for f, lines in sorted(hits.items())
                for line, module in lines
            )
>           raise AssertionError(
                f"Found module-level import(s) of {'/'.join(HEAVY_MODULES)}:\n{listing}\n\n"
                "These libraries load large native extensions into every process that "
                "transitively imports the module. Import them inside the function or "
                "method that uses them; imports needed only for type annotations go "
                "under `if TYPE_CHECKING:`."
            )
E           AssertionError: Found module-level import(s) of cv2/open3d/rerun:
E             - dimos/msgs/image.py:22: `cv2`
E             - dimos/msgs/pointcloud.py:24: `open3d`
E           
E           These libraries load large native extensions into every process that transitively imports the module. Import them inside the function or method that uses them; imports needed only for type annotations go under `if TYPE_CHECKING:`.

hits       = {'msgs/image.py': [(22, 'cv2')], 'msgs/pointcloud.py': [(24, 'open3d')]}
listing    = '  - dimos/msgs/image.py:22: `cv2`\n  - dimos/msgs/pointcloud.py:24: `open3d`'

dimos/codebase_checks/test_inline_heavy_imports.py:92: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

This branch was successfully deployed

1 active deployment
cachix — 254cb8b8 Deployed Oct 20, 2026 by TomCC7 via cachix-build-macos #13574
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant