Repository navigation
convert: read and write MuData (.h5mu) (#31) - #72
Conversation
Reading writes one AnnData per modality (or the one --mod names): the modality copied as stored, then the global obs/var columns, obsm/varm, obsp/varp and uns taken onto its rows through obsmap/varmap, in the modality's order, a block at a time; graphs remapped through on-disk buckets, CSR or CSC kept. A global name the modality already has keeps the modality's own, with a warning; columns pulled from a modality (<mod>:col) go back only to it. Without maps, rows match by name; a map that does not place each row once is refused before writing. .h5mu files and MuData Zarr stores are sniffed apart from AnnData. Writing splits var by feature type as muon does (Gene Expression -> rna, Peaks -> atac, Antibody Capture -> prot), or writes the whole AnnData under --mod: each modality's X, layers, var, varm, varp and raw in one fan-out pass; obs, obsm, obsp and uns once, globally; maps, masks, root attributes and the 512-byte MuData userblock. Also: anndata 0.7's dataframes (encoding-version 0.1.0) broke every copy, since their categoricals point into __categories by an object reference that cannot leave the file: subset wrote an h5ad anndata could not read, and Zarr outputs were refused. Copies now write them as current categorical groups in a 0.2.0 frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02f1b999b8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| stems = unique_names(names) | ||
| for (label, _), name in zip(counted, names): | ||
| mods.append((stems[name] if names.count(name) > 1 else name, | ||
| np.flatnonzero(types == label))) |
There was a problem hiding this comment.
When two different feature-type labels normalize to the same modality name, such as Foo Bar and foo_bar, unique_names(names) returns a dictionary keyed by that repeated name, so both iterations select the same final value (for example, foo_bar_1). The write then attempts to create the same modality group twice and fails after staging has begun; assign unique names by position and detect any remaining collision while planning.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 5935e2b: modality names are claimed by position (claim over the names taken so far), so Foo Bar and foo_bar become foo_bar and foo_bar_1. Test: test_feature_types_that_meet_as_names_get_a_modality_each.
| appender = CsrAppender(parent, name, shape=(out_major, out_minor), data_dtype=data.dtype, | ||
| compression=compression) |
There was a problem hiding this comment.
When importing a compressed or int64-indexed global sparse obsm, varm, obsp, or varp element without --compression, this constructs CsrAppender without source templates or index_dtype. Its defaults therefore rewrite indices as int32 and create fresh uncompressed/default-chunked datasets, rather than retaining the source's index dtype, chunks, and compression as other subset paths do.
AGENTS.md reference: AGENTS.md:L134-L135
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 5935e2b: CsrAppender takes optional templates (the source's data, indices, indptr), used for chunks and codecs, and the source's indices dtype (and a wider indptr dtype) is kept. Test: test_a_remapped_sparse_element_keeps_its_layout (int64, gzip in → int64, gzip out).
| kw = fit_chunks( | ||
| dataset_create_kwargs(codes_src, target_backend=target_backend, dst_parent=group), | ||
| codes.shape, backend=target_backend, | ||
| ) | ||
| create_dataset(group, "codes", data=np.asarray(codes), **kw) |
There was a problem hiding this comment.
When a legacy categorical is row-selected—such as a global anndata 0.7 obs column imported from MuData—the codes is not None branch forwards only the source creation settings and never applies the supplied compression. Consequently --compression gzip can compress the categories while leaving the codes uncompressed or in their original codec, despite the explicit conversion option.
AGENTS.md reference: AGENTS.md:L134-L135
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 5935e2b: taken legacy codes go through with_compression when --compression is given. (The categories are HDF5 strings, which with_compression leaves uncompressed by design.) Test: test_compression_applies_to_legacy_codes_taken_onto_a_modality.
Test Results (py3.13)1 791 tests 1 791 ✅ 4m 45s ⏱️ Results for commit 7152d9e. ♻️ This comment has been updated with latest results. |
Test Results (py3.12)1 791 tests 1 791 ✅ 3m 59s ⏱️ Results for commit 7152d9e. ♻️ This comment has been updated with latest results. |
…acy codes compressed - Feature types that normalise to one name (`Foo Bar`, `foo_bar`) get distinct modality names by position; they shared one, and the second group failed with the output half written. - Global sparse elements taken onto a modality keep the source's index dtypes, chunks and codecs (CsrAppender takes the source arrays as templates). - --compression applies to legacy categorical codes taken by row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5935e2bed0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| own = read_names(root["mod"][mod], axis) | ||
| missing = [n for n in own if n not in lookup] | ||
| if missing: | ||
| raise ValueError( | ||
| f"There is no {axis}map/{mod}, and {len(missing):,} of its {axis} " | ||
| f"names are not global ones (first: {missing[0]!r})." | ||
| ) | ||
| warnings.append(f"No {axis}map/{mod}: its {axis} are matched to the global ones by name.") | ||
| return np.fromiter((lookup[n] for n in own), dtype=np.int64, count=len(own)), n_global |
There was a problem hiding this comment.
When an older MuData has no map and a modality contains duplicate obs/var names, both rows are assigned the same global position here. Dense global elements happen to duplicate those rows, but _positions() can retain only the last local position, so importing a sparse global obsm/varm/obsp/varp silently leaves earlier duplicate rows empty or drops their edges. Reject repeated modality names during planning, as is already done for repeated global names, rather than producing a corrupted sparse element.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 80ad2e9: without a map, a modality whose names repeat is refused in the plan, as repeated global names already were. Test: test_without_maps_repeated_modality_names_are_refused.
| if options.mod is not None and (not options.mod or "/" in options.mod): | ||
| raise ValueError(f"--mod {options.mod!r}: a modality name cannot be empty or hold '/'.") |
There was a problem hiding this comment.
With --mod . or --mod .., this validation passes even though those names denote the current or parent group rather than a storable child name (as the imported storable() helper documents). The write later calls mod_group.create_group(name), so it cannot create the intended modality and fails only after staging has begun or targets the wrong group. Reject these reserved names alongside empty names and names containing /.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 80ad2e9: --mod must equal storable(--mod), so "", ".", ".." and names holding "/" are refused before writing. Tests: the parametrised test_export_refusals_leave_nothing.
…d names Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 80ad2e9463
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| keep = set(read_names(src, "var", rows)) if "raw" in src else None | ||
| targets.append(Target(group, None, None if len(rows) == detail.n_var else rows, keep)) |
There was a problem hiding this comment.
When --mod is used, or when no feature-types column exists, the single modality is supposed to contain the whole AnnData, but this still builds var_keep from the current var names. _fan_out_raw interprets that set as a selection, so a common filtered AnnData whose raw snapshot contains additional genes silently loses those genes from raw/var, raw/X, and raw/varm. Pass None for var_keep when rows covers the full current axis, just as var_idx is already set to None in that case.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in f5aa0b0: the whole AnnData as one modality (--mod, or no feature-types column) now passes var_keep=None, so raw is kept whole. Going further for the split case: each modality takes raw's features of its own type by raw/var's feature-types column when it has one, so raw-only genes and peaks land in rna and atac; without such a column, raw's features var lacks are left out with a warning naming how many. Tests: test_the_whole_anndata_as_one_modality_keeps_raw_whole, test_split_modalities_take_raws_features_of_their_type, test_raw_features_no_modality_can_claim_are_left_out_with_a_warning.
…feature types Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f5aa0b0bfb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return [set(np.asarray(raw_names, dtype=object)[raw_types == types[rows[0]]].tolist()) if len(rows) else set() | ||
| for _, rows in mods] | ||
| var_names = read_names(root, "var") | ||
| keeps = [set(var_names[i] for i in rows) for _, rows in mods] |
There was a problem hiding this comment.
When a split export has non-unique raw.var_names, converting each modality's selection to a set loses occurrence identity. _fan_out_raw subsequently resolves that set with indices_from_name_set, which stops after the first occurrence of each name, so duplicate raw columns—and the corresponding raw/X and raw/varm columns—are silently dropped; if the same name occurs under multiple feature types, it can also be assigned to the wrong modality. Keep positional raw indices or reject duplicate names during planning.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in f13eea9: a split export whose raw/var names repeat is refused in the plan (names cannot place a repeated feature), pointing to --mod, which writes the whole AnnData with raw intact (no name selection then). Test: test_repeated_raw_names_refuse_a_split.
| return [set(np.asarray(raw_names, dtype=object)[raw_types == types[rows[0]]].tolist()) if len(rows) else set() | ||
| for _, rows in mods] |
There was a problem hiding this comment.
When raw/var contains a feature type that has been completely filtered out of the current var, no output modality is created for that type, and this early return silently omits all of those raw features. The fallback branch warns about unclaimable raw features, but the typed branch does not; detect these unmatched raw types during planning and warn or refuse before writing.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in f13eea9: raw feature types with no modality (filtered out of var entirely) are now reported in the plan with how many features they hold. Test: test_raw_features_of_a_type_var_no_longer_has_are_reported.
…ypes var lacks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f13eea9607
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if mudata.is_mudata_root(handle) or isinstance(handle.get("mod"), h5py.Group): | ||
| return False |
There was a problem hiding this comment.
For an AnnData 0.7-era HDF5 file with obs and var but no root encoding attributes, an unrelated top-level mod group now makes this sniffer return false even when mudata.sniff() has already rejected the file as MuData. If such a store uses .h5 or another non-.h5ad extension, detect_format() no longer recognizes it by content and conversion fails unless the user explicitly supplies --from h5ad; before this change, the obs/var fallback accepted it. Only reject roots that actually satisfy the MuData signature.
AGENTS.md reference: AGENTS.md:L96-L101
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 416f2da: _sniff_h5ad turns a file away only when mudata.sniff takes it (root encoding-type: mudata, or muon's MuData userblock with a mod group); a bare mod group no longer does. Test: test_an_old_anndata_with_a_mod_group_is_still_an_anndata.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 416f2da296
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if len(mods) == 1 and len(mods[0][1]) == n_var: | ||
| return [None] |
There was a problem hiding this comment.
When feature-type splitting is active but current var contains only one surviving type—for example, RNA-only var with a raw snapshot that still contains Peaks—this returns None before inspecting raw's types. _fan_out_raw consequently copies every raw feature into the RNA modality instead of reporting and excluding features whose type has no modality, contaminating that modality's raw matrix. The fresh evidence after the prior fix is that this early return still checks only modality count and coverage, not whether types_col is active; whole-raw behavior should be reserved for --mod or an unsplit export.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in fba0c5c: raw is kept whole only for an unsplit export (--mod, or no feature-types column); with a types column active, even a single modality takes only raw's features of its type, and the others are reported. Test: test_one_type_left_in_var_still_splits_raw_by_type.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fba0c5cf7f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if options.mod is not None and options.mod not in mods: | ||
| raise ValueError(f"No modality {options.mod!r}; this file has: {', '.join(mods)}.") | ||
| wanted = [options.mod] if options.mod is not None else mods | ||
| stems = unique_names(wanted) |
There was a problem hiding this comment.
When modality names include a b, a_b, and a_b_1, unique_names() produces a_b, a_b_1, and a_b_1, so two modalities share an output and staging path. The second write truncates the first temporary store, and the subsequent duplicate rename fails instead of producing all modalities. Claim each generated stem against every previously generated stem and reject any remaining output collision during planning.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 7ed65ec: each modality's stem is claim(sanitize_label(mod), taken) against every stem so far, so a b, a_b, a_b_1 give a_b, a_b_1, a_b_1_1. Test: test_modality_names_that_meet_as_file_names_each_get_a_file.
| templates=(data, indices, obj["indptr"]), | ||
| ) | ||
| write_runs( | ||
| blocks(), n_rows=out_major, n_targets=1, destinations=[[appender]], scratch_parent=parent, |
There was a problem hiding this comment.
For a Zarr destination whose global obsm/varm/obsp/varp contains an entry named __csr_scratch__, any sparse remap that needs sorting—for example, a modality whose map reverses rows—passes that user mapping as the scratch parent. write_runs() then tries to create its fixed __csr_scratch__ group where the legitimate output entry already exists and the conversion fails. Use a collision-free temporary name or a scratch location outside the user mapping.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 7ed65ec: the buckets' scratch group is created under the output's root, not inside the user's obsm/varm/obsp/varp mapping. Test: test_a_global_entry_named_like_the_scratch_group_is_taken_too (a Zarr output, a reversed modality, an obsp entry named __csr_scratch__).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ed65ecfcf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| var_names = read_names(root, "var") | ||
| keeps = [set(var_names[i] for i in rows) for _, rows in mods] |
There was a problem hiding this comment.
When raw/var lacks the feature-types column and the current var has the same name under two feature types, these sets both claim the single matching raw feature. This can occur when a unique raw snapshot is retained and current var names are later renamed to duplicates; the export then silently copies the same raw column into both modalities even though it can belong to only one. Detect overlapping keeps during planning and refuse the ambiguous split.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 579bcfa: when raw is shared out by var's names (raw/var has no types column), names in more than one modality are refused in the plan, pointing to a raw/var types column or --mod. Test: test_var_names_in_two_modalities_refuse_a_split_by_name.
| counted = type_counts(types) | ||
| # By position: two types can name one modality (`Foo Bar`, | ||
| # `foo_bar`), and each needs a group of its own. | ||
| taken: set = set() | ||
| for label, _ in counted: | ||
| mods.append((claim(modality_name(label), taken), np.flatnonzero(types == label))) |
There was a problem hiding this comment.
When a valid AnnData has zero variables but still declares a feature_types column, type_counts() is empty and this loop creates no modalities. The writer consequently emits an .h5mu with an empty mod group, which this converter's own importer immediately refuses as having no modalities. Create a default empty modality or reject this case during planning rather than successfully writing an unusable output.
AGENTS.md reference: AGENTS.md:L127-L128
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 579bcfa: with no features (so no types), the AnnData becomes one empty modality, rna or --mod's name; the result converts back. Test: test_an_anndata_without_features_is_one_empty_modality.
…ared by name; one modality for no features Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| types_col = "feature_types" | ||
| mods: List[Tuple[str, np.ndarray]] = [] | ||
| if types_col is not None: | ||
| types = np.asarray(read_str_all(var[types_col], parent_group=var), dtype=object) |
There was a problem hiding this comment.
For AnnData stores with a large var axis, read_str_all() retains the entire feature-types column as Python strings and then copies it into an object array, so export memory grows with the input rather than the configured chunk size. Build the modality selections incrementally from chunks or spill them to disk instead of materializing the column.
AGENTS.md reference: AGENTS.md:L6-L11
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Leaving this as is, deliberately. The feature-types column is one var-length array — the same budget as the var index, which this export reads whole anyway to write the global var (and which every converter here holds whole: names are per-axis state, not per-entry data). It does not grow with the matrix, which is what --chunk bounds. Streaming it would not lower the export's peak, since the stacked var names are the larger of the two.
| write_runs( | ||
| blocks(), n_rows=out_major, n_targets=1, destinations=[[appender]], scratch_parent=scratch, | ||
| grouped=grouped, canonical=False, counts=counts, value_dtype=data.dtype, chunk=1 << 20, | ||
| ) |
There was a problem hiding this comment.
When a global CSR/CSC entry contains duplicate coordinates, this call routes it through write_runs(), whose DuplicateSummer unconditionally coalesces those entries even though the modality map is injective. Such duplicates are valid in stored sparse matrices, so importing a modality can change their representation and numeric values through rounding or integer overflow; the remapping path should retain source duplicates rather than applying coordinate-builder semantics.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 31aa048: write_runs takes sum_duplicates=False, which the MuData remap passes, so duplicates go through untouched on both the grouped and the bucket path. Test: test_duplicate_entries_of_a_global_graph_are_kept (in-order and reversed modalities; nnz and values kept).
The remap is one to one, so a duplicate is the stored matrix's own; the bucket builder summed it as if building from coordinates. write_runs takes sum_duplicates=False for that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31aa048273
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| templates=(data, indices, obj["indptr"]), | ||
| ) | ||
| write_runs( | ||
| blocks(), n_rows=out_major, n_targets=1, destinations=[[appender]], scratch_parent=scratch, |
There was a problem hiding this comment.
For a Zarr output where the modality already contains a top-level __csr_scratch__ element and a global sparse element needs reordering, the modality element is copied into the destination root before this call makes write_runs() create its fixed scratch group in that same root, so Zarr raises because the name already exists. Fresh evidence beyond the prior scratch fix is that modality-root passthrough keys—not just global obsm/varm/obsp/varp entries—can occupy this name; use a collision-free name or a separate temporary store.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 7152d9e, at the source: csr_build.scratch_group picks __csr_scratch__, or the first __csr_scratch___<n> the store does not hold, so no copied key can collide, for this converter or any other user of the buckets. (On HDF5 the scratch was already a separate temporary file.) Test: test_a_modality_key_named_like_the_scratch_group_is_left_alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Part of #31. MuData — muon, mudata, SCENIC+ — in and out of
adata convert.Reading (
adata convert x.h5mu -o x.h5ad): one AnnData per modality, tox.<mod>.h5ad(or the one--modnames), from an.h5muor a MuData Zarr store.obsmap/varmap, in the modality's order, a block at a time. Graphs are remapped as they stream and rebuilt throughcsr_build's on-disk buckets, CSR or CSC kept.<mod>:col) go back only to it.--compressionapplies to the global parts too..h5muand MuData Zarr are sniffed apart from AnnData (both used to be taken for one).Writing (
adata convert x.h5ad -o x.h5mu): var split by feature type as muon does (Gene Expression →rna, Peaks →atac, Antibody Capture →prot), or the whole AnnData under--mod. Each modality's X, layers, var, varm, varp and raw in one_write_targetsfan-out pass (newskip/verbarguments); obs, obsm, obsp and uns once, globally;obsmap/varmap, masks, root attrs and the 512-byteMuDatauserblock. HDF5 only; a MuData Zarr output is not added.Fixed on the way: anndata 0.7's dataframes (
encoding-version0.1.0) broke every copy — their categoricals point into__categoriesby an HDF5 object reference, which cannot leave the file:subsetwrote an h5ad anndata could not read, andsubset/convertto Zarr failed ("Object of type Reference is not JSON serializable").copy_treeandsubset_axis_groupnow write them as currentcategoricalgroups in a 0.2.0 frame (tests/test_legacy_copies.py).concat's_aligned_rowsmoved tocore/subset.aligned_rowsto be shared.Verified:
minipbcite.h5mu(mudatasets): both modalities, global parts included, equal mudata 0.2.1's reading (current mudata cannot read that file'sencoding-type: AnnDatamodalities; we can).filtered_feature_bc_matrix.h5→ h5ad →.h5mu: current mudata reads it without warnings,rna(36,601) andatac(98,319) equal the AnnData's subsets.🤖 Generated with Claude Code