Skip to content

Make string-cache compatible with Strict Provenance #260

Open

Description

@saethlin

Original title: Status of provenance/aliasing issues detected by Miri

Hi, I wish I were being helpful here but I'm just reporting the status of some hackery I've done. (hours wasted here, etc)

This is one of the most-downloaded crates that roundtrips pointer values through integers. I'm looking into most such crates to understand if it's possible to rewrite such roundtrips or if there's something better that can be done to support what libraries want to do.

For this crate, I do not think that it is possible to patch out the roundtrips, but the problem isn't the round-tripping. This crate wants to store pointers inside Atom, and to do that in a well-defined way we need a type which is capable of storing provenance and also plain old bytes, so we could use a raw pointer type or MaybeUninit. But we want to uphold the niche which it currently has, so the only type which is eligible is NonNull. But putting NonNull in that field breaks string-cache-codegen, because it puts Atom in a macro_rules! pattern:

error: to use a constant of type `std::ptr::NonNull` in a pattern, `std::ptr::NonNull` must be annotated with `#[derive(PartialEq, Eq)]`
    --> /tmp/markup5ever-0.11.0/target/miri/x86_64-unknown-linux-gnu/debug/build/markup5ever-85445da718f099e3/out/generated.rs:3381:39
     |
3381 | ...html") => { $ crate :: ATOM_NAMESPACE__68_74_74_70_3A_2F_2F_77_77_77_2E_77_33_2E_6F_72_67_2F_31_39_39_39_2F_78_68_74_6D_6C } ;
     |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
    ::: interface/tree_builder.rs:89:9
     |
89   |         expanded_name!(html "template") => flags.template = true,
     |         ------------------------------- in this macro invocation
     |
     = note: this error originates in the macro `namespace_url` (in Nightly builds, run with -Z macro-backtrace for more info)

I've put up the changes that would remove the ptr-int-ptr roundtrips (as well as some aliasing issues with Box that current SB has) in a branch in case anyone is interested: master...saethlin:fix-aliasing

Activity

  1. SimonSapin commented on Aug 19, 2026

    @SimonSapin
    Member

    In addition to the constant-in-pattern issue, we want something that is 64 bits wide even of 32-bit platforms

    As a work-around, MIRIFLAGS=-Zmiri-permissive-provenance can silence this warning:

      = note: integer-to-pointer cast
      |
      = help: this program is using integer-to-pointer casts or (equivalently) `ptr::with_exposed_provenance`, which means that Miri might miss pointer bugs in this program
      = help: see https://doc.rust-lang.org/nightly/std/ptr/fn.with_exposed_provenance.html for more details on that operation
      = help: to ensure that Miri does not miss bugs in your program, use Strict Provenance APIs (https://doc.rust-lang.org/nightly/std/ptr/index.html#strict-provenance, https://crates.io/crates/sptr) instead
      = help: you can then set `MIRIFLAGS=-Zmiri-strict-provenance` to ensure you are not relying on `with_exposed_provenance` semantics
      = help: alternatively, `MIRIFLAGS=-Zmiri-permissive-provenance` disables this warning
    
  2. SimonSapin commented on Aug 20, 2026

    @SimonSapin
    Member

    we want something that is 64 bits wide even of 32-bit platforms

    This could be worked around with enough cfgs. The real issue is the competing constraints of string provenance wants use to store a real pointer, and constant-in-pattern wants us to store an integer. Even though in practice the cases where we can use a pattern (string part of the static set) is disjoint from cases where we use a pointer (heap-allocated ref-counted strings)

  3. adamreichold commented on Aug 20, 2026

    @adamreichold
    Contributor

    Even though in practice the cases where we can use a pattern (string part of the static set) is disjoint from cases where we use a pointer (heap-allocated ref-counted strings)

    Would using a union help with this? Meaning keeping a pointer a pointer and a (64-bit) integer a (64-bit) integer and figuring this out at the union access sites?

  4. RalfJung commented on Aug 20, 2026

    @RalfJung

    Could you summarize what the type looks like and what kinds of values should be used as constants? I assume it is this one?

    pub struct Atom<Static> {
        unsafe_data: NonZeroU64,
        phantom: PhantomData<Static>,
    }

    I think we do allow some raw pointers to be used in patterns, so I guess we could allow NonNull if we do things carefully enough.

  5. SimonSapin commented on Aug 20, 2026

    @SimonSapin
    Member

    Yes this is about the Atom struct. Its representation is 64 bits (with a niche for Option<Atom>) whose two least significant bits form a tag to distinguish three cases:

    • 0b00: a heap allocated pointer. alignment ensures the lower pointer bits are indeed zero
    • 0b01: a short inline string stored in the 7 upper bytes. The upper nibble of the lower byte stores the length.
    • 0b10: a string in a static (indexed) set. The upper 32 bits give the index.

    For statically known strings, we generate an atom!("example") macro that expands to a named const item that can be used in either expression or patterns. Such consts use the inline or static representation, never pointer.

  6. SimonSapin commented on Aug 20, 2026

    @SimonSapin
    Member

    we want something that is 64 bits wide even of 32-bit platforms

    This could be worked around with enough cfgs

    This might look like:

    • 64-bit plaftorms: ptr::NonNull
    • 32-bit little-endian: #[repr(C)] struct { ptr::NonNull, u32 }
    • 32-bit big-endian: #[repr(C)] struct { u32, ptr::NonNull }

    Would using a union help with this?

    I would like to use a union to encode the triple representation in the type system, but I suspect it’d hinder both having a niche for Option<Atom> and being able to use const items in patterns.

  7. changed the title [-]Status of provenance/aliasing issues detected by Miri[/-] [+]Make string-cache compatible with Strict Provenance[/+] on Aug 20, 2026
  8. added a commit that references this issue on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions