Skip to content

document that extern statics may be bigger than the declared size - #2334

Open
RalfJung wants to merge 15 commits into
rust-lang:masterfrom
RalfJung:extern-static-oversize
Open

RalfJung wants to merge 15 commits into
rust-lang:masterfrom
RalfJung:extern-static-oversize

Conversation

@RalfJung

@RalfJung RalfJung commented Aug 22, 2026 •

Copy link
Copy Markdown
Member

I am fairly sure we discussed at some point whether it's okay to access memory beyond the declared size of an extern static, and @nikic confirmed that this is fine. But I couldn't find any official place where we document this. So, let's add it to the Reference.

I wondered if we also need the converse on the UB page: it is UB to even start the program if an extern static ends up being backed by memory that is smaller or less-aligned than the declared type. What do you think?

Cc @rust-lang/opsem
Fixes rust-lang/unsafe-code-guidelines#259
Fixes rust-lang/unsafe-code-guidelines#554

@rustbot rustbot added the S-waiting-on-review Status: The marked PR is awaiting review from a maintainer label Aug 22, 2026
Comment thread src/items/external-blocks.md Outdated

r[items.extern.static.size]
The actual memory that the extern static resolves to must have *at least* the size and alignment of the type that it was declared with in the extern block.
If the actual memory is bigger, then it is permitted to access that extra memory by creating a raw pointer to the extern static and then doing appropriate pointer arithmetic.

@Jules-Bertholet Jules-Bertholet Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps be more explicit about the need to use &raw (and avoid an intermediate reference)?

View changes since the review

@RalfJung RalfJung Aug 24, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the &raw requirement, anticipating #2338. I don't think we want subobject provenance to get in the way here, so this should be fine (if the static is big enough to read element N):

extern { static mut X: (); }

let ptr = &X as *const () as *const i32;
ptr.add(N).read();

This is a bad idea because of aliasing rules (the pointer gets invalidated on writes), but we already lint against that.

@DianaNites

Copy link
Copy Markdown

This would formally resolve/answer rust-lang/unsafe-code-guidelines#259 and maybe some of rust-lang/unsafe-code-guidelines#546 right?

@RalfJung

Copy link
Copy Markdown
Member Author

The latter has already been resolved by #1657.

And this does resolve the former, yes. Thanks for digging that up.

Comment thread src/behavior-considered-undefined.md Outdated
* Incorrect use of inline assembly. For more details, refer to the [rules] to follow when writing code that uses inline assembly.

r[undefined.extern-static]
* Declaring an `extern static` with some size/alignment/mutability, when the actual symbol this resolves to is smaller / less aligned / less mutable.

@RalfJung RalfJung Aug 24, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new kind of UB is kind of remarkable in that it does not require any code to trigger. But I don't see an alternative...

View changes since the review

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems we sure did declare extern "abi" to be unsafe for a good reason.

@RalfJung
RalfJung force-pushed the extern-static-oversize branch from ad86228 to ec8f4ef Compare August 24, 2026 14:45
Comment thread src/items/external-blocks.md Outdated

r[items.extern.static.size]
The actual memory that the extern static resolves to [must have][extern-static-ub] *at least* the size and alignment of the type that it was declared with in the extern block.
If the actual memory is bigger, then it is permitted to access that extra memory.

@bjorn3 bjorn3 Aug 24, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For dynamic libraries that may be used by a PIE executable, the size given by the symbol must be exact given that the PIE executable will emit a copy relocation that copies a block with exactly the size the symbol had at link time to memory the executable image has reserved for this and redirect all accesses to the static to this copy. This way the executable can avoid GOT indirection, which is a slight perf win. And yes, this means adding elements to a static array in a dylib (or otherwise changing the size) is an ABI breaking change on Linux.

View changes since the review

@RalfJung RalfJung Aug 24, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh... I understand like maybe half of those words. (Can I have some 🥧 please? :D )
"Copying" sounds wrong, statics are places and if you copy them, well, you have two copies so that can't be right?

But it sounds like you are saying linkme and inventory are unsound? IIRC they rely on extern statics that are bigger than declared, filled in by the linker.

@bjorn3 bjorn3 Aug 26, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically when linking the executable, space is reserved for each static referenced from a dylib inside the executable based on symbol size. Then at runtime the dynamic linker will copy the contents of the static in the dylib to the reserved space in the executable and redirect all references to this static to the version in the executable. So there are two copies in memory, but only the one inside the executable is observable by the user. The copy inside the dylib is unused and would be safe to unmap from memory after the dynamic linker has copied it.

linkme and inventory don't work across dylib boundaries anyway as they need the section they emit to be contiguous in memory. They would only see items defined inside the same dylib/executable as the one where the access code happened to be codegened.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

And what does that mean for crates like linkme/inventory?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linkme and inventory only see items defined in the same dylib/executable as the access code is codegened. So if you depend on them to have a global view, you will get misbehavior.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only ELF has copy relocations for PIE executables. The other object file formats don't implement this "optimization".

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see.

Not sure if we should make our UB-rules format-dependent, though.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two cases here, IIUC:

  • Rust is a dylib using the static. Access to the static goes through GOT lookup, independent of whether Rust is the definition site of the static or not. When linking to the dylib, the linker fixups must supply a memory region at least the declared size.
  • Rust is the binary using a static from a dylib. Rust supplies the memory region for the static matching the declared size. At dynamic linking time (before main), the dynamic linker writes the initialization value to the provided memory region for the static.

The memory region of the static always matches the declaration exactly if Rustc is the one allocating it; it's only when Rust is linking to an external memory region that it may potentially be larger. The case where Rust allocates the static and dynamic linking initializes it could be declared morally equivalent to life-before-main w.r.t. the abstract machine, and thus the initialization going out of bounds of the static is an out-of-bounds access unsoundness. Notably, if the Rust code declares a larger static that doesn't require the entire static value to be initialized, it's perfectly fine for the linker to copy initialize a smaller value in that memory region.

In conclusion, I think ELF copy relocation doesn't preclude allowing Rust extern static from linking to a memory region that ends up being larger than declared. But it is another weird case of where the wrong signature in an unsafe extern block can lead to UB even if the symbols go unused by Rust. Which isn't pretty, but it is the reality of today's compiler/linker plumbing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So... do you think the wording in the PR right now is accurate?
Note that this is about extern blocks. If you define a static in a Rust crate, compile that as a dylib, build a binary against that, and then load a different dylib into the binary, then there are all sorts of limitations that we have not even begun to document. That's not what I am trying to do here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe change r[items.extern.attributes.link.kind-raw-dylib.size] to clarify that the size needs to match exactly, but that is effectively already implied when combined with r[items.extern.static.size], so fine either way.

@traviscross traviscross added the I-lang-docs-nominated Nominated for discussion during a lang-docs team meeting. label Aug 25, 2026
@RalfJung
RalfJung force-pushed the extern-static-oversize branch from 9e483d7 to 199eef1 Compare August 26, 2026 17:11
Comment thread src/items/external-blocks.md Resolved
Comment thread src/behavior-considered-undefined.md Outdated
* Incorrect use of inline assembly. For more details, refer to the [rules] to follow when writing code that uses inline assembly.

r[undefined.extern-static]
* Declaring an `extern static` with some size/alignment/mutability, when the actual symbol this resolves to is smaller / less aligned / less mutable.

@joshlf joshlf Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the ground truth with respect to which "less mutable" is evaluated? I understand what "declaring an extern static with some... mutability" means, but what does the mutability of an "actual symbol" refer to?

View changes since the review

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It refers to whether that symbol is in actually mutable memory. My understanding is that in ELF there are flags indicating which sections are mutable and which are not, and I presume other binary formats have something similar.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible alternative wording instead of symbol: the linking-resolved memory region.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went for "The actual memory that the extern static resolves to ", which is also what I used in the other file.

Co-authored-by: Pete LeVasseur <plevasseur@gmail.com>
@RalfJung

RalfJung commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

I guess this does make new commitments, we are officially allowing symbols-bigger-than-the-static-they-are-imported-with. So this should probably be FCP'd. I guess t-opsem is enough?

@CAD97

CAD97 commented Sep 1, 2026

Copy link
Copy Markdown

We probably want to rope the linker-format experts in to the FCP as well at least, given potential limiting factors like ELF copy relocation could exist in other object formats. (Although I don't think ELF copy relocation precludes allowing this.)

Comment thread src/behavior-considered-undefined.md Resolved
@RalfJung

RalfJung commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

I think this is ready to go now... what's the best way to make progress @traviscross ? Some sort of FCP for opsem or opsem+lang?

@traviscross

Copy link
Copy Markdown
Contributor

I think this is ready to go now... what's the best way to make progress @traviscross ? Some sort of FCP for opsem or opsem+lang?

Sounds right. I'll check this over editorially, then propose dual FCP merge.

The new rules talked about "the actual memory" that an extern static
resolves to.  It'd be better to talk about the *allocation*, as the
Reference already does elsewhere.  Let's do that, and let's talk about
allocations being *larger* rather than *bigger*.
Every word in the Reference is important, so we generally don't
emphasize words such as *exactly*, *at least*, and *not*.  This helps
us reserve emphasis for when we're defining a term.  Let's remove the
stray emphasis added in the new rules.
The bytes used for an extern static will often fall within a linker
*section* that starts earlier.  Let's clarify that the *allocation*
starts at the address of the extern static.

r[items.extern.static.size]
The allocation that an extern static resolves to must have at least the size and alignment of the type it is declared with (see [undefined.extern-static]). If the allocation is larger, accessing the extra bytes is allowed.
An extern static resolves to an allocation that starts at the address of the static; the allocation must have at least the size and alignment of the type it is declared with (see [undefined.extern-static]). If the allocation is larger, accessing the extra bytes is allowed.

@RalfJung RalfJung Oct 12, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure whether it really has to start there. Having extra bytes to the left seems fine? (As long as the bytes are still only in one allocation, of course.)

Cc @nikic does LLVM have any trouble with statics having dereferenceable bytes before their start address?

View changes since the review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cc @nikic does LLVM have any trouble with statics having dereferenceable bytes before their start address?

Yes: https://github.com/llvm/llvm-project/blob/7159ce54c436f91a9437eced53800b8fcb4c34bc/llvm/lib/Analysis/AliasAnalysis.cpp#L927-L933

Global variables must point to the start of the allocation.

@traviscross traviscross added the I-lang-docs-nominated Nominated for discussion during a lang-docs team meeting. label Oct 13, 2026
Comment thread src/items/external-blocks.md Outdated
Comment on lines +83 to +87
r[items.extern.static.allocations-disjoint]
The allocation that an extern static resolves to does not include the bytes of any other static, even when the linker places the statics in the same section.

> [!NOTE]
> Pointer arithmetic from one static cannot reach another. In particular, an extern static that marks the start of a linker section cannot be used to reach other statics placed in that section.

@RalfJung RalfJung Oct 14, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If multiple extern statics have the same link_name, then they are all valid aliases of each other AFAIK.

I don't know if there is a way to "remap" symbol names and legally alias two statics with different names?

View changes since the review

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use a linker script to assign the address of one symbol as address of another symbol. Or just assign them both the same fixed address.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that will be UB as LLVM assumes statics do not alias, but let's check with @nikic to confirm.

@traviscross traviscross Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If multiple extern statics have the same link_name, then they are all valid aliases of each other AFAIK.

Thanks. I've clarified this in the rules.

You can use a linker script to assign the address of one symbol as address of another symbol. Or just assign them both the same fixed address.

It's not clear that the behavior of that is defined.1

use core::ffi::c_char;
use core::hint::{assert_unchecked, black_box};

unsafe extern "C" {
    // Glibc makes one of these a weak alias for the other.
    static environ: *const *const c_char;
    static __environ: *const *const c_char;
}

fn main() {
    let (a, b) = (&raw const environ, &raw const __environ);
    assert!(a != b);
    assert!(black_box(a) == black_box(b));
    if black_box(a) == black_box(b) {
        unsafe { assert_unchecked(a == b) }; //~ UB.
    }
}

Playground link

Footnotes

  1. Edit: I hadn't refreshed and so hadn't seen @RalfJung's concurrent comment above. ↩

Comment thread src/items/external-blocks.md Outdated
The allocation that an extern static resolves to does not include the bytes of any other static, even when the linker places the statics in the same section.

> [!NOTE]
> Pointer arithmetic from one static cannot reach another. In particular, an extern static that marks the start of a linker section cannot be used to reach other statics placed in that section.

@joshtriplett joshtriplett Oct 14, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In particular, an extern static that marks the start of a linker section cannot be used to reach other statics placed in that section.

Does this include walking the entire contents of the section, in the case of sections that have well-defined contents? For instance, one common stunt with linker sections is to have a section that accumulates pointers to things, and walk the section (from a pointer to the beginning to a pointer to the end), looking at each pointer-sized value within. Logically, this isn't accessing "the bytes of another static", it's accessing the contents of the whole section. How can we do that in Rust? Use raw pointers to the start/end of the section?

View changes since the review

@joshtriplett joshtriplett Oct 14, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this particular restriction coming from? People do this in C all the time, and there's presumably a way to convince LLVM/clang to accept that.

How can we craft this language such that there is a correct way to do this in Rust, interoperating with the same mechanisms used in C, using #[link_section] and not having to use inline assembly?

Note in particular that the exact ordering of the section is not a guarantee. The usual reason for not allowing walks off the end of one object into another is that we don't actually guarantee that they're placed adjacent to each other. But if unsafe code ensures that everything in the section is the same type, or is otherwise walkable in sequence (e.g. every record has a length at the beginning), that should be sufficient for code to walk it from beginning to end. (A combination of linker scripts and section naming can also affect ordering, such as by merging numbered sections in numeric order.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, I guess you can always use the wrapping_* functions to compare the pointers. You just can't use add() or sub() which require both pointers to be in the same allocation.

@RalfJung RalfJung Oct 15, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this include walking the entire contents of the section, in the case of sections that have well-defined contents? For instance, one common stunt with linker sections is to have a section that accumulates pointers to things, and walk the section (from a pointer to the beginning to a pointer to the end), looking at each pointer-sized value within. Logically, this isn't accessing "the bytes of another static", it's accessing the contents of the whole section. How can we do that in Rust? Use raw pointers to the start/end of the section?

That's completely fine as long as those bytes are not part of any other static.

What's not okay is have two statics alias each other, where you have

extern "C" {
  static mut A: [u8; 16];
  static mut B: [u8; 16];
}

and then you access the same byte (i.e., the same absolute address) via both of them.

What is completely okay is to have

extern "C" {
  static mut A: ();
  static mut B: ();
}

and then say that at A an allocation starts that reaches all the way to B, and at B a zero-sized allocation starts. That's my model for this "beginning and end marker" pattern. There just can't be any other static in between these two then, and pointers derived from B must not be used to access memory (or to do inbounds pointer arithmetic).

Where is this particular restriction coming from?

IIUC, LLVM alias analysis assumes that different statics do not alias.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People do this in C all the time

I do wonder if it's actually defined in C either, or "happens to work", since provenance is a thing there too, even if the compiler might not in practice -- especially on weak vendor compilers -- take advantage of it.


I do think there's probably a way to do it even if the allocations are different so long as you pick up the correct provenance for the pointers. I could imagine you iterating by integer address, then using from_exposed_provenance for each item you're trying to read to ensure you get the correct provenance on the pointer -- assuming either that linker script addresses are deemed exposed or that you otherwise have exposed those pointers in whatever created those allocations.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RalfJung To be explicit, the case I'm thinking of would have two pointers (not Rust allocations, more like raw pointers) provided as symbols by the linker and imported via Rust as extern, and then walked as raw pointers via unsafe code. And then separately you have constant (non-mut) statics that define a non-mutable frozen data structure (e.g. a pointer to an init function) that's being read, not written by any part of the program (it is defined with a fixed value and never changed).

What is the mechanism to define and implement that pattern without using any inline assembly?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're discussing this on Zulip: #t-opsem > linker section slice iteration

That some extern statics may be used to access more bytes than would
be implied by the size of their types does not imply that they may be
used to access the bytes of other statics (with different symbols),
even if those bytes are in the same linker section.  Let's make this
explicit.

And since statics with the same symbol do resolve to the same
allocation, let's add a rule making that explicit too.
@traviscross
traviscross force-pushed the extern-static-oversize branch from 6818de5 to d3a2589 Compare October 15, 2026 03:29
@tmandry

tmandry commented Oct 21, 2026

Copy link
Copy Markdown
Member

@rfcbot reviewed

2 similar comments
@CAD97

CAD97 commented Oct 21, 2026

Copy link
Copy Markdown

@rfcbot reviewed

@saethlin

Copy link
Copy Markdown
Member

@rfcbot reviewed

@joshtriplett

Copy link
Copy Markdown
Member

@rfcbot concern document-pattern-for-section-iteration

(Filing this about the ongoing conversation iterating on how to correctly do section iteration, without requiring any form of asm. I expect to resolve it and check a box the moment we have a documented incantation for that.)

@RalfJung

Copy link
Copy Markdown
Member Author

@digama0 @joshlf friendly opsem checkbox reminder :)

@joshtriplett

Copy link
Copy Markdown
Member

Ongoing conversation in https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/linker.20section.20slice.20iteration , and the general trend seems to be "yeah, this should work", with some thought going in to making sure that's the case.

>
> In particular, an extern static that marks the start of a linker section cannot be used to access statics with different symbols placed in that section.

```rust,standalone_crate

@joshlf joshlf Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example is pretty subtle. It'd be good to, before this example, precisely document the conditions under which a static can be used to access bytes. Currently the only prose that exists says when it can't be:

In particular, an extern static that marks the start of a linker section cannot be used to access statics with different symbols placed in that section.

View changes since the review

Comment on lines +146 to +177
```rust,standalone_crate
// A static placed in a section is an allocation of its own, which the
// section's start symbol cannot be used to read.

// A static placed in a section named `entries`.
# #[cfg(target_os = "linux")]
#[used]
#[unsafe(link_section = "entries")]
static ENTRY: [u8; 3] = [1, 2, 3];

# #[cfg(target_os = "linux")]
unsafe extern "C" {
// On ELF targets, the linker defines these symbols at the start
// and the end of the `entries` section.
static __start_entries: [u8; 0];
static __stop_entries: [u8; 0];
}

fn main() {
# #[cfg(target_os = "linux")] {
let start = &raw const __start_entries as *const u8;
let stop = &raw const __stop_entries as *const u8;
// The section holds exactly the bytes of `ENTRY`.
assert_eq!(stop.addr() - start.addr(), 3);
assert_eq!(start.addr(), (&raw const ENTRY).addr());
// But those bytes are the allocation of `ENTRY`, not of
// `__start_entries`, so reading them through `start` is undefined
// behavior.
//let x = unsafe { *start }; // UB.
# }
}
```

@RalfJung RalfJung Oct 21, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly the pattern @joshtriplett asked about, right?

View changes since the review

@joshtriplett joshtriplett Oct 23, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I'm hoping the conversation on Zulip turns up a pattern we can document as successfully allowing this. Not least of which because it won't use mut, which should make Rust okay with it, so we just need to make sure LLVM is too.

@joshtriplett

Copy link
Copy Markdown
Member

We talked about this in today's lang meeting. I'm hoping that this is a relatively straightforward thing to confirm with LLVM. If it is indeed straightforward, I'm hoping we can document it rather than explicitly restricting it (rather than before, when we implicitly restricted it). If it turns out to not be straightforward, then given that people already do it anyway in C, and need to be able to interoperate with that pattern from Rust, then I can drop the concern and we can proceed with what we have. That said, it'd be helpful to have a non-normative note saying essentially "we know this pattern exists, this currently makes it UB but we're trying to document an interoperable way to interface with it that isn't UB", so that this doesn't get lost. (For calibration on importance, interoperating with this pattern will be essential for writing some types of Linux kernel code.)

@RalfJung

RalfJung commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

I'm hoping that this is a relatively straightforward thing to confirm with LLVM.

I hope so too. I'm waiting for confirmation from @nikic.

@traviscross

traviscross commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

I'm hoping that this is a relatively straightforward thing to confirm with LLVM.

I hope so too. I'm waiting for confirmation from @nikic.

I wonder about other backends. E.g., GCC closed a similar issue, involving Linux kernel code, as INVALID:1

And given gcc 7 is to be released yet, can we have a switch to disable this optimization?

This is nothing new in GCC 7, you've most likely just been extremely lucky in the past that it happened to work as you expected. Other projects had to change similar UB code years ago. It isn't just a single optimization, but lots of them that rely on pointer arithmetics being defined only within the same object.

...

In C

extern struct builtin_fw __start_builtin_fw[];
extern struct builtin_fw __end_builtin_fw[];

declares two external arrays, thus they are two independent objects. It is like if you have:

int a[10];
int b[10];

in your program, although they might be allocated adjacent, such that

int *p = &a[10]; int *q = &b[0]; memcmp (&p, &q, sizeof (p)) == 0;
&b[0] - &a[0]

is still UB. What you do with __start_*/__end_* symbols is nothing you can define in C, you need linker support or asm for that, and to use it without UB you also need to use an optimization barrier that has been suggested.

The kernel side discussion was here. Linus wrote:

This is the best I could come up with: assuming gcc is not allowed to reason about what's inside the asm(), this is the only way I could think of to lose the array information without incurring unnecessary overheads. It should also be relatively safe as there is no way to accidentally use the underlying arrays without explicitly declaring them.

Ugh. I worry about the other places where we do things like this, depending on the linker just assigning the addresses and us being able to compare them.

If there is a compiler option to disable this optimization, I would almost prefer that.. Because we really do have a whole slew of these things.

And:

You could define a wrapper if you like, something like

/*
 * Assign dst = src, but prevent the compiler from inferring anything
 * about the assigned value, so it can't do any unwanted optimization.
 */
#define blind_assign(dst,src) asm("" : "=X" (dst) : "0" (src))

We already have RELOC_HIDE() and OPTIMIZER_HIDE_VAR() that basically do this.

They both use r. I guess they could use X.

Nor does it seem that GCC's position has shifted here.2 When the matter was raised on 2026-08-13, a GCC maintainer responded:

No this is NOT a false positive.

And:

The subtraction is well-defined in practice

NO It is not. See PR 77964 for why it shows up as being invalid thing to do.

GCC has an issue for documenting the correct way to iterate a "section array"; the proposal is to document the pattern of using inline assembly as an optimization barrier, as they suggested to the kernel developers.

cc @rust-lang/wg-gcc-backend

Footnotes

  1. Note that the example of defined behavior in this PR is written carefully, and does not use offset_from, to avoid this same issue. ↩

  2. Though that particular fold does not happen, currently, for reasons that are unrelated to section walking and are similar to the environ case I had mentioned. ↩

@traviscross

Copy link
Copy Markdown
Contributor

cc rust-lang/rust#125119

    /// Allows registering static items globally, possibly across crates, to iterate over at runtime.
    (unstable, global_registration, "CURRENT_RUSTC_VERSION", Some(125119)),

@RalfJung

RalfJung commented Nov 5, 2026 •

Copy link
Copy Markdown
Member Author

IMO it is the responsibility of GCC to properly model Rust's semantics, not the other way around. This is assuming that the semantics work fine in LLVM, which demonstrates that the semantics can reasonably be implemented.

@traviscross

Copy link
Copy Markdown
Contributor

IMO it is the responsibility of GCC to properly model Rust's semantics, not the other way around.

I don't disagree.

@RalfJung

RalfJung commented Nov 5, 2026 •

Copy link
Copy Markdown
Member Author

Looking at the issue, it seems like GCC folks claim that comparing unrelated pointers, such as pointers to two different statics, is UB in C. In think they are wrong even in C, but they are definitely wrong in Rust. This isn't just about statics, or overlapping statics. Pointer comparison is always well-defined in Rust and the GCC backend will have to find a way to deal with this.

(Subtracting unrelated pointers is indeed UB, and on the Rust side that would be offset_from which is quite clear about this.)

@Jules-Bertholet

Jules-Bertholet commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

comparing unrelated pointers, such as pointers to two different statics, is UB in C.

IIUC, == and != between such pointers is not UB, but </<=/>/>= are. https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#subsection.6.5.9

@RalfJung

RalfJung commented Nov 5, 2026 via email

Copy link
Copy Markdown
Member Author

Sometimes it's useful for multiple crates (or external compilation
units) to be able to each add entries at link time to a single table
that can be iterated over at runtime.  Let's show how to soundly
encode this pattern using the guarantees that we're adding.
@traviscross

Copy link
Copy Markdown
Contributor

The guarantees we're making here support writing the decentralized registration pattern. I've pushed a guide-like example section that demonstrates how to soundly encode this, including the optimized form, with place-relative offsets, that's used by the Linux kernel. This more-optimized form saves up to 28 bytes per table entry and allows the pages backing the table to be marked read-only, which lets them be shared across processes.

Please have a look.

My feeling is that even if we were to make the additional guarantee that we're discussing, people would be better off not using it, since there's no way to use statics with unsafe(link_section = …) to write this better encoding. The assembler directives needed for this optimized form are shared across architectures, so there's no loss of portability on ELF targets. Regardless of how we write this, it relies on (documented) linker behavior, not something the language provides.

@RalfJung

RalfJung commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

saves up to 28 bytes per table entry

A table entry is usually ptr-sized, i.e., 4/8 bytes. How can one save 28 bytes over that...?

@traviscross

Copy link
Copy Markdown
Contributor

saves up to 28 bytes per table entry

A table entry is usually ptr-sized, i.e., 4/8 bytes. How can one save 28 bytes over that...?

See the text in the commit. The relocation record is up to 24 bytes.

@RalfJung

RalfJung commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

Ah so this is about on-disk binary size, not runtime RAM usage?

TBH I feel like that example has way too much detail for a section of the Rust reference.^^ Going into so much detail distracts from the main content. But that's a separate question.

@traviscross

traviscross commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Ah so this is about on-disk binary size, not runtime RAM usage?

There are three aspects of this form that save RAM:

  • On 64-bit targets, there's a 2x savings from using 4-byte entries rather than pointer-sized entries.
  • Because there are no writes to the table at load time, the pages stay file-backed and can be shared between processes; with pointer entries, the relocations make the pages of the table private to each process. This is particularly relevant for shared libraries.
  • (Minor:) The on-disk representation (where we save up to 24 bytes) will sit in the page cache.

TBH I feel like that example has way too much detail for a section of the Rust reference.^^ Going into so much detail distracts from the main content. But that's a separate question.

Yes, there's some tension there. I have ideas about how to better organize or present this long term. I do think this sort of material adds value to the Reference.

(Also, even today, looking at the rendered form rather than the Markdown may affect one's sense of this: it's set off visually from the main body of the section in a kind of admonition designed for these asides.)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

disposition-merge I-lang-docs-nominated Nominated for discussion during a lang-docs team meeting. I-lang-nominated Nominated for discussion during a lang team meeting. I-lang-radar Items that are on lang's radar and will need eventual work or consideration. P-lang-drag-1 Lang team prioritization drag level 1. proposed-final-comment-period S-waiting-on-review Status: The marked PR is awaiting review from a maintainer T-lang Relevant to the language team. T-opsem Team: opsem

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How to explain linker symbols used as integers (and not pointers to an allocation)? Are statics confined to the size indicated by their type?