authorgravatar for mlugg@mlugg.co.ukMatthew Lugg <mlugg@mlugg.co.uk> 2026-08-12 12:24:45+01:00
committergravatar for mlugg@mlugg.co.ukMatthew Lugg <mlugg@mlugg.co.uk> 2026-08-16 10:19:44+01:00
logc7fed8d0c6efa1ac0c72566cd397acf1210203e3
tree8a9a1147548e880692c68e56b859b5004a048b85
parentc7a169280d51359f8206e86cf0f6289e902d0c2e
signaturelock-open Commit is signed but in an unrecognized format.

Elf2: implement `.hash`

This section, of type `SHT_HASH`, is a hash table used for looking up symbol names in DSOs. It is referenced in `.dynamic` by the `DT_HASH` entry. Without this, glibc assumes we export no symbols, and musl refuses to load our DSOs at all! Modern ELF systems broadly consider `SHT_HASH`/`DT_HASH` deprecated in favour of `SHT_GNU_HASH`/`DT_GNU_HASH`, another hash table with a different format which is more efficient for misses. However, as with so many things in ELF, GNU's hash table format seems to be carefully engineered to make incremental linking as difficult as possible. Every dynamic linker out there continues to support `DT_HASH`, which actually plays pretty nicely with incremental compilation, so I think we'll stick with `DT_HASH` when doing incremental links. I would now like to take a moment to complain about these hash tables. These tables exist in order to make symbol lookups efficient when applying relocations. The ldso implementations for both glibc and musl libc work by iterating all relocations, and, for each one, looking up the target symbol in each candidate DSO based on their `DT_HASH` or `DT_GNU_HASH` table. The `DT_GNU_HASH` format came about to speed up this lookup, since it's happening very often, so a small speedup can lead to major improvements in load time. However, I don't understand why nobody first solved the really obvious inefficiency: every single symbol lookup is doing N different hash table lookups where N is the number of loaded DSOs! A clearly better approach for a dynamic linker to take is to have a single global hash table, and every time a DSO is loaded, to add its symbols to that table. Then each symbol lookup requires only *one* hash map lookup instead of N! Plus, this change doesn't affect ELF files at all---it could be made in libc implementations tomorrow. Yes, you need to iterate each DSO's dynamic symbol table once in full, but that shouldn't be a big deal when you're already iterating their *relocations* (those are far more numerous than dynamic symbols!). Here's another idea for free---after building the hash map, iterate everyone's dynamic symbol table a second time, and build a lookup table from their dynamic symbol index to the resolved symbol in the hash map. Now relocation application doesn't even need a hash map; just a lookup table access! Maybe this one isn't worthwhile, since the difference between one LUT access and one hash map lookup isn't enough to warrant the extra allocation and iteration, but it seems worth a try... computers are fast, y'know! What drives me a little crazy is that when someone ran into performance problems with relocation application, instead of just spending an hour speeding up lookups in ldso, they chose to invent lazy PLT binding. In doing so, they opted to inflict needless complexity, unpredictable runtime performance characteristics, unreportable failures, and security risks in the form of a mutable jump table, upon every ELF system, for ever. But I guess at least no poor soul had to implement a hash map. But fine, let's look at status quo, and compare the standard `SHT_HASH` format with `SHT_GNU_HASH`. As we all know, GNU's track record terms of high-quality contributions to the ELF ecosystem is flawless, so there must have been some good reasons for the changes. Some small things which seem fine: they changed the hash function, which I'll assume was with fair reason, and they added a bloom filter to allow lookups to fail early (important if you accept the design of looking up symbol names in individual DSOs). The next change was to add a header field called `symoffset`, so that the hash table can avoid wasting space on chains for the first N symbol table entries which aren't actually global symbols. Unfortunately, it seems that someone missed a memo, because this field, in a format which debuted in 2006, has been redundant since the mid-90s. ELF files pretty much universally use a separate `SHT_DYNSYM` section for the dynamic symbol table, with the express purpose of *omitting* all the `STB_LOCAL` symbols to save space. If a symbol doesn't appear in the hash table, it shouldn't be in the dynamic symbol table to begin with. This quirk of the `SHT_GNU_HASH` section isn't particularly *offensive*, it's just... not useful. Then the actual hash table representation. The "chains" array no longer forms a linked list; instead, `chains[sym_idx]` now holds the *hash* for the symbol's name. The linked list is eliminated in favour of an assumption that symbols in the same bucket are contiguous in the symbol table, and the least-significant bit of `chains[sym_idx]` is repurposed to indicate the last symbol in a bucket (so you know when to stop iterating). In general I'm all for replacing linked lists with arrays, but this particular change is really annoying. Setting aside the fact that it's kinda odd for metadata *about* the symbol table to mandate a specific ordering *within* the symbol table, this rigid ordering requirement is also terrible for incremental linkers! It means that adding or removing a symbol requires shifting potentially the entire symbol table up or down to make sure the new symbol is in the right place, unless you're happy having gaps all over the table (which I've been trying to avoid in `Elf2`). I'm sure the contiguous-symbols assumption and the caching of the hashes does improve lookup performance, but it seems like a far more complicated solution than just doing some kind of caching in ldso implementations. How this became the accepted solution to slow symbol lookups is truly beyond me.

1 files changed, 189 insertions(+), 6 deletions(-)

src/link/Elf2.zig+189-6
......@@ -36,6 +36,7 @@ shndx: struct {
3636 dynsym: Section.Index,
3737 dynstr: Section.Index,
3838 dynamic: Section.Index,
39 hash: Section.Index,
3940 tdata: Section.Index,
4041 rela_dyn: Section.Index,
4142 rela_plt: Section.Index,
......@@ -1751,6 +1752,150 @@ const SymbolReloc = struct {
17511752 }
17521753};
17531754
1755fn ensureDynsymHashCapacity(elf: *Elf, max_dynsym_count: u32) Error!void {
1756 const min_buckets = max_dynsym_count / 2;
1757
1758 const cur_dynsym_count: u32 = switch (elf.shdrPtr(elf.shndx.dynsym)) {
1759 inline else => |shdr, class| @intCast(@divExact(
1760 elf.targetLoad(&shdr.size),
1761 @sizeOf(class.ElfN().Sym),
1762 )),
1763 };
1764
1765 {
1766 const section_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
1767 const header: *std.elf.hash.Header = @ptrCast(section_slice[0..@sizeOf(std.elf.hash.Header)]);
1768 assert(elf.targetLoad(&header.nchain) == cur_dynsym_count);
1769 const nbucket = elf.targetLoad(&header.nbucket);
1770 if (nbucket >= min_buckets) {
1771 // We don't need to add any buckets, but we still need to make sure the section is large
1772 // enough to fit `max_dynsym_count` chains.
1773 const need_size = @sizeOf(std.elf.hash.Header) + (nbucket + max_dynsym_count) * 4;
1774 try elf.ensureNodeSize(elf.shndx.hash.get(elf).ni, need_size);
1775 return;
1776 }
1777 // We need more buckets, so we'll have to rebuild the hash table.
1778 }
1779
1780 // Rebuilding the hash table is quite expensive, so to avoid doing it too often we use a large
1781 // growth factor (* 2) for `nbucket`.
1782 const new_nbucket = min_buckets * 2;
1783
1784 {
1785 const need_size = @sizeOf(std.elf.hash.Header) + (new_nbucket + max_dynsym_count) * 4;
1786 try elf.ensureNodeSize(elf.shndx.hash.get(elf).ni, need_size);
1787 }
1788
1789 elf.mf.nodes_lock.lock();
1790 defer elf.mf.nodes_lock.unlock();
1791
1792 const section_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
1793 const header: *std.elf.hash.Header = @ptrCast(section_slice[0..@sizeOf(std.elf.hash.Header)]);
1794 const trailing: []u32 = @ptrCast(section_slice[@sizeOf(std.elf.hash.Header)..]);
1795
1796 header.* = .{ .nbucket = new_nbucket, .nchain = cur_dynsym_count };
1797 if (elf.targetEndian() != std.lang.Endian.native) {
1798 std.mem.byteSwapAllFields(std.elf.hash.Header, header);
1799 }
1800 const buckets: []u32 = trailing[0..elf.targetLoad(&header.nbucket)];
1801 const chains: []u32 = trailing[elf.targetLoad(&header.nbucket)..][0..elf.targetLoad(&header.nchain)];
1802
1803 @memset(buckets, 0);
1804 chains[0] = 0;
1805 for (1..cur_dynsym_count, chains[1..]) |dynsym_index_usize, *chain| {
1806 const dynsym_index: u32 = @intCast(dynsym_index_usize);
1807 const sym_name: String(.dynstr) = switch (elf.dynsymPtr(dynsym_index)) {
1808 inline else => |sym| @fromBackingInt(elf.targetLoad(&sym.name)),
1809 };
1810 const b = std.elf.hash.calculate(sym_name.slice(elf)) % buckets.len;
1811 // Make this symbol the head of that bucket, and chain to the old head.
1812 chain.* = buckets[b];
1813 elf.targetStore(&buckets[b], dynsym_index);
1814 }
1815}
1816
1817fn appendDynsymHashEntry(elf: *Elf, dynsym_index: u32) void {
1818 const section_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
1819 const header: *std.elf.hash.Header = @ptrCast(section_slice[0..@sizeOf(std.elf.hash.Header)]);
1820 assert(elf.targetLoad(&header.nchain) == dynsym_index);
1821 elf.targetStore(&header.nchain, dynsym_index + 1);
1822
1823 switch (elf.shdrPtr(elf.shndx.hash)) {
1824 inline else => |shdr| elf.targetStore(&shdr.size, elf.targetLoad(&shdr.size) + 4),
1825 }
1826
1827 elf.populateDynsymHashEntry(dynsym_index);
1828}
1829fn populateDynsymHashEntry(elf: *Elf, dynsym_index: u32) void {
1830 elf.mf.nodes_lock.lock();
1831 defer elf.mf.nodes_lock.unlock();
1832
1833 assert(dynsym_index != 0);
1834
1835 const section_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
1836 const header: *std.elf.hash.Header = @ptrCast(section_slice[0..@sizeOf(std.elf.hash.Header)]);
1837 const trailing: []u32 = @ptrCast(section_slice[@sizeOf(std.elf.hash.Header)..]);
1838
1839 const buckets: []u32 = trailing[0..elf.targetLoad(&header.nbucket)];
1840 const chains: []u32 = trailing[elf.targetLoad(&header.nbucket)..][0..elf.targetLoad(&header.nchain)];
1841
1842 const sym_name: String(.dynstr) = switch (elf.dynsymPtr(dynsym_index)) {
1843 inline else => |sym| @fromBackingInt(elf.targetLoad(&sym.name)),
1844 };
1845 const b = std.elf.hash.calculate(sym_name.slice(elf)) % buckets.len;
1846 // Make this symbol the head of that bucket, and chain to the old head.
1847 chains[dynsym_index] = buckets[b];
1848 elf.targetStore(&buckets[b], dynsym_index);
1849}
1850fn popDynsymHashEntry(elf: *Elf, dynsym_index: u32) void {
1851 elf.clearDynsymHashEntry(dynsym_index);
1852
1853 const section_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
1854 const header: *std.elf.hash.Header = @ptrCast(section_slice[0..@sizeOf(std.elf.hash.Header)]);
1855 assert(elf.targetLoad(&header.nchain) == dynsym_index + 1);
1856 elf.targetStore(&header.nchain, dynsym_index);
1857
1858 switch (elf.shdrPtr(elf.shndx.hash)) {
1859 inline else => |shdr| elf.targetStore(&shdr.size, elf.targetLoad(&shdr.size) - 4),
1860 }
1861}
1862fn clearDynsymHashEntry(elf: *Elf, dynsym_index: u32) void {
1863 elf.mf.nodes_lock.lock();
1864 defer elf.mf.nodes_lock.unlock();
1865
1866 assert(dynsym_index != 0);
1867
1868 const section_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
1869 const header: *std.elf.hash.Header = @ptrCast(section_slice[0..@sizeOf(std.elf.hash.Header)]);
1870 const trailing: []u32 = @ptrCast(section_slice[@sizeOf(std.elf.hash.Header)..]);
1871
1872 const buckets: []u32 = trailing[0..elf.targetLoad(&header.nbucket)];
1873 const chains: []u32 = trailing[elf.targetLoad(&header.nbucket)..][0..elf.targetLoad(&header.nchain)];
1874
1875 const sym_name: String(.dynstr) = switch (elf.dynsymPtr(dynsym_index)) {
1876 inline else => |sym| @fromBackingInt(elf.targetLoad(&sym.name)),
1877 };
1878 const b = std.elf.hash.calculate(sym_name.slice(elf)) % buckets.len;
1879
1880 const next_dynsym_index = elf.targetLoad(&chains[dynsym_index]);
1881 elf.targetStore(&chains[dynsym_index], 0);
1882
1883 // To remove `dynsym_index` from the singly-linked list, we need to iterate the chain to find
1884 // and replace it. But since this is, well, a hash table, that's actually fine.
1885 if (elf.targetLoad(&buckets[b]) == dynsym_index) {
1886 elf.targetStore(&buckets[b], next_dynsym_index);
1887 } else {
1888 var cur = elf.targetLoad(&buckets[b]);
1889 while (true) {
1890 assert(cur != 0); // `dynsym_index` is definitely somewhere in the chain
1891 if (elf.targetLoad(&chains[cur]) == dynsym_index) break;
1892 cur = elf.targetLoad(&chains[cur]);
1893 }
1894 // We found `dynsym_index`; replace it with `next_dynsym_index`.
1895 elf.targetStore(&chains[cur], next_dynsym_index);
1896 }
1897}
1898
17541899fn ensureUnusedSymbolCapacity(elf: *Elf, len: u32, kind: enum { all_local, maybe_global }) Error!void {
17551900 const gpa = elf.base.comp.gpa;
17561901
......@@ -1780,12 +1925,19 @@ fn ensureUnusedSymbolCapacity(elf: *Elf, len: u32, kind: enum { all_local, maybe
17801925 try elf.node_global_symbols.ensureUnusedCapacity(gpa, len);
17811926
17821927 if (elf.shndx.dynsym != .UNDEF) {
1783 // Ensure the `.dynsym` section's node is big enough
1784 const dynsym_need_size: u64 = switch (elf.shdrPtr(elf.shndx.dynsym)) {
1785 inline else => |shdr, class| elf.targetLoad(&shdr.size) + len * @sizeOf(class.ElfN().Sym),
1928 const dynsym_cur_size: u64, const dynsym_ent_size: u32 = switch (elf.shdrPtr(elf.shndx.dynsym)) {
1929 inline else => |shdr, class| .{
1930 elf.targetLoad(&shdr.size),
1931 @sizeOf(class.ElfN().Sym),
1932 },
17861933 };
1934 const dynsym_cur_len: u32 = @intCast(@divExact(dynsym_cur_size, dynsym_ent_size));
1935
1936 const dynsym_need_size: u64 = (dynsym_cur_len + len) * dynsym_ent_size;
17871937 try elf.ensureNodeSize(elf.shndx.dynsym.get(elf).ni, dynsym_need_size);
17881938
1939 try elf.ensureDynsymHashCapacity(dynsym_cur_len + len);
1940
17891941 try elf.ensureUnusedPltCapacity(len);
17901942 }
17911943 },
......@@ -2122,6 +2274,7 @@ fn addGlobalSymbolAssumeCapacity(elf: *Elf, opts: AddGlobalSymbolOptions) error{
21222274 if (elf.targetEndian() != native_endian) {
21232275 std.mem.byteSwapAllFields(Sym, sym);
21242276 }
2277 elf.appendDynsymHashEntry(dynsym_index);
21252278 break :dynsym_index dynsym_index;
21262279 },
21272280 }
......@@ -2396,6 +2549,8 @@ fn moveDemotedGlobal(elf: *Elf, global_ptr: *Symbol.Global) void {
23962549 const new_size = old_size - ent_size;
23972550 const remove_dynsym_index: u32 = @intCast(@divExact(new_size, ent_size));
23982551
2552 elf.popDynsymHashEntry(remove_dynsym_index);
2553
23992554 const free_dynsym_index = global_ptr.dynsym_index;
24002555 global_ptr.dynsym_index = 0;
24012556
......@@ -2403,6 +2558,8 @@ fn moveDemotedGlobal(elf: *Elf, global_ptr: *Symbol.Global) void {
24032558 // The demoted global wasn't the last entry, so move whatever entry we just
24042559 // truncated out of dynsym into its place.
24052560
2561 elf.clearDynsymHashEntry(free_dynsym_index);
2562
24062563 const src_dynsym_ptr = @field(elf.dynsymPtr(remove_dynsym_index), @tagName(class));
24072564 const dest_dynsym_ptr = @field(elf.dynsymPtr(free_dynsym_index), @tagName(class));
24082565
......@@ -2415,6 +2572,8 @@ fn moveDemotedGlobal(elf: *Elf, global_ptr: *Symbol.Global) void {
24152572 assert(moved_global_ptr.dynsym_index == remove_dynsym_index);
24162573 moved_global_ptr.dynsym_index = free_dynsym_index;
24172574
2575 elf.populateDynsymHashEntry(free_dynsym_index);
2576
24182577 // Since that symbol's dynsym index has changed, we'll have to update any
24192578 // relocation entries targeting it.
24202579 elf.changed_symtab_index.putAssumeCapacity(moved_name, {});
......@@ -3161,6 +3320,7 @@ fn create(
31613320 .dynsym = .UNDEF,
31623321 .dynstr = .UNDEF,
31633322 .dynamic = .UNDEF,
3323 .hash = .UNDEF,
31643324 .tdata = .UNDEF,
31653325 .rela_dyn = .UNDEF,
31663326 .rela_plt = .UNDEF,
......@@ -3298,6 +3458,7 @@ fn initHeaders(
32983458 shnum += 1; // .dynamic
32993459 shnum += 1; // .dynstr
33003460 shnum += 1; // .dynsym
3461 shnum += 1; // .hash
33013462 shnum += 1; // .rela.dyn
33023463 shnum += 1; // .rela.plt
33033464 }
......@@ -3926,6 +4087,26 @@ fn initHeaders(
39264087 .entsize = @intCast(addr_align.toByteUnits() * 2),
39274088 .node_align = addr_align,
39284089 });
4090 elf.shndx.hash = try elf.addSection(elf.ni.rodata, .{
4091 .name = ".hash",
4092 .type = .HASH,
4093 .flags = .{ .ALLOC = true },
4094 .link = elf.shndx.dynsym.toSection().?,
4095 .addralign = .@"4",
4096 // initially: nbucket = 8, nchain = 1
4097 .size = @sizeOf(std.elf.hash.Header) + (8 + 1) * 4,
4098 });
4099 {
4100 const hash_slice: []align(4) u8 = @alignCast(elf.shndx.hash.get(elf).ni.slice(&elf.mf));
4101 const header: *std.elf.hash.Header = @ptrCast(hash_slice[0..@sizeOf(std.elf.hash.Header)]);
4102 header.* = .{ .nbucket = 8, .nchain = 1 };
4103 if (elf.targetEndian() != std.lang.Endian.native) {
4104 std.mem.byteSwapAllFields(std.elf.hash.Header, header);
4105 }
4106 // The initial bucket and chain values are all 0, but `MappedFile` initialized the
4107 // node with zeroes anyway, so no need to memset.
4108 }
4109
39294110 switch (machine) {
39304111 .AARCH64, .PPC64, .RISCV => @panic(@tagName(machine)),
39314112 .X86_64 => {
......@@ -5931,7 +6112,7 @@ fn prepareDynamic(elf: *Elf) Error!void {
59316112 @as(usize, @intFromBool(elf.shndx.preinit_array != .UNDEF)) * 2 +
59326113 @as(usize, @intFromBool(use_plt)) * 4 +
59336114 @intFromBool(comp.config.output_mode == .Exe) +
5934 @intFromBool(elf.textrel_count > 0) + 8;
6115 @intFromBool(elf.textrel_count > 0) + 9;
59356116
59366117 const dynamic_size = dynamic_len * 2 * elf.targetPtrSize();
59376118
......@@ -6031,7 +6212,7 @@ fn flushDynamic(elf: *Elf) void {
60316212 dynamic_index += 4;
60326213 }
60336214
6034 dynamic_entries[dynamic_index..][0..8].* = .{
6215 dynamic_entries[dynamic_index..][0..9].* = .{
60356216 .{ std.elf.DT_RELA, @intCast(elf.shndx.rela_dyn.vaddr(elf)) },
60366217 .{ std.elf.DT_RELASZ, @intCast(elf.shndx.rela_dyn.size(elf)) },
60376218 .{ std.elf.DT_RELAENT, @sizeOf(ElfN.Rela) },
......@@ -6039,9 +6220,10 @@ fn flushDynamic(elf: *Elf) void {
60396220 .{ std.elf.DT_SYMENT, @sizeOf(ElfN.Sym) },
60406221 .{ std.elf.DT_STRTAB, @intCast(elf.shndx.dynstr.vaddr(elf)) },
60416222 .{ std.elf.DT_STRSZ, @intCast(elf.shndx.dynstr.size(elf)) },
6223 .{ std.elf.DT_HASH, @intCast(elf.shndx.hash.vaddr(elf)) },
60426224 .{ std.elf.DT_NULL, 0 },
60436225 };
6044 dynamic_index += 8;
6226 dynamic_index += 9;
60456227
60466228 assert(dynamic_index == dynamic_entries.len);
60476229 if (elf.targetEndian() != native_endian) for (dynamic_entries) |*dynamic_entry|
......@@ -7821,6 +8003,7 @@ fn flushResized(elf: *Elf, ni: MappedFile.Node.Index) std.mem.Allocator.Error!vo
78218003 .REL,
78228004 .RELA,
78238005 .DYNSYM,
8006 .HASH,
78248007 => return,
78258008 }
78268009 if (shndx != elf.shndx.plt and