Hacker Newsnew | past | comments | ask | show | jobs | submit | MaskRay's commentslogin

shameless plug: lld 23.x is significantly faster than 22.x! I wrote about the optimizations in https://maskray.me/blog/2026-04-12-recent-lld-elf-performanc... As a side note, parallel input file parsing and symbol resolution aren't a silver bullet. I have an experimental branch (net ~800 lines of code) adding this to lld/ELF, but it actually loses performance when restricted to --threads={1,2}

I read how mold implements parallel input file parsing a while ago. It is overly complicated. wild's model seems superior.

---

"Table 4." in the paper feels misleading to me. I have never measured such high speedups. In addition, on debug builds (especially --gdb-index builds), mold's edge over lld is smaller.

On ICF, some unknown fraction of mold's speedup is due to "doesn't do the symbol bootkeeping". The symbol tables are different in mold but identical in lld.


Table 4 shows actual measurements on my Threadripper 7980X machine, so I'm not sure whether the numbers look too good or not. I chose large programs more or less at random as the test set without thinking too much about it, so they were not cherry-picked. I'm developing mold on this machine, so mold may naturally be optimized for this configuration. That's why we have Table 6. My benchmark is available at https://zenodo.org/records/21882261. It's pretty easy to run, so you may want to try it out.


Whoa, it's MaskRay! Your blog has been a lifesaver for me every time I've needed to understand some obscure detail about linker behavior. Thanks a ton for all your work in this area, and thanks even more for writing about it.


Interesting to see that people are still spending much time on assembly languages :) I've had a lot of fun with it in recent years as well. (Shameless plug: I've written a few on LLVM integrated assembler regarding better fragments and improving expressions and relocations).

When comparing GNU Assembler and MASM, GAS is missing many features: while loop, string processing (.e.g strlen)

> page 3: "It’s important to understand that MASM converts macro invocation arguments to text values before doing the macro expansion"

Like MASM, GAS uses call-by-name evaluation by default. While GAS's altmacro mode does allow for expression evaluation using syntax like `%(1+2)`, it has strict limitations: it only supports absolute expressions and is restricted to argument positions.

In contrast, MASM's % operator (page 8) looks far more general.


The GNU assembler is not intended to be used by humans, but its goal is to assemble the output of compilers.

On Linux, the "standard" assembler for humans is "nasm", though there are also others.

For anyone writing a non-negligible amount of code in an assembly language it is essential to develop or get from elsewhere a comprehensive library of macros, for avoiding to write huge amounts of boilerplate.


That is true of all UNIX assemblers, they never had the "used by humans" culture, rather being part of a C compiler pipeline approach.

At least since UNIX v4.

All good Assemblers have come from Amiga, Atari, PC world.


While we could utilize zigzag encoding (i>>31) ^ (i<<1) to convert SLEB128-encoded type/addend to use ULEB128 instead, the generate code is inferior to or on par with SLEB128 for one-byte encodings on x86, AArch64, and RISC-V. Haven't tried wider values - but zigzag encoding is likely slower as well

// One-byte case for SLEB128 int64_t from_signext(uint64_t v) { return v < 64 ? v - 128 : v; }

// One-byte case for ULEB128 with zig-zag encoding int64_t from_zigzag(uint64_t z) { return (z >> 1) ^ -(z & 1); }


GetProgramPath consulting PATH is a pretty standard behavior inspired by GCC. Clang driver may find an external program from its program paths (e.g. -ccc-install-dir, the GCC installation's bin directory, -B, ) and then PATH.

> Clang relies on address layout for ordering things

Every such instance is a bug. I have fixed many issues in 2023. There is even an upstream build bot https://discourse.llvm.org/t/reverse-iteration-bots/72224


I want to verify the claim that it builds the Linux kernel. It quickly runs into errors, but yeah, still pretty cool!

make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc -j30 defconfig all

``` /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:44:184: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) ~0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (~0x80000000); (void)pto_tmp__; } asm ("and" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:49:183: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) 0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (0x80000000); (void)pto_tmp__; } asm ("or" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:61:212: error: expected ';' after expression before 'pao_tmp__' ```


They said it builds Linux 6.9, maybe you are trying to compile a newer version there?


git switch v6.9

The riscv build succeeded. For the x86-64 build I ran into

    % make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 HOSTCC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 LDFLAGS=-fuse-ld=bfd LD=ld.bfd -j30 vmlinux -k
    make[1]: Entering directory '/tmp/linux/x86'
    ...
      CC      arch/x86/platform/intel/iosf_mbi.o
    ccc: error: lgdtl requires memory operand
      AR      arch/x86/platform/intel-mid/built-in.a
    make[6]: *** [/home/ray/Dev/linux/scripts/Makefile.build:362: arch/x86/realmode/rm/wakeup_asm.o] Error 1
    ld.bfd: arch/x86/entry/vdso/vdso32/sigreturn.o: warning: relocation in read-only section `.eh_frame'
    ld.bfd: error in arch/x86/entry/vdso/vdso32/sigreturn.o(.eh_frame); no .eh_frame_hdr table will be created
    ld.bfd: warning: creating DT_TEXTREL in a shared object
    ccc: error: unsupported pushw operand
There are many other errors.

tinyconfig and allnoconfig have fewer errors.

    RELOCS  arch/x86/realmode/rm/realmode.relocs
    Invalid absolute R_386_32 relocation: real_mode_seg
Still very impressive.


They said that it wasn't able to support 16 bit real mode. Needs to call gcc for that.


I feel like I could have done this in a much shorter time, for much less tokens, but still very impressive!


I accidentally published handling-long-branches.html . I actually removed it quickly, but Google Search already got it....

https://news.ycombinator.com/item?id=46759921 (Long branches in compilers, assemblers, and linkers), 2 days ago


Amazing! I read you article in 2012 when the link was https://homepages.cwi.nl/~tromp/maze.html I was learning Haskell and Ocaml and wrote my own article in Chinese then https://maskray.me/blog/2012-11-02-perfect-maze-generation

Now I should fix the link.


I recall that google3 switched to -funsigned-char for x86-64 a long time ago.


A cursory Chromium code search does not find anything outside third_party/ forcing either signed or unsigned char.

I suspect if I dug into the archives, I'd find a discussion on cxx@ with some comments about how doing this would result in some esoteric risk. If I was still on the Chrome team I'd go looking and see if it made sense to reraise the issue now; I know we had at least one stable branch security bug this caused.


Open source project maintenance follows a similar model, but with a different set of stakes.

The "price tag" of voicing concerns is lower, yet raise them too often and you still earn a reputation as obstructionist. Meanwhile, the cost of accepting problematic changes can be higher—you may end up maintaining that code long after changing jobs. And unlike corporate politics, the "influence bank account" is public: communications are archived indefinitely.

There is a fascinating shift in how "withdrawals" are calculated: In a corporate hierarchy, the cost of dissent feels exponential: something like `cost = exp(their_level - your_level)`. Say, as a Google L3/L4/L5 engineer, opposing L6-L8 feels like trying to make a massive withdrawal with a tiny balance. In contrast, in OSS the cost almost stays constant despite the corporate level difference.

This created a paradox for me: leaving Google means less time for LLVM maintenance, but it also lets me voice objections more freely, without the shadow of internal performance ratings or hierarchical friction.

That said, I know I've been "withdrawing" heavily, including from a lot of previous colleagues. In a recent LLVM Project Council meeting:

> There is a pattern of behavior here of blocking contributions due to concerns about maintenance cost and design simplicity.

(I appreciate the transparency of making these meetings public, by the way.)

I had to respond at https://discourse.llvm.org/t/llvm-project-council-meeting-no...


We are missing tooling to partition a huge binary into a few larger shared objects.

As my https://maskray.me/blog/2023-05-14-relocation-overflow-and-c... (linked by author, thanks! But I maintain lld/ELF instead of "wrote" it - it's engineer work of many folks)

Quoting the relevant paragraphs below:

## Static linking

In this section, we will deviate slightly from the main topic to discuss static linking. By including all dependencies within the executable itself, it can run without relying on external shared objects. This eliminates the potential risks associated with updating dependencies separately.

Certain users prefer static linking or mostly static linking for the sake of deployment convenience and performance aspects:

* Link-time optimization is more effective when all dependencies are known. Providing shared object information during executable optimization is possible, but it may not be a worthwhile engineering effort.

* Profiling techniques are more efficient dealing with one single executable.

* The traditional ELF dynamic linking approach incurs overhead to support [symbol interposition](https://maskray.me/blog/2021-05-16-elf-interposition-and-bsy...).

* Dynamic linking involves PLT and GOT, which can introduce additional overhead. Static linking eliminates the overhead.

* Loading libraries in the dynamic loader has a time complexity `O(|libs|^2*|libname|)`. The existing implementations are designed to handle tens of shared objects, rather than a thousand or more.

Furthermore, the current lack of techniques to partition an executable into a few larger shared objects, as opposed to numerous smaller shared objects, exacerbates the overhead issue.

In scenarios where the distributed program contains a significant amount of code (related: software bloat), employing full or mostly static linking can result in very large executable files. Consequently, certain relocations may be close to the distance limit, and even a minor disruption (e.g. add a function or introduce a dependency) can trigger relocation overflow linker errors.


> We are missing tooling to partition a huge binary into a few larger shared objects

Those who do not understand dynamic linking are doomed to reinvent it.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: