(original) (raw)

Leonardo, to answer to your questions, yes to all of them  You can take a

look at this prototype/proposal: https://reviews.llvm.org/D55585

Overall, computing ghashes in parallel at link-time and merging Types with them

is less costly that the current approach to merging. The 35sec you’re seeing

for merging should go down to about 15sec. The patch doesn’t parallelize

(yet) the Type merging itself, but we have an alternate multithread-suitable

implementation of DenseHash which already supports lockless, wait-free,

insert/fetch/resize.

The prototype allows for testing different hashing algorithms, and indeed

xxHash seems to be the best general-purpose choice. I’ve also added support

for more specialized hardware-based hashes, like Casey Muratori’s Meow Hash

(uses hardware AES SSE 4.2 instructions), which brings the figures down a bit.

Future changes could write back the computed ghash stream back to OBJs if

/INCREMENTAL is specified (just an idea). Incrementally linking will be faster

that way when working with MSVC OBJs.

As for creating PDBs for independent projects, that would help most likely.

However the ghash stream would need to be stored in the PDB in that case

(currently, ghashes are dropped after merging). That could help when using

rarely compiled projects, used along with network caches.

I will start sending smaller patches to converge towards the functionally of

the prototype above.

Best,

Alex.

From: Zachary Turner
Sent: Sunday, February 24, 2019 1:20 AM
To: Leonardo Santagada
Cc: Alexandre Ganea ; Reid Kleckner ; llvm-dev
Subject: Re: \[llvm-dev\] Making LLD PDB generation faster