[llvm-dev] Multi-Threading Compilers (original) (raw)

Nicholas Krause via llvm-dev llvm-dev at lists.llvm.org
Fri Feb 28 08:35:06 PST 2020

On 2/28/20 12:56 AM, Chris Lattner wrote:

On Feb 27, 2020, at 9:44 PM, Nicholas Krause <xerofoify at gmail.com_ _<mailto:xerofoify at gmail.com>> wrote:

On 2/28/20 12:19 AM, Chris Lattner wrote:

Hi Nicholas,

You might want to check out MLIR: its pass manager is already automatically and implicitly multithreaded. -Chris Chris, I was aware that LLVM was moving to MLIR at some point due to this. I've curious as to how MLIR deals with IPO as that's the problem I was running into. Even if you have pipelines what guarantees do we have about IPO or are there any. For example if an analysis pass runs after a loop pass for heuristics to get good data,  does MLIR ensure that?   The problem isn't getting a pipeline as much as IPO issues in terms of rescheduling in a pipeline or incorrectly splitting up into pipelines. I yet to find a good solution on the GCC side as well and it seems that there will be issues with this no matter what, not sure of what trade offs the LLVM side is willing to make. Hi Nicholas, It is very difficult to mix things like loop passes and IPO passes in any system, unless one is prepared to preserve the analysis results that the other depend on.  One nice thing about MLIR is that it defines this away, by allowing explicit representation of loops in the IR.  This means that it isn’t an analysis pass that gets “invalidated” like the existing LLVM LoopInfo analysis pass.  It is just structurally impossible for this to happen.  I don’t think that all of the AnalysisPass’s in LLVM have been super successful. The other issues is that graph data structures to my knowledge do not allow insertion of multiple nodes at the same time or how to scale the graphs for callers or SSA. Not sure if you guys have encapsulated the operators and data structures in a series of classes as that would be the first part. The other part is figuring out how to link and interact with build systems to launch threads from make -j or other similar things. Yeah, MLIR handles that by factoring global use-def chains on symbols (e.g. functions) as being different than local use-def chains. This makes it much more efficient. You can find more information on MLIR symbols here <https://mlir.llvm.org/docs/SymbolsAndSymbolTables/>. Thanks for the notice about MLIR through maybe my IPO is not really there but after reading parts of it seems to be a issue through a little smaller still and thanks for the  prompt response, Sure, happy to help.  It would be great to see a GIMPLE dialect in MLIR someday :-) -Chris Chris,

I asked the GCC what they want to do about MLIR and am waiting for a reply. Anyhow what is the status and what parts are we planning to move to MLIR in LLVM/Clang.  I've not seen any discussion on that other than starting to plan for it.  Perhaps we should start a wiki page for that as I don't think all parts need to be MLIR. I don't have access to writing for the wiki so unfortunately I can't start writing it up unless I get access.

Regards and this is one reason you do your research before writing a multi-threading program a lot of the research or work has been done at this point, Nick

Nick On Feb 27, 2020, at 7:05 PM, Nicholas Krause via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:

Greetings All, For the last few months since the GCC Cauldron I've been researching/helping out plan for multi-threading GCC. I've posted my GCC ideas here: https://docs.google.com/document/d/1poRRgSCtRyYgMHjV0itW8iOzJXpTdHYIpC9gUMjOxk/edit as I've heard there is interest on the LLVM side as well. I've talked to some of the IBM folks from the Toronto area about it face to face due to them having more middle end and back end knowledge then me. Not sure if anyone is interested in my ideas or wants me to extend my help on the LLVM side, Nick

LLVM Developers mailing list llvm-dev at lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200228/a50b3cb3/attachment.html>

More information about the llvm-dev mailing list