[llvm-dev] [cfe-dev] Remove unused types from ASTContext (original) (raw)

mateusz janek via llvm-dev llvm-dev at lists.llvm.org
Thu Apr 27 14:33:51 PDT 2017

My bad, I was sure that I'm "replying all", thanks for that.

Of course I don't want you to feel any kind of pressure from my side. I'm just curious if I can somehow adjust llvm/clang to help me with such problem.

I'll try to provide MCVE, but unfortunately probably tomorrow.

2017-04-27 21:56 GMT+02:00 mats petersson <mats at planetcatfish.com>:

It would be good if you did "reply all" so that those other people that have read and is interested in this discussion can see the answers. I've added Brian and the mailing list back into this reply.

That is an impressive amount of template-code. I've never really done much TMP, but I will see if I can at least somewhat analyse where the memory/time goes in this. It would probably have been better to have a SMALL example, that compiles in a few seconds to a few minutes, and a "plain" program that achieves roughly the same thing, but I'll see what I can do. When I get home (it's about 9pm, and I'm still at work), and I make no promises of ANY kind... -- Mats

On 27 April 2017 at 18:51, mateusz janek <stryku2393 at gmail.com> wrote: Thanks for the replies (:

@Brian Cain >Do other compilers behave similarly in terms if memory consumption for your example code? Good point. I'll check that today with the g++.

@Mats Petersson >You will need ALL types (that are used at all) to be present at semantic analysis, since types are used to determine for example implicit casts (float to int, int to float, short to int, int to short - and even if your type can't be converted, it still needs to be present during that phase). Since Semantic analysis is done at the end of the compilation of a translation unit. And of course, types are used during translation from AST to IR, so need to be (at least somewhat) present during the IR generation phase too. [And the compiler WILL need to know what Bar<FooParam> is to_ _find Bar<FooParam>::BarResult, even if that is then translated to_ _sizet or int._ _Thanks a lot for an explanation (:_ _>Also, are you sure that it's the number of types that is the real cause_ _of the problem?_ _Yes, I'm pretty sure. Almost everything in this project is a template_ _type._ _>Can you provide a more complete example (preferrably one that compiles,_ _but does not take "over night", in a few minutes would be good, that_ _illustrates that it uses a lot more memory than a comparable piece of code_ _with less types introduced in it)._ _Of course. Check the repo: https://github.com/stryku/ctai (Not sure if you've asked for this or for a complete new code which illustrates the problem, if the new code - sorry and let me know I'll try to prepare something) There is still v1.0 version and it compiles just fine in about 1-2s. To increase the time and memory usage you can just add a couple of dummy asm instructions in main.cpp, e.g.: _ _using code = decltype(_ _"mov ebp , esp "_ _"push eax " // declare fourusing code = decltype(_ _"mov ebp , esp "_ _"push eax " // declare four variables_ _"push eax "_ _"push eax "_ _"push eax "_ _"mov DWORD PTR [ ebp + 8 ] , 0 "_ _"mov DWORD PTR [ ebp + 12 ] , 1 "_ _"mov DWORD PTR [ ebp + 16 ] , 1 "_ _"mov DWORD PTR [ ebp + 4 ] , 1 "_ _":looplabel "_ _"mov eax , DWORD PTR [ ebp + 4 ] "_ _"mov ebx , 15 " //we want to get 15th fibonacci element_ _"cmp eax , ebx "_ _"jge .endlabel "_ _"mov edx , DWORD PTR [ ebp + 8 ] "_ _"mov eax , DWORD PTR [ ebp + 12 ] "_ _"add eax , edx "_ _"mov DWORD PTR [ ebp + 16 ] , eax "_ _"mov eax , DWORD PTR [ ebp + 12 ] "_ _"mov DWORD PTR [ ebp + 8 ] , eax "_ _"mov eax , DWORD PTR [ ebp + 16 ] "_ _"mov DWORD PTR [ ebp + 12 ] , eax "_ _"mov eax , DWORD PTR [ ebp + 4 ] "_ _"mov ebx , 1 "_ _"add eax , ebx "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"mov DWORD PTR [ ebp + 4 ] , eax "_ _"jmp .looplabel "_ _":endlabel "_ _"mov eax , DWORD PTR [ ebp + 16 ] "_ _"exit"s);_ _ Version v1.0, from the github is simpler and works in a more naive way (e.g. every asm instruction is recursively expanded) but basically, it has same problems as the current version - there is a lot of types. I know that I can optimize the template code, but I'm sure that at some point I'll hit the barrier and the only way will be to modify llvm/clang, which I'm trying to understand and eventually do. Thanks, Stryku

2017-04-27 10:54 GMT+02:00 mats petersson <mats at planetcatfish.com>:

On 27 April 2017 at 03:13, Brian Cain via cfe-dev <_ _cfe-dev at lists.llvm.org> wrote: Do other compilers behave similarly in terms if memory consumption for your example code? On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <_ _cfe-dev at lists.llvm.org> wrote: Hi, Before I'll begin, I want to say that I don't want to implement below 'feature' in a clang production code. I want to create my own custom build for my own needs. Abstract: I'm developing a hobby project which creates a lot of types in a compile time, in one .cpp file. Really a lot. I tried to compile next version, but it was compiling whole night and took ~30G of memory (16G RAM + swap) and wasn't able to finish. I know that almost all of these types are 'used' only once, they're kind of helpers. Basically what I want to achieve is to decrease memory usage to not need to use the swap, because it increases compilation time a lot. Question: Is there a way to implement a removing no longer used types? E.g. template struct Foo { using FooResult = typename Bar::BarResult; }; And I'd want to remove the 'Bar' type from a 'ASTContext' because I know that I won't need it anymore. (then, if there will again occur an initialization of very same 'Bar', I'm ok with that I'll need to create it from the scratch). So basically is there any way to detect if type is still referenced somewhere, so I can remove it if it isn't?

You will need ALL types (that are used at all) to be present at semantic analysis, since types are used to determine for example implicit casts (float to int, int to float, short to int, int to short - and even if your type can't be converted, it still needs to be present during that phase). Since Semantic analysis is done at the end of the compilation of a translation unit. And of course, types are used during translation from AST to IR, so need to be (at least somewhat) present during the IR generation phase too. [And the compiler WILL need to know what Bar<FooParam> is to find Bar<FooParam>::BarResult, even if that is then translated to sizet or int. Also, are you sure that it's the number of types that is the real cause of the problem? Can you provide a more complete example (preferrably one that compiles, but does not take "over night", in a few minutes would be good, that illustrates that it uses a lot more memory than a comparable piece of code with less types introduced in it). -- Mats I hope I've described my problem well. I know that in fact I'm asking for a explanation how types in clang are related to each other - that's a lot, but I'd really like to learn and understand that (: Thanks in advance for help! Stryku

cfe-dev mailing list cfe-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

cfe-dev mailing list cfe-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170427/44a7279d/attachment-0001.html>

More information about the llvm-dev mailing list