Mapfiles and static linking of standard libraries (original) (raw)

David Holmes david.holmes at oracle.com
Wed Feb 19 09:33:35 UTC 2014

Note that we (hotspot) do use a mapfile on OSX. This was needed to avoid every symbol in libjvm being exported resulting in the wrong versions of operator new/delete being used.

David

On 19/02/2014 7:22 PM, Magnus Ihse Bursie wrote:

(Adding in build-dev since we're moving into the jdk build as well now.)

On 2014-02-17 17:39, Volker Simonis wrote: On Mon, Feb 17, 2014 at 4:08 PM, Dmitry Samersoff <dmitry.samersoff at oracle.com> wrote:

Dan,

It was my bad - missed two related threads. We have two problems related to map files: 1. We have hand-written mapfiles and have to manage it manually. It's better to generate map file automatically or use visibility attributes. I would strongly vote against automatically generating the map files from sources by parsing for special patterns as proposed by Magnus because I think this will just introduce another level of complexity and another point of failure. My priorities is to prefer no map files if we can avoid it, but to prefer automatically generated over static, checked in, mapfiles if they cannot be avoided. So I'll try to join you in the fight to get rid of them altogether, but if that fails, I still want to generate them. :-) Having static map files are a source of complexity and point of failure in itself as well. >From this discussion so far I learned the following: - as long as Oracle insists on static linking libstdc++ and libgcc there's no way of getting rid of the map files. - using -fvisibility=hidden/attribute((visibility("default"))) is still desirable because it has positive performance impacts on some platforms and because it is the easiest and cleanest solution for the future when we can finally get rid of the map files. Moreover it is already used anyway. attribute((visibility("default"))) sounds very much like a gcc extension. Is there a similar construct for solaris studio? Otherwise we would still need mapfiles on solaris. Also, does attribute((visibility("default"))) work with clang? When I check the jdk build system, I notice that we do not use mapfiles on macosx. (This does not stop us from having a "mapfile-macosx" in a library... :-/) Also, it seems that the same goes for xlc on aix; we will process mapfiles but in the end ignore them. I have no idea what this means for the situation on the exported symbols on these platforms. And then we have the reorder files in jdk, where we extend the mapfiles with a reorder section which specify a specific order in which to store functions, based on a (believed, perceived, or once upon a time, real) performance boost. Oh my! When I carefully checked the jdk code, it seems that there's a bug in NativeCompilation, that makes all mapfiles ignored unless there is also a reorder file. That's only true for 7 libraries, out of the 40 that has mapfiles. The same goes for launchers, we point to an incorrect directory containing the mapfiles. So either I'm mis-reading the code, or we have not actually used most of the mapfiles for possibly a very long time. I'm not sure why this has not been noticed. - This solution would also establish something like a very low level contract that the VM will only ever export JVM*/JNI* symbols. Currently, the following symbols which don't start with JNI and JVM and are exported by the hotspot/make/linux/makefiles/mapfile-vers-product map file from libjvm.so on Linux: # miscellaneous functions jiofprintf; jiosnprintf; jiovfprintf; jiovsnprintf; - exported and used in the jdk/ (trough jdk/src/share/javavm/export/jvm.h) - could be easily renamed to JVM* Unfortunately, it's not that easy. While not nicely designed, those function that by tradition has been exported by libjvm.so can be considered part of the already-existing JVM/JDK contract. For the shared libraries in the jdk/ repository, the situation is a little different. Only the following three libraries do statically link libstdc++/libgcc: libfontmanager.so libunpack.so libsunec.so Yeah, that's a story in itself, isn't it. I remember being upset about it when I realized this was the case, but then forgot about it. We shouldn't mix statically and dynamically linking the standard library, that's just dumb. * The benefits of static linking is that we do not depend on a specific version of the library being available on the customer's computer; we can run with our own well known, tested and trusted version. But if you still do dynamic linking, other libraries are susceptible to this problem. * The benefits of dynamic linking is that we get a smaller footprint, and do not need to keep track of security patches in the standard libraries. But if we ship some libraries with static linking, we get a unneccessary large footprint and we still need to keep track of security issues. I can see good points for both approaches (even though I would tend to prefer dynamic linking), but mixing them just gives us the bad parts from both worlds. /Magnus

More information about the build-dev mailing list