Possibility of code cache unloading (original) (raw)

Eric Caspole eric.caspole at amd.com
Mon Oct 12 17:51:18 PDT 2009

Tom, Thanks for your comments. Your idea is a good extension of what I
have been playing with. I am going to do some experiments on the big
apps I have access to. Eric

On Oct 12, 2009, at 12:45 PM, Tom Rodriguez wrote:

There are two main problems. First is how to identify nmethods which aren't being used without penalizing execution too much and also without introducing new races. Reclamation of nmethods is somewhat tricky and reusing existing mechanisms would be good. Second is the policy part. Reclamation of nmethods isn't prompt so you have to get enough warning that the code cache is getting full to start reclamation before you run out. It's basically a GC policy problem though we can always simply delay compilation until some storage has been reclaimed. A hiccup near the transition would still be better than our current policy of simply stopping compilation.

For the first part, I'd always imagined we end up with some strategy where we marked methods as speculatively not entrant and if they weren't used within some time period we'd transition them fully to not entrant and reclaim the nmethod storage. One problem with this is the repatching of the nmethod back into a normal state. I guess if it's done at a safepoint it wouldn't be a big deal. On the other hand it occurred to me today that we could rely on the same guarantees that the nmethod sweeper uses to do this without requiring patching. There are only two ways to get into compiled code. You either grab the nmethod entry point from the methodOop you are invoking, which is how interpreter invokes and polymorphic dispatch work, or you have a CompiledIC that jumps you there directly which is the normal way compiled code works. If you break both of these kinds of links then an nmethod can no longer be invoked. This is essentially how making an nmethod not entrant works but since the cleaning of inline caches is done at a safepoint we also patch the entry point of the nmethod to dispatch back into the call resolution code. This allows the cleaning of the CompiledICs to be done lazily because any caller that still references the nmethod will still end up back in the resolution code. I think we could use the same strategy for identifying unused nmethods. We simply break the reference from the methodOop to the nmethod and start a cache cleaning but without patching the entry point. If we ever invoke those nmethods again we'll end up going through the call resolution code where we can detect that the nmethod is really being used and simply restore the nmethod and continue using it. If it's never referenced within some time period the nmethod can simply be freed because we know that it's clean. As far as policy goes we could simply mark every nmethod this way when we get into trouble but I think we'd want a more GC like policy that would begin identifying nmethods once the code cache started filling up and reclaim them before we actually run out. tom On Oct 12, 2009, at 9:53 AM, Eric Caspole wrote:

We discovered some app servers do not have really well behaved class loader schemes, so that in a long running process with multiple re-deployments of the same web app, the old stuff never gets unloaded and the first thing to run out of space in this case was the code cache. In this event the compiler shuts itself off and does not turn itself on regardless of what happens in later GC/ sweep cycles.

We have been wondering what it would take to do some kind of code cache unloading so long running applications like this won't end up having the later redeployments running interpreter-only. Many users do not even know when this has happened and may or may not notice a gradual mysterious slowdown until they just restart the process. In various discussions ideas have popped up from marking some amount of existing nmethods non-entrant so they will get unloaded in the existing code cache, to more elaborate reallocation schemes for the whole code cache. If necessary, a short term slowdown before getting back to having all the hot methods recompiled seems better than restarting the process. Perhaps there could be some JMX notification for this situation. One idea that came out of our testing was the idea of a current working set of hot methods. We saw that if enough space could be regained in the code cache, the program's normal operation would require recompiles only of the current hot method set, which is hopefully a lot smaller than the whole code cache size. Then it would quickly resume normal operation, and only in the event of one or more web app redeployments as mentioned above would the code cache require another flush operation, hopefully days or weeks later. Lastly, it is probably desirable to have a fallback plan of giving up and shutting off the compiler if the flush cycles are happening too often, for example if the hot method working set size is too close to the whole code cache size, and the application performance won't be any worse than it gets today. We'd like to hear if anyone else has a strong opinion or great idea on this topic, and what corner case did we not think of. I remember a wise crack about this topic at the JVM Languages Summit a few weeks ago so it seems someone out there is thinking about it. Thanks, Eric

More information about the hotspot-compiler-dev mailing list