State of Java Elasticity. Tuning Java Efficiency - GIDS.JAVA LIVE 2020 (original) (raw)

• 1. A STATE OF JAVA ELASTICITY Tuning Java Efficiency

• 2. About Speaker ● Ruslan Synytsky ● CEO and co-founder of Jelastic PaaS ● Java Champion ● Two-times Duke’s Choice Award Winner ● Former lead of engineering team at National Data Center (NDC) at National Space Agency of Ukraine ● @siruslan

• 3. Agenda ● Java Memory Usage Problems and Use Cases ● Recent Improvements in OpenJDK ● Testing Elasticity with Different Garbage Collectors ● Upcoming Improvements in OpenJDK

• 4. Unreleased Heap Memory The Problem Symptoms Over-Allocation and Underutilization

• 5. Java Memory Usage Pain Points Q: Return memory from JVM to OS Q: Java VM - does the freed memory return to the OS? Q: More flexible memory management Q: Force JVM to free memory Q: Why Java does not release memory? Q: Does GC release back memory to OS? Q: Why does this java process not release memory? ...

• 6. Large Java Memory Requirements WHAT IS THE MOST CHALLENGING ASPECT OF WORKING WITH JAVA EE? Jakarta EE Developer Survey 2018 “The most widely acknowledged issue when employing with Java EE is large memory requirements (40%)”

• 7. Reasons to Seek Java Elasticity Getting resources from common Desktop Applications, IDEs Scheduled tasks for data processing Cloud cost saving with pay-per-use NEED FOR ELASTICITY Automated scaling of resources on the fly without JVM restart and downtimes Dynamic stateful workloads Right-sizing problem with containers

• 8. Transition from Monolith to Microservices

• 9. Resource Limit vs Real Usage in VM

• 10. Resource Limit vs Real Usage in Container

• 11. Elasticy Use Case in the Cloud

• 12. Elasticy Use Case in the Cloud Pool of Resources

• 13. Elasticy Use Case in the Cloud

• 14. Elasticy Use Case in the Cloud

• 15. Elasticy Use Case in the Cloud

• 16. Elasticy Use Case in the Cloud

• 17. Real Statistics of Resource Consumption with Containers

• 18. Using automatic vertical scaling, cloud provides can offer economically advantageous pricing based on the actual resource consumption Forbes - Deceptive Cloud Efficiency: Do You Really Pay As You Use? Pay-As-You-Go Pay-per-Use Pay-As-You-Go vs Pay-per-Use

• 19. Understanding JVM Footprint

• 20. Understanding JVM Footprint MAXIMUM MEMORY USAGE [-Xmx] + [-XX:MaxMetaspaceSize] + [-XX:MaxDirectMemorySize] + Num_Of_Threads * [-Xss] Useful discussion at SoW Java using much more memory than heap size and presentation Memory Footprint of a Java Process

• 21. -Xmx=[25% of OS Total Memory] Useful options to adjust defaults in % -XX:MaxRAMPercentage -XX:MaxMetaspaceSize=[unlimited] -XX:MaxDirectMemorySize=[-Xmx] -Xss[1m] Alternative syntax -XX:ThreadStackSize=[1m] Specify these options explicitly if you face related OutOfMemoryError Default Limits (Depend on The Platform!)

• 22. Understanding of the OutOfMemoryError Exceptions OutOfMemoryError exception is usually thrown when there is insufficient space to allocate an object in the Java heap or insufficient native memory to support the loading of a Java class https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html ● java.lang.OutOfMemoryError: Java heap space ● java.lang.OutOfMemoryError: GC Overhead limit exceeded ● java.lang.OutOfMemoryError: Requested array size exceeds VM limit ● java.lang.OutOfMemoryError: Metaspace ● java.lang.OutOfMemoryError: Compressed class space ● java.lang.OutOfMemoryError: reason stack_trace_with_native_method ● java.lang.OutOfMemoryError: request size bytes for reason. Out of swap space?

• 23. An Example of OutOfMemoryError Exception Used Heap > Xmx

• 24. An Example of OOM Killer Situation (Even Worse) oom_kill is a job that helps to sacrifice one or more processes in order to free up memory for the system

• 25. The Heap Vertical Scaling Problem Symptoms Unreleased Heap Memory

• 26. Calling Full GC Periodically (Workaround) https://github.com/jelastic-jps/java-memory-agent As compacting GC cycles are not triggered automatically, we execute them explicitly by injecting an agent which monitors the memory usage and calls System.gc() periodically: -javaagent:jelastic-gc-agent.jar=period=300,debug=true Consider enabling -XX:+ExplicitGCInvokesConcurrent

• 27. G1 and Full GC java -XX:+UseG1GC -Xmx2g -jar app.jar https://github.com/jelastic/java-vertical-scaling-test

• 28. Simple Jenkins Example After Full GC call initiated by Jelastic GC agent the memory consumption goes down to 0.5G. If we disable the agent then the memory usage at the idle JVM will stay at 3G forever which is a significant waste of resources. If you put additional load on Jenkins the picture will look even worse. Container limit - 8G, max heap limit (Xmx) - 6.5G and the memory consumption after the simple initial sign in action goes to 3G.

• 29. Timely Reduce Unused Committed Memory (JEP 346) Make the G1 garbage collector automatically give back Java heap memory to the operating system when idle ● -XX:G1PeriodicGCInterval=[milliseconds] ● -XX:G1PeriodicGCSystemLoadThreshold=[float] ● -XX:+G1PeriodicGCInvokesConcurrent JEP 346: Promptly Return Unused Committed Memory from G1 java -Xmx2g -XX:+UseG1GC -XX:G1PeriodicGCInterval=900k -XX:G1PeriodicGCSystemLoadThreshold=0.6 -jar app.jar Available from Java 12

• 30. Immediately Improved Heap Elasticity Automatically Released Heap

• 31. Community Recognition

• 32. G1PeriodicGCSystemLoadThreshold in Docker Container Using LXCFS to Improve Container Resource Visibility For getting correct loadavg while running in Docker container use https://github.com/lxc/lxcfs/

• 33. Running GC Tests in Jelastic

• 34. Load Testing Logic https://github.com/jelastic/java-vertical-scaling-test/blob /master/src/com/jelastic/verticalscaling/Load.java#L50 java [OPTIONS] -jar app.jar

• 35. Auto Testing Package https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest.yml

• 36. G1 Collector (-XX:+UseG1GC) The Garbage-First (G1) is a server-style Garbage Collector for multiprocessor machines with a large amount of memory. The heap is partitioned into fixed-sized regions and G1 tracks the live data in those regions. When Garbage Collection is required, it collects from the regions with less live data first. ● 2004, Sun Microsystems JEP 346: Promptly Return Unused Committed Memory from G1

• 37. G1 -Xmx3g -XX:+UseCompressedOops -XX:+UseG1GC -XX:G1PeriodicGCInterval=1k

• 38. Shenandoah GC (-XX:+UseShenandoahGC) Shenandoah GC is a concurrent garbage collector for the JVM. GC tries to perform most of the activities in parallel without interrupting application performance. Such parallelism makes “stop-the-world” (STW) pauses extremely short. Another inherent advantage is an efficient work with small and large heaps with no impact on STW pauses’ length. ● 2014, Christine H. Flood, Red Hat https://wiki.openjdk.java.net/display/shenandoah/Main#Main-Heuristics

• 39. -Xmx3g -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact Shenandoah

• 40. ZGC (-XX:+UseZGC) ZGC is low latency scalable garbage collector. Designed for use with applications that require a large heap and low latency. It uses a bunch of one generation and performs most (but not all) garbage collection in parallel with uninterrupted application work. This greatly limits the impact of garbage collection on your application response time. ● 2018, Per Liden, Oracle JEP 351: ZGC: Uncommit Unused Memory - available from JDK 13 Release

• 41. -Xmx3g -XX:+UseZGC -XX:ZUncommitDelay=1 -XX:ZCollectionInterval=30 ZGC @ Oracle OpenJDK

• 42. C4 (-XX:+UseZST) The C4 (Continuously Concurrent Compacting Collector) is an updated generational form of the Azul Pauseless GC Algorithm and is the default collector of Zing®. C4 differentiates itself from other generational garbage collectors by supporting simultaneous – generational concurrency: the different generations are collected using concurrent (non-stop-the-world) mechanisms that can be simultaneously and independently active. Unlike other algorithms, it is not ‘mostly’ concurrent, but fully concurrent, so it never falls back to a stop-the-world compaction. ● 2010, Gil Tene, Azul Systems

• 43. -Xmx500m -XX:+UseZST C4 @ Zing pmem.conf -> cgroups enabled

• 44. ConcMarkSweep GC (-XX:+UseConcMarkSweepGC) ConcMarkSweep GC collector is designed for applications that prefer shorter garbage collection pauses and which can afford to share processor resources with the garbage collector while the application is running. It makes sense to use such a collector when applications requirements for time garbage collection pauses are low. ● 2004, Sun Microsystems

• 45. -Xmx3g -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC + 4 calls of Full GC via jcmd GC.run ConcMarkSweep (Deprecated since Java 9) Consider using -XX:-ShrinkHeapInSteps

• 46. Serial GC (-XX:+UseSerialGC) Serial GC performs garbage collection in a single thread and has the lowest consumption of memory among all GC types but, at the same time, it makes long pauses that can lead to application performance degradation. ● 2004, Sun Microsystems

• 47. -Xmx3g -XX:+UseCompressedOops -XX:+UseSerialGC + 4 calls of Full GC via jcmd GC.run Serial Consider using -XX:-ShrinkHeapInSteps

• 48. OpenJ9 OpenJ9 uses the Generational Concurrent (-Xgcpolicy:gencon) policy by default, which is best suited to transactional applications that have many short lived objects. Alternative policies are available, including those that cater for applications with large Java heaps (-Xgcpolicy:balanced), applications that are sensitive to response-time (-Xgcpolicy:metronome), or applications that require high application throughput (-Xgcpolicy:optthruput). ● 2017, Eclipse Foundation

• 49. -Xmx3g -XX:+UseCompressedOops -XX:+IdleTuningCompactOnIdle -XX:+IdleTuningGcOnIdle -XX:IdleTuningMinIdleWaitTime=1 -Xjit:waitTimeToEnterDeepIdleMode=1000 Bash command to check the real usage while true do pid=$(pgrep -f java | tail -n1) used=$(ps -orss --no-headers --pid pid)echo"scale=2;pid) echo "scale=2 ; pid)echo"scale=2;used / 1024/1024" | bc sleep 1 done OpenJ9 (Not Enough Tuning and Testing Experience*) Inconsistent behaviour with -XX:+IdleTuningGcOnIdle, mem not released back to OS on Idle

• 50. Parallel GC (-XX:+UseParallelGC) Parallel GC is a “stop-the-world” multithreaded Garbage Collector similar to the serial collector. The primary difference is that multiple threads are used to speed up garbage collection. By default, both minor and major collections are executed in parallel to further reduce garbage collection costs. ● 2000, Sun Microsystems

• 51. -Xmx3g -XX:+UseCompressedOops -XX:+UseParallelGC + periodical jcmd GC.run Parallel

• 52. Main Points of Elastic Vertical Scaling A - initial usage B - maximum usage C - growth speed D - duration before release E - release speed F - minimum usage after release Different GCs provide different results and fine tuning options

• 53. Running GC Tests in Kubernetes

• 54. Auto Testing Package for Kubernetes https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest-k8s.yaml

• 55. Load Testing Logic java [OPTIONS] -jar app.jar where sleep - 100 mode - 2 https://github.com/jelastic/java-vertical-scaling-test/blob/ma ster/src/com/jelastic/verticalscaling/Load.java#L64

• 56. G1 in Kubernetes

• 57. Shenandoah in Kubernetes

• 58. ZGC @ Oracle OpenJDK in Kubernetes

• 59. Joint Comparison - Several Load Cycles RAM CPU

• 60. Running GC Tests with Jakarta EE

• 61. ● simple .war artifact deployed with GlassFish 5 ● JSP that sets 1MB attribute in session ● 1 min session timeout ● https://github.com/jelastic/java-vertical-scaling-test/tree/payara/web app Load Testing Logic Load test GETs webapp endpoint n times for i in {1..n}; do curl -s localhost:8080 > /dev/null; done

• 62. GlassFish with Shenandoah -Xmx3g -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact for i in {1..1000}; do curl -s localhost:8080 > /dev/null; done

• 63. Resizing Xmx On the Fly

• 64. Dynamic Max Heap Resizing - Solving Right Sizing Problem Restart for Xmx Resize

• 65. -XX:SoftMaxHeapSize @ ZGC SoftMaxHeapSize is set for the GC to strive not to grow heap size beyond the specified size unless it is highly needed: ● to keep the heap footprint down, while maintaining the capability to deal with a temporary increase in heap space requirement ● with lots of margin, to increase confidence that you will not run into an allocation stall because of an unforeseen increase in allocation rate Using -XX:SoftMaxHeapSize

• 66. -XX:G1PeriodicGCInterval and -XX:SoftMaxHeapSize G1PeriodicGCInterval (G1) / ZCollectionInterval (ZGC) is a time-based solution which has no direct impact on how much the heap will grow during the given interval. SoftMaxHeapSize is a size-based solution which controls how large the heap can grow, but has no direct relation with uncommit (returning unused memory back to OS) Both options can be used, whichever condition is met first will trigger a GC. ZGC (Java13+) https://bugs.openjdk.java.net/browse/JDK-8222181 G1, Shenandoah (TBD) https://bugs.openjdk.java.net/browse/JDK-8236073

• 67. -Xsoftmx @ OpenJ9 https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/openj9/xsoftmx/index.html Runtime adjustable heap size (-Xsoftmx) allows to adjust heap size dynamically and take advantage of hot-add of memory. You can set this option on the command line, then modify it at run time by using the com.ibm.lang.management.MemoryMXBean.setMaxHeapSize(). This option can be useful in virtualized or cloud environments, for example, where the available memory might change dynamically to meet business needs. By default, -Xsoftmx is set to the same value as -Xmx.

• 68. C4 is fully elastic and can return all empty pages to the OS after each GC cycle. However, C4 sticks to the Xmx it was given, and avoid doing heavy elastic memory dance, since relinquishing memory mappings and reestablishing them on Linux kernels is bandwidth-limited in practice by the rate of page mapping invalidation the kernel can handle. C4 goes above Xmx rather than go between Xms and Xmx. JavaMemMax option controls the true maximum. In the future it will allow both scenarios where above-Xmx is allowed and where above-Xmx is prohibited. Two modes: ● Contingency (default mode) - goes above Xmx if it absolutely has to and will work hard to collect and stay below Xmx. ● Insurance (best effort elasticity) - borrows available memory and goes above Xmx in order to delay GC whenever possible. JavaMemMax @ С4 + ZST (Zing System Tools)

• 69. Xmx can be set higher than the Container Limit. Also both SoftMaxHeapSize and Container Limit can be adjusted on the fly without the need to restart JVM or container. At the moment the heap size can go beyond SoftMaxHeapSize and there is no guarantee on how much the heap will grow other than up to Xmx. The problem arises when Used Heap comes close to Container Limit. Most likely the JVM will be killed by the OOM Killer as it exceeds the amount of memory available in the container. JEP Draft: Dynamic Max Memory Limit @ G1 We can introduce manageable HardMaxHeapSize. In this case JVM will throw OOM Error (losing one operation) instead of OOM Kill (losing whole JVM). Xmx Container Limit SoftMaxHeapSize

• 70. JEP 387: Elastic Metaspace (Target Release Java 16) To deal with potential problems involving virtual memory fragmentation or uncommit speed, we will add a new production command-line option to control metaspace reclamation behavior: -XX:MetaspaceReclaimPolicy=(balanced|aggressive|none) ■ balanced: Most applications should see an improvement in metaspace memory footprint while the negative effects of memory reclamation should be marginal. This mode is the default, and aims for backward compatibility. ■ 'aggressive': Offers increased memory-reclamation rates at the cost of increased virtual-memory fragmentation. ■ 'none': Disables memory reclamation altogether. Applications that use many small class loaders may suffer from unreasonably high metaspace usage. Applications with heavy class loading and unloading activity can thus accrue a lot of unused space in the metaspace freelists. That space can be returned to the operating system to be used for other purposes if it is not fragmented, but that’s often not the case.

• 71. Keep Only Best Java Memories Learn More Get In Touch @jelastic info@jelastic.com