Fix failures to allocate arrays with a very large Java heap size by EthanNelson-Moore · Pull Request #1986 · broadinstitute/picard (original) (raw)

Description

The maximum length of a Java array is not exactly Integer.MAX_VALUE, but slightly less due to the space taken up by the object header. The exact maximum differs depending on the platform and Java version. This was already accounted for in one instance, but not others. This was causing MarkDuplicates to fail with a Java heap size of slightly less than 1TB when MAX_RECORDS_IN_RAM was not specified (and therefore calculated based on the amount of available memory). This commit fixes the other instances and changes the maximum size in the existing instance to Integer.MAX_VALUE - 32 instead of Integer.MAX_VALUE - 5 to decrease the likelihood of allocation failures on different Java versions and platforms.

Checklist (never delete this)

Never delete this, it is our record that procedure was followed. If you find that for whatever reason one of the checklist points doesn't apply to your PR, you can leave it unchecked but please add an explanation below.

Content

• Added or modified tests to cover changes and any new functionality - It would be hard to automatically test this change since triggering the issue requires an extremely large amount of memory, which is not likely to be the case in a CI pipeline.
• Edited the README / documentation (if applicable) - This is a bug fix, so there is no need to update the README or documentation, only the changelog of a future release.
• All tests passing on github actions

Review

• Final thumbs-up from reviewer
• Rebase, squash and reword as applicable

For more detailed guidelines, see https://github.com/broadinstitute/picard/wiki/Guidelines-for-pull-requests