ThreadLocalRandom clinit troubles (original) (raw)

Peter Levart peter.levart at gmail.com
Tue Jun 24 14:03:17 UTC 2014

Hi Martin,

On 06/22/2014 07:12 PM, Martin Buchholz wrote:

We know that loading the networking machinery is problematic. On Linux we would be content to hard-code a read from /dev/urandom, which is safer and strictly more random than the existing network hardware determination, but y'all will reject that as too system-dependent (insufficient machinery!). Hmmmm .... maybe not .... as long as we code up a good fallback ...

I learned that SecureRandom by default on Unix uses /dev/random for "seed bytes" and /dev/urandom for nextBytes. Here's my proposal, that in the default case on Unix doesn't load any machinery, and as a fallback loads the SecureRandom machinery instead of the network machinery, while maintaining the ultra-secure behavior of the java.util.secureRandomSeed system property:

private static long initialSeed() { byte[] seedBytes = initialSeedBytes(); long s = (long)(seedBytes[0]) & 0xffL; for (int i = 1; i < seedBytes.length; ++i) s = (s << 8) | ((long)(seedBytes[i]) & 0xffL); return s ^ mix64(System.currentTimeMillis()) ^ mix64(System.nanoTime()); } private static byte[] initialSeedBytes() { String pp = java.security.AccessController.doPrivileged( new sun.security.action.GetPropertyAction( "java.util.secureRandomSeed")); boolean secureRandomSeed = (pp != null && pp.equalsIgnoreCase("true")); if (secureRandomSeed) return java.security.SecureRandom.getSeed(8); final byte[] seedBytes = new byte[8]; File seedSource = new File("/dev/urandom"); if (seedSource.exists()) { try (FileInputStream stream = new FileInputStream(seedSource)) { if (stream.read(seedBytes) == 8) return seedBytes; } catch (IOException ignore) { } } new java.security.SecureRandom().nextBytes(seedBytes); return seedBytes; }

So on platforms lacking "/dev/urandom" special file (Windows only?), the fall-back would be to use SecureRandom.nextBytes() whatever default SecureRandom PRNG is configured to be on such platform. This might be a user-supplied SecureRandom PRNG. The user might want it's own default SecureRandom but not use it for TLR's seed computation (unless requested by "java.util.secureRandomSeed" system property).

I would rather use SecureRandom.generateSeed() instance method instead of SecureRandom.nextBytes(). Why? Because every SecureRandom instance has to initialize it's seed 1st before getBytes() can provide the next random bytes from the PRNG. Since we only need the 1st 8 bytes from the SecureRandom instance to initialize TLR's seeder, we might as well directly call the SecureRandom.generateSeed() method. That's one reason. The other is the peculiar initialization of default SecureRandom algorithm on Windows (see below)...

Even if user does not provide it's own default SecureRandom PRNG, there are basically two algorithms that are used by default on OpenJDK:

On Solaris/Linux/Mac/AIX:

the "NativePRNG" algorithm from "SUN" provider (implemented by sun.security.provider.NativePRNG) which uses /dev/random for getBytes() and /dev/urandom (or whatever is configured with "java.security.egd" or "securerandom.source" system properties) for generateSeed(), and

On Windows:

the "SHA1PRNG" algorithm from "SUN" provider (implemented by sun.security.provider.SecureRandom) which uses SHA1 message digest for generating random numbers with seed computed by gathering system entropy...

The most problematic one is the default on Windows platform (the platform that does not have the "/dev/urandom" special file and would be used as a fall-back by your proposal) - sun.security.provider.SecureRandom. This one seeds itself by constructing an instance of itself with the result returned from SeedGenerator.getSystemEntropy() method. This method, among other things, uses networking code to gather system entropy:

                     ...
                     md.update

(InetAddress.getLocalHost().toString().getBytes()); ...

This is problematic since it not only initializes NameService providers but also uses them to resolve local host name. This can block for several seconds on unusual configurations and was the main motivation to replace similar code in TLR with code that uses NetworkInterface.getHardwareAddress() instead. Using SecureRandom.generateSeed() instead of SecureRandom.getBytes() does not invoke SeedGenerator.getSystemEntropy(), but uses just SeedGenerator.generateSeed(), which by default on Windows uses ThreadedSeedGenerator.getSeedBytes()...

I showed how we could suppress NameService providers initialization while still using NetworkInterface.getHardwareAddress() for TLR's seeder initialization. If that's not enough and we would like to get-away without using networking code at all, then I propose the following:

 private static byte[] initialSeedBytes() {

     String secureRandomSeed =

java.security.AccessController.doPrivileged( new sun.security.action.GetPropertyAction( "java.util.secureRandomSeed", "")) .toLowerCase(Locale.ROOT);

     String osName = java.security.AccessController.doPrivileged(
         new sun.security.action.GetPropertyAction(
             "os.name"))
         .toLowerCase(Locale.ROOT);

     if (!secureRandomSeed.equals("true") &&

!secureRandomSeed.equals("blocking")) { secureRandomSeed = "nonblocking"; // default }

     SecureRandom srnd = null;

     if (secureRandomSeed.equals("nonblocking")) { // the default
         try {
             if (osName.startsWith("windows")) {
                 // native implementation using MSCAPI implemented by
                 // sun.security.mscapi.PRNG
                 srnd = SecureRandom.getInstance("Windows-PRNG",

"SunMSCAPI"); } else { // Solaris/Linux/Mac/AIX // a non-blocking native implementation using /dev/urandom for both // generateSeed() and nextBytes() implemented by // sun.security.provider.NativePRNG$NonBlocking srnd = SecureRandom.getInstance("NativePRNGNonBlocking", "SUN"); } } catch (NoSuchProviderException | NoSuchAlgorithmException ignore) {} } else if (secureRandomSeed.equals("blocking")) { try { if (osName.startsWith("windows")) { // native implementation using MSCAPI implemented by // sun.security.mscapi.PRNG srnd = SecureRandom.getInstance("Windows-PRNG", "SunMSCAPI"); } else { // Solaris/Linux/Mac/AIX // a blocking native implementation using /dev/random for both // generateSeed() and nextBytes() implemented by // sun.security.provider.NativePRNG$Blocking srnd = SecureRandom.getInstance("NativePRNGBlocking", "SUN"); } } catch (NoSuchProviderException | NoSuchAlgorithmException ignore) {} } else { assert secureRandomSeed.equals("true"); }

     if (srnd == null) { // fall back to default SecureRandom

algorithm / provider srnd = new SecureRandom(); }

     return srnd.generateSeed(8);
 }

By default (or when "java.util.secureRandomSeed" is set to "nonblocking" or unrecognized value) this would use /dev/urandom on UNIX-es and MSCAPI on Windows. More entropy for TLR's initial seeder could be provided on UNIX-es by risking some blocking with "java.util.secureRandomSeed" set to "blocking". This would still be independend of user's choice of default SecureRandom provider. The backward-compatible ("java.util.secureRandomSeed" set to "true") or fall-back would be to use default SecureRandom algorithm / provider.

Regards, Peter

On Sat, Jun 21, 2014 at 9:05 PM, Martin Buchholz <martinrb at google.com> wrote: While looking at NativePRNG, I filed https://bugs.openjdk.java.net/browse/JDK-8047769 SecureRandom should be more frugal with file descriptors If I run this java program on Linux public class SecureRandoms { public static void main(String[] args) throws Throwable { new java.security.SecureRandom(); } } it creates 6 file descriptors for /dev/random and /dev/urandom, as shown by: strace -q -ff -e open java SecureRandoms |& grep /dev/ [pid 20769] open("/dev/random", ORDONLY) = 5 [pid 20769] open("/dev/urandom", ORDONLY) = 6 [pid 20769] open("/dev/random", ORDONLY) = 7 [pid 20769] open("/dev/random", ORDONLY) = 8 [pid 20769] open("/dev/urandom", ORDONLY) = 9 [pid 20769] open("/dev/urandom", ORDONLY) = 10 Looking at jdk/src/solaris/classes/sun/security/provider/NativePRNG.java it looks like 2 file descriptors are created for every variant of NativePRNG, whether or not they are ever used. Which is wasteful. In fact, you only ever need at most two file descriptors, one for /dev/random and one for /dev/urandom. Further, it would be nice if the file descriptors were closed when idle and lazily re-created. Especially /dev/random should typically be used at startup and never thereafter.

On Fri, Jun 20, 2014 at 7:59 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote: On 20/06/2014 15:02, Peter Levart wrote:

And, as Martin pointed out, it seems to be used for tests that exercise particular responses from NameService API to test the behaviour of JDK classes. It would be a shame for those tests to go away. We've been talking about removing it for many years because it has been so troublesome. If we really need to having something for testing then I don't think it needs to be general purpose, we can get right of the lookup at least. -Alan.

More information about the security-dev mailing list