Is anyone able to build on Win 7 (original) (raw)

Kelly O'Hair kelly.ohair at oracle.com
Tue Feb 14 19:07:01 UTC 2012

Fantastic information set. Many thanks for all this digging.

I suspect, that our build infrastructure work may help, in that if we get rid of the nested makes, then I can only assume we will be doing fewer stat() calls, but I think we still have a problem.

A year or so ago, I managed to reduce the fork/exec count by 50% thinking it would improve windows build times, it did very little, I was disappointed, and assumed the bigger spike had to be related to I/O. So I tried a fast SSD on a Windows machine, which sure made it quiet, but also didn't help much.

So I'm with you on the stat() theory, makes a great deal of sense.

-kto

On Feb 14, 2012, at 6:59 AM, Volker Simonis wrote:

On Tue, Feb 14, 2012 at 12:43 PM, Fredrik Öhrström <fredrik.ohrstrom at oracle.com> wrote:

2012-02-14 12:29, Volker Simonis skrev:

To cut a long story short: - disabling "on access" scanning of *.{java,c,cpp,h,hpp} seems to resolve the file io problems (permission denied, access denied) - disabling ASLR seems to resolve the "fork" problems

Great work! Do you know if disabling ASLR affects the fork performance on 64 bit windows? No, definitely not. The build time on my DualCore i7 @ 2.7 GHz notebook with 8GB Ram is constantly 1h43min for a full JDK8 opt build. Now that that the build seems stable I want to compare it with a Linux build in a VirtualBox VM on the same host. I'll let you know the results once I'm done, but I don't think that "fork" is the big problem - at least not the only one. But once you asked here's my main suspect since long time (you asked so you have to read the conspiracy:) stat (or lstat/fstat/stat64 ...) --------------------------------- I'm quite sure that the poor implementation of stat in Cygwin and its usage in make is one of the main reasons why the build Windows build is so terrible slow in Windows compared to Linux. I first realized this years ago when I was first building Windows inside a VMWare image with the sources on network shares. The build times were excessively long and it was not the compile times because SMB shares do a quite efficient caching - it was the times which make needed in order to check the build dependencies. The build times could be improved by factors if the sources were moved to a local disc. But even locallly, I think that the stat calls for large dependency lists as we have them in the OpenJDK build may still be one of the causes for the slow build. Running make inside strace will reveal the shear number of these calls and there is evidence that the stat performance is really poor in Cygwin. I'll attach some unsorted links to some of the resources which discuss this issue just in case somebody wants to deep dive into this topic. I'm definitely not a Windows/Cygwin expert and I can't promise I'll have the time to solve it 'real soon' but I'll definitely stick to this topic because it is a real PITA (especially if you want to change your build from MKS to Cygwin the developers who will first love you because freed them from of the MKS licensing night mare will very soon kill you because of the build times:) "performance issue with cgywin make" http://www.mail-archive.com/make-w32@gnu.org/msg01353.html - mailing list thread which compares the bad performance of Cygwin make with MS nmake and discusses how some of the problems have been solved in CMake by using direct windows sys-calls instead of stat. "make 3.82 performing more stat() calls than 3.81" http://lists.gnu.org/archive/html/bug-make/2011-09/msg00025.html - mailing list thread discussing a bug in make 3.82 which leads to even more calls to stat (and has been fixed already in the head revision of make) "Cygwin Performance and stat()" http://omgili.com/mailinglist/cygwin/cygwin/com/efe8a37b2e4466daa7b6eb1aa610c3d7squirrelwwwwebmailwingertorg.html - very long thrad, very ‘flamy’, no outcome, some interesting infos nevertheless "cygwin: Use native Win32 API for stat" http://marc.info/?l=git&m=122278284210941 - partial patch for the problem by implementing a stripped down version of stat which does just enough... "NutFastStat(), NutFastStat64()" http://www.mkssoftware.com/docs/man3/NutFastStat.3.asp - MKSs limited but faster version of stat (not sure if MKS make uses this, but that may be an explanation why builds under MKS are reported to be considerably faster compared to Cygwin builds "Managing Projects with GNU Make, 3rd Edition, The Power of GNU make for Building Anything, By Robert Mecklenburg" http://shop.oreilly.com/product/9780596006105.do - Chapter 10: "Improving the Performance of make" freely available as PDF download: http://reilly.com/catalog/make3/book/ch10.pdf "Compile time Local Cygwin vs. VMware session on same system" http://cygwin.com/ml/cygwin/2008-10/threads.html#00415 - mail thread suggesting the usage of ‘dash’ instead of ‘bash’ for a 10% performance improvement //Fredrik

More information about the build-dev mailing list