RFR: race with nested repos in hgforest.sh (original) (raw)

Fredrik Öhrström oehrstroem at gmail.com
Tue Feb 5 06:38:04 UTC 2013

Fixing get_source.sh problem:

We can hardcode pre-creation of the closed repo locations as mkdir -p calls. We can do the openjdk clone first (in parllell) and when that is finished do the closedjdk clone. We can drop parallell clone altogether because it does not give the speedup we are looking for? We could actually fix how the SCM handles the source code? (Yes, I know that I am reiterating my statement from 2011 here, but I find it incomprehensible that Java is stored in an SCM, in such a way that you cannot reliably bisect to a faulty commit, nor in fact extract any old code from the SCM, in a state that is guaranteed to work or even compile. Johan Walles and I developed a reasonably non-intrusive solution for this where get_source.sh is not needed. So, fixing get_source.sh is clearly in an uphill battle.)

//Fredrik

2013/2/5 David Holmes <david.holmes at oracle.com>:

Chris,

When these failures occur does the failure get reflected in an error exit code? I'm seeing hudson builds merilly buildinh OpenJDK instead of Oracle JDK because the closed repos were skipped due to this bug :( David

On 2/02/2013 12:40 AM, Chris Hegarty wrote:

[ to build-dev and core-libs-dev, expect reviewer from either, but will integrate through jdk8/tl ] This issue is mainly of interest to Oracle engineers, but it effects the public hgforest script. When hgforest.sh is run with an addition argument to specify a closed server, there is a problem/race between the creation of the directories to hold nested repositories and the clone itself. These directories need to be created before the clone command is executed, otherwise it will fail, as below. The trivial fix is to back off these nested repos until their containing directory exists. Webrev: http://cr.openjdk.java.net/~chegar/hgforestnestedRepos/webrev/ sh ./getsource.sh http://xxx.yyy.oracle.com | & tee clone.log # Repositories: corba jaxp jaxws langtools jdk hotspot jdk/src/closed jdk/make/closed jdk/test/closed hotspot/make/closed hotspot/src/closed hotspot/test/closed deploy install sponsors pubs Waiting 5 secs before spawning next background command. Waiting 5 secs before spawning next background command. Waiting 5 secs before spawning next background command. jdk/src/closed: /java/devtools/sparc/mercurial/0.9.5/bin/python -u /java/devtools/sparc/mercurial/latest/bin/hg clone http://xxx.yyy.oracle.com/jdk8/tl/jdk/src/closed jdk/src/closed jdk/src/closed: abort: No such file or directory: jdk/src/closed jdk/make/closed: /java/devtools/sparc/mercurial/0.9.5/bin/python -u /java/devtools/sparc/mercurial/latest/bin/hg clone http://xxx.yyy.oracle.com/jdk8/tl/jdk/make/closed jdk/make/closed jdk/make/closed: abort: No such file or directory: jdk/make/closed Waiting 5 secs before spawning next background command. ....

-Chris.

More information about the core-libs-dev mailing list