RFR: race with nested repos in hgforest.sh (original) (raw)

Chris Hegarty chris.hegarty at oracle.com
Fri Feb 1 23:06:32 UTC 2013

On 1 Feb 2013, at 22:37, David Holmes <david.holmes at oracle.com> wrote:

Hi Chris,

On 2/02/2013 12:40 AM, Chris Hegarty wrote: [ to build-dev and core-libs-dev, expect reviewer from either, but will integrate through jdk8/tl ]

This issue is mainly of interest to Oracle engineers, but it effects the public hgforest script. When hgforest.sh is run with an addition argument to specify a closed server, there is a problem/race between the creation of the directories to hold nested repositories and the clone itself. These directories need to be created before the clone command is executed, otherwise it will fail, as below. I think I reported this myself just last week - probably internally to build or build-infra.

Ah, I missed that.

The weird thing is that based on the list of repos it shouldn't be grabbing the closed one yet.

In my remote/slow environment I end up with all clones running concurrently.

The trivial fix is to back off these nested repos until their containing directory exists. That seems to assume/require that the nested repo will always be later in the list. They happen to be but ....

No so. It will just wait for the process to clone the outer repo to start.

Workaround is to run getsource.sh twice first without the "extra base" then with it.

Good suggestion.

-Chris

David -----

Webrev: http://cr.openjdk.java.net/~chegar/hgforestnestedRepos/webrev/ sh ./getsource.sh http://xxx.yyy.oracle.com | & tee clone.log # Repositories: corba jaxp jaxws langtools jdk hotspot jdk/src/closed jdk/make/closed jdk/test/closed hotspot/make/closed hotspot/src/closed hotspot/test/closed deploy install sponsors pubs Waiting 5 secs before spawning next background command. Waiting 5 secs before spawning next background command. Waiting 5 secs before spawning next background command. jdk/src/closed: /java/devtools/sparc/mercurial/0.9.5/bin/python -u /java/devtools/sparc/mercurial/latest/bin/hg clone http://xxx.yyy.oracle.com/jdk8/tl/jdk/src/closed jdk/src/closed jdk/src/closed: abort: No such file or directory: jdk/src/closed jdk/make/closed: /java/devtools/sparc/mercurial/0.9.5/bin/python -u /java/devtools/sparc/mercurial/latest/bin/hg clone http://xxx.yyy.oracle.com/jdk8/tl/jdk/make/closed jdk/make/closed jdk/make/closed: abort: No such file or directory: jdk/make/closed Waiting 5 secs before spawning next background command. ....

-Chris.

More information about the core-libs-dev mailing list