hg clone is unbelievably slow (original) (raw)

Mario Torre neugens.limasoftware at gmail.com
Tue Feb 6 15🔞42 UTC 2018

Nice, I didn't know about that extension, but seems exactly what we are searching for.

What would it take to add this support to the repos? From the linked documentation I understand this is transparent for clients, so we should see immediate improvements just by enabling it on the servers.

Cheers, Mario

2018-02-06 16:04 GMT+01:00 Erik Helin <erik.helin at oracle.com>:

There is no need to re-invent the wheel :)

Mercurial (hg) has had support for clonebundles for a some time by now, see [0] for details. If the servers where configured to use clonebundles then users can continue to just use hg clone but get the performance of downloading compressed bundles. clonebundles can be stored on various CDN servers, then hg clone will download the closest bundle, unpack it and eventually do hg pull --update to get the latest commits (that might not be present in the bundle). For more information about clonebundles, just run hg help -e clonebundles. Thanks, Erik [0]: https://www.mercurial-scm.org/wiki/ClonebundlesExtension

On 02/06/2018 03:42 PM, Mario Torre wrote:

I think it should be a fully functional mercurial repository once is unpacked. What I have in mind is something that support this workflow (or similar): $ hg clone -r 12345 http://hg.openjdk.java.net/jdk10/client-gz $ tar xvfz client-gz.tgz && cd client-gz (do whatever, then some time later) $ hg log || hg pull -u || commit etc... This should be just a few lines of scripting away. Cheers, Mario

2018-02-06 15:35 GMT+01:00 Thomas Stüfe <thomas.stuefe at gmail.com>: Side question, we would only need to tar the .hg folder, no? And revive the workspace with hg update? On Tue, Feb 6, 2018 at 3:10 PM, Mario Torre <neugens.limasoftware at gmail.com> wrote:

I think it should probably be done on demand, it may be more expensive to keep a commit based tgz otherwise. In fact the most awesome thing would be to have a special repository that we clone but gives us back only the tgz to unpack locally, with all the history and correct hg paths. Cheers, Mario 2018-02-06 15:04 GMT+01:00 Remi Forax <forax at univ-mlv.fr>: +1 creating a tgz at each commit doesn't seems that difficult ... Rémi ----- Mail original ----- De: "Andrew Haley" <aph at redhat.com> À: "jdk-dev" <jdk-dev at openjdk.java.net> Envoyé: Mardi 6 Février 2018 11:50:07 Objet: hg clone is unbelievably slow

Half an hour or more here. AFAIK the problem is due to the inefficiency of Mercurial itself and the hg protocol. Aleksey Shipilev has done an experiment whereby trees are regularly cloned and compressed tarballs created; these can be downloaded in a couple of minutes. But really we don't want to depend on the largesse of one developer: if we could download the OpenJDK trees directly by means of wget (or something similar) we would reduce the load on the servers and reduce the time taken to download as well. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. <https://www.redhat.com> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

-- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens Proud GNU Classpath developer: http://www.classpath.org/ OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/

-- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF

Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens Proud GNU Classpath developer: http://www.classpath.org/ OpenJDK: http://openjdk.java.net/projects/caciocavallo/

Please, support open standards: http://endsoftpatents.org/

More information about the jdk-dev mailing list