Kaveh R. Ghazi - Re: svn speed traversing slow filesystems (original) (raw)

This is the mail archive of the gcc@gcc.gnu.orgmailing list for the GCC project.

• From: "Kaveh R. Ghazi"
• To: richard dot guenther at gmail dot com
• Cc: brane at xbc dot nu, dberlin at dberlin dot org, gcc at gcc dot gnu dot org
• Date: Fri, 25 Nov 2005 12:59:46 -0500 (EST)
• Subject: Re: svn speed traversing slow filesystems
• References: <200511191514.jAJFEIU4008764@caipclassic.rutgers.edu> <1132427291.22177.39.camel@dyn9002219071> <200511220320.jAM3KQe7024493@caipclassic.rutgers.edu> <438669A8.4030700@xbc.nu> <84fc9c000511250145p69de72dav16587078b2f217b5@mail.gmail.com>

Actually, i just removed the need for most stat calls during update in 1.4.

Thanks Dan, that's great, but for the remaining i/o calls, it really does matter if you use long/paths/with/lots/of/slashes rather than chdir and ./filenames instead. I believe other recursive gnu utils besides gnufind like "rm -r" or "mkdir -p" etc were modified to use the chdir mechanism also because the benefit was so great.

Yes that's fine, but we can't do this in SVN. We do the real work in libraries that are supposed to be thread-safe. The cwd is per-process on POSIX systems, as far as I know.

Yes, for this problem there may exist openat and friends. Of course you'd need to check for the availability of them and provide a fallback. Richard.

Certainly removing the need for some of these calls like Dan did is the best option. But for the remaining open, lstat and/or unlink calls, a faster mechanism than long/directory/paths/in/every/syscall is preferred for speed. As Richard mentioned the openat() et al. interfaces from Solaris serve this purpose in multi-threaded situations if that is a design requirement for SVN.

And AFAICT, these APIs are being considered (or have been implemented already) for linux as well. http://lkml.org/lkml/2005/11/9/290

But while I'd like to see openat etc adopted in SVN, unfortunately that doesn't help systems without those calls like older solaris2, e.g. solaris2.7.

    --Kaveh

-- Kaveh R. Ghazi ghazi@caip.rutgers.edu

• Follow-Ups:
  • Re: svn speed traversing slow filesystems
    * From: Daniel Berlin
• References:
  • svn speed traversing slow filesystems
    * From: Kaveh R. Ghazi
  • Re: svn speed traversing slow filesystems
    * From: Daniel Berlin
  • Re: svn speed traversing slow filesystems
    * From: Kaveh R. Ghazi
  • Re: svn speed traversing slow filesystems
    * From: Branko Äibej
  • Re: svn speed traversing slow filesystems
    * From: Richard Guenther