bug#10355: Add an option to {md5,sha*} to ignore directories (original) (raw)

[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From: Gilles Espinasse
Subject: bug#10355: Add an option to {md5,sha*} to ignore directories
Date: Fri, 23 Dec 2011 14:45:10 +0100

I was using a way to check md5sum on a lot of file using for myfile in cat <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mrow><mi>A</mi><mi>L</mi><mi>L</mi><mi>F</mi><mi>I</mi><mi>L</mi><mi>E</mi><mi>S</mi></mrow><mi mathvariant="normal">‘</mi><mo separator="true">;</mo><mi>d</mi><mi>o</mi><mi>i</mi><mi>f</mi><mo stretchy="false">[</mo><mo>−</mo><mi>f</mi><mi mathvariant="normal">/</mi></mrow><annotation encoding="application/x-tex">{ALLFILES}; do if [ -f /ALLFILES‘;doif[−f/{myfile} ]; then md5sum /$myfile >> $ALLFILES}.md5; fi; done

But this is slow, comparing with xargs md5sum way. time (for myfile in cat <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mrow><mi>A</mi><mi>L</mi><mi>L</mi><mi>F</mi><mi>I</mi><mi>L</mi><mi>E</mi><mi>S</mi></mrow><mi mathvariant="normal">‘</mi><mo separator="true">;</mo><mi>d</mi><mi>o</mi><mi>i</mi><mi>f</mi><mo stretchy="false">[</mo><mo>−</mo><mi>f</mi><mi mathvariant="normal">/</mi></mrow><annotation encoding="application/x-tex">{ALLFILES}; do if [ -f /ALLFILES‘;doif[−f/{myfile} ]; then md5sum /$myfile >> ${ALLFILES}.md5; fi; done)

real 0m26.907s user 0m40.019s sys 0m10.253s

This is faster using xargs md5sum. time (sed -e '/./$/d' -e 's|^.|/&|g' ${ALLFILES} | xargs md5sum

${ALLFILES}.md5) md5sum: /etc/ipsec.d/cacerts: Is a directory md5sum: /etc/ipsec.d/certs: Is a directory md5sum: /etc/ipsec.d/crls: Is a directory md5sum: /etc/ppp/chap-secrets: No such file or directory md5sum: /etc/ppp/pap-secrets: No such file or directory md5sum: /etc/squid/squid.conf: No such file or directory

real 0m1.176s user 0m0.780s sys 0m0.400s

That run mostly 30 times faster. In the above example, I already skipped most of the directories in the list, removing lines that end with / but not all directories in my list match on that condition.

So the fast solution emit errors and end with status 123. I know I could hide error messages and status error but that start to be ugly. sed -e'/./$/d' -e 's|^.|/&|g' ALLFILES∣xargsmd5sum>{ALLFILES} | xargs md5sum > ALLFILES∣xargsmd5sum>{ALLFILES}.md5 2>/dev/null || test $? -eq 123

Would it not be great to support an option in {md5,sha*} to ignore directory error? I may even be able to produce a patch if there is a real interest.

Gilles