Re: Shred: Add recursion operations [PATCH] (original) (raw)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From: Amr Ali
Subject: Re: Shred: Add recursion operations [PATCH]
Date: Sat, 10 Dec 2011 01:27:47 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111124 Thunderbird/8.0

On 12/09/2011 08:24 PM, Eric Blake wrote:

On 12/09/2011 10:52 AM, Amr Ali wrote: > Hi Eric,

Please don't top-post on technical lists.

> > I completely agree about the philosophy. But ... > > 1) I've seen a comment at shred.c:26 that indicates adding -R/-r/--recursive > as > a TODO.

Hmm, we probably ought to nuke that comment instead, unless there are other implementations of shred that already provide -r.

> 2) Adding recursion to shred is not bloat, it is in fact enabling shred to do > it's job, and to do it well. > 3) On the command line, the overhead of first, having to type down all of the > find parameters, and second, the overhead of executing shred, quite > literally, > on every single file is a waste of both time and resources. Adding a simple > -r > flag, however, enhances usability dramatically compared to the find(1) > solution.

You misunderstand 'find -exec shred {} +', which executes as few shreds as necessary to fit within command-line limits. Shred operates on multiple files per pass, and I don't see how teaching shred the ability to recurse adds any noticeable speed over using find to do the recursion. The bulk of your time is spent not in recursion, but in shredding, and using -exec + to avoid fork()ing one shred per file should put the overhead in the noise. Which leaves shred to do one thing well - shredding individual files.

You do have a point. Indeed the quantum taken by shred processing files far out weights that of the fork-per-directory CPU utilization. Despite that this can be argued as it largely depends on the directory structure depth and spread, but it's arguable too that the likelihood of a complex directory structure layout lies in the 10th percentile.

> 4) It became more than obvious that a lot of users do want such ability if > you > will, given the amount of postings online that talks/addresses the shred > recursion issue.

You didn't post any such links to support your claim. Meanwhile, I can point to the coreutils FAQ: https://www.gnu.org/software/coreutils/faq/#Why-don0027t-the-utilities-have-built-in-directory-recursion003f and state definitively that I have used that FAQ less than 5% of the time where the FAQ already provided my answer (my most frequent uses are about 'sort not working' and 'date not working').

As for the links, a simple Google query like "shred+no+recurse" should show that people were either disappointed or looking for a workaround/alternative that works recursively on a directory. e.g. ..

http://www.anchor.com.au/hosting/dedicated/WipingDeletingAndShreddingCustomerDataBeforeReleasingHardware http://www.linuxforums.org/forum/miscellaneous/46693-how-shred-entire-directory-tree.html http://ubuntuforums.org/archive/index.php/t-1082954.html

These mainly show that there is a "usability" issue since 2005 and probably a little earlier than that, and it still remain!

Point really is, I'd naturally expect shred to have a recursive mode, why go against nature? some users always appreciate a consistent way of doing things.

Apparently this is a far more cultured and dense mailing list than expected. Silly me, apologies for not taking the necessary steps before going into PATCH mode. I should have read that before having you to explain it again.

> 5) If find truly is made to enable recursion for all programs, why > recursion > is available for chmod or chgrp, or the plethora of other tools?

'chmod' and 'chgrp' have recursion because they affect the recursion itself. I can dream up recursion situations where find cannot operate unless you change permissions as you go, but the only way to change permissions as you go is to do one chmod invocation per name, which is inefficient.

'du' and 'ls' have recursion because they output a total - if you were to break things up by command-line length limits, you would have multiple totals.

'rm' has recursion even though 'find -delete' does the same, because 'find -delete' is a GNU extension not required by the standards.

The goal is that if 'find -exec +' can do your task, because no permissions are being modified by your task and your task has no output that would be adversely broken up by multiple max-command-line-length invocations, then your task doesn't need builtin recursion. 'shred' fits this goal, 'chmod', 'ls', and 'rm' do not.

The act of outputting a total, modifying permissions, or standards that focus on low level technical particularities is irrelevant. It's the process of working on multiple items to arrive at a quantifiable result that illustrates the purposefulness of recursion.

Purpose is what drives my modifications towards shred. If you do work on multiple files (and that does have a high probability), with a simple command line regular expression (e.g. './*') you are in fact recursing over files of the first level of depth of the current directory node, which conditionally, having it recurse on all directory nodes on multiple levels is only an expected behavior or for the very least having the ability to switch between modes is always thoughtful. (If you have a robot arm, and it moves only in 2D-space, moving in 3D-space is not bloat or out of line, it's only an expected behavior, if unavailable then it is an expected upgrade)

Adding recursion to coreutils is NOT done lightly. You can still manage to convince me that 'shred' fits one of the exceptions I listed above for needing built-in recursion, but the burden is on you to do so, and so far you have not convinced me. Before taking a dive at coreutils source code, which was my very first time, I always enjoyed just using the functionalities it does provide, I knew how they do work at very low level details, but this is not the drive behind using them. It's simply the desired quantifiable result. The low level technical particularities are almost always expected to be abstracted away from usage.

Finally, if you DO manage to convince me, then we would need copyright assignment before your patch could be applied.

Should I use assign.change.manual? if so which email I should be sending this to, after signing it? That of course if I did convince you.

-- Amr Ali

Attachment: signature.asc
Description: OpenPGP digital signature