EmacsWiki: Dynamic Isearch Filtering (original) (raw)

This page is about using Isearch+, that is, library isearch+.el, to dynamically add and remove any number of Isearch filter predicates (aka search filters) while searching incrementally. This Isearch+ feature is available starting with Emacs 24.4.

The predicate that is the value of ‘isearch-filter-predicate’ is advised by additional predicates that you add, creating a complex suite of predicates that act together.

This lets you search only given zones of text, where those zones can be defined on the fly in complex ways that are easy to specify.

Note: This is different from the feature provided by companion Isearch+ library isearch-prop.el, which also lets you constrain search to zones of text. When you use that library you first define the zones, and then search is limited to them. With dynamic filtering, search is not limited to certain zones ahead of time. Instead, after a search hit is found the filters are applied to see if it should be retained as a search hit. This is a posteriori filtering, whereas isearch-prop.el uses a priori filtering. Each has its advantages.

You can use the dynamic filtering described here at the same time as the zone-constraining provided by isearch-prop.el - you can use them together.

Isearch Filter Predicates

An Isearch filter predicate is a function that accepts two buffer positions, BEG and END, as its first two arguments. These values are the beginning and ending positions of a search hit. If the return value of the function is ‘nil’ then the search hit is excluded from searching; otherwise it is included.

The value of standard variable (but not a user option) ‘isearch-filter-predicate’ is the filter predicate used by Isearch. By default, the value is predicate ‘isearch-filter-visible’, which returns non-‘nil’ for any search hit that is visible (not rendered invisible by a text property, overlay property, etc.)

Using Filter Predicates Easily

If you search the Emacs Lisp source code, you will find only two uses, so far, of variable ‘isearch-filter-predicate’, even though such filtering has been around since Emacs 23. It’s hardly ever used. Why?

Because it’s not so easy to use, out of the box. And it’s not thought of as a way to refine searches, but rather as a way to wall off certain areas from searching.

Yes, those are in fact the same thing, but I don’t think people think this way . . . because Isearch does not make it particularly easy to use filters. Isearch+ tries to do that, to let you refine searches by adding filters incrementally.

The idea is simple: Isearch+ defines some keys that prompt you for a filter. You can enter any filter predicates at the prompts. There are also some predefined predicates that you can choose from, using completion. You can combine predicates using AND, OR, and NOT.

A Filter Predicate Can Help You Search

A filter predicate does essentially the same thing as the search pattern that you type at the Isearch prompt. Each restricts the search space (the buffer text) to certain zones: those that satisfy the predicate and those that match the search pattern.

But a predicate can be much more general than is the predefined pattern-matching provided by Emacs Isearch.

Suppose that you want to find lines of text that contain ‘cat’, ‘dog’, and ‘turtle’. There is no simple search pattern that lets you do this. A regexp would need to explicitly express each possible order, and there are 6 of them - not so simple.

But a predicate can just check each line for ‘cat’ AND check for ‘dog’ AND check for ‘turtle’. It is usually much easier to combine simple patterns than it is to come up with a complex pattern that does the same thing. And the way to combine patterns in Emacs Isearch is to use one or more filter predicates.

A Filter Predicate Can Perform Side Effects

A filter predicate can even perform side effects, if you like. Only the return value is used by Isearch. For example, if you wanted to more easily see the cursor position each time search stops at a search hit, you could use something like this as a filter predicate. (This requires library crosshairs.el, which highlights the current column and line using crosshairs.)

(lambda (beg end)
   (save-excursion (goto-char end)) 
   
   (unless isearchp-in-lazy-highlight-update-p (crosshairs))
   t)  

The side-effect-producing call to function ‘crosshairs’ is guarded by variable ‘isearchp-in-lazy-highlight-update-p’ here, so that it is invoked only when the cursor is moved to a search hit, not also when lazy highlighting is performed. (Filtering applies also to lazy highlighting: it filters out search hits that are not being used. But in this case no real filtering is done, and there is no need to show crosshairs moving across the buffer during lazy highlighting.)

(You can choose that crosshairs-showing filter predicate by the name ‘crosshairs’ when prompted for a predicate. It corresponds to predicate ‘isearchp-show-hit-w-crosshairs’.)

What You Can Do

Here are some of the things you can do using dynamic filtering:

• Search for a number of things at the same time, in any order. Example: ‘cat’, ‘dog’, and ‘turtle’, in any order, within the same context (e.g. within a line).
• Search in a progressive way, using multiple simple search patterns instead of a single complicated pattern. The previous example is also an example of this:
  1. Search for lines: ` C-M-s .+’.
  2. Refine that search to lines that contain a match for ‘cat’: ` C-z . cat’.
  3. Do the same for ‘dog’, and then ‘turtle’ (` C-z . dog’, then ` C-z . turtle’).
• Search for stuff within contexts that correspond to a type of THING. You can add to the list of context types, using option ‘isearchp-filter-predicates-alist’. When prompted to enter a filter predicate you can use completion against the predicates named in that list (e.g. ‘in-comment’, ‘in-file-name’). Examples of context type:
  • color name or #RGB
  • comment
  • defun (any top-level sexp)
  • sexp
  • symbol
  • Lisp variable
  • email address
  • file name
  • URL
  • file name or URL
  • number, decimal number, or hex number
  • string
  • string or comment
  • line
  • sentence
  • page
  • word
• Search for something that is near something else – within a given number of characters, words, lists, or sentences (you can add to this list of distance units, using option ‘isearchp-movement-unit-alist’). You specify the nearness. You can also constrain the nearby pattern to be only before or only after the search hit.
• Search for stuff in the union of given contexts. Example: search for text that is near ‘cat’ or near ‘dog’.
• Search for stuff outside contexts: Use the complement of any set of contexts as the search space. Example: search for text that is not near ‘cat’ and not near ‘dog’.

Filtering Commands Available During Search

The following commands are available during Isearch. They are all on prefix key ‘C-z’, by default. They are on prefix keymap ‘isearchp-filter-map’, which you can bind to any key in ‘isearch-mode-map’. If you forget a ‘C-z’ key, you can use ‘C-z C-h’ while searching to show them all.

• **` C-z &**’ (‘isearchp-add-filter-predicate’) adds a filter predicate, _AND_-ing it as an additional ` :after-while’ filter.

• **` C-z %**’ (‘isearchp-add-regexp-filter-predicate’) adds a filter predicate that requires search hits to match a given regexp.

• *` C-z .**’ (‘isearchp-add-inline-regexp-filter-predicate’) is really just ` C-z %’, but ` `.’ is added to each side of the regexp you enter. You can use this multiple times when regexp searching for full lines with \ .+’, to find the lines that contain multiple regexp matches in any order.

• **` C-z ||**’ (‘isearchp-or-filter-predicate’) adds a filter predicate, _OR_-ing it as an additional ` :before-until’ filter.

• **` C-z |1**’ (‘isearchp-or-last-filter’) replaces the last-added filter by its disjunction with another predicate, which you specify.

• **` C-z ~~**’ (‘isearchp-complement-filter’) _complements_ the current filter. It either adds an ` :around’ filter that complements or it removes an existing top-level complementing filter.

• **`C-z ~1**’ (‘isearchp-negate-last-filter’) replaces the last-added filter by its complement.

• ‘C-z -’ (‘isearchp-remove-filter-predicate’) removes a filter predicate that you specify, using completion. The last-added is the default – retrieve it using ‘M-n’.

• **` C-z !**’ (‘isearchp-set-filter-predicate’) _sets_ the overall filter predicate (advised ‘isearch-filter-predicate’) to a single predicate.

• ‘C-z 0’ – zero, not letter ‘O’ – (‘isearchp-reset-filter-predicate’) resets ‘isearch-filter-predicate’ to its original (default) value (which is typically ‘isearch-filter-visible’).

• ‘C-z c’ (‘isearchp-columns’) adds a filter predicate that limits search between two columns (or before/after a column).

• ‘C-z n’ (‘isearchp-defun-filter-predicate’) names the current suite of filter predicates, creating a named predicate that does the same thing. With a prefix arg it can also set or keep it (for this Emacs session) - that is, do what ` C-z !’ or ‘C-z s’ does. You can use that name with ‘C-z -’ to remove that predicate. You can also use it to create a custom Isearch command that uses it for filtering. For example:

     (defun foo ()
       "Isearch with filter predicate `my-filter-pred'."
       (interactive)
       (let ((isearch-filter-predicate  'my-filter-pred))
         (isearch-forward)))

• ‘C-z p’ (‘isearchp-toggle-showing-filter-prompt-prefixes’) toggles option ‘isearchp-show-filter-prompt-prefixes-flag’, which controls whether to show filter prefixes in the Isearch prompt.

• ‘C-z s’ (‘isearchp-keep-filter-predicate’) keeps the current filter-predicate suite for subsequent searches (in this Emacs session only). Unless you do this (and unless auto-keeping is turned on), the next Isearch starts out from scratch, using the default value of ‘isearch-filter-predicate’. (To remove the kept predicate suite, use ‘C-z 0’.)

• ‘C-z S’ (‘isearchp-toggle-auto-keep-filter-predicate’) toggles option ‘isearchp-auto-keep-filter-predicate-flag’, which automatically keeps the current filter-predicate suite, so that it is used for subsequent searches (so no need to use ‘C-z s’). (To remove a kept predicate suite, use ‘C-z 0’.)

• **` C-z ?**’ (‘isearchp-show-filters’) echoes the current suite of filter predicates (advice and original, unadvised predicate).

• **` C-z @**’, **` C-z <**’, and **` C-z >**’ (‘isearchp-near’, ‘isearchp-near-before’, and ‘isearchp-near-after’) constrain searching to be within a given distance of (_near_) another search pattern. For example, you can limit search hits to those whose end (or beginning, if searching backward) is within, say, 4 words of another search pattern. You are prompted for the search pattern for the nearby text, the “near” distance, and the unit of distance measurement (default: characters). You can define the list of acceptable units by customizing option **‘isearchp-movement-unit-alist’**. The default option value includes units character, word, sexp, list, and sentence. You can also use functions ‘isearch-near-predicate’, ‘isearchp-near-before-predicate’, and ‘isearchp-near-after-predicate’ to define your own nearness predicates, which incorporate particular patterns and distances. You can then simply add such a predicate using ` C-z &’ (no prompting for pattern or distance).

Typically you add (` C-z &’, ` C-z %’, etc.) a filter predicate to those already active, or you remove one (‘C-z -’). Adding is implicitly an AND operation: the list of current predicates must all be satisfied. You can also OR a predicate against either the entire _AND_ed list of predicates (` C-z ||’) or against only the last-added one (` C-z |1’). And you can complement either the entire _AND_ed list (` C-z ~~’) or just the last-added predicate (` C-z ~1’).

This _OR_ing and _NOT_ing, together with adding and removing predicates in a given order (implicitly _AND_ing them), gives you complete Boolean combination flexibility.

The list of filter predicates is always a conjunction. But you can use, as any of the conjuncts, a predicate that implements a disjunction or a negation. Or you can replace the entire list by a single predicate that implements a disjunction or a negation.

When you use one of the commands that adds a filter predicate as advice to ‘isearch-filter-predicate’ you can be prompted for two things: (1) a short name for the predicate and (2) text to add to the Isearch prompt as a reminder of filtering. The optional short name is a convenience for referring to the predicate - for adding it again or removing it, for example.

Two user options control this prompting:

• ‘isearchp-prompt-for-filter-name’ says whether to prompt you always, never, or only when the predicate that you provide is not a symbol (it is a lambda form). The last of these is the default behavior. If you are prompted and provide a name, you can use that name with ‘C-z -’ to remove that predicate.
• ‘isearchp-prompt-for-prompt-prefix-flag’ says whether to prompt you for a prefix to add to the Isearch prompt. You are prompted by default, but if you don’t care to see such a prompt prefix and you don’t want to be bothered by it, you can customize this to skip prompting.

In addition, whatever the value of these options, when you add a filter predicate you can override the option values by using a prefix argument. A non-positive prefix arg overrides the option for name prompting, and a non-negative prefix arg overrides the option for prompt-prefix prompting. (So zero, e.g., ‘M-0’, overrides both.)

Option ‘isearchp-show-filter-prompt-prefixes-flag’ controls whether prefixes for filters are added to the Isearch prompt. You can toggle this option during search using ‘C-z p’.

User option ‘isearchp-filter-predicates-alist’ contains filter predicates that are available as completion candidates whenever you are prompted for one. This is an important option. The alist entries can be of several forms, which affect the behavior differently.

In particular, instead of choosing a filter predicate as a completion candidate, you can choose a function that creates and returns a filter predicate, after prompting you for some more information.

This is the case, for example, for function ‘isearchp-near-before-predicate’. It is used in the predefined alist entry **` ("near<..." isearchp-near-before-predicate)**’, which associates the short name ` near<...’, as a completion candidate, with the function.

When you choose this candidate, function ‘isearchp-near-before-predicate’ prompts you for another pattern for Isearch to match, a max number of units of nearness, and which units to measure with. It constructs and returns a predicate that checks those match parameters. As usual, you can be prompted for a short name and an Isearch prompt prefix to associate with the newly defined predicate, so that you can easily choose it again (no prompting).

Similarly, candidate ` not...’ prompts you for a predicate to negate, and candidate ` or...’ prompts you for two predicates to combine using ‘or’.

For the completion candidates that are predefined, this naming convention is used:

• Bracketed names (**` [...]**’) stand for predicates that check that the search hit is (entirely) within something. For example, name ` [;]’ means each search hit must be inside a comment (` ;’ is the EmacsLisp comment-start character), and name ` [defun]’ means each search hit must be inside a defun.
• A **` ~**’ in front of a name means _“not”_. For example, **` ~[;]**’ means the filtered search hits must not be in comments.
• Names that end in **` ...**’ indicate candidates that prompt you for more information. These names represent, not filter predicates, but functions that return filter predicates. For example, ` near<...’ stands for function ‘isearchp-near-before-predicate’ (see above).

Filter predicates that you add dynamically are added as completion candidates for the current Emacs session. If option ‘isearchp-update-filter-predicates-alist-flag’ is non-‘nil’ then they are also added to ‘isearchp-filter-predicates-alist’. That updated option value is NOT SAVED, however. If you want to save your additions to it for future Emacs sessions then use ‘M-x customize-option isearchp-filter-predicates-alist’.

You can use command ‘isearchp-reset-filter-preds-alist’ (not bound) to reset the filter predicates available for completion to those in option ‘isearchp-filter-predicates-alist’. A prefix arg with ‘C-z 0’ also resets this, along with resetting to the unadvised value of ‘isearch-filter-predicate’.

If option ‘isearchp-lazy-dim-filter-failures-flag’ is non-‘nil’ then search hits that are skipped because they are removed by filtering are nevertheless lazy-highlighted, but using a face that dims the background. You can toggle this highlighting of filter-failure search hits using ‘M-s h d’ (command ‘isearchp-toggle-dimming-filter-failures’).

The dimming face for this is hard-coded as having background color #9abfca, unless you also use library isearch-prop.el (recommended). If you use isearch-prop.el then you can control the dimming color using option ‘isearchp-dimming-color’. It specifies a given background color to use always, or it specifies that the current background color is to be dimmed a given amount.

See Also: Isearch+

CategorySearchAndReplace