Home » Understanding the -exec option of `find`

Understanding the -exec option of `find`

Solutons:


This answer comes in the following parts:

  • Basic usage of -exec
  • Using -exec in combination with sh -c
  • Using -exec ... {} +
  • Using -execdir

Basic usage of -exec

The -exec option takes an external utility with optional arguments as its argument and executes it.

If the string {} is present anywhere in the given command, each instance of it will be replaced by the pathname currently being processed (e.g. ./some/path/FILENAME). In most shells, the two characters {} does not need to be quoted.

The command needs to be terminated with a ; for find to know where it ends (as there may be further options afterwards). To protect the ; from the shell, it needs to be quoted as ; or ';', otherwise the shell will see it as the end of the find command.

Example (the at the end of the first two lines are just for line continuations):

find . -type f -name '*.txt'      
   -exec grep -q 'hello' {} ';'   
   -exec cat {} ';'

This will find all regular files (-type f) whose names matches the pattern *.txt in or below the current directory. It will then test whether the string hello occurs in any of the found files using grep -q (which does not produce any output, just an exit status). For those files that contain the string, cat will be executed to output the contents of the file to the terminal.

Each -exec also acts like a “test” on the pathnames found by find, just like -type and -name does. If the command returns a zero exit status (signifying “success”), the next part of the find command is considered, otherwise the find command continues with the next pathname. This is used in the example above to find files that contain the string hello, but to ignore all other files.

The above example illustrates the two most common use cases of -exec:

  1. As a test to further restrict the search.
  2. To perform some kind of action on the found pathname (usually, but not necessarily, at the end of the find command).

Using -exec in combination with sh -c

The command that -exec can execute is limited to an external utility with optional arguments. To use shell built-ins, functions, conditionals, pipelines, redirections etc. directly with -exec is not possible, unless wrapped in something like a sh -c child shell.

If bash features are required, then use bash -c in place of sh -c.

sh -c runs /bin/sh with a script given on the command line, followed by optional command line arguments to that script.

A simple example of using sh -c by itself, without find:

sh -c 'echo  "You gave me $1, thanks!"' sh "apples"

This passes two arguments to the child shell script. These will be placed in $0 and $1 for the script to use.

  1. The string sh. This will be available as $0 inside the script, and if the internal shell outputs an error message, it will prefix it with this string.

  2. The argument apples is available as $1 in the script, and had there been more arguments, then these would have been available as $2, $3 etc. They would also be available in the list "$@" (except for $0 which would not be part of "$@").

This is useful in combination with -exec as it allows us to make arbitrarily complex scripts that acts on the pathnames found by find.

Example: Find all regular files that have a certain filename suffix, and change that filename suffix to some other suffix, where the suffixes are kept in variables:

from=text  #  Find files that have names like something.text
to=txt     #  Change the .text suffix to .txt

find . -type f -name "*.$from" -exec sh -c 'mv "$3" "${3%.$1}.$2"' sh "$from" "$to" {} ';'

Inside the internal script, $1 would be the string text, $2 would be the string txt and $3 would be whatever pathname find has found for us. The parameter expansion ${3%.$1} would take the pathname and remove the suffix .text from it.

Or, using dirname/basename:

find . -type f -name "*.$from" -exec sh -c '
    mv "$3" "$(dirname "$3")/$(basename "$3" ".$1").$2"' sh "$from" "$to" {} ';'

or, with added variables in the internal script:

find . -type f -name "*.$from" -exec sh -c '
    from=$1; to=$2; pathname=$3
    mv "$pathname" "$(dirname "$pathname")/$(basename "$pathname" ".$from").$to"' sh "$from" "$to" {} ';'

Note that in this last variation, the variables from and to in the child shell are distinct from the variables with the same names in the external script.

The above is the correct way of calling an arbitrary complex script from -exec with find. Using find in a loop like

for pathname in $( find ... ); do

is error prone and inelegant (personal opinion). It is splitting filenames on whitespaces, invoking filename globbing, and also forces the shell to expand the complete result of find before even running the first iteration of the loop.

See also:

  • Why is looping over find’s output bad practice?
  • Is it possible to use `find -exec sh -c` safely?

Using -exec ... {} +

The ; at the end may be replaced by +. This causes find to execute the given command with as many arguments (found pathnames) as possible rather than once for each found pathname. The string {} has to occur just before the + for this to work.

find . -type f -name '*.txt' 
   -exec grep -q 'hello' {} ';' 
   -exec cat {} +

Here, find will collect the resulting pathnames and execute cat on as many of them as possible at once.

find . -type f -name "*.txt" 
   -exec grep -q "hello" {} ';' 
   -exec mv -t /tmp/files_with_hello/ {} +

Likewise here, mv will be executed as few times as possible. This last example requires GNU mv from coreutils (which supports the -t option).

Using -exec sh -c ... {} + is also an efficient way to loop over a set of pathnames with an arbitrarily complex script.

The basics is the same as when using -exec sh -c ... {} ';', but the script now takes a much longer list of arguments. These can be looped over by looping over "$@" inside the script.

Our example from the last section that changes filename suffixes:

from=text  #  Find files that have names like something.text
to=txt     #  Change the .text suffix to .txt

find . -type f -name "*.$from" -exec sh -c '
    from=$1; to=$2
    shift 2  # remove the first two arguments from the list
             # because in this case these are *not* pathnames
             # given to us by find
    for pathname do  # or:  for pathname in "$@"; do
        mv "$pathname" "${pathname%.$from}.$to"
    done' sh "$from" "$to" {} +

Using -execdir

There is also -execdir (implemented by most find variants, but not a standard option).

This works like -exec with the difference that the given shell command is executed with the directory of the found pathname as its current working directory and that {} will contain the basename of the found pathname without its path (but GNU find will still prefix the basename with ./, while BSD find or sfind won’t).

Example:

find . -type f -name '*.txt' 
    -execdir mv -- {} 'done-texts/{}.done' ;

This will move each found *.txt-file to a pre-existing done-texts subdirectory in the same directory as where the file was found. The file will also be renamed by adding the suffix .done to it. --, to mark the end of options is needed here in those find implementations that don’t prefix the basename with ./. The quotes around the argument that contains {} not as a whole are needed if your shell is (t)csh. Also note that not all find implementations will expand that {} there (sfind won’t).

This would be a bit trickier to do with -exec as we would have to get the basename of the found file out of {} to form the new name of the file. We also need the directory name from {} to locate the done-texts directory properly.

With -execdir, some things like these becomes easier.

The corresponding operation using -exec instead of -execdir would have to employ a child shell:

find . -type f -name '*.txt' -exec sh -c '
    for name do
        mv "$name" "$( dirname "$name" )/done-texts/$( basename "$name" ).done"
    done' sh {} +

or,

find . -type f -name '*.txt' -exec sh -c '
    for name do
        mv "$name" "${name%/*}/done-texts/${name##*/}.done"
    done' sh {} +

Related Solutions

Joining bash arguments into single string with spaces

[*] I believe that this does what you want. It will put all the arguments in one string, separated by spaces, with single quotes around all: str="'$*'" $* produces all the scripts arguments separated by the first character of $IFS which, by default, is a space....

AddTransient, AddScoped and AddSingleton Services Differences

TL;DR Transient objects are always different; a new instance is provided to every controller and every service. Scoped objects are the same within a request, but different across different requests. Singleton objects are the same for every object and every...

How to download package not install it with apt-get command?

Use --download-only: sudo apt-get install --download-only pppoe This will download pppoe and any dependencies you need, and place them in /var/cache/apt/archives. That way a subsequent apt-get install pppoe will be able to complete without any extra downloads....

What defines the maximum size for a command single argument?

Answers Definitely not a bug. The parameter which defines the maximum size for one argument is MAX_ARG_STRLEN. There is no documentation for this parameter other than the comments in binfmts.h: /* * These are the maximum length and maximum number of strings...

Bulk rename, change prefix

I'd say the simplest it to just use the rename command which is common on many Linux distributions. There are two common versions of this command so check its man page to find which one you have: ## rename from Perl (common in Debian systems -- Ubuntu, Mint,...

Output from ls has newlines but displays on a single line. Why?

When you pipe the output, ls acts differently. This fact is hidden away in the info documentation: If standard output is a terminal, the output is in columns (sorted vertically) and control characters are output as question marks; otherwise, the output is...

mv: Move file only if destination does not exist

mv -vn file1 file2. This command will do what you want. You can skip -v if you want. -v makes it verbose - mv will tell you that it moved file if it moves it(useful, since there is possibility that file will not be moved) -n moves only if file2 does not exist....

Is it possible to store and query JSON in SQLite?

SQLite 3.9 introduced a new extension (JSON1) that allows you to easily work with JSON data . Also, it introduced support for indexes on expressions, which (in my understanding) should allow you to define indexes on your JSON data as well. PostgreSQL has some...

Combining tail && journalctl

You could use: journalctl -u service-name -f -f, --follow Show only the most recent journal entries, and continuously print new entries as they are appended to the journal. Here I've added "service-name" to distinguish this answer from others; you substitute...

how can shellshock be exploited over SSH?

One example where this can be exploited is on servers with an authorized_keys forced command. When adding an entry to ~/.ssh/authorized_keys, you can prefix the line with command="foo" to force foo to be run any time that ssh public key is used. With this...

Why doesn’t the tilde (~) expand inside double quotes?

The reason, because inside double quotes, tilde ~ has no special meaning, it's treated as literal. POSIX defines Double-Quotes as: Enclosing characters in double-quotes ( "" ) shall preserve the literal value of all characters within the double-quotes, with the...

What is GNU Info for?

GNU Info was designed to offer documentation that was comprehensive, hyperlinked, and possible to output to multiple formats. Man pages were available, and they were great at providing printed output. However, they were designed such that each man page had a...

Set systemd service to execute after fstab mount

a CIFS network location is mounted via /etc/fstab to /mnt/ on boot-up. No, it is not. Get this right, and the rest falls into place naturally. The mount is handled by a (generated) systemd mount unit that will be named something like mnt-wibble.mount. You can...

Merge two video clips into one, placing them next to each other

To be honest, using the accepted answer resulted in a lot of dropped frames for me. However, using the hstack filter_complex produced perfectly fluid output: ffmpeg -i left.mp4 -i right.mp4 -filter_complex hstack output.mp4 ffmpeg -i input1.mp4 -i input2.mp4...

How portable are /dev/stdin, /dev/stdout and /dev/stderr?

It's been available on Linux back into its prehistory. It is not POSIX, although many actual shells (including AT&T ksh and bash) will simulate it if it's not present in the OS; note that this simulation only works at the shell level (i.e. redirection or...

How can I increase the number of inodes in an ext4 filesystem?

It seems that you have a lot more files than normal expectation. I don't know whether there is a solution to change the inode table size dynamically. I'm afraid that you need to back-up your data, and create new filesystem, and restore your data. To create new...

Why doesn’t cp have a progress bar like wget?

The tradition in unix tools is to display messages only if something goes wrong. I think this is both for design and practical reasons. The design is intended to make it obvious when something goes wrong: you get an error message, and it's not drowned in...