Home » How can I replace a string in a file(s)?

How can I replace a string in a file(s)?

Solutons:


1. Replacing all occurrences of one string with another in all files in the current directory:

These are for cases where you know that the directory contains only regular files and that you want to process all non-hidden files. If that is not the case, use the approaches in 2.

All sed solutions in this answer assume GNU sed. If using FreeBSD or macOS, replace -i with -i ''. Also note that the use of the -i switch with any version of sed has certain filesystem security implications and is inadvisable in any script which you plan to distribute in any way.

  • Non recursive, files in this directory only:

     sed -i -- 's/foo/bar/g' *
     perl -i -pe 's/foo/bar/g' ./* 
    

(the perl one will fail for file names ending in | or space)).

  • Recursive, regular files (including hidden ones) in this and all subdirectories

     find . -type f -exec sed -i 's/foo/bar/g' {} +
    

    If you are using zsh:

     sed -i -- 's/foo/bar/g' **/*(D.)
    

    (may fail if the list is too big, see zargs to work around).

    Bash can’t check directly for regular files, a loop is needed (braces avoid setting the options globally):

     ( shopt -s globstar dotglob;
         for file in **; do
             if [[ -f $file ]] && [[ -w $file ]]; then
                 sed -i -- 's/foo/bar/g' "$file"
             fi
         done
     )
    

    The files are selected when they are actual files (-f) and they are writable (-w).

2. Replace only if the file name matches another string / has a specific extension / is of a certain type etc:

  • Non-recursive, files in this directory only:

    sed -i -- 's/foo/bar/g' *baz*    ## all files whose name contains baz
    sed -i -- 's/foo/bar/g' *.baz    ## files ending in .baz
    
  • Recursive, regular files in this and all subdirectories

    find . -type f -name "*baz*" -exec sed -i 's/foo/bar/g' {} +
    

    If you are using bash (braces avoid setting the options globally):

    ( shopt -s globstar dotglob
        sed -i -- 's/foo/bar/g' **baz*
        sed -i -- 's/foo/bar/g' **.baz
    )
    

    If you are using zsh:

    sed -i -- 's/foo/bar/g' **/*baz*(D.)
    sed -i -- 's/foo/bar/g' **/*.baz(D.)
    

The -- serves to tell sed that no more flags will be given in the command line. This is useful to protect against file names starting with -.

  • If a file is of a certain type, for example, executable (see man find for more options):

    find . -type f -executable -exec sed -i 's/foo/bar/g' {} +
    

zsh:

    sed -i -- 's/foo/bar/g' **/*(D*)

3. Replace only if the string is found in a certain context

  • Replace foo with bar only if there is a baz later on the same line:

     sed -i 's/foo(.*baz)/bar1/' file
    

In sed, using ( ) saves whatever is in the parentheses and you can then access it with 1. There are many variations of this theme, to learn more about such regular expressions, see here.

  • Replace foo with bar only if foo is found on the 3d column (field) of the input file (assuming whitespace-separated fields):

     gawk -i inplace '{gsub(/foo/,"baz",$3); print}' file
    

(needs gawk 4.1.0 or newer).

  • For a different field just use $N where N is the number of the field of interest. For a different field separator (: in this example) use:

     gawk -i inplace -F':' '{gsub(/foo/,"baz",$3);print}' file
    

Another solution using perl:

    perl -i -ane '$F[2]=~s/foo/baz/g; $" = " "; print "@Fn"' foo 

NOTE: both the awk and perl solutions will affect spacing in the file (remove the leading and trailing blanks, and convert sequences of blanks to one space character in those lines that match). For a different field, use $F[N-1] where N is the field number you want and for a different field separator use (the $"=":" sets the output field separator to :):

    perl -i -F':' -ane '$F[2]=~s/foo/baz/g; $"=":";print "@F"' foo 
  • Replace foo with bar only on the 4th line:

     sed -i '4s/foo/bar/g' file
     gawk -i inplace 'NR==4{gsub(/foo/,"baz")};1' file
     perl -i -pe 's/foo/bar/g if $.==4' file
    

4. Multiple replace operations: replace with different strings

  • You can combine sed commands:

     sed -i 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
    

Be aware that order matters (sed 's/foo/bar/g; s/bar/baz/g' will substitute foo with baz).

  • or Perl commands

     perl -i -pe 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
    
  • If you have a large number of patterns, it is easier to save your patterns and their replacements in a sed script file:

     #! /usr/bin/sed -f
     s/foo/bar/g
     s/baz/zab/g
    
  • Or, if you have too many pattern pairs for the above to be feasible, you can read pattern pairs from a file (two space separated patterns, $pattern and $replacement, per line):

     while read -r pattern replacement; do   
         sed -i "s/$pattern/$replacement/" file
     done < patterns.txt
    
  • That will be quite slow for long lists of patterns and large data files so you might want to read the patterns and create a sed script from them instead. The following assumes a <<!>space<!>> delimiter separates a list of MATCH<<!>space<!>>REPLACE pairs occurring one-per-line in the file patterns.txt :

     sed 's| *([^ ]*) *([^ ]*).*|s/1/2/g|' <patterns.txt |
     sed -f- ./editfile >outfile
    

The above format is largely arbitrary and, for example, doesn’t allow for a <<!>space<!>> in either of MATCH or REPLACE. The method is very general though: basically, if you can create an output stream which looks like a sed script, then you can source that stream as a sed script by specifying sed‘s script file as -stdin.

  • You can combine and concatenate multiple scripts in similar fashion:

     SOME_PIPELINE |
     sed -e'#some expression script'  
         -f./script_file -f-          
         -e'#more inline expressions' 
     ./actual_edit_file >./outfile
    

A POSIX sed will concatenate all scripts into one in the order they appear on the command-line. None of these need end in a newline.

  • grep can work the same way:

     sed -e'#generate a pattern list' <in |
     grep -f- ./grepped_file
    
  • When working with fixed-strings as patterns, it is good practice to escape regular expression metacharacters. You can do this rather easily:

     sed 's/[]$&^*./[]/\&/g
          s| *([^ ]*) *([^ ]*).*|s/1/2/g|
     ' <patterns.txt |
     sed -f- ./editfile >outfile
    

5. Multiple replace operations: replace multiple patterns with the same string

  • Replace any of foo, bar or baz with foobar

     sed -Ei 's/foo|bar|baz/foobar/g' file
    
  • or

     perl -i -pe 's/foo|bar|baz/foobar/g' file
    

A good replacement Linux tool is rpl, that was originally written for the Debian project, so it is available with apt-get install rpl in any Debian derived distro, and may be for others, but otherwise you can download the tar.gz file from SourceForge.

Simplest example of use:

 $ rpl old_string new_string test.txt

Note that if the string contains spaces it should be enclosed in quotation marks. By default rpl takes care of capital letters but not of complete words, but you can change these defaults with options -i (ignore case) and -w (whole words). You can also specify multiple files:

 $ rpl -i -w "old string" "new string" test.txt test2.txt

Or even specify the extensions (-x) to search or even search recursively (-R) in the directory:

 $ rpl -x .html -x .txt -R old_string new_string test*

You can also search/replace in interactive mode with -p (prompt) option:

The output shows the numbers of files/string replaced and the type of search (case in/sensitive, whole/partial words), but it can be silent with the -q (quiet mode) option, or even more verbose, listing line numbers that contain matches of each file and directory with -v (verbose mode) option.

Other options that are worth remembering are -e (honor escapes) that allow regular expressions, so you can search also tabs (t), new lines (n),etc. You can use -f to force permissions (of course, only when the user has write permissions) and -d to preserve the modification times`).

Finally, if you are unsure what exactly will happen, use the -s (simulate mode).

How to do a search and replace over multiple files suggests:

You could also use find and sed, but I find that this little line of
perl works nicely.

perl -pi -w -e 's/search/replace/g;' *.php
  • -e means execute the following line of code.
  • -i means edit in-place
  • -w write warnings
  • -p loop over the input file, printing each line after the script is applied to it.

My best results come from using perl and grep (to ensure that file have the search expression )

perl -pi -w -e 's/search/replace/g;' $( grep -rl 'search' )

Related Solutions

Winsock programming connecting to a public ip

The issue is with your server. You are binding it to 127.0.0.1. This means your server will only bind to the loopback interface, so only clients running on the same machine as the server will be able to connect to the server using this same interface. If you...

Pin-board effect with CSS [closed]

You can use JavaScript to accomplish this but it can't be done with CSS floats alone. A library like jQuery masonry will do it well. The reason? The specs on floats. In particular #5 which says, "The outer top of a floating box may not be higher than the outer...

Calculate the sum with minimum usage of numbers

Here's a hint: 23 : 11 + 11+ 1 ( 3 magic numbers) 120: 110+ 10 (2 magic numbers) The highest digit in the target number is the answer, since you need exactly k magic numbers (all having 1 in the relevant position) in order for the sum to contain the digit k. So...

Why not drop the “auto” keyword? [duplicate]

Your proposal would be rejected on the basis of backward compatibility alone. But let's say for the sake of argument that the standards committee like your idea. You don't take into account the numerous ways you can initialize a variable widget w; // (a) widget...

Recursive to iterative using a systematic method [closed]

So, to restate the question. We have a function f, in our case fac. def fac(n): if n==0: return 1 else: return n*fac(n-1) It is implemented recursively. We want to implement a function facOpt that does the same thing but iteratively. fac is written almost in...

How can I match values in one file to ranges from another?

if the data file sizes are not huge, there is a simpler way $ join input1 input2 | awk '$5<$4 && $3<$5 {print $2, $5-$3+1}' B100002 32 B100043 15 B123465 3 This Perl code seems to solve your problem It is a common idiom: to load the entire...

Javascript difference between “=” and “===” [duplicate]

You need to use == or === for equality checking. = is the assignment operator. You can read about assignment operators here on MDN. As a quick reference as you are learning JS: = assignment operator == equal to === equal value and equal type != not equal !==...

Compiler complains about misplaced else [closed]

Your compiler complains about an misplaced else because, well, there is an else without a preceding if: // ... for (j=1; j<n-i; j++) { if(a[j]<=a[j+1]) { // ... } // END OF IF } // END OF FOR else { continue; } // ... The else in your code does not follow...

Bootstrap – custom alerts with progress bar

/* !important are just used to overide the bootstrap css in the snippet */ .alertContainer { border-radius: 0 !important; border-width: 0 !important; padding: 0 !important; height: auto !important; position: absolute !important; bottom: 15px !important; left:...

How to Garbage Collect an external Javascript load?

Yes, s.onload = null is useful and will garbage collect! As of 2019, it is not possible to explicitly or programmatically trigger garbage collection in JavaScript. That means it collects when it wants. Although there is cases where setting to null may do a GC...

Math programming with python

At first, what you are looking for is the modulo operator and the function math.floor() Modulo from wikipedia: In computing, the modulo operation finds the remainder after division of one number by another (sometimes called modulus). for example: 12%12=0...

Android slide over letters to create a word [closed]

Here some advice you can use: First for each cell you can create an object that represents the state of that cell: class Cell { char mChar; int row,column; boolean isSelected; } then you can create a 2D array of your cells Cell[][] mTable = ... For views you...

Sum two integers in Java

You reused the x and y variable names (hence the variable x is already defined in method main error), and forgot to assign the ints read from the Scanner to the x and y variables. Besides, there's no need to create two Scanner objects. public static void...