Home » Find and replace text within a file using commands

Find and replace text within a file using commands

Solutons:


sed -i 's/original/new/g' file.txt

Explanation:

  • sed = Stream EDitor
  • -i = in-place (i.e. save back to the original file)
  • The command string:

    • s = the substitute command
    • original = a regular expression describing the word to replace (or just the word itself)
    • new = the text to replace it with
    • g = global (i.e. replace all and not just the first occurrence)
  • file.txt = the file name

There’s multitude of ways to achieve it. Depending on the complexity of what one tries to achieve with string replacement, and depending on tools with which user is familiar, some methods may be preferred more than others.

In this answer I am using simple input.txt file, which you can use to test all examples provided here. The file contents:

roses are red , violets are blue
This is an input.txt and this doesn't rhyme

BASH

Bash isn’t really meant for text processing, but simple substitutions can be done via parameter expansion , in particular here we can use simple structure ${parameter/old_string/new_string}.

#!/bin/bash
while IFS= read -r line
do
    case "$line" in
       *blue*) printf "%sn" "${line/blue/azure}" ;;
       *) printf "%sn" "$line" ;;
    esac
done < input.txt

This small script doesn’t do in-place replacement, meaning that you would have to save new text to new file, and get rid of the old file, or mv new.txt old.txt

Side note: if you’re curious about why while IFS= read -r ; do ... done < input.txt is used, it’s basically shell’s way of reading file line by line. See this for reference.

AWK

AWK, being a text processing utility, is quite appropriate for such task. It can do simple replacements and much more advanced ones based on regular expressions. It provides two functions: sub() and gsub(). The first one only replaces only the first occurrence, while the second – replaces occurrences in whole string. For instance, if we have string one potato two potato , this would be the result:

$ echo "one potato two potato" | awk '{gsub(/potato/,"banana")}1'
one banana two banana

$ echo "one potato two potato" | awk '{sub(/potato/,"banana")}1'                                      
one banana two potato 

AWK can take an input file as argument, so doing same things with input.txt , would be easy:

awk '{sub(/blue/,"azure")}1' input.txt

Depending on the version of AWK you have, it may or may not have in-place editing, hence the usual practice is save and replace new text. For instance something like this:

awk '{sub(/blue/,"azure")}1' input.txt > temp.txt && mv temp.txt input.txt

SED

Sed is a line editor. It also uses regular expressions, but for simple substitutions it’s sufficient to do:

sed 's/blue/azure/' input.txt

What’s good about this tool is that it has in-place editing, which you can enable with -i flag.

Perl

Perl is another tool which is often used for text processing, but it’s a general purpose language, and is used in networking, system administration, desktop apps, and many other places. It borrowed a lot of concepts/features from other languages such as C,sed,awk, and others. Simple substitution can be done as so:

perl -pe 's/blue/azure/' input.txt

Like sed, perl also has the -i flag.

Python

This language is very versatile and is also used in a wide variety of applications. It has a lot of functions for working with strings, among which is replace(), so if you have variable like var="Hello World" , you could do var.replace("Hello","Good Morning")

Simple way to read file and replace string in it would be as so:

python -c "import sys;lines=sys.stdin.read();print lines.replace('blue','azure')" < input.txt

With Python, however, you also need to output to new file , which you can also do from within the script itself. For instance, here’s a simple one:

#!/usr/bin/env python
import sys
import os
import tempfile

tmp=tempfile.mkstemp()

with open(sys.argv[1]) as fd1, open(tmp[1],'w') as fd2:
    for line in fd1:
        line = line.replace('blue','azure')
        fd2.write(line)

os.rename(tmp[1],sys.argv[1])

This script is to be called with input.txt as command-line argument. The exact command to run python script with command-line argument would be

 $ ./myscript.py input.txt

or

$ python ./myscript.py input.txt

Of course, make sure that ./myscript.py is in your current working directory and for the first way, ensure it is set executable with chmod +x ./myscript.py

Python can also have regular expressions , in particular, there’s re module, which has re.sub() function, which can be used for more advanced replacements.

There are a number of different ways to do this. One is using sed and Regex. SED is a Stream Editor for filtering and transforming text. One example is as follows:

marco@imacs-suck: ~$ echo "The slow brown unicorn jumped over the hyper sleeping dog" > orly
marco@imacs-suck: ~$ sed s/slow/quick/ < orly > yarly
marco@imacs-suck: ~$ cat yarly
The quick brown unicorn jumped over the hyper sleeping dog

Another way which may make more sense than < strin and > strout is with pipes!

marco@imacs-suck: ~$ cat yarly | sed s/unicorn/fox/ | sed s/hyper/lazy/ > nowai
marco@imacs-suck: ~$ cat nowai 
The quick brown fox jumped over the lazy sleeping dog

Related Solutions

Calculate the sum with minimum usage of numbers

Here's a hint: 23 : 11 + 11+ 1 ( 3 magic numbers) 120: 110+ 10 (2 magic numbers) The highest digit in the target number is the answer, since you need exactly k magic numbers (all having 1 in the relevant position) in order for the sum to contain the digit k. So...

Why not drop the “auto” keyword? [duplicate]

Your proposal would be rejected on the basis of backward compatibility alone. But let's say for the sake of argument that the standards committee like your idea. You don't take into account the numerous ways you can initialize a variable widget w; // (a) widget...

Recursive to iterative using a systematic method [closed]

So, to restate the question. We have a function f, in our case fac. def fac(n): if n==0: return 1 else: return n*fac(n-1) It is implemented recursively. We want to implement a function facOpt that does the same thing but iteratively. fac is written almost in...

How can I match values in one file to ranges from another?

if the data file sizes are not huge, there is a simpler way $ join input1 input2 | awk '$5<$4 && $3<$5 {print $2, $5-$3+1}' B100002 32 B100043 15 B123465 3 This Perl code seems to solve your problem It is a common idiom: to load the entire...

Javascript difference between “=” and “===” [duplicate]

You need to use == or === for equality checking. = is the assignment operator. You can read about assignment operators here on MDN. As a quick reference as you are learning JS: = assignment operator == equal to === equal value and equal type != not equal !==...

Compiler complains about misplaced else [closed]

Your compiler complains about an misplaced else because, well, there is an else without a preceding if: // ... for (j=1; j<n-i; j++) { if(a[j]<=a[j+1]) { // ... } // END OF IF } // END OF FOR else { continue; } // ... The else in your code does not follow...

Bootstrap – custom alerts with progress bar

/* !important are just used to overide the bootstrap css in the snippet */ .alertContainer { border-radius: 0 !important; border-width: 0 !important; padding: 0 !important; height: auto !important; position: absolute !important; bottom: 15px !important; left:...

How to Garbage Collect an external Javascript load?

Yes, s.onload = null is useful and will garbage collect! As of 2019, it is not possible to explicitly or programmatically trigger garbage collection in JavaScript. That means it collects when it wants. Although there is cases where setting to null may do a GC...

Math programming with python

At first, what you are looking for is the modulo operator and the function math.floor() Modulo from wikipedia: In computing, the modulo operation finds the remainder after division of one number by another (sometimes called modulus). for example: 12%12=0...

Android slide over letters to create a word [closed]

Here some advice you can use: First for each cell you can create an object that represents the state of that cell: class Cell { char mChar; int row,column; boolean isSelected; } then you can create a 2D array of your cells Cell[][] mTable = ... For views you...

Sum two integers in Java

You reused the x and y variable names (hence the variable x is already defined in method main error), and forgot to assign the ints read from the Scanner to the x and y variables. Besides, there's no need to create two Scanner objects. public static void...

Extend three classes that implements an interface in Java

Using this simplified implementation of the library, using method() instead of M(): interface IFC { void method(); } class A implements IFC { public void method() { System.out.println("method in A"); }; } As akuzminykh mentions in their comment You'd write a...

How to set the stream content in PHPExcel? [closed]

Okey, First thing first PHPExcel_Worksheet_MemoryDrawing() can't solve your problem if you insist to use stream content and pass that to your worksheet your PDF will not render your image. But you can use `PHPExcel_Worksheet_Drawing()' if you want to render...

How to remove all files from a directory?

Linux does not use extensions. It is up to the creator of the file to decide whether the name should have an extension. Linux looks at the first few bytes to figure out what kind of file it is dealing with. To remove all non-hidden files* in a directory use: rm...