linux sed I : substitution
The man page for sed:
"Sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed's ability to filter text in a pipeline which particularly distinguishes it from other types of editors."
Here is a sample text, saved as text:
i wandered lonely as a cloud that floats on high o'er vales and hills, when all at once I saw a crowd, a host, of golden daffodils; beside the lake, beneath the trees, fluttering and dancing in the breeze.
Let's try replace 'a' with 'A':
$ cat text | sed 's/a/A/'
$ cat text | sed 's/a/A/' i wAndered lonely as a cloud thAt floats on high o'er vales and hills, when All at once I saw a crowd, A host, of golden daffodils; beside the lAke, beneath the trees, fluttering And dancing in the breeze.
We replaced only the first occurrence of 'a' in each line. Note that we haven't modified original text at all. We just replace a character and displayed it.
The pipe isn't necessary, and we can achieve the same result using this:
$ sed 's/a/A/' text
To replace all 'a' with 'A', we need to use g which means 'global':
$ sed 's/a/A/g' text
$ sed 's/a/A/g' text i wAndered lonely As A cloud thAt floAts on high o'er vAles And hills, when All At once I sAw A crowd, A host, of golden dAffodils; beside the lAke, beneAth the trees, fluttering And dAncing in the breeze.
At this point, the original file is intact:
$ cat text i wandered lonely as a cloud that floats on high o'er vales and hills, when all at once I saw a crowd, a host, of golden daffodils; beside the lake, beneath the trees, fluttering and dancing in the breeze.
To modify our initial text, we need to i which means edit file in-place:
$ sed -i 's/a/A/g' text
$ sed -i 's/a/A/g' text $ cat text i wAndered lonely As A cloud thAt floAts on high o'er vAles And hills, when All At once I sAw A crowd, A host, of golden dAffodils; beside the lAke, beneAth the trees, fluttering And dAncing in the breeze.
Of course we can put it back by reversing the substitute:
$ sed -i 's/A/a/g' text
$ sed -i 's/A/a/g' text $ cat text i wandered lonely as a cloud that floats on high o'er vales and hills, when all at once I saw a crowd, a host, of golden daffodils; beside the lake, beneath the trees, fluttering and dancing in the breeze.
Note that we can replace not only a character but also a string:
$ sed -i 's/tree/TREE/g' text
Replacing a string recursively (for all files including sub-directories):
$ find . -type f -name "*.txt" -print0 | xargs -0 sed -i '' 's/abcde/12345/g'
find . -type f -name '*.txt'finds, in the current directory (.) and below, all regular files (-type f) whose names end in .txt- The "print0" puts ASCII NUL instead of '\n', and this requires "-0" after "xargs".
- passes the output of that command (a list of filenames) to the next command
xargsgathers up those filenames and hands them one by one tosedsed -i '' -e 's/abcde/12345/g'means "edit the file in place, without a backup (here, zero length string, '' is provided), and make the following substitution (s/abcde/12345) multiple times per line (/g)" (see man sed)
We can also do the same with -exec:
$ find ./ -type f -name '*.txt' -exec sed -i 's/abcde/12345/g' {} \; -print
The "{}" has the result from the find command, and the find command needs to know when the arguments of exec are terminated.
So, we added ';'. The escape ('\') is needed because ';' has its own meaning within a shell.
On MacOS, however, the sed -i requires backup, which is an option in Linux.
So, it should look like this on MacOS (note that no space between i and '.bak'):
$ find ./ -type f -name "*.txt" -exec sed -i'.bak' "s/12345/54321/g" {} \; -print
Don't like the require backup option? Then, install gnu-sed, and use gsed instead:
$ brew install gnu-sed
$ find ./ -type f -name "*.txt" -exec gsed -i "s/12345/54321/g" {} \; -print
If don't like the escape ("\"), we can use quote("") instead as shown below:
$ find ./ -type f -name "*.txt" -exec sed -i "s/abcde/12345/g" {} ";" -print
Linux - system, cmds & shell
- Linux Tips - links, vmstats, rsync
- Linux Tips 2 - ctrl a, curl r, tail -f, umask
- Linux - bash I
- Linux - bash II
- Linux - Uncompressing 7z file
- Linux - sed I (substitution: sed 's///', sed -i)
- Linux - sed II (file spacing, numbering, text conversion and substitution)
- Linux - sed III (selective printing of certain lines, selective definition of certain lines)
- Linux - 7 File types : Regular, Directory, Block file, Character device file, Pipe file, Symbolic link file, and Socket file
- Linux shell programming - introduction
- Linux shell programming - variables and functions (readonly, unset, and functions)
- Linux shell programming - special shell variables
- Linux shell programming : arrays - three different ways of declaring arrays & looping with $*/$@
- Linux shell programming : operations on array
- Linux shell programming : variables & commands substitution
- Linux shell programming : metacharacters & quotes
- Linux shell programming : input/output redirection & here document
- Linux shell programming : loop control - for, while, break, and break n
- Linux shell programming : string
- Linux shell programming : for-loop
- Linux shell programming : if/elif/else/fi
- Linux shell programming : Test
- Managing User Account - useradd, usermod, and userdel
- Linux Secure Shell (SSH) I : key generation, private key and public key
- Linux Secure Shell (SSH) II : ssh-agent & scp
- Linux Secure Shell (SSH) III : SSH Tunnel as Proxy - Dynamic Port Forwarding (SOCKS Proxy)
- Linux Secure Shell (SSH) IV : Local port forwarding (outgoing ssh tunnel)
- Linux Secure Shell (SSH) V : Reverse SSH Tunnel (remote port forwarding / incoming ssh tunnel) /)
- Linux Processes and Signals
- Linux Drivers 1
- tcpdump
- Linux Debugging using gdb
- Embedded Systems Programming I - Introduction
- Embedded Systems Programming II - gcc ARM Toolchain and Simple Code on Ubuntu/Fedora
- LXC (Linux Container) Install and Run
- Linux IPTables
- Hadoop - 1. Setting up on Ubuntu for Single-Node Cluster
- Hadoop - 2. Runing on Ubuntu for Single-Node Cluster
- ownCloud 7 install
- Ubuntu 14.04 guest on Mac OSX host using VirtualBox I
- Ubuntu 14.04 guest on Mac OSX host using VirtualBox II
- Windows 8 guest on Mac OSX host using VirtualBox I
- Ubuntu Package Management System (apt-get vs dpkg)
- RPM Packaging
- How to Make a Self-Signed SSL Certificate
- Linux Q & A
- DevOps / Sys Admin questions
Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization
My YouTube channel