How to remove ^[, and all of the escape sequences in a file using linux shell scripting

Asked
Active3 hr before
Viewed126 times

7 Answers

escapeusingremove
90%

groff produces bold characters like you would on a typewriter: print a letter, move one character back with backspace (you can't erase text on a typewriter), print the same letter again to make the character more pronounced. So simply omitting backspaces produces "SSYYNNOOPPSSIISS". col -b fixes this by interpreting backspaces correctly, quote from the manual:,I've stumbled upon this post when looking for a way to strip extra formatting from man pages. ansifilter did it, but it was far from desired result (for example all previously-bold characters were duplicated, like SSYYNNOOPPSSIISS)., 1 @rth Unclear what you are proposing; trimming exactly four characters is wrong because the escape sequences are different lengths. You'd have to write an escape sequence parser to know how many to remove. – tripleee Mar 23 '20 at 5:20 , 1 It seems to be the col -b option that does this. The documentation says that this removes backspaces characters :/, go figure. It is nevertheless the most compact option that I could find that doesn't require one to install things (outside of one's package manager) – Att Righ Mar 8 '17 at 15:31

Using keyboard entry:

sed 's/Ctrl-vEsc//g'

alternatively

sed 's/Ctrl-vCtrl-[//g'

Or you can use character escapes:

sed 's/\x1b//g'

or for all control characters:

sed 's/[\x01-\x1F\x7F]//g'
# NOTE: zaps TAB character too!
load more v
88%

$ file input.txt
input.txt: ASCII English text, with escape sequences
load more v
72%

The first and last " characters wrap the whole lot into one parameter passed to echo so that the spacing between the two words is kept as is. But the code:,would be interpreted as three parameters:, So backslash itself must be escaped to show that it is to be taken literally. The other special character, the backtick, is discussed later in Chapter 12, External Programs. , Most characters (*, ', etc) are not interpreted (ie, they are taken literally) by means of placing them in double quotes (""). They are taken as is and passed on to the command being called. An example using the asterisk (*) goes:

Certain characters are significant to the shell; we have seen, for example, that the use of double quotes (") characters affect how spaces and TAB characters are treated, for example:

$ echo Hello World
Hello World
$ echo "Hello       World"
Hello World
load more v
65%

You need to escape the [ and \ characters with a backslash:,Thanks for contributing an answer to Unix & Linux Stack Exchange!, Stack Exchange network consists of 178 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. ,Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.

If you want in the output just foo then

printf '%s'
'foo.bar\033[00m' | sed 's/\.bar\\033\[00m//'

After question edit: If your file contains control characters such as

printf "foo.bar\033[00m" > file
load more v
75%

You would get your shell prompt back, as if nothing happened! But then there would be a new file, with the name 5, containing “2”, the names of all files in your current directory, and then the string 3 is a valid inequality. Make sure you understand why. [17] ,You can use wildcards in the string, but you must quote them, so that the find command itself can match them against names of files in each directory it searches. The command find . -name ' * .c ' will match all files whose names end in .c anywhere in your current directory, subdirectories, sub-subdirectories, etc. ,[17] This should also teach you something about the flexibility of placing I/O redirectors anywhere on the command line—even in places where they don’t seem to make sense.,the result is the string, taken literally. You needn’t quote the entire line, just the portion containing special characters (or characters you think might be special, if you just want to be sure):

$ echo 2 * 3 > 5 is a valid inequality.
$ echo '2 * 3 > 5 is a valid inequality.'
$ echo '2 * 3 > 5'
is a valid inequality.
load more v
40%

If the command substitution consists of a single subshell, such as:,If the command name matches the name of a utility listed in the following table, that utility shall be invoked.,The shell reads its input from a file (see sh), from the -c option or from the system() and popen() functions defined in the System Interfaces volume of IEEE Std 1003.1-2001. If the first line of a file of shell commands starts with the characters "#!", the results are unspecified., The shell reads its input from a file (see sh), from the -c option or from the system() and popen() functions defined in the System Interfaces volume of IEEE Std 1003.1-2001. If the first line of a file of shell commands starts with the characters "#!", the results are unspecified.

| & ; <> ( ) $ ` \ " ' <space>
      <tab>
         <newline>
load more v
22%

bash, available from http://www.gnu.org/directory/GNU/. The Bash shell is available on nearly every Linux system, and can these days be found on a wide variety of UNIX systems.,Using the file descriptors to read from and write to multiple files,Then she displays the line numbers containing this search string.,The file /etc/shells gives an overview of known shells on a Linux system:

* Copyright(c) 2002 - 2007, Machtelt Garrels *
   All rights reserved.*Redistribution and use in source and binary forms, with or without *
   modification, are permitted provided that the following conditions are met:
   *
   *
   * Redistributions of source code must retain the above copyright *
   notice, this list of conditions and the following disclaimer.* * Redistributions in binary form must reproduce the above copyright *
   notice, this list of conditions and the following disclaimer in the *
   documentation and / or other materials provided with the distribution.* * Neither the name of the author, Machtelt Garrels, nor the *
   names of its contributors may be used to endorse or promote products *
   derived from this software without specific prior written permission.*
   *
   THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS "AS IS"
AND ANY
   *
   EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED *
   WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE *
   DISCLAIMED.IN NO EVENT SHALL THE AUTHOR AND CONTRIBUTORS BE LIABLE FOR ANY *
   DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES *
   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
      * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND *
   ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT *
   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS *
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
load more v

Other "escape-using" queries related to "How to remove ^[, and all of the escape sequences in a file using linux shell scripting"