Sed Replace A Multi-Line String: Difference between revisions

From Chorke Wiki
Jump to navigation Jump to search
(Created page with "{| |valign="top" colspan="2"| <source lang="bash"> mkdir -p ${HOME}/Documents/sed_playground sudo tee -a ${HOME}/Documents/sed_playground/000-default.conf >/dev/null <<EOF...")
 
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Example==
{|
{|
|valign="top" colspan="2"|
|valign="top" colspan="2"|
<source lang="bash">
<source lang="bash">
mkdir   -p ${HOME}/Documents/sed_playground
mkdir -p ${HOME}/Documents/sed_playground
sudo tee -a ${HOME}/Documents/sed_playground/000-default.conf >/dev/null <<EOF
tee   -a ${HOME}/Documents/sed_playground/000-default.conf >/dev/null <<EOF
#
#
#This is some test comments
#This is some test comments
Line 24: Line 25:
|colspan="2" valign="top"|
|colspan="2" valign="top"|
<source lang="bash">
<source lang="bash">
CONF_FILE="${HOME}/Documents/sed_playground/000-default.conf"
APACHE_CONF_FILE="${HOME}/Documents/sed_playground/000-default.conf"
DIRECTORY_FIND_LEAD_EXP='<Directory "\/var\/www\/cgi-bin">'
DIRECTORY_FIND_LEAD_EXP='<Directory "\/var\/www\/cgi-bin">'
</source>
</source>
Line 63: Line 64:
|-
|-
|colspan="2" valign="top"|
|colspan="2" valign="top"|
'''Complex:'''
<source lang="bash">
<source lang="bash">
echo ""
echo "complex"
echo "Sedding"
echo "sedding"
sed -i "/${DIRECTORY_FIND_LEAD_EXP}/{
sed -i "/${DIRECTORY_FIND_LEAD_EXP}/{N;N;N;N;N;s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}|}" ${APACHE_CONF_FILE}
    N;N;N;N;N
</source>
    s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}|
 
}" ${CONF_FILE}
|-
|colspan="2"|
----
|-
|colspan="2" valign="top"|
'''Simple:'''
<source lang="bash">
echo "simple"
echo "sedding"
sed -z "s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}|" -i ${APACHE_CONF_FILE}
</source>
</source>


Line 77: Line 88:
|-
|-
|valign="top"|
|valign="top"|
'''${HOME}/Documents/sed_playground/000-default.conf'''
<source lang="apache">
<source lang="apache">
#
#
Line 92: Line 104:


|valign="top"|
|valign="top"|
'''${HOME}/Documents/sed_playground/000-default.conf'''
<source lang="apache">
<source lang="apache">
#
#
Line 105: Line 118:
#</Directory>
#</Directory>
</source>
</source>
|-
|colspan="2"|
----
|-
|colspan="2" valign="top"|
* <code>sed</code> separate <code>|</code> used to instead of <code>/</code> to avoid escaping of <code>/</code> and <code>.</code>
* <code>EOF</code> block used to avoid escaping of <code>"</code>
* Lets have a look:
|-
|colspan="2"|
----
|-
|valign="top"|
<source lang="bash">
DIRECTORY_FIND_FULL_EXP=$(cat <<EOF
<Directory "/var/www/cgi-bin">\n\
[ ]*AllowOverride None\n\
[ ]*Options +ExecCGI\n\
[ ]*AddHandler cgi-script .cgi .pl\n\
[ ]*Require all granted\n\
</Directory>
EOF
)
</source>
|valign="top"|
<source lang="bash">
DIRECTORY_FILL_FULL_EXP=$(cat <<EOF
#<Directory "/var/www/cgi-bin">\n\
    #AllowOverride None\n\
    #Options +ExecCGI\n\
    #AddHandler cgi-script .cgi .pl\n\
    #Require all granted\n\
#</Directory>
EOF
)
</source>
|}
==Explanation==
# <code>DIRECTORY_FIND_LEAD_EXP='<Directory "\/var\/www\/cgi-bin">'</code> Special characters must be escaped with a backslash <code>\</code>.
# <code>[ ]*</code> This will match 0 or many whitespaces. Standard RegEx notation
# <code>sed -i "/${DIRECTORY_FIND_LEAD_EXP}/{</code> This will search the file, line-by-line, for <code>${DIRECTORY_FIND_LEAD_EXP}</code> <code><Directory "/var/www/cgi-bin"></code>. Note that the search pattern '''must not contain newline'''. If, and only if, sed finds  <code>${DIRECTORY_FIND_LEAD_EXP}</code> in the file, then it will proceed with executing the sub-code within the curly braces <code>{}</code>. It will start from the pattern matching line of file.
# <code>N;N;N;N;N;</code> '''N''' tells sed to read the next line after the pattern and attach it to current line. It is important to understand that sed is designed to replace only 1 line at a time, so N will basically force sed to read 2 lines and consider them as a single line with a single newline <code>\n</code> between the two. The newline in the second line will be ignored. By chaining '''5 Ns''', we are instructing sed to read 6 lines of the file starting with the pattern matching line.
# <code>s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}|</code> Replace <code>${DIRECTORY_FIND_FULL_EXP}</code> with <code>${DIRECTORY_FILL_FULL_EXP}</code>. Note the presence of newlines in the variables. Understanding this part will take some trial and error.
==References==
{|
| valign="top" |
* [https://unix.stackexchange.com/questions/26284/ Replace a Multi-Line String Using <code>sed</code>]
* [https://serverfault.com/questions/283129/ SSH Connection Hang Forever]
* [https://en.wikipedia.org/wiki/Regular_expression Regular Expression]
| valign="top" |
| valign="top" |
|-
| colspan="3" |
----
|-
| valign="top" |
| valign="top" |
| valign="top" |


|}
|}

Latest revision as of 08:04, 24 December 2022

Example

mkdir -p ${HOME}/Documents/sed_playground
tee   -a ${HOME}/Documents/sed_playground/000-default.conf >/dev/null <<EOF
#
#This is some test comments
#    Skip this
#

<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options +ExecCGI
    AddHandler cgi-script .cgi .pl
    Require all granted
</Directory>
EOF

APACHE_CONF_FILE="${HOME}/Documents/sed_playground/000-default.conf"
DIRECTORY_FIND_LEAD_EXP='<Directory "\/var\/www\/cgi-bin">'

DIRECTORY_FIND_FULL_EXP=$(cat <<EOF
${DIRECTORY_FIND_LEAD_EXP}\n\
[ ]*AllowOverride None\n\
[ ]*Options +ExecCGI\n\
[ ]*AddHandler cgi-script \.cgi \.pl\n\
[ ]*Require all granted\n\
<\/Directory>
EOF
)
DIRECTORY_FILL_FULL_EXP=$(cat <<EOF
#<Directory "\/var\/www\/cgi-bin">\n\
    #AllowOverride None\n\
    #Options +ExecCGI\n\
    #AddHandler cgi-script \.cgi \.pl\n\
    #Require all granted\n\
#<\/Directory>
EOF
)

Complex:

echo "complex"
echo "sedding"
sed -i "/${DIRECTORY_FIND_LEAD_EXP}/{N;N;N;N;N;s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}|}" ${APACHE_CONF_FILE}

Simple:

echo "simple"
echo "sedding"
sed -z "s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}|" -i ${APACHE_CONF_FILE}

${HOME}/Documents/sed_playground/000-default.conf
#
#This is some test comments
#    Skip this
#

<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options +ExecCGI
    AddHandler cgi-script .cgi .pl
    Require all granted
</Directory>
${HOME}/Documents/sed_playground/000-default.conf
#
#This is some test comments
#    Skip this
#

#<Directory "/var/www/cgi-bin">
    #AllowOverride None
    #Options +ExecCGI
    #AddHandler cgi-script .cgi .pl
    #Require all granted
#</Directory>

  • sed separate | used to instead of / to avoid escaping of / and .
  • EOF block used to avoid escaping of "
  • Lets have a look:

DIRECTORY_FIND_FULL_EXP=$(cat <<EOF
<Directory "/var/www/cgi-bin">\n\
[ ]*AllowOverride None\n\
[ ]*Options +ExecCGI\n\
[ ]*AddHandler cgi-script .cgi .pl\n\
[ ]*Require all granted\n\
</Directory>
EOF
)
DIRECTORY_FILL_FULL_EXP=$(cat <<EOF
#<Directory "/var/www/cgi-bin">\n\
    #AllowOverride None\n\
    #Options +ExecCGI\n\
    #AddHandler cgi-script .cgi .pl\n\
    #Require all granted\n\
#</Directory>
EOF
)

Explanation

  1. DIRECTORY_FIND_LEAD_EXP='<Directory "\/var\/www\/cgi-bin">' Special characters must be escaped with a backslash \.
  2. [ ]* This will match 0 or many whitespaces. Standard RegEx notation
  3. sed -i "/${DIRECTORY_FIND_LEAD_EXP}/{ This will search the file, line-by-line, for ${DIRECTORY_FIND_LEAD_EXP} <Directory "/var/www/cgi-bin">. Note that the search pattern must not contain newline. If, and only if, sed finds ${DIRECTORY_FIND_LEAD_EXP} in the file, then it will proceed with executing the sub-code within the curly braces {}. It will start from the pattern matching line of file.
  4. N;N;N;N;N; N tells sed to read the next line after the pattern and attach it to current line. It is important to understand that sed is designed to replace only 1 line at a time, so N will basically force sed to read 2 lines and consider them as a single line with a single newline \n between the two. The newline in the second line will be ignored. By chaining 5 Ns, we are instructing sed to read 6 lines of the file starting with the pattern matching line.
  5. s|${DIRECTORY_FIND_FULL_EXP}|${DIRECTORY_FILL_FULL_EXP}| Replace ${DIRECTORY_FIND_FULL_EXP} with ${DIRECTORY_FILL_FULL_EXP}. Note the presence of newlines in the variables. Understanding this part will take some trial and error.

References