[defensive-coding-guide] 120/120: GNUTLS: Document that the
pitfalls have been addressed
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 9eb72b454b83441329e9c84eb88041140d201ea3
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Thu Nov 13 12:24:37 2014 +0100
GNUTLS: Document that the pitfalls have been addressed
Suggested by Nikos Mavrogiannopoulos.
---
defensive-coding/en-US/Features-TLS.xml | 85 ++++++++++++++++++++-------------
1 file changed, 51 insertions(+), 34 deletions(-)
diff --git a/defensive-coding/en-US/Features-TLS.xml b/defensive-coding/en-US/Features-TLS.xml
index 5d9e39d..a2c0afd 100644
--- a/defensive-coding/en-US/Features-TLS.xml
+++ b/defensive-coding/en-US/Features-TLS.xml
@@ -215,41 +215,58 @@
<section id="sect-Defensive_Coding-TLS-Pitfalls-GNUTLS">
<title>GNUTLS Pitfalls</title>
<para>
- <filename>libgnutls.so.26</filename> links to
- <filename>libpthread.so.0</filename>. Loading the threading
- library too late causes problems, so the main program should
- be linked with <literal>-lpthread</literal> as well. As a
- result, it can be difficult to use GNUTLS in a plugin which is
- loaded with the <function>dlopen</function> function. Another
- side effect is that applications which merely link against
- GNUTLS (even without actually using it) may incur a
- substantial overhead because other libraries automatically
- switch to thread-safe algorithms.
- </para>
- <para>
- The <function>gnutls_global_init</function> function must be
- called before using any functionality provided by the library.
- This function is not thread-safe, so external locking is
- required, but it is not clear which lock should be used.
- Omitting the synchronization does not just lead to a memory
- leak, as it is suggested in the GNUTLS documentation, but to
- undefined behavior because there is no barrier that would
- enforce memory ordering.
- </para>
- <para>
- The <function>gnutls_global_deinit</function> function does
- not actually deallocate all resources allocated by
- <function>gnutls_global_init</function>. It is currently not
- thread-safe. Therefore, it is best to avoid calling it
- altogether.
- </para>
- <para>
- The X.509 implementation in GNUTLS is rather lenient. For
- example, it is possible to create and process X.509
- version 1 certificates which carry extensions. These
- certificates are (correctly) rejected by other
- implementations.
+ Older versions of GNUTLS had several peculiarities. As of
+ GNUTLS 3.3.10, they have been addressed, so these are only a
+ concern for applications which have to run with older GNUTLS
+ versions.
</para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ The dynamic shared object provided by GNTULS links to
+ <filename>libpthread.so.0</filename>. Loading the
+ threading library too late causes problems, so the main
+ program should be linked with <literal>-lpthread</literal>
+ as well. As a result, it can be difficult to use GNUTLS
+ in a plugin which is loaded with the
+ <function>dlopen</function> function. Another side effect
+ is that applications which merely link against GNUTLS
+ (even without actually using it) may incur a substantial
+ overhead because other libraries automatically switch to
+ thread-safe algorithms.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The <function>gnutls_global_init</function> function must
+ be called before using any functionality provided by the
+ library. This function is not thread-safe, so external
+ locking is required, but it is not clear which lock should
+ be used. Omitting the synchronization does not just lead
+ to a memory leak, as it is suggested in the GNUTLS
+ documentation, but to undefined behavior because there is
+ no barrier that would enforce memory ordering.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The <function>gnutls_global_deinit</function> function
+ does not actually deallocate all resources allocated by
+ <function>gnutls_global_init</function>. It is currently
+ not thread-safe. Therefore, it is best to avoid calling
+ it altogether.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The X.509 implementation in GNUTLS is rather lenient. For
+ example, it is possible to create and process X.509
+ version 1 certificates which carry extensions. These
+ certificates are (correctly) rejected by other
+ implementations.
+ </para>
+ </listitem>
+ </itemizedlist>
</section>
<section id="sect-Defensive_Coding-TLS-Pitfalls-OpenJDK">
<title>OpenJDK Pitfalls</title>
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 119/120: Update revision history in
preparation of publication
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 0ef43f912113fd6cccc4b5d7b19066d83ce2aee6
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Mon Oct 13 09:54:53 2014 +0200
Update revision history in preparation of publication
---
defensive-coding/en-US/Revision_History.xml | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/defensive-coding/en-US/Revision_History.xml b/defensive-coding/en-US/Revision_History.xml
index f0f0c78..e2dfe12 100644
--- a/defensive-coding/en-US/Revision_History.xml
+++ b/defensive-coding/en-US/Revision_History.xml
@@ -8,6 +8,21 @@
<simpara>
<revhistory>
<revision>
+ <revnumber>1.3-1</revnumber>
+ <date>Mon Oct 13 2014</date>
+ <author>
+ <firstname>Florian</firstname>
+ <surname>Weimer</surname>
+ <email>fweimer(a)redhat.com</email>
+ </author>
+ <revdescription>
+ <simplelist>
+ <member>Go: Mention default value handling in deserialization</member>
+ <member>Shell: New chapter</member>
+ </simplelist>
+ </revdescription>
+ </revision>
+ <revision>
<revnumber>1.2-1</revnumber>
<date>Wed Jul 16 2014</date>
<author>
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 118/120: Shell: Use a snippet for the
input validation example
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 0c1d3d46838c1427d17cadabf4000444bb614046
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Mon Oct 13 09:51:42 2014 +0200
Shell: Use a snippet for the input validation example
Add self-tests to the snippet code. Mention that this construct is
bash-specific.
Fixes the broken regular expression spotted by Eric Blake.
---
defensive-coding/en-US/Shell.xml | 27 +++++++-------
.../en-US/snippets/Shell-Input_Validation.xml | 12 +++++++
defensive-coding/src/Shell-Input_Validation.sh | 41 ++++++++++++++++++++++
3 files changed, 66 insertions(+), 14 deletions(-)
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
index f889dc1..d6a9465 100644
--- a/defensive-coding/en-US/Shell.xml
+++ b/defensive-coding/en-US/Shell.xml
@@ -398,23 +398,22 @@ trap cleanup 0
linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
</para>
<para>
- The following construct can be used to check if a string
- “<literal>$value</literal>” is an integer.
+ <xref linkend="ex-Defensive_Coding-Shell-Input_Validation"/>
+ shows a construct which can be used to check if a string
+ “<literal>$value</literal>” is an integer. This construct is
+ specific to <application>bash</application> and not portable to
+ POSIX shells.
</para>
- <informalexample>
- <programlisting language="Bash">
-if [[ $value =~ ^-?[0-9]$ ]] ; then
- echo value is an integer
-else
- echo "value is not an integer" 1>&2
- exit 1
-fi
- </programlisting>
- </informalexample>
+ <example id="ex-Defensive_Coding-Shell-Input_Validation">
+ <title>Input validation in <application>bash</application></title>
+ <xi:include href="snippets/Shell-Input_Validation.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+ </example>
<para>
Using <literal>case</literal> statements for input validation is
- also possible, but the pattern language is more restrictive, and
- it can be difficult to write suitable patterns.
+ also possible and supported by other (POSIX) shells, but the
+ pattern language is more restrictive, and it can be difficult to
+ write suitable patterns.
</para>
<para>
The <literal>expr</literal> external command can give misleading
diff --git a/defensive-coding/en-US/snippets/Shell-Input_Validation.xml b/defensive-coding/en-US/snippets/Shell-Input_Validation.xml
new file mode 100644
index 0000000..61cb7d1
--- /dev/null
+++ b/defensive-coding/en-US/snippets/Shell-Input_Validation.xml
@@ -0,0 +1,12 @@
+<?xml version='1.0' encoding='utf-8' ?>
+<!DOCTYPE programlisting PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+]>
+<!-- Automatically generated file. Do not edit. -->
+<programlisting language="Bash">
+if [[ $value =~ ^-?[0-9]+$ ]] ; then
+ echo value is an integer
+else
+ echo "value is not an integer" 1>&2
+ exit 1
+fi
+</programlisting>
diff --git a/defensive-coding/src/Shell-Input_Validation.sh b/defensive-coding/src/Shell-Input_Validation.sh
new file mode 100644
index 0000000..2b86a49
--- /dev/null
+++ b/defensive-coding/src/Shell-Input_Validation.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+validate () {
+ local value="$1"
+ #+ Shell Input_Validation
+ if [[ $value =~ ^-?[0-9]+$ ]] ; then
+ echo value is an integer
+ else
+ echo "value is not an integer" 1>&2
+ exit 1
+ fi
+ #-
+}
+
+check_validate () {
+ local value="$1"
+ local expected="$2"
+ (
+ validate "$value"
+ ) >/dev/null 2>/dev/null
+ result="$?"
+ if ! test "$result" -eq "$expected" ; then
+ echo "failure: validate \"$value\" $expected -> got $result"
+ fi
+}
+
+check_validate "" 1
+check_validate "0" 0
+check_validate "9" 0
+check_validate "-0" 0
+check_validate "-9" 0
+check_validate "10" 0
+check_validate "19" 0
+check_validate "-10" 0
+check_validate "-19" 0
+check_validate " 0" 1
+check_validate "--1" 1
+check_validate "1-" 1
+check_validate "1 || 0" 1
+check_validate '1$(kill -9 $PPID)' 1
+check_validate '2$(id)' 1
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 117/120: Add support for shell snippets
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 0026cc05cf2568357e2d8d91aa8da1f1e4b84bfa
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Mon Oct 13 09:49:48 2014 +0200
Add support for shell snippets
---
defensive-coding/Makefile | 2 +-
defensive-coding/scripts/split-snippets.py | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/defensive-coding/Makefile b/defensive-coding/Makefile
index 2090dad..f57f808 100644
--- a/defensive-coding/Makefile
+++ b/defensive-coding/Makefile
@@ -9,7 +9,7 @@ build: build-src build-manual
build-snippets:
mkdir -p en-US/snippets
python scripts/split-snippets.py . \
- src/*.c src/*.cpp src/*.java src/*.py src/*.go
+ src/*.c src/*.cpp src/*.java src/*.py src/*.go src/*.sh
build-manual: build-snippets
publican build --formats=html,epub,pdf --langs=en-US
diff --git a/defensive-coding/scripts/split-snippets.py b/defensive-coding/scripts/split-snippets.py
index 7244ced..6f87b6e 100644
--- a/defensive-coding/scripts/split-snippets.py
+++ b/defensive-coding/scripts/split-snippets.py
@@ -34,6 +34,7 @@ def extension_to_language(path, map={
'c' : 'C',
'py' : 'Python',
'java' : 'Java',
+ 'sh' : 'Bash',
}):
return map.get(path.split('.')[-1], 'C')
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 116/120: Shell: Fix internal reference
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit b7ec6fc7882d999c57ce47fc0b667c2c96647c7c
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Mon Oct 13 09:34:16 2014 +0200
Shell: Fix internal reference
Spotted by Kamil Dudka.
Also use "double expansion" consistently.
---
defensive-coding/en-US/Shell.xml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
index 042ac61..f889dc1 100644
--- a/defensive-coding/en-US/Shell.xml
+++ b/defensive-coding/en-US/Shell.xml
@@ -93,7 +93,7 @@ external-program "$arg1" "$arg2"
shell scripts difficult.
</para>
<para>
- Double evaluation can be requested explicitly with the
+ Double expansion can be requested explicitly with the
<literal>eval</literal> built-in command, or by invoking a
subshell with “<literal>bash -c</literal>”. These constructs
should not be used.
@@ -108,8 +108,8 @@ external-program "$arg1" "$arg2"
<emphasis>Arithmetic evaluation</emphasis> is a process by which
the shell computes the integer value of an expression specified
as a string. It is highly problematic for two reasons: It
- triggers double evaluation (see <xref
- linkend="sect-Defensive_Coding-Shell-Arithmetic"/>), and the
+ triggers double expansion (see <xref
+ linkend="sect-Defensive_Coding-Shell-Double_Expansion"/>), and the
language of arithmetic expressions is not self-contained. Some
constructs in arithmetic expressions (notably array subscripts)
provide a trapdoor from the restricted language of arithmetic
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 115/120: Shell: Update section on input
validation
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit e23c38377538e4c9f0311347b6fc15b8c1dddd37
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Fri Oct 10 16:44:53 2014 +0200
Shell: Update section on input validation
Also mention safety of [[ $var =~ regexp ]].
---
defensive-coding/en-US/Shell.xml | 36 +++++++++++++++++++++---------------
1 file changed, 21 insertions(+), 15 deletions(-)
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
index 24554b1..042ac61 100644
--- a/defensive-coding/en-US/Shell.xml
+++ b/defensive-coding/en-US/Shell.xml
@@ -162,6 +162,14 @@ external-program "$arg1" "$arg2"
evaluation, even with integer operators such as
<literal>-eq</literal>.)
</para>
+ <para>
+ The conditional expression
+ “<literal>[[ $</literal><emphasis>variable</emphasis><literal> =~ </literal><emphasis>regexp</emphasis><literal> ]]</literal>”
+ can be used for input validation, assuming that
+ <emphasis>regexp</emphasis> is a constant regular
+ expression.
+ See <xref linkend="sect-Defensive_Coding-Shell-Input_Validation"/>.
+ </para>
</listitem>
<listitem>
<para>
@@ -391,29 +399,27 @@ trap cleanup 0
</para>
<para>
The following construct can be used to check if a string
- “<literal>$value</literal>” is not a non-negative integer.
+ “<literal>$value</literal>” is an integer.
</para>
<informalexample>
<programlisting language="Bash">
-case "$value" in
- *[!0-9]*)
- echo "invalid input value" 1>&2
- exit 1
- ;;
-esac
+if [[ $value =~ ^-?[0-9]$ ]] ; then
+ echo value is an integer
+else
+ echo "value is not an integer" 1>&2
+ exit 1
+fi
</programlisting>
</informalexample>
<para>
- The pattern “<literal>*[!0-9]*</literal>” is not special shell
- syntax—it matches any string which contains arbitrary characters,
- followed by a non-digit, followed by arbitrary characters.
+ Using <literal>case</literal> statements for input validation is
+ also possible, but the pattern language is more restrictive, and
+ it can be difficult to write suitable patterns.
</para>
<para>
- Using <literal>case</literal> statements is the most reliable way
- for performing input validation, although constructing proper
- patterns is difficult. The <literal>expr</literal> external
- command and the built-in operator <literal>=~</literal> can give
- misleading results.
+ The <literal>expr</literal> external command can give misleading
+ results (e.g., if the value being checked contains operators
+ itself) and should not be used.
</para>
</section>
<section id="sect-Defensive_Coding-Shell-Edit_Guard">
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 114/120: Shell: Example code for
recognizing non-negative integers
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 01cf74aac034286d6dd3a341dcaab9cabcdb938f
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Fri Oct 10 16:19:15 2014 +0200
Shell: Example code for recognizing non-negative integers
---
defensive-coding/en-US/Shell.xml | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
index 9c9ec1f..24554b1 100644
--- a/defensive-coding/en-US/Shell.xml
+++ b/defensive-coding/en-US/Shell.xml
@@ -203,6 +203,11 @@ external-program "$arg1" "$arg2"
linkend="sect-Defensive_Coding-Shell-Alternatives"/>.
</para>
</important>
+ <para>
+ If it is impossible to avoid shell arithmetic on untrusted
+ inputs, refer to <xref
+ linkend="sect-Defensive_Coding-Shell-Input_Validation"/>.
+ </para>
</section>
<section id="sect-Defensive_Coding-Shell-Types">
<title>Type declarations</title>
@@ -376,6 +381,41 @@ trap cleanup 0
</informalexample>
</example>
</section>
+<section id="sect-Defensive_Coding-Shell-Input_Validation">
+ <title>Performing input validation</title>
+ <para>
+ In some cases, input validation cannot be avoided. For example,
+ if arithmetic evaluation is absolutely required, it is imperative
+ to check that input values are, in fact, integers. See <xref
+ linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
+ </para>
+ <para>
+ The following construct can be used to check if a string
+ “<literal>$value</literal>” is not a non-negative integer.
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+case "$value" in
+ *[!0-9]*)
+ echo "invalid input value" 1>&2
+ exit 1
+ ;;
+esac
+ </programlisting>
+ </informalexample>
+ <para>
+ The pattern “<literal>*[!0-9]*</literal>” is not special shell
+ syntax—it matches any string which contains arbitrary characters,
+ followed by a non-digit, followed by arbitrary characters.
+ </para>
+ <para>
+ Using <literal>case</literal> statements is the most reliable way
+ for performing input validation, although constructing proper
+ patterns is difficult. The <literal>expr</literal> external
+ command and the built-in operator <literal>=~</literal> can give
+ misleading results.
+ </para>
+</section>
<section id="sect-Defensive_Coding-Shell-Edit_Guard">
<title>Guarding shell scripts against changes</title>
<para>
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 113/120: Add a chapter on shell programming
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 2a829115ffd64e3ce6c6a281498f8a0006a77af3
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Fri Oct 10 15:36:28 2014 +0200
Add a chapter on shell programming
---
defensive-coding/en-US/Defensive_Coding.xml | 1 +
defensive-coding/en-US/Shell.xml | 409 ++++++++++++++++++++++++++++
2 files changed, 410 insertions(+)
diff --git a/defensive-coding/en-US/Defensive_Coding.xml b/defensive-coding/en-US/Defensive_Coding.xml
index ee96c8d..58d9121 100644
--- a/defensive-coding/en-US/Defensive_Coding.xml
+++ b/defensive-coding/en-US/Defensive_Coding.xml
@@ -8,6 +8,7 @@
<xi:include href="CXX.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Java.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Python.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
+ <xi:include href="Shell.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Go.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Vala.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
</part>
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
new file mode 100644
index 0000000..9c9ec1f
--- /dev/null
+++ b/defensive-coding/en-US/Shell.xml
@@ -0,0 +1,409 @@
+<?xml version='1.0' encoding='utf-8' ?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+]>
+<chapter id="chap-Defensive_Coding-Shell">
+<title>Shell Programming and <application>bash</application></title>
+<para>
+ This chapter contains advice about shell programming, specifically
+ in <application>bash</application>. Most of the advice will apply
+ to scripts written for other shells because extensions such as
+ integer or array variables have been implemented there as well, with
+ comparable syntax.
+</para>
+<section id="sect-Defensive_Coding-Shell-Alternatives">
+ <title>Consider alternatives</title>
+ <para>
+ Once a shell script is so complex that advice in this chapter
+ applies, it is time to step back and consider the question: Is
+ there a more suitable implementation language available?
+ </para>
+ <para>
+ For example, Python with its <literal>subprocess</literal> module
+ can be used to write scripts which are almost as concise as shell
+ scripts when it comes to invoking external programs, and Python
+ offers richer data structures, with less arcane syntax and more
+ consistent behavior.
+ </para>
+</section>
+<section id="sect-Defensive_Coding-Shell-Language">
+<title>Shell language features</title>
+<para>
+ The following sections cover subtleties concerning the shell
+ programming languages. They have been written with the
+ <application>bash</application> shell in mind, but some of these
+ features apply to other shells as well.
+</para>
+<para>
+ Some of the features described may seem like implementation defects,
+ but these features have been replicated across multiple independent
+ implementations, so they now have to be considered part of the shell
+ programming language.
+</para>
+<section id="sect-Defensive_Coding-Shell-Parameter_Expansion">
+ <title>Parameter expansion</title>
+ <para>
+ The mechanism by which named shell variables and parameters are
+ expanded is called <emphasis>parameter expansion</emphasis>. The
+ most basic syntax is
+ “<literal>$</literal><emphasis>variable</emphasis>” or
+ “<literal>${</literal><emphasis>variable</emphasis><literal>}</literal>”.
+ </para>
+ <para>
+ In almost all cases, a parameter expansion should be enclosed in
+ double quotation marks <literal>"</literal>…<literal>"</literal>.
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+external-program "$arg1" "$arg2"
+ </programlisting>
+ </informalexample>
+ <para>
+ If the double quotation marks are omitted, the value of the
+ variable will be split according to the current value of the
+ <envar>IFS</envar> variable. This may allow the injection of
+ additional options which are then processed by
+ <literal>external-program</literal>.
+ </para>
+ <para>
+ Parameter expansion can use special syntax for specific features,
+ such as substituting defaults or performing string or array
+ operations. These constructs should not be used because they can
+ trigger arithmetic evaluation, which can result in code execution.
+ See <xref linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
+ </para>
+</section>
+<section id="sect-Defensive_Coding-Shell-Double_Expansion">
+ <title>Double expansion</title>
+ <para>
+ <emphasis>Double expansion</emphasis> occurs when, during the
+ expansion of a shell variable, not just the variable is expanded,
+ replacing it by its value, but the <emphasis>value</emphasis> of
+ the variable is itself is expanded as well. This can trigger
+ arbitrary code execution, unless the value of the variable is
+ verified against a restrictive pattern.
+ </para>
+ <para>
+ The evaluation process is in fact recursive, so a self-referential
+ expression can cause an out-of-memory condition and a shell crash.
+ </para>
+ <para>
+ Double expansion may seem like as a defect, but it is implemented
+ by many shells, and has to be considered an integral part of the
+ shell programming language. However, it does make writing robust
+ shell scripts difficult.
+ </para>
+ <para>
+ Double evaluation can be requested explicitly with the
+ <literal>eval</literal> built-in command, or by invoking a
+ subshell with “<literal>bash -c</literal>”. These constructs
+ should not be used.
+ </para>
+ <para>
+ The following sections give examples of places where implicit
+ double expansion occurs.
+ </para>
+ <section id="sect-Defensive_Coding-Shell-Arithmetic">
+ <title>Arithmetic evaluation</title>
+ <para>
+ <emphasis>Arithmetic evaluation</emphasis> is a process by which
+ the shell computes the integer value of an expression specified
+ as a string. It is highly problematic for two reasons: It
+ triggers double evaluation (see <xref
+ linkend="sect-Defensive_Coding-Shell-Arithmetic"/>), and the
+ language of arithmetic expressions is not self-contained. Some
+ constructs in arithmetic expressions (notably array subscripts)
+ provide a trapdoor from the restricted language of arithmetic
+ expressions to the full shell language, thus paving the way
+ towards arbitrary code execution. Due to double expansion,
+ input which is (indirectly) referenced from an arithmetic
+ expression can trigger execution of arbitrary code, which is
+ potentially harmful.
+ </para>
+ <para>
+ Arithmetic evaluation is triggered by the follow constructs:
+ </para>
+ <!-- The list was constructed by looking at the bash sources and
+ search for the string "expand_". -->
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>expression</emphasis> in
+ “<literal>$((</literal><emphasis>expression</emphasis><literal>))</literal>”
+ is evaluated. This construct is called <emphasis>arithmetic
+ expansion</emphasis>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ “<literal>$[</literal><emphasis>expression</emphasis><literal>]</literal>”
+ is a deprecated syntax with the same effect.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The arguments to the <literal>let</literal> shell built-in
+ are evaluated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ “<literal>((</literal><emphasis>expression</emphasis><literal>))</literal>”
+ is an alternative syntax for “<literal>let
+ </literal><emphasis>expression</emphasis>”.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Conditional expressions surrounded by
+ “<literal>[[</literal>…<literal>]]</literal>” can trigger
+ arithmetic evaluation if certain operators such as
+ <literal>-eq</literal> are used. (The
+ <literal>test</literal> built-in does not perform arithmetic
+ evaluation, even with integer operators such as
+ <literal>-eq</literal>.)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Certain parameter expansions, for example
+ “<literal>${</literal><emphasis>variable</emphasis><literal>[</literal><emphasis>expression</emphasis><literal>]}</literal>”
+ (array indexing) or
+ “<literal>${</literal><emphasis>variable</emphasis><literal>:</literal><emphasis>expression</emphasis><literal>}</literal>”
+ (string slicing), trigger arithmetic evaluation of
+ <emphasis>expression</emphasis>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Assignment to array elements using
+ “<emphasis>array_variable</emphasis><literal>[</literal><emphasis>subscript</emphasis><literal>]=</literal><emphasis>expression</emphasis>”
+ triggers evaluation of <emphasis>subscript</emphasis>, but
+ not <emphasis>expression</emphasis>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The expressions in the arithmetic <literal>for</literal>
+ command,
+ “<literal>for ((</literal><emphasis>expression1</emphasis><literal>; </literal><emphasis>expression2</emphasis><literal>; </literal><emphasis>expression3</emphasis><literal>)); do </literal><emphasis>commands</emphasis><literal>; done</literal>”
+ are evaluated. This does not apply to the regular
+ for command,
+ “<literal>for </literal><emphasis>variable</emphasis><literal> in </literal><emphasis>list</emphasis><literal>; do </literal><emphasis>commands</emphasis><literal>; done</literal>”.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <important>
+ <para>
+ Depending on the <application>bash</application> version, the
+ above list may be incomplete.
+ </para>
+ <para>
+ If faced with a situation where using such shell features
+ appears necessary, see <xref
+ linkend="sect-Defensive_Coding-Shell-Alternatives"/>.
+ </para>
+ </important>
+ </section>
+ <section id="sect-Defensive_Coding-Shell-Types">
+ <title>Type declarations</title>
+ <para>
+ <application>bash</application> supports explicit type
+ declarations for shell variables:
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+ declare -i integer_variable
+ declare -a array_variable
+ declare -A assoc_array_variable
+
+ typeset -i integer_variable
+ typeset -a array_variable
+ typeset -A assoc_array_variable
+
+ local -i integer_variable
+ local -a array_variable
+ local -A assoc_array_variable
+
+ readonly -i integer_variable
+ readonly -a array_variable
+ readonly -A assoc_array_variable
+ </programlisting>
+ </informalexample>
+ <para>
+ Variables can also be declared as arrays by assigning them an
+ array expression, as in:
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+array_variable=(1 2 3 4)
+ </programlisting>
+ </informalexample>
+ <para>
+ Some built-ins (such as <literal>mapfile</literal>) can
+ implicitly create array variables.
+ </para>
+ <para>
+ Such type declarations should not be used because assignment to
+ such variables (independent of the concrete syntax used for the
+ assignment) triggers arithmetic expansion (and thus double
+ expansion) of the right-hand side of the assignment operation.
+ See <xref linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
+ </para>
+ <para>
+ Shell scripts which use integer or array variables should be
+ rewritten in another, more suitable language. Se <xref
+ linkend="sect-Defensive_Coding-Shell-Alternatives"/>.
+ </para>
+ </section>
+</section>
+<section id="sect-Defensive_Coding-Shell-Obscure">
+ <title>Other obscurities</title>
+ <para>
+ Obscure shell language features should not be used. Examples are:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Exported functions (<literal>export -f</literal> or
+ <literal>declare -f</literal>).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Function names which are not valid variable names, such as
+ “<literal>module::function</literal>”.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The possibility to override built-ins or external commands
+ with shell functions.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Changing the value of the <envar>IFS</envar> variable to
+ tokenize strings.
+ </para>
+ </listitem>
+ </itemizedlist>
+</section>
+</section>
+<section id="sect-Defensive_Coding-Shell-Invoke">
+<title>Invoking external commands</title>
+<para>
+ When passing shell variables as single command line arguments,
+ they should always be surrounded by double quotes. See
+ <xref linkend="sect-Defensive_Coding-Shell-Parameter_Expansion"/>.
+</para>
+<para>
+ Care is required when passing untrusted values as positional
+ parameters to external commands. If the value starts with a hyphen
+ “<literal>-</literal>”, it may be interpreted by the external
+ command as an option. Depending on the external program, a
+ “<literal>--</literal>” argument stops option processing and treats
+ all following arguments as positional parameters. (Double quotes
+ are completely invisible to the command being invoked, so they do
+ not prevent variable values from being interpreted as options.)
+</para>
+<para>
+ Cleaning the environment before invoking child processes is
+ difficult to implement in script. <application>bash</application>
+ keeps a hidden list of environment variables which do not correspond
+ to shell variables, and unsetting them from within a
+ <application>bash</application> script is not possible. To reset
+ the environment, a script can re-run itself under the “<literal>env
+ -i</literal>” command with an additional parameter which indicates
+ the environment has been cleared and suppresses a further
+ self-execution. Alternatively, individual commands can be executed
+ with “<literal>env -i</literal>”.
+</para>
+<important>
+ <para>
+ Completely isolation from its original execution environment
+ (which is required when the script is executed after a trust
+ transition, e.g., triggered by the SUID mechanism) is impossible
+ to achieve from within the shell script itself. Instead, the
+ invoking process has to clear the process environment (except for
+ few trusted variables) before running the shell script.
+ </para>
+</important>
+<para>
+ Checking for failures in executed external commands is recommended.
+ If no elaborate error recovery is needed, invoking “<literal>set
+ -e</literal>” may be sufficient. This causes the script to stop on
+ the first failed command. However, failures in pipes
+ (“<literal>command1 | command2</literal>”) are only detected for the
+ last command in the pipe, errors in previous commands are ignored.
+ This can be changed by invoking “<literal>set -o pipefail</literal>”.
+ Due to architectural limitations, only the process that spawned
+ the entire pipe can check for failures in individual commands;
+ it is not possible for a process to tell if the process feeding
+ data (or the process consuming data) exited normally or with
+ an error.
+</para>
+<para>
+ See <xref linkend="sect-Defensive_Coding-Tasks-Processes-Creation"/>
+ for additional details on creating child processes.
+</para>
+</section>
+<section id="sect-Defensive_Coding-Shell-Temporary_Files">
+ <title>Temporary files</title>
+ <para>
+ Temporary files should be created with the
+ <literal>mktemp</literal> command, and temporary directories with
+ “<literal>mktemp -d</literal>”.
+ </para>
+ <para>
+ To clean up temporary files and directories, write a clean-up
+ shell function and register it as a trap handler, as shown in
+ <xref linkend="ex-Defensive_Coding-Tasks-Temporary_Files"/>.
+ Using a separate function avoids issues with proper quoting of
+ variables.
+ </para>
+ <example id="ex-Defensive_Coding-Tasks-Temporary_Files">
+ <title>Creating and cleaning up temporary files</title>
+ <informalexample>
+ <programlisting language="Bash">
+tmpfile="$(mktemp)"
+
+cleanup () {
+ rm -f -- "$tmpfile"
+}
+
+trap cleanup 0
+ </programlisting>
+ </informalexample>
+ </example>
+</section>
+<section id="sect-Defensive_Coding-Shell-Edit_Guard">
+ <title>Guarding shell scripts against changes</title>
+ <para>
+ <application>bash</application> only reads a shell script up to
+ the point it is needed for executed the next command. This means
+ that if script is overwritten while it is running, execution can
+ jump to a random part of the script, depending on what is modified
+ in the script and how the file offsets change as a result. (This
+ behavior is needed to support self-extracting shell archives whose
+ script part is followed by a stream of bytes which does not follow
+ the shell language syntax.)
+ </para>
+ <para>
+ Therefore, long-running scripts should be guarded against
+ concurrent modification by putting as much of the program logic
+ into a <literal>main</literal> function, and invoking the
+ <literal>main</literal> function at the end of the script, using
+ this syntax:
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+main "$@" ; exit $?
+ </programlisting>
+ </informalexample>
+ <para>
+ This construct ensures that <application>bash</application> will
+ stop execution after the <literal>main</literal> function, instead
+ of opening the script file and trying to read more commands.
+ </para>
+</section>
+</chapter>
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 112/120: Go: Fix typo
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit fab2049127ec0a116a15d1eff478ce14b31d15af
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Tue Oct 7 14:16:27 2014 +0200
Go: Fix typo
---
defensive-coding/en-US/Go.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/defensive-coding/en-US/Go.xml b/defensive-coding/en-US/Go.xml
index e1285ed..06098e7 100644
--- a/defensive-coding/en-US/Go.xml
+++ b/defensive-coding/en-US/Go.xml
@@ -22,7 +22,7 @@
Code which does not run in parallel and does not use the
<literal>unsafe</literal> package (or other packages which expose
unsafe constructs) is memory-safe. For example, invalid casts and
- out-of-range subscripting cause panics and run time.
+ out-of-range subscripting cause panics at run time.
</para>
<para>
Keep in mind that finalization can introduce parallelism because
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months
[defensive-coding-guide] 111/120: Go Marshaling: Fix section title
by git repository hosting
This is an automated email from the git hooks/post-receive script.
sparks pushed a commit to branch master
in repository defensive-coding-guide.
commit 00ae7f52e51702cdc856cd60ff1a34f4f5046c87
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Thu Aug 14 10:03:03 2014 +0200
Go Marshaling: Fix section title
---
defensive-coding/en-US/Go.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/defensive-coding/en-US/Go.xml b/defensive-coding/en-US/Go.xml
index b5529a6..e1285ed 100644
--- a/defensive-coding/en-US/Go.xml
+++ b/defensive-coding/en-US/Go.xml
@@ -88,7 +88,7 @@
</para>
</section>
<section id="chap-Defensive_Coding-Go-Marshaling">
- <title>Marshaling and marshaling</title>
+ <title>Marshaling and unmarshaling</title>
<para>
Several packages in the <literal>encoding</literal> hierarchy
provide support for serialization and deserialization. The usual
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
7 years, 9 months