Exit Status and Bash

I'm @balupton, author of dorothy which is the largest public bash codebase that I am aware of.

Dorothy is coded using set -e (errexit) to avoid || return $? statements on every single one of its thousands of lines of code, for the most part this has worked well, however I was suprised when the down command would report a failure via:

• https://github.com/bevry/dorothy/blob/107a34dd01f73b8fc53aee032d7dffe5e7aee5b5/commands/down#L293-L307
• https://github.com/bevry/dorothy/blob/107a34dd01f73b8fc53aee032d7dffe5e7aee5b5/commands/eval-helper#L119

However it would also report a success within the called function:

• https://github.com/bevry/dorothy/blob/107a34dd01f73b8fc53aee032d7dffe5e7aee5b5/commands/down#L255-L280

Asking stackoverflow and asking the bug-bash mailing list revealed this pecularity with errexit was at fault:

The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in an if statement, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command’s return status is being inverted with !. https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html

1. When this option is on, when any command fails (for any of the reasons listed in Consequences of Shell Errors or by returning an exit status greater than zero), the shell immediately shall exit, as if by executing the exit special built-in utility with no arguments, with the following exceptions: The failure of any individual command in a multi-command pipeline shall not cause the shell to exit. Only the failure of the pipeline itself shall be considered.

2. The -e setting shall be ignored when executing the compound list following the while, until, if, or elif reserved word, a pipeline beginning with the ! reserved word, or any command of an AND-OR list other than the last.

3. If the exit status of a compound command other than a subshell command was the result of a failure while -e was being ignored, then -e shall not apply to this command.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#set

Which put another way means using any way of acting upon the exit status of a command, will permantly disable errexit on that command.

This is what caused the unexpected behaviour in my earlier code, eval-wrapper was detecting the exit status of the eval'd act function, but in turn, the act function was executing with errexit completely disabled! Not what I wanted!!!

This gist is my attempt to showcase all the techniques to demonstrate and workaround this issue, in a neat, composable, and definitive way. Licensed under the Attribution-ShareAlike 4.0 International (include credit when you copy and paste, submit your improvements).

TLDR: Show me the ultimate solution

Copy the eval_capture function (and its authorship/license header) from technique-09-trap+stdout+stderr+output+script.bash, listen to this song, and use it like so:

local status=0 stdout='' stderr=''
eval_capture [--statusvar=status] [--stdoutvar=stdout] [--stderrvar=stderr] [--] cmd ...

Reproducing the problem, and comparing techniques

I have attached each technique, and their outputs, inside this gist, so you don't have to run them locally, you can just scroll to them.

If you do wish to run the techniques locally:

1. Clone this repository and cd into it
2. Run chmod +x ./script.bash so you can run it
3. Copy and paste the technique you want to trial into script.bash at the top, replacing any prior techniques you copied and pasted in there
4. Run ./script.bash to see the result

While we are at it, be aware of another batch gotha that return is equivalent to return $? and should not be used if you actually want return 0.

What about capturing status and stdout?

One could have hoped for:

function capture_failure_and_output {
	log "$FUNCNAME: start"

	local status output
	output="$(errexit_workound --statusvar=status indirect_failure)"
	log "$FUNCNAME: \$? = $?, \$status = $status [this should report: 0 / 99]"
	echo "output=[$output]"

	log "$FUNCNAME: end"
}

However because $(this is a subshell) it produces the unexpected but correct result of:

capture_failure_and_output: $? = 0, $status =  [this should report: 0 / 99]
output=[14: indirect_failure: start
15: direct_failure: start]

Same issue with:

local status output
output="$(mktemp)"
errexit_workound --statusvar=status indirect_failure > >(tee "$output")
output="$(cat "$output")"
log "$FUNCNAME: \$? = $?, \$status = $status [this should report: 0 / 99]"
echo "output=[$output]"

Same issue with:

local status output=''
shopt -s lastpipe
errexit_workound --statusvar=status indirect_failure | read -r output
echo "output=[$output]"

However that one has a single line of output:

capture_failure_and_output: $? = 0, $status =  [this should report: 0 / 99]
output=[14: indirect_failure: start]

As such, we should update our methods with support for --stdoutvar=....

However, not to fear, I figured it out with technique-07-trap+stdout+script.bash which includes the script.bash code as well, due to modifications.

What about capturing status, stdout, and stderr?

Got you covered, check out technique-08-trap+stdout+stderr+script.bash

What about cpaturing status, stdout, stderr, and both stdout+stderr?

Got you covered, check out technique-09-trap+stdout+stderr+output+script.bash