Home » Do progress reports/logging information belong on stderr or stdout?

Do progress reports/logging information belong on stderr or stdout?

Solutons:


POSIX defines standard error as

for writing diagnostic output

This doesn’t limit its use to error messages only. I would consider progress information as diagnostic output, so it belongs on standard error.

Posix defines the standard streams thus:

At program start-up, three streams shall be predefined and need not be opened explicitly: standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). When opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.

The GNU C Library describes the standard streams similarly:

Variable: FILE * stdout
The standard output stream, which is used for normal output from the program.

Variable: FILE * stderr
The standard error stream, which is used for error messages and diagnostics issued by the program.

Thus, standard definitions have little guidance for stream usage beyond “conventional/normal output” and “diagnostic/error output.” In practice, it’s common to redirect either or both of these streams to files and pipelines, where progress indicators will be a problem. Some systems even monitor stderr for output and consider it a sign of problems. Purely auxiliary progress information is therefore problematic on either stream.

Instead of sending progress indicators unconditionally to either standard stream, it’s important to recognize that progress output is only appropriate for interactive streams. With that in mind, I recommend writing progress counters only after checking whether the stream is interactive (e.g., with isatty()) or when explicitly enabled by a command-line option. That’s especially important for progress meters that rely on terminal update behavior to make sense, like %-complete bars.

For certain very simple progress messages (“Starting X” … “Done with X”) it’s more reasonable to include the output even for non-interactive streams. In that case, consider how users might interact with the streams, like searching with grep or paging with less or monitoring with tail -f. If it makes sense to see the progress messages in those contexts, they will be much easier to consume from stdout.

POSIX is slightly more concrete about “diagnostic information” in Shell and Utilities, 1.4: Utility Description Defaults (emphasis mine):

STDERR

The STDERR section describes the standard error output of the utility.
Only those messages that are purposely sent by the utility are
described. Use of a terminal for standard error may cause any of the
standard utilities that write standard error output to stop when used
in the background. For this reason, applications should not use
interactive features in scripts to be placed in the background.

The format of diagnostic messages for most utilities is unspecified,
but the language and cultural conventions of diagnostic and
informative messages whose format is unspecified by this volume of
POSIX.1-2008 should be affected by the setting of LC_MESSAGES and
[XSI] [Option Start] NLSPATH. [Option End]

The specified standard error output of standard utilities shall not
depend on the existence or value of the environment variables defined
in this volume of POSIX.1-2008, except as provided by this volume of
POSIX.1-2008.

Default Behavior: When this section is listed as “The standard error shall be used only for diagnostic messages.”, it means that,
unless otherwise stated, the diagnostic messages shall be sent to
the standard error only when the exit status indicates that an error
occurred
and the utility is used as described by this volume of
POSIX.1-2008.

When this section is listed as “Not used.”, it means that the standard
error shall not be used when the utility is used as described in this
volume of POSIX.1-2008.

IANASL, but I interpret that to mean that stderr will have output only if the utility will return an error exit code. Since this should not be the normal course of events for successful execution, no progress information should be printed by a POSIX utility unless an error occurs (unless, of course, otherwise specified, etc.).

Related Solutions

Joining bash arguments into single string with spaces

[*] I believe that this does what you want. It will put all the arguments in one string, separated by spaces, with single quotes around all: str="'$*'" $* produces all the scripts arguments separated by the first character of $IFS which, by default, is a space....

AddTransient, AddScoped and AddSingleton Services Differences

TL;DR Transient objects are always different; a new instance is provided to every controller and every service. Scoped objects are the same within a request, but different across different requests. Singleton objects are the same for every object and every...

How to download package not install it with apt-get command?

Use --download-only: sudo apt-get install --download-only pppoe This will download pppoe and any dependencies you need, and place them in /var/cache/apt/archives. That way a subsequent apt-get install pppoe will be able to complete without any extra downloads....

What defines the maximum size for a command single argument?

Answers Definitely not a bug. The parameter which defines the maximum size for one argument is MAX_ARG_STRLEN. There is no documentation for this parameter other than the comments in binfmts.h: /* * These are the maximum length and maximum number of strings...

Bulk rename, change prefix

I'd say the simplest it to just use the rename command which is common on many Linux distributions. There are two common versions of this command so check its man page to find which one you have: ## rename from Perl (common in Debian systems -- Ubuntu, Mint,...

Output from ls has newlines but displays on a single line. Why?

When you pipe the output, ls acts differently. This fact is hidden away in the info documentation: If standard output is a terminal, the output is in columns (sorted vertically) and control characters are output as question marks; otherwise, the output is...

mv: Move file only if destination does not exist

mv -vn file1 file2. This command will do what you want. You can skip -v if you want. -v makes it verbose - mv will tell you that it moved file if it moves it(useful, since there is possibility that file will not be moved) -n moves only if file2 does not exist....

Is it possible to store and query JSON in SQLite?

SQLite 3.9 introduced a new extension (JSON1) that allows you to easily work with JSON data . Also, it introduced support for indexes on expressions, which (in my understanding) should allow you to define indexes on your JSON data as well. PostgreSQL has some...

Combining tail && journalctl

You could use: journalctl -u service-name -f -f, --follow Show only the most recent journal entries, and continuously print new entries as they are appended to the journal. Here I've added "service-name" to distinguish this answer from others; you substitute...

how can shellshock be exploited over SSH?

One example where this can be exploited is on servers with an authorized_keys forced command. When adding an entry to ~/.ssh/authorized_keys, you can prefix the line with command="foo" to force foo to be run any time that ssh public key is used. With this...

Why doesn’t the tilde (~) expand inside double quotes?

The reason, because inside double quotes, tilde ~ has no special meaning, it's treated as literal. POSIX defines Double-Quotes as: Enclosing characters in double-quotes ( "" ) shall preserve the literal value of all characters within the double-quotes, with the...

What is GNU Info for?

GNU Info was designed to offer documentation that was comprehensive, hyperlinked, and possible to output to multiple formats. Man pages were available, and they were great at providing printed output. However, they were designed such that each man page had a...

Set systemd service to execute after fstab mount

a CIFS network location is mounted via /etc/fstab to /mnt/ on boot-up. No, it is not. Get this right, and the rest falls into place naturally. The mount is handled by a (generated) systemd mount unit that will be named something like mnt-wibble.mount. You can...

Merge two video clips into one, placing them next to each other

To be honest, using the accepted answer resulted in a lot of dropped frames for me. However, using the hstack filter_complex produced perfectly fluid output: ffmpeg -i left.mp4 -i right.mp4 -filter_complex hstack output.mp4 ffmpeg -i input1.mp4 -i input2.mp4...

How portable are /dev/stdin, /dev/stdout and /dev/stderr?

It's been available on Linux back into its prehistory. It is not POSIX, although many actual shells (including AT&T ksh and bash) will simulate it if it's not present in the OS; note that this simulation only works at the shell level (i.e. redirection or...

How can I increase the number of inodes in an ext4 filesystem?

It seems that you have a lot more files than normal expectation. I don't know whether there is a solution to change the inode table size dynamically. I'm afraid that you need to back-up your data, and create new filesystem, and restore your data. To create new...

Why doesn’t cp have a progress bar like wget?

The tradition in unix tools is to display messages only if something goes wrong. I think this is both for design and practical reasons. The design is intended to make it obvious when something goes wrong: you get an error message, and it's not drowned in...