xargsposix - Online in the Cloud

PROGRAM:

NAME

xargs — construct argument lists and invoke utility

SYNOPSIS

xargs [−ptx] [−E eofstr] [−I replstr|−L number|−n number]
[−s size] [utility [argument...]]

DESCRIPTION

The xargs utility shall construct a command line consisting of the utility and argument
operands specified followed by as many arguments read in sequence from standard input as
fit in length and number constraints specified by the options. The xargs utility shall
then invoke the constructed command line and wait for its completion. This sequence shall
be repeated until one of the following occurs:

* An end-of-file condition is detected on standard input.

* An argument consisting of just the logical end-of-file string (see the −E eofstr
option) is found on standard input after double-quote processing, <apostrophe>
processing, and <backslash>-escape processing (see next paragraph). All arguments up
to but not including the argument consisting of just the logical end-of-file string
shall be used as arguments in constructed command lines.

* An invocation of a constructed command line returns an exit status of 255.

The application shall ensure that arguments in the standard input are separated by
unquoted <blank> characters, unescaped <blank> characters, or <newline> characters. A
string of zero or more non-double-quote ('"') characters and non-<newline> characters can
be quoted by enclosing them in double-quotes. A string of zero or more non-<apostrophe>
('\'') characters and non-<newline> characters can be quoted by enclosing them in
<apostrophe> characters. Any unquoted character can be escaped by preceding it with a
<backslash>. The utility named by utility shall be executed one or more times until the
end-of-file is reached or the logical end-of file string is found. The results are
unspecified if the utility named by utility attempts to read from its standard input.

The generated command line length shall be the sum of the size in bytes of the utility
name and each argument treated as strings, including a null byte terminator for each of
these strings. The xargs utility shall limit the command line length such that when the
command line is invoked, the combined argument and environment lists (see the exec family
of functions in the System Interfaces volume of POSIX.1‐2008) shall not exceed
{ARG_MAX}−2048 bytes. Within this constraint, if neither the −n nor the −s option is
specified, the default command line length shall be at least {LINE_MAX}.

OPTIONS

The xargs utility shall conform to the Base Definitions volume of POSIX.1‐2008, Section
12.2, Utility Syntax Guidelines.

The following options shall be supported:

−E eofstr Use eofstr as the logical end-of-file string. If −E is not specified, it is
unspecified whether the logical end-of-file string is the <underscore> character
('_') or the end-of-file string capability is disabled. When eofstr is the null
string, the logical end-of-file string capability shall be disabled and
<underscore> characters shall be taken literally.

−I replstr
Insert mode: utility is executed for each logical line from standard input.
Arguments in the standard input shall be separated only by unescaped <newline>
characters, not by <blank> characters. Any unquoted unescaped <blank> characters
at the beginning of each line shall be ignored. The resulting argument shall be
inserted in arguments in place of each occurrence of replstr. At least five
arguments in arguments can each contain one or more instances of replstr. Each
of these constructed arguments cannot grow larger than an implementation-defined
limit greater than or equal to 255 bytes. Option −x shall be forced on.

−L number The utility shall be executed for each non-empty number lines of arguments from
standard input. The last invocation of utility shall be with fewer lines of
arguments if fewer than number remain. A line is considered to end with the
first <newline> unless the last character of the line is a <blank>; a trailing
<blank> signals continuation to the next non-empty line, inclusive.

−n number Invoke utility using as many standard input arguments as possible, up to number
(a positive decimal integer) arguments maximum. Fewer arguments shall be used
if:

* The command line length accumulated exceeds the size specified by the −s
option (or {LINE_MAX} if there is no −s option).

* The last iteration has fewer than number, but not zero, operands remaining.

−p Prompt mode: the user is asked whether to execute utility at each invocation.
Trace mode (−t) is turned on to write the command instance to be executed,
followed by a prompt to standard error. An affirmative response read from
/dev/tty shall execute the command; otherwise, that particular invocation of
utility shall be skipped.

−s size Invoke utility using as many standard input arguments as possible yielding a
command line length less than size (a positive decimal integer) bytes. Fewer
arguments shall be used if:

* The total number of arguments exceeds that specified by the −n option.

* The total number of lines exceeds that specified by the −L option.

* End-of-file is encountered on standard input before size bytes are
accumulated.

Values of size up to at least {LINE_MAX} bytes shall be supported, provided that
the constraints specified in the DESCRIPTION are met. It shall not be considered
an error if a value larger than that supported by the implementation or
exceeding the constraints specified in the DESCRIPTION is given; xargs shall use
the largest value it supports within the constraints.

−t Enable trace mode. Each generated command line shall be written to standard
error just prior to invocation.

−x Terminate if a constructed command line will not fit in the implied or specified
size (see the −s option above).

OPERANDS

The following operands shall be supported:

utility The name of the utility to be invoked, found by search path using the PATH
environment variable, described in the Base Definitions volume of POSIX.1‐2008,
Chapter 8, Environment Variables. If utility is omitted, the default shall be
the echo utility. If the utility operand names any of the special built-in
utilities in Section 2.14, Special Built-In Utilities, the results are
undefined.

argument An initial option or operand for the invocation of utility.

STDIN

The standard input shall be a text file. The results are unspecified if an end-of-file
condition is detected immediately following an escaped <newline>.

INPUT FILES

The file /dev/tty shall be used to read responses required by the −p option.

ENVIRONMENT VARIABLES

The following environment variables shall affect the execution of xargs:

LANG Provide a default value for the internationalization variables that are unset or
null. (See the Base Definitions volume of POSIX.1‐2008, Section 8.2,
Internationalization Variables for the precedence of internationalization
variables used to determine the values of locale categories.)

LC_ALL If set to a non-empty string value, override the values of all the other
internationalization variables.

LC_COLLATE
Determine the locale for the behavior of ranges, equivalence classes, and multi-
character collating elements used in the extended regular expression defined for
the yesexpr locale keyword in the LC_MESSAGES category.

LC_CTYPE Determine the locale for the interpretation of sequences of bytes of text data
as characters (for example, single-byte as opposed to multi-byte characters in
arguments and input files) and the behavior of character classes used in the
extended regular expression defined for the yesexpr locale keyword in the
LC_MESSAGES category.

LC_MESSAGES
Determine the locale used to process affirmative responses, and the locale used
to affect the format and contents of diagnostic messages and prompts written to
standard error.

NLSPATH Determine the location of message catalogs for the processing of LC_MESSAGES.

PATH Determine the location of utility, as described in the Base Definitions volume
of POSIX.1‐2008, Chapter 8, Environment Variables.

ASYNCHRONOUS EVENTS

Default.

STDOUT

Not used.

STDERR

The standard error shall be used for diagnostic messages and the −t and −p options. If the
−t option is specified, the utility and its constructed argument list shall be written to
standard error, as it will be invoked, prior to invocation. If −p is specified, a prompt
of the following format shall be written (in the POSIX locale):

"?..."

at the end of the line of the output from −t.

OUTPUT FILES

None.

EXTENDED DESCRIPTION

None.

EXIT STATUS

The following exit values shall be returned:

0 All invocations of utility returned exit status zero.

1‐125 A command line meeting the specified requirements could not be assembled, one or
more of the invocations of utility returned a non-zero exit status, or some other
error occurred.

126 The utility specified by utility was found but could not be invoked.

127 The utility specified by utility could not be found.

CONSEQUENCES OF ERRORS

If a command line meeting the specified requirements cannot be assembled, the utility
cannot be invoked, an invocation of the utility is terminated by a signal, or an
invocation of the utility exits with exit status 255, the xargs utility shall write a
diagnostic message and exit without processing any remaining input.

The following sections are informative.

APPLICATION USAGE

The 255 exit status allows a utility being used by xargs to tell xargs to terminate if it
knows no further invocations using the current data stream will succeed. Thus, utility
should explicitly exit with an appropriate value to avoid accidentally returning with 255.

Note that since input is parsed as lines, <blank> characters separate arguments, and
<backslash>, <apostrophe>, and double-quote characters are used for quoting, if xargs is
used to bundle the output of commands like find dir −print or ls into commands to be
executed, unexpected results are likely if any filenames contain <blank>, <newline>, or
quoting characters. This can be solved by using find to call a script that converts each
file found into a quoted string that is then piped to xargs, but in most cases it is
preferable just to have find do the argument aggregation itself by using −exec with a '+'
terminator instead of ';'. Note that the quoting rules used by xargs are not the same as
in the shell. They were not made consistent here because existing applications depend on
the current rules. An easy (but inefficient) method that can be used to transform input
consisting of one argument per line into a quoted form that xargs interprets correctly is
to precede each non-<newline> character with a <backslash>. More efficient alternatives
are shown in Example 2 and Example 5 below.

On implementations with a large value for {ARG_MAX}, xargs may produce command lines
longer than {LINE_MAX}. For invocation of utilities, this is not a problem. If xargs is
being used to create a text file, users should explicitly set the maximum command line
length with the −s option.

The command, env, nice, nohup, time, and xargs utilities have been specified to use exit
code 127 if an error occurs so that applications can distinguish ``failure to find a
utility'' from ``invoked utility exited with an error indication''. The value 127 was
chosen because it is not commonly used for other meanings; most utilities use small values
for ``normal error conditions'' and the values above 128 can be confused with termination
due to receipt of a signal. The value 126 was chosen in a similar manner to indicate that
the utility could be found, but not invoked. Some scripts produce meaningful error
messages differentiating the 126 and 127 cases. The distinction between exit codes 126 and
127 is based on KornShell practice that uses 127 when all attempts to exec the utility
fail with [ENOENT], and uses 126 when any attempt to exec the utility fails for any other
reason.

EXAMPLES

1. The following command combines the output of the parenthesized commands (minus the
<apostrophe> characters) onto one line, which is then appended to the file log. It
assumes that the expansion of "$0$*" does not include any <apostrophe> or <newline>
characters.

(logname; date; printf "'%s'\n$0 $*") | xargs −E "" >>log

2. The following command invokes diff with successive pairs of arguments originally typed
as command line arguments. It assumes there are no embedded <newline> characters in
the elements of the original argument list.

printf "%s\n$@" | sed 's/[^[:alnum:]]/\\&/g' |
xargs −E "" −n 2 −x diff

3. In the following commands, the user is asked which files in the current directory
(excluding dotfiles) are to be archived. The files are archived into arch; a, one at a
time or b, many at a time. The commands assume that no filenames contain <blank>,
<newline>, <backslash>, <apostrophe>, or double-quote characters.

a. ls | xargs −E "" −p −L 1 ar −r arch

b. ls | xargs −E "" −p −L 1 | xargs −E "" ar −r arch

4. The following command invokes command1 one or more times with multiple arguments,
stopping if an invocation of command1 has a non-zero exit status.

xargs −E "" sh −c 'command1 "$@" || exit 255' sh < xargs_input

5. On XSI-conformant systems, the following command moves all files from directory $1 to
directory $2, and echoes each move command just before doing it. It assumes no
filenames contain <newline> characters and that neither $1 nor $2 contains the
sequence "{}".

ls −A "$1" | sed −e 's/"/"\\""/g' −e 's/.*/"&"/' |
xargs −E "" −I {} −t mv "$1"/{} "$2"/{}

RATIONALE

The xargs utility was usually found only in System V-based systems; BSD systems included
an apply utility that provided functionality similar to xargs −n number. The SVID lists
xargs as a software development extension. This volume of POSIX.1‐2008 does not share the
view that it is used only for development, and therefore it is not optional.

The classic application of the xargs utility is in conjunction with the find utility to
reduce the number of processes launched by a simplistic use of the find −exec combination.
The xargs utility is also used to enforce an upper limit on memory required to launch a
process. With this basis in mind, this volume of POSIX.1‐2008 selected only the minimal
features required.

Although the 255 exit status is mostly an accident of historical implementations, it
allows a utility being used by xargs to tell xargs to terminate if it knows no further
invocations using the current data stream shall succeed. Any non-zero exit status from a
utility falls into the 1‐125 range when xargs exits. There is no statement of how the
various non-zero utility exit status codes are accumulated by xargs. The value could be
the addition of all codes, their highest value, the last one received, or a single value
such as 1. Since no algorithm is arguably better than the others, and since many of the
standard utilities say little more (portably) than ``pass/fail'', no new algorithm was
invented.

Several other xargs options were removed because simple alternatives already exist within
this volume of POSIX.1‐2008. For example, the −i replstr option can be just as efficiently
performed using a shell for loop. Since xargs calls an exec function with each input line,
the −i option does not usually exploit the grouping capabilities of xargs.

The requirement that xargs never produces command lines such that invocation of utility is
within 2048 bytes of hitting the POSIX exec {ARG_MAX} limitations is intended to guarantee
that the invoked utility has room to modify its environment variables and command line
arguments and still be able to invoke another utility. Note that the minimum {ARG_MAX}
allowed by the System Interfaces volume of POSIX.1‐2008 is 4096 bytes and the minimum
value allowed by this volume of POSIX.1‐2008 is 2048 bytes; therefore, the 2048 bytes
difference seems reasonable. Note, however, that xargs may never be able to invoke a
utility if the environment passed in to xargs comes close to using {ARG_MAX} bytes.

The version of xargs required by this volume of POSIX.1‐2008 is required to wait for the
completion of the invoked command before invoking another command. This was done because
historical scripts using xargs assumed sequential execution. Implementations wanting to
provide parallel operation of the invoked utilities are encouraged to add an option
enabling parallel invocation, but should still wait for termination of all of the children
before xargs terminates normally.

The −e option was omitted from the ISO POSIX‐2:1993 standard in the belief that the eofstr
option-argument was recognized only when it was on a line by itself and before quote and
escape processing were performed, and that the logical end-of-file processing was only
enabled if a −e option was specified. In that case, a simple sed script could be used to
duplicate the −e functionality. Further investigation revealed that:

* The logical end-of-file string was checked for after quote and escape processing,
making a sed script that provided equivalent functionality much more difficult to
write.

* The default was to perform logical end-of-file processing with an <underscore> as the
logical end-of-file string.

To correct this misunderstanding, the −E eofstr option was adopted from the X/Open
Portability Guide. Users should note that the description of the −E option matches
historical documentation of the −e option (which was not adopted because it did not
support the Utility Syntax Guidelines), by saying that if eofstr is the null string,
logical end-of-file processing is disabled. Historical implementations of xargs actually
did not disable logical end-of-file processing; they treated a null argument found in the
input as a logical end-of-file string. (A null string argument could be generated using
single or double-quotes ('' or ""). Since this behavior was not documented historically,
it is considered to be a bug.

The −I, −L, and −n options are mutually-exclusive. Some implementations use the last one
specified if more than one is given on a command line; other implementations treat
combinations of the options in different ways.

FUTURE DIRECTIONS

None.

Use xargsposix online using onworks.net services