Special Kwargs
These arguments alter a command’s behavior. They are not passed to the program. You can use them on any command that you run, but some may not be used together. sh will tell you if there are conflicts.
To set default special keyword arguments on every command run, you may use Default Arguments.
Controlling Output
_out
Default value: None
What to redirect STDOUT to. If this is a string, it will be treated as a file
name. You may also pass a file object (or file-like object), an int
(representing a file descriptor, like the result of os.pipe()
), a
io.StringIO
object, or a callable.
import sh
sh.ls(_out="/tmp/output")
See also
_err
Default value: None
What to redirect STDERR to. See _out.
_err_to_out
Default value: False
If True
, duplicate the file descriptor bound to the process’s STDOUT also to
STDERR, effectively causing STDERR and STDOUT to go to the same place.
_encoding
Default value: sh.DEFAULT_ENCODING
The character encoding of the process’s STDOUT. By default, this is the locale’s default encoding.
_decode_errors
New in version 1.07.0.
Default value: "strict"
This is how Python should handle decoding errors of the process’s output.
By default, this is "strict"
, but you can use any value that’s valid
to bytes.decode()
, such as "ignore"
.
_tee
New in version 1.07.0.
Default value: None
As of 1.07.0, any time redirection is used, either for STDOUT or STDERR, the
respective internal buffers are not filled. For example, if you’re downloading
a file and using a callback on STDOUT, the internal STDOUT buffer, nor the pipe
buffer be filled with data from STDOUT. This option forces one of stderr
(_tee='err'
) or stdout (_tee='out'
or _tee=True
) to be filled
anyways, in effect “tee-ing” the output into two places (the callback/redirect
handler, and the internal buffers).
_truncate_exc
New in version 1.12.0.
Default value: True
Whether or not exception ouput should be truncated.
Execution
_fg
New in version 1.12.0.
Default value: False
Runs a command in the foreground, meaning it is spawned using os.spawnle()
. The current process’s STDIN/OUT/ERR
is os.dup2()
’d to the new process and so the new process becomes the foreground of the shell executing the
script. This is only really useful when you want to launch a lean, interactive process that sh is having trouble
running, for example, ssh.
Warning
_fg=True
side-steps a lot of sh’s functionality. You will not be returned a process object and most (likely
all) other special kwargs will not work.
If you are looking for similar functionality, but still retaining sh’s features, use the following:
import sh
import sys
sh.your_command(_in=sys.stdin, _out=sys.stdout, _err=sys.stderr)
_bg
Default value: False
Runs a command in the background. The command will return immediately, and you
will have to run RunningCommand.wait()
on it to ensure it terminates.
See also
_bg_exc
New in version 1.12.9.
Default value: True
Automatically report exceptions for the background command. If you set this to
False
you should make sure to call RunningCommand.wait()
or you may
swallow exceptions that happen in the background command.
_async
New in version 2.0.0.
Default value: False
Allows your command to become awaitable. Use in combination with _iter
and async for
to incrementally await output as it is produced.
_env
Default value: None
A dictionary defining the only environment variables that will be made accessible to the process. If not specified, the calling process’s environment variables are used.
Note
This dictionary is the authoritative environment for the process. If you wish to change a single variable in your current environement, you must pass a copy of your current environment with the overriden variable to sh.
See also
_timeout
Default value: None
How much time, in seconds, we should give the process to complete. If the process does not finish within the timeout, it will be sent the signal defined by _timeout_signal.
_timeout_signal
Default value: signal.SIGKILL
The signal to be sent to the process if _timeout is not None
.
_cwd
Default value: None
A string that sets the current working directory of the process.
_ok_code
Default value: 0
Either an integer, a list, or a tuple containing the exit code(s) that are considered “ok”, or in other words: do not raise an exception. Some misbehaved programs use exit codes other than 0 to indicate success.
import sh
sh.weird_program(_ok_code=[0,3,5])
See also
_new_session
Default value: False
Determines if our forked process will be executed in its own session via
os.setsid()
.
Changed in version 2.0.0: The default value of _new_session
was changed from True
to False
because it makes more sense for a launched process to default to being in
the process group of python script, so that it receives SIGINTs correctly.
Note
If _new_session
is False
, the forked process will be put into its
own group via os.setpgrp()
. This way, the forked process, and all of
it’s children, are always alone in their own group that may be signalled
directly, regardless of the value of _new_session
.
See also
_uid
New in version 1.12.0.
Default value: None
The user id to assume before the child process calls os.execv()
.
_preexec_fn
New in version 1.12.0.
Default value: None
A function to be run directly before the child process calls os.execv()
.
Typically not used by normal users.
_pass_fds
New in version 1.13.0.
Default value: {}
(empty set)
A whitelist iterable of integer file descriptors to be inherited by the child. Passing anything in this argument causes _close_fds to be True
.
_close_fds
New in version 1.13.0.
Default value: True
Causes all inherited file descriptors besides stdin, stdout, and stderr to be automatically closed. This option is automatically enabled when _pass_fds is given a value.
Communication
_in
Default value: None
Specifies an argument for the process to use as its standard input. This may be
a string, a queue.Queue
, a file-like object, or any iterable.
See also
_piped
Default value: None
May be True
, "out"
, or "err"
. Signals a command that it is being
used as the input to another command, so it should return its output
incrementally as it receives it, instead of aggregating it all at once.
See also
_iter
Default value: None
May be True
, "out"
, or "err"
. Puts a command in iterable mode. In
this mode, you can use a for
or while
loop to iterate over a command’s
output in real-time.
import sh
for line in sh.cat("/tmp/file", _iter=True):
print(line)
See also
_iter_noblock
Default value: None
Same as _iter, except the loop will not block if there is no
output to iterate over. Instead, the output from the command will be
errno.EWOULDBLOCK
.
import sh
import errno
import time
for line in sh.tail("-f", "stuff.log", _iter_noblock=True):
if line == errno.EWOULDBLOCK:
print("doing something else...")
time.sleep(0.5)
else:
print("processing line!")
See also
_with
Default value: False
Explicitly tells us that we’re running a command in a with
context. This is
only necessary if you’re using a command in a with
context and passing
parameters to it.
import sh
with sh.contrib.sudo(password="abc123", _with=True):
print(sh.ls("/root"))
See also
_done
New in version 1.11.0.
Default value: None
A callback that is always called when the command completes, even if it completes with an exit code that would raise an exception. After the callback is run, any exception that would be raised is raised.
The callback is passed the RunningCommand instance, a boolean indicating success, and the exit code.
Here’s an example of using _done to create a multiprocess pool, where
sh.your_parallel_command
is executed concurrently at no more than 10 at a
time:
import sh
from threading import Semaphore
pool = Semaphore(10)
def done(cmd, success, exit_code):
pool.release()
def do_thing(arg):
pool.acquire()
return sh.your_parallel_command(arg, _bg=True, _done=done)
procs = []
for arg in range(100):
procs.append(do_thing(arg))
# essentially a join
[p.wait() for p in procs]
TTYs
_tty_in
Default value: False
, meaning a os.pipe()
will be used.
If True
, sh creates a TTY for STDIN, essentially emulating a terminal, as if
your command was entered from the commandline. This is necessary for commands
that require STDIN to be a TTY.
_tty_out
Default value: True
If True
, sh creates a TTY for STDOUT, otherwise use a os.pipe()
. This
is necessary for commands that require STDOUT to be a TTY.
See also
_unify_ttys
New in version 1.13.0.
Default value: False
If True
, sh will combine the STDOUT and STDIN TTY into a single
pseudo-terminal. This is sometimes required by picky programs which expect to be
dealing with a single pseudo-terminal, like SSH.
See also
_tty_size
Default value: (20, 80)
The (rows, columns) of stdout’s TTY. Changing this may affect how much your program prints per line, for example.
Performance & Optimization
_in_bufsize
Default value: 0
The STDIN buffer size. 0 for unbuffered, 1 for line buffered, anything else for a buffer of that amount.
_out_bufsize
Default value: 1
The STDOUT buffer size. 0 for unbuffered, 1 for line buffered, anything else for a buffer of that amount.
_err_bufsize
Default value: 1
Same as _out_bufsize, but with STDERR.
_internal_bufsize
Default value: 3 * 1024**2
chunks
How much of STDOUT/ERR your command will store internally. This value represents the number of bufsize chunks not the total number of bytes. For example, if this value is 100, and STDOUT is line buffered, you will be able to retrieve 100 lines from STDOUT. If STDOUT is unbuffered, you will be able to retrieve only 100 characters.
_no_out
New in version 1.07.0.
Default value: False
Disables STDOUT being internally stored. This is useful for commands that produce huge amounts of output that you don’t need, that would otherwise be hogging memory if stored internally by sh.
_no_err
New in version 1.07.0.
Default value: False
Disables STDERR being internally stored. This is useful for commands that produce huge amounts of output that you don’t need, that would otherwise be hogging memory if stored internally by sh.
_no_pipe
New in version 1.07.0.
Default value: False
Similar to _no_out
, this explicitly tells the sh command that it will never
be used for piping its output into another command, so it should not fill its
internal pipe buffer with the process’s output. This is also useful for
conserving memory.
Program Arguments
These are options that affect how command options are fed into the program.
_long_sep
New in version 1.12.0.
Default value: "="
This is the character(s) that separate a program’s long argument’s key from the
value, when using kwargs to specify your program’s long arguments. For example,
if your program expects a long argument in the form --name value
, the way to
achieve this would be to set _long_sep=" "
.
import sh
sh.your_program(key=value, _long_sep=" ")
Would send the following list of arguments to your program:
["--key value"]
If your program expects the long argument name to be separate from its value,
pass None
into _long_sep
instead:
import sh
sh.your_program(key=value, _long_sep=None)
Would send the following list of arguments to your program:
["--key", "value"]
_long_prefix
New in version 1.12.0.
Default value: "--"
This is the character(s) that prefix a long argument for the program being run. Some programs use single dashes, for example, and do not understand double dashes.
_arg_preprocess
New in version 1.12.0.
Default value: None
This is an advanced option that allows you to rewrite a command’s arguments on the fly, based on other command arguments, or some other variable. It is really only useful in conjunction with baking, and only currently used when constructing contrib wrappers.
Example:
import sh
def processor(args, kwargs):
return args, kwargs
my_ls = sh.bake.ls(_arg_preprocess=processor)
Warning
The interface to the _arg_preprocess
function may change without
warning. It is generally only for internal sh use, so don’t use it unless
you absolutely have to.
Misc
_log_msg
Default value: None
New in version 1.12.0.
This allows for a custom logging header for Command Class instances. For example, the default logging looks like this:
import logging
import sh
logging.basicConfig(level=logging.INFO)
sh.ls("-l")
INFO:sh.command:<Command '/bin/ls -l'>: starting process
INFO:sh.command:<Command '/bin/ls -l', pid 28952>: process started
INFO:sh.command:<Command '/bin/ls -l', pid 28952>: process completed
People can find this <Command ..
section long and not relevant. _log_msg
allows you to customize this:
import logging
import sh
logging.basicConfig(level=logging.INFO)
def custom_log(ran, call_args, pid=None):
return ran
sh.ls("-l", _log_msg=custom_log)
INFO:sh.command:/bin/ls -l: starting process
INFO:sh.command:/bin/ls -l: process started
INFO:sh.command:/bin/ls -l: process completed
The first argument, ran
, is the program’s execution string and arguments, as close as we can get it to be how you’d
type in the shell. call_args
is a dictionary of all of the special kwargs that were passed to the command. And pid
is the process id of the forked process. It defaults to None
because the _log_msg
callback is actually called
twice: first to construct the logger for the RunningCommand Class instance, before the process itself is spawned, then
a second time after the process is spawned via OProc Class, when we have a pid.