Skip to content

Latest commit

 

History

History
222 lines (182 loc) · 7.96 KB

README-shell-integration.md

File metadata and controls

222 lines (182 loc) · 7.96 KB
Raw

Shell integration for shournal

Basic setup (interactive)

After installation, to start observing your interactive shell-sessions append the following to your shell's rc:
~/.bashrc 

HISTCONTROL=ignoredups:erasedups # NOT ALLOWED: ignorespace,ignoreboth
source /usr/share/shournal/SOURCE_ME.bash
SHOURNAL_ENABLE

~/.zshrc 

source /usr/share/shournal/SOURCE_ME.zsh
SHOURNAL_ENABLE

Launch a new shell afterwards and check whether it's working:

$ echo foo > bar
$ shournal --query --wfile bar
cmd-id 66075 $?=0 2021-11-02 14:23 $ echo foo > bar
Working directory: /home/tycho
session-uuid 3hIZtDwhEey5WPDVv9W/Cw==
  1 written file:
     /home/tycho/bar (4 bytes) Hash: 8087352826690557229
$ # or just look into the history:
$ shournal --query --history 3
# ...

The shell-integration injects code into PROMPT_COMMAND, PS0 and PS1 (bash) or the preexec/precmd_functions (zsh), so please do not overwrite those after having enabled shournal. Further basic history functionality must be available, e.g. in bash HISTCONTROL must not ignore commands with leading spaces (see above). shournal's shell integration checks the typical variables and gives hints, if there is need for action.

Other commands include
SHOURNAL_DISABLE to disable the observation
SHOURNAL_PRINT_VERSIONS to print the version of each component
SHOURNAL_SET_VERBOSITY to change the default verbosity ("dbg, info, warning, critical"). For dbg, shournal must have been compiled with debugging symbols. A verbosity higher than warning is not recommended.

Advanced setup (non-interactive)

To also observe non-interactive commands executed via ssh

ssh localhost echo foo

or the Sun Grid Engine (SGE) the following setup is recommended:

bash
Put the following near the beginning of your bashrc:

if [[ -n ${SGE_O_WORKDIR+x} || (
     -n ${BASH_EXECUTION_STRING+x} &&
    ( -n ${SSH_CLIENT+x} || -n ${SSH_TTY+x} )
    ) ]]; then
    source /usr/share/shournal/SOURCE_ME.bash
    SHOURNAL_ENABLE
fi

In particular that code has to run before the sourcing of ~/.bashrc stops due to a negative interactive-check. For example, some distributions place the following near the top of the bashrc:

case $- in
    *i*) ;;
      *) return;;
esac

zsh
Put the following into ~/.zprofile

if [[ -n ${SGE_O_WORKDIR+x} || (
    -n ${ZSH_EXECUTION_STRING+x} &&
    ( -n ${SSH_CLIENT+x} || -n ${SSH_TTY+x} )
    ) ]]; then
    source /usr/share/shournal/SOURCE_ME.zsh
    SHOURNAL_ENABLE
fi

Note that depending on your server environment, this requires zsh to be executed as login shell, e.g.
ssh HOST zsh -l -c command. Alternatively you may use ~/.zshenv but beware that this file is always sourced, also during zsh -c ':' invocations on the interactive command-line, so at least an additional check for
[ $SHLVL -eq 1 ] is recommended.

For cluster software systems other than SGE, you may export SHOURNAL_IS_CLUSTERJOB=true, before SHOURNAL_ENABLE, if and only if the shell is about to execute a cluster job. Note that in this case, shournal performs a re-execution of the current command and only returns control flow after flushing the database, because cluster software systems tend to kill background processes, once the main job script finished. To totally disable cluster job detection, set SHOURNAL_NO_CLUSTER_JOB_DETECT=true before SHOURNAL_ENABLE.

Prerequisites of the fanotify backend

If the fanotify backend is used, please ensure the following:

  • The shell must be linked dynamically against (g)libc (default case, can be tested e.g. with
    file $(which bash) | grep "dynamically linked" ).
  • Sourcing of SOURCE_ME.$shell must be within the shell's rc-file.
  • SHOURNAL_ENABLE should be within the shell's rc, because on the very first enable the shell is re-executed, purging all non-exported variables.
  • For non-interactive commands SHOURNAL_ENABLE must be called before the actual execution begins.

Note that the kernel module backend does not have those prerequisites and should be preferred in most cases.

Updates

If the shell-integration is running while shournal is updated, it is recommended, to restart your shell. A more elegant way than logout-login might be to exec your $shell.

FAQ

  • How to obtain the value of variables?.
    If shell-variables are used within a command, shournal's reports might not seem to be very helpful. However, the shell-integration assigns each shell-session a unique identifier (uuid). In the likely case that the variable was assigned during that session, you might be able to obtain its value. This of course only works, if SHOURNAL_ENABLE was called, before a variable was assigned. Example:
    shournal --query --shell-session-id 'L/932KZTEemRB/dOGB9LOA==' | grep var_name
  • What about new, nested shell-sessions?
    By new shell-sessions it is meant to call e.g. bash within an already running bash-process. What happens next really depends on whether the shell is itself observed by shournal or not (e.g. whether SHOURNAL_ENABLE is within the .bashrc or not). On calling SHOURNAL_ENABLE file-events are then considered to belong to the new shell-session and are no longer reported to the original observation-process of the caller. If a non-observed shell is a called, shournal's later report will not be very helpful: all file-modifications caused by that process will yield the plain shell-command (and not individual commands possibly entered within the new shell session).

Limitations

  • File-operations (redirections) which spread over multiple command-sequences within the interactive shell might lead to surprising (kernel module backend) or incorrect (fanotify backend) results.
    Example: 
    $ exec 3> /tmp/foo  # open fd 3
$ echo "test" >&3
$ exec 3>&-  # close fd 3.
    In case of the kernel module backend as usual the close event is tracked, however shournal -q -wf /tmp/foo prints only the command exec 3>&-. By using the shell-session uuid it should be possible to reconstruct those cases.
    In case of the fanotify backend the close-event is lost.
  • Additional limitations of the fanotify-backend:
    Filesystem-events of asynchronously launched processes, which close the inherited shournal-socket, might be lost, because an external shournal-run process waits until all instances of that socket are closed. Steps to reproduce: In an observed shell-session enter
    bash -c 'eval "exec $_SHOURNAL_SOCKET_NB>&-"; sleep 1; echo foo > bar' &
    Note that e.g. in Python processes launched via its subprocess-module do not inherit file descriptors by default. There seems to be no general solution to this problem, but in most cases there should be some mechanism to wait for the processes to finish, within the interactive shell-session or a script.
  • For further limitations please also read the general README.

Motivation

For a general introduction about the data and meta-data shournal stores please visit the general README.

Having to type shournal before every single command one wants to observe can be tiresome. Another typing-overhead would be introduced by using pipes or redirections. Consider the following broken example:

shournal --exec echo hi > foo    # Don't do this.

As many shell users know the redirection applies to the whole command, while shournal itself only observes "echo hi". The file modification event ('hi' written to 'foo') is hence not tracked by shournal. To actually observe such a command one must rather type

shournal --exec sh -c 'echo hi > foo'

That's annoying, right?

Therefore before observing one or multiple commands, source the respective integration-file within your shell's rc (e.g. .bashrc) and type

SHOURNAL_ENABLE

That's (almost) all. Forget about shournal until needed ( e.g. you want to know how a certain file was created).