timmy/doc/source/usage.rst

Usage

NOTICE: Even though Timmy uses nice and ionice to limit impact on the cloud, you should still expect 1 core utilization both locally (where Timmy is launched) and on each node where commands are executed or logs collected. Additionally, if logs are collected, local disk (log destination directory) may get utilized significantly.

WARNING If modifying the outdir config parameter, please first read the related warning on configuration </configuration> page.

The easiest way to launch Timmy would be running the timmy.py script / timmy command: * Timmy will perform all actions defined in the default.yaml rq-file. The file is located in timmy_data/rq folder in Python installation directory. Specifically: * run diagnostic scripts on all nodes, including Fuel server * collect configuration files for all nodes * Timmy will NOT collect log files when executed this way.

Basically, timmy.py is a simple wrapper that launches cli.py. * Current page does not reference all available CLI options. Full reference </cli> for command line interface. * You may also want to create a custom configuration </configuration> for Timmy, depending on your use case.

Basic parameters:

• --only-logs only collect logs (skip files, filelists, commands and scripts)
• -l, --logs also collect logs (logs are not collected by default due to their size)
• -e, --env filter by environment ID
• -R, --role filter by role
• -c, --config use custom configuration file to overwrite defaults. See timmy_data/config/example.yaml as an example
• -j, --nodes-json use json file instead of polling Fuel (to generate json file use fuel node --json) - speeds up initialization
• -o, --dest-file the name/path for output archive, default is general.tar.gz and put into /tmp/timmy/archives. A folder will be created if it does not exist. It's not recommended to use /var/log as destination because subsequent runs with log collection may cause Timmy to collect it's own previously created files or even update them while reading from them. The general idea is that a destination directory should contain enough space to hold all collected data and should not be in collection paths.
• -v, --verbose verbose(INFO) logging. Use -vv to enable DEBUG logging.

Shell Mode

Shell Mode is activated whenever any of the following parameters are used via CLI: -C, -S, -P, -G.

A mode of execution which makes the following changes:

• rqfile (timmy_data/rq/default.yaml by default) is skipped
• Fuel node is skipped. If for some reason you need to run specific scripts/actions via Timmy on Fuel and on other nodes at the same time, create an rqfile instead (see configuration </configuration> for details, see timmy_data/rq/neutron.yaml as an example), coupled with --rqfile option or a custom config file to override default rqfile.
• outputs of commands (specified with -C options) and scripts (specified with -S) are printed on screen
• any actions (cmds, scripts, files, filelists, put, except logs) and Parameter Based configuration defined in config are ignored.

The following parameters ("actions") are available via CLI:

• -C <command> - Bash command (string) to execute on nodes. Using multiple -C statements will produce the same result as using one with several commands separated by ; (traditional Shell syntax), but for each -C statement a new SSH connection is established.
• -S <script> - name of the Bash script file to execute on nodes (if you do not have a path separator in the filename, you need to put the file into scripts folder inside a path specified by rqdir config parameter, defaults to rq. If a path separator is present, the given filename will be used directly as provided)
• -P <file/path> <dest> - upload local data to nodes (wildcards supported). You must specify 2 values for each -P switch.
• -G <file/path> - download (collect) data from nodes

Logs

It's possible to specify custom log collection when using CLI: * -L <base-path> <include-regex> <exclude-regex>, --get-logs - specify a base path, include regex and exclude regex to collect logs. This option can be specified more than once, in this case log lists will be united. This option does not disable default log collection defined in timmy_data/rq/default.yaml. * --logs-no-default - use this option of you only need logs specified via -L.

Execution order

Specified actions are executed for all applicable nodes, always in the following order: 1. put 2. commands 3. scripts 4. get, filelists 5. logs

Examples

• timmy - run according to the default configuration and default actions. Default actions are defined in timmy_data/rq/default.yaml. Logs are not collected.
• timmy -l - run default actions and also collect logs. Such execution is similar to Fuel's "diagnostic snapshot" action, but will finish faster and collect less logs. There is a default log collection period based on file modification time, only files modified within the last 30 days are collected.
• timmy -l --days 3 - same as above but only collect log files updated within the last 3 days.
• timmy --only-logs - only collect logs, no actions (files, filelists, commands, scripts, put, get) performed.
• timmy -C 'uptime; free -m' - check uptime and memory on all nodes
• timmy -G /etc/nova/nova.conf - get nova.conf from all nodes
• timmy -R controller -P package.deb '' -C 'dpkg -i package.deb' -C 'rm package.deb' -C 'dpkg -l | grep [p]ackage' - push a package to all nodes, install it, remove the file and check that it is installed. Commands are executed in the order in which they are provided.
• timmy -с myconf.yaml - use a custom config file and run the program according to it. Custom config can specify any actions, log setup, and other settings. See configuration doc for more details.

Using custom configuration file

If you want to perform a set of actions on the nodes without writing a long command line (or if you want to use the options only available in config), you may want to set up config file instead. An example config structure would be:

rqdir: './pacemaker-debug' # a folder which should contain any filelists and/or scripts if they are defined later, should contain folders 'filelists' and/or 'scripts' 
rqfile: null # explicitly undefine rqfile to skip default filelists and scripts
hard_filter:
  roles: # only execute on Fuel and controllers
    - fuel
    - controller 
cmds: # some commands to run on all nodes (after filtering). cmds syntax is {name: value, ...}. cmds are executed in alphabetical order.
  01-my-first-command: 'uptime'
  02-disk-check: 'df -h'
  and-also-ram: 'free -m'
logs:
  path: '/var/log' # base path to search for logs
  exclude: # a list of exclude regexes
    - '.*' # exclude all logs by default - does not make much sense - just an example. If the intention is to not collect all logs then this 'logs' section can be removed altogether, just ensure that either rqfile is custom or 'null', or '--logs-no-default' is set via CLI / 'logs_no_default: True' set in config.
logs_days: 5 # collect only log files updated within the last 5 days
# an example of parameter-based configuration is below:
by_roles:
  controller:
    scripts: # I use script here to not overwrite the cmds we have already defined for all nodes 
      - pacemaker-debug.sh # the name of the file inside 'scripts' folder inside 'rqdir' path, which will be executed (by default) on all nodes
    files:
      - '/etc/coros*' # get all files from /etc/coros* wildcard path
  fuel:
    logs:
      path: '/var/log/remote'
      include: # include regexp - non-matching log files will be excluded.
        - 'crmd|lrmd|corosync|pacemaker'

Then you would run timmy -l -c my-config.yaml to execute Timmy with such config.

Instead of putting all structure in a config file you can move actions (cmds, files, filelists, scripts, logs) to an rqfile, and specify rqfile path in config (although in this example the config-way is more compact). rqfile structure is a bit different:

cmds: # top-level elements are node parameters, __default will be assigned to all nodes
  __default:
    - 01-my-first-command: 'uptime'
    - 02-disk-check: 'df -h'
    - and-also-ram: 'free -m'
scripts:
  by_roles: # all non "__default" keys should match, "by_<parameter>"
    controller: 
      - pacemaker-debug.sh
files:
  by_roles:
    controller:
      - '/etc/coros*'
logs:
  by_roles:
    fuel:
      path: '/var/log/remote'
      include:
        - 'crmd|lrmd|corosync|pacemaker'
  __default: # again, this default section is useless, just serving as an example here.
    path: '/var/log'
    exclude:
      - '.*'

Then the config should look like this:

rqdir: './pacemaker-debug'
rqfile:
  - file: './pacemaker-rq.yaml'
hard_filter:
  roles:
    - fuel
    - controller

And you run timmy -l -c my-config.yaml.

Back to Index </index>.