x/stacktach-klugman
Code Issues Proposed changes

Switch to argparse for better client libary support.

Browse Source 
Docs changed too to reflect better count support (--count).
Removed v2 api implementation example (Archive resource). We
can always revisit once we nail down how archive retrieval
will work.

Change-Id: I79d1929a14fe51d36ecc5c54707902d5e772ad89
This commit is contained in:
Sandy Walsh 2015-04-22 15:00:34 -07:00
parent 312dbbdb1c
commit 0d491a9366
6 changed files with 468 additions and 517 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

392
README.md
View File

@ -3,166 +3,300 @@ klugman
Python library and cmdline tools for accessing Quincy. Python library and cmdline tools for accessing Quincy.
Using Klugman as a client library:
Examples:
``` ```
$ klugman from klugman import v1
Usage:
klugman.py [options] <command> [<args>...]
klugman.py (-h | --help)
klugman.py --version
events = v1.Events('http://www.example.com:8000')
data = events.get_events_count(name='compute.instance.update')
$ klugman -h streams = v1.Streams('http://www.example.com:8000')
Klugman - cmdline and client library for StackTach.v3 data = streams.get_streams(state='completed', details=True)
```
Usage: Command-line Examples:
klugman.py [options] <command> [<args>...]
klugman.py (-h | --help)
klugman.py --version
Options: ```
-h --help Show this help message $ klugman http://127.0.0.1 streams -h
--version Show klugman version usage: klugman url streams [-h] [--name trigger_name] [--from datetime]
--debug Debug mode [--to datetime] [--traits trait_list] [--details]
-a, --api_version <api_version> [--state {active,firing,expiring,error,expire_error,completed,retry_fire,retry_expire}]
Which API version to use [default: latest] [--count | --id stream_id]
--url <url> StackTach.v3 server url [default: http://localhost:8000]
For a list of possible StackTach commands, use: optional arguments:
klugman help -h, --help show this help message and exit
--name trigger_name Return streams of type trigger_name.
--from datetime Return streams last updated after datetime
--to datetime Return streams last updated before datetime
--traits trait_list Return streams with specific distinguishing traits.
--details Return full event details.
--state {active,firing,expiring,error,expire_error,completed,retry_fire,retry_expire}
Only return streams in this state.
--count Return a count of streams matching filter criteria.
--id stream_id Return a single specific stream by id.
$ klugman http://stacktach3-api01.example.com:8000 streams --count
$ klugman help
Klugman - StackTach.v3 client
Usage:
klugman.py [options] streams [<args>...]
klugman.py num-streams [<args>...] [options]
klugman.py [options] archives [<args>...]
Options:
-h, --help show command options
$ klugman num-streams
+----------+-------+ +----------+-------+
| Property | Value | | Property | Value |
+----------+-------+ +----------+-------+
| count | 19 | | count | 44216 |
+----------+-------+ +----------+-------+
$ klugman num-streams --state completed $ klugman http://stacktach3-api01.example.com:8000 streams --count --state completed
+----------+-------+ +----------+-------+
| Property | Value | | Property | Value |
+----------+-------+ +----------+-------+
| count | 4 | | count | 42571 |
+----------+-------+ +----------+-------+
$ klugman num-streams --state active
+----------+-------+
| Property | Value |
+----------+-------+
| count | 15 |
+----------+-------+
$ klugman streams -h $ klugman http://stacktach3-api01.example.com:8000 streams
usage: +---------+------------------+---------------------------------------------------------------------------------------------+
klugman.py streams [options] | Section | Property | Value |
+---------+------------------+---------------------------------------------------------------------------------------------+
| Stream | id | 44171 |
| Stream | state | active |
| Stream | name | test_trigger |
| Stream | first_event | 2015-04-22 21:06:09.400561 |
| Stream | last_event | 2015-04-22 21:07:17.317974 |
| Stream | fire_timestamp | None |
| Stream | expire_timestamp | 2015-04-24 21:07:17.317974 |
| D.Trait | instance_id | 3ed27346-5906-4790-9b6e-e095e5b0cfa4 |
| D.Trait | timestamp | TimeRange from datetime.datetime(2015, 4, 22, 0, 0) to datetime.datetime(2015, 4, 23, 0, 0) |
+---------+------------------+---------------------------------------------------------------------------------------------+
options: $ klugman http://stacktach3-api01.example.com:8000 streams --id 44171 --detail
--id <id> +---------+------------------+---------------------------------------------------------------------------------------------+
get stream with id | Section | Property | Value |
--details +---------+------------------+---------------------------------------------------------------------------------------------+
return events with each stream | Stream | id | 44171 |
--state <state> | Stream | state | active |
return streams in state | Stream | name | test_trigger |
--older_than <datetime> | Stream | first_event | 2015-04-22 21:06:09.400561 |
list streams where first_event < <datetime> | Stream | last_event | 2015-04-22 21:15:05.962515 |
--younger_than <datetime> | Stream | fire_timestamp | None |
list streams where last_event > <datetime> | Stream | expire_timestamp | 2015-04-24 21:15:05.962515 |
--trigger_name <name> | D.Trait | instance_id | 3ed12384-5906-4790-9b6e-e095e5b0cfa4 |
list streams with given trigger definition | D.Trait | timestamp | TimeRange from datetime.datetime(2015, 4, 22, 0, 0) to datetime.datetime(2015, 4, 23, 0, 0) |
--distinguishing_traits <dtraits> +---------+------------------+---------------------------------------------------------------------------------------------+
list stream with specific distriquishing traits Events:
Stream states:
collecting - collecting events
ready - ready for processing
triggered - being processed
processed - processing completed
error - pipeline processing failed
commit_error - pipeline result commit failed
Distinguishing trait format:
"trait:value;trait:value;..."
$ klugman streams
+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | 17 |
| state | completed |
| name | test_trigger |
| first_event | 2015-01-31 13:21:50.679800 |
| last_event | 2015-01-31 13:36:46.981800 |
| fire_timestamp | 2015-01-30 18:52:03.196602 |
| expire_timestamp | 2015-01-31 14:36:46.981800 |
| distinguishing_traits | {u'instance_id': u'bd8b66f6-745b-45ce-b9c2-43010f8d9cdf', u'timestamp': TimeRange from datetime.datetime(2015, 1, 31, 0, 0) to datetime.datetime(2015, 2, 1, 0, 0)} |
| events | None |
+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ klugman streams --distinguishing_traits instance_id:1438620b-e426-4911-81cd-e5f82219a390
...
$ klugman streams --older_than 01-31-2015T13:30
...
$ klugman streams --older_than 01-31-2015T13:30 --state completed
...
$ klugman events --traits=os_distro:com.redhat
+--------------------+------------------------------------------+ +--------------------+------------------------------------------+
| Property | Value | | Property | Value |
+--------------------+------------------------------------------+ +--------------------+------------------------------------------+
| _mark | 1 | | disk_gb | 40 |
| bandwidth_in | 537783 | | display_name | my_ubuntu1404 |
| bandwidth_out | 19189871 |
| disk_gb | 160 |
| display_name | Instance_296624 |
| ephemeral_gb | 0 | | ephemeral_gb | 0 |
| event_type | compute.instance.update | | event_type | compute.instance.update |
| instance_flavor | 4GB Standard Instance | | host | nova-api05.example.com |
| instance_flavor_id | 5 | | instance_flavor | 1GB Standard Instance |
| instance_id | 60c52a73-ec8e-47bc-81eb-eee38931a60e | | instance_flavor_id | 3 |
| instance_type | 4GB Standard Instance | | instance_id | 3ed23782-5906-4790-9b6e-e095e5b0cfa4 |
| launched_at | 2014-04-17 11:40:15.321940 | | instance_type | 1GB Standard Instance |
| memory_mb | 4096 | | memory_mb | 1024 |
| message_id | 8eca72ba-3adb-4354-aaa4-f61980549e07 | | message_id | a58e25ed-01f3-42d8-8979-5f1603ab2468 |
| os_architecture | x64 | | os_architecture | x64 |
| os_distro | com.redhat | | os_distro | com.ubuntu |
| os_version | 6.3 | | os_version | 14.04 |
| request_id | req-511c28a6-c6ec-4173-a124-7c92989e443c | | request_id | req-758de1f9-03e9-4337-af7d-d9efe3efc730 |
| root_gb | 160 | | root_gb | 40 |
| service | publisher-189550 | | service | api |
| state | active | | state | building |
| state_description | powering-off | | state_description | scheduling |
| tenant_id | 854126 | | tenant_id | 1234 |
| timestamp | 2015-03-03 20:13:56.560940 | | timestamp | 2015-04-22 21:06:09.400561 |
| user_id | 366869 | | user_id | 4567 |
| vcpus | 1 |
+--------------------+------------------------------------------+
+--------------------+------------------------------------------+
| Property | Value |
+--------------------+------------------------------------------+
| disk_gb | 40 |
| display_name | my_ubuntu1404 |
| ephemeral_gb | 0 |
| event_type | compute.instance.update |
| host | c-88-77-44-2 |
| instance_flavor | 1GB Standard Instance |
| instance_flavor_id | 3 |
| instance_id | 3ed23782-5906-4790-9b6e-e095e5b0cfa4 |
| instance_type | 1GB Standard Instance |
| memory_mb | 1024 |
| message_id | dbb6a5c0-08b8-45c8-85b4-a77b2b876bc3 |
| os_architecture | x64 |
| os_distro | com.ubuntu |
| os_version | 14.04 |
| request_id | req-758de1f9-03e9-4337-af7d-d9efe3efc730 |
| root_gb | 40 |
| service | None |
| state | building |
| state_description | |
| tenant_id | 1234 |
| timestamp | 2015-04-22 21:06:09.888688 |
| user_id | 4567 |
| vcpus | 1 |
+--------------------+------------------------------------------+
$ klugman http://stacktach3-api01.example.com:8000 streams --traits instance_id:633fe23b-7c6a-dead-beef-55fcc6803cbc
+---------+------------------+---------------------------------------------------------------------------------------------+
| Section | Property | Value |
+---------+------------------+---------------------------------------------------------------------------------------------+
| Stream | id | 43993 |
| Stream | state | active |
| Stream | name | test_trigger |
| Stream | first_event | 2015-04-22 18:21:28.462931 |
| Stream | last_event | 2015-04-22 18:21:28.462931 |
| Stream | fire_timestamp | None |
| Stream | expire_timestamp | 2015-04-24 18:21:28.462931 |
| D.Trait | instance_id | 633fe23b-7c6a-dead-beef-55fcc6803cbc |
| D.Trait | timestamp | TimeRange from datetime.datetime(2015, 4, 22, 0, 0) to datetime.datetime(2015, 4, 23, 0, 0) |
+---------+------------------+---------------------------------------------------------------------------------------------+
+---------+------------------+---------------------------------------------------------------------------------------------+
| Section | Property | Value |
+---------+------------------+---------------------------------------------------------------------------------------------+
| Stream | id | 43992 |
| Stream | state | active |
| Stream | name | test_trigger |
| Stream | first_event | 2015-04-22 18:21:27.905027 |
| Stream | last_event | 2015-04-22 18:24:18.985118 |
| Stream | fire_timestamp | None |
| Stream | expire_timestamp | 2015-04-24 18:24:18.985118 |
| D.Trait | instance_id | 633fe23b-7c6a-dead-beef-55fcc6803cbc |
| D.Trait | timestamp | TimeRange from datetime.datetime(2015, 4, 22, 0, 0) to datetime.datetime(2015, 4, 23, 0, 0) |
+---------+------------------+---------------------------------------------------------------------------------------------+
$ klugman http://stacktach3-api01.example.com:8000 streams --state completed
+---------+------------------+---------------------------------------------------------------------------------------------+
| Section | Property | Value |
+---------+------------------+---------------------------------------------------------------------------------------------+
| Stream | id | 43498 |
| Stream | state | completed |
| Stream | name | test_trigger |
| Stream | first_event | 2015-04-20 00:00:18.740466 |
| Stream | last_event | 2015-04-20 00:00:18.740466 |
| Stream | fire_timestamp | 2015-04-22 11:53:37.441695 |
| Stream | expire_timestamp | 2015-04-22 00:00:18.740466 |
| D.Trait | instance_id | 7080633a-ffa3-dead-beef-d1549b4ac049 |
| D.Trait | timestamp | TimeRange from datetime.datetime(2015, 4, 19, 0, 0) to datetime.datetime(2015, 4, 20, 0, 0) |
+---------+------------------+---------------------------------------------------------------------------------------------+
$ klugman http://127.0.0.1 events -h
usage: klugman url events [-h] [--name event_name] [--from datetime]
[--to datetime] [--traits trait_list]
[--count | --msg_id message_id]
optional arguments:
-h, --help show this help message and exit
--name event_name Return events of type event_name.
--from datetime Return events generated before datetime
--to datetime Return events generated after datetime
--traits trait_list Return events with specific traits.
--count Return a count of events matching filter criteria.
--msg_id message_id Return a single specific event by message id.
$ klugman http://stacktach3-api01.example.com:8000 events
+--------------------+------------------------------------------+
| Property | Value |
+--------------------+------------------------------------------+
| _mark | 6544b |
| disk_gb | 80 |
| display_name | My_Display_Name |
| ephemeral_gb | 0 |
| event_type | compute.instance.update |
| host | c-11-22-33-4 |
| instance_flavor | 2GB Standard Instance |
| instance_flavor_id | 4 |
| instance_id | 85240caf-71cf-dead-beef-6735298e6090 |
| instance_type | 2GB Standard Instance |
| memory_mb | 2048 |
| message_id | c49d08be-dae5-4740-8f04-fb4cc27ac2fa |
| os_architecture | x64 |
| os_distro | org.centos |
| os_version | 7 |
| request_id | req-16b4995e-8a14-4b25-bd1c-ba68e82773f7 |
| root_gb | 80 |
| service | None |
| state | building |
| state_description | spawning |
| tenant_id | 725 |
| timestamp | 2015-04-22 21:43:29.940846 |
| user_id | 945 |
| vcpus | 2 | | vcpus | 2 |
+--------------------+------------------------------------------+ +--------------------+------------------------------------------+
$ klugman events --msg_id=8eca72ba-3adb-4354-aaa4-f61980549e07
... $ klugman http://stacktach3-api01.example.com:8000 events --name compute.instance.create.end
+--------------------+------------------------------------------+
| Property | Value |
+--------------------+------------------------------------------+
| _mark | 6547e |
| disk_gb | 80 |
| display_name | My_Display_Name |
| ephemeral_gb | 0 |
| event_type | compute.instance.create.end |
| host | c-11-22-33-4 |
| instance_flavor | 2GB Standard Instance |
| instance_flavor_id | 4 |
| instance_id | 0c30d4be-409d-dead-beef-6d8a3744e80f |
| instance_type | 2GB Standard Instance |
| launched_at | 2015-04-22 21:46:04 |
| memory_mb | 2048 |
| message | Success |
| message_id | 51bd735f-9817-4940-9826-5b057bf51f70 |
| os_architecture | x64 |
| os_distro | com.microsoft.server |
| os_version | 2012.0 |
| rax_options | 4 |
| request_id | req-108e129d-49fd-1213-8e93-3f5595fd4d6c |
| root_gb | 80 |
| service | compute |
| state | active |
| state_description | |
| tenant_id | 3334 |
| timestamp | 2015-04-22 21:46:05.066171 |
| user_id | 3848 |
| vcpus | 2 |
+--------------------+------------------------------------------+
$ klugman events --name=compute.instance.power_off.end $ klugman http://stacktach3-api01.example.com:8000 events --name compute.instance.create.end --count
... +----------+-------+
| Property | Value |
+----------+-------+
| count | 10280 |
+----------+-------+
$ klugman events --from="2015-03-04T22:25" --to="2015-03-04T22:45"
$ klugman http://stacktach3-api01.example.com:8000 events --msg_id 047d9d5c-9190-4b85-9963-35d4cd095d07
+--------------------+------------------------------------------+
| Property | Value |
+--------------------+------------------------------------------+
| disk_gb | 80 |
| display_name | My_Display_Name |
| ephemeral_gb | 0 |
| event_type | compute.instance.create.end |
| host | c-11-22-33-4 |
| instance_flavor | 2GB Standard Instance |
| instance_flavor_id | 4 |
| instance_id | b3c1be31-0ebe-4d6d-9663-b617eabac421 |
| instance_type | 2GB Standard Instance |
| launched_at | 2015-04-22 18:02:03 |
| memory_mb | 2048 |
| message | Success |
| message_id | 047d9d5c-9190-4b85-9963-35d4cd095d07 |
| os_architecture | x64 |
| os_distro | org.debian |
| os_version | 7 |
| rax_options | 0 |
| request_id | req-597b08a3-faef-425a-8f63-b91abcb1e1dd |
| root_gb | 80 |
| service | compute |
| state | active |
| tenant_id | 3334 |
| timestamp | 2015-04-22 21:46:05.066171 |
| user_id | 3848 |
| vcpus | 2 |
+--------------------+------------------------------------------+
``` ```

28
klugman/base.py
View File

@ -13,6 +13,8 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import jsonutil
from docopt import docopt from docopt import docopt
import prettytable import prettytable
import requests import requests
@ -32,26 +34,12 @@ def dump_response(keys, rows):
print x print x
def get(url, cmd, params): def get(url, cmd, params, debug=False):
final = "%s/%s" % (url, cmd) final = "%s/%s" % (url, cmd)
if debug:
print 'URL: %s' % final
for item in params.items():
print " : %s='%s'" % item
ret = requests.get(final, params=params) ret = requests.get(final, params=params)
ret.raise_for_status() ret.raise_for_status()
return ret return ret.json(object_hook=jsonutil.object_hook)
class Impl(object):
def __init__(self, base_url, base_args, cmds, docs):
self.base_url = base_url
self.base_args = base_args
self.cmds = cmds
self.docs = docs
def dispatch(self, cmdline):
arguments = docopt(self.docs, argv=cmdline, help=False,
options_first=True)
if self.base_args['--debug']:
print arguments
for key in self.cmds.keys():
if arguments.get(key):
self.cmds[key].cmdline(self, cmdline)

66
klugman/klugman.py
View File

@ -13,57 +13,41 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
"""Klugman - cmdline and client library for StackTach.v3 import argparse
Usage:
klugman.py [options] <command> [<args>...]
klugman.py (-h | --help)
klugman.py --version
Options:
-h --help Show this help message
--version Show klugman version
--debug Debug mode
-a, --api_version <api_version>
Which API version to use [default: latest]
--url <url> StackTach.v3 server url [default: http://localhost:8000]
For a list of possible StackTach commands, use:
klugman help
"""
from docopt import docopt
import v1 import v1
import v2
versions = {1: v1.V1, 2: v2.V2} versions = {1: v1.V1}
latest = 1 latest = 1
def main(): def _get_base_parser():
arguments = docopt(__doc__, options_first=True) parser = argparse.ArgumentParser(description='Klugman cmdline tool')
if arguments['--debug']: parser.add_argument('--version', metavar='version', type=int,
print arguments default=latest, help='Which api version to use.')
parser.add_argument('url', metavar='url',
help='The API endpoint url')
parser.add_argument('--debug', action='store_true',
default=False, help='Enable debugging.')
return parser
version = arguments["--api_version"]
if version == "latest": def main():
version = latest parser = _get_base_parser()
else: parser.add_argument('args', nargs=argparse.REMAINDER)
version = int(version) arguments = parser.parse_args()
version = arguments.version
impl = versions[version] impl = versions[version]
url = "%s/v%d" % (arguments["--url"], version) url = "%s/v%d" % (arguments.url, version)
cmd = arguments['<command>']
argv = [cmd] + arguments['<args>']
api = impl(url, arguments) # Ok, we got past the basics. Add the subparsers and try again ...
if cmd == 'help': parser = _get_base_parser()
print api.__doc__ api = impl(url, parser)
else: arguments = parser.parse_args()
api.dispatch(argv)
api.dispatch(arguments)
if __name__ == '__main__': if __name__ == '__main__':

417
klugman/v1.py
View File

@ -14,76 +14,105 @@
# limitations under the License. # limitations under the License.
import base import argparse
import json
import jsonutil
import prettytable import prettytable
import sys import sys
from docopt import docopt import base
def check_state(state):
if state and state not in ["active", "firing", "expiring", "error",
"expire_error", "completed",
"retry_fire", "retry_expire"]:
print "Invalid state. %s not one of 'active, firing, " \
"expiring, error, expire_error, completed, " \
"retry_fire or retry_expire" % state
sys.exit(1)
class Streams(object): class Streams(object):
"""usage: def __init__(self, url, subparser=None):
klugman.py streams [options] self.url = url
if subparser:
parser = subparser.add_parser('streams')
parser.add_argument('--name', metavar='trigger_name',
help='Return streams of type trigger_name.')
parser.add_argument('--from', metavar='datetime', dest='_from',
help='Return streams last updated after datetime')
parser.add_argument('--to', metavar='datetime',
help='Return streams last updated before datetime')
parser.add_argument('--traits', metavar='trait_list',
help='Return streams with specific distinguishing traits.')
parser.add_argument('--details', action='store_true',
default=False, help='Return full event details.')
parser.add_argument('--state', choices=['active', 'firing',
'expiring', 'error', 'expire_error', 'completed',
'retry_fire', 'retry_expire'],
help='Only return streams in this state.')
options: group = parser.add_mutually_exclusive_group()
--id <id> group.add_argument('--count', action='store_true',
get stream with id default=False,
--details help='Return a count of streams matching filter criteria.')
return events with each stream group.add_argument('--id', metavar='stream_id', dest='stream_id',
--state <state> help='Return a single specific stream by id.')
return streams in state
--older_than <datetime>
list streams where first_event < <datetime>
--younger_than <datetime>
list streams where last_event > <datetime>
--trigger_name <name>
list streams with given trigger definition
--distinguishing_traits <dtraits>
list stream with specific distinguishing traits
Stream states: def get_streams_count(self, from_datetime=None, to_datetime=None,
collecting - collecting events traits=None, state=None, name=None, debug=False):
ready - ready for processing if traits:
triggered - being processed traits = ",".join(["%s:%s" % item for item in traits.items()])
processed - processing completed cmd = "streams/count"
error - pipeline processing failed params = base.remove_empty({'older_than': from_datetime,
commit_error - pipeline result commit failed 'younger_than': to_datetime,
'name': name,
'state': state,
'distinguishing_traits': traits})
Distinguishing trait format: return base.get(self.url, cmd, params, debug=debug)
"trait:value;trait:value;..."
"""
def cmdline(self, version, cmdline): def get_streams(self, from_datetime=None, to_datetime=None,
arguments = docopt(Streams.__doc__, argv=cmdline) traits=None, name=None, stream_id=None, debug=False,
debug = version.base_args['--debug'] state=None, details=None):
if debug: if stream_id:
print arguments params = base.remove_empty({'details': details})
return base.get(self.url, "streams/%s" % stream_id, params,
debug=debug)
response = self.do_streams(version, arguments) if traits:
# Handle cmdline output here, not in do_foo() traits = ",".join(["%s:%s" % item for item in traits.items()])
raw_rows = response.json(object_hook=jsonutil.object_hook) cmd = "streams"
params = base.remove_empty({'older_than': from_datetime,
'younger_than': to_datetime,
'name': name,
'details': details,
'state': state,
'distinguishing_traits': traits})
if debug: return base.get(self.url, cmd, params, debug=debug)
print json.dumps(raw_rows, indent=2)
# TODO(sandy): This should come from the server-issued def _cmdline(self, arguments):
# schema at some point. _from = arguments._from
_to = arguments.to
_name = arguments.name
_traits = arguments.traits
_debug = arguments.debug
_state = arguments.state
_details = arguments.details
trait_dict = None
if _traits:
trait_pairs = _traits.split(',')
trait_dict = dict([tuple(item.split(':')) for item in trait_pairs])
if arguments.count:
rows = self.get_streams_count(from_datetime=_from, to_datetime=_to,
name=_name, traits=trait_dict,
state=_state, debug=_debug)
print rows
keys = ['count']
base.dump_response(keys, rows)
return
_stream_id = arguments.stream_id
rows = self.get_streams(from_datetime=_from, to_datetime=_to,
name=_name, traits=trait_dict,
details=_details, state=_state,
stream_id=_stream_id, debug=_debug)
keys = ['id', 'state', 'name', 'first_event', 'last_event', keys = ['id', 'state', 'name', 'first_event', 'last_event',
'fire_timestamp', 'expire_timestamp'] 'fire_timestamp', 'expire_timestamp']
for row in raw_rows: for row in rows:
x = prettytable.PrettyTable(["Section", "Property", "Value"]) x = prettytable.PrettyTable(["Section", "Property", "Value"])
for key in keys: for key in keys:
x.add_row(["Stream", key, row.get(key)]) x.add_row(["Stream", key, row.get(key)])
@ -105,209 +134,105 @@ class Streams(object):
print x print x
def do_streams(self, version, arguments):
sid = arguments.get('--id')
state = arguments.get('--state')
older = arguments.get('--older_than')
younger = arguments.get('--younger_than')
trigger = arguments.get('--trigger_name')
traits = arguments.get('--distinguishing_traits')
details = arguments.get('--details')
check_state(state)
cmd = "streams"
if sid:
cmd = "streams/%s" % sid
return base.get(version.base_url, cmd, {'details': details})
params = base.remove_empty({'state': state,
'older_than': older,
'younger_than': younger,
'trigger_name': trigger,
'distinguishing_traits': traits,
'details': details})
return base.get(version.base_url, cmd, params)
class NumStreams(object):
"""usage:
klugman.py num-streams [options]
options:
--state <state>
return streams in state
--older_than <datetime>
list streams older than datetime
--older_than <datetime>
list streams where first_event < <datetime>
--younger_than <datetime>
list streams where last_event > <datetime>
--distinguishing_traits <dtraits>
list stream with specific distinguishing traits
Stream states:
active = collecting events
firing = about to process events
expiring = about to expire stream
error = pipeline processing failed
expire_error = pipeline expiry failed
completed = stream processing completed
retry_fire = re-attempt pipeline firing
retry_expire = re-attempt pipeline expiry
Distinguishing trait format:
"trait:value;trait:value;..."
"""
def cmdline(self, version, cmdline):
arguments = docopt(NumStreams.__doc__, argv=cmdline)
debug = version.base_args['--debug']
if debug:
print arguments
response = self.do_stream_count(version, arguments)
raw_rows = response.json(object_hook=jsonutil.object_hook)
keys = ['count']
base.dump_response(keys, raw_rows)
def do_stream_count(self, version, arguments):
state = arguments.get('--state')
older = arguments.get('--older_than')
younger = arguments.get('--younger_than')
trigger = arguments.get('--trigger_name')
traits = arguments.get('--distinguishing_traits')
check_state(state)
cmd = "streams/count"
params = base.remove_empty({'state': state,
'older_than': older,
'younger_than': younger,
'trigger_name': trigger,
'distinguishing_traits': traits})
return base.get(version.base_url, cmd, params)
class Events(object): class Events(object):
"""usage: def __init__(self, url, subparser=None):
klugman.py events [options] self.url = url
if subparser:
parser = subparser.add_parser('events')
parser.add_argument('--name', metavar='event_name',
help='Return events of type event_name.')
parser.add_argument('--from', metavar='datetime', dest='_from',
help='Return events generated before datetime')
parser.add_argument('--to', metavar='datetime',
help='Return events generated after datetime')
parser.add_argument('--traits', metavar='trait_list',
help='Return events with specific traits.')
options: group = parser.add_mutually_exclusive_group()
--debug group.add_argument('--count', action='store_true',
--name <name> default=False,
return events of type <name> help='Return a count of events matching filter criteria.')
--from <datetime> group.add_argument('--msg_id', metavar='message_id',
list events generated before datetime help='Return a single specific event by message id.')
--to <datetime>
list events generated after datetime
--traits <traits>
list events with specific traits
--msg_id <message_id>
get event with <message_id>
Trait format:
"trait:value;trait:value;..."
"""
def cmdline(self, version, cmdline): def get_events_count(self, from_datetime=None, to_datetime=None,
arguments = docopt(Events.__doc__, argv=cmdline) traits=None, name=None, debug=False):
debug = version.base_args['--debug'] if traits:
if debug: traits = ",".join(["%s:%s" % item for item in traits.items()])
print arguments cmd = "events/count"
params = base.remove_empty({'from_datetime': from_datetime,
'to_datetime': to_datetime,
'event_name': name,
'traits': traits})
response = self.do_event(version, arguments) return base.get(self.url, cmd, params, debug=debug)
raw_rows = response.json(object_hook=jsonutil.object_hook)
if isinstance(raw_rows, dict): def get_events(self, from_datetime=None, to_datetime=None,
raw_rows = [raw_rows] traits=None, name=None, msg_id=None, debug=False):
if msg_id:
return base.get(self.url, "events/%s" % msg_id, {}, debug=debug)
if traits:
traits = ",".join(["%s:%s" % item for item in traits.items()])
cmd = "events"
params = base.remove_empty({'from_datetime': from_datetime,
'to_datetime': to_datetime,
'event_name': name,
'traits': traits})
return base.get(self.url, cmd, params, debug=debug)
def _cmdline(self, arguments):
_from = arguments._from
_to = arguments.to
_name = arguments.name
_traits = arguments.traits
_debug = arguments.debug
trait_dict = None
if _traits:
trait_pairs = _traits.split(',')
trait_dict = dict([tuple(item.split(':')) for item in trait_pairs])
if arguments.count:
rows = self.get_events_count(from_datetime=_from, to_datetime=_to,
name=_name, traits=trait_dict,
debug=_debug)
keys = ['count']
base.dump_response(keys, rows)
return
_msg_id = arguments.msg_id
rows = self.get_events(from_datetime=_from, to_datetime=_to,
name=_name, traits=trait_dict,
msg_id=_msg_id, debug=_debug)
if isinstance(rows, dict):
rows = [rows]
keys = set() keys = set()
for row in raw_rows: for row in rows:
keys.update(row.keys()) keys.update(row.keys())
keys = sorted(list(keys)) keys = sorted(list(keys))
base.dump_response(keys, raw_rows) base.dump_response(keys, rows)
def do_event(self, version, arguments):
_from = arguments.get('--from')
_to = arguments.get('--to')
name = arguments.get('--name')
traits = arguments.get('--traits')
msg_id = arguments.get('--msg_id')
if msg_id:
cmd = "events/%s" % msg_id
return base.get(version.base_url, cmd, {})
cmd = "events"
params = base.remove_empty({'from_datetime': _from,
'to_datetime': _to,
'event_name': name,
'traits': traits})
return base.get(version.base_url, cmd, params)
class NumEvents(object): class V1(object):
"""usage: def __init__(self, url, parser):
klugman.py num-events [options] subparser = parser.add_subparsers(dest='command',
help="V1 API commands")
self.resources = {'events': Events(url, subparser),
'streams': Streams(url, subparser)}
options: def dispatch(self, arguments):
--debug # We could let argparse dispatch automatically, but I
--name <name> # want to make this work as a library as well as a
return events of type <name> # cmdline tool, so it goes to an object vs. a function.
--from <datetime> cmd = arguments.command
list events generated before datetime
--to <datetime>
list events generated after datetime
--traits <traits>
list events with specific traits
Trait format: # This shouldn't be needed, but I'm being paranoid.
"trait:value;trait:value;..." if cmd not in self.resources:
""" print "Unknown command: '%s'" % cmd
return
def cmdline(self, version, cmdline): self.resources[cmd]._cmdline(arguments)
arguments = docopt(NumEvents.__doc__, argv=cmdline)
debug = version.base_args['--debug']
if debug:
print arguments
response = self.do_event(version, arguments)
raw_rows = response.json(object_hook=jsonutil.object_hook)
keys = ['count']
base.dump_response(keys, raw_rows)
def do_event(self, version, arguments):
_from = arguments.get('--from')
_to = arguments.get('--to')
name = arguments.get('--name')
traits = arguments.get('--traits')
cmd = "events/count"
params = base.remove_empty({'from_datetime': _from,
'to_datetime': _to,
'event_name': name,
'traits': traits})
return base.get(version.base_url, cmd, params)
class V1(base.Impl):
"""usage:
klugman.py streams [<args>...] [options]
klugman.py num-streams [<args>...] [options]
klugman.py events [<args>...] [options]
klugman.py num-events [<args>...] [options]
-h, --help show command options
"""
def __init__(self, base_url, base_args):
cmds = {'streams': Streams(),
'num-streams': NumStreams(),
'events': Events(),
'num-events': NumEvents()}
super(V1, self).__init__(base_url, base_args, cmds, V1.__doc__)

80
klugman/v2.py
View File

@ -1,80 +0,0 @@
# Copyright (c) 2014 Dark Secret Software Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import base
import v1
import jsonutil
from docopt import docopt
import requests
class Archives(object):
"""usage:
klugman.py archives <start_datetime> <end_datetime> [options]
options:
-h, --help
<start_datetime> starting datetime range
<end_datetime> ending datetime range
"""
def cmdline(self, version, cmdline):
arguments = docopt(Archives.__doc__, argv=cmdline)
if version.base_args['--debug']:
print arguments
response = self.do_archives(version, arguments)
raw_rows = response.json(object_hook=jsonutil.object_hook)
keys = ['id', 'filename']
base.dump_response(keys, raw_rows)
def do_archives(self, version, arguments):
start = arguments.get('--start')
end = arguments.get('--end')
cmd = "archives"
params = base.remove_empty({'start_ts': start,
'end_ts': end})
return base.get(version.base_url, cmd, params)
class V2(base.Impl):
# Note the [<args>...] [options] approach
# which basically says "anything is acceptable".
# We will be more strict in the actual command handler.
"""Klugman - StackTach.v3 client
Usage:
klugman.py [options] streams [<args>...]
klugman.py num-streams [<args>...] [options]
klugman.py events [<args>...] [options]
klugman.py [options] archives [<args>...]
Options:
-h, --help show command options
"""
def __init__(self, base_url, base_args):
cmds = {'streams': v1.Streams(),
'num-streams': v1.NumStreams(),
'events': v1.Events(),
'archives': Archives()}
super(V2, self).__init__(base_url, base_args, cmds, V2.__doc__)

2
setup.cfg
View File

@ -1,6 +1,6 @@
[metadata] [metadata]
name = klugman name = klugman
version = 0.4 version = 0.5
author = Dark Secret Software Inc. author = Dark Secret Software Inc.
author-email = admin@darksecretsoftware.com author-email = admin@darksecretsoftware.com
summary = StackTach.v3 Client summary = StackTach.v3 Client