openstack/adjutant
Code Issues Proposed changes

Updating the docs to match current state

Browse Source 
* While the incode documentation has been updated quite
  a bit, the readme has remained mostly untouched for too
  long and needed some updates.
* Also added some notes about deprecation of older style
  taskviews that ended up never being used in production
  anyway.

Change-Id: I7f21b68e1e37fec0bcba75eeccc35213c4e0a5c0
This commit is contained in:
adrian-turjak 2017-05-26 17:46:11 +12:00
parent dba11b87d1
commit 3a35e66a66
3 changed files with 192 additions and 226 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

123
DEVSTACK_GUIDE.md
View File

@ -1,18 +1,18 @@
# Deploying Adjutant in Devstack
This is a guide to setting up Adjutant in a running Devstack environment similar to how we have been running it for development purposes.
This is a guide to setting up Adjutant in a running Devstack environment close to how we have been running it for development purposes.
This guide assumes you are running this in a clean ubuntu 14.04 virtual machine with sudo access.
This guide assumes you are running this in a clean ubuntu 16.04 virtual machine with sudo access.
## Deploy Devstack
Grab the Devstack repo. For this we are going to focus on mitaka.
Grab the Devstack repo.
```
git clone https://github.com/openstack-dev/devstack.git -b stable/mitaka
git clone https://github.com/openstack-dev/devstack.git
```
And then define a basic localrc file with the password set and place that in the devstack folder:
And then define a basic localrc file with the password set and place that in the devstack folder (adjutant's default conf assumes 'openstack' as the admin password):
```
ADMIN_PASSWORD=openstack
MYSQL_PASSWORD=openstack
@ -44,8 +44,9 @@ virtualenv venv
source venv/bin/activate
```
Once that is done you can install Adjutant into your virtual environment, which will also install all the required python libraries:
Once that is done you can install Adjutant and its requirements:
```
pip install -r requirements.txt
python setup.py develop
```
@ -74,8 +75,11 @@ In a new SSH termimal connected to your ubuntu VM setup your credentials as envi
```
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_TENANT_NAME=demo
export OS_AUTH_URL=http://localhost:5000/v2.0
export OS_PROJECT_NAME=demo
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_DOMAIN_NAME=default
export OS_AUTH_URL=http://localhost:5000/
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME=RegionOne
```
@ -95,6 +99,10 @@ To allow certain actions, Adjutant requires two special roles to exist. You can
openstack role create project_admin
openstack role create project_mod
```
Also because Adjutant by default also adds the role, you will want to create 'heat_stack_owner' which isn't by default present in devstack unless you install Heat:
```
openstack role create heat_stack_owner
```
## Testing Adjutant via the CLI
@ -103,11 +111,11 @@ Now that the service is running, and the endpoint setup, you will want to instal
```
sudo pip install python-adjutantclient
```
In this case the client is safe to install globally as none of its requirements will conflict with OpenStack.
In this case the client should be safe to install globally with sudo, but you can also install it in the same virtualenv as Adjutant itself, or make a new virtualenv.
Now lets check the status of the service:
```
adjutant status
openstack adjutant status
```
What you should get is:
@ -120,16 +128,17 @@ What you should get is:
```
Seeing as we've done nothing to the service yet this is the expected output.
To list the users on your current project (admin users are hidden):
To list the users on your current project (admin users are hidden by default):
```
adjutant user-list
openstack project user list
```
The above action is only possibly for users with the following roles: 'admin', 'project_admin', 'project_mod'
Now lets try inviting a new user:
```
adjutant user-invite --email bob@example.com --roles project_admin
openstack project user invite bob@example.com project_admin
```
You then then get a note saying your invitation has been sent followed by a print out of the users on your current project (same as doing 'adjutant user-list').
You will then get a note saying your invitation has been sent. You can list your project users again with 'openstack project user list' to see your invite.
Now if you look at the log in the Adjutant terminal you should still have open, you will see a print out of the email that would have been sent to bob@example.com. In the email is a line that looks like this:
@ -140,12 +149,13 @@ Normally that would direct the user to a Horizon dashboard page where they can s
Since we don't have that running, your only option is to submit it via the CLI. This is cumbersome, but doable. From that url in your Adjutant output, grab the values after '.../token/'. That is bob's token. You can submit that via the CLI:
```
adjutant token-submit <bobs_token> --data '{"password": "123456"}'
openstack admin task token submit <token> <json_data>
openstack admin task token submit e86cbfb187d34222ace90845f900893c '{"password": "123456"}'
```
Now if you get the user list, you will see bob is now active:
```
adjutant user-list
openstack project user list
```
And also shows up as a user if you do:
@ -153,76 +163,19 @@ And also shows up as a user if you do:
openstack user list
```
And since you are an admin, you can even take a look at the tasks themselves:
```
openstack admin task list
```
The topmost one should be your invite, and if you then do a show using that id you can see some details about it:
```
openstack admin task show <UUID>
```
## Setting Up Adjutant on Horizon
This is a little annoying, since we can't simple install the changes as a plugin, but for the purposes of our demo we will be using a fork of Horizon with our internal Horizon changes rebased on top.
Adjutant has a Horizon UI plugin, the code and setup instructions for it can be found here:
https://github.com/catalyst/adjutant-ui
First grab the Adjutant fork of horizon:
```
git clone https://github.com/catalyst/horizon.git -b stable/mitaka_adjutant
```
Now we will copy the code from that repo to replace the code devstack is using:
```
cp -r horizon/* /opt/stack/horizon/
```
We will also need to add the Adjutant url to the local_settings file for the non-authed views:
```
echo 'OPENSTACK_REGISTRATION_URL="http://0.0.0.0:5050/v1"' >> /opt/stack/horizon/openstack_dashboard/local/local_settings.py
```
Now we need to restart apache:
```
sudo service apache2 restart
```
Now if you go to your devstack Horizon dashboard you will be able to access the new panel and new un-authed views.
To help testing token submission, you probably will want to update this line in the Adjutant conf from:
```
TOKEN_SUBMISSION_URL: http://192.168.122.160:8080/dashboard/token/
```
To point to the ip or url of your devstack VM:
```
TOKEN_SUBMISSION_URL: <ip_or_url_to_devstack>/dashboard/token/
```
Setting that url will mean the token links send in the emails will be usable via your Horizon.
## Sending email via a mail server
By default the conf is set to output emails to console, but if you have a mail server you can use update the conf to reflect that and the service will send emails through that.
Simply replace this part of the conf:
```yaml
EMAIL_SETTINGS:
EMAIL_BACKEND: django.core.mail.backends.console.EmailBackend
```
With this, adding in your server and port:
```yaml
EMAIL_SETTINGS:
EMAIL_BACKEND: django.core.mail.backends.smtp.EmailBackend
EMAIL_HOST: <server_url>
EMAIL_PORT: <server_port>
```
Or this if your server needs a username and password:
```yaml
EMAIL_SETTINGS:
EMAIL_BACKEND: django.core.mail.backends.smtp.EmailBackend
EMAIL_HOST: <server_url>
EMAIL_PORT: <server_port>
EMAIL_HOST_USER: <username>
EMAIL_HOST_PASSWORD: <password>
```
Once the service has reset, it should now send emails via that server rather than print them to console.
## Updating adjutant
Adjutant doesn't have a typical manage.py file, instead this functionality is installed into the virtual enviroment when adjutant is installed.
All of the expected Django functionality can be used using the 'adjutant-api' cli.
If you do set this up, you will want to edit the default Adjutant conf to so that the TOKEN_SUBMISSION_URL is correctly set to point at your Horizon.

149
README.md
View File

@ -2,7 +2,7 @@
A basic workflow framework built using Django and Django-Rest-Framework to help automate basic Admin tasks within an OpenStack cluster.
Primarily built as user registration service that fits into the OpenStack ecosystem alongside Keystone, its purpose to fill functionality missing from Keystone. Ultimately it is just a framework with actions that are tied to an endpoint and can require certain data fields and perform actions via the OpenStack clients.
Primarily built as user registration service that fits into the OpenStack ecosystem alongside Keystone, its purpose to fill functionality missing from Keystone. Ultimately it is just a framework with actions that are tied to an endpoint and can require certain data fields and perform actions via the OpenStack clients as well as talk to external systems as needed.
Useful for automating generic admin tasks that users might request but otherwise can't do without the admin role. Also allows automating the signup and creation of new users, but also allows such requests to require approval first if wanted. Due to issuing of uri+tokens for final steps of some actions, allows for a password submit/reset system as well.
@ -33,13 +33,13 @@ There are cases and TaskViews that auto-approve, and thus automatically do the m
Actions themselves can also effectively do anything within the scope of those three stages, and there is even the ability to chain multiple actions together, and pass data along to other actions.
The points that are modular, or will be made more modular in future, are the TaskViews and the actions tied to them. Adding new actions is easy, and attaching them to existing TaskViews is as well. Adding new TaskViews is also fairly easy, but will be made more modular in future (see Future Plans).
The points that are modular, or will be made more modular in future, are the TaskViews and the actions tied to them. Adding new actions is easy, and attaching them to existing TaskViews is as well. Adding new TaskViews is also almost entirely modular.
Creation and management of Tasks, Tokens, and Notifications is not modular and is the framework around the defined Actions and TaskViews that handles how they are executed. This helps keep the way Actions are executed consistent and simpler to maintain, but does also allow Actions to run almost any logic within those consistent steps.
#### Admin Endpoints:
Endpoints for the management of tasks, tokens, and notifications. Most of these are limited by roles, and are for admin use only.
Endpoints for the management of tasks, tokens, and notifications. Most of these are limited by roles, or are for admin use only.
* ../v1/tasks - GET
* A json containing all tasks.
@ -55,7 +55,7 @@ Endpoints for the management of tasks, tokens, and notifications. Most of these
* approve a task
* ../v1/token - GET
* A json containing all tokens.
* This will be updated to take parameters to refine the list.
* Can also be filtered.
* ../v1/token - POST
* Reissue tokens for a given task.
* ../v1/token - DELETE
@ -66,6 +66,7 @@ Endpoints for the management of tasks, tokens, and notifications. Most of these
* submit the token.
* ../v1/notification - GET
* Get a list of all unacknowledged notifications.
* Can also be filtered.
* ../v1/notification - POST
* Acknowledge a list of notifications.
* ../v1/notification/<id> - GET
@ -87,12 +88,56 @@ Example:
{'filters': {'task_id': { 'exact': '842433bb-fa08-4fc1-8c3b-aa9904ceb370'}}
```
This looks a bit messy in the url as that json ends up being url-safe encoded, but doing the filters this way gives us a fairly large amount of flexibility.
Possible field lookup operations:
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#id4
#### Default TaskView Endpoints:
Basic default endpoints for the TaskViews.
#### OpenStack Style TaskView Endpoints:
For ease of integration with OpenStack, these endpoints are setup to work and partly mimic the way similar ones would work in Keystone. They work and use the TaskViews, with some specical changes for certain required endpoints.
* ../v1/openstack/users - GET
* Returns a list of users on your project, and their roles.
* Also returns a list of pending user invites.
* ../v1/openstack/users - POST
* authenticated endpoint limited by role
* auto-approved
* add/invite a user to your project
* adds an existing user with the selected role, or if non-existent user sends a uri+token to them for setup user before adding the role.
* allows adding of users to own project without needing an admin role
* ../v1/openstack/users/<user_id> - GET
* Get details on the given user, including their roles on your project.
* ../v1/openstack/users/<user_id> - DELETE
* Used to cancel a pending user invite.
* ../v1/openstack/users/<user_id>/roles - GET
* Returns a list of roles for the user on your project.
* ../v1/openstack/users/<user_id>/roles - PUT
* Add roles to a user.
* ../v1/openstack/users/<user_id>/roles - DELETE
* Remove roles from a user.
* ../v1/openstack/roles - GET
* Returns a list of roles you are allowed to edit on your project.
* ../v1/openstack/forgotpassword - POST
* Submit a username/email to have a password reset token emailed to it.
* ../v1/openstack/email-update - POST
* Submit a new email address for your user.
* Will notifiy old email address, and send a uri+token to new email to confirm.
* On confirm will update email (or username if username is email)
* ../v1/openstack/sign-up - POST
* unauthenticated endpoint
* for signup of new users/projects.
* task requires manual approval, sends a uri+token for password setup after the project is created and setup.
* create project
* setup basic networking if needed
* create user with random password
* set user given password on token submit
#### (DEPRECATED) Default TaskView Endpoints:
Basic default endpoints for the TaskViews. These are still mostly used for testing, but are not really for production use. We will be getting rid of them eventually, although they do currently serve as a good basis for some of the more complex views.
* ../v1/actions/CreateProject - GET
* return a json describing the actions and required fields for the endpoint.
@ -101,9 +146,9 @@ Basic default endpoints for the TaskViews.
* for signup of new users/projects.
* task requires manual approval, sends a uri+token for password setup after the project is created and setup.
* create project
* add admin user to project
* setup basic networking if needed
* create user on password submit
* create user with random password
* set user given password on token submit
* ../v1/actions/InviteUser - GET
* return a json describing the actions and required fields for the endpoint.
* ../v1/actions/InviteUser - POST
@ -133,31 +178,6 @@ Basic default endpoints for the TaskViews.
* takes an email address
* issue a uri+token to new email to update to that email
#### OpenStack Style TaskView Endpoints:
For ease of integration with OpenStack, these endpoints are setup to work and partly mimic the way similar ones would work in Keystone. They work and use the TaskViews, with some specical changes for certain required endpoints.
* ../v1/openstack/users - GET
* Returns a list of users on your project, and their roles.
* Also returns a list of pending user invites.
* ../v1/openstack/users - POST
* See above, same as ../v1/actions/InviteUser - POST
* ../v1/openstack/users/<user_id> - GET
* Get details on the given user, including their roles on your project.
* ../v1/openstack/users/<user_id> - DELETE
* Used to cancel a pending user invite.
* ../v1/openstack/users/<user_id>/roles - GET
* Returns a list of roles for the user on your project.
* ../v1/openstack/users/<user_id>/roles - PUT
* Add roles to a user.
* ../v1/openstack/users/<user_id>/roles - DELETE
* Remove roles from a user.
* ../v1/openstack/roles - GET
* Returns a list of roles you are allowed to edit on your project.
* ../v1/openstack/forgotpassword - POST
* Submit a username/email to have a password reset token emailed to it.
#### More API Documentation:
While in debug mode the service will supply online browsable documentation via Django REST Swagger.
@ -187,7 +207,7 @@ Each action class has the functions "pre_approve", "post_approve", and "submit".
Multiple actions can be chained together under one Task and will execute in the defined order. Actions can pass information along via an in memory cache/field on the task object, but that is only safe for the same stage of execution. Actions can also store data back to the database if their logic requires some info passed along to a later step of execution.
See **actions.models** for a good idea of Actions.
See **actions.models** and **actions.v1** for a good idea of Actions.
#### What is a Task?
@ -207,7 +227,7 @@ TaskViews are classes which extend the base TaskView class and use its imbuilt f
The TaskView will process incoming data and build it into a Task, and the related Action classes.
They are very simple to define as the inbuilt functions handle all the real logic, but defining which functions of those are called changes the view to create a task that either requires approval or auto-approves.
They are very simple to define as the inbuilt functions handle all the real logic, but defining which functions of those are called changes the view to create a task that either requires approval or auto-approves, with some cases auto-approval coming from the actions themselves if setup to do so.
The base TaskView class has three functions:
@ -225,6 +245,8 @@ See **api.v1.tasks** and look at the TaskView class to get a better idea.
For a more complex variant, look at **api.v1.openstack** to see some more unique TaskViews specific to certain endpoints we needed to mimic OpenStack functionality.
In fact, at base these are ApiViews from Django Rest Framework, with some magic built in functions for task processing. You normally will want to pick a HTTP method for your core task itself, but having the other methods return data related to the task, or even trigger slight variants of a task is doable.
## Development:
@ -234,7 +256,7 @@ While this is a Django application, it does not follow the standard Django folde
Rather than a standard Django application, treat this as a more standard python application in this regard.
Once installed, all the normal manage.py functions can be called directly on the 'adjutant' commandline function.
Once installed, all the normal manage.py functions can be called directly on the 'adjutant-api' commandline function.
### Dev Environment:
@ -256,49 +278,30 @@ $ tox
```
To run just action unit tests:
```
$ tox adjutant.actions
$ tox -- adjutant.actions
```
To run a single api test:
```
$ tox adjutant.api.v1.tests.test_api_taskview.TaskViewTests.test_duplicate_tasks_new_user
$ tox -- adjutant.api.v1.tests.test_api_taskview.TaskViewTests.test_duplicate_tasks_new_user
```
### Adding Actions:
Adding new actions is done by creating a new django app in the actions module and defining the action models and their serializers. Action must extend the BaseAction class as defined in the **actions.models** module. They also must add themselves to the global store of actions (see the bottom of existing models modules).
Adding new actions is done by creating a new django app in the actions module and defining the action models and their serializers. Action must extend the BaseAction class as defined in the **actions.models.v1.base** module. They also must register themselves to the global store of actions in **action.models**.
The documentation for this is mainly inline.
For examples of actions look in: **action.models** and **actions.tenant_setup.models**.
For examples of actions look in: **action.models.v1.users**, **action.models.v1.project**, and **action.models.v1.resources**
### Adding TaskViews:
Ideally this will be made pluggable in future, but for now requires updating and adding a url entry in api.v1.urls and adding an additional TaskView linked to that url.
TaskViews also need to be registered, and then are made active in the conf. To see how to register a TaskView look at **api.v1.models** and in the default conf under **ACTIVE_TASKVIEWS** too see how they are enabled.
For examples see the classes CreateProject, InviteUser, and ResetPassword in the **api.v1.tasks** module, and **api.v1.openstack**.
For examples see **api.v1.openstack**.
## Setup/Deployment:
### Debian packaging
Debian packaging is done with dh-virtualenv (https://github.com/spotify/dh-virtualenv).
When a package is built, a new virtualenv is created and populated using the contents of requirements.txt.
Package building setup:
apt-get install build-essential devscripts dh-virtualenv
Incrementing package version:
'vim setup.py' and increment version parameter.
'dch -i' and write release notes, set version.
Build the package:
dpkg-buildpackage -us -uc
Now a debian package has been built that will unpack a virtualenv containing adjutant and all dependencies in a self-contained package, so they do not conflict with other python packages on the system.
### Puppet module
Then a puppet module will be able to install the debian package, setup a database, and run the service via nginx and uwsgi in the virtualenv.
### Custom Email Templates
Custom email templates are placed in:
@ -307,36 +310,42 @@ Custom email templates are placed in:
```
This is so that adding personalised or deployment specific templates is kept outside of the scope of the service itself and managed by the deployer.
## Plugins
As Adjutant is built on top of Django, we've used parts of Django's installed apps system to allow us a plugin mechanism that allows additional actions and views to be brought in via external sources. This allows company specific or deployer specific changes to easily live outside of the core service and simply extend the core service where and when need.
An example of such a plugin is here:
https://github.com/catalyst/adjutant-odoo
## Future Plans:
Most future plans are around adding additional Actions to the service, but there will be some features that will require some refactoring.
While we are presently working with the Keystone V3 API, groups are not being used, but we intend to update the service to also manage and handle user groups. Managing Domains isn't really doable, but having the service be able to accept Domains, and multiple Domain back-ends is being planned.
Additional Actions we wish to add in the near future:
Additional Actions and TaskViews we wish to add in the near future:
* Update Quota
* Admin is required to do this
* Allows users to request a quota increase and by requiring an admin to simply check, and confirm the request, will make the process faster.
* Makes it effectively a quick 2 step process.
* For eventual heirarchical-multi-tenancy this would ideally send these qouta increase requests to the parent for approval.
* Hierarchical Multi-Tenancy in a single domain environment
* 'project_admin' to be able to create sub-projects off the current scoped project.
* This works as per normal in Keystone but does not require an admin role and enforces a naming convention to ensure unique namespaces per sub-projects and somewhat avoid the project name uniqueness issues per domain.
* This also adds inherited role support to the already existing user invite and user role editing features.
* some VERY basic sub-project quota (number of sub-projects allowed) via metadata stored on a project in Keystone, with quota calculations in Adjutant checking against number of sub-projects created in your WHOLE tree within given shifting window.
* Stand-alone Setup Network Action + TaskView
* For users who missed or forgot the step at Project creation, and want a quick network setup.
* Blank Slate
* Doesn't need admin, and could reuse your own token.
* Clear my entire project of any and all resources.
* Already doable via the APIs, but would be nice to have it in an easy to request action so clients don't need to code it themselves.
Additional Actions for our own deployment (may be useful to others):
* openERP client create
* An Action that will create a client in openERP and link them to the Project
* Or if the Client already exists, just link them to the project.
* Will need to automatically validate that the client doesn't already exist.
* A way to setup and manage MFA credentials for a user
* relies on work in Keystone around MFA, and serves only as a means to allow an initial challenge response that requires an initial passcode before MFA would become active for the user in Keystone.
Features that might require a slight refactor:
* Break out the TaskViews into a set of explorable endpoints so adding and removing TaskViews is easier.
* Add optional serializers for token data.
Even less likely, and further far-future additions:

4
adjutant/api/v1/tasks.py
View File

@ -365,6 +365,10 @@ class TaskView(APIViewWithLogger):
return {'notes': ["Task completed successfully."]}, 200
# NOTE(adriant): We should deprecate these TaskViews properly and switch tests
# to work against the openstack ones. One option is making these abstract
# classes, so we retain the code here, but make them useless without extension.
class CreateProject(TaskView):
task_type = "create_project"