Initial import of osel code
This is an initial import of the osel codebase. The osel tool is a tool that initiates external security scans (initially through Qualys) upon reciept of AMQP events that indicate certain sensitive events have occurred, like a security group rule change. The commit history had to be thrown away because it contained some non-public data, so I would like to call out the following contributors: This uses go 1.10 and vgo for dependency management. Co-Authored-By: Charles Bitter <Charles_Bitter@cable.comcast.com> Co-Authored-By: Olivier Gagnon <Olivier_Gagnon@cable.comcast.com> Co-Authored-By: Joseph Sleiman <Joseph_Sleiman@comcast.com> Change-Id: Ib6abe2024fd91978b783ceee4cff8bb4678d7b15
This commit is contained in:
Nate Johnston
parent
22f4339944
commit
ca0e1ca769
1
.gitignore
vendored
Normal file
1
.gitignore
vendored
Normal file
@ -0,0 +1 @@
|
||||
osel
|
||||
18
Makefile
Normal file
18
Makefile
Normal file
@ -0,0 +1,18 @@
|
||||
# Makefile for the 'osel' project.
|
||||
|
||||
# Note that the installation of go and vgo is accomplished by
|
||||
# tools/test-setup.sh
|
||||
|
||||
SOURCE?=./...
|
||||
|
||||
env:
|
||||
@echo "Running build"
|
||||
$(HOME)/go/bin/vgo build
|
||||
|
||||
test:
|
||||
@echo "Running tests"
|
||||
$(HOME)/go/bin/vgo test $(SOURCE) -cover
|
||||
|
||||
fmt:
|
||||
@echo "Running fmt"
|
||||
go fmt $(SOURCE)
|
||||
89
README.md
Normal file
89
README.md
Normal file
@ -0,0 +1,89 @@
|
||||
OpenStack Event Listener
|
||||
========================
|
||||
|
||||
What does this do?
|
||||
------------------
|
||||
|
||||
The OpenStack Event Listener connects to the OpenStack message bus (RabbitMQ)
|
||||
and listens for certain kinds of events. When it detects those events, it will
|
||||
gather additional data and forward the information to external systems for
|
||||
processing. It integrates with syslog and the Qualys API.
|
||||
|
||||
The initial use case that inspired this project was to detect when security
|
||||
group changes occurred and to trigger an external port scan of the affected IP
|
||||
addresses so that we could ensure that the change did not create a new
|
||||
vulnerability by opening something up to the Internet.
|
||||
|
||||
For more background information on this project, see [the story of
|
||||
osel](STORY.md).
|
||||
|
||||
Current State
|
||||
-------------
|
||||
Code maturity is considered experimental.
|
||||
|
||||
Installation
|
||||
------------
|
||||
Use `go get git.openstack.org/openstack/osel`. Or alternatively,
|
||||
download or clone the repository.
|
||||
|
||||
The lib was developed and tested on go 1.10.
|
||||
|
||||
Configuration
|
||||
-------------
|
||||
|
||||
Configuration resides in a YAML-format configuration file. Before running the
|
||||
os_event_listener process set the EL_CONFIG environment variable to the
|
||||
absolute path to that file.
|
||||
|
||||
This is an example of the configuration format:
|
||||
|
||||
```yaml
|
||||
debug: true
|
||||
batch_interval: 2
|
||||
rabbit_uri: "amqp://amqp_user:amqp_password@amqp_host:amqp_port//"
|
||||
logfile: "/var/log/os_event_listener.log"
|
||||
syslog_server: your.syslog.server.fqdn
|
||||
syslog_port: "514"
|
||||
syslog_protocol: "tcp"
|
||||
retry_syslog: "false"
|
||||
openstack:
|
||||
identity_endpoint: "https://keystone.url:5000/v2.0/"
|
||||
tenant_name: "tenant_to_authenticate_against"
|
||||
user: "username"
|
||||
password: "password"
|
||||
region: "region_name"
|
||||
qualys:
|
||||
username: "qualys_username"
|
||||
password: "qualys_password"
|
||||
option: "Name Of The Qualys Scan Profile"
|
||||
proxy_url: "http://in.case.you.need.to.proxy.to.reach.qualys/"
|
||||
url: "https://qualysapi.qualys.com/api/2.0/fo/scan/"
|
||||
drop6: true
|
||||
```
|
||||
|
||||
Testing
|
||||
-------
|
||||
There is one type of test file. The `*_test.go` are standard golang unit test
|
||||
files. The examples can be run as integration tests.
|
||||
|
||||
License
|
||||
-------
|
||||
Apache v2.
|
||||
|
||||
Contributing
|
||||
------------
|
||||
The code repository utilizes the OpenStack CI infrastructure. Please use the
|
||||
[recommended
|
||||
workflow](http://docs.openstack.org/infra/manual/developers.html#development-workflow).
|
||||
If you are not a member yet, please consider joining as an [OpenStack
|
||||
contributor](http://docs.openstack.org/infra/manual/developers.html). If you
|
||||
have questions or comments, you can email the maintainer(s).
|
||||
|
||||
Coding Style
|
||||
------------
|
||||
The source code is automatically formatted to follow `go fmt`.
|
||||
|
||||
OpenStack Environment
|
||||
---------------------
|
||||
* Release note management is done using [reno](https://docs.openstack.org/reno/latest/user/usage.html)
|
||||
* Zuul CI jobs are defined in-repo, [using these techniques](https://docs.openstack.org/infra/manual/zuulv3.html#howto-in-repo)
|
||||
141
STORY.md
Normal file
141
STORY.md
Normal file
@ -0,0 +1,141 @@
|
||||
# OpenStack Event Listener
|
||||
|
||||
## What is STORY.md?
|
||||
|
||||
Retweeted Safia Abdalla (@captainsafia):
|
||||
|
||||
From now on, you can expect to see a "STORY.md" in each of my @github repos
|
||||
that describes the technical story/process behind the project.
|
||||
|
||||
https://twitter.com/captainsafia/status/839587421247389696
|
||||
|
||||
## Introduction
|
||||
|
||||
I wanted to write a little about a project that I enjoyed working on, called
|
||||
the OpenStack Event Listener, or "OSEL" for short. This project bridged the
|
||||
OpenStack control plane on one hand, and an external scanning facility provided
|
||||
by Qualys. It had a number of interesting challenges. I was never able to
|
||||
really concentrate on it - this project took about 20% of my time for a period
|
||||
of about 3 months.
|
||||
|
||||
I am writing this partially as catharsis, to allow my brain to mark this part
|
||||
of my mental inventory as ripe for reclamation. I am also writing on the off
|
||||
chance that someone might find this useful in the future.
|
||||
|
||||
## The Setting
|
||||
|
||||
Let me paint a picture of the environment in which this development occurred.
|
||||
|
||||
The Comcast OpenStack environment was transitioning from the OpenStack Icehouse
|
||||
release (very old) to the Newton release (much more current). This development
|
||||
occurred within the context of the Icehouse environment.
|
||||
|
||||
Comcast's security team uses S3 RiskFabric to manage auditing and tracking
|
||||
security vulnerabilities across the board. They also engage the services of
|
||||
Qualys to perform network scanning (in a manner very similar to Nessus) once a
|
||||
day against all the CIDR blocks that comprise Comcast's Internet-routable IP
|
||||
addresses. Qualys scanning could also be triggered on-demand.
|
||||
|
||||
## Technical Requirements
|
||||
|
||||
First, let me describe the technical requirements for OSEL:
|
||||
|
||||
* OSEL would connect to the OpenStack RabbitMQ message bus and register as a
|
||||
listener for "notification" events. This would allow OSEL to inspect all
|
||||
events, including security group changes.
|
||||
* When a security group change occurred, OSEL would ensure that it had the
|
||||
details of the change (ports permitted or blocked) as well as a list of all
|
||||
affected IP addresses.
|
||||
* OSEL would initiate a Qualys scan using the Qualys API. This would return a
|
||||
scan ID.
|
||||
* OSEL would log the change as well as the Qualys scan ID to the Security
|
||||
instance of Splunk to create an audit trail.
|
||||
* Qualys scan results would be imported into S3 RiskFabric for security audit
|
||||
management.
|
||||
|
||||
## Implementation Approach
|
||||
|
||||
My group does most of it's development in Go, and this was no exception.
|
||||
|
||||
This is what the data I was getting back from the AMQP message looked like.
|
||||
All identifiers have been scrambled.
|
||||
|
||||
```json
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-f96ea9a5-435e-4177-8e51-bfe60d0fae2a",
|
||||
"event_type":"security_group_rule.create.end",
|
||||
"timestamp":"2016-10-03 18:10:59.112712",
|
||||
"_context_tenant_id":"ada3b9b06482909f9361e803b54f5f32",
|
||||
"_unique_id":"eafc9362327442b49d8c03b0e88d0216",
|
||||
"_context_tenant_name":"BLURP",
|
||||
"_context_user":"bca89c1b248e4a78282899ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4a78282899ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule_id":"bf8318fc-f9cb-446b-ffae-a8de016c562"
|
||||
},
|
||||
"_context_project_name":"BLURP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b06482909f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b06482909f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 18:10:59.079179",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl1",
|
||||
"message_id":"e75fb2ee-85bf-44ba-a083-2445eca2ae10"
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Pattern
|
||||
|
||||
I leaned heavily on dependency injection to make this code as testable as
|
||||
possible. For example, I needed an object that would contain the persistent
|
||||
`syslog.Writer`. I created a `SyslogActioner` interface to represent all
|
||||
interactions with syslog. When the code is operating normally, interactions
|
||||
with syslog occur through methods of the `SyslogActions` struct. But in unit
|
||||
testing mode the `SyslogTestActions` struct is used instead, and all that does
|
||||
is save copies of all messages that would have been sent so they can be
|
||||
compared against the intended messages. This facilitates good testing.
|
||||
|
||||
## Fate of the Project
|
||||
|
||||
The OSEL project was implemented and installed into production. There were two
|
||||
problems with it.
|
||||
|
||||
The first to become visible is that there was no exponential backoff for the
|
||||
AMQP connection to the OpenStack control plane's RabbitMQ. When that RabbitMQ
|
||||
had issues - which was surprisingly often - OSEL would hanner away, trying to
|
||||
connect to it. That would not be too much of an issue; despite what was
|
||||
effectively an infinite loop, CPU usage was not extreme. The real problem was
|
||||
that connection failures were logged - and logs could become several gigabytes
|
||||
in a matter of hours. This was mitigated by the OpenStack operations team
|
||||
rotating the logs hourly, and alerting if an hour's worth of logs exceeded a
|
||||
set size. It was my intention to use one of the many [exponential backoff
|
||||
modules](https://github.com/cenkalti/backoff) available out there to make this
|
||||
more graceful.
|
||||
|
||||
The second - and fatal - issue is that S3 RiskFabric was not configured to
|
||||
ingest from Qualys scans more than once a day. Since Qualys was already
|
||||
scanning the CIDR block that corresponded to our OpenStack instances once a
|
||||
day, we were essentially just adding noise to the system. The frequency of the
|
||||
S3-Qualys imports could not be easily altered, and as a result the project was
|
||||
shelved.
|
||||
|
||||
## Remaining Work
|
||||
|
||||
If OSEL were ever to be un-shelved, here are a few of the things that I wish I
|
||||
had time to implement.
|
||||
|
||||
- Neutron Port Events: The initial release of OSEL processed only security
|
||||
group rule additions, modifications, or deletions. So that covered the base
|
||||
case for when a security group was already associated with a set of OpenStack
|
||||
Networking (neutron) ports. But a scan should be similarly launched when a
|
||||
new port is created and associated to a security group. This is what happens
|
||||
when a new host is created.
|
||||
- Modern OpenStack: In order to make this work with a more modern OpenStack, it
|
||||
would probably best to integrate with events generated through Aodh. Aodh
|
||||
seems to be built for this kind of reporting.
|
||||
- Implement exponential backoff for AMQP connections as mentioned earlier.
|
||||
92
amqp.go
Normal file
92
amqp.go
Normal file
@ -0,0 +1,92 @@
|
||||
package main
|
||||
|
||||
/*
|
||||
|
||||
amqp - This file includes all of the logic necessary to interact with the amqp
|
||||
library. This is extrapolated out so that a AmqpInterface interface can be
|
||||
passed to functions. Doing this allows testing by mock classes to be created
|
||||
that can be passed to functions.
|
||||
|
||||
Since this is a wrapper around the amqp library, this does not need testing.
|
||||
|
||||
*/
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"github.com/streadway/amqp"
|
||||
)
|
||||
|
||||
// AmqpActioner is an interface for an AmqpActions class. Having
|
||||
// this as an interface allows us to pass in a dummy class for testing that
|
||||
// just returns mocked data.
|
||||
type AmqpActioner interface {
|
||||
Connect() (<-chan amqp.Delivery, error)
|
||||
}
|
||||
|
||||
// AmqpActions is a class that handles all interactions directly with Amqp.
|
||||
// See the comment on AmqpActioner for rationale.
|
||||
type AmqpActions struct {
|
||||
Incoming *<-chan amqp.Delivery
|
||||
Options AmqpOptions
|
||||
AmqpConnection *amqp.Connection
|
||||
AmqpChannel *amqp.Channel
|
||||
NotifyError chan *amqp.Error
|
||||
}
|
||||
|
||||
// AmqpOptions is a class to convey all of the configurable options for the
|
||||
// AmqpActions class.
|
||||
type AmqpOptions struct {
|
||||
RabbitURI string
|
||||
}
|
||||
|
||||
// Connect initiates the initial connection to the AMQP.
|
||||
func (s *AmqpActions) Connect() (<-chan amqp.Delivery, chan *amqp.Error, error) {
|
||||
var err error
|
||||
|
||||
s.AmqpConnection, err = amqp.Dial(s.Options.RabbitURI)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("Failed to connect to RabbitMQ: %s", err)
|
||||
}
|
||||
s.NotifyError = s.AmqpConnection.NotifyClose(make(chan *amqp.Error)) //error channel
|
||||
|
||||
s.AmqpChannel, err = s.AmqpConnection.Channel()
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("Failed to open a channel: %s", err)
|
||||
}
|
||||
|
||||
amqpQueue, err := s.AmqpChannel.QueueDeclare(
|
||||
"notifications.info", // name
|
||||
false, // durable
|
||||
false, // delete when usused
|
||||
false, // exclusive
|
||||
false, // no-wait
|
||||
nil, // arguments
|
||||
)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("Failed to declare a queue: %s", err)
|
||||
}
|
||||
|
||||
amqpIncoming, err := s.AmqpChannel.Consume(
|
||||
amqpQueue.Name, // queue
|
||||
"osel", // consumer
|
||||
true, // auto-ack
|
||||
false, // exclusive
|
||||
false, // no-local
|
||||
false, // no-wait
|
||||
nil, // args
|
||||
)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("Failed to register a consumer: %s", err)
|
||||
}
|
||||
s.Incoming = &amqpIncoming
|
||||
return amqpIncoming, s.NotifyError, nil
|
||||
}
|
||||
|
||||
// Close closes connections
|
||||
func (s AmqpActions) Close() {
|
||||
log.Println("Closing AMQP connection")
|
||||
s.AmqpConnection.Close()
|
||||
s.AmqpChannel.Close()
|
||||
}
|
||||
3
bindep.txt
Normal file
3
bindep.txt
Normal file
@ -0,0 +1,3 @@
|
||||
build-essential
|
||||
make
|
||||
clang
|
||||
57
events.go
Normal file
57
events.go
Normal file
@ -0,0 +1,57 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"log"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// EventProcessor is an Interface for event-specific classes that will process
|
||||
// events based on their specific fiends.
|
||||
type EventProcessor interface {
|
||||
FormatLogs(*Event, []string) ([]string, error)
|
||||
FillExtraData(*Event, OpenStackActioner) error
|
||||
}
|
||||
|
||||
// Event is a class representing an event accepted from the AMQP, and the
|
||||
// additional attributes that have been parsed from it.
|
||||
type Event struct {
|
||||
EventData *openStackEvent
|
||||
RawData []byte
|
||||
IPs map[string][]string
|
||||
SecurityGroupRules []*osSecurityGroupRule
|
||||
LogLines []string
|
||||
Processor EventProcessor
|
||||
QualysScanID string
|
||||
QualysScanError string
|
||||
}
|
||||
|
||||
// ParseEvent takes the []byte that has been received from the AMQP message,
|
||||
// demarshals the JSON, and then returns the event data as well as an event
|
||||
// processor specific to that type of event.
|
||||
func ParseEvent(message []byte) (Event, error) {
|
||||
var osEvent openStackEvent
|
||||
if err := json.Unmarshal(message, &osEvent); err != nil {
|
||||
return Event{}, err
|
||||
}
|
||||
|
||||
e := Event{
|
||||
EventData: &osEvent,
|
||||
RawData: message,
|
||||
}
|
||||
|
||||
if Debug {
|
||||
log.Printf("Event detected: %s\n", osEvent.EventType)
|
||||
}
|
||||
|
||||
switch {
|
||||
case strings.Contains(e.EventData.EventType, "security_group_rule.create.end"):
|
||||
e.Processor = EventSecurityGroupRuleChange{ChangeType: "sg_rule_add"}
|
||||
case strings.Contains(e.EventData.EventType, "security_group_rule.delete.end"):
|
||||
e.Processor = EventSecurityGroupRuleChange{ChangeType: "sg_rule_del"}
|
||||
// case strings.Contains(e.EventData.EventType, "port.create.end"):
|
||||
// e.Processor = EventPortChange{ChangeType: "port_create"}
|
||||
}
|
||||
|
||||
return e, nil
|
||||
}
|
||||
299
events_json_fixtures_test.go
Normal file
299
events_json_fixtures_test.go
Normal file
@ -0,0 +1,299 @@
|
||||
package main
|
||||
|
||||
const (
|
||||
portCreateWhenCreatingInstance = `
|
||||
{
|
||||
"_context_roles": [
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id": "req-fdb23f2e-9c0e-46b1-802f-3194c1fad251",
|
||||
"event_type": "port.create.end",
|
||||
"timestamp": "2016-10-03 18:40:34.596836",
|
||||
"_context_tenant_id": "0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"_unique_id": "bca88f14c46e40559e981ac0b4ffebf5",
|
||||
"_context_tenant_name": "services",
|
||||
"_context_user": "31055c32b50442e5a4eb4c0f0cb3430b",
|
||||
"_context_user_id": "31055c32b50442e5a4eb4c0f0cb3430b",
|
||||
"payload": {
|
||||
"port": {
|
||||
"status": "DOWN",
|
||||
"binding:host_id": "oscomp-ch2-a06",
|
||||
"name": "",
|
||||
"allowed_address_pairs": [
|
||||
|
||||
],
|
||||
"admin_state_up": true,
|
||||
"network_id": "af33487a-4e96-4499-bfcd-4f741617a763",
|
||||
"tenant_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"binding:vif_details": {
|
||||
"port_filter": true,
|
||||
"ovs_hybrid_plug": true
|
||||
},
|
||||
"binding:vnic_type": "normal",
|
||||
"binding:vif_type": "ovs",
|
||||
"device_owner": "compute:None",
|
||||
"mac_address": "fa:16:3e:4a:ac:75",
|
||||
"binding:profile": {
|
||||
},
|
||||
"fixed_ips": [
|
||||
{
|
||||
"subnet_id": "4a23cb36-b861-4daa-a8ef-c61360663669",
|
||||
"ip_address": "162.150.0.117"
|
||||
},
|
||||
{
|
||||
"subnet_id": "244c99a6-8011-4177-855b-dd493c5175c5",
|
||||
"ip_address": "2001:558:fe21:403:f816:3eff:fe4a:ac75"
|
||||
}
|
||||
],
|
||||
"id": "a6c671d7-b4d5-4ebb-afaf-0c822bcc8948",
|
||||
"security_groups": [
|
||||
"0783a151-768c-49d3-a31d-178f70fabd51",
|
||||
"46d46540-98ac-4c93-ae62-68dddab2282e"
|
||||
],
|
||||
"device_id": "128bc33a-22ae-48b4-8283-093b6ec749d0"
|
||||
}
|
||||
},
|
||||
"_context_project_name": "services",
|
||||
"_context_read_deleted": "no",
|
||||
"_context_tenant": "0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"priority": "INFO",
|
||||
"_context_is_admin": true,
|
||||
"_context_project_id": "0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"_context_timestamp": "2016-10-03 18:40:34.477012",
|
||||
"_context_user_name": "neutron",
|
||||
"publisher_id": "network.osctrl-ch2-a03",
|
||||
"message_id": "71047538-531f-4aca-be09-a31bec441d16"
|
||||
}
|
||||
|
||||
`
|
||||
|
||||
securityGroupRuleCreateWithCustomProtocall = `
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-a17c784c-fec9-4077-8908-44b6f56b6196",
|
||||
"event_type":"security_group_rule.create.end",
|
||||
"timestamp":"2016-10-03 17:50:59.982008",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"a7452605170c4979b2c6b76911d22026",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule":{
|
||||
"remote_group_id":null,
|
||||
"direction":"ingress",
|
||||
"protocol":10,
|
||||
"remote_ip_prefix":"10.0.0.0/8",
|
||||
"port_range_max":null,
|
||||
"dscp":null,
|
||||
"security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"port_range_min":null,
|
||||
"ethertype":"IPv4",
|
||||
"id":"3eff38bb-eb03-450b-aed4-019d612baeec"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 17:50:59.925462",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a03",
|
||||
"message_id":"6c93e24f-0892-494b-8e68-46252ceb9611"
|
||||
}
|
||||
`
|
||||
|
||||
securityGroupRuleCreateWithIcmpAndCider = `
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-c584fd21-9e58-4624-b316-b53487eed98e",
|
||||
"event_type":"security_group_rule.create.end",
|
||||
"timestamp":"2016-10-03 18:05:35.836029",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"cd280fd4f1474266bd0ad6e3ee5933a6",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule":{
|
||||
"remote_group_id":null,
|
||||
"direction":"ingress",
|
||||
"protocol":"icmp",
|
||||
"remote_ip_prefix":"192.168.1.0/24",
|
||||
"port_range_max":null,
|
||||
"dscp":null,
|
||||
"security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"port_range_min":null,
|
||||
"ethertype":"IPv4",
|
||||
"id":"66d7ac79-3551-4436-83c7-103b50760cfb"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 18:05:35.769947",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a03",
|
||||
"message_id":"f67b70d5-a782-4c5e-a274-a7ff197b73ec"
|
||||
}
|
||||
`
|
||||
securityGroupRuleCreateWithports = `
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-1f17d667-c33f-4fa4-a026-8e2872dbf1d8",
|
||||
"event_type":"security_group_rule.create.end",
|
||||
"timestamp":"2016-10-03 17:32:25.723344",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"2fad8ecdd86e4748850d91bb0c83d625",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule":{
|
||||
"remote_group_id":null,
|
||||
"direction":"ingress",
|
||||
"protocol":"tcp",
|
||||
"remote_ip_prefix":"10.0.0.0/8",
|
||||
"port_range_max":443,
|
||||
"dscp":null,
|
||||
"security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"port_range_min":443,
|
||||
"ethertype":"IPv4",
|
||||
"id":"2b84d898-67b4-4370-9808-40a3fdb55a64"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 17:32:25.665588",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a03",
|
||||
"message_id":"4df01871-8bdb-4b85-bb34-cbff59ee6034"
|
||||
}
|
||||
`
|
||||
securityGroupRuleCreateWithSecurityGroupAsRemoteIPPrefix = `
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-9e0360c7-786f-4a5b-84b6-7d2ccd23cbdd",
|
||||
"event_type":"security_group_rule.create.end",
|
||||
"timestamp":"2016-10-03 17:36:58.780554",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"b38fe8caed514eb2ba910e1ae74c6321",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule":{
|
||||
"remote_group_id":"0783a151-768c-49d3-a31d-178f70fabd51",
|
||||
"direction":"ingress",
|
||||
"protocol":"tcp",
|
||||
"remote_ip_prefix":null,
|
||||
"port_range_max":25,
|
||||
"dscp":null,
|
||||
"security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"port_range_min":20,
|
||||
"ethertype":"IPv6",
|
||||
"id":"7b14b6cd-f966-4b61-aaad-c03d8eacc830"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 17:36:58.712962",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a03",
|
||||
"message_id":"e2d7c089-8194-4523-8f84-ae22db497f60"
|
||||
}
|
||||
`
|
||||
securityGroupRuleCreateWithSSHOpenToTheInternet = `
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-94df69c6-1c3f-48bd-b2f6-f47abdef5d9b",
|
||||
"event_type":"security_group_rule.create.end",
|
||||
"timestamp":"2016-10-03 18:09:11.938476",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"09412fff881543679f30412ef2342954",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule":{
|
||||
"remote_group_id":null,
|
||||
"direction":"ingress",
|
||||
"protocol":"tcp",
|
||||
"remote_ip_prefix":"0.0.0.0/0",
|
||||
"port_range_max":22,
|
||||
"dscp":null,
|
||||
"security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"port_range_min":22,
|
||||
"ethertype":"IPv4",
|
||||
"id":"bf288dfc-f9cb-446b-bacc-a8de016c9b11"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 18:09:11.876789",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a03",
|
||||
"message_id":"afb043b6-fa56-470b-b17e-984fb4cb6505"
|
||||
}
|
||||
`
|
||||
securityGroupRuleDeleteWithIcmpAndCider = `
|
||||
{
|
||||
"_context_roles": [
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id": "req-836eb80f-c6eb-459b-87b6-a093ebac3051",
|
||||
"event_type": "security_group_rule.delete.end",
|
||||
"timestamp": "2016-10-03 18:14:33.007074",
|
||||
"_context_tenant_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id": "04beeb34769b43bca09ec837d86ed18b",
|
||||
"_context_tenant_name": "VOIP",
|
||||
"_context_user": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload": {
|
||||
"security_group_rule_id": "7b14b6cd-f966-4b61-aaad-c03d8eacc830"
|
||||
},
|
||||
"_context_project_name": "VOIP",
|
||||
"_context_read_deleted": "no",
|
||||
"_context_tenant": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority": "INFO",
|
||||
"_context_is_admin": false,
|
||||
"_context_project_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp": "2016-10-03 18:14:32.962116",
|
||||
"_context_user_name": "admin",
|
||||
"publisher_id": "network.osctrl-ch2-a03",
|
||||
"message_id": "9bc5106c-a08b-4cda-9311-20bc16bc3008"
|
||||
}
|
||||
`
|
||||
)
|
||||
61
events_test.go
Normal file
61
events_test.go
Normal file
@ -0,0 +1,61 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestParseEventWillReturnAnEventStruct(t *testing.T) {
|
||||
event, err := ParseEvent([]byte(securityGroupRuleCreateWithports))
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, "main.Event", reflect.TypeOf(event).String(),
|
||||
"ParseEvent should return an Event struct")
|
||||
assert.Equal(t, "security_group_rule.create.end", event.EventData.EventType)
|
||||
assert.Equal(t, "bca89c1b248e4aef9c69ece9e744cc54", event.EventData.UserID)
|
||||
assert.Equal(t, "admin", event.EventData.UserName)
|
||||
assert.Equal(t, "ada3b9b0dbac429f9361e803b54f5f32", event.EventData.TenantID)
|
||||
assert.Equal(t, "VOIP", event.EventData.TenantName)
|
||||
}
|
||||
|
||||
func TestParseEventWillCreateTheProperEventProcessor(t *testing.T) {
|
||||
e, err := ParseEvent([]byte(securityGroupRuleCreateWithports))
|
||||
assert.Nil(t, err)
|
||||
//assert.Equal(t, "main.EventSecurityGroupRuleChange", reflect.TypeOf(e.Processor).String(),
|
||||
// "ParseEvent should return the proper implementation of EventProcessor")
|
||||
assert.Equal(t, EventSecurityGroupRuleChange{"sg_rule_add"}, e.Processor,
|
||||
"ParseEvent should return the proper implementation of EventProcessor")
|
||||
|
||||
e, err = ParseEvent([]byte(securityGroupRuleDeleteWithIcmpAndCider))
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, "main.EventSecurityGroupRuleChange", reflect.TypeOf(e.Processor).String(),
|
||||
"ParseEvent should return the proper implementation of EventProcessor")
|
||||
|
||||
// _, eventProcessor, err = ParseEvent([]byte(portCreateWhenCreatingInstance))
|
||||
// assert.Nil(t, err)
|
||||
// assert.Equal(t, "main.EventPortChange", reflect.TypeOf(eventProcessor).String(),
|
||||
// "ParseEvent should return the proper implementation of EventProcessor")
|
||||
|
||||
}
|
||||
|
||||
// func TestPortCreateEvent(t *testing.T) {
|
||||
// fakeOpenStack := connectFakeOpenstack()
|
||||
// event, eventProcessor, err := ParseEvent([]byte(portCreateWhenCreatingInstance))
|
||||
// assert.Nil(t, err)
|
||||
// eventProcessor.FillExtraData(&event, fakeOpenStack)
|
||||
//}
|
||||
|
||||
func TestEventSecurityGroupRuleCreateEvent(t *testing.T) {
|
||||
fakeOpenStack := connectFakeOpenstack()
|
||||
event, err := ParseEvent([]byte(securityGroupRuleCreateWithports))
|
||||
assert.Nil(t, err)
|
||||
event.Processor.FillExtraData(&event, fakeOpenStack)
|
||||
}
|
||||
|
||||
func TestEventSecurityGroupRuleDeleteEvent(t *testing.T) {
|
||||
fakeOpenStack := connectFakeOpenstack()
|
||||
event, err := ParseEvent([]byte(securityGroupRuleDeleteWithIcmpAndCider))
|
||||
assert.Nil(t, err)
|
||||
event.Processor.FillExtraData(&event, fakeOpenStack)
|
||||
}
|
||||
6
fixtures/README.md
Normal file
6
fixtures/README.md
Normal file
@ -0,0 +1,6 @@
|
||||
# Test Fixtures
|
||||
|
||||
The fixtures folder is meant to hold test files that can be used to create positive and negative tests.
|
||||
|
||||
The test files should be organized in a folder per _test.go file. The folder should be named after the test. For example foo_test.go
|
||||
should use a folder called foo. Sub folders are fine. Just don't go nuts and KISS
|
||||
@ -0,0 +1,90 @@
|
||||
{
|
||||
"_context_roles": [
|
||||
"Member",
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id": "req-49d3158a-1288-4b76-bed9-907666d7d8c5",
|
||||
"_context_quota_class": null,
|
||||
"event_type": "compute.instance.create.start",
|
||||
"_context_service_catalog": [
|
||||
{
|
||||
"endpoints": [
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "public",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "7d0a1277410648d4a3e0164ad233adea"
|
||||
},
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "admin",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "df06789d23684643a0b59bfa7ee26064"
|
||||
}
|
||||
],
|
||||
"type": "volume",
|
||||
"id": "e90968e859474737a4ad7aadffb2f578"
|
||||
}
|
||||
],
|
||||
"timestamp": "2016-10-03 18:40:33.699871",
|
||||
"_context_user": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_unique_id": "ab304ee51028479d899f38cd100b9b36",
|
||||
"_context_instance_lock_checked": false,
|
||||
"_context_user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload": {
|
||||
"state_description": "scheduling",
|
||||
"availability_zone": null,
|
||||
"terminated_at": "",
|
||||
"ephemeral_gb": 0,
|
||||
"instance_type_id": 2,
|
||||
"message": "",
|
||||
"deleted_at": "",
|
||||
"reservation_id": "r-vgn2gv93",
|
||||
"instance_id": "128bc33a-22ae-48b4-8283-093b6ec749d0",
|
||||
"user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"hostname": "test-sc-group",
|
||||
"state": "building",
|
||||
"launched_at": "",
|
||||
"metadata": {
|
||||
},
|
||||
"node": null,
|
||||
"ramdisk_id": "",
|
||||
"access_ip_v6": null,
|
||||
"disk_gb": 1,
|
||||
"access_ip_v4": null,
|
||||
"kernel_id": "",
|
||||
"image_name": "cirros-0.3.4",
|
||||
"host": null,
|
||||
"display_name": "test_sc_group",
|
||||
"image_ref_url": "http://172.28.195.37:9292/images/caf23aab-f6ec-4691-8f50-b0c70a33eca3",
|
||||
"root_gb": 1,
|
||||
"tenant_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"created_at": "2016-10-03T18:40:33.000000",
|
||||
"memory_mb": 512,
|
||||
"instance_type": "m1.tiny",
|
||||
"vcpus": 1,
|
||||
"image_meta": {
|
||||
"description": "cirros-0.3.4",
|
||||
"container_format": "bare",
|
||||
"min_ram": "0",
|
||||
"disk_format": "qcow2",
|
||||
"min_disk": "1",
|
||||
"base_image_ref": "caf23aab-f6ec-4691-8f50-b0c70a33eca3"
|
||||
},
|
||||
"architecture": null,
|
||||
"os_type": null,
|
||||
"instance_flavor_id": "1"
|
||||
},
|
||||
"_context_project_name": "VOIP",
|
||||
"_context_read_deleted": "no",
|
||||
"_context_auth_token": "7b82d6dec2eb457d85e08940a7f7b33e",
|
||||
"_context_tenant": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority": "INFO",
|
||||
"_context_is_admin": false,
|
||||
"_context_project_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp": "2016-10-03T18:40:32.636697",
|
||||
"_context_user_name": "admin",
|
||||
"publisher_id": "compute.oscomp-ch2-a06",
|
||||
"message_id": "0553fb1f-42d1-4959-8991-4b5034701d9d",
|
||||
"_context_remote_address": "127.0.0.1"
|
||||
}
|
||||
@ -0,0 +1,88 @@
|
||||
{
|
||||
"_context_roles": [
|
||||
"Member",
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id": "req-dceb796f-f5cf-45b0-8f6c-78b669304f4d",
|
||||
"_context_quota_class": null,
|
||||
"event_type": "compute.instance.delete.end",
|
||||
"_context_service_catalog": [
|
||||
{
|
||||
"endpoints": [
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "public",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "7d0a1277410648d4a3e0164ad233adea"
|
||||
},
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "admin",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "df06789d23684643a0b59bfa7ee26064"
|
||||
}
|
||||
],
|
||||
"type": "volume",
|
||||
"id": "e90968e859474737a4ad7aadffb2f578"
|
||||
}
|
||||
],
|
||||
"timestamp": "2016-10-03 18:25:18.941599",
|
||||
"_context_user": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_unique_id": "a4c13937ec624259856d11e0ae0717cf",
|
||||
"_context_instance_lock_checked": false,
|
||||
"_context_user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload": {
|
||||
"state_description": "",
|
||||
"availability_zone": null,
|
||||
"terminated_at": "2016-10-03T18:25:18.000000",
|
||||
"ephemeral_gb": 0,
|
||||
"instance_type_id": 2,
|
||||
"deleted_at": "2016-10-03T18:25:18.857623",
|
||||
"reservation_id": "r-osn0qo9l",
|
||||
"instance_id": "563d0899-d1cc-4154-bb58-f932ad0b255c",
|
||||
"user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"hostname": "test",
|
||||
"state": "deleted",
|
||||
"launched_at": "2016-10-03T18:23:11.000000",
|
||||
"metadata": {
|
||||
},
|
||||
"node": "openstack-host2",
|
||||
"ramdisk_id": "",
|
||||
"access_ip_v6": null,
|
||||
"disk_gb": 1,
|
||||
"access_ip_v4": null,
|
||||
"kernel_id": "",
|
||||
"host": "oscomp-ch2-a07",
|
||||
"display_name": "test",
|
||||
"image_ref_url": "http://172.28.195.38:9292/images/caf23aab-f6ec-4691-8f50-b0c70a33eca3",
|
||||
"root_gb": 1,
|
||||
"tenant_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"created_at": "2016-10-03 18:23:03+00:00",
|
||||
"memory_mb": 512,
|
||||
"instance_type": "m1.tiny",
|
||||
"vcpus": 1,
|
||||
"image_meta": {
|
||||
"description": "cirros-0.3.4",
|
||||
"container_format": "bare",
|
||||
"min_ram": "0",
|
||||
"disk_format": "qcow2",
|
||||
"min_disk": "1",
|
||||
"base_image_ref": "caf23aab-f6ec-4691-8f50-b0c70a33eca3"
|
||||
},
|
||||
"architecture": null,
|
||||
"os_type": null,
|
||||
"instance_flavor_id": "1"
|
||||
},
|
||||
"_context_project_name": "VOIP",
|
||||
"_context_read_deleted": "no",
|
||||
"_context_auth_token": "7b82d6dec2eb457d85e08940a7f7b33e",
|
||||
"_context_tenant": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority": "INFO",
|
||||
"_context_is_admin": false,
|
||||
"_context_project_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp": "2016-10-03T18:25:16.888382",
|
||||
"_context_user_name": "admin",
|
||||
"publisher_id": "compute.oscomp-ch2-a07",
|
||||
"message_id": "cf848ea3-300e-4b9a-8672-2306e9413cbf",
|
||||
"_context_remote_address": "127.0.0.1"
|
||||
}
|
||||
@ -0,0 +1,88 @@
|
||||
{
|
||||
"_context_roles": [
|
||||
"Member",
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id": "req-71eec143-a576-49aa-84f6-eeaccd4f6619",
|
||||
"_context_quota_class": null,
|
||||
"event_type": "compute.instance.shutdown.end",
|
||||
"_context_service_catalog": [
|
||||
{
|
||||
"endpoints": [
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "public",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "7d0a1277410648d4a3e0164ad233adea"
|
||||
},
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "admin",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "df06789d23684643a0b59bfa7ee26064"
|
||||
}
|
||||
],
|
||||
"type": "volume",
|
||||
"id": "e90968e859474737a4ad7aadffb2f578"
|
||||
}
|
||||
],
|
||||
"timestamp": "2016-10-03 18:38:47.143591",
|
||||
"_context_user": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_unique_id": "2740b7baa2fb48fea3c3654bf94c304e",
|
||||
"_context_instance_lock_checked": false,
|
||||
"_context_user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload": {
|
||||
"state_description": "deleting",
|
||||
"availability_zone": null,
|
||||
"terminated_at": "",
|
||||
"ephemeral_gb": 0,
|
||||
"instance_type_id": 2,
|
||||
"deleted_at": "",
|
||||
"reservation_id": "r-s8qu0d07",
|
||||
"instance_id": "8bca63d6-0d51-465b-9379-c20ccc8d3e17",
|
||||
"user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"hostname": "test-instance-one",
|
||||
"state": "active",
|
||||
"launched_at": "2016-10-03T17:23:59.000000",
|
||||
"metadata": {
|
||||
},
|
||||
"node": "openstack-host1",
|
||||
"ramdisk_id": "",
|
||||
"access_ip_v6": null,
|
||||
"disk_gb": 1,
|
||||
"access_ip_v4": null,
|
||||
"kernel_id": "",
|
||||
"host": "oscomp-ch2-a06",
|
||||
"display_name": "test_instance_one",
|
||||
"image_ref_url": "http://172.28.195.37:9292/images/caf23aab-f6ec-4691-8f50-b0c70a33eca3",
|
||||
"root_gb": 1,
|
||||
"tenant_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"created_at": "2016-10-03 17:23:40+00:00",
|
||||
"memory_mb": 512,
|
||||
"instance_type": "m1.tiny",
|
||||
"vcpus": 1,
|
||||
"image_meta": {
|
||||
"description": "cirros-0.3.4",
|
||||
"container_format": "bare",
|
||||
"min_ram": "0",
|
||||
"disk_format": "qcow2",
|
||||
"min_disk": "1",
|
||||
"base_image_ref": "caf23aab-f6ec-4691-8f50-b0c70a33eca3"
|
||||
},
|
||||
"architecture": null,
|
||||
"os_type": null,
|
||||
"instance_flavor_id": "1"
|
||||
},
|
||||
"_context_project_name": "VOIP",
|
||||
"_context_read_deleted": "no",
|
||||
"_context_auth_token": "7b82d6dec2eb457d85e08940a7f7b33e",
|
||||
"_context_tenant": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority": "INFO",
|
||||
"_context_is_admin": true,
|
||||
"_context_project_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp": "2016-10-03T18:38:45.381205",
|
||||
"_context_user_name": "admin",
|
||||
"publisher_id": "compute.oscomp-ch2-a06",
|
||||
"message_id": "1cffe568-1eee-4150-89a0-937973ce15e4",
|
||||
"_context_remote_address": "127.0.0.1"
|
||||
}
|
||||
@ -0,0 +1,88 @@
|
||||
{
|
||||
"_context_roles": [
|
||||
"Member",
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id": "req-dceb796f-f5cf-45b0-8f6c-78b669304f4d",
|
||||
"_context_quota_class": null,
|
||||
"event_type": "compute.instance.shutdown.start",
|
||||
"_context_service_catalog": [
|
||||
{
|
||||
"endpoints": [
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "public",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "7d0a1277410648d4a3e0164ad233adea"
|
||||
},
|
||||
{
|
||||
"url": "https://openstack-host:8776/v1/ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"interface": "admin",
|
||||
"region": "ndc_ch2_a",
|
||||
"id": "df06789d23684643a0b59bfa7ee26064"
|
||||
}
|
||||
],
|
||||
"type": "volume",
|
||||
"id": "e90968e859474737a4ad7aadffb2f578"
|
||||
}
|
||||
],
|
||||
"timestamp": "2016-10-03 18:25:17.121727",
|
||||
"_context_user": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_unique_id": "b7c7128bd595423daeed9d6ad36c2b2a",
|
||||
"_context_instance_lock_checked": false,
|
||||
"_context_user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload": {
|
||||
"state_description": "deleting",
|
||||
"availability_zone": null,
|
||||
"terminated_at": "",
|
||||
"ephemeral_gb": 0,
|
||||
"instance_type_id": 2,
|
||||
"deleted_at": "",
|
||||
"reservation_id": "r-osn0qo9l",
|
||||
"instance_id": "563d0899-d1cc-4154-bb58-f932ad0b255c",
|
||||
"user_id": "bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"hostname": "test",
|
||||
"state": "active",
|
||||
"launched_at": "2016-10-03T18:23:11.000000",
|
||||
"metadata": {
|
||||
},
|
||||
"node": "openstack-host2",
|
||||
"ramdisk_id": "",
|
||||
"access_ip_v6": null,
|
||||
"disk_gb": 1,
|
||||
"access_ip_v4": null,
|
||||
"kernel_id": "",
|
||||
"host": "oscomp-ch2-a07",
|
||||
"display_name": "test",
|
||||
"image_ref_url": "http://172.28.195.38:9292/images/caf23aab-f6ec-4691-8f50-b0c70a33eca3",
|
||||
"root_gb": 1,
|
||||
"tenant_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"created_at": "2016-10-03 18:23:03+00:00",
|
||||
"memory_mb": 512,
|
||||
"instance_type": "m1.tiny",
|
||||
"vcpus": 1,
|
||||
"image_meta": {
|
||||
"description": "cirros-0.3.4",
|
||||
"container_format": "bare",
|
||||
"min_ram": "0",
|
||||
"disk_format": "qcow2",
|
||||
"min_disk": "1",
|
||||
"base_image_ref": "caf23aab-f6ec-4691-8f50-b0c70a33eca3"
|
||||
},
|
||||
"architecture": null,
|
||||
"os_type": null,
|
||||
"instance_flavor_id": "1"
|
||||
},
|
||||
"_context_project_name": "VOIP",
|
||||
"_context_read_deleted": "no",
|
||||
"_context_auth_token": "7b82d6dec2eb457d85e08940a7f7b33e",
|
||||
"_context_tenant": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority": "INFO",
|
||||
"_context_is_admin": true,
|
||||
"_context_project_id": "ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp": "2016-10-03T18:25:16.888382",
|
||||
"_context_user_name": "admin",
|
||||
"publisher_id": "compute.oscomp-ch2-a07",
|
||||
"message_id": "41963fa2-be81-42ee-b31e-e2f727f109b7",
|
||||
"_context_remote_address": "127.0.0.1"
|
||||
}
|
||||
@ -0,0 +1,36 @@
|
||||
{
|
||||
"_context_roles":[
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id":"req-b22260da-ecd2-4eb6-a7bb-e07a216450de",
|
||||
"event_type":"port.create.start",
|
||||
"timestamp":"2016-10-03 18:23:04.894439",
|
||||
"_context_tenant_id":"0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"_unique_id":"d133f63bb45740f587b780d4dac7e2ee",
|
||||
"_context_tenant_name":"services",
|
||||
"_context_user":"31055c32b50442e5a4eb4c0f0cb3430b",
|
||||
"_context_user_id":"31055c32b50442e5a4eb4c0f0cb3430b",
|
||||
"payload":{
|
||||
"port":{
|
||||
"binding:host_id":"oscomp-ch2-a07",
|
||||
"admin_state_up":true,
|
||||
"network_id":"af33487a-4e96-4499-bfcd-4f741617a763",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"device_owner":"compute:None",
|
||||
"security_groups":[
|
||||
"0783a151-768c-49d3-a31d-178f70fabd51"
|
||||
],
|
||||
"device_id":"563d0899-d1cc-4154-bb58-f932ad0b255c"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"services",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":true,
|
||||
"_context_project_id":"0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"_context_timestamp":"2016-10-03 18:23:04.892974",
|
||||
"_context_user_name":"neutron",
|
||||
"publisher_id":"network.osctrl-ch2-a02",
|
||||
"message_id":"b256daa5-3eed-4fec-8c54-d93fa79ab753"
|
||||
}
|
||||
@ -0,0 +1,38 @@
|
||||
{
|
||||
"_context_roles":[
|
||||
"admin"
|
||||
],
|
||||
"_context_request_id":"req-662e5998-9dcd-4b76-a726-5d815a1ab9ef",
|
||||
"event_type":"port.create.start",
|
||||
"timestamp":"2016-10-03 17:25:04.189288",
|
||||
"_context_tenant_id":"0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"_unique_id":"edd46be30b524fe9a3af78cec809db0e",
|
||||
"_context_tenant_name":"services",
|
||||
"_context_user":"31055c32b50442e5a4eb4c0f0cb3430b",
|
||||
"_context_user_id":"31055c32b50442e5a4eb4c0f0cb3430b",
|
||||
"payload":{
|
||||
"port":{
|
||||
"binding:host_id":"oscomp-ch2-a07",
|
||||
"admin_state_up":true,
|
||||
"network_id":"af33487a-4e96-4499-bfcd-4f741617a763",
|
||||
"tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"device_owner":"compute:None",
|
||||
"security_groups":[
|
||||
"0783a151-768c-49d3-a31d-178f70fabd51",
|
||||
"46d46540-98ac-4c93-ae62-68dddab2282e"
|
||||
],
|
||||
"device_id":"bb64fe08-0eae-4c83-8bc2-457b6cb7e9a3"
|
||||
}
|
||||
},
|
||||
"_context_project_name":"services",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":true,
|
||||
"_context_project_id":"0b65cf220eab4a3cbd68681d188d7dc7",
|
||||
"_context_timestamp":"2016-10-03 17:25:04.187847",
|
||||
"_context_user_name":"neutron",
|
||||
"publisher_id":"network.osctrl-ch2-a02",
|
||||
"message_id":"56253f06-7350-47c0-8284-d74701d11698"
|
||||
}
|
||||
|
||||
@ -0,0 +1,26 @@
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-61fa3b99-0582-43a3-a5da-71465e5b56b6",
|
||||
"event_type":"port.delete.end",
|
||||
"timestamp":"2016-10-03 18:25:18.645306",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"7296f75a9e0f4993bde5cc2b6d5cbe7d",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"port_id":"d781dc1e-f940-4de8-ad2f-b1286b75efe0"
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 18:25:18.562793",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a02",
|
||||
"message_id":"3375a032-8bd2-45b1-b3d8-29bdc5aacbf5"
|
||||
}
|
||||
@ -0,0 +1,28 @@
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-d9daa316-5075-4702-961a-6a808db40c1e",
|
||||
"event_type":"port.delete.end",
|
||||
"timestamp":"2016-10-03 17:28:04.004860",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"e7a124d2056c476696a5efada8ddfbf2",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"port_id":"57545a0d-ab4e-4a18-bba2-d7ee8bc9c3c1"
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 17:28:03.916009",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a02",
|
||||
"message_id":"9573b2e3-9f33-499d-ad31-43840f5c1c58"
|
||||
}
|
||||
|
||||
|
||||
@ -0,0 +1,26 @@
|
||||
{
|
||||
"_context_roles":[
|
||||
"Member"
|
||||
],
|
||||
"_context_request_id":"req-f96ea9a5-435e-4177-8e51-ebb60d0fae2a",
|
||||
"event_type":"security_group_rule.delete.end",
|
||||
"timestamp":"2016-10-03 18:10:59.112712",
|
||||
"_context_tenant_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_unique_id":"eafc9362327442b49d8c03b0e88d0216",
|
||||
"_context_tenant_name":"VOIP",
|
||||
"_context_user":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"_context_user_id":"bca89c1b248e4aef9c69ece9e744cc54",
|
||||
"payload":{
|
||||
"security_group_rule_id":"bf288dfc-f9cb-446b-bacc-a8de016c9b11"
|
||||
},
|
||||
"_context_project_name":"VOIP",
|
||||
"_context_read_deleted":"no",
|
||||
"_context_tenant":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"priority":"INFO",
|
||||
"_context_is_admin":false,
|
||||
"_context_project_id":"ada3b9b0dbac429f9361e803b54f5f32",
|
||||
"_context_timestamp":"2016-10-03 18:10:59.079179",
|
||||
"_context_user_name":"admin",
|
||||
"publisher_id":"network.osctrl-ch2-a03",
|
||||
"message_id":"e75fb2ee-85bf-44ba-a083-2445eca2ae10"
|
||||
}
|
||||
6
fixtures/viper/test.yml
Normal file
6
fixtures/viper/test.yml
Normal file
@ -0,0 +1,6 @@
|
||||
required_string: "required_string value"
|
||||
test_alias: "test_alias value"
|
||||
nested:
|
||||
one: "nested.one value"
|
||||
|
||||
|
||||
20
go.mod
Normal file
20
go.mod
Normal file
@ -0,0 +1,20 @@
|
||||
module "git.openstack.org/openstack/osel"
|
||||
|
||||
require (
|
||||
"github.com/fsnotify/fsnotify" v1.4.7
|
||||
"github.com/google/go-querystring" v0.0.0-20170111101155-53e6ce116135
|
||||
"github.com/hashicorp/hcl" v0.0.0-20171017181929-23c074d0eceb
|
||||
"github.com/magiconair/properties" v1.7.6
|
||||
"github.com/mitchellh/mapstructure" v0.0.0-20180220230111-00c29f56e238
|
||||
"github.com/nate-johnston/viper" v1.0.1
|
||||
"github.com/pelletier/go-toml" v1.1.0
|
||||
"github.com/racker/perigee" v0.1.0
|
||||
"github.com/rackspace/gophercloud" v1.0.0
|
||||
"github.com/spf13/afero" v1.0.2
|
||||
"github.com/spf13/cast" v1.2.0
|
||||
"github.com/spf13/pflag" v1.0.0
|
||||
"github.com/streadway/amqp" v0.0.0-20180131094250-fc7fda2371f5
|
||||
"golang.org/x/sys" v0.0.0-20180302081741-dd2ff4accc09
|
||||
"golang.org/x/text" v0.0.0-20171214130843-f21a4dfb5e38
|
||||
"gopkg.in/yaml.v2" v1.1.1-gopkgin-v2.1.1
|
||||
)
|
||||
129
main.go
Normal file
129
main.go
Normal file
@ -0,0 +1,129 @@
|
||||
package main // import "git.openstack.org/openstack/osel"
|
||||
|
||||
import (
|
||||
"log"
|
||||
"net/url"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/fsnotify/fsnotify"
|
||||
"github.com/nate-johnston/viper"
|
||||
)
|
||||
|
||||
// OselVersion is exposed in the logged JSON in the "source_type" field. This
|
||||
// will allow us to track the version of the logging specification.
|
||||
// 1.0: Initial revision
|
||||
// 1.1: Added qualys_scan_id and qualys_scan_error
|
||||
const OselVersion = "osel1.1"
|
||||
|
||||
// Debug is a global variable to toggle debug logging
|
||||
var Debug bool
|
||||
|
||||
// RabbitMQ URI
|
||||
var rabbitURI string
|
||||
|
||||
func main() {
|
||||
// Declare the configuration
|
||||
viperConfigs := []ViperConfig{
|
||||
ViperConfig{Key: "batch_interval", Description: "Interval of time in minutes for message batching"},
|
||||
ViperConfig{Key: "debug", Description: "Output additional messages for debugging"},
|
||||
ViperConfig{Key: "rabbit_uri", Description: "AMQP connection uri. See: https://www.rabbitmq.com/uri-spec.html"},
|
||||
ViperConfig{Key: "openstack.identity_endpoint", Description: "Openstack Keystone Endpoint"},
|
||||
ViperConfig{Key: "openstack.user", Description: "Openstack user that has at least read only access to all tenants/ports/security groups in the region."},
|
||||
ViperConfig{Key: "openstack.password", Description: "Password for the Openstack user"},
|
||||
ViperConfig{Key: "openstack.region", Description: "The name of the region running this process"},
|
||||
ViperConfig{Key: "qualys.drop6", Description: "Should IPv6 addresses be incorporated in Qualys scans? true or false."},
|
||||
ViperConfig{Key: "qualys.username", Description: "Username for credentials for the Qualys external scanning service"},
|
||||
ViperConfig{Key: "qualys.password", Description: "Password for credentials for the Qualys external scanning service"},
|
||||
ViperConfig{Key: "qualys.url", Description: "URL for thw Qualys service"},
|
||||
ViperConfig{Key: "qualys.proxy_url", Description: "URL for an HTTP proxy that will permit access to the Qualys service"},
|
||||
ViperConfig{Key: "syslog_server", Description: "FQDN of the server for events to log to over the network"},
|
||||
ViperConfig{Key: "syslog_port", Description: "Port for communication to syslog, defaults to 514"},
|
||||
ViperConfig{Key: "syslog_protocol", Description: "tcp or udp, defaults to tcp"},
|
||||
ViperConfig{Key: "retry_syslog", Description: "Should the process keep trying if it cannot reach syslog? true or false."},
|
||||
}
|
||||
configPath := os.Getenv("EL_CONFIG") //The config path comes from ENV.
|
||||
if configPath == "" {
|
||||
log.Fatalln("Fatal Error: The Config file was not set to EL_CONFIG.")
|
||||
}
|
||||
if err := InitViper(configPath, viperConfigs); err != nil {
|
||||
log.Fatalf("Fatal Error: (%s) while reading config file %s", err, configPath)
|
||||
}
|
||||
|
||||
// Set defaults
|
||||
viper.SetDefault("batch_interval", 60)
|
||||
viper.SetDefault("debug", true)
|
||||
viper.SetDefault("qualys.drop6", true)
|
||||
viper.SetDefault("qualys.url", "https://qualysapi.qualys.com/api/2.0/fo/scan/")
|
||||
viper.SetDefault("syslog_port", "514")
|
||||
viper.SetDefault("syslog_protocol", "tcp")
|
||||
|
||||
// Watch for config changes
|
||||
viper.WatchConfig()
|
||||
viper.OnConfigChange(func(fsnotify.Event) {
|
||||
if err := ValidateConfig(viperConfigs); err != nil {
|
||||
log.Printf("Fatal Error: %s while refreshing config file %s\n", err, configPath)
|
||||
}
|
||||
})
|
||||
|
||||
batchInterval := viper.GetInt("batch_interval")
|
||||
Debug = viper.GetBool("debug")
|
||||
|
||||
// Initialize AMQP
|
||||
rabbitURI = viper.GetString("rabbit_uri")
|
||||
amqpBus := new(AmqpActions)
|
||||
amqpBus.Options = AmqpOptions{
|
||||
RabbitURI: rabbitURI,
|
||||
}
|
||||
amqpIncoming, amqpErrorNotify, err := amqpBus.Connect()
|
||||
if err != nil {
|
||||
log.Fatalln(err)
|
||||
}
|
||||
|
||||
// Initialize Qualys
|
||||
qualysURL, err := url.Parse(viper.GetString("qualys.url"))
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
qualysProxyURL, err := url.Parse(viper.GetString("qualys.proxy_url"))
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
qualys := new(QualysActions)
|
||||
qualys.Options = QualysOptions{
|
||||
DropIPv6: viper.GetBool("qualys.drop6"),
|
||||
Password: viper.GetString("qualys.password"),
|
||||
ProxyURL: qualysProxyURL,
|
||||
QualysURL: qualysURL,
|
||||
ScanOptionName: viper.GetString("qualys.option"),
|
||||
MinRemaining: viper.GetInt("qualys.min_remaining"),
|
||||
UserName: viper.GetString("qualys.username"),
|
||||
}
|
||||
|
||||
// Initialize OpenStack
|
||||
openstack := new(OpenStackActions)
|
||||
openstack.Options = OpenStackOptions{
|
||||
KeystoneURI: viper.GetString("openstack.identity_endpoint"),
|
||||
Password: viper.GetString("openstack.password"),
|
||||
RegionName: viper.GetString("openstack.region"),
|
||||
UserName: viper.GetString("openstack.user"),
|
||||
}
|
||||
|
||||
// Initialize Syslog
|
||||
logger := new(SyslogActions)
|
||||
logger.Options = SyslogOptions{
|
||||
Host: viper.GetString("syslog_server"),
|
||||
Port: viper.GetString("syslog_port"),
|
||||
Protocol: viper.GetString("syslog_protocol"),
|
||||
Retry: viper.GetBool("retry_syslog"),
|
||||
}
|
||||
err = logger.Connect()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// run main loop
|
||||
batchDuration := time.Duration(batchInterval) * time.Minute
|
||||
mainLoop(batchDuration, amqpIncoming, amqpErrorNotify, openstack, logger, qualys)
|
||||
defer amqpBus.Close()
|
||||
}
|
||||
126
openstack.go
Normal file
126
openstack.go
Normal file
@ -0,0 +1,126 @@
|
||||
package main
|
||||
|
||||
/*
|
||||
|
||||
openstack - This file includes all of the logic necessary to interact with
|
||||
OpenStack. This is extrapolated out so that an OpenStackActioner
|
||||
interface can be passed to functions. Doing this allows testing by mock
|
||||
classes to be created that can be passed to functions.
|
||||
|
||||
Since this is a wrapper around the gophercloud libraries, this does not need
|
||||
testing.
|
||||
|
||||
*/
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"github.com/rackspace/gophercloud"
|
||||
"github.com/rackspace/gophercloud/openstack"
|
||||
"github.com/rackspace/gophercloud/openstack/networking/v2/ports"
|
||||
"github.com/rackspace/gophercloud/pagination"
|
||||
)
|
||||
|
||||
// OpenStackActioner is an interface for an OpenStackActions class.
|
||||
// Having this as an interface allows us to pass in a dummy class for testing
|
||||
// that just returns mocked data.
|
||||
type OpenStackActioner interface {
|
||||
GetPortList() []OpenStackIPMap
|
||||
Connect(string, string) error
|
||||
}
|
||||
|
||||
// OpenStackActions is a class that handles all interactions directly with
|
||||
// OpenStack. See the comment on OpenStackActioner for rationale.
|
||||
type OpenStackActions struct {
|
||||
gopherCloudClient *gophercloud.ProviderClient
|
||||
neutronClient *gophercloud.ServiceClient
|
||||
Options OpenStackOptions
|
||||
}
|
||||
|
||||
// OpenStackOptions is a class to convey all of the configurable options for the
|
||||
// OpenStackActions class.
|
||||
type OpenStackOptions struct {
|
||||
KeystoneURI string
|
||||
Password string
|
||||
RegionName string
|
||||
TenantID string
|
||||
UserName string
|
||||
}
|
||||
|
||||
// OpenStackIPMap is a struct that is used to capture the mapping of IP address
|
||||
// to security group. It is what is returned, in array form, from port list.
|
||||
type OpenStackIPMap struct {
|
||||
ipAddress string
|
||||
securityGroup string
|
||||
}
|
||||
|
||||
// GetPortList is a method that uses GopherCloud to query OpenStack for a
|
||||
// list of ports, with their associated security group. It returns an array of
|
||||
// OpenStackIPMap.
|
||||
func (s OpenStackActions) GetPortList() []OpenStackIPMap {
|
||||
// Make port list request to neutron
|
||||
var ips []OpenStackIPMap
|
||||
portListOpts := ports.ListOpts{
|
||||
TenantID: s.Options.TenantID,
|
||||
}
|
||||
if s.neutronClient == nil {
|
||||
log.Println("Error: neutronClient is nil")
|
||||
}
|
||||
pager := ports.List(s.neutronClient, portListOpts)
|
||||
|
||||
// Define an anonymous function to be executed on each page's iteration
|
||||
pager.EachPage(func(page pagination.Page) (bool, error) {
|
||||
portList, err := ports.ExtractPorts(page)
|
||||
if err != nil {
|
||||
// ignore ?
|
||||
}
|
||||
|
||||
for _, p := range portList {
|
||||
// "p" will be a ports.Port
|
||||
for _, fixedIP := range p.FixedIPs {
|
||||
for _, securityGroup := range p.SecurityGroups {
|
||||
ips = append(ips, OpenStackIPMap{
|
||||
ipAddress: fixedIP.IPAddress,
|
||||
securityGroup: securityGroup,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
return true, err
|
||||
})
|
||||
return ips
|
||||
}
|
||||
|
||||
// Connect is the method that establishes a connection to the OpenStack
|
||||
// service.
|
||||
func (s *OpenStackActions) Connect(tenantID string, username string) error {
|
||||
var err error
|
||||
keystoneOpts := gophercloud.AuthOptions{
|
||||
IdentityEndpoint: s.Options.KeystoneURI,
|
||||
TenantID: tenantID,
|
||||
Username: username,
|
||||
Password: s.Options.Password,
|
||||
AllowReauth: true,
|
||||
}
|
||||
|
||||
log.Println(fmt.Sprintf("Connecting to keystone %q in region %q for tenant %q with user %q", s.Options.KeystoneURI,
|
||||
s.Options.RegionName, tenantID, username))
|
||||
s.gopherCloudClient, err = openstack.AuthenticatedClient(keystoneOpts)
|
||||
if err != nil {
|
||||
return fmt.Errorf("unable to connect to %s using user %s for tenant %s: %s",
|
||||
s.Options.KeystoneURI, s.Options.UserName, s.Options.TenantID, err)
|
||||
}
|
||||
log.Println("Connected to gophercloud ", s.Options.KeystoneURI)
|
||||
|
||||
neutronOpts := gophercloud.EndpointOpts{
|
||||
Name: "neutron",
|
||||
Region: s.Options.RegionName,
|
||||
}
|
||||
s.neutronClient, err = openstack.NewNetworkV2(s.gopherCloudClient, neutronOpts)
|
||||
if err != nil {
|
||||
return fmt.Errorf("unable to connect to neutron using user %s in region %s: %s",
|
||||
s.Options.UserName, s.Options.RegionName, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
31
openstack_mock_test.go
Normal file
31
openstack_mock_test.go
Normal file
@ -0,0 +1,31 @@
|
||||
package main
|
||||
|
||||
type OpenStackTestActions struct {
|
||||
regionName string
|
||||
tenantID string
|
||||
}
|
||||
|
||||
func (s OpenStackTestActions) Connect(tenantID string, username string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s OpenStackTestActions) GetPortList() []OpenStackIPMap {
|
||||
return []OpenStackIPMap{
|
||||
{
|
||||
ipAddress: "10.0.0.1",
|
||||
securityGroup: "46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
},
|
||||
{
|
||||
ipAddress: "10.0.0.2",
|
||||
securityGroup: "groupTwo",
|
||||
},
|
||||
{
|
||||
ipAddress: "10.0.0.3",
|
||||
securityGroup: "46d46540-98ac-4c93-ae62-68dddab2282e",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func connectFakeOpenstack() *OpenStackTestActions {
|
||||
return new(OpenStackTestActions)
|
||||
}
|
||||
133
processing.go
Normal file
133
processing.go
Normal file
@ -0,0 +1,133 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"net"
|
||||
"time"
|
||||
|
||||
"github.com/streadway/amqp"
|
||||
)
|
||||
|
||||
func processWaitingEvent(delivery amqp.Delivery, openstackActions OpenStackActioner) (Event, error) {
|
||||
// executes when an event is waiting
|
||||
event, err := ParseEvent(delivery.Body)
|
||||
if err != nil {
|
||||
return Event{}, fmt.Errorf("Failed to parse event due to error: %s", err)
|
||||
}
|
||||
if event.Processor == nil {
|
||||
if !Debug {
|
||||
return Event{}, nil
|
||||
}
|
||||
return Event{}, fmt.Errorf("Ignoring event type %s", event.EventData.EventType)
|
||||
}
|
||||
if Debug {
|
||||
log.Printf("Processing event type %s\n", event.EventData.EventType)
|
||||
}
|
||||
err = event.Processor.FillExtraData(&event, openstackActions)
|
||||
if err != nil {
|
||||
return Event{}, fmt.Errorf("Error fetching extra data: %s", err)
|
||||
}
|
||||
return event, nil
|
||||
}
|
||||
|
||||
func logEvents(events []Event, logger SyslogActioner, qualys QualysActioner) {
|
||||
var ipAddresses []string
|
||||
var qualysIPAddresses []string
|
||||
|
||||
if Debug {
|
||||
log.Println("Timer Expired")
|
||||
}
|
||||
|
||||
// De-dupe IP addresses and get them into a single struct
|
||||
dedupIPAddresses := make(map[string]struct{})
|
||||
for _, event := range events {
|
||||
for _, IPs := range event.IPs {
|
||||
for _, IP := range IPs {
|
||||
if _, ok := dedupIPAddresses[IP]; !ok {
|
||||
ipAddresses = append(ipAddresses, IP)
|
||||
}
|
||||
dedupIPAddresses[IP] = struct{}{}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Disregard the scan if no targets have been found
|
||||
if len(ipAddresses) == 0 {
|
||||
if Debug {
|
||||
log.Println("Nothing to scan, skipping...")
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Remove IPv6 addresses
|
||||
if qualys.DropIPv6() {
|
||||
for ipAddressIndex := range ipAddresses {
|
||||
testIPAddress := ipAddresses[ipAddressIndex]
|
||||
if net.ParseIP(testIPAddress).To4() != nil {
|
||||
qualysIPAddresses = append(qualysIPAddresses, testIPAddress)
|
||||
} else {
|
||||
log.Println("Disregarded IPv6 address", testIPAddress)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Execute Qualys scan
|
||||
|
||||
log.Println("Qualys Scan Starting")
|
||||
scanID, scanError := qualys.InitiateScan(qualysIPAddresses)
|
||||
log.Printf("Qualys Scan Complete: scan ID='%s'; scan_error='%v'", scanID, scanError)
|
||||
|
||||
// Iterate through entries and format the logs
|
||||
log.Printf("Processing %d events\n", len(events))
|
||||
for _, event := range events {
|
||||
event.QualysScanID = scanID
|
||||
if scanError != nil {
|
||||
event.QualysScanError = scanError.Error()
|
||||
}
|
||||
event.LogLines, _ = event.Processor.FormatLogs(&event, qualysIPAddresses)
|
||||
|
||||
// Output the logs
|
||||
log.Printf("Processing %d loglines\n", len(event.LogLines))
|
||||
for lineToLog := range event.LogLines {
|
||||
logger.Info(event.LogLines[lineToLog])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func mainLoop(batchInterval time.Duration, deliveries <-chan amqp.Delivery, amqpNotifyError chan *amqp.Error, openstackActions OpenStackActioner, logger SyslogActioner, qualys QualysActioner) {
|
||||
var events []Event
|
||||
ticker := time.NewTicker(batchInterval)
|
||||
amqpReconnectTimer := time.NewTimer(1)
|
||||
for {
|
||||
select {
|
||||
case e := <-deliveries:
|
||||
event, err := processWaitingEvent(e, openstackActions)
|
||||
if err != nil {
|
||||
log.Printf("Event skipped: %s\n", err)
|
||||
continue
|
||||
}
|
||||
events = append(events, event)
|
||||
case <-ticker.C:
|
||||
logEvents(events, logger, qualys)
|
||||
events = nil
|
||||
case err := <-amqpNotifyError:
|
||||
// Reinitialize AMQP on connection error
|
||||
log.Printf("AMQP connection error: %s\n", err)
|
||||
amqpReconnectTimer = time.NewTimer(time.Second * 30)
|
||||
case <-amqpReconnectTimer.C:
|
||||
var err error
|
||||
amqpBus := new(AmqpActions)
|
||||
amqpBus.Options = AmqpOptions{
|
||||
RabbitURI: rabbitURI,
|
||||
}
|
||||
deliveries, amqpNotifyError, err = amqpBus.Connect()
|
||||
if err != nil {
|
||||
log.Printf("AMQP retry connection error: %s\n", err)
|
||||
amqpReconnectTimer = time.NewTimer(time.Second * 30)
|
||||
} else {
|
||||
log.Printf("AMQP reconnected\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
48
processing_test.go
Normal file
48
processing_test.go
Normal file
@ -0,0 +1,48 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/streadway/amqp"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestProcessWaitingEvent(t *testing.T) {
|
||||
var delivery amqp.Delivery
|
||||
openstackActions := connectFakeOpenstack()
|
||||
|
||||
delivery.Body = []byte(securityGroupRuleCreateWithIcmpAndCider)
|
||||
event, err := processWaitingEvent(delivery, openstackActions)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_ = event
|
||||
}
|
||||
|
||||
func TestLogEvents(t *testing.T) {
|
||||
hostName, _ := os.Hostname()
|
||||
IPList := []string{"10.0.0.1", "10.0.0.3"}
|
||||
logLines := []string{fmt.Sprintf(`{"security_group_rule":{"remote_group_id":null,"direction":"ingress","protocol":"icmp","remote_ip_prefix":"192.168.1.0/24","port_range_max":null,"rule_direction":"","security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e","tenant_id":"ada3b9b0dbac429f9361e803b54f5f32","port_range_min":null,"ethertype":"IPv4","id":"66d7ac79-3551-4436-83c7-103b50760cfb"},"affected_ip_address":"10.0.0.1","change_type":"sg_rule_add","source_type":"osel","source_message_bus":"%s"}`, hostName), fmt.Sprintf(`{"security_group_rule":{"remote_group_id":null,"direction":"ingress","protocol":"icmp","remote_ip_prefix":"192.168.1.0/24","port_range_max":null,"rule_direction":"","security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e","tenant_id":"ada3b9b0dbac429f9361e803b54f5f32","port_range_min":null,"ethertype":"IPv4","id":"66d7ac79-3551-4436-83c7-103b50760cfb"},"affected_ip_address":"10.0.0.3","change_type":"sg_rule_add","source_type":"osel","source_message_bus":"%s"}`, hostName)}
|
||||
logger := connectFakeSyslog()
|
||||
qualys := connectFakeQualys()
|
||||
IPs := make(map[string][]string)
|
||||
|
||||
IPs["46d46540-98ac-4c93-ae62-68dddab2282e"] = IPList
|
||||
fakeEvent := Event{
|
||||
RawData: []byte(securityGroupRuleCreateWithIcmpAndCider),
|
||||
LogLines: logLines,
|
||||
Processor: EventSecurityGroupRuleChange{ChangeType: "sg_rule_add"},
|
||||
IPs: IPs,
|
||||
}
|
||||
events := []Event{fakeEvent}
|
||||
|
||||
logEvents(events, logger, qualys)
|
||||
savedLogs := logger.GetLogs()
|
||||
assert.Equal(t, 2, len(savedLogs))
|
||||
|
||||
logLine1 := fmt.Sprintf(`{"affected_ip_address":"10.0.0.1","change_type":"sg_rule_add","qualys_scan_id":"","qualys_scan_error":"Not scanned by Qualys","security_group_rule":{"remote_group_id":null,"direction":"ingress","protocol":"icmp","remote_ip_prefix":"192.168.1.0/24","port_range_max":null,"rule_direction":"","security_group_id":"46d46540-98ac-4c93-ae62-68dddab2282e","tenant_id":"ada3b9b0dbac429f9361e803b54f5f32","port_range_min":null,"ethertype":"IPv4","id":"66d7ac79-3551-4436-83c7-103b50760cfb"},"source_type":"osel1.1","source_message_bus":"%s"}`, hostName)
|
||||
assert.Equal(t, logLine1, savedLogs[0])
|
||||
}
|
||||
104
qualys.go
Normal file
104
qualys.go
Normal file
@ -0,0 +1,104 @@
|
||||
package main
|
||||
|
||||
/*
|
||||
|
||||
qualys - This file includes all of the logic necessary to interact with the
|
||||
go-qualys library. This is extrapolated out so that a QualysInterface
|
||||
interface can be passed to functions. Doing this allows testing by mock
|
||||
classes to be created that can be passed to functions.
|
||||
|
||||
Since this is a wrapper around the go-qualys library, this does not need
|
||||
testing.
|
||||
|
||||
*/
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
||||
"git.openstack.org/openstack/osel/qualys"
|
||||
)
|
||||
|
||||
// QualysActioner is an interface for an QualysActions class. Having
|
||||
// this as an interface allows us to pass in a dummy class for testing that
|
||||
// just returns mocked data.
|
||||
type QualysActioner interface {
|
||||
InitiateScan([]string) (string, error)
|
||||
DropIPv6() bool
|
||||
}
|
||||
|
||||
// QualysActions is a class that handles all interactions directly with Qualys.
|
||||
// See the comment on QualysActioner for rationale.
|
||||
type QualysActions struct {
|
||||
Options QualysOptions
|
||||
}
|
||||
|
||||
// QualysOptions is a class to convey all of the configurable options for the
|
||||
// QualysActions class.
|
||||
type QualysOptions struct {
|
||||
DropIPv6 bool
|
||||
MinRemaining int
|
||||
ProxyURL *url.URL
|
||||
Password string
|
||||
QualysURL *url.URL
|
||||
ScanOptionName string
|
||||
UserName string
|
||||
}
|
||||
|
||||
// InitiateScan is the main method for the QualysActioner class, it
|
||||
// makes a call to the Qualys API to start a scan and harvests a scan ID, and
|
||||
// an optional error string if there is a problem contacting Qualys.
|
||||
func (s *QualysActions) InitiateScan(targetIPAddresses []string) (string, error) {
|
||||
var err error
|
||||
|
||||
// create client with proxy so the qualys service can be accessed
|
||||
qualysCreds := qualys.Credentials{
|
||||
Username: s.Options.UserName,
|
||||
Password: s.Options.Password,
|
||||
}
|
||||
c, err := qualys.NewClient(&http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(s.Options.ProxyURL)}}, &qualysCreds)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
c.BaseURL = s.Options.QualysURL
|
||||
|
||||
// create the options
|
||||
opts := qualys.LaunchScanOptions{
|
||||
ScanTitle: "osel",
|
||||
ScannerName: "External",
|
||||
OptionTitle: s.Options.ScanOptionName,
|
||||
IP: targetIPAddresses,
|
||||
}
|
||||
|
||||
// launch the request
|
||||
launchScanResponse, err := c.LaunchScan(&opts)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// process the request response
|
||||
scanID := launchScanResponse.ScanReference
|
||||
remainingQualysRequests := launchScanResponse.RateLimitations.Remaining
|
||||
allowedQualysRequests := launchScanResponse.RateLimitations.Limit
|
||||
if Debug {
|
||||
log.Printf("Qualys Rate Limit: %d of %d total requests remaining, concurrency of %d out of %d, %d seconds remaining in limit window and %d seconds until a request can be made again\n",
|
||||
remainingQualysRequests, allowedQualysRequests, launchScanResponse.RateLimitations.CurrentConcurrency,
|
||||
launchScanResponse.RateLimitations.ConcurrencyLimit, launchScanResponse.RateLimitations.LimitWindow, launchScanResponse.RateLimitations.WaitingPeriod)
|
||||
}
|
||||
if launchScanResponse.Text != "" {
|
||||
err = errors.New(launchScanResponse.Text)
|
||||
}
|
||||
if remainingQualysRequests <= s.Options.MinRemaining {
|
||||
err = fmt.Errorf("halting Qualys processing! Only %d Qualys calls remain out of a total of %d. Waiting for %d seconds before resuming", remainingQualysRequests,
|
||||
allowedQualysRequests, launchScanResponse.RateLimitations.LimitWindow)
|
||||
}
|
||||
return scanID, err
|
||||
}
|
||||
|
||||
// DropIPv6 is an accessor method to allow other code to make decisions based on whether this flag is enabled.
|
||||
func (s *QualysActions) DropIPv6() bool {
|
||||
return s.Options.DropIPv6
|
||||
}
|
||||
170
qualys/assets.go
Normal file
170
qualys/assets.go
Normal file
@ -0,0 +1,170 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"net"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
assetsBasePath = "asset"
|
||||
groupsBasePath = "group"
|
||||
)
|
||||
|
||||
// AssetsService is an interface for interfacing with the Assets
|
||||
// endpoints of the Qualys API
|
||||
type AssetsService interface {
|
||||
ListAssetGroups(*ListAssetGroupOptions) ([]AssetGroup, *Response, error)
|
||||
GetAssetGroupByID(groupID string) (*AssetGroup, *Response, error)
|
||||
AddIPsToGroup(*AddIPsToGroupOptions) (*Response, error)
|
||||
}
|
||||
|
||||
// AssetsServiceOp handles communication with the asset related methods of the
|
||||
// Qualys API.
|
||||
type AssetsServiceOp struct {
|
||||
client *Client
|
||||
}
|
||||
|
||||
var _ AssetsService = &AssetsServiceOp{}
|
||||
|
||||
// AssetGroup represents a Qualys HostGroup
|
||||
type AssetGroup struct {
|
||||
ID string `xml:"ID"`
|
||||
Title string `xml:"TITLE"`
|
||||
OwnerUserID string `xml:"OWNER_USER_ID"`
|
||||
OwnerUnitID string `xml:"OWNER_UNIT_ID"`
|
||||
IPs AssetGroupIPs `xml:"IP_SET"`
|
||||
}
|
||||
|
||||
// AssetGroupIPs represents one or more IP addresses assigned to the AssetGroup
|
||||
type AssetGroupIPs struct {
|
||||
IPs []string `xml:"IP"`
|
||||
IPRanges []string `xml:"IP_RANGE"`
|
||||
}
|
||||
|
||||
type ipRange struct {
|
||||
Min net.IP
|
||||
Max net.IP
|
||||
}
|
||||
|
||||
func newIPRange(rangeString string) *ipRange {
|
||||
var r = strings.Split(rangeString, "-")
|
||||
return &ipRange{Min: net.ParseIP(r[0]), Max: net.ParseIP(r[1])}
|
||||
}
|
||||
|
||||
func (ip *ipRange) Contains(ipString string) bool {
|
||||
var myIP = net.ParseIP(ipString)
|
||||
if bytes.Compare(myIP, ip.Min) >= 0 && bytes.Compare(myIP, ip.Max) <= 0 {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ContainsIP returns true when the AssetGroupIPs matches the provided IP
|
||||
func (agp *AssetGroupIPs) ContainsIP(ip string) bool {
|
||||
if containsString(agp.IPs, ip) {
|
||||
return true
|
||||
}
|
||||
if agp.IPRanges != nil && len(agp.IPRanges) > 0 {
|
||||
for _, ipRange := range agp.IPRanges {
|
||||
if newIPRange(ipRange).Contains(ip) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ContainsIP returns true when the AssetGroup has any assets matching the provided IP
|
||||
func (ag *AssetGroup) ContainsIP(ip string) bool {
|
||||
return ag.IPs.ContainsIP(ip)
|
||||
}
|
||||
|
||||
type assetGroupsRoot struct {
|
||||
AssetGroups []AssetGroup `xml:"RESPONSE>ASSET_GROUP_LIST>ASSET_GROUP"`
|
||||
}
|
||||
|
||||
// AssetGroupUpdateRequest represents a request to update a group
|
||||
type AssetGroupUpdateRequest struct {
|
||||
}
|
||||
|
||||
// ListAssetGroupOptions represents the AssetGroup retrieval options
|
||||
type ListAssetGroupOptions struct {
|
||||
Ids []string `url:"ids,comma,omitempty"`
|
||||
Action string `url:"action,omitempty"`
|
||||
}
|
||||
|
||||
// AddIPsToGroupOptions represents the update request for an AssetGroup
|
||||
type AddIPsToGroupOptions struct {
|
||||
GroupID string `url:"id,omitempty"`
|
||||
IPs []string `url:"add_ips,comma,omitempty"`
|
||||
}
|
||||
|
||||
// ListAssetGroups retrieves a list of AssetGroups
|
||||
func (s *AssetsServiceOp) ListAssetGroups(opt *ListAssetGroupOptions) ([]AssetGroup, *Response, error) {
|
||||
return s.listAssetGroups(opt)
|
||||
}
|
||||
|
||||
// GetAssetGroupByID retrieves an AssetGroup by id.
|
||||
func (s *AssetsServiceOp) GetAssetGroupByID(groupID string) (*AssetGroup, *Response, error) {
|
||||
return s.getAssetGroup(groupID)
|
||||
}
|
||||
|
||||
// AddIPsToGroup adds the IPs in AddIPsToGroupOptions to the AssetGroup
|
||||
func (s *AssetsServiceOp) AddIPsToGroup(opt *AddIPsToGroupOptions) (*Response, error) {
|
||||
return s.addIPsToGroup(opt)
|
||||
}
|
||||
|
||||
func (s *AssetsServiceOp) getAssetGroup(groupID string) (*AssetGroup, *Response, error) {
|
||||
opts := ListAssetGroupOptions{Ids: []string{groupID}}
|
||||
groups, response, err := s.listAssetGroups(&opts)
|
||||
if err != nil {
|
||||
return nil, response, err
|
||||
}
|
||||
if len(groups) == 0 {
|
||||
return nil, response, nil
|
||||
}
|
||||
return &groups[0], response, nil
|
||||
}
|
||||
|
||||
func (s *AssetsServiceOp) addIPsToGroup(opt *AddIPsToGroupOptions) (*Response, error) {
|
||||
path := fmt.Sprintf("%s/%s/?action=edit", assetsBasePath, groupsBasePath)
|
||||
|
||||
req, err := s.client.NewRequest("POST", path, opt)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := s.client.MakeRequest(req, nil)
|
||||
if err != nil {
|
||||
return resp, err
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// Helper method for listing asset groups
|
||||
func (s *AssetsServiceOp) listAssetGroups(listOpt *ListAssetGroupOptions) ([]AssetGroup, *Response, error) {
|
||||
path := fmt.Sprintf("%s/%s/", assetsBasePath, groupsBasePath)
|
||||
if listOpt == nil {
|
||||
listOpt = &ListAssetGroupOptions{}
|
||||
}
|
||||
listOpt.Action = "list"
|
||||
path, err := addURLParameters(path, listOpt)
|
||||
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
req, err := s.client.NewRequest("GET", path, nil)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
root := new(assetGroupsRoot)
|
||||
resp, err := s.client.MakeRequest(req, root)
|
||||
if err != nil {
|
||||
return nil, resp, err
|
||||
}
|
||||
return root.AssetGroups, resp, err
|
||||
}
|
||||
251
qualys/assets_test.go
Normal file
251
qualys/assets_test.go
Normal file
@ -0,0 +1,251 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestListAssetGroups(t *testing.T) {
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
response string
|
||||
expected []AssetGroup
|
||||
opts *ListAssetGroupOptions
|
||||
isErr bool
|
||||
}{
|
||||
{
|
||||
name: "ListAssetGroups - single item, without list options",
|
||||
response: assetGroupsXMLSingleGroup,
|
||||
expected: []AssetGroup{
|
||||
{
|
||||
ID: "1759735",
|
||||
Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{
|
||||
IPs: []string{"10.1.1.1", "10.10.10.11"},
|
||||
IPRanges: nil,
|
||||
},
|
||||
},
|
||||
},
|
||||
opts: nil,
|
||||
},
|
||||
{
|
||||
name: "ListAssetGroups - single item, with list options",
|
||||
response: assetGroupsXMLSingleGroup,
|
||||
expected: []AssetGroup{
|
||||
{
|
||||
ID: "1759735",
|
||||
Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{
|
||||
IPs: []string{"10.1.1.1", "10.10.10.11"},
|
||||
IPRanges: nil,
|
||||
},
|
||||
},
|
||||
},
|
||||
opts: &ListAssetGroupOptions{Ids: []string{}},
|
||||
},
|
||||
{
|
||||
name: "ListAssetGroups - multi item",
|
||||
response: assetGroupsXMLMultiGroups,
|
||||
expected: []AssetGroup{
|
||||
{ID: "1759734", Title: "AG - New"},
|
||||
{ID: "1759735", Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{
|
||||
IPs: []string{"10.10.10.14"},
|
||||
IPRanges: []string{"10.10.10.3-10.10.10.6"},
|
||||
},
|
||||
},
|
||||
},
|
||||
opts: &ListAssetGroupOptions{Ids: []string{"1", "2"}},
|
||||
},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
setup()
|
||||
defer teardown()
|
||||
mux.HandleFunc("/asset/group/", func(w http.ResponseWriter, r *http.Request) {
|
||||
testMethod(t, r, "GET")
|
||||
fmt.Fprint(w, c.response)
|
||||
})
|
||||
|
||||
assetGroups, _, err := client.Assets.ListAssetGroups(c.opts)
|
||||
if err != nil {
|
||||
t.Errorf("Assets.ListAssetGroups returned error: %v", err)
|
||||
}
|
||||
|
||||
if !reflect.DeepEqual(assetGroups, c.expected) {
|
||||
t.Errorf("Assets.ListAssetGroups case: %s returned %+v, expected %+v", c.name, assetGroups, c.expected)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetAssetGroupByID(t *testing.T) {
|
||||
setup()
|
||||
defer teardown()
|
||||
|
||||
mux.HandleFunc("/asset/group/", func(w http.ResponseWriter, r *http.Request) {
|
||||
testMethod(t, r, "GET")
|
||||
fmt.Fprint(w, assetGroupsXMLSingleGroup)
|
||||
})
|
||||
|
||||
groupID := "1759735"
|
||||
|
||||
assetGroup, _, err := client.Assets.GetAssetGroupByID(groupID)
|
||||
if err != nil {
|
||||
t.Errorf("Assets.GetAssetGroupByID(%s) returned error: %v", groupID, err)
|
||||
}
|
||||
|
||||
expected := &AssetGroup{
|
||||
ID: "1759735",
|
||||
Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{
|
||||
IPs: []string{"10.1.1.1", "10.10.10.11"},
|
||||
IPRanges: nil,
|
||||
},
|
||||
}
|
||||
if !reflect.DeepEqual(assetGroup, expected) {
|
||||
t.Errorf("Assets.GetAssetGroupByID(%s) returned %+v, expected %+v", groupID, assetGroup, expected)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddIPsToGroup(t *testing.T) {
|
||||
setup()
|
||||
defer teardown()
|
||||
|
||||
groupID := "1759735"
|
||||
ip := "10.10.10.10"
|
||||
|
||||
mux.HandleFunc("/asset/group/", func(w http.ResponseWriter, r *http.Request) {
|
||||
testMethod(t, r, "POST")
|
||||
if r.FormValue("add_ips") != ip {
|
||||
t.Errorf("Request form data did not include the correct IP")
|
||||
}
|
||||
if r.FormValue("id") != groupID {
|
||||
t.Errorf("Request form data did not include the correct asset group ID")
|
||||
}
|
||||
fmt.Fprint(w, assetGroupsAddIPsResponse)
|
||||
})
|
||||
opts := &AddIPsToGroupOptions{
|
||||
GroupID: groupID,
|
||||
IPs: []string{ip},
|
||||
}
|
||||
|
||||
_, err := client.Assets.AddIPsToGroup(opts)
|
||||
if err != nil {
|
||||
t.Errorf("Assets.AddIPsToGroup returned error: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssetGroupContainsIP(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
ip string
|
||||
group *AssetGroup
|
||||
expected bool
|
||||
}{
|
||||
{
|
||||
name: "AssetGroup.ContainsIP - nil",
|
||||
ip: "10.1.1.1",
|
||||
group: &AssetGroup{ID: "1759735", Title: "AG - Elastic Cloud Dynamic Perimeter"},
|
||||
expected: false,
|
||||
},
|
||||
{
|
||||
name: "AssetGroup.ContainsIP - empty",
|
||||
ip: "10.1.1.1",
|
||||
group: &AssetGroup{
|
||||
ID: "1759735",
|
||||
Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{}},
|
||||
expected: false,
|
||||
},
|
||||
{
|
||||
name: "AssetGroup.ContainsIP - single item list",
|
||||
ip: "10.1.1.1",
|
||||
group: &AssetGroup{
|
||||
ID: "1759735",
|
||||
Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{
|
||||
IPs: []string{"10.1.1.1"},
|
||||
IPRanges: []string{},
|
||||
},
|
||||
},
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
name: "AssetGroup.ContainsIP - multi item list",
|
||||
ip: "10.1.1.1",
|
||||
group: &AssetGroup{
|
||||
ID: "1759735",
|
||||
Title: "AG - Elastic Cloud Dynamic Perimeter",
|
||||
IPs: AssetGroupIPs{
|
||||
IPs: []string{"10.1.1.1"},
|
||||
IPRanges: []string{"10.10.1.1-10.10.10.10"},
|
||||
},
|
||||
},
|
||||
expected: true,
|
||||
},
|
||||
}
|
||||
for _, c := range cases {
|
||||
contains := c.group.ContainsIP(c.ip)
|
||||
if contains != c.expected {
|
||||
t.Errorf("%s - AssetGroup.ContainsIP(%s) returned %v, expected %v", c.name, c.ip, contains, c.expected)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssetGroupIPsContainsIP(t *testing.T) {
|
||||
group := AssetGroupIPs{IPs: []string{"10.0.1.1"}, IPRanges: []string{"10.10.10.3-10.10.10.6"}}
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
ip string
|
||||
group AssetGroupIPs
|
||||
expected bool
|
||||
}{
|
||||
{
|
||||
name: "AssetGroupIPs.ContainsIP - IP value match",
|
||||
ip: "10.0.1.1",
|
||||
group: group,
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
name: "AssetGroupIPs.ContainsIP - IP value no match",
|
||||
ip: "192.0.1.1",
|
||||
group: group,
|
||||
expected: false,
|
||||
},
|
||||
{
|
||||
name: "AssetGroupIPs.ContainsIP - IP Range value match",
|
||||
ip: "10.10.10.4",
|
||||
group: group,
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
name: "AssetGroupIPs.ContainsIP - IP Range value no match",
|
||||
ip: "10.10.10.1",
|
||||
group: group,
|
||||
expected: false,
|
||||
},
|
||||
{
|
||||
name: "AssetGroupIPs.ContainsIP - IP Range value match",
|
||||
ip: "10.10.0.4",
|
||||
group: AssetGroupIPs{IPs: []string{"10.0.1.1"}, IPRanges: []string{"10.10.0.0-10.10.10.6"}},
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
name: "AssetGroupIPs.ContainsIP - IP Range value no match",
|
||||
ip: "10.10.0.4",
|
||||
group: AssetGroupIPs{IPs: []string{"10.0.1.1"}, IPRanges: []string{"10.10.1.3-10.10.10.6"}},
|
||||
expected: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
contains := c.group.ContainsIP(c.ip)
|
||||
if contains != c.expected {
|
||||
t.Errorf("%s - AssetGroupIPs.ContainsIP(%s) returned %v, expected %v", c.name, c.ip, contains, c.expected)
|
||||
}
|
||||
}
|
||||
}
|
||||
68
qualys/assets_xml_fixtures_test.go
Normal file
68
qualys/assets_xml_fixtures_test.go
Normal file
@ -0,0 +1,68 @@
|
||||
package qualys
|
||||
|
||||
const (
|
||||
assetGroupsXMLSingleGroup = `
|
||||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!DOCTYPE ASSET_GROUP_LIST_OUTPUT SYSTEM "https://qualysapi.qualys.com/api/2.0/fo/asset/group/asset_group_list_output.dtd">
|
||||
<ASSET_GROUP_LIST_OUTPUT>
|
||||
<RESPONSE>
|
||||
<DATETIME>2016-10-05T19:00:22Z</DATETIME>
|
||||
<ASSET_GROUP_LIST>
|
||||
<ASSET_GROUP>
|
||||
<ID>1759735</ID>
|
||||
<TITLE><![CDATA[AG - Elastic Cloud Dynamic Perimeter]]></TITLE>
|
||||
<IP_SET>
|
||||
<IP>10.1.1.1</IP>
|
||||
<IP>10.10.10.11</IP>
|
||||
</IP_SET>
|
||||
</ASSET_GROUP>
|
||||
</ASSET_GROUP_LIST>
|
||||
</RESPONSE>
|
||||
</ASSET_GROUP_LIST_OUTPUT>
|
||||
<!-- CONFIDENTIAL AND PROPRIETARY INFORMATION. Qualys provides the QualysGuard Service "As Is," without any warranty of any kind. Qualys makes no warranty that the information contained in this report is complete or error-free. Copyright 2016, Qualys, Inc. //-->
|
||||
`
|
||||
|
||||
assetGroupsXMLMultiGroups = `
|
||||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!DOCTYPE ASSET_GROUP_LIST_OUTPUT SYSTEM "https://qualysapi.qualys.com/api/2.0/fo/asset/group/asset_group_list_output.dtd">
|
||||
<ASSET_GROUP_LIST_OUTPUT>
|
||||
<RESPONSE>
|
||||
<DATETIME>2016-10-05T19:00:22Z</DATETIME>
|
||||
<ASSET_GROUP_LIST>
|
||||
<ASSET_GROUP>
|
||||
<ID>1759734</ID>
|
||||
<TITLE><![CDATA[AG - New]]></TITLE>
|
||||
<DEFAULT_APPLIANCE_ID>105102</DEFAULT_APPLIANCE_ID>
|
||||
<APPLIANCE_IDS>105102</APPLIANCE_IDS>
|
||||
</ASSET_GROUP>
|
||||
<ASSET_GROUP>
|
||||
<ID>1759735</ID>
|
||||
<TITLE><![CDATA[AG - Elastic Cloud Dynamic Perimeter]]></TITLE>
|
||||
<IP_SET>
|
||||
<IP_RANGE>10.10.10.3-10.10.10.6</IP_RANGE>
|
||||
<IP>10.10.10.14</IP>
|
||||
</IP_SET>
|
||||
</ASSET_GROUP>
|
||||
</ASSET_GROUP_LIST>
|
||||
</RESPONSE>
|
||||
</ASSET_GROUP_LIST_OUTPUT>
|
||||
<!-- CONFIDENTIAL AND PROPRIETARY INFORMATION. Qualys provides the QualysGuard Service "As Is," without any warranty of any kind. Qualys makes no warranty that the information contained in this report is complete or error-free. Copyright 2016, Qualys, Inc. //-->
|
||||
`
|
||||
|
||||
assetGroupsAddIPsResponse = `
|
||||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!DOCTYPE SIMPLE_RETURN SYSTEM "https://qualysapi.qualys.com/api/2.0/simple_return.dtd">
|
||||
<SIMPLE_RETURN>
|
||||
<RESPONSE>
|
||||
<DATETIME>2016-10-12T14:16:22Z</DATETIME>
|
||||
<TEXT>Asset Group Updated Successfully</TEXT>
|
||||
<ITEM_LIST>
|
||||
<ITEM>
|
||||
<KEY>ID</KEY>
|
||||
<VALUE>1759735</VALUE>
|
||||
</ITEM>
|
||||
</ITEM_LIST>
|
||||
</RESPONSE>
|
||||
</SIMPLE_RETURN>
|
||||
`
|
||||
)
|
||||
130
qualys/client.go
Normal file
130
qualys/client.go
Normal file
@ -0,0 +1,130 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
)
|
||||
|
||||
const (
|
||||
libraryVersion = "0.1.0"
|
||||
defaultBaseURL = "https://qualysapi.qualys.com/api/2.0/fo/"
|
||||
userAgent = "go-qualys"
|
||||
mediaType = "application/xml"
|
||||
|
||||
headerUserAgent = "X-Requested-With"
|
||||
headerRateLimit = "X-RateLimit-Limit"
|
||||
headerRateLimitWindow = "X-RateLimit-Window-Sec"
|
||||
headerRateRemaining = "X-RateLimit-Remaining"
|
||||
headerRateLimitWait = "X-RateLimit-ToWait-Sec"
|
||||
headerConcurrencyLimit = "X-Concurrency-Limit-Limit"
|
||||
headerConcurrencyLimitRunning = "X-Concurrency-Limit-Running"
|
||||
)
|
||||
|
||||
// Client for Qualys API
|
||||
type Client struct {
|
||||
// Credentials used to authenticate to the Qualys API
|
||||
Credentials *Credentials
|
||||
|
||||
// HTTP client used to communicate with the Qualys API
|
||||
client *http.Client
|
||||
|
||||
// Base URL for API requests.
|
||||
BaseURL *url.URL
|
||||
|
||||
// User agent for client
|
||||
UserAgent string
|
||||
|
||||
// Rate contains the current rate limit for the client as determined by the most recent
|
||||
// API call.
|
||||
Rate Rate
|
||||
|
||||
// Services used for communicating with the API
|
||||
Assets AssetsService
|
||||
}
|
||||
|
||||
// Rate contains the rate limit for the current client.
|
||||
type Rate struct {
|
||||
// The number of requests within the limit window of seconds the client is allowed
|
||||
Limit int
|
||||
|
||||
// The number of seconds remaining in the limit window
|
||||
LimitWindow int
|
||||
|
||||
// The number of remaining requests the client can make during the limit window period
|
||||
Remaining int
|
||||
|
||||
// The number of seconds to wait before requests can be made again -- headerRateLimitWait
|
||||
WaitingPeriod int
|
||||
|
||||
// The number of API calls permitted to be executed concurrrently
|
||||
ConcurrencyLimit int
|
||||
|
||||
// The number of API calls currently running
|
||||
CurrentConcurrency int
|
||||
}
|
||||
|
||||
// Credentials holds the credentials and endpoint for the Qualys Client
|
||||
type Credentials struct {
|
||||
Username string
|
||||
Password string
|
||||
}
|
||||
|
||||
// ClientOpt are options for New.
|
||||
type ClientOpt func(*Client) error
|
||||
|
||||
// New returns a new API client instance.
|
||||
func New(httpClient *http.Client, credentials *Credentials, opts ...ClientOpt) (*Client, error) {
|
||||
c, err := NewClient(httpClient, credentials)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, opt := range opts {
|
||||
if err := opt(c); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// NewClient returns a new Qualys API client.
|
||||
func NewClient(httpClient *http.Client, credentials *Credentials) (*Client, error) {
|
||||
if httpClient == nil {
|
||||
httpClient = http.DefaultClient
|
||||
}
|
||||
if credentials == nil || credentials.Username == "" || credentials.Password == "" {
|
||||
return nil, fmt.Errorf("Credentials must be provided")
|
||||
}
|
||||
|
||||
baseURL, err := url.Parse(defaultBaseURL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
c := &Client{client: httpClient, Credentials: credentials, BaseURL: baseURL, UserAgent: userAgent}
|
||||
|
||||
c.Assets = &AssetsServiceOp{client: c}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// SetBaseURL is a client option for setting the base URL.
|
||||
func SetBaseURL(bu string) ClientOpt {
|
||||
return func(c *Client) error {
|
||||
u, err := url.Parse(bu)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
c.BaseURL = u
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// SetUserAgent is a client option for setting the user agent.
|
||||
func SetUserAgent(ua string) ClientOpt {
|
||||
return func(c *Client) error {
|
||||
c.UserAgent = fmt.Sprintf("%s+%s", ua, c.UserAgent)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
207
qualys/client_test.go
Normal file
207
qualys/client_test.go
Normal file
@ -0,0 +1,207 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
var (
|
||||
mux *http.ServeMux
|
||||
|
||||
client *Client
|
||||
|
||||
server *httptest.Server
|
||||
|
||||
creds *Credentials
|
||||
)
|
||||
|
||||
func setup() {
|
||||
mux = http.NewServeMux()
|
||||
server = httptest.NewServer(mux)
|
||||
creds = &Credentials{Username: "bogus", Password: "bogus"}
|
||||
client, _ = NewClient(nil, creds)
|
||||
url, _ := url.Parse(server.URL)
|
||||
client.BaseURL = url
|
||||
}
|
||||
|
||||
func teardown() {
|
||||
server.Close()
|
||||
}
|
||||
|
||||
func testMethod(t *testing.T, r *http.Request, expected string) {
|
||||
if expected != r.Method {
|
||||
t.Errorf("Request method = %v, expected %v", r.Method, expected)
|
||||
}
|
||||
}
|
||||
|
||||
type values map[string]string
|
||||
|
||||
func testFormValues(t *testing.T, r *http.Request, values values) {
|
||||
expected := url.Values{}
|
||||
for k, v := range values {
|
||||
expected.Add(k, v)
|
||||
}
|
||||
|
||||
err := r.ParseForm()
|
||||
if err != nil {
|
||||
t.Fatalf("parseForm(): %v", err)
|
||||
}
|
||||
|
||||
if !reflect.DeepEqual(expected, r.Form) {
|
||||
t.Errorf("Request parameters = %v, expected %v", r.Form, expected)
|
||||
}
|
||||
}
|
||||
|
||||
func testURLParseError(t *testing.T, err error) {
|
||||
if err == nil {
|
||||
t.Errorf("Expected error to be returned")
|
||||
}
|
||||
if err, ok := err.(*url.Error); !ok || err.Op != "parse" {
|
||||
t.Errorf("Expected URL parse error, got %+v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func testClientDefaultBaseURL(t *testing.T, c *Client) {
|
||||
if c.BaseURL == nil || c.BaseURL.String() != defaultBaseURL {
|
||||
t.Errorf("NewClient BaseURL = %v, expected %v", c.BaseURL, defaultBaseURL)
|
||||
}
|
||||
}
|
||||
|
||||
func testClientDefaultUserAgent(t *testing.T, c *Client) {
|
||||
if c.UserAgent != userAgent {
|
||||
t.Errorf("NewClick UserAgent = %v, expected %v", c.UserAgent, userAgent)
|
||||
}
|
||||
}
|
||||
|
||||
func testClientDefaults(t *testing.T, c *Client) {
|
||||
testClientDefaultBaseURL(t, c)
|
||||
testClientDefaultUserAgent(t, c)
|
||||
}
|
||||
|
||||
func TestNewClient(t *testing.T) {
|
||||
c, _ := NewClient(nil, creds)
|
||||
testClientDefaults(t, c)
|
||||
}
|
||||
|
||||
func TestNew(t *testing.T) {
|
||||
c, err := New(nil, creds)
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("New(): %v", err)
|
||||
}
|
||||
testClientDefaults(t, c)
|
||||
}
|
||||
|
||||
func TestNewClientWithoutCredentials(t *testing.T) {
|
||||
_, err := NewClient(nil, nil)
|
||||
|
||||
if err == nil {
|
||||
t.Errorf("NewClient() expected error when Credentials are not set")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCustomUserAgent(t *testing.T) {
|
||||
c, err := New(nil, creds, SetUserAgent("testing"))
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("New() unexpected error: %v", err)
|
||||
}
|
||||
|
||||
expected := fmt.Sprintf("%s+%s", "testing", userAgent)
|
||||
if got := c.UserAgent; got != expected {
|
||||
t.Errorf("New() UserAgent = %s; expected %s", got, expected)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddURLParameters(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
path string
|
||||
expected string
|
||||
opts *ListAssetGroupOptions
|
||||
isErr bool
|
||||
}{
|
||||
{
|
||||
name: "addURLParameters",
|
||||
path: "/asset/group/",
|
||||
expected: "/asset/group/?ids=1",
|
||||
opts: &ListAssetGroupOptions{Ids: []string{"1"}},
|
||||
isErr: false,
|
||||
},
|
||||
{
|
||||
name: "addURLParameters with slice parameter",
|
||||
path: "/asset/group/",
|
||||
expected: "/asset/group/?ids=1,2",
|
||||
opts: &ListAssetGroupOptions{Ids: []string{"1", "2"}},
|
||||
isErr: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
got, err := addURLParameters(c.path, c.opts)
|
||||
if c.isErr && err == nil {
|
||||
t.Errorf("%q expected error but none was encountered", c.name)
|
||||
continue
|
||||
}
|
||||
|
||||
if !c.isErr && err != nil {
|
||||
t.Errorf("%q unexpected error: %v", c.name, err)
|
||||
continue
|
||||
}
|
||||
|
||||
gotURL, err := url.Parse(got)
|
||||
if err != nil {
|
||||
t.Errorf("%q unable to parse returned URL", c.name)
|
||||
continue
|
||||
}
|
||||
|
||||
expectedURL, err := url.Parse(c.expected)
|
||||
if err != nil {
|
||||
t.Errorf("%q unable to parse expected URL", c.name)
|
||||
continue
|
||||
}
|
||||
|
||||
if g, e := gotURL.Path, expectedURL.Path; g != e {
|
||||
t.Errorf("%q path = %q; expected %q", c.name, g, e)
|
||||
continue
|
||||
}
|
||||
|
||||
if g, e := gotURL.Query(), expectedURL.Query(); !reflect.DeepEqual(g, e) {
|
||||
t.Errorf("%q query = %#v; expected %#v", c.name, g, e)
|
||||
continue
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormPostBody(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
expected string
|
||||
opts *AddIPsToGroupOptions
|
||||
}{
|
||||
{
|
||||
name: "formPostBody single IP",
|
||||
expected: "add_ips=10.10.10.10&id=1234",
|
||||
opts: &AddIPsToGroupOptions{GroupID: "1234", IPs: []string{"10.10.10.10"}},
|
||||
},
|
||||
{
|
||||
name: "formPostBody multi IP",
|
||||
expected: "add_ips=10.10.10.10%2C10.10.10.11&id=1234",
|
||||
opts: &AddIPsToGroupOptions{GroupID: "1234", IPs: []string{"10.10.10.10", "10.10.10.11"}},
|
||||
},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got, _ := formPostBody(c.opts)
|
||||
buf := new(bytes.Buffer)
|
||||
buf.ReadFrom(got)
|
||||
bodyString := buf.String()
|
||||
if c.expected != bodyString {
|
||||
t.Errorf("%q expected %s but got %s", c.name, c.expected, bodyString)
|
||||
}
|
||||
}
|
||||
}
|
||||
137
qualys/requests.go
Normal file
137
qualys/requests.go
Normal file
@ -0,0 +1,137 @@
|
||||
package qualys // import "git.openstack.org/openstack/osel/qualys"
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/xml"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
|
||||
"github.com/google/go-querystring/query"
|
||||
)
|
||||
|
||||
// Response is a Qualys API response. This wraps the standard http.Response returned from Qualys.
|
||||
type Response struct {
|
||||
*http.Response
|
||||
|
||||
Rate
|
||||
}
|
||||
|
||||
// NewRequest creates an API request. A relative URL can be provided in urlStr, which will be resolved to the
|
||||
// BaseURL of the Client. Relative URLS should always be specified without a preceding slash. If specified, the
|
||||
// value pointed to by body is form encoded and included as the request body.
|
||||
func (c *Client) NewRequest(method, urlStr string, body interface{}) (*http.Request, error) {
|
||||
rel, err := url.Parse(urlStr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
u := c.BaseURL.ResolveReference(rel)
|
||||
buf := new(bytes.Buffer)
|
||||
|
||||
if method == http.MethodPost {
|
||||
buf, err = formPostBody(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
req, err := http.NewRequest(method, u.String(), buf)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if method == http.MethodPost {
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
}
|
||||
req.SetBasicAuth(c.Credentials.Username, c.Credentials.Password)
|
||||
req.Header.Set(headerUserAgent, userAgent)
|
||||
return req, nil
|
||||
}
|
||||
|
||||
// newResponse creates a new Response for the provided http.Response
|
||||
func newResponse(r *http.Response) *Response {
|
||||
response := Response{Response: r}
|
||||
response.populateRate()
|
||||
|
||||
return &response
|
||||
}
|
||||
|
||||
// populateRate parses the rate related headers and populates the response Rate.
|
||||
func (r *Response) populateRate() {
|
||||
// TODO - deal with the rest of the headers
|
||||
if limit := r.Header.Get(headerRateLimit); limit != "" {
|
||||
r.Rate.Limit, _ = strconv.Atoi(limit)
|
||||
}
|
||||
if remaining := r.Header.Get(headerRateRemaining); remaining != "" {
|
||||
r.Rate.Remaining, _ = strconv.Atoi(remaining)
|
||||
}
|
||||
if rateLimitWindow := r.Header.Get(headerRateLimitWindow); rateLimitWindow != "" {
|
||||
r.Rate.LimitWindow, _ = strconv.Atoi(rateLimitWindow)
|
||||
}
|
||||
if waitingPeriod := r.Header.Get(headerRateLimitWait); waitingPeriod != "" {
|
||||
r.Rate.WaitingPeriod, _ = strconv.Atoi(waitingPeriod)
|
||||
}
|
||||
if concurrencyLimit := r.Header.Get(headerConcurrencyLimit); concurrencyLimit != "" {
|
||||
r.Rate.ConcurrencyLimit, _ = strconv.Atoi(concurrencyLimit)
|
||||
}
|
||||
if runningConcurrencyLimit := r.Header.Get(headerConcurrencyLimitRunning); runningConcurrencyLimit != "" {
|
||||
r.Rate.CurrentConcurrency, _ = strconv.Atoi(runningConcurrencyLimit)
|
||||
}
|
||||
}
|
||||
|
||||
// MakeRequest sends an API request and returns the API response. The API response is XML decoded and stored in the value
|
||||
// pointed to by v, or returned as an error if an API error has occurred..
|
||||
func (c *Client) MakeRequest(req *http.Request, v interface{}) (*Response, error) {
|
||||
resp, err := c.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
defer func() {
|
||||
if rerr := resp.Body.Close(); err == nil {
|
||||
err = rerr
|
||||
}
|
||||
}()
|
||||
|
||||
response := newResponse(resp)
|
||||
c.Rate = response.Rate
|
||||
|
||||
err = CheckResponse(resp)
|
||||
if err != nil {
|
||||
return response, err
|
||||
}
|
||||
|
||||
bodyContents, err := ioutil.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return response, err
|
||||
}
|
||||
|
||||
if v != nil {
|
||||
err := xml.Unmarshal(bodyContents, v)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
return response, err
|
||||
}
|
||||
|
||||
// CheckResponse checks the API response for errors, and returns them if present. A response is considered an
|
||||
// error if it has a status code outside the 200 range.
|
||||
func CheckResponse(r *http.Response) error {
|
||||
if c := r.StatusCode; c >= 200 && c <= 299 {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("Response status is: %v", r.StatusCode)
|
||||
}
|
||||
|
||||
func formPostBody(opt interface{}) (*bytes.Buffer, error) {
|
||||
vals, err := query.Values(opt)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return bytes.NewBufferString(vals.Encode()), nil
|
||||
}
|
||||
186
qualys/scans.go
Normal file
186
qualys/scans.go
Normal file
@ -0,0 +1,186 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"encoding/xml"
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var ErrMalformedResponse error = errors.New("malformed xml response from server")
|
||||
|
||||
// LaunchScanResponse is the expected response for a scan launch
|
||||
// xml tag: simple_return
|
||||
// DateTime is the date and time the scan was issued
|
||||
// It includes a key/value map of pertinent information.
|
||||
type LaunchScanResponse struct {
|
||||
XMLName xml.Name `xml:"SIMPLE_RETURN"`
|
||||
Value []string `xml:"RESPONSE>ITEM_LIST>ITEM>VALUE"`
|
||||
Key []string `xml:"RESPONSE>ITEM_LIST>ITEM>KEY"`
|
||||
Datetime string `xml:"RESPONSE>DATETIME"`
|
||||
Text string `xml:"RESPONSE>TEXT"`
|
||||
|
||||
ScanReference string
|
||||
|
||||
RateLimitations Rate
|
||||
}
|
||||
|
||||
type LaunchScanOptions struct {
|
||||
ScanTitle string `url:"scan_title"`
|
||||
OptionID int64 `url:"option_id,omitempty"`
|
||||
OptionTitle string `url:"option_title"`
|
||||
ScannerName string `url:"iscanner_name"`
|
||||
IP []string `url:"ip"`
|
||||
Action string `url:"action"`
|
||||
}
|
||||
|
||||
// ScanLaunch will try and run a scan on demand
|
||||
// has certain parameters that must be filled in
|
||||
func (client *Client) LaunchScan(options *LaunchScanOptions) (*LaunchScanResponse, error) {
|
||||
options.Action = "launch"
|
||||
|
||||
urlString, err := addURLParameters(client.BaseURL.String(), options)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// i don't think there's any header data here?
|
||||
req, err := client.NewRequest(http.MethodPost, urlString, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp LaunchScanResponse
|
||||
|
||||
response, err := client.MakeRequest(req, &resp)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.RateLimitations = response.Rate
|
||||
|
||||
for key, val := range resp.Key {
|
||||
if strings.ToUpper(val) == "REFERENCE" {
|
||||
// should check len first
|
||||
if len(resp.Value) > key {
|
||||
resp.ScanReference = resp.Value[key]
|
||||
} else {
|
||||
return nil, ErrMalformedResponse
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
type PollScanResponse struct {
|
||||
XMLName xml.Name `xml:"SCAN_LIST_OUTPUT"`
|
||||
Datetime string `xml:"RESPONSE>DATETIME"`
|
||||
Processing_priority string `xml:"RESPONSE>SCAN_LIST>SCAN>PROCESSING_PRIORITY"`
|
||||
Processed string `xml:"RESPONSE>SCAN_LIST>SCAN>PROCESSED"`
|
||||
Launch_datetime string `xml:"RESPONSE>SCAN_LIST>SCAN>LAUNCH_DATETIME"`
|
||||
Target string `xml:"RESPONSE>SCAN_LIST>SCAN>TARGET"`
|
||||
Type string `xml:"RESPONSE>SCAN_LIST>SCAN>TYPE"`
|
||||
Title string `xml:"RESPONSE>SCAN_LIST>SCAN>TITLE"`
|
||||
User_login string `xml:"RESPONSE>SCAN_LIST>SCAN>USER_LOGIN"`
|
||||
Ref string `xml:"RESPONSE>SCAN_LIST>SCAN>REF"`
|
||||
Duration string `xml:"RESPONSE>SCAN_LIST>SCAN>DURATION"`
|
||||
State string `xml:"RESPONSE>SCAN_LIST>SCAN>STATUS>STATE"`
|
||||
}
|
||||
|
||||
type PollScanOptions struct {
|
||||
Action string `url:"action"` // list
|
||||
ScanRef string `url:"scan_ref"`
|
||||
}
|
||||
|
||||
func (client *Client) PollScanResults(options *PollScanOptions) (*PollScanResponse, error) {
|
||||
options.Action = "list"
|
||||
|
||||
urlString, err := addURLParameters(client.BaseURL.String(), options)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
req, err := client.NewRequest(http.MethodGet, urlString, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp PollScanResponse
|
||||
|
||||
_, requestError := client.MakeRequest(req, &resp)
|
||||
|
||||
if requestError != nil {
|
||||
return nil, requestError
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
type CompletedScanResponse struct {
|
||||
XMLName xml.Name `xml:"SIMPLE_RETURN"` // the actual outp was supposed to be: SCAN_LIST_OUTPUT?
|
||||
City string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>COMPANY_INFO>CITY"`
|
||||
Key []key `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>KEY"`
|
||||
Name string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>COMPANY_INFO>NAME"`
|
||||
ZIPCode string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>COMPANY_INFO>ZIP_CODE"`
|
||||
Address string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>COMPANY_INFO>ADDRESS"`
|
||||
NameUserInfoHeaderComplianceScanResponse string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>USER_INFO>NAME"`
|
||||
Username string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>USER_INFO>USERNAME"`
|
||||
Role string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>USER_INFO>ROLE"`
|
||||
Hosts string `xml:"RESPONSE>COMPLIANCE_SCAN>APPENDIX>TARGET_DISTRIBUTION>SCANNER>HOSTS"`
|
||||
Type string `xml:"RESPONSE>COMPLIANCE_SCAN>APPENDIX>AUTHENTICATION>AUTH>TYPE"`
|
||||
Datetime string `xml:"RESPONSE>DATETIME"`
|
||||
IP string `xml:"RESPONSE>COMPLIANCE_SCAN>APPENDIX>AUTHENTICATION>AUTH>SUCCESS>IP"`
|
||||
State string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>COMPANY_INFO>STATE"`
|
||||
Country string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>COMPANY_INFO>COUNTRY"`
|
||||
HostsScanned string `xml:"RESPONSE>COMPLIANCE_SCAN>APPENDIX>TARGET_HOSTS>HOSTS_SCANNED"`
|
||||
NameScannerTargetDistributionAppendixComplianceScanResponse string `xml:"RESPONSE>COMPLIANCE_SCAN>APPENDIX>TARGET_DISTRIBUTION>SCANNER>NAME"`
|
||||
GenerationDateTime string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>GENERATION_DATETIME"`
|
||||
NameHeaderComplianceScanResponse string `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>NAME"`
|
||||
OptionProfileTitle optionProfileTitle `xml:"RESPONSE>COMPLIANCE_SCAN>HEADER>OPTION_PROFILE>OPTION_PROFILE_TITLE"`
|
||||
}
|
||||
|
||||
type key struct {
|
||||
XMLName xml.Name `xml:"KEY"`
|
||||
Value string `xml:"value,attr"`
|
||||
Text string `xml:",chardata"`
|
||||
}
|
||||
type optionProfileTitle struct {
|
||||
XMLName xml.Name `xml:"OPTION_PROFILE_TITLE"`
|
||||
Option_profile_default string `xml:"option_profile_default,attr"`
|
||||
Text string `xml:",chardata"`
|
||||
}
|
||||
|
||||
type CompletedScanOptions struct {
|
||||
Action string `url:"action"` // fetch
|
||||
ScanRef string `url:"scan_ref"`
|
||||
}
|
||||
|
||||
func (client *Client) GetScanResults(options *CompletedScanOptions) (*CompletedScanResponse, error) {
|
||||
options.Action = "fetch"
|
||||
|
||||
urlString, err := addURLParameters(client.BaseURL.String(), options)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// i don't think there's any header data here?
|
||||
req, err := client.NewRequest(http.MethodGet, urlString, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp CompletedScanResponse
|
||||
|
||||
_, requestError := client.MakeRequest(req, &resp)
|
||||
|
||||
if requestError != nil {
|
||||
return nil, requestError
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
71
qualys/scans_test.go
Normal file
71
qualys/scans_test.go
Normal file
@ -0,0 +1,71 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var devCreds Credentials = Credentials{
|
||||
Username: "cmcas_ae2",
|
||||
Password: "D02debLYko",
|
||||
}
|
||||
|
||||
func TestLiveScan(t *testing.T) {
|
||||
// create client
|
||||
c, clientErr := NewClient(&http.Client{}, &devCreds)
|
||||
|
||||
if clientErr != nil {
|
||||
t.Error(clientErr)
|
||||
}
|
||||
|
||||
if baseURL, err := url.Parse("https://qualysapi.qualys.com/api/2.0/fo/scan/"); err != nil {
|
||||
t.Error(err)
|
||||
} else {
|
||||
c.BaseURL = baseURL
|
||||
}
|
||||
|
||||
// create the options
|
||||
opts := LaunchScanOptions{
|
||||
ScanTitle: "hello_world",
|
||||
ScannerName: "External",
|
||||
// OptionID: 923922,
|
||||
OptionTitle: "Elastic Cloud Option Profile with Password Guessing",
|
||||
IP: []string{"96.119.99.178"},
|
||||
}
|
||||
|
||||
// launch the request
|
||||
|
||||
launchScanResponse, err := c.LaunchScan(&opts)
|
||||
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
|
||||
// not sure if necessary
|
||||
time.Sleep(time.Minute * 1)
|
||||
|
||||
//time to poll the scan results
|
||||
pollOpts := PollScanOptions{
|
||||
ScanRef: launchScanResponse.ScanReference,
|
||||
}
|
||||
|
||||
_, pollRespErr := c.PollScanResults(&pollOpts)
|
||||
|
||||
if pollRespErr != nil {
|
||||
t.Error(pollRespErr)
|
||||
}
|
||||
|
||||
// now need to keep polling until the results are all in...
|
||||
|
||||
resultsOptions := CompletedScanOptions{
|
||||
ScanRef: launchScanResponse.ScanReference,
|
||||
}
|
||||
|
||||
_, resultsRespErr := c.GetScanResults(&resultsOptions)
|
||||
|
||||
if resultsRespErr != nil {
|
||||
t.Error(resultsRespErr)
|
||||
}
|
||||
}
|
||||
44
qualys/utils.go
Normal file
44
qualys/utils.go
Normal file
@ -0,0 +1,44 @@
|
||||
package qualys
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"reflect"
|
||||
|
||||
"github.com/google/go-querystring/query"
|
||||
)
|
||||
|
||||
func containsString(strList []string, testStr string) bool {
|
||||
for _, str := range strList {
|
||||
if testStr == str {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func addURLParameters(urlString string, opt interface{}) (string, error) {
|
||||
v := reflect.ValueOf(opt)
|
||||
|
||||
if v.Kind() == reflect.Ptr && v.IsNil() {
|
||||
return urlString, nil
|
||||
}
|
||||
|
||||
origURL, err := url.Parse(urlString)
|
||||
if err != nil {
|
||||
return urlString, err
|
||||
}
|
||||
|
||||
origValues := origURL.Query()
|
||||
|
||||
newValues, err := query.Values(opt)
|
||||
if err != nil {
|
||||
return urlString, err
|
||||
}
|
||||
|
||||
for k, v := range newValues {
|
||||
origValues[k] = v
|
||||
}
|
||||
|
||||
origURL.RawQuery = origValues.Encode()
|
||||
return origURL.String(), nil
|
||||
}
|
||||
18
qualys/xml/scan_launch.xml
Normal file
18
qualys/xml/scan_launch.xml
Normal file
@ -0,0 +1,18 @@
|
||||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!DOCTYPE SIMPLE_RETURN SYSTEM "https://qualysapi.qualys.com/api/2.0/simple_return.dtd">
|
||||
<SIMPLE_RETURN>
|
||||
<RESPONSE>
|
||||
<DATETIME>2017-02-02T14:28:35Z</DATETIME>
|
||||
<TEXT>New vm scan launched</TEXT>
|
||||
<ITEM_LIST>
|
||||
<ITEM>
|
||||
<KEY>ID</KEY>
|
||||
<VALUE>26148615</VALUE>
|
||||
</ITEM>
|
||||
<ITEM>
|
||||
<KEY>REFERENCE</KEY>
|
||||
<VALUE>scan/1486045714.48615</VALUE>
|
||||
</ITEM>
|
||||
</ITEM_LIST>
|
||||
</RESPONSE>
|
||||
</SIMPLE_RETURN>
|
||||
23
qualys/xml/scan_list_output.xml
Normal file
23
qualys/xml/scan_list_output.xml
Normal file
@ -0,0 +1,23 @@
|
||||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!DOCTYPE SCAN_LIST_OUTPUT SYSTEM "https://qualysapi.qualys.com/api/2.0/fo/scan/scan_list_output.dtd">
|
||||
<SCAN_LIST_OUTPUT>
|
||||
<RESPONSE>
|
||||
<DATETIME>2017-02-02T14:48:09Z</DATETIME>
|
||||
<SCAN_LIST>
|
||||
<SCAN>
|
||||
<REF>scan/1486045714.48615</REF>
|
||||
<TYPE>API</TYPE>
|
||||
<TITLE><![CDATA[Elastic Cloud Automated Scan]]></TITLE>
|
||||
<USER_LOGIN>cmcas_at1</USER_LOGIN>
|
||||
<LAUNCH_DATETIME>2017-02-02T14:28:34Z</LAUNCH_DATETIME>
|
||||
<DURATION>00:08:44</DURATION>
|
||||
<PROCESSING_PRIORITY>0 - No Priority</PROCESSING_PRIORITY>
|
||||
<PROCESSED>1</PROCESSED>
|
||||
<STATUS>
|
||||
<STATE>Finished</STATE>
|
||||
</STATUS>
|
||||
<TARGET><![CDATA[96.119.99.90]]></TARGET>
|
||||
</SCAN>
|
||||
</SCAN_LIST>
|
||||
</RESPONSE>
|
||||
</SCAN_LIST_OUTPUT>
|
||||
81
qualys/xml/scan_results.xml
Normal file
81
qualys/xml/scan_results.xml
Normal file
@ -0,0 +1,81 @@
|
||||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!DOCTYPE COMPLIANCE_SCAN_RESULT_OUTPUT SYSTEM
|
||||
"https://qualysapi.qualys.com/api/2.0/fo/scan/compliance/compliance_s
|
||||
can_result_output.dtd">
|
||||
<COMPLIANCE_SCAN_RESULT_OUTPUT>
|
||||
<RESPONSE>
|
||||
<DATETIME>2012-09-17T10:23:53Z</DATETIME>
|
||||
<COMPLIANCE_SCAN>
|
||||
<HEADER>
|
||||
<NAME>
|
||||
<![CDATA[Compliance Scan Results]]>
|
||||
</NAME>
|
||||
<GENERATION_DATETIME>
|
||||
2012-09-17T10:23:53Z
|
||||
</GENERATION_DATETIME>
|
||||
<COMPANY_INFO>
|
||||
<NAME><![CDATA[Qualys]]></NAME>
|
||||
<ADDRESS><![CDATA[1600 Bridge Parkway]]></ADDRESS>
|
||||
<CITY><![CDATA[Redwood Shores]]></CITY>
|
||||
<STATE><![CDATA[California]]></STATE>
|
||||
<COUNTRY><![CDATA[United States]]></COUNTRY>
|
||||
<ZIP_CODE><![CDATA[94065]]></ZIP_CODE>
|
||||
</COMPANY_INFO>
|
||||
<USER_INFO>
|
||||
<NAME><![CDATA[NAME]]></NAME>
|
||||
<USERNAME>USERNAME</USERNAME>
|
||||
<ROLE>Manager</ROLE>
|
||||
</USER_INFO>
|
||||
<KEY value="USERNAME">USERNAME</KEY>
|
||||
<KEY value="COMPANY"><![CDATA[Qualys]]></KEY>
|
||||
<KEY value="DATE">2012-09-15T11:49:08Z</KEY>
|
||||
<KEY value="TITLE"><![CDATA[My PC Scan]]></KEY>
|
||||
<KEY value="TARGET">10.10.10.29</KEY>
|
||||
<KEY value="EXCLUDED_TARGET"><![CDATA[N/A]]></KEY>
|
||||
<KEY value="DURATION">00:01:00</KEY>
|
||||
<KEY value="SCAN_HOST">
|
||||
10.10.21.122 (Scanner 6.6.28-1, Vulnerability Signatures 2.2.215-2)
|
||||
</KEY>
|
||||
<KEY value="NBHOST_ALIVE">1</KEY>
|
||||
<KEY value="NBHOST_TOTAL">1</KEY>
|
||||
<KEY value="REPORT_TYPE">Scheduled</KEY>
|
||||
<KEY value="OPTIONS">
|
||||
File Integrity Monitoring: Enabled,
|
||||
Scanned Ports: Standard Scan, Hosts to Scan in Parallel - External
|
||||
Scanners: 15, Hosts to Scan in Parallel - Scanner Appliances: 30,
|
||||
Total Processes to Run in Parallel: 10, HTTP Processes to Run in
|
||||
Parallel: 10,
|
||||
Packet (Burst) Delay: Medium, Intensity: Normal, Overall Performance:
|
||||
Normal, ICMP Host Discovery, Ignore RST packets: Off, Ignore firewallgenerated
|
||||
SYN-ACK packets: Off, Do not send ACK or SYN-ACK packets
|
||||
during host discovery: Off
|
||||
</KEY>
|
||||
<KEY value="STATUS">FINISHED</KEY>
|
||||
<OPTION_PROFILE>
|
||||
<OPTION_PROFILE_TITLE option_profile_default="0">
|
||||
<![CDATA[11412]]>
|
||||
</OPTION_PROFILE_TITLE>
|
||||
</OPTION_PROFILE>
|
||||
</HEADER>
|
||||
<APPENDIX>
|
||||
<TARGET_HOSTS>
|
||||
<HOSTS_SCANNED>10.10.10.29</HOSTS_SCANNED>
|
||||
</TARGET_HOSTS>
|
||||
<TARGET_DISTRIBUTION>
|
||||
<SCANNER>
|
||||
<NAME><![CDATA[iscan_sx]]></NAME>
|
||||
<HOSTS>10.10.10.29</HOSTS>
|
||||
</SCANNER>
|
||||
</TARGET_DISTRIBUTION>
|
||||
<AUTHENTICATION>
|
||||
<AUTH>
|
||||
<TYPE>Windows</TYPE>
|
||||
<SUCCESS>
|
||||
<IP>10.10.10.29</IP>
|
||||
</SUCCESS>
|
||||
</AUTH>
|
||||
</AUTHENTICATION>
|
||||
</APPENDIX>
|
||||
</COMPLIANCE_SCAN>
|
||||
</RESPONSE>
|
||||
</COMPLIANCE_SCAN_RESULT_OUTPUT>
|
||||
36
qualys_mock_test.go
Normal file
36
qualys_mock_test.go
Normal file
@ -0,0 +1,36 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
//"github.com/satori/go.uuid"
|
||||
)
|
||||
|
||||
// QualysActions is a class that handles all interactions directly with Qualys.
|
||||
// See the comment on QualysActioner for rationale.
|
||||
type QualysTestActions struct {
|
||||
testUUID string
|
||||
}
|
||||
|
||||
// InitiateScan is the main method for the QualysActioner class, it
|
||||
// makes a call to the Qualys API to start a scan and harvests a scan ID, and
|
||||
// an optional error string if there is a problem contacting Qualys.
|
||||
func (s *QualysTestActions) InitiateScan(ipAddresses []string) (string, error) {
|
||||
//testUUID = uuid.NewV4().String()
|
||||
s.testUUID = `5fbf3cef-976e-475d-bd84-47ef23638a6b`
|
||||
log.Printf("FAKE QUALYS SCAN: %s\n", s.testUUID)
|
||||
return s.testUUID, nil
|
||||
}
|
||||
|
||||
// GetTestScanID returns the fake UUID created in testing. This allows for
|
||||
// inspection of the UUID in unit tests.
|
||||
func (s *QualysTestActions) GetTestScanID() string {
|
||||
return s.testUUID
|
||||
}
|
||||
|
||||
func (s *QualysTestActions) DropIPv6() bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func connectFakeQualys() *QualysTestActions {
|
||||
return new(QualysTestActions)
|
||||
}
|
||||
24
releasenotes/notes/initial-import-8cdbc214e8596521.yaml
Normal file
24
releasenotes/notes/initial-import-8cdbc214e8596521.yaml
Normal file
@ -0,0 +1,24 @@
|
||||
---
|
||||
prelude: >
|
||||
This is the first public release of the OpenStack Event Listener (OSEL).
|
||||
It had previously been a project within Comcast, but was open-sourced
|
||||
under the Apache license.
|
||||
features:
|
||||
- |
|
||||
Connects to RabbitMQ to listen for notification events specific to security
|
||||
group changes. When those are intercepted, query Nova for information about
|
||||
what the affected IP addresses are, then initiate a Qualys scan. Finally
|
||||
send info in the IP addresses and the Qualys scan ID to syslog.
|
||||
issues:
|
||||
- |
|
||||
Only processes security group changes, should also process new port events
|
||||
as well.
|
||||
- |
|
||||
Needs to exponential backoff for AMQP connections.
|
||||
- |
|
||||
Needs to be integrated with Aodh for modern OpenStacks.
|
||||
security:
|
||||
- |
|
||||
Requires access to RabbitMQ as well as OpenStack credentials that have access
|
||||
to data in all projects, so this should be considered a privileged process and
|
||||
should be run in a properly secured context.
|
||||
81
security_group_events.go
Normal file
81
security_group_events.go
Normal file
@ -0,0 +1,81 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// EventSecurityGroupRuleChange is the event processor class for all changes to
|
||||
// security groups. This includes additions and deletions. This must conform
|
||||
// to the EventProcessor interface (see events.go).
|
||||
type EventSecurityGroupRuleChange struct {
|
||||
ChangeType string
|
||||
}
|
||||
|
||||
// FillExtraData takes a security group change and enriches it with additional
|
||||
// information about the affected IP addresses using the
|
||||
// OpenStackActionInterface getPortList function.
|
||||
func (s EventSecurityGroupRuleChange) FillExtraData(e *Event, openstack OpenStackActioner) error {
|
||||
// PopulateIps: This function returns a map of security group to array of IP addresses for all ports in the specified tenantID.
|
||||
|
||||
err := openstack.Connect(e.EventData.TenantID, e.EventData.UserName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Make port list request to neutron
|
||||
resultMap := openstack.GetPortList()
|
||||
resultIPAddresses := make(map[string][]string)
|
||||
for _, ipMap := range resultMap {
|
||||
resultIPAddresses[ipMap.securityGroup] = append(resultIPAddresses[ipMap.securityGroup], ipMap.ipAddress)
|
||||
}
|
||||
e.IPs = resultIPAddresses
|
||||
return nil
|
||||
}
|
||||
|
||||
// FormatLogs takes the accumulated event data and composes the JSON message to
|
||||
// be logged.
|
||||
func (s EventSecurityGroupRuleChange) FormatLogs(e *Event, scannedIPAddresses []string) ([]string, error) {
|
||||
var es osSecurityGroupRuleChange
|
||||
var logLines []string
|
||||
if e == nil {
|
||||
return logLines, fmt.Errorf("Event must not be nil")
|
||||
}
|
||||
|
||||
if err := json.Unmarshal(e.RawData, &es); err != nil {
|
||||
return logLines, err
|
||||
}
|
||||
|
||||
hostName, err := os.Hostname()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
es.Payload.ChangeType = s.ChangeType
|
||||
es.Payload.SourceType = OselVersion
|
||||
es.Payload.SourceMessageBus = hostName
|
||||
es.Payload.QualysScanID = e.QualysScanID
|
||||
es.Payload.QualysScanError = e.QualysScanError
|
||||
|
||||
affectedIPArray := e.IPs[es.Payload.SecurityGroupRule.SecurityGroupID]
|
||||
qualysScanJoin := fmt.Sprintf("|%s|", strings.Join(scannedIPAddresses, "|"))
|
||||
for _, affectedIPAddr := range affectedIPArray {
|
||||
es.Payload.QualysScanID = ""
|
||||
es.Payload.QualysScanError = ""
|
||||
if strings.Index(qualysScanJoin, fmt.Sprintf("|%s|", affectedIPAddr)) > -1 {
|
||||
es.Payload.QualysScanID = e.QualysScanID
|
||||
es.Payload.QualysScanError = e.QualysScanError
|
||||
} else {
|
||||
es.Payload.QualysScanID = ""
|
||||
es.Payload.QualysScanError = "Not scanned by Qualys"
|
||||
}
|
||||
es.Payload.AffectedIPAddr = affectedIPAddr
|
||||
jsonLine, err := json.Marshal(es.Payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
logLines = append(logLines, string(jsonLine))
|
||||
}
|
||||
return logLines, nil
|
||||
}
|
||||
80
structs.go
Normal file
80
structs.go
Normal file
@ -0,0 +1,80 @@
|
||||
package main
|
||||
|
||||
type openStackEvent struct {
|
||||
EventType string `json:"event_type"`
|
||||
Timestamp string `json:"timestamp"`
|
||||
TenantID string `json:"_context_tenant_id"`
|
||||
TenantName string `json:"_context_tenant_name"`
|
||||
User string `json:"_context_user"`
|
||||
UserName string `json:"_context_user_name"`
|
||||
UserID string `json:"_context_user_id"`
|
||||
IsAdmin bool `json:"_context_is_admin"`
|
||||
PublisherID string `json:"publisher_id"`
|
||||
MessageID string `json:"message_id"`
|
||||
}
|
||||
|
||||
type osSecurityGroupRule struct {
|
||||
RemoteGroupID interface{} `json:"remote_group_id"`
|
||||
Direction string `json:"direction"`
|
||||
Protocol interface{} `json:"protocol"`
|
||||
RemoteIPPrefix string `json:"remote_ip_prefix"`
|
||||
PortRangeMax interface{} `json:"port_range_max"`
|
||||
// Dscp interface{} `json:"dscp"`
|
||||
Rule string `json:"rule_direction"`
|
||||
SecurityGroupID string `json:"security_group_id"`
|
||||
TenantID string `json:"tenant_id"`
|
||||
PortRangeMin interface{} `json:"port_range_min"`
|
||||
Ethertype string `json:"ethertype"`
|
||||
ID string `json:"id"`
|
||||
}
|
||||
|
||||
type osSecurityGroupRuleChange struct {
|
||||
Payload struct {
|
||||
AffectedIPAddr interface{} `json:"affected_ip_address"`
|
||||
ChangeType string `json:"change_type"`
|
||||
QualysScanID string `json:"qualys_scan_id"`
|
||||
QualysScanError string `json:"qualys_scan_error"`
|
||||
SecurityGroupRule osSecurityGroupRule `json:"security_group_rule"`
|
||||
SourceType string `json:"source_type"`
|
||||
SourceMessageBus string `json:"source_message_bus"`
|
||||
} `json:"payload"`
|
||||
}
|
||||
|
||||
type osSecurityGroupRuleDelete struct {
|
||||
Payload struct {
|
||||
SecurityGroupRuleID string `json:"security_group_rule_id"`
|
||||
} `json:"payload"`
|
||||
}
|
||||
|
||||
type osPortCreate struct {
|
||||
Payload struct {
|
||||
Port osPort `json:"port"`
|
||||
} `json:"payload"`
|
||||
}
|
||||
|
||||
type osPort struct {
|
||||
Status string `json:"status"`
|
||||
BindingHostID string `json:"binding:host_id"`
|
||||
Name string `json:"name"`
|
||||
AllowedAddressPairs []interface{} `json:"allowed_address_pairs"`
|
||||
AdminStateUp bool `json:"admin_state_up"`
|
||||
NetworkID string `json:"network_id"`
|
||||
TenantID string `json:"tenant_id"`
|
||||
BindingVifDetails struct {
|
||||
PortFilter bool `json:"port_filter"`
|
||||
OvsHybridPlug bool `json:"ovs_hybrid_plug"`
|
||||
} `json:"binding:vif_details"`
|
||||
BindingVnicType string `json:"binding:vnic_type"`
|
||||
BindingVifType string `json:"binding:vif_type"`
|
||||
DeviceOwner string `json:"device_owner"`
|
||||
MacAddress string `json:"mac_address"`
|
||||
BindingProfile struct {
|
||||
} `json:"binding:profile"`
|
||||
FixedIps []struct {
|
||||
SubnetID string `json:"subnet_id"`
|
||||
IPAddress string `json:"ip_address"`
|
||||
} `json:"fixed_ips"`
|
||||
ID string `json:"id"`
|
||||
SecurityGroups []string `json:"security_groups"`
|
||||
DeviceID string `json:"device_id"`
|
||||
}
|
||||
81
syslog.go
Normal file
81
syslog.go
Normal file
@ -0,0 +1,81 @@
|
||||
package main
|
||||
|
||||
/*
|
||||
|
||||
syslog - This file includes all of the logic necessary to interact with syslog.
|
||||
This is extrapolated out so that a SyslogActioner interface can be
|
||||
passed to functions. Doing this allows testing by mock classes to be created
|
||||
that can be passed to functions.
|
||||
|
||||
Since this is a wrapper around the log/syslog library, this does not need
|
||||
testing.
|
||||
|
||||
*/
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"log/syslog"
|
||||
"net"
|
||||
)
|
||||
|
||||
// SyslogActioner is an interface for an SyslogActions class. Having
|
||||
// this as an interface allows us to pass in a dummy class for testing that
|
||||
// just returns mocked data.
|
||||
type SyslogActioner interface {
|
||||
Connect() error
|
||||
Info(string)
|
||||
}
|
||||
|
||||
// SyslogActions is a class that handles all interactions directly with Syslog.
|
||||
// See the comment on SyslogActioner for rationale.
|
||||
type SyslogActions struct {
|
||||
logger *syslog.Writer
|
||||
Options SyslogOptions
|
||||
}
|
||||
|
||||
// SyslogOptions is a class to convey all of the configurable options for the
|
||||
// SyslogActions class.
|
||||
type SyslogOptions struct {
|
||||
Host string
|
||||
Protocol string
|
||||
Retry bool
|
||||
Port string
|
||||
}
|
||||
|
||||
// Info is the main method for the SyslogActioner class, it writes an
|
||||
// info-level message to the syslog stream.
|
||||
func (s SyslogActions) Info(writeMe string) {
|
||||
log.Println("Logged: ", writeMe)
|
||||
s.logger.Info(writeMe)
|
||||
}
|
||||
|
||||
// Connect is the method that establishes the connection to the syslog server
|
||||
// over the network.
|
||||
func (s *SyslogActions) Connect() error {
|
||||
var err error
|
||||
|
||||
address := net.JoinHostPort(s.Options.Host, s.Options.Port)
|
||||
|
||||
if Debug {
|
||||
log.Printf("Opening %q syslog socket to %q\n", s.Options.Protocol, s.Options.Host)
|
||||
}
|
||||
|
||||
s.logger, err = syslog.Dial(s.Options.Protocol, address, syslog.LOG_INFO, "osel")
|
||||
if err != nil {
|
||||
log.Printf("error opening syslog socket to %s: %s\n", s.Options.Host, err)
|
||||
if s.Options.Retry {
|
||||
for err != nil {
|
||||
log.Println("retrying")
|
||||
s.logger, err = syslog.Dial(s.Options.Protocol, address, syslog.LOG_INFO, "osel")
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("error opening syslog socket to %s: %s", s.Options.Host, err)
|
||||
}
|
||||
|
||||
if Debug {
|
||||
log.Println("Successfully connected to syslog host", s.Options.Host)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
24
syslog_mock_test.go
Normal file
24
syslog_mock_test.go
Normal file
@ -0,0 +1,24 @@
|
||||
package main
|
||||
|
||||
import "log"
|
||||
|
||||
type SyslogTestActions struct {
|
||||
savedLogs []string
|
||||
}
|
||||
|
||||
func (s *SyslogTestActions) Connect() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *SyslogTestActions) Info(writeMe string) {
|
||||
log.Printf("FAKE SYSLOG LINE: %s\n", writeMe)
|
||||
s.savedLogs = append(s.savedLogs, writeMe)
|
||||
}
|
||||
|
||||
func (s *SyslogTestActions) GetLogs() []string {
|
||||
return s.savedLogs
|
||||
}
|
||||
|
||||
func connectFakeSyslog() *SyslogTestActions {
|
||||
return new(SyslogTestActions)
|
||||
}
|
||||
59
tools/test-setup.sh
Executable file
59
tools/test-setup.sh
Executable file
@ -0,0 +1,59 @@
|
||||
#!/bin/bash
|
||||
|
||||
# You cannot go build-time dependency fetching from projects hosted on github
|
||||
# without a github token, otherwise you get restricted by API throttling.
|
||||
# See: https://github.com/golang/go/issues/23955
|
||||
echo "machine api.github.com login openstackzuul password dba1634cb701f1c514f3268784b1d0a6512c12d4" >> $HOME/.netrc
|
||||
mkdir -p /home/zuul/go/src/v 2>/dev/null
|
||||
|
||||
# Setup the environment prior to testing.
|
||||
export PATH=$PATH:$GOPATH/bin
|
||||
|
||||
# Get OS
|
||||
case $(uname -s) in
|
||||
Darwin)
|
||||
OS=darwin
|
||||
;;
|
||||
Linux)
|
||||
if LSB_RELEASE=$(which lsb_release); then
|
||||
OS=$($LSB_RELEASE -s -c)
|
||||
else
|
||||
# No lsb-release, trya hack or two
|
||||
if which dpkg 1>/dev/null; then
|
||||
OS=debian
|
||||
elif which yum 1>/dev/null || which dnf 1>/dev/null; then
|
||||
OS=redhat
|
||||
else
|
||||
echo "Linux distro not yet supported"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "Unsupported OS"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
echo "Depected OS is '$OS'"
|
||||
|
||||
echo | sudo -S /bin/true 2>/dev/null
|
||||
if [ $? != 0 ]; then
|
||||
echo "Sudo does not work, so packages can not be installed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Now install go
|
||||
case $OS in
|
||||
xenial)
|
||||
sudo add-apt-repository ppa:longsleep/golang-backports
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y golang-go golint
|
||||
;;
|
||||
esac
|
||||
|
||||
# Install vgo https://github.com/golang/go/wiki/vgo
|
||||
if which go 1>/dev/null; then
|
||||
sudo go get -u -v golang.org/x/vgo
|
||||
else
|
||||
echo "go not found, install golang from source?"
|
||||
fi
|
||||
55
viper.go
Normal file
55
viper.go
Normal file
@ -0,0 +1,55 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/nate-johnston/viper"
|
||||
)
|
||||
|
||||
// ViperConfig holds information per config item. If an Default
|
||||
// is not set then it is assumed that the value is Required.
|
||||
// NOTE: You can not set a default on a nested value. i.e. a value
|
||||
// within a has in a json or yaml file. (nested.value) you can
|
||||
// set nested values as required.
|
||||
type ViperConfig struct {
|
||||
Key string // The config key that is required.
|
||||
Default interface{} // Default Value to set.
|
||||
Alias []string // Any key Aliases that should be registered
|
||||
Description string // Description of the config.
|
||||
}
|
||||
|
||||
// InitViper with the passed path and config.
|
||||
func InitViper(path string, viperConfigs []ViperConfig) error {
|
||||
viper.SetConfigFile(path)
|
||||
if err := viper.ReadInConfig(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ValidateConfig(viperConfigs); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ValidateConfig will check the defined var ViperConfigs []ViperConfig and validate
|
||||
// the existances of the required keys, and set defaults for all keys where defaults are
|
||||
// defined.
|
||||
func ValidateConfig(viperConfigs []ViperConfig) error {
|
||||
var errs []error
|
||||
for _, rc := range viperConfigs {
|
||||
if rc.Default == nil && viper.Get(rc.Key) == nil {
|
||||
errs = append(errs, fmt.Errorf("Key: %s, Description: %s", rc.Key, rc.Description))
|
||||
} else {
|
||||
viper.SetDefault(rc.Key, rc.Default)
|
||||
}
|
||||
|
||||
if len(rc.Alias) > 0 {
|
||||
for _, a := range rc.Alias {
|
||||
viper.RegisterAlias(a, rc.Key)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(errs) > 0 {
|
||||
return fmt.Errorf("Required Configuration Missing: %v", errs)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
42
viper_test.go
Normal file
42
viper_test.go
Normal file
@ -0,0 +1,42 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/nate-johnston/viper"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInitViperWillReturnErrorOnReadError(t *testing.T) {
|
||||
err := InitViper("fixtures/viper/not_there.yml", nil)
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, "open fixtures/viper/not_there.yml: no such file or directory", err.Error())
|
||||
}
|
||||
|
||||
func TestInitViperEnsureItCallsValidateConfigWithConfigErrors(t *testing.T) {
|
||||
// By calling InitViper and expecting an error case we prove that IniViper is
|
||||
// calling ValidateConfig() and its working as expected.
|
||||
viperConfigs := []ViperConfig{
|
||||
ViperConfig{Key: "missing_required_string", Description: "Required String"},
|
||||
}
|
||||
err := InitViper("fixtures/viper/test.yml", viperConfigs)
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, "Required Configuration Missing: [Key: missing_required_string, Description: Required String]",
|
||||
err.Error())
|
||||
}
|
||||
|
||||
func TestValidateConfig(t *testing.T) {
|
||||
viperConfigs := []ViperConfig{
|
||||
ViperConfig{Key: "required_string", Description: "Required String"},
|
||||
ViperConfig{Key: "nested.one", Description: "Required Nested One"},
|
||||
ViperConfig{Key: "test_default", Default: "Optional Value", Description: "Optional Value"},
|
||||
ViperConfig{Key: "test_alias", Alias: []string{"bubba", "forest"}, Description: "Optional Value"},
|
||||
}
|
||||
InitViper("fixtures/viper/test.yml", viperConfigs)
|
||||
|
||||
err := ValidateConfig(viperConfigs)
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, "Optional Value", viper.GetString("test_default"))
|
||||
assert.Equal(t, "test_alias value", viper.GetString("test_alias"))
|
||||
assert.Equal(t, "test_alias value", viper.GetString("bubba"))
|
||||
assert.Equal(t, "test_alias value", viper.GetString("forest"))
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user