diff --git a/doc/source/test_plans/neutron_features/index.rst b/doc/source/test_plans/neutron_features/index.rst index 905d2e4..346346b 100644 --- a/doc/source/test_plans/neutron_features/index.rst +++ b/doc/source/test_plans/neutron_features/index.rst @@ -9,4 +9,5 @@ Neutron features test plans .. toctree:: :maxdepth: 3 - l3_ha/plan \ No newline at end of file + l3_ha/plan + resource_density/plan diff --git a/doc/source/test_plans/neutron_features/resource_density/plan.rst b/doc/source/test_plans/neutron_features/resource_density/plan.rst new file mode 100644 index 0000000..9443c08 --- /dev/null +++ b/doc/source/test_plans/neutron_features/resource_density/plan.rst @@ -0,0 +1,141 @@ +.. _neutron_resource_density_test_plan: + +============================================ +OpenStack Neutron Resource Density Test Plan +============================================ + +:status: **draft** +:version: 1.0 + +:Abstract: + Is data-plane performance affected by existence of other OpenStack resources? + + + +Test Plan +========= + +The goal of this test plan is to investigate whether existing OpenStack +resources affect data-plane performance. + +Out of all resources the following may theoretically affect performance: + * instances running on the same compute host - because of CPU consumption, + additional network namespaces, iptables, OVS ports and flows; + * routers - because of Linux network and OVS resources; + * security groups and rules - because of iptables rules. + + +Test Environment +---------------- + +Preparation +^^^^^^^^^^^ + +This test plan is performed against existing OpenStack cloud. + + +Environment description +^^^^^^^^^^^^^^^^^^^^^^^ + +The environment description includes hardware specification of servers, +network parameters, operation system and OpenStack deployment characteristics. + +Hardware +~~~~~~~~ + +This section contains list of all types of hardware nodes. + ++-----------+-------+----------------------------------------------------+ +| Parameter | Value | Comments | ++-----------+-------+----------------------------------------------------+ +| model | | e.g. Supermicro X9SRD-F | ++-----------+-------+----------------------------------------------------+ +| CPU | | e.g. 6 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz | ++-----------+-------+----------------------------------------------------+ +| role | | e.g. compute or network | ++-----------+-------+----------------------------------------------------+ + +Network +~~~~~~~ + +This section contains list of interfaces and network parameters. +For complicated cases this section may include topology diagram and switch +parameters. + ++------------------+-------+-------------------------+ +| Parameter | Value | Comments | ++------------------+-------+-------------------------+ +| network role | | e.g. provider or public | ++------------------+-------+-------------------------+ +| card model | | e.g. Intel | ++------------------+-------+-------------------------+ +| driver | | e.g. ixgbe | ++------------------+-------+-------------------------+ +| speed | | e.g. 10G or 1G | ++------------------+-------+-------------------------+ +| MTU | | e.g. 9000 | ++------------------+-------+-------------------------+ +| offloading modes | | e.g. default | ++------------------+-------+-------------------------+ + +Software +~~~~~~~~ + +This section describes installed software. + ++-----------------+-------+---------------------------+ +| Parameter | Value | Comments | ++-----------------+-------+---------------------------+ +| OS | | e.g. Ubuntu 14.04.3 | ++-----------------+-------+---------------------------+ +| OpenStack | | e.g. Liberty | ++-----------------+-------+---------------------------+ +| Hypervisor | | e.g. KVM | ++-----------------+-------+---------------------------+ +| Neutron plugin | | e.g. ML2 + OVS | ++-----------------+-------+---------------------------+ +| L2 segmentation | | e.g. VLAN or VxLAN or GRE | ++-----------------+-------+---------------------------+ +| virtual routers | | HA | ++-----------------+-------+---------------------------+ + +Test Case: Data-plane performance measurements +---------------------------------------------- + +Description +^^^^^^^^^^^ + +Measurements are performed between 2 instances running on different compute +nodes. One of instances has floating IP assigned and thus is reachable from +outside. We are interested in following metrics: TCP max throughput and +UDP top packets throughput (for 64-byte packets). + +TCP throughput is measured with `flent`_:: + + flent -H -f stats tcp_download + +UDP throughput is measured with `iperf3`_:: + + iperf3 -c -u -l 64 -b 0 -t 20 + +List of performance metrics +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +======== =============== ================= ====================================== +Priority Value Measurement Units Description +======== =============== ================= ====================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size +======== =============== ================= ====================================== + +Reports +======= + +Test plan execution reports: + * :ref:`neutron_neutron_resource_test_report` + +.. references: + +.. _flent: http://flent.org/ +.. _iperf3: http://iperf.fr/ diff --git a/doc/source/test_results/neutron_features/index.rst b/doc/source/test_results/neutron_features/index.rst index b4bec7a..9b0e38d 100644 --- a/doc/source/test_results/neutron_features/index.rst +++ b/doc/source/test_results/neutron_features/index.rst @@ -11,4 +11,4 @@ Neutron features scale testing l3_ha/test_results_liberty l3_ha/test_results_mitaka - + resource_density/index diff --git a/doc/source/test_results/neutron_features/resource_density/chart_latency.png b/doc/source/test_results/neutron_features/resource_density/chart_latency.png new file mode 100644 index 0000000..a491792 Binary files /dev/null and b/doc/source/test_results/neutron_features/resource_density/chart_latency.png differ diff --git a/doc/source/test_results/neutron_features/resource_density/chart_tcp.png b/doc/source/test_results/neutron_features/resource_density/chart_tcp.png new file mode 100644 index 0000000..0c5bdf7 Binary files /dev/null and b/doc/source/test_results/neutron_features/resource_density/chart_tcp.png differ diff --git a/doc/source/test_results/neutron_features/resource_density/chart_udp.png b/doc/source/test_results/neutron_features/resource_density/chart_udp.png new file mode 100644 index 0000000..f4f6a3f Binary files /dev/null and b/doc/source/test_results/neutron_features/resource_density/chart_udp.png differ diff --git a/doc/source/test_results/neutron_features/resource_density/index.rst b/doc/source/test_results/neutron_features/resource_density/index.rst new file mode 100644 index 0000000..b9b9892 --- /dev/null +++ b/doc/source/test_results/neutron_features/resource_density/index.rst @@ -0,0 +1,216 @@ +.. _neutron_neutron_resource_test_report: + +============================================== +OpenStack Neutron Resource Density Test Report +============================================== + +This report is generated for :ref:`neutron_resource_density_test_plan`. + +Environment description +======================= + +Cluster description +------------------- +* 3 controllers +* 2 compute nodes + +Software versions +----------------- + +**OpenStack/System**: + Fuel/MOS 9.0, Ubuntu 14.04, Linux kernel 3.13, OVS 2.4.1 +**Networking** + Neutron ML2 + OVS plugin, DVR, L2pop, MTU 1500 + +Hardware configuration of each server +------------------------------------- + +Description of servers hardware + +**Compute Vendor**: + HP ProLiant DL380 Gen9, +**CPU** + 2 x Intel(R) Xeon(R) CPU E5-2680 v3 @2.50GHz (48 cores) +**RAM**: + 256 Gb +**NIC** + 2 x Intel Corporation Ethernet 10G 2P X710 + + +Report +====== + +Preparation +----------- + +Measurements are performed between 2 instances deployed in L3 east-west +topology. To prepare: + #. Create new Neutron router. Plug into external network. + #. Create 2 Neutron networks, e.g. `east` and `west`. Plug them into + the router. + #. Boot 2 instances (Ubuntu Cloud Image can be used). Each instance should be + in its own network and on its own compute node. Public key may be required + to login into instance (in case of using Ubuntu). + #. Assign floating IP to one of instances. + +The traffic is sent between 2 instances, from the one with floating IP to the +other. Phisically packets will go from one compute node to the other directly. + +Measurements +------------ + +Baseline +^^^^^^^^ + +We start with measuring baseline numbers. During this test only 2 instances +exist. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 1.8 [0.6] 7445 [850] 132 [6.5] + ================= ============================= ========================== + + +10 instances per compute in separate network +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 20 micro instances (10 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Instances are plugged into separate +network. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.0 [0.7] 6500 [807] 106 [4.5] + ================= ============================= ========================== + +30 instances per compute in separate network +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 60 micro instances (30 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Instances are plugged into separate +network. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.2 [0.8] 6650 [600] 88 [3.0] + ================= ============================= ========================== + +10 instances per compute in tester network +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 20 micro instances (10 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Instances are plugged into `east` +network. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.7 [1.0] 4930 [499] 101 [3.6] + ================= ============================= ========================== + +30 instances per compute in tester network +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 60 micro instances (30 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Instances are plugged into `east` +network. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 5.4 [2.1] 2470 [130] 88 [2.3] + ================= ============================= ========================== + +10 instances and 10 routers per compute +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 20 micro instances (10 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Each instance is plugged into its own +network, which in turn is plugged into its own router. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.3 [0.8] 5690 [630] 107 [5.0] + ================= ============================= ========================== + +30 instances and 30 routers per compute +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 60 micro instances (30 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Each instance is plugged into its own +network, which in turn is plugged into its own router. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.2 [0.8] 5890 [707] 85 [1.9] + ================= ============================= ========================== + +30 instances and 30 routers per compute, QEMU paused +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 60 micro instances (30 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Each instance is plugged into its own +network, which in turn is plugged into its own router. QEMU processes are +paused to reduce noise load on CPU. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.0 [0.7] 6640 [623] 103 [4.6] + ================= ============================= ========================== + +30 instances, 30 routers and 500 SG rules per instance, QEMU paused +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are 60 micro instances (30 per compute node). Each instance is +Cirros-based with 1 VCPU and 64MB RAM. Each instance is plugged into its own +network, which in turn is plugged into its own router. QEMU processes are +paused to reduce noise load on CPU. All instances have security group which +contains 500 rules (this results in ~17k iptables rules) + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 2.0 [0.7] 6658 [576] 97 [1.0] + ================= ============================= ========================== + +1000 security group rules +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In this case we assign security group containing 1000 rules to test instances. +There is no measurable difference from the baseline data. + + ================= ============================= ========================== + Latency, ms [σ] TCP throughput, Mbits/s [σ] UDP throughput, kpps [σ] + ================= ============================= ========================== + 1.8 [0.6] 7039 [653] 130 [2.0] + ================= ============================= ========================== + +Conclusion +^^^^^^^^^^ + +It appears that two things have most of impact: + * neighbours, which produce slight CPU load + * OVS flows: the more flows, the less performance. Adding 30 instances to + compute node results in 180 new flows. + +.. image:: chart_latency.png + +The worst case for latency is when instances are co-located in the same network +as the tester. While presence of security group rules have no measurable effect +at all. + +.. image:: chart_tcp.png + +The worst case again when instances are co-located in the same network. CPU +noise also affects - the throughput reduces on ~10%. + +.. image:: chart_udp.png + +For UDP metric the most impact is introduced by CPU noise and presence of OVS +rules. With security group rules the numbers are the same as in the baseline. diff --git a/doc/source/test_results/neutron_features/resource_density/router_vm.hot b/doc/source/test_results/neutron_features/resource_density/router_vm.hot new file mode 100644 index 0000000..b6a8f7a --- /dev/null +++ b/doc/source/test_results/neutron_features/resource_density/router_vm.hot @@ -0,0 +1,41 @@ +heat_template_version: 2013-05-23 + +resources: + east_private_net: + type: OS::Neutron::Net + properties: + name: my_net_east + + east_private_subnet: + type: OS::Neutron::Subnet + properties: + network_id: { get_resource: east_private_net } + cidr: 10.1.0.0/16 + + router: + type: OS::Neutron::Router + properties: + external_gateway_info: + network: admin_floating_net + + router_interface: + type: OS::Neutron::RouterInterface + properties: + router_id: { get_resource: router } + subnet_id: { get_resource: east_private_subnet } + + server_1: + type: OS::Nova::Server + properties: + name: server_1 + image: TestVM + flavor: m1.micro + networks: + - port: { get_resource: port_1 } + + port_1: + type: OS::Neutron::Port + properties: + network_id: { get_resource: east_private_net } + fixed_ips: + - subnet_id: { get_resource: east_private_subnet }