openstack/trove
Code Issues Proposed changes
trove/doc/source/index.rst
Petr Malik 4899f49401 Implement dangling mock detector for unittests 
The Issue:

Dangling mock objects in global modules (mocked members of imported
modules that never get restored) have been causing various transient
failures in the unit test suite.

The main issues posed by dangling mock objects include:
- Such object references propagate across the entire test suite. Any
caller may be hit by a non-functional or worse crippled module member
because some other (potentially totally unrelated) test case failed to
restore it.
- Dangling mock references shared across different test modules may
lead to unexpected results/behavior in multi-threaded environments. One
example could be a test case failing because a mock got called multiple
times from unrelated modules.

Such issues are likely to exhibit transient random behavior depending
on the runtime environment making them difficult to debug.

This contribution is aiming to provide a simple transparent detection
layer that would help us prevent (most of) such issues.

Solution Strategy:

We extend the 'testtools.TestCase' class currently used as the base
class of Trove unit tests with functions that attempts to detect leaked
mock objects in imported modules. The new 'trove_testtools.TestCase'
implementation basically retrieves all loaded modules and scans their
members for mock objects before and after each test case.

An outline of the procedure follows:
1. Override the setUp() method and add a cleanup call (executed after
the tearDown()) to collect mock references before and after a test case.
2. Subtract the two sets after each test case and mark it as failed if
there are any new mock references left over.

Code Impact:

The existing test classes replace 'testtools.TestCase' base class with
'trove_testtools.TestCase'.

Documentation Impact:

Added a short document on recommended mocking strategies in unit tests
to the Trove Developer Documentation.

Known Limitations:

The current implementation has a configurable recursion depth which is
the number of nested levels to examine when searching for mocks.
Higher setting will potentially uncover more dangling objects,
at the cost of increased scanning time.
We set it to 2 by default to get better coverage.
This setting will increase test runtime.

Recommended Mocking Patterns:

Mock Guide: https://docs.python.org/3/library/unittest.mock.html

- Mocking a class or object shared across multiple test cases.
  Use the patcher pattern in conjunction with the setUp() and tearDown()
  methods [ see section 26.4.3.5. of Mock Guide ].

def setUp(self):
    super(CouchbaseBackupTests, self).setUp()
    self.exe_timeout_patch = patch.object(utils, 'execute_with_timeout')

def test_case(self):
    # This line can be moved to the setUp() method if the mock object
    # is not needed.
    mock_object = self.exe_timeout_patch.start()

def tearDown(self):
    super(CouchbaseBackupTests, self).tearDown()
    self.exe_timeout_patch.stop()

Note also: patch.stopall()
Stop all active patches. Only stops patches started with start.

- Mocking a class or object for a single entire test case.
  Use the decorator pattern.

@patch.object(utils, 'execute_with_timeout')
@patch.object(os, 'popen')
def test_case(self, popen_mock, execute_with_timeout_mock):
    pass

@patch.multiple(utils, execute_with_timeout=DEFAULT,
                generate_random_password=MagicMock(return_value=1))
def test_case(self, generate_random_password, execute_with_timeout):
    pass

- Mocking a class or object for a smaller scope within one test case.
  Use the context manager pattern.

def test_case(self):
    # Some code using real implementation of 'generate_random_password'.
    with patch.object(utils, 'generate_random_password') as pwd_mock:
        # Using the mocked implementation of 'generate_random_password'.
    # Again code using the actual implementation of the method.

def test_case(self):
    with patch.multiple(utils, execute_with_timeout_mock=DEFAULT,
                        generate_random_password=MagicMock(
                                return_value=1)) as mocks:
        password_mock = mocks['generate_random_password']
        execute_mock = mocks['execute_with_timeout_mock']

Change-Id: Ia487fada249aa903410a1a3fb3f717d6e0d581d2
Closes-Bug: 1447833
2015-05-04 17:53:53 -04:00

2.4 KiB
Raw Blame History

Welcome to Trove's developer documentation!

Introduction

Trove is Database as a Service for OpenStack. It's designed to run entirely on OpenStack, with the goal of allowing users to quickly and easily utilize the features of a relational database without the burden of handling complex administrative tasks. Cloud users and database administrators can provision and manage multiple database instances as needed.

Initially, the service will focus on providing resource isolation at high performance while automating complex administrative tasks including deployment, configuration, patching, backups, restores, and monitoring.

For an in-depth look at the project's design and structure, see the dev/design page.

Installation And Deployment

Trove is constantly under development. The easiest way to install Trove is using the Trove integration scripts that can be found in git in the Trove Integration Repository.

For further details on how to install Trove using the integration scripts please refer to the dev/install page.

For further details on how to install Trove to work with existing OpenStack environment please refer to the dev/manual_install page.

Developer Resources

For those wishing to develop Trove itself, or to extend Trove's functionality, the following resources are provided.

dev/design dev/testing dev/install dev/manual_install.rst dev/building_guest_images.rst

Guest Images

In order to use Trove, you need to have Guest Images for each datastore and version. These images are loaded into Glance and registered with Trove.

For those wishing to develop guest images, please refer to the dev/building_guest_images.rst page.

Search Trove Documentation

  • search