Create Custom Plugin

This example shows how to create a custom plugin that counts the number of passing and failing assertions when running a specified test suite. The plugin prints a brief summary at the end of the testing.

Create AssertionCountingPlugin Class
Extend Running of TestSuite
Extend Creation of Shared Test Fixtures and TestCase Instances
Extend Running of Single Test Suite Element
Define Helper Functions
AssertionCountingPlugin Class Definition Summary
Create Example Test Class
Add Plugin to TestRunner and Run Tests

Create AssertionCountingPlugin Class

In a file in your working folder, create a new class, AssertionCountingPlugin, that inherits from the matlab.unittest.plugins.TestRunnerPlugin class. For a complete version of the code for an AssertionCountingPlugin, see "AssertionCountingPlugin Class Definition Summary".

Keep track of the number of passing and failing assertions. Within a properties block, create NumPassingAssertions and NumFailingAssertions properties to pass the data between methods.

properties
    NumPassingAssertions = 0;
    NumFailingAssertions = 0;
end

Extend Running of TestSuite

Implement the runTestSuite method in a methods block with protected access.

methods (Access = protected)
    function runTestSuite(plugin, pluginData)
        suiteSize = numel(pluginData.TestSuite);
        fprintf('## Running a total of %d tests\n', suiteSize)

        plugin.NumPassingAssertions = 0;
        plugin.NumFailingAssertions = 0;

        runTestSuite@matlab.unittest.plugins.TestRunnerPlugin(...
            plugin, pluginData);

        fprintf('## Done running tests\n')
        plugin.printAssertionSummary()
    end
end

The test framework evaluates this method one time. It displays information about the total number of tests, initializes the assertion count, and invokes the superclass method. After the framework completes evaluating the superclass method, the runTestSuite method displays the assertion count summary.

Extend Creation of Shared Test Fixtures and TestCase Instances

Add listeners to AssertionPassed and AssertionFailed events to count the assertions. To add these listeners, extend the methods that the test framework uses to create the test content. The test content comprises TestCase instances for each Test element, class-level TestCase instances for the TestClassSetup and TestClassTeardown methods, and Fixture instances that are used when a TestCase class has the SharedTestFixtures attribute.

Invoke the corresponding superclass method when you override the creation methods. The creation methods return the content that the test framework creates for each of their respective contexts. When implementing one of these methods, pass this argument out of your own implementation, and add the listeners required by this plugin.

Add these creation methods to a methods block with protected access.

methods (Access = protected)
    function fixture = createSharedTestFixture(plugin, pluginData)
        fixture = createSharedTestFixture@...
            matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);

        fixture.addlistener('AssertionPassed', ...
            @(~,~)plugin.incrementPassingAssertionsCount);
        fixture.addlistener('AssertionFailed', ...
            @(~,~)plugin.incrementFailingAssertionsCount);
    end

    function testCase = createTestClassInstance(plugin, pluginData)
        testCase = createTestClassInstance@...
            matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);

        testCase.addlistener('AssertionPassed', ...
            @(~,~)plugin.incrementPassingAssertionsCount);
        testCase.addlistener('AssertionFailed', ...
            @(~,~)plugin.incrementFailingAssertionsCount);
    end

    function testCase = createTestMethodInstance(plugin, pluginData)
        testCase = createTestMethodInstance@...
            matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);

        testCase.addlistener('AssertionPassed', ...
            @(~,~)plugin.incrementPassingAssertionsCount);
        testCase.addlistener('AssertionFailed', ...
            @(~,~)plugin.incrementFailingAssertionsCount);
    end
end

Extend Running of Single Test Suite Element

Extend runTest to display the name of each test at run time. Include this function in a methods block with protected access. Like all plugin methods, when you override this method you must invoke the corresponding superclass method.

methods (Access = protected)
    function runTest(plugin, pluginData)
        fprintf('### Running test: %s\n', pluginData.Name)

        runTest@matlab.unittest.plugins.TestRunnerPlugin(...
            plugin, pluginData);
    end
end

Define Helper Functions

In a methods block with private access, define three helper functions. These functions increment the number of passing or failing assertions, and print out the assertion count summary.

methods (Access = private)
    function incrementPassingAssertionsCount(plugin)
        plugin.NumPassingAssertions = plugin.NumPassingAssertions + 1;
        end

        function incrementFailingAssertionsCount(plugin)
        plugin.NumFailingAssertions = plugin.NumFailingAssertions + 1;
    end

    function printAssertionSummary(plugin)
        fprintf('%s\n', repmat('_', 1, 30))
        fprintf('Total Assertions: %d\n', ...
            plugin.NumPassingAssertions + plugin.NumFailingAssertions)
        fprintf('\t%d Passed, %d Failed\n', ...
            plugin.NumPassingAssertions, plugin.NumFailingAssertions)
    end
end

AssertionCountingPlugin Class Definition Summary

classdef AssertionCountingPlugin < ...
        matlab.unittest.plugins.TestRunnerPlugin
    
    properties
        NumPassingAssertions = 0;
        NumFailingAssertions = 0;
    end
    
    methods (Access = protected)
        function runTestSuite(plugin, pluginData)
            suiteSize = numel(pluginData.TestSuite);
            fprintf('## Running a total of %d tests\n', suiteSize)
            
            plugin.NumPassingAssertions = 0;
            plugin.NumFailingAssertions = 0;
            
            runTestSuite@matlab.unittest.plugins.TestRunnerPlugin(...
                plugin, pluginData);
            
            fprintf('## Done running tests\n')
            plugin.printAssertionSummary()
        end
        
        function fixture = createSharedTestFixture(plugin, pluginData)
            fixture = createSharedTestFixture@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            
            fixture.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            fixture.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
        
        function testCase = createTestClassInstance(plugin, pluginData)
            testCase = createTestClassInstance@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            
            testCase.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            testCase.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
        
        function testCase = createTestMethodInstance(plugin, pluginData)
            testCase = createTestMethodInstance@...
                matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            
            testCase.addlistener('AssertionPassed', ...
                @(~,~)plugin.incrementPassingAssertionsCount);
            testCase.addlistener('AssertionFailed', ...
                @(~,~)plugin.incrementFailingAssertionsCount);
        end
        
        function runTest(plugin, pluginData)
            fprintf('### Running test: %s\n', pluginData.Name)
            
            runTest@matlab.unittest.plugins.TestRunnerPlugin(...
                plugin, pluginData);
        end 
    end
    
    methods (Access = private)
        function incrementPassingAssertionsCount(plugin)
            plugin.NumPassingAssertions = plugin.NumPassingAssertions + 1;
        end
        
        function incrementFailingAssertionsCount(plugin)
            plugin.NumFailingAssertions = plugin.NumFailingAssertions + 1;
        end
        
        function printAssertionSummary(plugin)
            fprintf('%s\n', repmat('_', 1, 30))
            fprintf('Total Assertions: %d\n', ...
                plugin.NumPassingAssertions + plugin.NumFailingAssertions)
            fprintf('\t%d Passed, %d Failed\n', ...
                plugin.NumPassingAssertions, plugin.NumFailingAssertions)
        end
    end
end

Create Example Test Class

In your working folder, create the file ExampleTest.m containing the following test class.

classdef ExampleTest < matlab.unittest.TestCase
    methods(Test)
        function testOne(testCase)  % Test fails
            testCase.assertEqual(5, 4)
        end
        function testTwo(testCase)  % Test passes
            testCase.verifyEqual(5, 5)
        end
        function testThree(testCase)  % Test passes
            testCase.assertEqual(7*2, 14)
        end
    end
end

Add Plugin to TestRunner and Run Tests

At the command prompt, create a test suite from the ExampleTest class.

import matlab.unittest.TestSuite
import matlab.unittest.TestRunner

suite = TestSuite.fromClass(?ExampleTest);

Create a test runner with no plugins. This code creates a silent runner and provides you with complete control over the installed plugins.

runner = TestRunner.withNoPlugins;

Run the tests.

result = runner.run(suite);

Add AssertionCountingPlugin to the runner and run the tests.

runner.addPlugin(AssertionCountingPlugin)
result = runner.run(suite);

## Running a total of 3 tests
### Running test: ExampleTest/testOne
### Running test: ExampleTest/testTwo
### Running test: ExampleTest/testThree
## Done running tests
______________________________
Total Assertions: 2
	1 Passed, 1 Failed

Documentation