Testing extensions

CKAN extensions can have their own tests that are run using pytest in much the same way as running CKAN’s own tests (see Testing CKAN).

Continuing with our example_iauthfunctions extension, first we need a CKAN config file to be used when running our tests. Create the file ckanext-iauthfunctions/test.ini with the following contents:

[app:main]
use = config:../ckan/test-core.ini

The use line declares that this config file inherits the settings from the config file used to run CKAN’s own tests (../ckan should be the path to your CKAN source directory, relative to your test.ini file).

The test.ini file is a CKAN config file just like your /etc/ckan/default/ckan.ini file, and it can contain any CKAN config file settings that you want CKAN to use when running your tests, for example:

[app:main]
use = config:../ckan/test-core.ini
ckan.site_title = My Test CKAN Site
ckan.site_description = A test site for testing my CKAN extension

Next, make the directory that will contain our test modules:

mkdir ckanext-iauthfunctions/ckanext/iauthfunctions/tests/

Finally, create the file ckanext-iauthfunctions/ckanext/iauthfunctions/tests/test_iauthfunctions.py with the following contents:

# encoding: utf-8
'''Tests for the ckanext.example_iauthfunctions extension.

'''
import pytest

import ckan.logic as logic
import ckan.tests.factories as factories
import ckan.tests.helpers as helpers
from ckan.plugins.toolkit import NotAuthorized, ObjectNotFound


@pytest.mark.ckan_config('ckan.plugins',
                         'example_iauthfunctions_v6_parent_auth_functions')
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
class TestAuthV6(object):
    def test_resource_delete_editor(self):
        '''Normally organization admins can delete resources
        Our plugin prevents this by blocking delete organization.

        Ensure the delete button is not displayed (as only resource delete
        is checked for showing this)

        '''
        user = factories.User()
        owner_org = factories.Organization(users=[{
            'name': user['id'],
            'capacity': 'admin'
        }])
        dataset = factories.Dataset(owner_org=owner_org['id'])
        resource = factories.Resource(package_id=dataset['id'])
        with pytest.raises(logic.NotAuthorized) as e:
            logic.check_access('resource_delete', {'user': user['name']},
                               {'id': resource['id']})

        assert e.value.message == 'User %s not authorized to delete resource %s' % (
            user['name'], resource['id'])

    def test_resource_delete_sysadmin(self):
        '''Normally organization admins can delete resources
        Our plugin prevents this by blocking delete organization.

        Ensure the delete button is not displayed (as only resource delete
        is checked for showing this)

        '''
        user = factories.Sysadmin()
        owner_org = factories.Organization(users=[{
            'name': user['id'],
            'capacity': 'admin'
        }])
        dataset = factories.Dataset(owner_org=owner_org['id'])
        resource = factories.Resource(package_id=dataset['id'])
        assert logic.check_access('resource_delete', {'user': user['name']},
                                  {'id': resource['id']})


@pytest.mark.ckan_config('ckan.plugins',
                         'example_iauthfunctions_v5_custom_config_setting')
@pytest.mark.ckan_config('ckan.iauthfunctions.users_can_create_groups', False)
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
class TestAuthV5(object):

    def test_sysadmin_can_create_group_when_config_is_false(self):
        sysadmin = factories.Sysadmin()
        context = {'ignore_auth': False, 'user': sysadmin['name']}
        helpers.call_action('group_create', context, name='test-group')

    def test_user_cannot_create_group_when_config_is_false(self):
        user = factories.User()
        context = {'ignore_auth': False, 'user': user['name']}
        with pytest.raises(NotAuthorized):
            helpers.call_action('group_create', context, name='test-group')

    def test_visitor_cannot_create_group_when_config_is_false(self):
        context = {'ignore_auth': False, 'user': None}
        with pytest.raises(NotAuthorized):
            helpers.call_action('group_create', context, name='test-group')


@pytest.mark.ckan_config('ckan.plugins',
                         'example_iauthfunctions_v5_custom_config_setting')
@pytest.mark.ckan_config('ckan.iauthfunctions.users_can_create_groups', True)
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
class TestAuthV5WithUserCreateGroup(object):

    def test_sysadmin_can_create_group_when_config_is_true(self):
        sysadmin = factories.Sysadmin()
        context = {'ignore_auth': False, 'user': sysadmin['name']}
        helpers.call_action('group_create', context, name='test-group')

    def test_user_can_create_group_when_config_is_true(self):
        user = factories.User()
        context = {'ignore_auth': False, 'user': user['name']}
        helpers.call_action('group_create', context, name='test-group')

    def test_visitor_cannot_create_group_when_config_is_true(self):
        context = {'ignore_auth': False, 'user': None}
        with pytest.raises(NotAuthorized):
            helpers.call_action('group_create', context, name='test-group')


@pytest.fixture
def curators_group():
    '''This is a helper method for test methods to call when they want
    the 'curators' group to be created.
    '''
    sysadmin = factories.Sysadmin()

    # Create a user who will *not* be a member of the curators group.
    noncurator = factories.User()

    # Create a user who will be a member of the curators group.
    curator = factories.User()

    # Create the curators group, with the 'curator' user as a member.
    users = [{'name': curator['name'], 'capacity': 'member'}]
    context = {'ignore_auth': False, 'user': sysadmin['name']}
    group = helpers.call_action('group_create',
                                context,
                                name='curators',
                                users=users)

    return (noncurator, curator, group)


@pytest.mark.ckan_config('ckan.plugins', 'example_iauthfunctions_v4')
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
def test_group_create_with_no_curators_group():
    '''Test that group_create doesn't crash when there's no curators group.
    '''
    sysadmin = factories.Sysadmin()

    # Make sure there's no curators group.
    assert 'curators' not in helpers.call_action('group_list', {})

    # Make our sysadmin user create a group. CKAN should not crash.
    context = {'ignore_auth': False, 'user': sysadmin['name']}
    helpers.call_action('group_create', context, name='test-group')


@pytest.mark.ckan_config('ckan.plugins', 'example_iauthfunctions_v4')
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
def test_group_create_with_visitor(curators_group):
    '''A visitor (not logged in) should not be able to create a group.

    Note: this also tests that the group_create auth function doesn't
    crash when the user isn't logged in.
    '''
    context = {'ignore_auth': False, 'user': None}
    with pytest.raises(NotAuthorized):
        helpers.call_action('group_create',
                            context,
                            name='this_group_should_not_be_created')


@pytest.mark.ckan_config('ckan.plugins', 'example_iauthfunctions_v4')
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
def test_group_create_with_non_curator(curators_group):
    '''A user who isn't a member of the curators group should not be able
    to create a group.
    '''
    noncurator, _, _ = curators_group
    context = {'ignore_auth': False, 'user': noncurator['name']}
    with pytest.raises(NotAuthorized):
        helpers.call_action('group_create',
                            context,
                            name='this_group_should_not_be_created')


@pytest.mark.ckan_config('ckan.plugins', 'example_iauthfunctions_v4')
@pytest.mark.usefixtures('clean_db', 'with_plugins', 'with_request_context')
def test_group_create_with_curator(curators_group):
    '''A member of the curators group should be able to create a group.
    '''
    _, curator, _ = curators_group
    name = 'my-new-group'
    context = {'ignore_auth': False, 'user': curator['name']}
    result = helpers.call_action('group_create', context, name=name)

    assert result['name'] == name

To run these extension tests, cd into the ckanext-iauthfunctions directory and run this command:

pytest --ckan-ini=test.ini ckanext/iauthfunctions/tests

Some notes on how these tests work:

  • Pytest has lots of useful functions for testing, see the pytest documentation.

  • We’re calling ckan.tests.call_action() This is a convenience function that CKAN provides for its own tests.

  • The CKAN core Testing coding standards can usefully be applied to writing tests for plugins.

  • CKAN core provides:

    which are also useful for testing extensions.

  • You might also find it useful to read the Flask testing documentation (or Pylons testing documentation for plugins using legacy pylons controllers).

  • Avoid importing the plugin modules directly into your test modules (e.g from example_iauthfunctions import plugin_v5_custom_config_setting). This causes the plugin to be registered and loaded before the entire test run, so the plugin will be loaded for all tests. This can cause conflicts and test failures.

Todo

Link to CKAN guidelines for how to write tests, once those guidelines have been written. Also add any more extension-specific testing details here.