Validator functions reference¶

Validators in CKAN are user-defined functions that serves two purposes:

Ensuring that the input satisfies certain requirements
Converting the input to an expected form

Validators can be defined as a function that accepts one, two or four arguments. But this is an implementation detail and validators should not be called directly. Instead, the ckan.plugins.toolkit.navl_validate() function must be used whenever input requires validation.

import ckan.plugins.toolkit as tk
from ckanext.my_ext.validators import is_valid

data, errors = tk.navl_validate(
    {"input": "value"},
    {"input": [is_valid]},
)

And in order to be more flexible and allow overrides, don’t import validator functions directly. Instead, register them via the IValidators interface and use the ckan.plugins.tookit.get_validator() function:

import ckan.plugins as p
import ckan.plugins.toolkit as tk

def is_valid(value):
    return value

class MyPlugin(p.SingletonPlugin)
    p.implements(p.IValidators)

    def get_validators(self):
        return {"is_valid": is_valid}

...
# somewhere in code
data, errors = tk.navl_validate(
    {"input": "value"},
    {"input": [tk.get_validator("is_valid")]},
)

As you should have already noticed, navl_validate requires two parameters and additionally accepts an optional one. That’s their purpose:

Data that requires validation. Must be a dict object, with keys being the names of the fields.
The validation schema. It’s a mapping of field names to the lists of validators for that particular field.
Optional context. Contains any extra details that can change validation workflow in special cases. For the simplicity sake, we are not going to use context in this section, and in general is best not to rely on context variables inside validators.

Let’s imagine an input that contains two fields first and second. The first field must be an integer and must be provided, while the second field is an optional string. If we have following four validators:

is_integer
is_string
is_required
is_optional

we can validate data in the following way:

input = {"first": "123"}
schema = {
    "first": [is_required, is_integer],
    "second": [is_optional, is_string],
}

data, errors = tk.navl_validate(input, schema)

If the input is valid, data contains validated input and errors is an empty dictionary. Otherwise, errors contains all the validation errors for the provided input.

Built-in validators¶

ckan.lib.navl.validators.keep_extras(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → None¶

Convert dictionary into simple fields.

data, errors = tk.navl_validate(
    {"input": {"hello": 1, "world": 2}},
    {"input": [keep_extras]}
)
assert data == {"hello": 1, "world": 2}

ckan.lib.navl.validators.not_missing(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → None¶

Ensure value is not missing from the input, but may be empty.

data, errors = tk.navl_validate(
    {},
    {"hello": [not_missing]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.not_empty(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → None¶

Ensure value is available in the input and is not empty.

data, errors = tk.navl_validate(
    {"hello": None},
    {"hello": [not_empty]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.if_empty_same_as(other_key: str) → Callable[[...], Any]¶

Copy value from other field when current field is missing or empty.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [], "world": [if_empty_same_as("hello")]}
)
assert data == {"hello": 1, "world": 1}

ckan.lib.navl.validators.both_not_empty(other_key: str) → Union[Callable[[Any], Any], Callable[[Any, Context], Any], Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]]¶

Ensure that both, current value and other field has value.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [], "world": [both_not_empty("hello")]}
)
assert errors == {"world": [error_message]}

data, errors = tk.navl_validate(
    {"world": 1},
    {"hello": [], "world": [both_not_empty("hello")]}
)
assert errors == {"world": [error_message]}

data, errors = tk.navl_validate(
    {"hello": 1, "world": 2},
    {"hello": [], "world": [both_not_empty("hello")]}
)
assert not errors

ckan.lib.navl.validators.empty(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → None¶

Ensure that value is not present in the input.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [empty]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.ignore(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → NoReturn¶

Remove the value from the input and skip the rest of validators.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [ignore]}
)
assert data == {}

ckan.lib.navl.validators.default(default_value: Any) → Union[Callable[[Any], Any], Callable[[Any, Context], Any], Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]]¶

Convert missing or empty value to the default one.

data, errors = tk.navl_validate(
    {},
    {"hello": [default("not empty")]}
)
assert data == {"hello": "not empty"}

ckan.lib.navl.validators.configured_default(config_name: str, default_value_if_not_configured: Any) → Union[Callable[[Any], Any], Callable[[Any, Context], Any], Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]]¶: When key is missing or value is an empty string or None, replace it with a default value from config, or if that isn’t set from the default_value_if_not_configured.

ckan.lib.navl.validators.ignore_missing(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → None¶

If the key is missing from the data, ignore the rest of the key’s schema.

By putting ignore_missing at the start of the schema list for a key, you can allow users to post a dict without the key and the dict will pass validation. But if they post a dict that does contain the key, then any validators after ignore_missing in the key’s schema list will be applied.

Raises: ckan.lib.navl.dictization_functions.StopOnError – if data[key] is ckan.lib.navl.dictization_functions.missing or None
Returns: None

ckan.lib.navl.validators.ignore_empty(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → None¶

Skip the rest of validators if the value is empty or missing.

data, errors = tk.navl_validate(
    {"hello": ""},
    {"hello": [ignore_empty, isodate]}
)
assert data == {}
assert not errors

ckan.lib.navl.validators.convert_int(value: Any) → int¶

Ensure that the value is a valid integer.

data, errors = tk.navl_validate(
    {"hello": "world"},
    {"hello": [convert_int]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.unicode_only(value: Any) → str¶

Accept only unicode values

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [unicode_only]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.unicode_safe(value: Any) → str¶

Make sure value passed is treated as unicode, but don’t raise an error if it’s not, just make a reasonable attempt to convert other types passed.

This validator is a safer alternative to the old ckan idiom of using the unicode() function as a validator. It tries not to pollute values with Python repr garbage e.g. when passed a list of strings (uses json format instead). It also converts binary strings assuming either UTF-8 or CP1252 encodings (not ASCII, with occasional decoding errors)

ckan.lib.navl.validators.limit_to_configured_maximum(config_option: str, default_limit: int) → Union[Callable[[Any], Any], Callable[[Any, Context], Any], Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]]¶: If the value is over a limit, it changes it to the limit. The limit is defined by a configuration option, or if that is not set, a given int default_limit.

ckan.logic.validators.owner_org_validator(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶

Validate organization for the dataset.

Depending on the settings and user’s permissions, this validator checks whether organization is optional and ensures that specified organization can be set as an owner of dataset.

ckan.logic.validators.package_id_not_changed(value: Any, context: Context) → Any¶: Ensures that package’s ID is not changed during the update.

ckan.logic.validators.int_validator(value: Any, context: Context) → Any¶

Return an integer for value, which may be a string in base 10 or a numeric type (e.g. int, long, float, Decimal, Fraction). Return None for None or empty/all-whitespace string values.

Raises: ckan.lib.navl.dictization_functions.Invalid for other inputs or non-whole values

ckan.logic.validators.natural_number_validator(value: Any, context: Context) → Any¶: Ensures that the value is non-negative integer.

ckan.logic.validators.is_positive_integer(value: Any, context: Context) → Any¶: Ensures that the value is an integer that is greater than zero.

ckan.logic.validators.datetime_from_timestamp_validator(value: Any, context: Context) → Any¶

ckan.logic.validators.boolean_validator(value: Any, context: Context) → Any¶: Return a boolean for value. Return value when value is a python bool type. Return True for strings ‘true’, ‘yes’, ‘t’, ‘y’, and ‘1’. Return False in all other cases, including when value is an empty string or None

ckan.logic.validators.isodate(value: Any, context: Context) → Any¶: Convert the value into datetime.datetime object.

ckan.logic.validators.package_id_exists(value: str, context: Context) → Any¶: Ensures that the value is an existing package’s ID or name.

ckan.logic.validators.package_id_does_not_exist(value: str, context: Context) → Any¶: Ensures that the value is not used as a package’s ID or name.

ckan.logic.validators.resource_id_does_not_exist(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶

ckan.logic.validators.package_name_exists(value: str, context: Context) → Any¶: Ensures that the value is an existing package’s name.

ckan.logic.validators.package_id_or_name_exists(package_id_or_name: str, context: Context) → Any¶

Return the given package_id_or_name if such a package exists.

Raises: ckan.lib.navl.dictization_functions.Invalid if there is no package with the given id or name

ckan.logic.validators.resource_id_exists(value: Any, context: Context) → Any¶: Ensures that the value is not used as a resource’s ID or name.

ckan.logic.validators.resource_id_validator(value: Any) → Any¶

ckan.logic.validators.user_id_exists(user_id: str, context: Context) → Any¶: Raises Invalid if the given user_id does not exist in the model given in the context, otherwise returns the given user_id.

ckan.logic.validators.user_id_or_name_exists(user_id_or_name: str, context: Context) → Any¶

Return the given user_id_or_name if such a user exists.

Raises: ckan.lib.navl.dictization_functions.Invalid if no user can be found with the given id or user name

ckan.logic.validators.group_id_exists(group_id: str, context: Context) → Any¶: Raises Invalid if the given group_id does not exist in the model given in the context, otherwise returns the given group_id.

ckan.logic.validators.group_id_or_name_exists(reference: str, context: Context) → Any¶: Raises Invalid if a group identified by the name or id cannot be found.

ckan.logic.validators.name_validator(value: Any, context: Context) → Any¶

Return the given value if it’s a valid name, otherwise raise Invalid.

If it’s a valid name, the given value will be returned unmodified.

This function applies general validation rules for names of packages, groups, users, etc.

Most schemas also have their own custom name validator function to apply custom validation rules after this function, for example a package_name_validator() to check that no package with the given name already exists.

Raises: ckan.lib.navl.dictization_functions.Invalid – if value is not a valid name

ckan.logic.validators.package_name_validator(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that value can be used as a package’s name

ckan.logic.validators.package_version_validator(value: Any, context: Context) → Any¶: Ensures that value can be used as a package’s version

ckan.logic.validators.duplicate_extras_key(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that there are no duplicated extras.

ckan.logic.validators.group_name_validator(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that value can be used as a group’s name

ckan.logic.validators.tag_length_validator(value: Any, context: Context) → Any¶: Ensures that tag length is in the acceptable range.

ckan.logic.validators.tag_name_validator(value: Any, context: Context) → Any¶: Ensures that tag does not contain wrong characters

ckan.logic.validators.tag_not_uppercase(value: Any, context: Context) → Any¶: Ensures that tag is lower-cased.

ckan.logic.validators.tag_string_convert(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Takes a list of tags that is a comma-separated string (in data[key]) and parses tag names. These are added to the data dict, enumerated. They are also validated.

ckan.logic.validators.ignore_not_package_admin(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ignore if the user is not allowed to administer the package specified.

ckan.logic.validators.ignore_not_sysadmin(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ignore the field if user not sysadmin or ignore_auth in context.

ckan.logic.validators.ignore_not_group_admin(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ignore if the user is not allowed to administer for the group specified.

ckan.logic.validators.user_name_validator(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶

Validate a new user name.

Append an error message to errors[key] if a user named data[key] already exists. Otherwise, do nothing.

Raises: ckan.lib.navl.dictization_functions.Invalid – if data[key] is not a string
Return type: None

ckan.logic.validators.user_both_passwords_entered(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that both password and password confirmation is not empty

ckan.logic.validators.user_password_validator(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that password is safe enough.

ckan.logic.validators.user_passwords_match(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that password and password confirmation match.

ckan.logic.validators.user_password_not_empty(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Only check if password is present if the user is created via action API. If not, user_both_passwords_entered will handle the validation

ckan.logic.validators.user_about_validator(value: Any, context: Context) → Any¶: Ensures that user’s about field does not contains links.

ckan.logic.validators.vocabulary_name_validator(name: str, context: Context) → Any¶: Ensures that the value can be used as a tag vocabulary name.

ckan.logic.validators.vocabulary_id_not_changed(value: Any, context: Context) → Any¶: Ensures that vocabulary ID is not changed during the update.

ckan.logic.validators.vocabulary_id_exists(value: Any, context: Context) → Any¶: Ensures that value contains existing vocabulary’s ID or name.

ckan.logic.validators.tag_in_vocabulary_validator(value: Any, context: Context) → Any¶: Ensures that the tag belongs to the vocabulary.

ckan.logic.validators.tag_not_in_vocabulary(key: tuple[Any, ...], tag_dict: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that the tag does not belong to the vocabulary.

ckan.logic.validators.url_validator(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Checks that the provided value (if it is present) is a valid URL

ckan.logic.validators.user_name_exists(user_name: str, context: Context) → Any¶: Ensures that user’s name exists.

ckan.logic.validators.role_exists(role: str, context: Context) → Any¶: Ensures that value is an existing CKAN Role name.

ckan.logic.validators.datasets_with_no_organization_cannot_be_private(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶

ckan.logic.validators.list_of_strings(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that value is a list of strings.

ckan.logic.validators.if_empty_guess_format(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Make an attempt to guess resource’s format using URL.

ckan.logic.validators.clean_format(format: str)¶: Normalize resource’s format.

ckan.logic.validators.no_loops_in_hierarchy(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Checks that the parent groups specified in the data would not cause a loop in the group hierarchy, and therefore cause the recursion up/down the hierarchy to get into an infinite loop.

ckan.logic.validators.filter_fields_and_values_should_have_same_length(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶

ckan.logic.validators.filter_fields_and_values_exist_and_are_valid(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶

ckan.logic.validators.extra_key_not_in_root_schema(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensures that extras are not duplicating base fields

ckan.logic.validators.empty_if_not_sysadmin(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Only sysadmins may pass this value

ckan.logic.validators.strip_value(value: str)¶: Trims the Whitespace

ckan.logic.validators.email_validator(value: Any, context: Context) → Any¶: Validate email input

ckan.logic.validators.collect_prefix_validate(prefix: str, *validator_names: str) → Union[Callable[[Any], Any], Callable[[Any, Context], Any], Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]]¶: Return a validator that will collect top-level keys starting with prefix then apply validator_names to each one. Results are moved to a dict under the prefix name, with prefix removed from keys

ckan.logic.validators.dict_only(value: Any) → dict[Any, Any]¶: Ensures that the value is a dictionary

ckan.logic.validators.email_is_unique(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Validate email is unique

ckan.logic.validators.one_of(list_of_value: Container[Any]) → Union[Callable[[Any], Any], Callable[[Any, Context], Any], Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]]¶: Checks if the provided value is present in a list or is an empty string

ckan.logic.validators.json_object(value: Any) → Any¶: Make sure value can be serialized as a JSON object

ckan.logic.validators.extras_valid_json(extras: Any, context: Context) → Any¶: Ensures that every item in the value dictionary is JSON-serializable.

ckan.logic.converters.convert_to_extras(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Convert given field into an extra field.

ckan.logic.converters.convert_from_extras(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Restore field using object’s extras.

ckan.logic.converters.extras_unicode_convert(extras: dict[FlattenKey, Any], context: Context)¶: Convert every value of the dictionary into string.

ckan.logic.converters.free_tags_only(key: tuple[Any, ...], data: dict[FlattenKey, Any], errors: dict[FlattenKey, list[str]], context: Context) → Any¶: Ensure that none of the tags belong to a vocabulary.

ckan.logic.converters.convert_to_tags(vocab: Any) → Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]¶: Convert list of tag names into a list of tag dictionaries

ckan.logic.converters.convert_from_tags(vocab: Any) → Callable[[tuple[Any, ...], dict[FlattenKey, Any], dict[FlattenKey, list[str]], Context], None]¶

ckan.logic.converters.convert_user_name_or_id_to_id(user_name_or_id: Any, context: Context) → Any¶

Return the user id for the given user name or id.

The point of this function is to convert user names to ids. If you have something that may be a user name or a user id you can pass it into this function and get the user id out either way.

Also validates that a user with the given name or id exists.

Returns: the id of the user with the given user name or id
Return type: string
Raises: ckan.lib.navl.dictization_functions.Invalid if no user can be found with the given id or user name

ckan.logic.converters.convert_package_name_or_id_to_id(package_name_or_id: Any, context: Context) → Any¶

Return the package id for the given package name or id.

The point of this function is to convert package names to ids. If you have something that may be a package name or id you can pass it into this function and get the id out either way.

Also validates that a package with the given name or id exists.

Returns: the id of the package with the given name or id
Return type: string
Raises: ckan.lib.navl.dictization_functions.Invalid if there is no package with the given name or id

ckan.logic.converters.convert_group_name_or_id_to_id(group_name_or_id: Any, context: Context) → Any¶

Return the group id for the given group name or id.

The point of this function is to convert group names to ids. If you have something that may be a group name or id you can pass it into this function and get the id out either way.

Also validates that a group with the given name or id exists.

Returns: the id of the group with the given name or id
Return type: string
Raises: ckan.lib.navl.dictization_functions.Invalid if there is no group with the given name or id

ckan.logic.converters.convert_to_json_if_string(value: Any, context: Context) → Any¶: Parse string value as a JSON object.

ckan.logic.converters.as_list(value: Any)¶: Convert whitespace separated string into a list of strings.

ckan.logic.converters.convert_to_list_if_string(value: Any) → Any¶: Transform string into one-item list

ckan.logic.converters.json_or_string(value: Any) → Any¶: parse string values as json, return string if that fails

ckan.logic.converters.json_list_or_string(value: Any) → Any¶: parse string values as json or comma-separated lists, return string as a one-element list if that fails

ckan.logic.converters.remove_whitespace(value: Any, context: Context) → Any¶: Trim whitespaces from the value.