schemaengine module¶

class genie.metaparser.util.schemaengine.Schema(schema, path=None)¶

Bases: object

Schema class

Provides the ability to define schema (schematics/requirements) for input data, and subsequently validate whether the input data meets requirements.

Example

data = Schema(str).validate(‘a string’) data = Schema({‘a’: str}).validate({‘a’: ‘some string’})

Parameters:	schema – the schema to be validated against. Any valid python datastructures/callable.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

validate(data, top=True)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

class genie.metaparser.util.schemaengine.Optional(schema, path=None)¶

Bases: genie.metaparser.util.schemaengine.Schema

Optional Class (Schema)

Marks an optional part of the schema.

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

validate(data, top=True)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

class genie.metaparser.util.schemaengine.Any¶

Bases: genie.metaparser.util.schemaengine.Optional

Any Class (Optional, Schema)

Marks a section of a schema that matches anything. This is effectively a wildcard (*).

Note that Any is also Optional.

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

validate(data, top=True)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

class genie.metaparser.util.schemaengine.And(*args)¶

Bases: genie.metaparser.util.schemaengine.Schema

And Class (Schema)

Defines a schema of AND relationship, eg, the input data must pass the validation of all of the requirements of this Schema.

Example

# requires a string of ‘left’ or ‘right’ And(str, lambda: s: s in (‘left’, ‘right’))

Parameters:	*args – arbitrary args of schema to apply AND to.

validate(data, top=False)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

class genie.metaparser.util.schemaengine.Or(*args)¶

Bases: genie.metaparser.util.schemaengine.Schema

Or Class (Schema)

Defines a schema of OR relationship, eg, the input data must pass the validation of one of the requirements of this Schema.

Example

# requires a string or an integer Or(str, int)

Parameters:	*args – arbitrary args of schema to apply OR to.

validate(data, top=False)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

class genie.metaparser.util.schemaengine.Default(schema, default)¶

Bases: genie.metaparser.util.schemaengine.Optional

Default Class (Optional, Schema)

Defines a schema with a default. Eg, if the schema was not satisfied, the default value is added to the input data. A Default schema is also Optional.

Note

using the Default schema changes the input data (due to application of default values)

Usage Criteria:: if a Default() is part of a dictionary schema type, then keys leading to this default value cannot be marked with any other schema objects, such as Optional(). Using Default() means that its key-chain is automatically mandatory, because regardless of input, the keys leading to the default value will always be there.

Parameters:	schema (any) – input schema requirements default (any) – default value to apply

validate(data, top=False)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

class genie.metaparser.util.schemaengine.Fallback(schema, fallback)¶

Bases: genie.metaparser.util.schemaengine.Optional

Fallback Class (Optional, Schema)

Defines a schema with a Fallback. Eg, if the schema was not satisfied, the value falls-back to another value in the given data. A Fallback schema is also Optional.

Fallbacks can only be used if the input schema and data to be validated are of type dict.

Note

using the Fallback schema changes the input data (due to application of fallback values)

Parameters:	schema (any) – input schema requirements fallback (str) – string representing the key to fallback to, using ‘.’ as separator to identify dict nesting levels.

Example

# fall back to data[‘a’][‘b’][‘c’] Fallback(str, ‘a.b.c’)

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

validate(data, top=True)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

class genie.metaparser.util.schemaengine.Use(schema, path=None)¶

Bases: genie.metaparser.util.schemaengine.Schema

validate(data, top=False)¶

validate input

Validates the given data against the current schema, and returns the correct, validated data. If there are defaults & fallbacks associated with the schema, augment the input data with them so that the final return contains validated data + the defaults/fallbacks.

Parameters:	data (any) – data to be validated
Returns:	Validated data with defaults & fallbacks applied

apply_defaults(data)¶

This API takes the current data and apply default fields to it wherever needed (eg, missing fields)

Parameters:	data (any) – input data
Returns:	input data augmented with default values

apply_fallback(data)¶

This API goes through the schema looking for fallback fields. When found, it takes the fallback key, pull up its data, and apply it to the current field, if it doesn’t exist.

Parameters:	data (any) – input data
Returns:	input data augmented with fallback values

collect_defaults()¶: Based on the given schema, compute the defaults, if any. This involves using the Default subclass and creating a new datastructure that represents the default state of this schema (eg, with a None input, the resulting data should look like the output of this)

collect_fallbacks()¶: Based on the given schema, compute the fallbacks, if any. This involves using the Fallback subclass and creating a new datastructure that represents the fallback states of this schema, eg, if a key does not exist, it should fallback to another key’s value.

Note

Fallbacks are only valid if the schema type is Dict.

classmethod priority(obj)¶

Function to make sure that we always process Any objects last.

Eg. process mandatory and optiona ones first before trying to match Any

class genie.metaparser.util.schemaengine.Path¶

Bases: tuple

Path object (tuple)

Defines a tuple-like object to be used with ListDict, extending a tuple’s native ability to compare to also support Any() objects, so that:

assert Path((1, Any(), 3)) == Path((1, 2, 3)) assert Path((1, Any(), 3)) == Path((1, 2, 3))

is_dynamic()¶

count(value) → integer -- return number of occurrences of value¶

index(value[, start[, stop]]) → integer -- return first index of value.¶: Raises ValueError if the value is not present.

missing_from(paths)¶: Compare the current path to a list of known paths, returning a list of matching paths that are missing from the known paths.

Example

Path((1, Any(), 2)).missing_from(((1,2,3), (2,3,4))) [(1,2,2)]