Schema Engine

Introduction

schemaengine module located in metaparser package util directory (metaparser.util), along with a few other utility modules like util.py and exceptions.py. This module defines all schema related classes and functions.

Parser schema

The term “schema” refers to the organization of data as a blueprint of how the parser output is constructed (dictionary or nested-dictionary structure).

The Metaparser uses the schema to control how and what the parsed information should be structured, which not only protects the existing scripts against unthoughtful parser API updates, but also empowers data-modeling agnostic testing by simply switching among different contexts within a same script.

Each parsing mechanism (cli, xml, yang) implemented in a same parser class has to follow the same schema definition. The developer who defines the first parsing mechanism (e.g.: cli ()) in parser class will also define the schema for the output structure. At the end of the parsing process, parser engine (MetaParser) will do schema checking to make sure the parser always returns the output (nested dict) that has the same data structure across all supported parsing mechanisms.

Schema class

Schema class defines the schema (schematics/requirements) for input data, and subsequently validates whether the input data meets the requirements.

class instance attributes:
schema: the schema to be validated against. It can be any valid Python data structures (eg.: dict) or callables.

To instantiate the Schema class:
from genie.metaparser.util.schemaengine import Schema
dic = {'a':1, 'b':'default'}
s = Schema(dic)
class functionalities:
apply_defaults: function takes the current data and applies default fields to wherever needed (e.g:, missing fields), and returns the input data augmented with default values.

collect_defaults: function computes the ‘defaults’ based on the given schema, returns a new data structure representing the default state of this schema.
validate: function validates the given data against the current schema, and returns the correct, validated data.

For example:
# validate simple 'str' type of schema
data = Schema(str).validate('a string')

# valide 'dict' type of schema
data = Schema({'a': str}).validate({'a': 'some string'})

Other types of schema

There are few more subclasses of Schema defined in the schemaengine module:

Any: Any schema indicates that the schema matches ‘anything’. It’s commonly been used in bigger and more complicated schemas to mark a section or a field of the schema to match with anything. This is effectively a wildcard (*) in schema data format.

from genie.metaparser.util.schemaengine import Schema, Any
dic = {'cmp': {
                'module': {
                         Any(): {
                                 'bios_compile_time': str,
                                 'bios_version': str,
                                 'image_compile_time': str,
                                 'image_version': str,
                                 'status': str},}},
       'hardware': {
                'bootflash': str,
                'chassis': str,
                'cpu': str,
                'device_name': str,
                'memory': str,
                'model': str,
                'processor_board_id': str,
                'slots': str,
                Any(): Any(),},}
s = Schema(dic)

Default: Default class defines a schema with a ‘default’ which if the schema is not satisfied, the default fields will be added to the input data.
And: And class defines a schema of AND relationship which the input data must pass the validation of all requirements of this schema.
# requires a string of 'left' or 'right' s = And(str, lambda: s: s in ('left', 'right'))
Or: Or class defines a schema of ‘OR’ relationship, which the input data must pass the validation of one of the requirements of this schema.
# requires a string or an integer s = Or(str, int)

Path class

Path class defines a tuple-like object to be used mainly for pyATS ListDict objects comparison - nested dictionary key path comparison. The class extends a tuple’s native comparing ability to also support Any objects.

assert Path((1, Any(), 3)) == Path((1, 2, 3))

Schema exceptions

All errors related to the schema module have been defined in genie.metaparser.util.exceptions

from genie.metaparser.util.schemaengine import SchemaValueError,\
                                               SchemaClassError,\
                                               SchemaTypeError,\
                                               SchemaMissingKeyError,\
                                               SchemaUnsupportedKeyError, \
                                               SchemaFallbackError, \
                                               SchemaFallbackLoopError, \
                                               SchemaEmptyParserError