Core documentation

Markup Syntax

CLI output parsing rules may be specified using a marked-up text input that is divided into OS sections and then further subdivided into 3-5 sub-sections. By default, the supported OS values are : IOX, IOS, NXOS and LINUX. The supported sub-subsections are CMD, SHOWCMD, PREFIX, ACTUAL and MARKUP. The ACTUAL subsection is optional and is used to simply store the unmarked up raw show output.

An example of a simple marked-up file is as follows:

SHOWCMD: show ip mroute
PREFIX: mrib-route

(,, uptime: 00:00:43, igmp pim ip
  Incoming interface: Ethernet2/1, RPF nbr:^M
  Outgoing interface list: (count: 1)^M
    Ethernet2/2, uptime: 00:00:43, igmp^M

(XP<source>X13.13.13.4/32, XP<group>X232.0.0.1/32), uptime: XT<uptime>X00:00:43,
  XR<protos>Xigmp pim ip
  Incoming interface: XI<ingress-intf>XEthernet2/1, RPF nbr: XA<rpf>X11.11.11.2
  Outgoing interface list: (count: XN<egress-count>X1)
    XI<egress-intf>XEthernet2/2, uptime: XT<egress-uptime>X00:00:43, XR<egress-protos>Xigmp

The above is used to tell core the details of how to go about parsing the show ip mroute command on a NXOS router.

The OS Section

Multiple OS architectures are supported. The OS section tells core which OS the following show commands (and their resulting regular expressions) are for. The currently supported operating systems are IOS, IOX, NXOS and LINUX.

The CMD Sub-Section

The CMD section is a single line that records the symbolic name for the show command, this name is passed to core so that it can look up the correct actual show command for the given OS. If this section is not specified then it will be created by uppercasing the SHOWCMD value and converting spaces and dashes to underscores.

The SHOWCMD Sub-Section

The SHOWCMD section is a single line that records the show command that generated the output that will follow.

The PREFIX Sub-Section

The prefix section indicates the prefix that should be used to identify all regular expressions when accessing core parsing classes.

The ACTUAL Sub-Section

One typically copies and pastes the output from a show command into the ACTUAL section, which is simply a place to keep an unmodified copy of the show command output.

The MARKUP Sub-Section

One typically copies the ACTUAL output into the MARKUP section and then uses an XxX or Xx<name>X notation to tell core what type of value follows. The markup notation allows for instructing core what to name the value as well. If the name is omitted then core takes a guess as to the name by using text immediately before or after the value. A warning is generated if duplicate names are detected.

The following are the available values for x in the XxX notation:

  • A - IPv4 or IPv6 address.

  • B - Value terminated with a close brace, bracket, or parenthesis.

  • C - Value terminated with a comma.

  • F - Floating point number.

  • H - Hexidecimal number.

  • I - Interface name.

  • M - Mac address.

  • N - Decimal number.

  • R - everything else to the newline.

  • P - IPv4 or IPv6 prefix.

  • Q - Value terminated by a double quote.

  • S - Non-space value.

  • T - Time (00:00:00)

  • W - A word.


Additionally if one uses lower case rather than upper case for x then the value is matched with a regular expression but no tag is created. This allows for specifying a single line multiple times in order to omit various values that may only optionally be present.

For example consider if the uptime and protos values were optional in the output from show ip mroute on NXOS:

(XP<source>X13.13.13.4/32, XP<group>X232.0.0.1/32)
(Xp<source>X13.13.13.4/32, Xp<group>X232.0.0.1/32), uptime: XT<uptime>X00:00:43
(Xp<source>X13.13.13.4/32, Xp<group>X232.0.0.1/32), uptime: Xt<uptime>X00:00:43,
XR<protos>Xigmp pim ip

This generates the following dictionary output:

'mrib-route.source': '\(([A-Fa-f0-9/:\.]+),\s+[A-Fa-f0-9/:\.]+\)',
'' : '\([A-Fa-f0-9/:\.]+,\s+([A-Fa-f0-9/:\.]+)\)',
'mrib-route.uptime': '\([A-Fa-f0-9/:\.]+,\s+[A-Fa-f0-9/:\.]+\),
'mrib-route.protos': '\([A-Fa-f0-9/:\.]+,\s+[A-Fa-f0-9/:\.]+\),

You can see above that uptime and protos are not present on the first and second lines because they do not need to be present for source or group to match.

A second markup notation which allows for specifying the exact regex to parse the value is allowed for cases where none of the above predefined value types works. This notation takes the form XXX<regex>XXX or XXX<regex><name>XXX. So using this notation in place of XSX would look like this:



Manual regex specification via XXX does not support the use of the repetition operators {m} and {m, n}.

Parsing Example

The following example shows how a user can specify marked-up text, and then use core to parse both tabular and non-tabular elements in a piece of show command output:

Sample Parsing Script

The following script illustrates the parsing of show command output that contains both tabular and non-tabular content. It also shows the automatic validation of non-tabular content. Notice the use of a tuple in the values field to indicate multiple acceptable matches. Notice also that in the cases where the user does not specify the field name, a name is automatically chosen based on surrounding text(table-id):

BGP router identifier, local AS number 100
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0xe0000000   RD version: 8
BGP main routing table version 8
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.

Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker               8          8          8          8           8           8

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd          0   100      63      55        8    0    0 00:51:24        100          0   200      63      55        8    0    0 00:40:16        200

SHOWCMD: show bgp summary


BGP router identifier XA<router-id>X192.168.0.12, local AS number XN<local-as>X100
BGP generic scan interval XN<gen-scan-interval>X60 secs
BGP table state: XW<table-state>XActive
Table ID: XHX0xe0000000   RD version: 8
BGP main routing table version 8
BGP scan interval XN<scan-interval>X60 secs

BGP is operating in XW<oper-mode>XSTANDALONE mode.

Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker               8          8          8          8           8           8

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd          0   100      63      55        8    0    0 00:51:24        100          0   200      63      55        8    0    0 00:40:16        200
# Parse non-tabular data
from genie.parsergen.core import extend_markup

from genie.parsergen.core import _parser_gen_t
from genie.parsergen import core
parse_key = 'rtr1'
parsergen.core.text_to_parse[parse_key] = bgp_summ_actual_output

print ("Parsing CLI show command output: ...")
tags_to_parse =     [
                     'bgp.local-as',             (100, 101),
                     'bgp.router-id',            None,
                     'bgp.gen-scan-interval',    None,
                     'bgp.table-state',          None,
                     'bgp.table-id',             None,
                     'bgp.scan-interval',        None,
                     'bgp.oper-mode',            None]
pg = _parser_gen_t('IOX', tags_to_parse[::2], tags_to_parse[1::2], fill = True,
(validate, dictio, msg) = pg._parser_validate()
print("     Parser's fill report :\n          validate={}," +
      " \n          dictio={}, \n          msg={}\n".
    format(validate, dictio, msg))
parsed_output = parsergen.core.ext_dictio[parse_key]

# Parse tabular data
from genie.parsergen.core import column_table_result_core_t
header = ['Neighbor', 'Spk', 'AS', 'MsgRcvd', 'MsgSent',
          'TblVer', 'InQ', 'OutQ', 'Up/Down', 'St/PfxRcd']
labels = ['neighbor', 'spk', 'as', 'msg_rcvd', 'msg_sent' ,
          'tbl_ver', 'in_q', 'out_q', 'time', 'prefixes']
tabular_parse_result = column_table_result_core_t(
                            header, label_fields=labels, index=[0,2],
                            right_justified=True, parse_key=parse_key);
parsed_output['bgp.neighbor-table'] = tabular_parse_result.entries

print ("CLI parse results for combination tabular/non-tabular data:")
import pprint
pprint.pprint(parsed_output, indent=5)

print ("\n\nTesting COMPARE mode for non-tabular data: ...")

tags_to_parse =     [
                     'bgp.local-as',             (100,101),
                     'bgp.router-id',            '',
                     'bgp.gen-scan-interval',    60,
                     'bgp.table-state',          'Active',
                     'bgp.table-id',             '0xe0000000',
                     'bgp.scan-interval',        60,
                     'bgp.oper-mode',            'STANDALONE']
pg = _parser_gen_t('IOX', tags_to_parse[::2], tags_to_parse[1::2],
        fill = False, parse_key=parse_key)
(validate, dictio, msg) = pg._parser_validate()

print("     Expected parser output     :\n      {}\n".format(tags_to_parse))
print("     Actual parser output       :\n      {}\n".format(parsergen.core.ext_dictio[parse_key]))
print("     Parser's comparison report :\n      validate={}," +
    " \n          dictio={}, \n          msg={}\n".
    format(validate, dictio, msg))

Use of Subclassing with the Tabular Parser

A subclass can define a class dictionary variable field_mapping whose keys are field names and whose value is the type of that field. When the results are parsed the value is cast to the given type. Values can also take the form of a generic mapping function.

A subclass can define a class list variable table_title_mapping which is a list of mapping functions for table title value keys. If specified, this list has an entry for each matching group present in table_title_pattern.

A subclass can also override the function cleanup_entry_field to cleanup a field value prior to it being stored. This is called prior to the mapping being done.

Sub-class Example:

from genie.parsergen import core
parse_key = ''
core.text_to_parse[parse_key]  = '''
RP/0/0/CPU0:iox#show isis database
Wed Dec 16 09:48:55.017 EST

IS-IS ring (Level-1) Link State Database
LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime  ATT/P/OL
iox.00-00           * 0x00000008   0xf9fd        1003            0/0/0
ioxbfd.00-00          0x00000004   0x8f36        862             0/0/0

Total Level-1 LSP count: 4     Local Level-1 LSP count: 1

IS-IS ring (Level-2) Link State Database
LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime  ATT/P/OL
iox.00-00           * 0x00000009   0x351a        1003            0/0/0
iox.01-00             0x00000002   0x0ead        922             0/0/0

Total Level-2 LSP count: 4     Local Level-2 LSP count: 1

def _hexint (val):
    return int(val, 16)

def cleanupLspId (field):
    return field

from core import column_table_result_core_t

class my_isis_database_column_parser_t (column_table_result_core_t):
    field_mapping = {
        'LSPID'       :   str,
        'LSP Seq Num' :   _hexint,
        'LSP Checksum':   _hexint,
        'LSP Holdtime':   None,

    table_title_mapping = [ int ]

    def __init__ (self):
        headers = ["LSPID", "LSP Seq Num", "LSP Checksum",
        "LSP Holdtime",  "ATT/P/OL"]
        labels = headers

         "Total Level-[12] LSP count:",
         table_title_pattern =
         r"IS-IS (?:[-\w]+ )?\(?Level-([12])\)? Link State Database:?",
         label_fields = labels)

    def cleanup_entry_field (self, header, field):
        if header == "LSPID":
            # Strip the "*" off the LSPID for the router's own LSPID.
            return cleanupLspId(field)
            return field

result = my_isis_database_column_parser_t()
print (result.entries[2]['iox.01-00']['LSP Holdtime'])