Genie Run

genie run is a one-stop-shop tool for executing all the Python functionality provided by Genie harness directly from linux command line. It eliminates the need for creating and maintaining job-files and having working Python knowledge prior to getting started with Genie.

For more information on Genie harness, refer to the user guide:

genie run supports all arguments from Genie, easypy and aetest.

For a complete list of arguments supported by genie run sub-components:

To see what functionality genie run offers, execute the following in your linux terminal:

(genie) bash-4.1$ genie run --help

usage: genie [--no-mail] [--mail-to] [--mail-subject] [--mail-html]
             [--submitter] [--image] [--release] [--html-logs [DIR]]
             [--xunit [DIR]] [--tims-post] [--tims-user] [--tims-dns]
             [--tims-options] [--tims-folder] [--tims-custom-attrs]
             [--tims-config-id] [--no-archive] [--runinfo-dir] [--archive-dir]
             [--no-upload] [--bg-upload] [-t] [--clean-file FILE]
             [--clean-devices [[...]]] [--clean-scope {job,task}]
             [--invoke-clean] [--logical-testbed-file] [--rerun-file FILE]
             [--rerun-condition  [...]] [-h] [-v] [-q] [--datafile FILE]
             [--uids LOGIC] [--groups LOGIC] [--random] [--random-seed INT]
             [--max-failures INT] [--pdb] [--step-debug FILE]
             [--pause-on PAUSE_ON] [--trigger-datafile FILE]
             [--verification-datafile FILE] [--trigger-file FILE]
             [--verification-file FILE] [--pts-datafile FILE]
             [--pts-features PTS_FEATURES] [--pts-golden-config FILE]
             [--config-datafile FILE] [--verification-uids LOGIC]
             [--trigger-uids LOGIC] [--verification-groups LOGIC]
             [--trigger-groups LOGIC] [--mapping-datafile FILE]
             [--subsection-datafile FILE] [--debug-plugin FILE]
             [--filetransfer-protocol FILETRANSFER_PROTOCOL]
             [-devices DEVICES]
             [--tgn-skip-configuration TGN_SKIP_CONFIGURATION] [--tgn-enable]
             [--tgn-traffic-convergence-threshold TGN_TRAFFIC_CONVERGENCE_THRESHOLD]
             [--tgn-reference-rate-threshold TGN_REFERENCE_RATE_THRESHOLD]
             [--tgn-first-sample-threshold TGN_FIRST_SAMPLE_THRESHOLD]
             [--tgn-disable-traffic-post-execution TGN_DISABLE_TRAFFIC_POST_EXECUTION]
             [--tgn-traffic-loss-recovery-threshold TGN_TRAFFIC_LOSS_RECOVERY_THRESHOLD]
             [--tgn-traffic-loss-tolerance-percentage TGN_TRAFFIC_LOSS_TOLERANCE_PERCENTAGE]
             [--tgn-enable-traffic-loss-check TGN_ENABLE_TRAFFIC_LOSS_CHECK]
             [--tgn-config-post-device-config TGN_CONFIG_POST_DEVICE_CONFIG]
             [--tgn-profile-snapshot-threshold TGN_PROFILE_SNAPSHOT_THRESHOLD]
             [--tgn-routing-threshold TGN_ROUTING_THRESHOLD]
             [--tgn-port-list TGN_PORT_LIST]
             [--tgn-arp-wait-time TGN_ARP_WAIT_TIME]
             [--tgntcl-enable-arp TGNTCL_ENABLE_ARP]
             [--tgntcl-learn-after-n-samples TGNTCL_LEARN_AFTER_N_SAMPLES]
             [--tgntcl-stream-sample-rate-percentage TGNTCL_STREAM_SAMPLE_RATE_PERCENTAGE]
             [--tgntcl-wait-multiplier TGNTCL_WAIT_MULTIPLIER] [-C FILE]

Easypy command line arguments.
Non-recognized args will pass-through to the user job file.

  genie /path/to/
  genie /path/to/ -testbed_file /path/to/testbed.yaml


  -C, --configuration FILE    easypy configuration yaml file for plugins

  --no-mail                   disable report email notifications
  --mail-to                   list of report email recipients
  --mail-subject              report email subject header
  --mail-html                 enable html format report email

  --submitter                 Specify the current submitter user id
  --image                     Specify the image under test
  --release                   Specify the release being tested
  --xunit [DIR]               Generate xunit report in the provided location. If used as a flag, generates xunit reports runtime directory

HTML Logging:
  --html-logs [DIR]           Experimental feature. Directory to generates HTML logs in addition to any existing log files.
                              Note - will increase archive size due to log duplication.

  --tims-post                 Enable posting results to tims
  --tims-user                 TIMS user. If not specified, the userstarting the run is used
  --tims-dns                  TIMS project dns name
  --tims-options              A string in the form of -key value sequence.
                              (Add a space at the beginning of the string to work around python parser bug. e.g. -a 1
  --tims-folder               A TIMS 3.x ID, a Logical ID or a complete, fully-qualified path
  --tims-custom-attrs         A str of dashed key/value pairs
                              e.g. -key1 val1 -key2 val2. The key value pairs are fed to TIMS as global custom attributes
  --tims-config-id            TIMS config_id, maps to Configuration ID, Related Config or logical identifier

  --no-archive                disable archive creation
  --runinfo-dir               specify alternate runinfo directory
  --archive-dir               specify alternate archive directory
  --no-upload                 Disable uploading archive to TRADe
  --bg-upload                 Upload to TRADe in background

  -t, --testbed-file          Specify testbed file location

  --clean-file FILE           Specify clean file location
  --clean-devices [ [ ...]]   Specify list of devices to clean, separated by spaces.  To clean groups of devices sequentially, specify as "[[dev1, dev2], dev3]".
  --clean-scope {job,task}    Specify whether clean runs before job or per task
  --invoke-clean              Clean is only invoked if this parameter is specified.

  --logical-testbed-file      Specify logical testbed file location

  --rerun-file FILE           rerun.results file that contains the information of tasks and testcases
  --rerun-condition  [ ...]   Results type list for the condition of rerun plugin.

General Options:
  -h, --help                  Show help information
  -v, --verbose               Give more output, additive up to 3 times.
  -q, --quiet                 Give less output, additive up to 3 times, corresponding to WARNING, ERROR, and CRITICAL logging levels

Input Datafile:
  --datafile FILE             input datafile, supplementing the script
                              with additional feature enablement & values

Execution Control:
  --uids LOGIC                logic string matching section uids to run.
                              eg: -uids="And('.*setup.*', 'test.+')"
  --groups LOGIC              logic string matching testcase groups to be run.
                              eg: -ids="And('sanity', 'regression')"
  --random                    flag to randomize the execution of testcases
  --random-seed INT           seed value for testcase randomization
  --max-failures INT          max number of failures before script abandon

  --pdb                       start the interactive debugger on failure
  --step-debug FILE           file containing the debug command to run specific step
  --pause-on PAUSE_ON         Pauses the testscript when there is a matching phrase in the log messages

Genie arguments:
  --trigger-datafile FILE     Trigger configuration datafile
  --verification-datafile     FILE
                              VerificationTrigger configuration datafile
  --trigger-file FILE         Trigger configuration file
  --verification-file FILE    VerificationTrigger configuration file
  --pts-datafile FILE         PTS configuration datafile
  --pts-features PTS_FEATURES
                              Features to learn for pts
  --pts-golden-config FILE    Golden configuration file
  --config-datafile FILE      File containing configuration information
  --verification-uids STR     string matching verifications uids to run.
                              eg: -uids="Verify_IpOspfNeighborDetail_vrf_all Verify_IpRoute_protocol_bgp"
                              or: -uids="And('Verify_IpOspfNeighborDetail_vrf_all',
  --trigger-uids STR          string matching triggers uids to run.
                              eg: -uids="TriggerShutNoShutEthernetInterface TriggerAddRemoveAdvertiseEvpnMulticast"
                              or: -uids="And('TriggerShutNoShutEthernetInterface',
  --verification-groups LOGIC
                              logic string matching verifications groups to be run.
                              eg: -ids="And('sanity', 'regression')"
  --trigger-groups LOGIC      logic string matching triggers groups to be run.
                              eg: -ids="And('sanity', 'regression')"
  --mapping-datafile FILE     File containing device to connection mapping
  --subsection-datafile FILE  File containing Common_(setup/cleanup)subsection information
  --debug-plugin FILE         File pointing to debug plugin
  --filetransfer-protocol     FILETRANSFER_PROTOCOL
                              File transfer protocol to be used in the run
  -devices DEVICES            List of devices to connect to

Genie Tgn arguments:
  --tgn-skip-configuration    TGN_SKIP_CONFIGURATION
                              Allows user to skip loading configuration onto TGN device if it has been configured prior to Genie run.
  --tgn-enable                Enable subsection 'initialize_traffic'.
                              Performs the following tasks:
                              * Connect to TGN device
                              * Load TGN configuration
                              * Learn TGN traffic streams
                              Create snapshot profile of traffic streams
                              Default value is False.
  --tgn-traffic-convergence-threshold TGN_TRAFFIC_CONVERGENCE_THRESHOLD
                              Wait time (seconds) to allow traffic streams to coverge to steady state.
                              Default value is 60 seconds.
  --tgn-reference-rate-threshold TGN_REFERENCE_RATE_THRESHOLD
                              Wait time (seconds) before checking traffic stream rates to create profile snapshot.
                              Default value is 100 seconds.
  --tgn-first-sample-threshold TGN_FIRST_SAMPLE_THRESHOLD
                              Wait time (seconds) before collecting the first sample of traffic stream rates.
                              Default value is 15 seconds.
  --tgn-disable-traffic-post-execution TGN_DISABLE_TRAFFIC_POST_EXECUTION
                              Allows user to stop traffic *AFTER* Genie has completed execution. This is useful for manual debugging after Genie runs complete.
                              Default value is False.
  --tgn-traffic-loss-recovery-threshold TGN_TRAFFIC_LOSS_RECOVERY_THRESHOLD
                              Wait time (seconds) for allowing traffic to recover to steady state AFTER a traffic loss was observed (say during trigger execution).
                              Default value is 5 seconds.
  --tgn-traffic-loss-tolerance-percentage TGN_TRAFFIC_LOSS_TOLERANCE_PERCENTAGE
                              Maximum allowable traffic loss percentage.
                              Default value is 15 percent.
  --tgn-enable-traffic-loss-check TGN_ENABLE_TRAFFIC_LOSS_CHECK
                              Enable checking of traffic loss after every trigger that is executed by Genie.
                              Default value is True.
  --tgn-config-post-device-config TGN_CONFIG_POST_DEVICE_CONFIG
                              Configure TGN device ONLY AFTER device configuration is successfully applied.
                              Default value is True.
  --tgn-profile-snapshot-threshold TGN_PROFILE_SNAPSHOT_THRESHOLD
                              Wait time (seconds) to collect reference rate while creating traffic snapshot profile.
                              Default is 1200 seconds.
  --tgn-routing-threshold TGN_ROUTING_THRESHOLD
                              Wait time (seconds) after enabling TGN routing engine and before starting traffic.
                              Default value is 120 seconds.
  --tgn-port-list TGN_PORT_LIST
                              Ports on the traffic generator device to connect to during Genie execution.
                              Default value is empty
  --tgn-arp-wait-time TGN_ARP_WAIT_TIME
                              Time to wait after sending ARP on interfaces before verifying ARP/ND successfully resolved.
                              Default value is 60 seconds
  --tgntcl-enable-arp TGNTCL_ENABLE_ARP
                              Send ARP to TGN device
  --tgntcl-learn-after-n-samples TGNTCL_LEARN_AFTER_N_SAMPLES
                              Create traffic profile after N number of samples
  --tgntcl-stream-sample-rate-percentage TGNTCL_STREAM_SAMPLE_RATE_PERCENTAGE
                              Specifies percentage tolerance that two samples of the same stream group must be within to be considered "the same"
  --tgntcl-wait-multiplier TGNTCL_WAIT_MULTIPLIER
                              Multiplier to increase the wait time for creating a traffic profile snapshot. This argument multiples the value of 'tgn_profile_snapshot_threshold'.

Running Triggers & Verifications

Let’s use genie run to execute a sample script containing 1 trigger and 1 verification. The linux CLI to do so would be:

genie run --testbed-file /path/to/testbed.yaml \
          --trigger-uids="TriggerShutNoShutBgp" \
          --verification-uids="Verify_BgpProcessVrfAll" \
          --devices nxos-osv-1 \
          -- html_logs .

Let’s examine each of those arguments above in more detail:

1. ‘–trigger-uids’ and/or ‘–trigger-groups’ arguments are used to specify which Trigger’s to execute from Genie. Complete list of Triggers available

to execute within Genie: Available Genie Triggers

When both ‘–trigger-uids’ and ‘–trigger-groups’ are provided, both will be used to elect triggers from trigger datafile.

For more information on how a Genie Trigger works, refer to: Genie Trigger Documentation

2. ‘–verification-uids’ and/or ‘–verification-groups’ arguments can be used to specify which Verification’s to execute from Genie. Complete list of Verifications available to execute within Genie: Available Genie Verifications

When both ‘–verification-uids’ and ‘–verification-groups’ are provided, both will be used to elect verifications from verification datafile.

For more information on how a Genie Verification works, refer to: Genie Verification Documentation

3. By default, genie run executes on all devices specified in the testbed YAML file. Use argumnet ‘–devices <name of device>’ to execute the Trigger’s and Verification’s on a specific device.

4. In order to create viewable logs, provide option ‘–html-logs .’ to specify that the logfile should be created in the directory we are running from.


  1. Ensure the device’s that we are exeucting triggers and verifications on have a connection marked as ‘cli’ within the testbed YAML. For more information, refer to the documentation on: Control Device Connections

  2. In case the device’s in the testbed YAML does not have any connections marked as ‘cli’, provide argument ‘–mapping-datafile’ containing the device mapping details to genie run. For more information, refer to the documentation on Mapping Datafile

Once genie run completes, you can see a table summarizing the results of the Trigger’s and Verification’s that were executed. The following is a sample:

2019-01-25T13:06:24: %EASYPY-INFO: +------------------------------------------------------------------------------+
2019-01-25T13:06:24: %EASYPY-INFO: |                                Easypy Report                                 |
2019-01-25T13:06:24: %EASYPY-INFO: +------------------------------------------------------------------------------+
2019-01-25T13:06:24: %EASYPY-INFO: pyATS Instance   : /ws/ellewoods/genie
2019-01-25T13:06:24: %EASYPY-INFO: Python Version   : cpython-3.4.1 (32bit)
2019-01-25T13:06:24: %EASYPY-INFO: CLI Arguments    : /ws/ellewoods/genie/bin/genie run --testbed-file /ws/ellewoods/genie/genie_testbeds/genie_n9kv_tt4.yaml --trigger-uids=And('TriggerSleep') --verification-uids=And('Verify_BgpProcessVrfAll$') -device uut --html-logs .
2019-01-25T13:06:24: %EASYPY-INFO: User             : ellewoods
2019-01-25T13:06:24: %EASYPY-INFO: Host Server      :
2019-01-25T13:06:24: %EASYPY-INFO: Host OS Version  : Red Hat Enterprise Linux Server 6.9 Santiago (x86_64)
2019-01-25T13:06:24: %EASYPY-INFO:
2019-01-25T13:06:24: %EASYPY-INFO: Job Information
2019-01-25T13:06:24: %EASYPY-INFO:     Name         : job
2019-01-25T13:06:24: %EASYPY-INFO:     Start time   : 2019-01-25 13:05:41.914951
2019-01-25T13:06:24: %EASYPY-INFO:     Stop time    : 2019-01-25 13:06:19.755553
2019-01-25T13:06:24: %EASYPY-INFO:     Elapsed time : 0:00:37.840602
2019-01-25T13:06:24: %EASYPY-INFO:     Archive      : /ws/ellewoods/genie/users/ellewoods/archive/19-01/
2019-01-25T13:06:24: %EASYPY-INFO:
2019-01-25T13:06:24: %EASYPY-INFO: Total Tasks    : 1
2019-01-25T13:06:24: %EASYPY-INFO:
2019-01-25T13:06:24: %EASYPY-INFO: Overall Stats
2019-01-25T13:06:24: %EASYPY-INFO:     Passed     : 5
2019-01-25T13:06:24: %EASYPY-INFO:     Passx      : 0
2019-01-25T13:06:24: %EASYPY-INFO:     Failed     : 0
2019-01-25T13:06:24: %EASYPY-INFO:     Aborted    : 0
2019-01-25T13:06:24: %EASYPY-INFO:     Blocked    : 0
2019-01-25T13:06:24: %EASYPY-INFO:     Skipped    : 0
2019-01-25T13:06:24: %EASYPY-INFO:     Errored    : 0
2019-01-25T13:06:24: %EASYPY-INFO:
2019-01-25T13:06:24: %EASYPY-INFO:     TOTAL      : 5
2019-01-25T13:06:24: %EASYPY-INFO:
2019-01-25T13:06:24: %EASYPY-INFO: Success Rate   : 100.00 %

2019-01-25T13:06:24: %EASYPY-INFO: +------------------------------------------------------------------------------+
2019-01-25T13:06:24: %EASYPY-INFO: |                             Task Result Summary                              |
2019-01-25T13:06:24: %EASYPY-INFO: +------------------------------------------------------------------------------+
2019-01-25T13:06:24: %EASYPY-INFO: Task-1: commons.commonSetup                                               PASSED
2019-01-25T13:06:24: %EASYPY-INFO: Task-1: base.Verify_BgpProcessVrfAll.uut.1                                PASSED
2019-01-25T13:06:24: %EASYPY-INFO: Task-1: sleep.TriggerSleep.uut                                            PASSED
2019-01-25T13:06:24: %EASYPY-INFO: Task-1: base.Verify_BgpProcessVrfAll.uut.2                                PASSED
2019-01-25T13:06:24: %EASYPY-INFO: Task-1: commons.commonCleanup                                             PASSED
2019-01-25T13:06:24: %EASYPY-INFO:
2019-01-25T13:06:24: %EASYPY-INFO: +------------------------------------------------------------------------------+
2019-01-25T13:06:24: %EASYPY-INFO: |                             Task Result Details                              |
2019-01-25T13:06:24: %EASYPY-INFO: +------------------------------------------------------------------------------+
2019-01-25T13:06:24: %EASYPY-INFO: Task-1: commons
2019-01-25T13:06:24: %EASYPY-INFO: |-- commonSetup                                                           PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   |-- connect                                                           PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   |-- configure                                                        SKIPPED
2019-01-25T13:06:24: %EASYPY-INFO: |   |-- configuration_snapshot                                            PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   |-- save_bootvar                                                      PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   |-- learn_system_defaults                                             PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   `-- initialize_traffic                                               SKIPPED
2019-01-25T13:06:24: %EASYPY-INFO: |-- Verify_BgpProcessVrfAll.uut.1                                         PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   `-- verify                                                            PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |-- TriggerSleep.uut                                                      PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   `-- sleep                                                             PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |-- Verify_BgpProcessVrfAll.uut.2                                         PASSED
2019-01-25T13:06:24: %EASYPY-INFO: |   `-- verify                                                            PASSED
2019-01-25T13:06:24: %EASYPY-INFO: `-- commonCleanup                                                         PASSED
2019-01-25T13:06:24: %EASYPY-INFO:     |-- verify_configuration_snapshot                                     PASSED
2019-01-25T13:06:24: %EASYPY-INFO:     `-- stop_traffic                                                     SKIPPED

genie run will also create a logfile named “TaskLog.html” that contains the complete logs of all the Trigger’s and Verification’s executed and are viewable in any browser.

The following is a sample log report:

Sample log

Recording with Genie

genie run in combination with Unicon Recording is capable of recording and saving all interactions with any device.

Simply provide argument ‘–record’ to any genie CLI command and the path to the directory into which recordings should be saved. Unicon will record all interactions with the device and create a pickle file under the directory specified. This file can later be unpicked using genie unpickle.

Let’s look at an example of using Unicon record with genie learn:

(genie) bash-4.1$ genie run --testbed-file /path/to/testbed.yaml \
                            --trigger-uids="TriggerShutNoShutBgp" \
                            --verification-uids="Verify_BgpProcessVrfAll" \
                            --devices nxos-osv-1 \
                            --html_logs . \
                            --record recorded_session

The above genie run command will do exactly as above however, it will record all device interactions into the directory specified in ‘–record’.

Recording device interacations is an extremely useful feature! It enables users to easily reproduce complicated issues observed on devices such as unexpected device outputs or crashes/traceback triggers etc

Replaying with Genie

genie run in combination with Unicon Replay is capable of replaying a saved Unicon recording to mimic all interactions with any device.

Simply provide argument ‘–replay’ to any genie CLI command and the path containing the saved pickled Unicon recording. Unicon will then replay the contents of the pickled file as a Mock device providing inputs to prompts as needed.

Let’s look at an example of using Unicon replay with genie learn:

(genie) bash-4.1$ genie run --testbed-file /path/to/testbed.yaml \
                            --trigger-uids="TriggerShutNoShutBgp" \
                            --verification-uids="Verify_BgpProcessVrfAll" \
                            --devices nxos-osv-1 \
                            --html_logs . \
                            --replay recorded_session

The above genie run command will do exactly as above however, it will replay all device interactions from the directory specified in ‘–replay’ instead of executing commands on actual device.

Once again, replaying a device recording is an extremely useful functionality in network automation! It significantly improves the ease of debugging issues reported on network device’s. Users can now simply request a recording of any issue and replay it to reproduce an issue for debugging.

For more information on the Unicon.playback package refer to the Unicon Playback Documentation


Ensure your python virtual environment has Unicon version 3.4.6 or higher installed, in order to use the Unicon.playback record/replay features.