Q

In previous sections we covered how to natively access Tcl, read/set variables, typecast variables, and making API calls. However, all of that was entirely based on “evaluating” Tcl code in Python. That is: evaluating each line of Tcl in its string format, and somehow typecasting the returning string back to Python objects.

There’s a certain degree of frustration with that. Yes, it is natively Tcl, yes it allows you to do anything and everything, but because the code to be evaluated is a string, when building the line, you must obey Python’s string formatting rules & use string substitution and such.

The Quartermaster

tcl module offers a magic quartermaster function that allows users to make Tcl calls as if they were Python objects, allowing object attribute chaining and support for mapping Python style arguments args``and``kwargs to Tcl’s positional and dashed argument styles.

../_images/q.jpg

A tribute to Desmond Llewelyn, Q from 1963 to 1999.

The magic Q function is built-in to each Interpreter object instance and can be accessed via either the q``or``Q attribute.

# Example
# -------
#
#   how to make fun of Q, 007 style.

from pyats import tcl

# create some procs/namespaces for use during this example
tcl.eval('''
    # for testing casting
    keylset klist a "some value"
    keylset klist b "more values"
    keylset klist c.d.e "nested keys"

    # for testing positional arguments
    proc testPositionalArgs {a b c} {
        return [list $a $b $c]
    }

    # for testing dashed arguments
    proc testDashedArgs {args} {
        return $args
    }

    # for testing mixed style arguments
    proc testMixedArgs {a b args} {
        return [list $a $b $args]
    }

    # for nesting namespaces
    namespace eval ::a {
        namespace eval b {
            proc c {args} {
                return $args
            }
        }
    }
''')

# what is this... Q?
tcl.q
# <pyats.tcl.Q magic, referring Tcl code>

# q is Q - the lowercase was only created to avoid pressing shift :)
assert tcl.q is tcl.Q
True

# setting and getting variables
tcl.q.set('myVar', 1)
ret = tcl.q.set('myVar')

# Q always performs a typecast (using Interpreter.cast_any) on the return
tcl.q.set('myVar')
1
tcl.q.set('klist')
KeyedList({'b': 'more values',
           'c': KeyedList({
                'd': KeyedList({
                    'e': 'nested keys'
                })
            }),
           'a': 'some value'})


# calling namespace APIS
# notice the chaining in arguments
result = tcl.q.a.b.c()

# calling with positional arguments
tcl.q.testPositionalArgs("pos 1", "pos 2", "pos 3")
# {pos 1} {pos 2} {pos 3}

# calling with kwargs: they get mapped to dashed args
tcl.q.testDashedArgs(arg_a = 'a string', arg_b = 1, arg_c = 1)
# -arg_a {a string} -arg_b 1 -arg_c 1

# calling with mixed arguments
tcl.q.testMixedargs('position 1',
                    'position 2',
                    arg_a = 'a string', arg_b = 1, arg_c = 1)
# {position 1} {position 2} {-arg_b 1 -arg_c true -arg_a {a string}}

Regulations

The Q mechanism works by converting Python calls into specific Tcl syntax. The following rules are followed:

  • Q calls are relative to the current Tcl namespace (usually global ::).

    # refer to Tcl's set command (built-in)
tcl.q.set

  • absolute namespaces in Tcl are represented as Q attribute chains.

    # refers to Tcl ::a::b::c procedure
tcl.q.a.b.c

  • call procedure as if you are making a Python call using ().

    # making a call
tcl.info('patchlevel')

  • Python *args arguments are converted into Tcl positional arguments, respecting the original argument position.

    # notice that set takes two arguments: name and value
tcl.q.set('varName', 'varValue')

  • Python **kwargs keyword arguments are converted into Tcl dashed arguments. Order is not preserved, as dashed arguments are not order-sensitive.

    # representation -a 1 -b 2 dashed arguments
tcl.q.testProc(a = 1, b = 2)

  • When *args``and``**kwargs are used together, positional arguments comes first, keywords/dashed arguments comes last.

  • All arguments are converted from Python objects to Tcl string forms using tclstr casting API. See data casting section for details.

Auto-Cast

By default, all Q function calls are always casted into Python objects using Interpreter.cast_any functionality. This allows maximum python-to-python look & feels. This functionality can be turned off using cast_ = False argument.

# Example
# -------
#   Q casting on/off

from pyats import tcl.

tcl.eval('set myVar 1')

# by default, Q casting is always on:
type(tcl.q.set('myVar'))
# <type 'int'>

# turn it off using cast_ argument
type(tcl.q.set('myVar', cast_ = False))
# <type 'str'>

Note

technically, cast_ uses off a “potential” argument that can be sent to the actual function called by Tcl. Thus, a trailing _ is added in order to minimize the chance of such collisions. If you do actually find a case where a Tcl API has an argument named cast_, please let us know.

Limitations

There are still some fundamental differences between Tcl and Python syntax, leading to corner conditions that cannot be handled even by the all-mighty Q.

  • Python identifiers are limited to A-Z,``a-z``,``0-9``and``_``. Thus, any procedure/namespace names with characters outside of this allowed set cannot be called with Q function.

  • Any Tcl procedure/namespaces with names the same as Python reserved keywords, statements and operators cannot be called. Eg: pass,``return``,``def``.

# Example
# -------
#
#   some example that cannot be called with Q

from pyats import tcl

tcl.eval('''
    proc procedureWith:Colon {} {}

    proc procedureWith-Dash {} {}

    namespace eval namespaceWith/\Slash {} {}

    # the null procedure
    proc {} {} {}
''')

# try to call them
tcl.q.procedureWith:Colon()
#   File "<stdin>", line 1
#     tcl.q.procedureWith:Colon()
#                        ^
# SyntaxError: invalid syntax

tcl.q.procedureWith:Colon
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# NameError: name 'Dash' is not defined

tcl.namespaceWith/\Slash
#   File "<stdin>", line 1
#     tcl.namespaceWith/\Slash
#                            ^
# SyntaxError: unexpected character after line continuation character

# and I honestly don't know how to call the null function in Python
# ...

The only workaround for the above limitations is to continue using the basic Interpreter.eval() method, and evaluate them as strings.

# continuing from the above example ...

tcl.eval('procedureWith:Colon')

tcl.eval('procedureWith-Dash')

tcl.eval('namespaceWith/\Slash')

tcl.eval('{}')

Hint

even though Tcl allows for special characters in procedure/namespace names, it is still considered extremely bad practice. Don’t do it.

Examples

Here’s some practical day-to-day usage examples using Q magic.

# Example
# -------
#
#   some actual examples of how to abuse Q
#   (don't forget to return his gadgets in a broken state)

import os
from pyats import tcl

# source some files
tcl.q.source(os.path.join('path','to','lib.tcl'))

# delete an array element
tcl.q.unset('myArray(index)')

# list appends
tcl.q.lappend('myList', 'value 1', 'value 2')

# keyed list operations
tcl.q.keylset('myKlist', 'key_a', 'value', 'key.subkey', 'value')

# load up some packages
tcl.q.package('require', 'Expect')