Automatic Descriptions

This module creates descriptions of C++ classes from source code, by using external parsers (GCC-XML, Clang AST) and the type system.

author:Anthony Scopatz <scopatz@gmail.com>

Descriptions

A key component of API wrapper generation is having a a top-level, abstract representation of the software that is being wrapped. In C++ there are three basic constructs which may be wrapped: variables, functions, and classes. Here we restrict ourselves to wrapping classes (though ironically these are the most complex of the three).

The abstract representation of a C++ class is known as a description (abbr. desc). This description is simply a Python dictionary with a specific structure. This structure makes heavy use of the type system to declare the types of all needed parameters.

Top-Level Keys

The following are valid top-level keys in a description dictionary: name, parents, namespace, attrs, methods, docstrings, and extra.

name:str, the class name parents:list of strings, the immediate parents of the class (not grandparents) namespace:str or None, the namespace or module the class lives in. attrs:dict or dict-like, the names of the attributes (member variables) of the class mapped to their types, given in the format of the type system. methods:dict or dict-like, similar to the attrs except that the keys are now function signatures and the values are the method return types. The signatures themselves are tuples. The first element of these tuples is the method name. The remaining elements (if any) are the function arguments. Arguments are themselves length-2 or -3 tuples whose first elements are the argument names, the second element is the argument type, and the third element (if present) is the default value. If the return type is None (as opposed to ‘void’), then this method is assumed to be a constructor or destructor. docstrings:dict, optional, this dictionary is meant for storing documentation strings. All values are thus either strings or dictionaries of strings. Valid keys include: module, class, attrs, and methods. The attrs and methods keys are dictionaries which may include keys that mirror the top-level keys of the same name. extra:dict, optional, this stores arbitrary metadata that may be used with different backends. It is not added by any auto-describe routine but may be inserted later if needed. One example use case is that the Cython generation looks for the pyx, pxd, and cpppxd keys for strings of supplemental Cython code to insert directly into the wrapper.

Toaster Example

Suppose we have a C++ class called Toaster that takes bread and makes delicious toast. A valid description dictionary for this class would be as follows:

desc = {
    'name': 'Toaster',
    'parents': ['FCComp'],
    'namespace': 'bright',
    'attrs': {
        'n_slices': 'int32',
        'rate': 'float64',
        'toastiness': 'str',
        },
    'methods': {
        ('Toaster',): None,
        ('Toaster', ('name', 'str', '""')): None,
        ('Toaster', ('paramtrack', ('set', 'str')), ('name', 'str', '""')): None,
        ('~Toaster',): None, 
        ('tostring',): 'str', 
        ('calc',): 'Material',
        ('calc', ('incomp', ('map', 'int32', 'float64'))): 'Material',
        ('calc', ('mat', 'Material')): 'Material',
        ('write', ('filename', 'str', '"toaster.txt"')): 'void',
        ('write', ('filename', ('char' '*'), '"toaster.txt"')): 'void',
        },
    'docstrings': {
        'module': "This is where Toaster lives.",
        'class': "I am a toaster!",
        'attrs': {
            'n_slices': 'the number of slices',
            'rate': 'the toast rate',
            'toastiness': 'the toastiness level',
            },
        'methods': {
            'Toaster': "Make me a toaster!",
            '~Toaster': "Noooooo",
            'tostring': "string representation of the toaster",
            'calc': "actually makes the toast.",
            'write': "persists the toaster state."
            },
        },
    'extra': {
        'pyx': 'toaster = Toaster()  # make toaster singleton'
        },
    }

Automatic Description Generation

The purpose of this module is to create description dictionaries like those above by automatically parsing C++ classes. In theory this parsing step may be handled by visiting any syntax tree of C++ code. Two options were pursued here: GCC-XML and the Python bindings to the Clang AST. Unfortunately, the Clang AST bindings lack exposure for template argument types. These are needed to use any standard library containers. Thus while the Clang method was pursued to a mostly working state, the GCC-XML version is the only fully functional automatic describer for the moment.

Automatic Descriptions API

class bright.apigen.autodescribe.ClangTypeVisitor(verbose=False)

For a Clang type located at a root node, compute the cooresponding typesystem type.

visit(root)

Takes a root type.

class bright.apigen.autodescribe.GccxmlClassDescriber(classname, root=None, onlyin=None, verbose=False)

Class used to generate descriptions via GCC-XML output.

Parameters :

classname : str

The classname, this may not have a None value.

root : element tree node, optional

The root element node of the class or struct to describe.

onlyin : str, optional

Filename the class or struct described must live in. Prevents finding classes of the same name coming from other libraries.

verbose : bool, optional

Flag to display extra information while visiting the class.

context(id)

Resolves the context from its id and information in the element tree.

type(id)

Resolves the type from its id and information in the root element tree.

visit(node=None)

Visits the class node and all sub-nodes, generating the description dictionary as it goes.

Parameters :

node : element tree node, optional

The element tree node to start from. If this is None, then the top-level class node is found and visited.

visit_argument(node)

visits a constructor, destructor, or method argument.

visit_arraytype(node)

visits an array type and maps it to a ‘*’ refinement type.

visit_base(node)

visits a base class.

visit_class(node)

visits a class or struct.

visit_constructor(node)

visits a class constructor.

visit_destructor(node)

visits a class destructor.

visit_field(node)

visits a member variable.

visit_fundamentaltype(node)

visits a base C++ type, mapping it to the approriate type in the type system.

visit_method(node)

visits a member function.

visit_namespace(node)

visits the namespace that a node is defined in.

visit_pointertype(node)

visits a pointer and maps it to a ‘*’ refinement type.

visit_referencetype(node)

visits a refernece and maps it to a ‘&’ refinement type.

visit_struct(node)

visits a class or struct.

visit_typedef(node)

visits a type definition anywhere.

bright.apigen.autodescribe.clang_describe(filename, classname, verbose=False)

Use clang to describe the class.

bright.apigen.autodescribe.clang_find_attributes(node)

Finds attributes one level below the Clang node.

bright.apigen.autodescribe.clang_find_class(node, classname, namespace=None)

Find the node for a given class underneath the current node.

bright.apigen.autodescribe.clang_find_declarations(node)

Finds declarations one level below the Clang node.

bright.apigen.autodescribe.clang_is_loc_in_range(location, source_range)

Returns whether a given Clang location is part of a source file range.

bright.apigen.autodescribe.clang_range_str(source_range)

Get the text present on a source range.

bright.apigen.autodescribe.describe(filename, classname=None, parser='gccxml', verbose=False)

Automatically describes a class in a file. This is the main entry point.

Parameters :

filename : str

The path to the file.

classname : str or None, optional

The classname, a ‘None’ value will attempt to infer this from the filename.

parser : str, optional

The parser / AST to use to use for the C++ file. Currently only ‘clang’ and ‘gccxml’ are supported, though others may be implemented in the future.

verbose : bool, optional

Flag to diplay extra information while describing the class.

Returns :

desc : dict

A dictionary describing the class which may be used to generate API bindings.

bright.apigen.autodescribe.gccxml_describe(filename, classname, verbose=False)

Use GCC-XML to describe the class.

Parameters :

filename : str

The path to the file.

classname : str or None, optional

The classname, a ‘None’ value will attempt to infer this from the filename.

verbose : bool, optional

Flag to diplay extra information while describing the class.

Returns :

desc : dict

A dictionary describing the class which may be used to generate API bindings.

bright.apigen.autodescribe.merge_descriptions(descriptions)

Given a sequence of descriptions, in order of increasing precedence, merge them into a single description dictionary.