Skip to main content

Configuration as a graph

I had the privilege to be part of the team that built Juniper’s EX switches; it was a fun experience as building something from scratch typically is. One episode I vividly remember is a week long discussion as to whether the new switch software should, in its configuration, list the interfaces and the features enabled in them (as Cisco devices typically do); or be driven by the feature configuration and list the interfaces the feature is enabled on (as juniper routers typically do).

On one side of the argument, listing the features under the interfaces can give one an immediate outlook of what to expect for that interface; besides if one is building a switch it is hard to ignore Cisco’s enterprise market share and how familiar networking people are with that model. On the other side of the argument, Juniper’s configuration model allows the operator to quickly understand what is going on for each feature: a protocol such as OSPF for instance.

It is very easy for people to argue for hours about such a simple thing; one can’t conclusively prove either viewpoint to be right or wrong. After a few meetings with heated arguments we ended up with a compromised solution: we would do both. Switching functionality would be configurable under the interface as well as under a configuration stanza specific to the feature. Once the product shipped, we ended up hearing many comments on how it was confusing to have both ways to do it. I’m convinced we would have ended up with a similar number of complaints had we chosen one approach or the other. There are scenarios where there simply is no perfect answer.

I’m quite happy with the fact that on my current project (http://www.opencontrail.org) we have neither model. Instead configuration is expressed as a graph of objects. So far i really like the outcome.

When we started working on network virtualization at contrail we took the approach that we should avoid inventing new things as much as possible. For configuration management we wanted a standards based approach that could allow for multiple producers to contribute information to as well as multiple consumers. We were looking for interoperability and we knew that we needed to have a configuration model that could be sharable by multiple independent systems.

The only reusable tool we found was IF-MAP. IF-MAP was originally targeted at the “identity” space. In a campus network, it is often desirable to be able to associate an IP address with the corresponding user-id. Users can enter the network from a wiring-closet switch, wireless or VPN concentrator. All devices that thread authentication differently and for which enterprise wants the ability to choose different vendors. IF-MAP has been quite successful in this space, supporting not just IP and user association but the ability to provide additional properties and policy configuration.

In the network virtualization space, the data-model is necessarily different. The concepts however are very similar. In on-demand infrastructure deployments the primary entities are virtual-machines and their respective interfaces vs users and devices. But similarly, interfaces can be associated with virtual-networks and there is the need to express the desired outcome as policy statements, for example: a VM in network blue can establish an http session with a VM on network red.

In the OpenContrail system, virtual-machine, interfaces and virtual-networks are IF-MAP identifiers. IF-MAP “metadata” can be associated with either an identifier or with two identifiers forming an edge in the graph. We use the metadata, for instance, in order to define the policy that determines how traffic can flow between networks. Two virtual-networks can be connected by metadata that defines what traffic is allowed to flow. From that information we later extract both access-control lists that apply to a specific “port” as well as the reachability information to be presented to a VM interface.

The data-model is something that keeps expanding as the functionality of the system increases; as such one needs to automate as much as possible the mechanical process of adding new attributes or identifiers. For that reason we built a code generator that takes as input an XML schema definition file and can output that code that is necessary to:

– Provide a REST API for the data-model in;

– Decode the corresponding IF-MAP objects.

In terms of the REST API, the generator is able to produce both python and Java client libraries as well as the server side code (in python). The server code that is manually coded is agnostic to the data model and capable of publishing objects to an IF-MAP server.

The generator also produces a C++ library that can decode the IF-MAP messages and generate C++ objects; another library which is agnostic to the data-model can provide a subset of the graph to specific clients. It is used to determine the minimal subset of the configuration graph that should be made available to the contrail vrouter agents.

These tools are quite independent of the XSD schema that one wishes to feed to IF-MAP. They can be useful to anyone else that is tired of arguing how the hierarchy should look like in a traditional configuration file approach in a world that is increasingly interconnected.

– Pedro Marques