1 Introduction
This is a small introduction to service VM’s. On what the minimum requirements are to build them, and how to test them in a Fabric. Some people use other names like Service-extraction, Service-chaining or Network Function Virtualisation (NFV) when they mean the whole construct. In this article, we’ll focus on what has to be done to create service VM’s, the extracted service that get’s this traffic and processes it, and how you can do simple tests with it.
An additional focus will be on the creation and testing of Layer 2 Forwarding Service Images since this is a bit more challenging compared to build-in a ‘traditional’ Layer 3 Forwarding Service Image.
2 Types of Service VM’s
There are different types of service VM’s depending on what Layer 4 to Layer 7 service will be offered. When you build these VM’s, you must know how they fit into the network and how data traffic is passed through them for inspection or manipulation.
2.1 Passive Collection of Data traffic or monitoring
A monitoring service VM get’s a configured amount of the traffic between two networks for analysis. The traffic in OpenContrail is selected by a Policy Rule, which can also be a part of a traffic analysis/debugging that resides inside the fabric. The rule may only select a portion of this traffic to be inspected by the service VM. The service VM itself does NOT manipulate or insert anything into the Data Traffic. It just observes the traffic; usually the interface is in a kind of Promiscuous Mode, and then can report events via the management interface to higher entities. For example, in sophisticated environments, the service VM can directly re-program the policies via the OpenContrail Controller API’s, based on an event seen, to stop malicious traffic.
2.2 Service VM in-line with Data traffic
Quite often you see Service VM’s that are in-line with the Data Traffic. This allows them to apply all kinds of manipulation of the Data traffic. Here are just a few of those applications that are in-line with the Data traffic:
- Caching Services
- Firewalling Services
- Tunnel Termination Service (mainly VPN-GWs of all types)
- Traffic Shaping Services
- Load-Balancing Services
- Layer 7 inspection and Manipulation Services
- HTTP Header
- SQL Injection prevention
- DPI
When you write these applications they usually have TWO or more interfaces to work within their environment. The Data Traffic is then manipulated or forwarded between those two interfaces. Should the Service Image for some reason NOT process the traffic then it is dropped or lost inside the Service VM, which it is not the case with the mirroring option but may be a desired behavior.
2.3 Traffic forwarding Types
2.3.1 Layer 3 forwarding
Layer 3 forwarding is probably the most used option as a service VM. The service VM needs two interfaces; say eth0 and eth1; and should configure the interfaces via DHCP as with every other application that has usually just one interface into the network.
If the Service Guest is run on Linux as OS, then the only configuration option for a test-image (apart from the interface configuration) is to enable Kernel Forwarding. Below is a command that can be issued on a running VM or as part of a Cron-Job or /etc/rc.local configuration.
echo 1 > /proc/sys/net/ipv4/ip_forward
2.3.2 Layer 2 forwarding
Certain Service VM’s will not be able to do Layer 3 forwarding and will act as a Layer2 Bridge only. Which means all Forwarding they are doing is based on Layer 2 MAC basis and not on Layer 3 IP-Addresses. There are a number of reasons why a Service VM should do this. Mainly it’s about speed and simplicity of a Service or this is a Service that today exists in a Bump-In-The-Wire Appliance and is now hosted in a virtual environment.
2.4 Optional management interfaces
Management Interfaces are meant to provide an out-of-bound management of the Service VM. In some cases the Service VM has a fixed behaviour that is build-in with and started as soon as the orchestration system launches the VM. In those cases, a management interface may not be needed. On the other hand if the Service VM needs to communicate with higher management entities in the network such as OSS/BSS, then you would certainly want to configure such an interface.
3 Building a simple image for an Layer 2 in-line Service VM
To build a Service Image for a Layer 2 forwarding VM we recommend building a regular Linux Image on KVM with virtio-support. This allows you to check the service in the same way it would later be used in OpenContrail.
In the example below we use a computer with three Ethernet NICs. The first eth0 will be used to access the system but is not mapped into the virtual machine. The Ethernet NICs eth1 and eth2 will be set into bridge mode and transparently appear in the Guest VM. This allows us to attach a regular service to these two interfaces before we load it into OpenContrail.
3.1 Building up the KVM install environment
Below are some simple instructions to install a KVM environment that allows you to install a VM into the KVM environment. On a desktop machine for a developer, this may have occurred in the past. Note the importance of installing the bridge-utils
yum -y install qemu-kvm libvirt python-virtinst
yum -y install virt-viewer virt-manager
yum -y groupinstall "Desktop" "Desktop Platform" "X Window System" "Fonts"
yum -y install bridge-utils tunctl
vi /etc/libvirt/qemu.conf
vnc_listen = "0.0.0.0"
user = "root"
group = "root"
dynamic_ownership = 1
service libvirtd restart
3.2 Setting up local bridging
The next step is to setup a local bridging to make the two interfaces eth1 and eth2 appear in the Guest VM so that we can use them.
brctl addbr br0
brctl addbr br1
brctl addif br0 eth1
brctl addif br1 eth2
brctl show
[root@sdn-cfgm ~]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.5cf3fcb79872 no eth1
br1 8000.5cf3fc6a633c no eth2
virbr0 8000.525400a2cfa0 yes virbr0-nic
3.3 Create and install the VM from ISO
Below are instructions on how to create the image and start the installation process from an ISO in KVM. Our Image is called ‘guest’. Please review the network mapping.
qemu-img create -f qcow2 /var/lib/libvirt/images/guest.img 20G
virt-install --connect qemu:///system -n guest -r 4096 --vcpus=2 --disk path=/var/lib/libvirt/images/guest.img,format=qcow2,size=20,device=disk,bus=virtio -c /home/linux.iso --graphics vnc,listen=0.0.0.0 --os-type linux --hvm --network=bridge:br0,model=virtio --network=bridge:br1,model=virtio
Starting install...
Creating domain... | 0 B 00:00
Cannot open display:
Run 'virt-viewer --help' to see a full list of available command line options
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
[root@sdn-cfgm ~]# virsh
Welcome to virsh, the virtualization interactive terminal.
Type: 'help' for help with commands
'quit' to quit
virsh # list --all
Id Name State
----------------------------------------------------
2 guest running
At this point, the installation usually requires commands to choose the installation type and packages and so on. This is represented with a graphical view via the build-in VNC-Server that was also installed as part of the X-Windows installation. Please use a VNC-Client to finish the installation. With only one VM running, it’s usually window “:0” that has a view of this VM.
3.4 Setup a simple Bridge inside a VM for testing
Configuration of the services and of the interfaces inside the VM is left to the one developing the service and building the image. Below, we use a simple install of the (again) needed ‘bridge-utils’ and then we simply re-write /etc/rc.local with the shell commands that configures this bridge after the VM was started. You can do similar things for your own test-images before you start developing the Service.
yum -y install bridge-utils
cat </etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don’t
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
ifconfig eth0 up
ifconfig eth1 up
brctl addbr mybridge
brctl addif mybridge eth0
brctl addif mybridge eth1
ifconfig eth0 0.0.0.0
ifconfig eth1 0.0.0.0
ifconfig mybridge up
EOF
3.5 Finishing the Installation
Please make sure that you gently halt the VM before you put it into production. Use “shutdown” and not “destroy” in virsh and wait until the VM is stopped.
There have been cases where the VM was not able to delete the file “/etc/udev/rules.d/70-persistent-net.rules”. This is especially the case when you use the un-gentle “destroy” command in virsh. When you then move this Image to an orchestration system (such as CloudStack or OpenStack) the MAC addresses of your Guest Interfaces change to the dynamic ones that are present inside the Guest. The original MAC addresses of the real interfaces we used to create the Image and test it are no longer available. An existing “/etc/udev/rules.d/70-persistent-net.rules” file that was not deleted may look like the previous interface name eth0 and eth1 inside the guest to the original MAC . This will result in those Interfaces names no longer being available and having the OS use new ones like eth2 and eth3. To avoid this (especially if you have hardcoded scripts like the one above), you need to delete the file from the image before you read it into your orchestration system.
Below we use ‘guestfish’ to mount the image and delete the file. You can also use this method to make small changes to the configuration of the image later on via a simple text editor.
yum -y install guestfish
guestfish --rw -a /var/lib/libvirt/images/guest.img
> run
> list-filesystems
/dev/vda1: ext3
/dev/vda2: ext4
/dev/vda3: swap
/dev/vda4: unknown
/dev/vda5: ext4
> mount /dev/vda2 /
> rm /etc/udev/rules.d/70-persistent-net.rules
> exit
The next step is an example to read -in the Image into OpenStack orchestration via CLI
source /etc/OpenContrail/openstackrc; glance add name='myservice' is_public=true container_format=ovf disk_format=qcow2
< /var/lib/libvirt/images/guest.img
4 Testing the Layer2 Service Image
We’ll now do a test of the new Layer 2 service into an OpenContrail environment to see how it works. To make this process easier, we use the same structure as above and use the OpenContrail GUI to test and make the relevant setup changes.
4.1 Setup the Virtual Networks
First we create two internal virtual networks that are on the right and left side of the Service VM.
4.2 Create a Service-Template
Now we need to create the Service Service-Template. It’s important to select Service-Mode=”Transparent” because this is the indicator for the system to know that this is a Layer 2 Service Image. The Service Type remains “firewall” indicating a Service VM that is in-line with the Data traffic.
4.3 Launch a Service Image
Now we are launching our Service Image to be used.
In this step, it is VERY IMPORTANT to let the system use “Auto Configured” Networks because it needs to change the default behaviour of the vRouters at both ends in this network.
The default is that the vRouter uses the same MAC-Address towards any Guest VM regardless if it is a service VM or a normal application VM. The vRouter always has the MAC Address 00:01:00:5e:00:00 (which is IP-Multicast) that it uses towards the VM.
This design decision makes configuration easy if you think about things like mobility of a VM. No matter where you hold a running VM, copy it’s RAM and disk to a new server and spin it up again, the local vRouter will have the same local MAC address, which means that the migrated VM would never see any difference on the new server even if it was in a remote Data Center. The vRouter always shields the physical network from the Guest VM.
In this special environment we must change the behaviour of the vRouters. You need different MAC addresses at both ends; otherwise, the service VM might treat this as a Loop and no Traffic would flow. Choosing Auto-Configure lets the vRouter use a new MAC address as indicated in the capture below.
Now the Orchestration System will start the Service VM
Ignore the IP-Address indicated as assigned to the VM. This may happen when the service VM ask for them via DHCP but this usually does not happen for Layer 2 Services.
4.4 Create a Policy
Now we need to create a Policy. This policy is similar to a Layer 3 service VM policy. You define the two networks Source/Destination for being left and right (don’t use the Auto-Configured Networks here). Check “Apply-Service” and select the running instance as below.
4.5 Assign Policy to Network
Now you need to assign the policy to the networks. Below you also see also the new networks created via the Auto-Configure option. Just edit the old (left/right) networks to apply the policy.
At this point the network configuration is ready to be tested.
4.6 Test the Service Image
Here we use OpenStack as the orchestration System. After we log into the project, we see the Service VM running on the Horizon Dashboard
For this demo, we just used an ordinary server image which has added bridge-utils support. After the orchestration system has started it, we need to login and manually configure, the same way as above, the Bridge.
Now we need to start two regular VM’s at both ends to be able to Ping each other through the Bridge Service.
After the two VM’s for our Test are started, just notify the two IP-Addresses they have to be able to ping each other.
Now we login to the left VM in the left network and ping the right-VM
The last thing is to check what we see in the Service VM. As you see below, our MAC based service runs perfectly and forwards traffic between both interfaces.
Now the rest is up for you to explore what new or existing services could be put in-between.
