Tuesday 15 September 2015

KVM Revisted Part I - Multiple VLANs

I've decided to rebuild my KVM lab on OpenSUSE 13.2 and to try and finish documenting the process of a fully functional KVM lab on multiple VLANs.

Install KVM with yast2
I used the "Install Hypervisor and Tools" in yast2 to install KVM.  I also allowed it to create the bridge (br0)

Access with virt-manager
This is the same as the steps I outlined in my previous post  http://pyn00b.blogspot.co.uk/2014/01/opensuse-kvm-getting-started-notes.html - just make sure to add yourself to the libvirt group and to restart the libvirtd.service

Multiple VLANs
The objective here is to have a single KVM host with a single Ethernet NIC to have multiple and routable segregated subnets. This is achieved by Bridging and VLAN tagging. We do this so that we can use the existing network infrastructure rather than using the KVM host to do any NATing or using KVM virtual networks.

Our solution is to have a single NIC, to which we create a VLAN interface, a Bridge is created from the VLAN interface and KVM attaches virtual NICs to the Bridge. The bridge is specified when creating a host and is therefore attached to the VLAN. This can be repeated multiple times for any number of VLANs. For example, a hierarchy of network devices might look like this:

Physical Nic (enp3s0) -> VLAN interface (enp3s0.69) -> Bridge (br69) -> vnet0
Physical Nic (enp3s0) -> VLAN interface (enp3s0.69) -> Bridge (br69) -> vnet1

Physical Nic (enp3s0) -> VLAN interface (enp3s0.101) -> Bridge (br101) -> vnet2

So in summary, for each new VLAN, a new VLAN interface is created with a VLAN tag, that interface is then Bridged and any number of KVM hosts attach their virtual NIC to the bridge and their traffic is VLAN tagged transparently.

Setting up VLAN interfaces
Let's start by creating an virtual interface that will tag traffic for VLAN 69.  We do so by creating the file /etc/sysconfig/network/ifcfg-enp3s0.69

BOOTPROTO='none'
STARTMODE='auto'
ETHERDEVICE='enp3s0'

Notice that by using the format ifcfg-enp3s0.VLANID sets the VLAN tag to 69 for us.  You can use VLAN_ID= as an option instead if you want to.  (see man ifcfg-vlan)

Also worth noting is that I have set BOOTPROTO='none' rather than 'static' and using IPADDR=''.  This is somewhat in contradiction to man ifcfg which states "Do  not  use  (none) to  just  skip  the IP setup".  However I noticed that when yast2 creates the bridge during the KVM install it sets BOOTPROTO='none'.  In my testing either option had the same result.  I used 'none' for consistency.  The yast convention is also to create ifcfg-vlanX which I did not care for.

Setting up Bridges
We now need to created a bridge from our VLAN interface.  We do so by creating /etc/sysconfig/network/ifcfg-br69

BOOTPROTO='none'
BRIDGE='yes'
BRIDGE_FORWARDDELAY='0'
BRIDGE_PORTS='enp3s0.69'
BRIDGE_STP='off'
STARTMODE='auto

Changes to sysctl
Disable any traffic over the bridges getting sent to iptables ( http://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf )

I created  /etc/sysctl.d/kvm.conf and reloaded the config with sudo sysctl --system

net.ipv4.ip_forward = 0
net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0

Checking config
I initially had issues getting wicked to work properly with the VLANs (see https://bugzilla.suse.com/show_bug.cgi?id=939580) I upgraded wicked from OBS and it was better after that although using ifup was still unreliable.  systemctl restart network.service works much better for now.

It turns out wicked network manager has some nice features too.  I particularly like this one as an alternative to ifconfig:

$ sudo wicked ifstatus all
...
 br69            up
      link:     #8, state up, mtu 1500
      type:     bridge
      config:   compat:suse:/etc/sysconfig/network/ifcfg-br69
...
 enp3s0.69       enslaved
      link:     #11, state up, mtu 1500, master br69
      type:     vlan enp3s0[69], hwaddr 67:89:d5:65:1e:66
      config:   compat:suse:/etc/sysconfig/network/ifcfg-enp3s0.69

vnet1           device-unconfigured
      link:     #12, state up, mtu 1500, master br69
      type:     tap, hwaddr fe:54:00:ac:0a:29

Monday 14 September 2015

Setting Up VLANs and DHCP server in ScreenOS

In this example I wanted to created a VLAN attached to e0/1 for use with my KVM lab.  You'll notice I give this interface an IP address, this is so that I can have my ScreenOS device do all the routing / NATing / firewalling in isolated VLANs. I also create a DHCP server.  I make the interface pingable as it may well be the internet gateway for anything inside the VLAN.

Create new zone and interface set zone name vlan69
set int ethernet0/1.1 tag 69 zone vlan69
set int e1.1 ip 10.10.69.254/24
set int e1.1 route
set int e1.1 manage ping
save

Setting Up DHCP Server
set int e1.1 dhcp server service
set int e1.1 dhcp server enable
set int e1.1 dhcp server option lease 71582788 #unlimted lease
set int e1.1 dhcp server option gateway 10.10.69.254
set int e1.1 dhcp server option netmask 255.255.255.0
set int e1.1 dhcp server option dns1 8.8.8.8
set int e1.1 dhcp server ip 10.10.69.10 to 10.10.69.240
set int e1.1 dhcp server config next-server-ip #unsets this
unset int e1.1 dhcp server config updatable
save

Checking DHCP Server Config # Check config options
get int e1.1 dhcp server option

# Check allocated leases
get int e1.1 dhcp server ip allocate