KVM Revisted Part I - Multiple VLANs

I've decided to rebuild my KVM lab on OpenSUSE 13.2 and to try and finish documenting the process of a fully functional KVM lab on multiple VLANs.

Install KVM with yast2
I used the "Install Hypervisor and Tools" in yast2 to install KVM.  I also allowed it to create the bridge (br0)

Access with virt-manager
This is the same as the steps I outlined in my previous post  http://pyn00b.blogspot.co.uk/2014/01/opensuse-kvm-getting-started-notes.html - just make sure to add yourself to the libvirt group and to restart the libvirtd.service

Multiple VLANs
The objective here is to have a single KVM host with a single Ethernet NIC to have multiple and routable segregated subnets. This is achieved by Bridging and VLAN tagging. We do this so that we can use the existing network infrastructure rather than using the KVM host to do any NATing or using KVM virtual networks.

Our solution is to have a single NIC, to which we create a VLAN interface, a Bridge is created from the VLAN interface and KVM attaches virtual NICs to the Bridge. The bridge is specified when creating a host and is therefore attached to the VLAN. This can be repeated multiple times for any number of VLANs. For example, a hierarchy of network devices might look like this:

Physical Nic (enp3s0) -> VLAN interface (enp3s0.69) -> Bridge (br69) -> vnet0
Physical Nic (enp3s0) -> VLAN interface (enp3s0.69) -> Bridge (br69) -> vnet1

Physical Nic (enp3s0) -> VLAN interface (enp3s0.101) -> Bridge (br101) -> vnet2

So in summary, for each new VLAN, a new VLAN interface is created with a VLAN tag, that interface is then Bridged and any number of KVM hosts attach their virtual NIC to the bridge and their traffic is VLAN tagged transparently.

Setting up VLAN interfaces
Let's start by creating an virtual interface that will tag traffic for VLAN 69.  We do so by creating the file /etc/sysconfig/network/ifcfg-enp3s0.69

BOOTPROTO='none'
STARTMODE='auto'
ETHERDEVICE='enp3s0'

Notice that by using the format ifcfg-enp3s0.VLANID sets the VLAN tag to 69 for us.  You can use VLAN_ID= as an option instead if you want to.  (see man ifcfg-vlan)

Also worth noting is that I have set BOOTPROTO='none' rather than 'static' and using IPADDR=''.  This is somewhat in contradiction to man ifcfg which states "Do  not  use  (none) to  just  skip  the IP setup".  However I noticed that when yast2 creates the bridge during the KVM install it sets BOOTPROTO='none'.  In my testing either option had the same result.  I used 'none' for consistency.  The yast convention is also to create ifcfg-vlanX which I did not care for.

Setting up Bridges
We now need to created a bridge from our VLAN interface.  We do so by creating /etc/sysconfig/network/ifcfg-br69

BOOTPROTO='none'
BRIDGE='yes'
BRIDGE_FORWARDDELAY='0'
BRIDGE_PORTS='enp3s0.69'
BRIDGE_STP='off'
STARTMODE='auto

Changes to sysctl
Disable any traffic over the bridges getting sent to iptables ( http://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf )

I created  /etc/sysctl.d/kvm.conf and reloaded the config with sudo sysctl --system

net.ipv4.ip_forward = 0
net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0

Checking config
I initially had issues getting wicked to work properly with the VLANs (see https://bugzilla.suse.com/show_bug.cgi?id=939580) I upgraded wicked from OBS and it was better after that although using ifup was still unreliable.  systemctl restart network.service works much better for now.

It turns out wicked network manager has some nice features too.  I particularly like this one as an alternative to ifconfig:

$ sudo wicked ifstatus all
...
 br69            up
      link:     #8, state up, mtu 1500
      type:     bridge
      config:   compat:suse:/etc/sysconfig/network/ifcfg-br69
...
 enp3s0.69       enslaved
      link:     #11, state up, mtu 1500, master br69
      type:     vlan enp3s0[69], hwaddr 67:89:d5:65:1e:66
      config:   compat:suse:/etc/sysconfig/network/ifcfg-enp3s0.69

vnet1           device-unconfigured
      link:     #12, state up, mtu 1500, master br69
      type:     tap, hwaddr fe:54:00:ac:0a:29

Setting Up VLANs and DHCP server in ScreenOS

In this example I wanted to created a VLAN attached to e0/1 for use with my KVM lab.  You'll notice I give this interface an IP address, this is so that I can have my ScreenOS device do all the routing / NATing / firewalling in isolated VLANs. I also create a DHCP server.  I make the interface pingable as it may well be the internet gateway for anything inside the VLAN.

Create new zone and interface set zone name vlan69
set int ethernet0/1.1 tag 69 zone vlan69
set int e1.1 ip 10.10.69.254/24
set int e1.1 route
set int e1.1 manage ping
save
Setting Up DHCP Server
set int e1.1 dhcp server service
set int e1.1 dhcp server enable
set int e1.1 dhcp server option lease 71582788 #unlimted lease
set int e1.1 dhcp server option gateway 10.10.69.254
set int e1.1 dhcp server option netmask 255.255.255.0
set int e1.1 dhcp server option dns1 8.8.8.8
set int e1.1 dhcp server ip 10.10.69.10 to 10.10.69.240
set int e1.1 dhcp server config next-server-ip #unsets this
unset int e1.1 dhcp server config updatable
save
Checking DHCP Server Config # Check config options
get int e1.1 dhcp server option

# Check allocated leases
get int e1.1 dhcp server ip allocate


OpenSUSE XDM SSH_ASKPASS

Every year or so I rebuild my machine and forget how to enable the ssh-askpass tool with ssh-agent from XDM display manager

cp /etc/X11/xdm/sys.xsession ~/.xession

cp ~/.xinitrc.template ~/.xinitrc

# Add ssh agent config for loading keys

if test -S "$SSH_AUTH_SOCK" -a -x "$SSH_ASKPASS"; then
        ssh-add ~/.ssh/id_nameofprivatekey < /dev/null
fi

Passing CFLAGS and LDFLAGS to cgo

I was trying to build https://github.com/systemfreund/go-libshout which wraps the C libshout library with Go.  On Linux this worked fine, but on OpenBSD this was a bit more of a challenge as cgo could not find libshout.  (install libshout with pkg_add first)

Although -lshout is specified as an LDFLAG  in the go source code, I needed to to specify the include path for libshout and also link to libspeex (-lsppex)

CFLAGS and LDFLAGS can be passed to go build as environmental variables so I set:

export CGO_CFLAGS='-I/usr/local/include'
export CGO_LDFLAGS='-L/usr/local/lib -lshout -lspeex'

Then I is was able to do

go build -x  github.com/systemfreund/go-libshout

(the -x is very useful for checking was gets passed to cgo)

Ansible ad-hoc commands

Create an inventory file

host1.something
host2.something
[group1]
group1host.something
group2host.something

Run a command on all hosts
ansible -i inventory_file all -m command -a "uptime"

This specifies the inventory file (-i) run on all hosts (all) and run the command module (runs a remote command) with the module argument uptime (-a)

Run a command on all hosts simpler
ansible -i inventory_file all  -a "uptime"

Same as above.  Notice that the module name was not specified as the command module is used by default

Run a command on hosts in group1 with sudo
ansible -i inventory_file group1 -s -K -a "service rsyslog status"

-s Use sudo
-K ask for the sudo password (can be omitted if not required)

Copy a file to all hosts
ansible -i inventory_file all -m copy -a "src=~/.vimrc dest=~"

Real Time Scheduling for audio in openSUSE

I was getting the following message from jack / hydogen
Cannot use real-time scheduling

To solve this I added the following to  /etc/security/limits.conf
@audio - rtprio 99
@audio - memlock 250000
@audio - nice -10

and also added myself to the audio group (sudo usermod -a -G audio jon)

Bulk Filename Rewriting

In zsh you'll find zmv (autoload zmv) and in zmv it's easy to rewrite bulk files.  For example, I had some downloaded files that were something.mp3?id=8795294528 etc and wanted to cut off the cruft at the end.

zmv '(*.mp3)*' '$1'