#Nuage #docker demo in a box

Hello There,

I’ve been working thru several PoCs last weeks. And always struggling over the installation of VSD and VSC. This post delivers some tips and installation notes to make things easier.

I’ve done that only on one physical box. then, I created a dummy interface as bridge. that way I don’t need to bother my self changing the Ip Segment

Prepare your environment

Create your dummy interface and bridge. Check out my notes at “Bridge your dummy interface in centos7”

Install whatever is required to get your KVM instances running. check out my notes at “Resize and manage cloud-init on kvm with centos cloud images”. You will need those scripts later.

Configure NTP and DNS server. If you don’t have one. You can create one of them using the script than I’ve mentioned before. I will annex some samples of bind and ntp configuration files on this post.

Also, you will need some nuage resources that are not free of access (ping me)

  • VSD and VSC QCOW2 images and user credentials
  • Evaluation license for VSD
  • VRS Nuage OpenVSwitch package
  • Nuage docker monitor app package

Nuage VSD: Create your Management Plane

These are the steps to create you VSD instance:

VSD – Step ONE: Create your VSD virtual instance

You will need the qcow2 image. Use “virt-install” app to do so as follow:

virt-install --connect qemu:///system -n vsd01.nuage40r4.lab -r 16384 --os-type=linux --os-variant=rhel6 --disk path=/var/lib/libvirt/images/vsd01.nuage40r4.lab/vsd.qcow2,device=disk,bus=virtio,format=qcow2 --vcpus=4 --graphics vnc,listen=0.0.0.0 --network bridge=bridge0,model=virtio --noautoconsole --import

Get into the server thru “virsh console ” as root.
Then, set the VSD’s eth0 interface to:

DEVICE="eth0"
BOOTPROTO="static"
IPV6INIT="yes"
MTU="1500"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"
DEFROUTE=yes
IPADDR=10.10.10.105
PREFIX=24
GATEWAY=10.10.10.1

Restart network services: service network restart.
Configure dns in /etc/resolv.conf changing nameserver to you local one (i.e. 10.10.10.2)
Set ntp services removing time servers and leaving just your local one: “server ns01.nuage.lab iburst”
Then, restart ntp services via: service ntpd restart (check with ntpstat if your server is already synched to your local ntp)
Add you host vsd.nuage.lab to your /etc/hosts
and set hostname to vsd.nuage.lab

VSD – Step TWO: Install VSD software

Finally execute /opt/vsd/install:

-------------------------------------------------------------
  V I R T U A L I Z E D  S E R V I C E S  D I R E C T O R Y  
  version 4.0.4_44
  (c) 2016 Nuage Networks
-------------------------------------------------------------
VSD supports two configurations:
1)  HA, consisting of 3 redundant installs of VSD.
2)  Standalone, where all services are installed on a single machine.
Is this a redundant (r) or standalone (s) installation [r|s]? (default=s): 
Deploy VSD on single host vsd01.nuage40r4.lab  ...
VSD node:      vsd01.nuage40r4.lab
Continue [y|n]? (default=y): y
Starting VSD deployment. This may take as long as 20 minutes in some situations ...
VSD package deployment and configuration DONE. Please initialize VSD.
DONE: VSD deployed.
Starting VSD initialization. This may take as long as 20 minutes in some situations ...
A self-signed certificate has been generated to get you started using VSD.
VSD installed and the services have started.

Check your services using “watch monit summary” until you get the following

[root@vsd01 ~]# monit summary
The Monit daemon 5.17.1 uptime: 2h 20m 

Process 'zookeeper'                 Running
Program 'zookeeper-status'          Status ok
Program 'vsd-core-status'           Status ok
Program 'vsd-common-status'         Status ok
Filesystem 'rootfs'                 Accessible
Program 'ntp-status'                Status ok
Process 'mysql'                     Running
Program 'mysql-status'              Status ok
Process 'mediator'                  Running
Program 'mediator-status'           Status ok
File 'jboss-console-log'            Accessible
File 'monit-log'                    Accessible
File 'mediator-out'                 Accessible
File 'zookeeper-out'                Accessible
Program 'keyserver-status'          Status ok
Process 'jboss'                     Running
Program 'jboss-status'              Status ok
Program 'ejbca-status'              Status ok
Process 'ejabberd'                  Running
Program 'ejabberd-status'           Status ok
System 'vsd01.nuage40r4.lab'        Running

VSD – Step THREE: Make VSD operational

You will need an eval license. Create an organization called “acme_corp” and also a template and domains to start testing your containers.

Nuage VSC: Create your Control Plane

VSC – Step ONE: Configure VSC IP settnigs

Set IP settings in bof.cfg file into VSC thru guestfish to:

primary-image    cf1:\timos\cpm.tim
primary-config   cf1:\config.cfg
address          10.10.10.106/24 active
primary-dns      10.10.10.2
dns-domain       nuage40r4.lab
static-route     0.0.0.0/1 next-hop 10.10.10.1
autonegotiate
duplex           full
speed            100
wait             3
persist          off
no li-local-save
no li-separate
console-speed    115200

Check my post “Nuage VSC – Modify QCOW2 images with guestfish” for further details

VSC – Step TWO: Create KVM instance

Install VSC thru the following script:

virt-install --connect qemu:///system -n vsc01.nuage40r4.lab -r 4096 --os-type=linux --disk path=/var/lib/libvirt/images/vsc01.nuage40r4.lab/vsc01.qcow2,device=disk,bus=virtio,format=qcow2 --vcpus=4 --graphics vnc,listen=0.0.0.0 --network bridge=bridge0,model=virtio --noautoconsole --import

VSC – Step THREE: Configure your VSC

Get access to vsc thru ssh.
And do the following to set ntp server and time zone

A:vm1# configure system 
A:vm1>config>system# name vsc01 
*A:vsc01>config>system# snmp 
*A:vsc01>config>system>snmp# exit 
*A:vsc01>config>system# time 
*A:vsc01>config>system>time# ntp 
*A:vsc01>config>system>time>ntp# server 10.10.10.2 
*A:vsc01>config>system>time>ntp# no shutdown 
*A:vsc01>config>system>time>ntp# exit 
*A:vsc01>config>system>time# sntp 
*A:vsc01>config>system>time>sntp# shutdown 
*A:vsc01>config>system>time>sntp# exit 
*A:vsc01>config>system>time# dst-zone CST 
*A:vsc01>config>system>time>dst-zone# start second sunday march 02:00 
*A:vsc01>config>system>time>dst-zone# end first sunday november 02:00 
*A:vsc01>config>system>time>dst-zone# exit 
*A:vsc01>config>system>time# zone CST 
*A:vsc01>config>system>time# exit 
*A:vsc01>config>system# thresholds 
*A:vsc01>config>system>thresholds# rmon 
*A:vsc01>config>system>thresh>rmon# exit 
*A:vsc01>config>system>thresholds# exit 
*A:vsc01>config>system# exit

Now set VSD setting into VSC

*A:vsc01# configure vswitch-controller 
*A:vsc01>config>vswitch-controller# xmpp-server vsd01:password@vsd01.nuage40r4.lab
*A:vsc01>config>vswitch-controller# open-flow 
*A:vsc01>config>vswitch-controller>open-flow# exit 
*A:vsc01>config>vswitch-controller# xmpp 
*A:vsc01>config>v-switch-controller>xmpp# exit 
*A:vsc01>config>vswitch-controller# ovsdb 
*A:vsc01>config>vswitch-controller>ovsdb# exit 
*A:vsc01>config>vswitch-controller# exit 
*A:vsc01# admin save

Check if setting are already working:

A:vsc01# show vswitch-controller vsd detail 

===============================================================================
VSD Server Table
===============================================================================
VSD User Name      : cna@vsd01.nuage40r4.lab/nuage
Uptime             : 0d 00:01:19
Status             : available          
Nuage Msg Tx.      : 1                  Nuage Msg Rx.      : 1
Nuage Msg Ack. Rx. : 1                  Nuage Msg Error    : 0
Nuage Msg TimedOut : 0                  Nuage Msg MinRtt   : 860
Nuage Msg MaxRtt   : 860                
 
===============================================================================

Create your docker server

Docker – Step ONE: Create your KVM instance

Create your docker server thru: “./virt-create-centos7.sh docker.nuage40r4.lab 4096 4 30 10.10.10.150”
Execute a “yum -y update” and then “yum -y docker install”

Now install docker module as following executing:

sudo yum -y install epel-release
sudo yum -y install python-pip
sudo pip install docker-py

Docker – Step TWO: Setup Nuage software

Install Nuage Docker-Monitor and VRS (OpenVSwitch) packages like the following:

sudo yum install nuage-openvswitch-4.0.4-45.el7.x86_64.rpm
sudo yum install nuage-docker-monitor-4.0.4-45.el7.x86_64.rpm

Modify ACTIVE CONTROLLER for VRS config file and set your VSC IP address as following

[centos@docker ~]$ cat /etc/default/openvswitch | grep ACTIVE
# ACTIVE_CONTROLLER: Primary controller IP. Only valid IP addresses will be
# accepted. To delete the controller comment out the ACTIVE_CONTROLLER
ACTIVE_CONTROLLER=10.10.10.106

Modify your Docker Monitor configuration (check annexed info) and restart both services. Don’t forget to create a user in the organization for this.

sudo /usr/share/openvswitch/scripts/nuage-docker-monitor.init restart
sudo service openvswitch  restart

Docker – Step THREE: Create you first container

Create your first container (remember use “–net=none” cause the network is being managed thru Nuage now). Remember create your domain, zones and subnets previously.

[centos@docker /]$ sudo docker run -d -i -t --name=docker_app01 --net=none centos /bin/bash
c57c77aecf715c3df427712dce63325e0aded69aff741943f14b552d4ea0d766
[centos@docker ~]$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
c57c77aecf71        centos              "/bin/bash"         7 minutes ago       Up 7 minutes                            docker_app01

Check out the container in your VSD console
nuage-docker-in-a-box-post-pinrojas-blog-02

See ya!

========================== Annexed Details

VSC configuration

Here you can see more details of my VSC settings

A:vsc01# show bof 
===============================================================================
BOF (Memory)
===============================================================================
    primary-image    cf1:\timos\cpm.tim
    primary-config   cf1:\config.cfg
    address          10.10.10.106/24 active
    primary-dns      10.10.10.2
    dns-domain       nuage40r4.lab
    static-route     0.0.0.0/1 next-hop 10.10.10.1
    autonegotiate
    duplex           full
    speed            100
    wait             3
    persist          off
    no li-local-save
    no li-separate
    console-speed    115200
===============================================================================
A:vsc01# admin display-config 
# TiMOS-DC-C-4.0.4-46 cpm/i386 NUAGE VSC Copyright (c) 2000-2016 Nokia.
# All rights reserved. All use subject to applicable license agreements.
# Built on Tue Aug 30 18:41:15 PDT 2016 [cf37f7] by builder in /rel4.0-DC/b1/4/panos/main

# Generated MON NOV 07 18:05:26 2016 UTC

exit all
configure
#--------------------------------------------------
echo "System Configuration"
#--------------------------------------------------
    system
        name "vsc01"
        snmp
        exit
        time
            ntp
                server 10.10.10.2
                no shutdown
            exit
            sntp
                shutdown
            exit
            dst-zone CST
                start second sunday march 02:00
                end first sunday november 02:00
            exit
            zone CST 
        exit
        thresholds
            rmon
            exit
        exit
    exit
#--------------------------------------------------
echo "System Security Configuration"
#--------------------------------------------------
    system
    exit
#--------------------------------------------------
echo "Log Configuration"              
#--------------------------------------------------
    log 
    exit 
#--------------------------------------------------
echo "System Security Cpm Hw Filters and PKI Configuration"
#--------------------------------------------------
    system
        security
        exit
    exit
#--------------------------------------------------
echo "QoS Policy Configuration"
#--------------------------------------------------
    qos
    exit
#--------------------------------------------------
echo "Card Configuration"
#--------------------------------------------------
#--------------------------------------------------
echo "Service Configuration"
#--------------------------------------------------
    service
    exit
#--------------------------------------------------
echo "LAG Configuration"
#--------------------------------------------------
    lag 98
        description "Multichassis interconnect LAG"
        encap-type dot1q
        qos
        exit
        lacp active administrative-key 36864 
        no shutdown
    exit
#--------------------------------------------------
echo "Virtual Switch Controller Configuration"
#--------------------------------------------------
    vswitch-controller
        xmpp-server "vsd01:password@vsd01.nuage40r4.lab"
        open-flow
        exit                          
        xmpp
        exit
        ovsdb
        exit
    exit
#--------------------------------------------------
echo "Management Router Configuration"
#--------------------------------------------------
    router management
    exit

#--------------------------------------------------
echo "Router (Network Side) Configuration"
#--------------------------------------------------
    router 
        interface "control"
            no shutdown
        exit
        interface "system"
            no shutdown
        exit
        vxlan
        exit
#--------------------------------------------------
echo "Web Portal Protocol Configuration"
#--------------------------------------------------
    exit

#--------------------------------------------------
echo "Service Configuration"
#--------------------------------------------------
    service
        customer 1 create
            description "Default customer"
        exit
    exit
#--------------------------------------------------
echo "Router (Service Side) Configuration"
#--------------------------------------------------
    router 
#--------------------------------------------------
echo "RADIUS Server Configuration"
#--------------------------------------------------
    exit

#--------------------------------------------------
echo "System Time NTP Configuration"
#--------------------------------------------------
    system
        time
            ntp
            exit
        exit
    exit

exit all

Monitor VSC and VRS

You will see your VSC and VRS into the VSD console like this:
nuage-docker-in-a-box-post-pinrojas-blog-01

Configuration yaml file Nuage docker monitor

First create the use in the organization as follow:
nuage-docker-in-a-box-post-pinrojas-blog-03

then modify yaml file:

[centos@docker /]$ cat /etc/default/nuage-docker-monitor.yaml 
# The user that will own the containers
User: docker
# The site ID value of the default VSD.  If your VSD(s) are not configured for
# geo redundancy, leave this option blank.
#SiteID:
# The default enterprise
Enterprise: acme_corp
# The default domain
Domain: docker_dom01
# The default zone
Zone: app
# The default network/subnet
Network: app_subnet01
# The default policy group
#PolicyGroup:
# The default domain type.  Valid values are `L2` and `L3`.  If no value is
# specified, `L3` is assumed
#DomainType:
# How long to keep container reservations in the VSD after the container goes
# down, in seconds.  A value of `0` means that reservations are immediately
# removed.  Default is `0`.
#RetentionTime:

# The default interface mtu. If no value is specified here, the default is 1460
#InterfaceMTU:

ntp.conf sample file

This is a file for a local and independent NTP server

driftfile /var/lib/ntp/drift
restrict default nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict ::1

restrict 10.10.10.0 mask 255.255.255.0 nomodify notrap

server 127.127.1.0 iburst # local clock
fudge 127.127.1.0 stratum 10

includefile /etc/ntp/crypto/pw

keys /etc/ntp/keys

disable monitor

named.conf sample file

This is a sample for a local and independent DNS server

acl "trusted" {
10.10.10.0/24;
};

options {
directory "/var/cache/bind";

recursion yes; # enables resursive queries
allow-recursion { trusted; }; # allows recursive queries from "trusted" clients
listen-on { 10.10.10.2; }; # ns1 private IP address - listen on private network only
allow-transfer { none; }; # disable zone transfers by default

forwarders {
8.8.8.8;
8.8.4.4;
};

};

zone "nuage.lab" {
 type master;
 file "/etc/named/zones/db.nuage.lab"; # zone file path
};

zone "10.10.10.in-addr.arpa" {
 type master;
 file "/etc/named/zones/db.10.10.10";
};


zone "nuage40r4.lab" {
 type master;
 file "/etc/named/zones/db.nuage40r4.lab"; # zone file path
};

db.zone sample file

An example of a db.zone file. first the reverse:

; file: /etc/named/zones/db.10.10.10
$TTL 604800
@ IN SOA ntp.nuage.lab. admin.nuage.lab. (
                           10 ; Serial
                           604800 ; Refresh
                           86400 ; Retry
                           2419200 ; Expire
                           604800 ) ; Negative Cache TTL
; name servers
@ IN NS ns01.nuage.lab.

; PTR Records
2 IN PTR ns01.nuage.lab.

; PTR records nuage40r4.lab
105 IN PTR vsd01.nuage40r4.lab.
106 IN PTR vsc01.nuage40r4.lab.
150 IN PTR docker.nuage40r4.lab.

Now the main zone:

; file: /etc/named/zones/db.nuage.lab
$TTL 604800
@ IN SOA ntp.nuage.lab. admin.nuage.lab (
                            10 ; Serial
                            604800 ; Refresh
                            86400 ; Retry
                            2419200 ; Expire
                            604800 ) ; Negative Cache TTL
;
; name servers – NS records

nuage.lab. IN NS ns01.nuage.lab.

; name servers – A records
ns01.nuage.lab. IN A 10.10.10.2

the app zone:

; file: /etc/named/zones/db.nuage40r4.lab
$TTL 604800
@ IN SOA ntp.nuage.lab. admin.nuage.lab (
                            12 ; Serial
                            604800 ; Refresh
                            86400 ; Retry
                            2419200 ; Expire
                            604800 ) ; Negative Cache TTL
;
; name servers – NS records

nuage40r4.lab. IN NS ns01.nuage.lab.

; 10.10.10.0/24 – A records
vsd01.nuage40r4.lab. IN A 10.10.10.105
xmpp IN CNAME vsd01
vsc01.nuage40r4.lab. IN A 10.10.10.106
docker.nuage40r4.lab. IN A 10.10.10.150


; SRV records
_xmpp-client._tcp IN SRV 10 0 5222 vsd01.nuage40r4.lab.

VRS: PAT to Underlay

Adding PAT2Underlay support using default namespace:
Default name space is utilizing the already configured eth0 or eth1 and just applying that to the ovs config file.

vi /etc/default/openvswitch "NETWORK_UPLINK_INTF=eth1"

Above example utilizes what’s already inside your /etc/sysconfig/network-scripts/ifcfg-eth1 as an example.

After you’ve applied the correct settings within the ovs config file restart the openvswitch process

systemctl restart openvswitch

Once all of this has been applied to each node launch the VSD (can utilize RestAPI’s as well)
Within the desired domain select the subnet that you want to PAT and enable the following:

Underlay Support – set to Enabled
Address Translation Support – set to Enabled

5 thoughts on “#Nuage #docker demo in a box

    1. Thanks. I am trying to create an ansible playbook to do it faster. I will publish in over this week or the next. Also, you can try nuagex.io and get management/control plane online to play with docker. Thanks for your comment.

  1. Hey Mauricio,

    Gr8 blog, can you please share the nuage VSD/VSC images as you rightly pointed out that its not free. i m looking for it to build a POC . Thanks

    Rgds.

    Shaz

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s