How to configure NAT with etherstub

We have a host with the only external IP-address. We want to install many non-global zones to build the system and to port packages or for other services. The crossbow helps us to solve this problem. Let’s look at the diagram of the host:

a host with etherstub and NAT

In this host we have a physical NIC atge0 which connected to an external hardware switch. A virtual interface ext0 (VNIC) was created over the physical NIC (for example, to make a tagged packets). Let’s see at the real example:

lenovo# dladm show-link LINK CLASS MTU STATE BRIDGE OVER atge0 phys 1500 up -- --

Create an extenal VNIC over the physical device:

lenovo# dladm create-vnic -l atge0 ext0

Let’s see the result:

lenovo# dladm show-link LINK CLASS MTU STATE BRIDGE OVER atge0 phys 1500 up -- -- ext0 vnic 1500 up -- atge0

Now configure the new VNIC as a regular network interface. Just create a file /etc/hostname.ext0 like this:

lenovo# cat /etc/hostname.ext0 192.168.0.3 netmask 255.255.255.0 broadcast + up

Create /etc/defaultrouter file and specify a default gateway like this:

lenovo cat /etc/defaultrouter 192.168.0.1

Now you can reconfigure the network, just restart the service network/physical:

lenovo# svcs network/physical STATE STIME FMRI disabled 9:48:02 svc:/network/physical:nwam online 9:48:11 svc:/network/physical:default lenovo# svcadm restart svc:/network/physical:default lenovo# svcs network/physical STATE STIME FMRI disabled 9:48:02 svc:/network/physical:nwam online 11:02:04 svc:/network/physical:default

You will see something like this:

lenovo# ifconfig lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 ext0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 2 inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 ether 2:8:20:f9:ec:e5 lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1 inet6 ::1/128 lenovo# netstat -rnf inet Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 192.168.0.1 UG 2 0 ext0 127.0.0.1 127.0.0.1 UH 2 28 lo0 192.168.0.0 192.168.0.3 U 4 60 ext0

Alternatively you can do it manually:

lenovo# ifconfig ext0 plumb lenovo# ifconfig ext0 inet 192.168.0.3/24 up lenovo# route add default 192.168.0.1

But… This is a temporary solution, it works until you reboot the system. You can use this method to access the system via ssh and then to configure the network permanently.

Now the global zone has configured partially, it means that we need second VNIC to do NAT for the non-global zones.

Configuring etherstub

Thechnically etherstub is a virtual software switch that does the same as a hardware switch. The differnce is the end point of the etherstub is a VNIC (instead of physical cable which connected to a physical NIC). So, to do a forward packets we need an interface which will receive packets from the non-global zones and forwards them to the external network (ext0). The VNIC will be first end point of the virtual switch (ethersub). Let’s create them in the host:

lenovo# dladm create-etherstub vsw0 lenovo# dladm create-vnic -l vsw0 int0 lenovo# dladm show-link LINK CLASS MTU STATE BRIDGE OVER atge0 phys 1500 up -- -- ext0 vnic 1500 up -- atge0 vsw0 etherstub 9000 up -- -- int0 vnic 9000 up -- vsw0

Line 1 creates a new etherstub named vsw0; line 2 creates a new VNIC over the switch and line 3 shows us the result. Now the interface int0 should be configured as we did it for ext0. I do it manually:

lenovo# ifconfig int0 plumb lenovo# ifconfig int0 inet 10.68.0.1/24 up lenovo# ifconfig int0 int0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9000 index 3 inet 10.68.0.1 netmask ffffff00 broadcast 10.68.0.255 ether 2:8:20:85:3d:bf

The interface int0 has created, but it will not forward anything outside. NAT and ip forwarding are not yet configured. Let’s configure one as it was described here :

lenovo# cat /etc/ipf/ipnat.conf map ext0 10.68.0.0/24 -> 0/32 proxy port ftp ftp/tcp map ext0 10.68.0.0/24 -> 0/32 portmap tcp/udp auto map ext0 10.68.0.0/24 -> 0/32

Enable ipfilter if it is still disabled:

lenovo# svcs ipfilter STATE STIME FMRI disabled 9:48:03 svc:/network/ipfilter:default lenovo# svcadm enable ipfilter lenovo# svcs ipfilter STATE STIME FMRI online 11:29:25 svc:/network/ipfilter:default

Examinging packets ip forwarding:

lenovo# routeadm Configuration Current Current Option Configuration System State --------------------------------------------------------------- IPv4 routing disabled disabled IPv6 routing disabled disabled IPv4 forwarding disabled disabled IPv6 forwarding disabled disabled Routing services "route:default ripng:default" Routing daemons: STATE FMRI disabled svc:/network/routing/ripng:default online svc:/network/routing/ndp:default disabled svc:/network/routing/legacy-routing:ipv4 disabled svc:/network/routing/legacy-routing:ipv6 disabled svc:/network/routing/rdisc:default disabled svc:/network/routing/route:default

IPv4 forwarding is disabled (line 7); enable it:

lenovo# routeadm -u -e ipv4-forwarding lenovo# routeadm Configuration Current Current Option Configuration System State --------------------------------------------------------------- IPv4 routing disabled disabled IPv6 routing disabled disabled IPv4 forwarding enabled enabled IPv6 forwarding disabled disabled Routing services "route:default ripng:default" Routing daemons: STATE FMRI disabled svc:/network/routing/ripng:default online svc:/network/routing/ndp:default disabled svc:/network/routing/legacy-routing:ipv4 disabled svc:/network/routing/legacy-routing:ipv6 disabled svc:/network/routing/rdisc:default disabled svc:/network/routing/route:default

Now it is enabled, configure the first zone to test the result. Here we do not explain step by step configuring of a non-global zone, it was described here , just do it:

lenovo# zonecfg -z zone01 zone01: No such zone configured Use 'create' to begin configuring a new zone. zonecfg:zone01> create zonecfg:zone01> set zonename=zone01 zonecfg:zone01> set zonepath=/zones/zone01 zonecfg:zone01> set autoboot=false zonecfg:zone01> set ip-type=exclusive zonecfg:zone01> add net zonecfg:zone01:net> set physical=iz0 zonecfg:zone01:net> set global-nic=vsw0 zonecfg:zone01:net> add property (name=ip,value="10.68.0.2") zonecfg:zone01:net> add property (name=gateway,value="10.68.0.1") zonecfg:zone01:net> add property (name=netmask,value="255.255.255.0") zonecfg:zone01:net> add property (name=primary,value="true") zonecfg:zone01:net> end zonecfg:zone01> add attr zonecfg:zone01:attr> set name=resolvers zonecfg:zone01:attr> set type=string zonecfg:zone01:attr> set value=192.168.0.1 zonecfg:zone01:attr> end zonecfg:zone01> verify zonecfg:zone01> commit zonecfg:zone01> exit

The only I would like to comment here is the lines 10-13.

In line 10 we specified a physical VNIC which will be created at zone boot time over the global nic vsw0 (line 11), instead of a global nic we specified an etherstub (virtual switch), in line 12 we assign an IP-address from our internal private network. As we remember from the previous step, we configured first IP-address on the interface int0 to forward the traffic from our zones. In line 13 we use the int0 as the default gateway. In line 20 I specified an IP-address of name server (external for the host, it could be google’s public addresses: 8.8.8.8, 8.8.4.4).

Install and boot the zone:

lenovo# zoneadm -z zone01 install A ZFS file system has been created for this zone. Image: Preparing at /zones/zone01/root. Installing: Packages (output follows) /* cut the logs */ Postinstall: Copying SMF seed repository ... done. Postinstall: Applying workarounds. Done: Installation completed in 62 seconds. Next Steps: Boot the zone, then log into the zone console (zlogin -C) to complete the configuration process. lenovo#

Boot it:

lenovo# zoneadm -z zone01 boot zone 'zone01': /zones/zone01/root lenovo#

After the zone booted we can login into its console:

lenovo# zlogin zone01 [Connected to zone 'zone01' pts/2] zlogin from global The programs included with the DilOS 2.0.2 system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. DilOS 2.0.2 comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. root@zone01:~#

Test the network:

root@zone01:~# ifconfig lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 iz0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9000 index 2 inet 10.68.0.2 netmask ffffff00 broadcast 10.68.0.255 ether 2:8:20:80:1e:99 lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1 inet6 ::1/128

Ping the name server we specified in the zone configuration:

root@zone01:~# ping 192.168.0.1 192.168.0.1 is alive

Ping google’s DNS (as the external resource):

root@zone01:~# ping 8.8.8.8 8.8.8.8 is alive

So, the NAT and the IP forwarding are working perfectly!

Let’s install the second zone (zone02) using the same template:

lenovo# zonecfg -z zone02 zone02: No such zone configured Use 'create' to begin configuring a new zone. zonecfg:zone02> create zonecfg:zone02> set zonename=zone02 zonecfg:zone02> set zonepath=/zones/zone02 zonecfg:zone02> set autoboot=false zonecfg:zone02> set ip-type=exclusive zonecfg:zone02> add net zonecfg:zone02:net> set physical=iz1 zonecfg:zone02:net> set global-nic=vsw0 zonecfg:zone02:net> add property (name=ip,value="10.68.0.3") zonecfg:zone02:net> add property (name=gateway,value="10.68.0.1") zonecfg:zone02:net> add property (name=netmask,value="255.255.255.0") zonecfg:zone02:net> add property (name=primary,value="true") zonecfg:zone02:net> end zonecfg:zone02> add attr zonecfg:zone02:attr> set name=resolvers zonecfg:zone02:attr> set type=string zonecfg:zone02:attr> set value="192.168.0.1,8.8.8.8" zonecfg:zone02:attr> end zonecfg:zone02> verify zonecfg:zone02> commit zonecfg:zone02> exit

Install and boot it as the zone01:

lenovo# zoneadm -z zone02 install A ZFS file system has been created for this zone. Image: Preparing at /zones/zone02/root. Installing: Packages (output follows) /* cut the logs */ Postinstall: Copying SMF seed repository ... done. Postinstall: Applying workarounds. Done: Installation completed in 61 seconds. Next Steps: Boot the zone, then log into the zone console (zlogin -C) to complete the configuration process. lenovo# zoneadm -z zone02 boot zone 'zone02': /zones/zone02/root

Login in the second zone:

lenovo# zlogin zone02 [Connected to zone 'zone02' pts/2] zlogin from global The programs included with the DilOS 2.0.2 system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. DilOS 2.0.2 comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law.

Examining the network:

root@zone02:~# ifconfig lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 iz1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9000 index 2 inet 10.68.0.3 netmask ffffff00 broadcast 10.68.0.255 ether 2:8:20:9f:99:97 lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1 inet6 ::1/128 root@zone02:~# ping 192.168.0.1 192.168.0.1 is alive root@zone02:~# ping 8.8.8.8 8.8.8.8 is alive

And now ping the first zone:

root@zone02:~# ping 10.68.0.2 10.68.0.2 is alive

Yes, we can communicate the neighbour zones.

Try to setup the zone03 from the figure as an exercise to understand every step in this article.