Managing Apache CloudStack Network

Networking is a separate component in CloudStack, supporting different types of guest network setups for instance to communicate with each other and the public internet. Guest networks in CloudStack provide the services for instances (guests) to communicate with each other. Guest networks in CloudStack coexist next to the storage and management networks, but the last two networks are internal and thus invisible to you.

There are 2 main types of guest networks to discern, Isolated network and Shared network. Also, there is the option to create a Virtual Private Cloud (VPC), grouping instances in separate tiers with their own IP subnet from a larger CIDR. In the base, networks leverage layer 2 MAC addressing to establish networking links between the available devices, both in Shared and Isolated networks. Each instance added to a network gets a virtual network interface card (NIC) with a unique MAC address assigned to it. This allows for an IP address to be assigned to the instance through one of its assigned virtual network interface cards.

IPv4 addresses are provided as subnets to be able to announce your own network to the public internet. Therefore, by default 5 IP addresses out of each subnet are reserved for the network:

  • 2 IPv4 addresses are assigned to the Leaseweb routers
  • 1 IPv4 address is assigned as a Network address
  • 1 IPv4 address is assigned as a Broadcast address
  • 1 IPv4 address is assigned as Gateway

Both the shared and the isolated networks can be assigned to instances at the same time. An instance can simultaneously have connections within multiple networks. For each connection to a network, a separate virtual network interface is created on the instance.

Note: If instances have multiple networks, manual configuration is required to ensure correct routing.


Contents

Managing Shared networks

The shared network type has a single virtual router running that provides DHCP and DNS services.  For each instance added to this network, a public IP address from your IP subnet is assigned making the instance available to the public internet. The IP address is directly assigned to the virtual network interface of the instance, making it directly connected with the internet.

Shared networks are configured and managed by Leaseweb, therefore you cannot create this network type in the CloudStack panel. Please contact support to have a shared network created. Also, make sure to have sufficient IP addresses available to be assigned to the shared network.

A public IPv4 range can be purchased separately, please contact Sales for more info.

  • All traffic to and from this network is measured as part of your billed data traffic.

Assigning an instance to a shared network

  1. From the left menu in the CloudStack panel, choose Compute > Instances and click on the instance.
  2. Choose the NICs tab in the instance overview page, and click + Add network to VM.
  3. From the drop-down menu select the shared network, and optionally provide an IP address from the subnet assigned to the network and that is still free.
  4. Confirm by clicking OK.
    The selected instances will be added to the shared network. You will see the network appear in the list of networks under the NICs tab for your instance. The NIC will have both a MAC and IP address assigned.

IP address usage in a shared network

Each shared network needs a public IPv4 address range assigned, from where the virtual router in the network can select an IP address to assign to an instance.

  1. From the left menu in the CloudStack panel, choose Networks > Guest networks and click on the network with type Shared.
  2. In the Details tab you will find the assigned IPv4 address range, under CIDR.
    N.B.: From each IPv4 address range the first IP address is assigned to Network, the last 4 IP address are assigned to router01, router02, Gateway and Broadcast in that order. The rest of the IP address in the range can be assigned to instances, by adding them to this network.
  3. In the Public IP addresses tab you can see all the IP addresses allocated to the shared network and if they are assigned to an instance.

Managing Isolated networks

The isolated network type has a virtual router running that provides multiple services like, DHCP, DNS, load balancing, port forwarding and firewall, but it can also be a standalone layer 2 network without any services. A simple layer 2 network will only assign MAC addresses to instances in that network, allowing you to setup your own DHCP and DNS services.

Available services

Service nameService description
DHCPManaging IP addresses within the network, assigning primary IP address and sharing network information to an instance.
FirewallManaging ingress on public IP addresses within the network and egress traffic from instances through Source NAT address.
Port forwardingAllowing to forward traffic from and to specific ports only to instances within an isolated network.
Source NATThe virtual router IP address that is also used by instances to send internet packets over to the public internet.
Static NATThis provides a public IP address directly to a specific instances, where Source NAT is attached to the virtual router and shared between instances from that network.
User dataAllowing to add of user-specific data when the instance gets deployed.
DNSProviding DNS to the network, with the virtual router being the primary DNS.
Load balancerProviding load balancing between multiple instances on a public IP address.
VPNIPsec VPN endpoint for a public IP address associated with the network, to extend the network as a local network with your own system.

An isolated network has a private IP address range assigned, which is normally 10.1.1.0/24 and from this subnet, a private IP address is assigned to each instance in the network. All instances within the same isolated network can communicate with each other using the private IP address space, thus there is by default no outside route to other networks or the public internet. To reach other networks, traffic destined for other IP addresses can be routed through the virtual router via a Source NAT address which also acts as the gateway and manages any public IPv4 addresses assigned to this network.

  • A public IPv4 range can be purchased separately, please contact Sales for more info.
  • The private IPv4 range is assigned by Leaseweb to the virtual router and a private IP address out of the range can be mapped by you to a virtual machine on the private network side of the virtual router.
  • Traffic between the virtual machines within this private network is free, only public traffic is measured as part of your billed data traffic

This type of network is useful if you do not want outside hosts initiating a connection directly to internal virtual machines, allowing only connections between virtual machines in the private network. Using the virtual router’s Static NAT function allows you to map a public IP address to a virtual machine with a private IP address.


Adding an Isolated Network

An isolated network can be deployed with different services like an HAproxy load balancer in the virtual router or source nat to communicate with other networks or a simple layer 2 network. A Network Offering in CloudStack defines how the isolated network is setup and which services to include with its deployment.

Available Network Offerings


Network Offering nameServices providedDescription

DefaultIsolatedNetworkOfferingWithSourceNatService

DHCP, firewall, port forwarding, Source NAT, Static NAT, User data, DNS, Load balancer and VPN

Creates a single virtual router with Source Nat service enabled and egress traffic blocked by default

Isolated Network with Source Nat and Dual VR (Deny)

DHCP, firewall, port forwarding, Source NAT, Static NAT, User data, DNS, Load balancer and VPN

Creates a redundant virtual router with Source Nat service enabled and egress traffic blocked by default

Isolated Network with Source Nat and Dual VR (Allow)

DHCP, firewall, port forwarding, Source NAT, Static NAT, User data, DNS, Load balancer and VPN

Creates a redundant virtual router with Source Nat service enabled and egress traffic allowed by default

DefaultL2NetworkOfferingNo services includedCreates a simple layer 2 network that assigns a MAC address to instances in the network
  1. From the left menu in the CloudStack panel, choose Networks > Guest networks and click Add network.
  2. Choose Isolated if you want a network that includes services like DHCP and DNS, or else choose L2 for a simple layer 2 networks.
  3. Provide a network name and description.
  4. Choose a network offering from the drop-down menu.
  5. Confirm by clicking OK.
  6. The network can be viewed by going to Network > Guest networks.

Acquiring New IP Address for an Isolated Network

You can allocate multiple public IP addresses to an isolated network. This allows you to i.e. statically NAT a public IP address to an instance on the network or get multiple IP addresses to load balance. If you do not statically NAT a new IP address to an instance, it will be assigned to the virtual router. Any IP address associated to the virtual router can be used to create load balancing rules, port forwarding, or firewall rules.

  1. From the left menu in the CloudStack panel, choose Network > Guest networks and click on the network.
  2. Choose the Public IP Addresses tab in the overview.
  3. Click Aquire new IP.
  4. From the drop-down choose a public IP address belonging to your domain.
    N.B. Only free IP addresses will appear here. If you are out of IP addresses, go to your customer portal where you can order an additional subnet to be assigned to your domain.

    Any public IP address assigned to the isolated network is within the range of IP addresses assigned to your CloudStack domain.

Restarting an Isolated Network

If any of the services within your network are failing, restarting your network might resolve it. On restarting the network, all services offered through the virtual router are restarted. The virtual router will be destroyed and recreated. All virtual machines within the network will loose public connectivity during this process.

When services within the network are unavailable or broken, you can restart the network. Please check the 'cleanup' checkbox.

You can only restart a network when the state of the network is either "Implemented" or "Setup". You can view the state of the network on the Details page of the network. If an isolated network is in the "Allocated" state, it means that the network was created, but there was never any running instance assigned to it yet. After assigning the first running instance to it, the network state would change to "implemented".

  1. From the left menu in the CloudStack panel choose Network > Guest networks and click on the network.

  2. In the top right options on the network overview page, click Restart network.
  3. Check the clean-up option and confirm by clicking OK.

Deleting an Isolated Network

You can only delete networks that do not have any instances running in the network. To remove an instance from the network, you must remove the NIC for that network on the instance, or destroy the instance.

  1. From the left menu in the CloudStack panel choose Network > Guest networks and click on the network.

  2. In the top right options on the network overview page, click Delete Network.
  3. Confirm by clicking OK.

Updating network offering for the isolated network

For existing isolated networks it is possible to change the network offering by editing the network settings. This allows to add a redundant virtual router to the network without having to recreate the entire network.

  1. From the left menu in the CloudStack panel choose Network > Guest networks and click on the network.

  2. In the top right options on the network overview page, click Edit.
  3. Choose the right Network Offering and confirm by clicking OK.

Assigning a Static NAT for an IP address to an instance

To attach a public IP address directly to an instance, it is possible to assign it as Static NAT. The instance with the Static NAT IP address in the isolated network, will be directly connected through this public IP address with the internet. 

Make sure to configure firewall rules on the Static NAT IP address to ensure only allowed hosts can communicate with the instance.

  1. From the left menu in the CloudStack panel choose Network > Guest networks and click on the network.

  2. Choose Public IP Addresses tab in the overview and click on the IP address.
  3. In the top right options on the IP address overview page, click Enable Static NAT.
  4. Select the instance to link the IP address with and confirm by clicking OK.
  5. You can view the public IP address and manage the firewall by going to Networks > Public IP addresses, click on the public IP address.

Enabling a VPN for an IP Address of an Isolated Network

By enabling VPN for an isolated network IP address, you can configure a remote access VPN (IPsec) connection. This allows you to directly access the instances within the network from a remote machine.

You can only set up one remote access VPN per network.


  1. From the left menu in the CloudStack panel choose Network > Guest networks and click on the network.
  2. Choose Public IP Addresses tab in the overview and click on the IP address with the source-nat label.
  3. Choose VPN tab in the overview.
  4. Click Enable Remote Access VPN, then Yes to confirm.
    An IPsec remote access VPN connection is enabled for the selected IP address.

  5. Click Manage VPN users, to add a new VPN user.

    The IPsec pre-shared key is used to connect from your client with the isolated network. Together with the VPN username and password, you need to setup the VPN connection from your local machine.

Managing Virtual Private Cloud (VPC)

A Virtual Private Cloud is a combination of isolated networks or tiers governed by one virtual router device, allowing you to create your own virtual network topology that resembles a traditional physical network. The virtual router device which normally exists per isolated network is promoted to control all isolated networks within the VPC. Instances in a VPC receive a private IP address within the subnet range that you define for the VPC and its network tiers. By defining the super CIDR (e.g. 10.0.0.0/16), you will be able to create network tiers with their own subnet within this super CIDR range. This enables you to group instances based on IP address range in a single virtual network and with ACL rules restricting internet traffic between the isolated networks.

The VPC router can be setup as a single virtual instance or with a backup, taking over when the main VPC router should fail. Also, keep in mind that the VPC router is responsible for all traffic between the different network tiers and the public internet. This creates a single point where traffic flows and could increase the chance of traffic congestion between the networks.

Adding a VPC

  1. From the left menu in the CloudStack panel choose Network > VPC and click Add VPC.
  2. Provide a Name and Description for the VPC.
  3. Provide the super CIDR for the VPC. This will be the main CIDR, from which you can create smaller subnets to assign to network tiers.
  4. Choose a VPC Offering and click OK.

    VPC Offering nameServices providedDescription
    Default VPC offeringDHCP, Network ACL, port forwarding, Source NAT, Static NAT, User data, DNS, Load balancer and VPNCreates a single VPC router with all services included
    Redundant VPC offeringDHCP, Network ACL, port forwarding, Source NAT, Static NAT, User data, DNS, Load balancer and VPNCreates a redundant VPC router with all services included

Adding a network tier to a VPC

Each network tier act as an isolated network with its own VLANs and CIDR list, where you can place instances. The tiers are segmented by means of VLAN, the NIC of each tier acts as its gateway.

You can create a VPC with up to 10 network tiers.


  1. From the left menu in the CloudStack panel choose Network > VPC and click on the VPC.
  2. On the VPC overview page choose the Networks tab and click Add Network.
  3. Provide a Name for the network tier.
  4. Choose a Network Offering from the drop-down menu.

    Network OfferingServices providedDescription
    DefaultIsolatedNetworkOfferingForVpcNetworksDHCP, Network ACL, port forwarding, Source NAT, Static NAT, User data, DNS, Load balancer, and VPNCreates a network tier with external load balancer. External load balancer listens to rules created on the VPC router to redirect the traffic received on public IP address. Traffic is load balanced within a tier based on your configuration.
    DefaultIsolatedNetworkOfferingForVpcNetworksNoLBDHCP, Network ACL, port forwarding, Source NAT, Static NAT, User data, DNS and VPNCreates a network tier without load balancer service.
    DefaultIsolatedNetworkOfferingForVpcNetworksWithInternalLBDHCP, Network ACL, Source NAT, User data, DNS, Load balancerCreates a network tier with internal load balancer. Internal load balancer listens to rules created on the VPC router to redirect the traffic received on internal IP addresses from a specific network tier. 
  5. Provide the Gateway IP address for the network tier. This should be the very last IP address from the CIDR you are adding to this network tier (e.g. 10.0.0.254 for Gateway IP, where CIDR will be 10.0.0.0/24).
  6. Provide the Netmask address for the network tier. Here you define the subnet class (Network and Host address) for the network tier (e.g., 255.255.255.0 is netmask for CIDR 10.0.0.0/24).
  7. Choose default ACL ruleset. 'default_allow' to allow all traffic through the firewall and 'default_deny' to deny all traffic through the firewall. To allow certain traffic you create specific ACL rules in the VPC.
  8. Click OK to create the network tier.

Managing ACL lists

With Network ACL you manage incoming and outgoing network traffic for the network tiers within the VPC. ACL rules are evaluated in order based on numbered priority, with the lowest number getting highest priority.

See the CloudStack official docs for detailed guidance on setting up and associating ACL lists and rules.

Making VPC router redundant

During the creation of a VPC, you can choose to setup the VPC router as a single instance or redundant, based on the VPC offering (Default VPC offering or Redundant VPC offering). 

By changing the network offering for a VPC you can change the VPC router redundancy, should you have a single VPC router deployed. By adding redundancy to the VPC router, CloudStack will switch traffic almost instantly to the backup router.  The switch-over is seamless and minimises impact on traffic flow to your network tiers in the VPC.


  1. From the left menu in the CloudStack panel choose Network > VPC and click on the VPC.
  2. In the top right options on the VPC overview page, click Restart VPC.
  3. Enable the option Make redundant and confirm by clicking OK.
    N.B. by restarting the VPC router a short interruption in traffic is expected.

Creating a site-to-site VPN

With a site-to-site VPN, you can setup a secure connection between your network and the CloudStack VPC. A site-to-site VPN connects networks to each other, sending TCP/IP traffic over the VPN gateway to the network tiers. This allows you to access all the instances in a VPC through a single VPN connection from your network and the VPC router. Also, it is possible to setup a site-to-site VPN connection between two VPC setups.


To establish a site-to-site VPN connection with your VPC, you will need to create a VPN customer gateway, VPN gateway for the VPC, and VPN connection from the VPC VPN gateway to the customer VPN gateway.

Please read the official CloudStack documentation for a step-by-step guide to setup a site-to-site VPN.

Managing load balancer 

Configuring load balancing for a public IP Address in Isolated Network

With an Isolated Network the load balancer is provided as service, balancing traffic over the instances that are running in the isolated network. Load balancing rules are set on each public IP address.


  1. From the left menu in the CloudStack panel choose Network > Public IP addresses.
  2. Click the public IP address associated to your isolated network.
  3. On the right panel, click Load Balancing tab.
  4. Name the load balancing rule and define the public and private ports that allow traffic through.
  5. Choose the Algorithm and Protocol for load balancing.
  6. Select an SSL certificate from the uploaded certificated in case you want SSL offloading by the router instead of the instance.
  7. Now add an instance by clicking the Add button under Add VMs, select one or more instances and click OK.

Changing the load balancer settings

The load balancer is the HAproxy application running on the virtual router instance. You can modify certain default HAproxy settings to align with your hosting needs.


  1. From the left menu in the CloudStack panel choose Network > Guest networks 
  2. Click the network with type Isolated.
  3. On the right panel, click LB Configs tab.
  4. Under LB Config name you can select from a list of HAproxy rules that can be modified.
    In the table below an overview of available rules to modify.

    CategoryNetwork tagLoad balancer tagSettings in Haproxy

    default timeout

    lb.timeout.client=value

    lb.timeout.server=value

    lb.timeout.connect=value


    timeout client $value (default value is 50000)

    timeout server $value (default value is 50000)

    timeout connect $value (default value is 5000)

    Enabling statslb.stats.enable
    stats enable (default value is true)
    URIlb.stats.uri
    /admin?stats

    Haproxy stats

    lb.stats.auth=

    username:password


    stats auth $username:$password (default value is admin1:AdMiN123)

    global

    connections

    global.maxconn=value

    global.maxpipes=value

    global.stats.socket=value


    maxconn $value (default value is 4096)

    maxpipes $value (default value is global.maxconn / 4)

    default value is false

    load balancer

    connections


    lb.maxconn=value

    lb.fullconn=value

    lb.maxconn.each=value

    lb.minconn.each=value

    lb.maxqueue.each=value

    lb.timeout.connect=value

    lb.timeout.client=value

    lb.timeout.server=value

    lb.server.maxconn=value

    lb.server.minconn=value

    lb.server.maxqueue=value

    maxconn $value (not set by default)

    fullconn $value ((not set by default. default value is $maxconn / 10 in haproxy)

    maxconn $value (per site/vm)

    minconn $value (per site/vm)

    maxqueue $value (per site/vm)

    timeout connect $value (default value is 5000ms)

    timeout server $value (default value is 50000ms)

    timeout client $value (default value ius 50000ms)

    server maxconn $value (default value is unlimited)

    server minconn $value

    server maxqueue $value (default value is unlimited)

    (TBD) default action

    lb.default.action=

    restart/reload


    (VR) service haproxy $value

    (the action if configurations change, default value is reload)

    HTTP settings


    lb.http=true/false

    lb.http.keepalive=true/false

    mode http (if true,default value true for 80, default value false for other ports)

    option httpclose (if false and lb.http set to true, default value is false)

    HTTPS settings
    lb.backend.https=true/falsessl verify none (default value is false)

    Transparent

    mode


    lb.transparent.mode=true/false

    source 0.0.0.0 usesrc clientip

To get the haproxy stats, please allow 8081 port in the firewall rules and then access the page with the username/password specified in lb.stats.auth
http://<Source NAT IP>:8081/admin?stats

Setting up sticky sessions on your load balancer in CloudStack

CloudStack supports sticky sessions through its default load balancer feature for isolated networks. With these sticky sessions, you can ensure a persistent state for user sessions across multiple requests.

Any load balancer rule defined in CloudStack can have a stickiness policy. The policy consists of a name, method, and additional parameters. The parameters are name-value pairs or flags, which are defined by HAProxy.

In CloudStack there are three sticky methods supported: load balancer-generated cookie, application-generated cookie, or source-based. The cookie generated by the load balancer or application is included in request and response URLs to create persistence. In the source-based method, the source IP address is used to identify the user and locate the user’s stored data.


To setup sticky sessions:

  1. Find or create a load balancer rule under one of your public IP addresses.
  2. From the left menu in the CloudStack panel choose Network > Public IP addresses.
  3. Click the public IP address associated to your isolated network.
  4. On the right panel, click Load Balancing tab.
  5. Click Configure under Stickiness on one of the created load balancer rules
    In the Configure Sticky Policy screen, based on the Stickiness method you select, you will see the following details below. There are 3 different Stickiness method to choose from.

Load balancer-generated cookie (LbCookie)

By selecting LbCookie you enable cookie-based persistency in your backend environment. Used by default to have the load balancer (HAProxy) insert the cookie in the HTTP response header. It is also possible to use modes  “Rewrite” when the server is already providing the cookie and HAProxy only needs to modify it with the server id, or “Prefix” when you don’t want a dedicated cookie, but the cookie prefixed with the server id and a delimiter in between.

ModesDescription
Insert

Used by default. Allows HAProxy to insert cookie in HTTP response

Rewrite

Allows HAProxy to modify the cookie that has been set by application

Prefix

Allows HAProxy to prefix cookie with server id that send the HTTP response

  1. From the Stickiness method drop-down menu, choose LBCookie.
  2. Fill in a Sticky Name that will be displayed in the dashboard for this rule
  3. Fill in a Cookie name that will be used as cookie to insert in the HTTP response headers
  4. Choose the preferred Mode from the options in the table above
  5. Optionally select No cache checkbox to prevent caching of responses and reuse of the same cookie by different clients. This option will tag responses as non-cacheable in an environment that is using response caching.
  6. Optionally select Indirect checkbox to instruct HAProxy to insert the cookie if the client does not already have one
  7. Optionally select Post only checkbox to insert cookie only for POST-request responses
  8. Optionally add a Domain to tell HAProxy to insert a cookie only for this domain specified

For more information on the different directives that are supported, please read the following HAProxy instruction: https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-cookie

Application-generated cookie (AppCookie)

By selecting AppCookie you can manage session stickiness on an existing application provided cookie.

  1. From the Stickiness method drop-down menu, choose AppCookie.
  2. Fill in a Sticky Name that will be displayed in the dashboard for this rule
  3. Fill in a Cookie name  that will be used as cookie to insert in the HTTP response headers
  4. Choose the preferred Mode:
    1. Use Mode path-parameter if the parser stores the appsession in the path parameter (e.g. /path/file123)
    2. Use query-string when returning a query string (e.g. /path?user=id).
  5. Optionally set the max character that will be checked by HAProxy
  6. Optionally set the Hold time (defined with suffixes ms, s, m, h etc.), the time an unused cookie is removed from memory

For more information on the different directives that are supported, please read the following HAProxy instruction: https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-appsession

Source based cookie (SourceBased)

When you want to track users based on IP address, the source-based cookie is the easiest option to go with.

  1. From the Stickiness method drop-down menu, choose SourceBased.
  2. Fill in a Sticky Name  that will be displayed in the dashboard for this rule
  3. Fill in the Table sizedefined in binary (2^10, 2^20, 2^30) with suffix k, m or g, ‘1k’ is 1024 bytes and ‘1m’ is 1.048.576 bytes etc.). With this option you can manage how many entries can fit in the table, this value has direct impact on memory usage.
  4. In the Expires field, fill in the desired expiration time. This defines the maximum duration (defined with suffixes ms, s, m, h, etc.) of an entry in the table since it was last created, refreshed or matched.

See for more information: https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#stick-table

Configuring a private network between private cloud and a  dedicated server

Leaseweb Private Networking enables you to connect different services from Leaseweb over a dedicated private connection. So, next to isolated networks, that are internal to CloudStack, you can run other services like dedicated servers within a private/ isolated network together with your CloudStack instances.

First enable private networking for your Elastic Compute domain, before you can configure it in CloudStack panel.


  1. In the Customer Portal from the left menu bar, select Network > Private Network and click the Elastic Compute tab.
  2. Click Add to Private Network.
  3. Select the Elastic Compute domain and confirm by clicking Add to Private Network in the pop-up window.

    Private Network in CloudStack

    After enabling Private Network for the Elastic Compute domain, a new network in CloudStack gets created of type Shared with suffix "-PrivateNetworking".

    DHCP

    Private networks between dedicated servers and cloud always have DHCP server providing internal IP addresses. However, obtaining the address via DHCP it is up to you: you can statically assign any IP address instead.

    Private network throughput

    Private networks between dedicated servers and cloud servers have throughput 100Mbit per second per virtual machine.

To add an instance in CloudStack to the private network you will need to add a NIC from the private network to the instance.

  1. From the left menu in the CloudStack panel choose Compute > Instances and click the instance.
  2. Choose the NICs tab in the instance overview page, click + Add network to VM.
  3. Select the private network from the drop-down menu and confirm by clicking OK.

    The new network interface should appear in your virtual machine operating system.

Get Support

Need Technical Support?

Have a specific challenge with your setup?

Create a Ticket