An introductory guide to VLAN configuration on a Proxmox virtualization server so that you can group related VMs onto their own subnet.
A virtualization server allows you to run multiple machines, virtual machines (VMs), on one physical device, also known as the host. In a commercial production system, these virtualization servers will often run similar VMs, but for us with homelabs, there could be many different VMs each for different tasks. One VM could be for development work, another could be handling HEVC encodes to save valuable storage, others might be hosting an NGINX reverse proxy and web servers. In this guide, we will discuss configuring your Proxmox virtualization server to use VLANs so that you can group related VMs onto their own subnet.
The Problem: Why Would You Want to Set Up VLANs on Proxmox?
Before we begin the technical how-to, let's begin with our motivation for setting up VLANs on Proxmox.
For security, as well as organizational purposes, physical machines are often separated on the network from each other by VLANs. By logically separating devices based on their functionality with these VLANs, we can make sure that our family's personal devices aren't sitting out in the open on the same subnet exposed to our internet-facing web servers. This is fairly easy on your regular network setup because the devices are physically separate from each other and so each ethernet port physically connected to a device can be assigned an individual VLAN.
However, this system starts to break down when faced with virtualization servers. This is because diverse virtual machines are all sitting on the same physical host, forcing each VM to share the same physical connection. With a standard bridge between the individual VM and the host's NIC, we necessarily end up with each VM on the same subnet as the Proxmox host itself. Additionally, we end up with each VM on the same subnet as every other VM on that host. Not ideal.
The Solution: Making Your Proxmox's Virtual Bridge VLAN-Aware
Thankfully there's a way around this. In Proxmox, you can make your virtual bridge VLAN-aware so you can pass multiple VLANs through to your Proxmox server using only a single physical port. The individual VMs can then be configured to use whichever VLAN you choose.
Assumptions/Prerequisites:
In writing this guide, I have made the following assumptions:
- You have already set up VLANs on your network. If you haven't, check out my other guide on Creating VLANs with an L3 Switch on pfSense.
- Since we're going to be passing multiple VLANs from our L3 switch to the Proxmox server over a single physical ethernet cable, we'll need a port on the switch to be configured as a trunk port. (To be covered in a future guide.)
- Your network interface card (NIC) has two ports. This is necessary since one port is claimed by the Proxmox host when you install Proxmox. You may be able to configure Proxmox's default virtual bridge the same way as I describe below to allow VLAN traffic (or you may be able to create a separate virtual bridge and apply it to the same physical port) but I haven't tried it. Since I am working remotely, I am not willing to risk knocking my virtualization server offline by monkeying with the Proxmox server's management bridge.
- You have Proxmox installed on a server. (This will also be covered in a future guide.)
Taking Inventory of Your Proxmox Server's Current Network Configuration:
Let's begin this tutorial by taking stock of your Proxmox server's current network configuration.
1. Log in to your server's administration page and navigate to Proxmox's network configuration. This can be found by navigating the following path: Server node > System > Network:
On the network config page, your configuration will look something like the following:
So you know what you're doing and not just blindly following my guide, let's break this down a little. This is a list of your network devices. The two network devices, eno1 and enp4s0, are physical network devices- i.e. they represent physical network interface ports. vmbr0 is a Linux bridge and, just as the name implies, it bridges your VMs to a physical network device (hence why it's called vmbr0). This one happens to be bridging to enp4s0, found in the "Ports/Slaves" column. Under the CIDR column you'll see that my Proxmox server is on the 10.0.20.0/24 subnet and the gateway in use for routing is found at 10.0.20.221.
(Again, this is discussed in my Guide for Setting Up VLANs with pfSense and an L3 Switch. Things will make a lot more sense if you read that article, so I highly recommend you do so, if for no other reason than for a better understanding of how networks/routing works).
Now, if you leave this set up as is and start to create VMs, they will default to bridging to vmbr0. By itself this is fine and, for the longest time, this has been my configuration on my Proxmox server since the 10.0.20.0/24 subnet is my DataCenter VLAN, but as I said above, eventually you may want to have very different VMs on your host and when that happens, you'll probably want to have multiple VLANs as well. This is the situation I find myself in. I want to ultimately migrate my web servers from my Raspberry Pi's to my Proxmox server to take advantage of both the superior hardware that a real server has but also to take advantage of the paravirtualization advantage inherent in having the same devices on the same virtual switch on the same machine.
Create a Linux Bridge to Connect Proxmox to Multiple VLANs:
2. If you haven't done so already, connect your Proxmox server's second network port to your L3 switch.
Remember: The port on the L3 switch you're connecting to has to be set up as trunk port with the VLANs you wish to used tagged on it.
3. Create a new Linux Bridge in Proxmox. Check the box next to "VLAN aware" and bind it to the physical network port you used in step 2 above.
By making our new network bridge, that's connected to our switch's trunk port, VLAN-aware, we are telling Proxmox to tag the traffic it carries with the VLAN ID so that when it reaches our real L3 switch, the switch knows what to do with it.
Now, unfortunately Proxmox seems to just make all VLANs available on this port, but let's say we want a little more control than that. In that case, we just need to complete one more step. If you're fine with all VLANs being available on that port, then you're already done.
4. Modify /etc/network/interfaces so that only certain VLANs are handled by your new Linux bridge.
SSH into your Proxmox server and run the following command to edit /etc/network/interfaces:
nano /etc/network/interfacesScroll down and you should see your new Linux bridge. Update bridge-vids with the VLAN IDs you want to make available. In my case, I only want VLAN 10 (my web server VLAN) available:
5. Restart your Proxmox server for changes to take effect.
Now, whenever you configure one of your VMs, you'll be able to add a network device and have it use one of your other VLANs:
