Skip to content

Instantly share code, notes, and snippets.

@jpawlowski
Created August 12, 2012 12:06
Show Gist options
  • Save jpawlowski/3331593 to your computer and use it in GitHub Desktop.
Save jpawlowski/3331593 to your computer and use it in GitHub Desktop.
Debian network configuration for Proxmox VE server running on a Hetzner host
# /etc/network/interfaces
#
auto lo
iface lo inet loopback
# device: eth0
iface eth0 inet manual
# IPv4 bridge
# (connect ONLY your firewall/router KVM instance here, this is the WAN device!)
auto vmbr0
iface vmbr0 inet static
# Hetzner primary WAN IP
address 176.9.xxx.xxx
# Hetzner primary WAN IP broadcast address
broadcast 176.9.xxx.xyz
# This netmask needs to have all bits set
netmask 255.255.255.255
# Our gateway is reachable via Point-to-Point tunneling
# put the Hetzner gateway IP address here twice
pointopoint 176.9.xxx.yyy
gateway 176.9.xxx.yyy
# Virtual bridge settings
# this one is bridging physical eth0 interface
bridge_ports eth0
bridge_stp off
bridge_fd 0
# add a static route through the Hetzner gateway IP
# for the subnet our primary IP belongs to
up route add -net 176.9.xxx.zyx netmask 255.255.255.224 gw 176.9.xxx.yyy vmbr0
# Add routing for up to 4 dedicated IP's we get from Hetzner
# You need to
up ip route add 176.9.xxx.xx1/32 dev vmbr0
up ip route add 176.9.xxx.xx2/32 dev vmbr0
up ip route add 176.9.xxx.xx3/32 dev vmbr0
up ip route add 176.9.xxx.xx4/32 dev vmbr0
# Assure local routing of private IPv4 IP's from our
# Proxmox host via our firewall's WAN port
up ip route add 192.168.0.0/16 via 176.9.xxx.xx1 dev vmbr0
up ip route add 172.16.0.0/12 via 176.9.xxx.xx1 dev vmbr0
up ip route add 10.0.0.0/8 via 176.9.xxx.xx1 dev vmbr0
# IPv6 bridge
# (connect ONLY your firewall/router KVM instance here, this is the WAN device!)
iface vmbr0 inet6 static
address 2a01:4f8:151:XXX::3
netmask 64
up ip -6 route add 2a01:4f8:151:XXX::1 dev vmbr0
up ip -6 route add default via 2a01:4f8:151:XXX::1
# Virtual switch for DMZ
# (connect your firewall/router KVM instance and private DMZ hosts here)
auto vmbr1
iface vmbr1 inet manual
bridge_ports none
bridge_stp off
bridge_fd 0
# Virtual switch for Private LAN
# (connect your firewall/router KVM instance and private hosts here)
auto vmbr2
iface vmbr2 inet manual
bridge_ports none
bridge_stp off
bridge_fd 0
# Virtual switch for Test Data Center
# (connect your firewall/router KVM instance and private hosts here)
auto vmbr3
iface vmbr3 inet manual
bridge_ports none
bridge_stp off
bridge_fd 0
# /etc/pve/qemu-server/100.conf
#
# This is an example KVM host configuration I use for my
# Vyatta router instance.
# It was created with the Proxmox web interface and should
# just introduce you to the used network configuration
# (see the 5 network interfaces and their bridge connections).
# That means you won't need to hack the configuration file yourself,
# just keep settings here in mind when creating your virtual host
# in the Proxmox interface.
#
# net0 - WAN interface used for dedicated IPv4 connection handling
# net1 - WAN interface used for dedicated IPv6 connection handling
# net2 - DMZ interface for the majority of my virtual OpenVZ and KVM hosts
# net3 - Private hosts only accessible via VPN
# net4 - Private hosts only accessible via VPN
#
# Of course you could use less interfaces (net0+net1 could be combined
# in a normal dual-stack configuration and net2/3/4 could be one
# interface only if you don't want/need such a complex configuration).
#
# IMPORTANT: net0 uses the MAC address that was assigned by Hetzner to one
# of my additional IPv4 addresses. This is essential to have a working connection.
# I also requested MAC addresses for the other IP's but it's sufficient to have only one of the MAC's
# in your configuration.
#
# Now go and have your firewall instance listening to all your additional
# IPv4 addresses and setup destination NAT to your private IPv4 addressed
# hosts connected to net2.
boot: cd
bootdisk: virtio0
cores: 2
cpu: host
ide2: none,media=cdrom
memory: 2048
name: msys-firewall-instance
# IMPORTANT: net0 has set the MAC address assigned by Hetzner
net0: virtio=00:50:56:00:XX:YY,bridge=vmbr0
net1: virtio=56:4A:05:76:D6:F8,bridge=vmbr0
net2: virtio=76:EE:88:BA:17:DF,bridge=vmbr1
net3: virtio=EE:10:82:D7:3C:14,bridge=vmbr2
net4: virtio=DA:18:23:03:D0:C7,bridge=vmbr3
onboot: 1
ostype: l26
sockets: 2
startup: order=1
virtio0: local:100/vm-100-disk-1.qcow2,cache=writeback
@nikole
Copy link

nikole commented Jun 5, 2013

I used you config on a Hetzner server successfully (thanks!) however only the KVMs with public IPs are working. I cannot access the internet from inside KVMs with private IPs.
Any ideas why?

I am guessing I did not provide the correct settings inside the gues KVM. Can you please post an example?

Thank you!!!!

@jpawlowski
Copy link
Author

You need a running firewall KVM (like pfSense) which is dual-homed (1 public IP and 1 private IP for each of the other virtual switches) to vmbr0 and vmbr1/2/3 etc. This instance needs to be configured to rewrite and translate your private IP traffic.

Just connect all your private KVM's to vmbr1/2/3 then and use the private IP of your firewall instance as gateway.

@mlanin
Copy link

mlanin commented Jul 2, 2013

Thank you for your gist. Helped much! But I have a problem anyway. Maybe because I have no experience in networking :)

First of all I set up a firewall machine with 2 interfaces: one for vmbr0 and one for vmbr1. Then I installed pfSense on it, and pfSense initiated first interface as WAN interface with my additional hetcner IP and LAN with static IP (192.168.0.1) on the second. Also it started DHCP server for LAN.

When I'm tryng to create new private VM I'm connecting it to vmbr1, but OS installer (ex. Debian) can't see my firewall and it's DHCP. So my new VM become cut from the network. Even if I set network manually and set it's address like 192.168.0.10 and gateway 192.168.0.1 (LAN IP of my firewall) it's still can't connect to the internet or LAN. What I'm doing wrong?

Thanks!

@key21
Copy link

key21 commented Oct 31, 2013

I have problems to undersstand this. Well, my subnet works fine, but I have one single IP that I can't make work.

The most important part is working (subnet IPs), and VMs are working fine. The wiki is clear on this part (http://wiki.hetzner.de/index.php/Proxmox_VE/en).

But need to know how to make extra IP work, I can't handle this and support doesn't seem to explain anything, just repeat something that is inconsistent with the wiki. Probably it's me, that I'm stupid .... but I need to understand.

The wiki says that I must configure the host like you say here in the host: up ip route add 176.9.xxx.xx1/32 dev vmbr0. Okay, but in the KVM connected to vmbr0 is it correct to configure it like this?

iface eth0 inet static
address
netmask 255.255.255.255
pointopoint


gateway

This doesn't work :(

@phob
Copy link

phob commented Apr 12, 2015

Two years later this is still the only source for the setup with proxmox and a firewall/router as vm.
But now with a firewall build in proxmox 3.4 and new ways to configure the networt in proxmox:
Is this still the best solution? Is this the solution you still use?

@mvaschetto
Copy link

mvaschetto commented Jan 25, 2018

Thanks for share this!

This configuration still working, I have use a bit different configuration, but I'm started from this!

What I do with the hetzner root server is use the single ipv4 for management in the physical server and restrict it to be accessed from certain ip, and the ipv6 /64 complete subnet to the pfsense; below the configuration I have use to be done with my porpouse:

auto lo
iface lo inet loopback
iface lo inet6 loopback


auto eth0
iface eth0 inet static
        address 195.xxx.yyy.zzz
        netmask 255.255.255.255 # netmask have to be .255 (/32) unless you want the server talk to neighbors 
        gateway 195.xxx.yyy.zzz # this is the gateway you find just after image restored
        pointopoint 195.xxx.yyy.zzz # insert gateway here! (yes, twice) 

iface eth0 inet6 static
        address aaaa:bbbb:cccc:dddd::2
        netmask 128 # change this to /128 (same reason of before and like this we can assign entire /64 ipv6 to the bridge)
        gateway fe80::1

auto vmbr0
iface vmbr0 inet manual
        # WAN Interface
#      address 195.xxx.yyy.zzz # Same address of physical interface
#      netmask 255.255.255.255 # same reason of before
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        # Route an eventually ipv4 subnet (!NOTE: in hetzner an ipv4 /29 subnet when routed is completely usable)
#        up route add -host 195.xxx.yyy.zz0 dev vmbr0
#        up route add -host 195.xxx.yyy.zz1 dev vmbr0
#        up route add -host 195.xxx.yyy.zz2 dev vmbr0
#        up route add -host 195.xxx.yyy.zz3 dev vmbr0
#        up route add -host 195.xxx.yyy.zz4 dev vmbr0
#        up route add -host 195.xxx.yyy.zz5 dev vmbr0
#        up route add -host 195.xxx.yyy.zz6 dev vmbr0
#        up route add -host 195.xxx.yyy.zz7 dev vmbr0

        # Says the host where the answer for subnet behind pfsense should be routed
        #up ip route add 172.xxx.yyy.zzz/24 via 195.xxx.yyy.zz0 dev vmbr0 #(the ip of via have to be the pfsense ipv4 WAN ip)

iface vmbr0 inet6 static
        address aaa:bbb:ccc:ddd::2
        netmask 126
        # Metric 1 because kernel set up a ipv6 route send the /64 subnet over (::) no next hop
        up ip -6 route add aaa:bbb:ccc:ddd::/64 via aaa:bbb:ccc:ddd::3 dev vmbr0 metric 1 #(the ip of via have to be the pfsense ipv6 WAN ip)

auto vmbr1
iface vmbr1 inet manual
        # LAN INTERFACE 
        bridge_ports none
        bridge_stp off
        bridge_fd 0

In the routed configuration do not need any additional subnet and/or second mac address, if you want to have "just!" the /64 subenet 💃

Copy link

ghost commented May 23, 2018

Thank you so much ... it helped me understand a lot with those two files! 🎁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment