TrueOS + ipfw + natd + dhcpd + fwbuilder + virtualbox
I recently started playing around with freebsd in the form of TrueOS (originally PC-BSD).
I inherited this maxed out 12 core Dell Precision T7500 beast that was decommissioned from our defunct bioinformatics section and due to a building power outage had just lost a 17 year old NAT box that I was using to share the net with a wireless access point.
I set out configuring the Dell to replace the failed NAT/wireless machine.
First I installed TrueOS a few dozen times and completely broke it a few times. At one point, I accidentally hosed the system by installing bash and using vipw to edit the shell for the root user and adding the path /bin/bash instead of /usr/local/bin/bash. Yeah, I could have probably fixed it with a rescue boot and a symlink, but I just re-installed... again.
One of the first things I found out is that searching for specific help with all things freebsd is a bit daunting. Luckily, I got my unix guru status in the mid-90's on solaris and sunOS so I was able to sort out where the good places are to find non-confusing non-obtuse examples when "man -k" was of no use.
As I was building yet another firewall and was using ipfw instead of the iptables and feeling a bit out of my element, I decided to complicate^H^H^H^H^H^H^H^H^Hsimplify things by introducing another learning curve on top of the learning curve. Fwbuilder is a great idea: create a firewall once, and then have the capability of generating firewall scripts that can work on numerous firewall appliances and different OS'en.
So I kicked fwbuilder around for a few weeks and had ZERO LUCK getting nat to work correctly. As it turns out fwbuilder does not generate NAT rules for ipfw EVEN IF YOU HAVE DEFINED THEM IN FWBUILDER. I also had an issue with uploading the firewall script through ssh stalling out and never uploading that was fixed by setting up an ssh-agent key and eliminating the password prompting.
I ended up writing a script called natstart in order to activate the NAT rules:
natstart script:
ipfw add 11 divert natd all from any to any via em0
ipfw add 12 check-state
So basically all I have to do to test the firewall setup is:
ipfw flush ; ./fwb_firewall_script.fw ; natstart
Fwbuilder has the ability to define prologue and epilogue scripts, but they don't seem to be working. I will kick that around eventually.
Update:
It seems like these prologue and epilogue scripts aren't even added to the generated firewall script. All I see added is a line with the command: epilog_commands - which does nothing.
If you add any ipfw commands to the generated script, keep in mind that the script is creating a rule "set" and then swaps that set for the active set at the end of the script. So my rules would have to be changed to:
"$IPFW" add 11 set 1 divert natd all from any to any via em0
"$IPFW" add 12 set 1 check-state
I was eventually able to modify one of the NAT firewall templates that fwbuilder includes to produce a half-working NAT firewall... and then our director's mac SSD decided to completely die - necessitating immediate access to our in-house wifi for a temporary laptop to work with. The firewall that I activated on the Dell had some issues when one tried to do X11 forwarding through ssh on the machine but still provided a usable gateway to the net for the wireless setup. So at this point I was unable to do any testing with the live firewall as it would interrupt my director's connections to file shares and any open documents on those shares...
So I decided that another level of slapdash fuckery was required: virtualbox
I updated my virtualbox install on my workstation and found a forgotten Ubuntu-studio 15.10 VM that I had installed for "shits and gigs". My mission was now to set up some kind of virtual network where this Ubuntu install is attached to the "inside" interface of a TrueOS VM.
Great.
So here's how that works:
On the TrueOS VM I set up two network interfaces:
interface 1: attached to NAT
interface 2: attached to Internal Network (which I named TrueOS_INSIDE)
On my Ubuntu studio VM I changed the adapter to attach to the internal network TrueOS_INSIDE
Then I played around with fwbuilder like a maniac, restarting my firewall a few dozen times after each change, until things were finally working.
Now I will find out when a good time is to test this firewall on the non-virtual machine.
Update:
Here are the contents of /etc/rc.conf that one needs to set this up:
hostname="trueos.mydomain.net"
ifconfig_em0="DHCP" # Outside Interface
ifconfig_dc0="inet 192.168.1.1 netmask 255.255.255.0" # Internal Interface
firewall_enable="YES"
firewall_script="/etc/fw/firewall.fw" # I usually make this a symlink pointing to the active firewall script
firewall_logging="YES"
gateway_enable="YES"
forward_sourceroot="YES"
natd_enable="YES"
natd_interface="em0" # Divert traffic to Outside Interface
natd_flags="-log -u -m -dynamic -n /dev/em0"
dhcpd_enable="YES"
dhcpd_flags="-q"
dhcpd_interface="dc0"
dhcpd_withumask="022"
dhcpd_chuser_enable="YES"
dhcpd_withuser="dhcpd"
dhcpd_withgroup="dhcpd"
dhcpd_chroot_enable="YES"
dhcpd_devfs_enable="YES"
dhcpd_rootdir="/var/db/dhcpd"
zfs_enable="YES"
ipv6_activate_all_interfaces="NO"
# for dhcpd
snmpd_enable="YES"
snmpd_flags="-a"
snmpd_conffile="/usr/local/share/snmp/snmpd.conf /etc/snmpd.conf"
snmptrapd_enable="YES"
snmptrapd_flags="-a -p /var/run/snmptrapd.pid"
/usr/local/etc/dhcpd.conf:
# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#
# option definitions common to all supported networks...
option domain-name "trueos.mydomain.net";
option domain-name-servers my.domain.name.server; # change to your DNS IP
option subnet-mask 255.255.255.0;
default-lease-time 600;
max-lease-time 7200;
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.2 192.168.1.255;
option routers 192.168.1.1;
}
You will likely have to install the following packages to get this kind of a setup to work:
# pkg install FreeBSD-natd
# pkg install isc-dhcp43-server
# pkg install fwbuilder
# pkg install net-snmp
I inherited this maxed out 12 core Dell Precision T7500 beast that was decommissioned from our defunct bioinformatics section and due to a building power outage had just lost a 17 year old NAT box that I was using to share the net with a wireless access point.
I set out configuring the Dell to replace the failed NAT/wireless machine.
First I installed TrueOS a few dozen times and completely broke it a few times. At one point, I accidentally hosed the system by installing bash and using vipw to edit the shell for the root user and adding the path /bin/bash instead of /usr/local/bin/bash. Yeah, I could have probably fixed it with a rescue boot and a symlink, but I just re-installed... again.
One of the first things I found out is that searching for specific help with all things freebsd is a bit daunting. Luckily, I got my unix guru status in the mid-90's on solaris and sunOS so I was able to sort out where the good places are to find non-confusing non-obtuse examples when "man -k" was of no use.
As I was building yet another firewall and was using ipfw instead of the iptables and feeling a bit out of my element, I decided to complicate^H^H^H^H^H^H^H^H^Hsimplify things by introducing another learning curve on top of the learning curve. Fwbuilder is a great idea: create a firewall once, and then have the capability of generating firewall scripts that can work on numerous firewall appliances and different OS'en.
So I kicked fwbuilder around for a few weeks and had ZERO LUCK getting nat to work correctly. As it turns out fwbuilder does not generate NAT rules for ipfw EVEN IF YOU HAVE DEFINED THEM IN FWBUILDER. I also had an issue with uploading the firewall script through ssh stalling out and never uploading that was fixed by setting up an ssh-agent key and eliminating the password prompting.
I ended up writing a script called natstart in order to activate the NAT rules:
natstart script:
ipfw add 11 divert natd all from any to any via em0
ipfw add 12 check-state
So basically all I have to do to test the firewall setup is:
ipfw flush ; ./fwb_firewall_script.fw ; natstart
Fwbuilder has the ability to define prologue and epilogue scripts, but they don't seem to be working. I will kick that around eventually.
Update:
It seems like these prologue and epilogue scripts aren't even added to the generated firewall script. All I see added is a line with the command: epilog_commands - which does nothing.
If you add any ipfw commands to the generated script, keep in mind that the script is creating a rule "set" and then swaps that set for the active set at the end of the script. So my rules would have to be changed to:
"$IPFW" add 11 set 1 divert natd all from any to any via em0
"$IPFW" add 12 set 1 check-state
I was eventually able to modify one of the NAT firewall templates that fwbuilder includes to produce a half-working NAT firewall... and then our director's mac SSD decided to completely die - necessitating immediate access to our in-house wifi for a temporary laptop to work with. The firewall that I activated on the Dell had some issues when one tried to do X11 forwarding through ssh on the machine but still provided a usable gateway to the net for the wireless setup. So at this point I was unable to do any testing with the live firewall as it would interrupt my director's connections to file shares and any open documents on those shares...
So I decided that another level of slapdash fuckery was required: virtualbox
I updated my virtualbox install on my workstation and found a forgotten Ubuntu-studio 15.10 VM that I had installed for "shits and gigs". My mission was now to set up some kind of virtual network where this Ubuntu install is attached to the "inside" interface of a TrueOS VM.
Great.
So here's how that works:
On the TrueOS VM I set up two network interfaces:
interface 1: attached to NAT
interface 2: attached to Internal Network (which I named TrueOS_INSIDE)
On my Ubuntu studio VM I changed the adapter to attach to the internal network TrueOS_INSIDE
Then I played around with fwbuilder like a maniac, restarting my firewall a few dozen times after each change, until things were finally working.
Now I will find out when a good time is to test this firewall on the non-virtual machine.
Update:
Here are the contents of /etc/rc.conf that one needs to set this up:
hostname="trueos.mydomain.net"
ifconfig_em0="DHCP" # Outside Interface
ifconfig_dc0="inet 192.168.1.1 netmask 255.255.255.0" # Internal Interface
firewall_enable="YES"
firewall_script="/etc/fw/firewall.fw" # I usually make this a symlink pointing to the active firewall script
firewall_logging="YES"
gateway_enable="YES"
forward_sourceroot="YES"
natd_enable="YES"
natd_interface="em0" # Divert traffic to Outside Interface
natd_flags="-log -u -m -dynamic -n /dev/em0"
dhcpd_enable="YES"
dhcpd_flags="-q"
dhcpd_interface="dc0"
dhcpd_withumask="022"
dhcpd_chuser_enable="YES"
dhcpd_withuser="dhcpd"
dhcpd_withgroup="dhcpd"
dhcpd_chroot_enable="YES"
dhcpd_devfs_enable="YES"
dhcpd_rootdir="/var/db/dhcpd"
zfs_enable="YES"
ipv6_activate_all_interfaces="NO"
# for dhcpd
snmpd_enable="YES"
snmpd_flags="-a"
snmpd_conffile="/usr/local/share/snmp/snmpd.conf /etc/snmpd.conf"
snmptrapd_enable="YES"
snmptrapd_flags="-a -p /var/run/snmptrapd.pid"
/usr/local/etc/dhcpd.conf:
# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#
# option definitions common to all supported networks...
option domain-name "trueos.mydomain.net";
option domain-name-servers my.domain.name.server; # change to your DNS IP
option subnet-mask 255.255.255.0;
default-lease-time 600;
max-lease-time 7200;
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.2 192.168.1.255;
option routers 192.168.1.1;
}
You will likely have to install the following packages to get this kind of a setup to work:
# pkg install FreeBSD-natd
# pkg install isc-dhcp43-server
# pkg install fwbuilder
# pkg install net-snmp