HomeLab Networking Upgrade [and progress update] !

Where We Started...

Way back in October of 2022, I began the creation of a personal project. A HomeLab. With this project, I planned on expediting my learning, hosting my own services, and getting into a wonderful new hobby!

When it started, my "Lab" really boiled down to one single reused and abused Lenovo ThinkPad L440 laptop. Boasting a staggeringly weak early i3 processor, and a screaming 8GB of RAM...it knew it was perfect for large-scale virtualization (jokes...of course).

With this laptop, I delved into Proxmox, PiHole, Windows NAS shares, basic network configuration etc. Backing up my mighty laptop, was the classic $40 Walmart-purchased router that you see here. It features NO VLAN Support, NO Subnetting Support, and no advanced customization of any kind. What it did, though, was give me a gateway to the internet. Armed with such a tool, I could tinker with all manner of .iso images on my tiny Proxmox hypervisor. This modest setup was the beginning of a passion for learning Information Technology!

The Upgrade:

<Hardware>

As can be seen from the topmost picture of this article, my hardware has come a long way since October. From a folding table and some empty boxes, to a genuine rack with business-class equipment! From the top, my rack includes the TP-Link Jetstream TL-SG3428 L2+ switch and two Dell PowerEdge R510 Servers that had been retired from an enterprise environment (this means iDRAC management!). Towards the bottom I have 2 *unused* Dell PowerVault MD1000 units. They generally remain off due to the excessive amount of heat and noise that they produce. Lastly, at the bottom of my rack, sits my Lenovo ThinkPad L440 laptop...still on, still connected, and waiting for a new purpose!

The scale of this upgrade is hard to overstate. The switch is amazing, and even includes 4 SFP ports for fiber uplink, if I so decide to make that switch! With gigabit connectivity, the potential bandwidth of THIS switch FAR exceeds that of the integrated switch on the old Archer Router. The Dell servers are also a huge bonus! Though they bare the same model designation, they differ in terms of configuration. The first R510, which I will call ServerA, boasts dual Intel Xeon X5650 CPUs clocked at 2.67Ghz for a total of 12 physical cores. It also has 80GB of installed RAM and 6 total Gigabit NICs, not counting the iDRAC. The other server, which I will call ServerB, boasts a single Intel Xeon E5649 CPU clocked at 2.533Ghz with 12 Physical Cores. It has 24GB of RAM and 2 Gigabit NICs (integrated iDRAC express). Though these servers are older and weaker than most offerings today, they provide a fantastic sandbox for somebody unfamiliar with real enterprise server hardware, like myself. It has been on these boxes, that I have learned virtually everything that I have used to better my IT knowledge today!

<Software / OS>

It's all well and good to have such beefy hardware in the server rack, but what good is it if it doesn't run anything of note? It is time to talk about my current software setup, and how it has improved from a simple one-Proxmox laptop situation, starting from the most important part...routing.

My network isn't much use without robust routing. Thankfully, I have made quite the improvement in that department. My routing is now being handled by the famed Pfsense Community Edition routing software. Based on the rock-solid OpenBSD operating system, this enterprise-quality routing package includes a fully customizable firewall, VLAN support, Subnetting support, syslog integrations, UTM plugins, network monitoring, and so much more! This isn't the coolest part, though. The best part about my Pfsense setup, is that it is not running on any kind of physical appliance. My Pfsense router is entirely virtual.

The virtualization of my routing solution provides a great segue into another pertinent software facet of my HomeLab, what are Servers A & B doing? Well, they are serving as hypervisors! This explanation may be best set out in bullet points:

SERVER A:

• Windows Server 2019 Datacenter w/ HyperV Role installed
• Guest 1: Pfsense router
• Guest 2: A Test Windows 10 PC (more later)
• Guest 3: A Test Windows 10 PC (more later)
• Guest 4: Ubuntu Server 22.04 (more later)

SERVER B:

• Proxmox VE 7.4-3
• Guest 1: PiHole DNS for custom name records and advertisement blocking
• Guest 2: TP-Link Omada Controller for managing SDN from a Single Pane of Glass
• Guest 3: Windows 10 VM that serves as a file share and printer share
• Guest 4: Ubuntu 22.04 LTS Minecraft Server

<Wireless>

As I am sure you can imagine, Dell R510 rackmount servers do not come with wireless radios built into them, and a virtualized Pfsense box can't exactly accept any wireless clients. So, where do I broadcast an SSID? How do I utilize Wi-Fi?

This question is answered by the triumphant return of my TP-Link Archer AX1500 wireless router! As it turns out, it can be configured to act as a wireless access point and bridge! In this mode, it disables its DHCP server, internal firewall, and all traffic shaping / QoS behaviors. It simply broadcasts a single SSID and that is it. Luckily, that is all I need (for now)!

The Process:

In this section of the article, I will attempt to give a high-level explanation of my migration process. Since I manage everything from my desktop computer, doing this changeover was tricky, and had to be done in pieces so as not to accidentally lock myself out of the GUIs of a major appliance due to not being able to route to my rack from my desk. That being said, the process went something like this:

1. Prepare the Windows Server HyperV host role
2. Create the Pfsense virtual machine
3. create two virtual external switches within HyperV, one for WAN and one for LAN
4. Make the WAS vSwitch inaccessible from the host (this is for security)
5. Assign these switches as the network interfaces of the Pfsense router
6. Give BOTH the WAN and LAN sides of the Pfsense VM bogus LAN addresses. This is to be able to access them via the web-interface
7. Access the web interface and set all settings to mimic the currently installed router EXCEPT for the IP Address (can't create an IP conflict)
8. Access old router's web GUI and change it over to Access Point mode
9. Unplug ISP WAN from old router and plug it into the Physical NIC on the server that corresponds to the vSwitch that you have labled for WAN
10. Assign a device static IP information that allows it to use Pfsense as its gateway.
11. Attempt to ping out to an external host such as 8.8.8.8
12. If successful, then the process is complete! Begin changing over your remaining clients and make any necessary changes. The DHCP clients should remain connected with their cached DHCP addresses, which fall into the same scope.

HAVING actual server hardware made this process go so much more smoothly. For example, being able to manage my Dell servers from OUT of band using the iDRACs was crucial. It allowed me to remote into them to make configuration changes WITHOUT having to have a physical display, or even while the hypervisors were experiencing issues. Even so, there were definitely still MANY speed bumps along the way. I will describe two of the main ones here!

<Issue 1: HyperV Software Incompatibility>

Believe it or not, Microsoft HyperV has a software compatibility issue with Pfsense's latest stable build. This is due to the usage of a technology called Reactive Segment Coalescing (RSC). In short, this feature attempts to reduce host CPU utilization and increase network throughput by combining multiple TCP segments into fewer, larger segments. For reasons that are still not clear to me, or the Pfsense forums, leaving this feature on causes my WAN side speeds to plummet to nearly Dial-Up level numbers ~56Kbps. Furthermore, this feature can only be disabled from Powershell. In case anybody happens across this article with the same issue, here is the command:

get-vmswitch -name * | set-vmswitch -EnableSoftwareRsc $false        

After disabling this feature, I watched my throughput balloon back up to expected rates! I also found that it helped to disable any hardware offloading being performed by the Pfsense virtual machine. This was at the suggestion of many seasoned forum-goers.

<Issue 2: Changing IP Information on Linux Server Guests>

Fortunately, this one was not so much of an issue as it was a gap in my knowledge. I had to learn some YAML syntax to be able to properly change over the IP configurations of some of my virtual guests. Specifically my TP-Link Omada controller, which runs on Ubuntu Server. After reading some documentation, and learning some YAML, I was able to properly write a new configuration file for "networkd" within Ubuntu Server 22.04 LTS!

What Did I Learn?

I want to take a minute here, as the article winds down, to list out some of the more notable things that I learned during this nearly day-long process of upgrading, reconfiguring, migrating, and re-racking various components of my HomeLab:

• I learned how to configure Pfsense at a basic - intermediate level
• I learned how to properly virtualize a routing appliance and troubleshot unique issues that arose as a result of this unique design choice using Powershell
• I learned how to manipulate and configure Microsoft HyperV, a relatively new hypervisor to me
• I learned a lot about virtual networking, NIC passthrough, and virtual switch topology
• I learned how to write some fundamental configuration files in YAML
• I learned how to perform a live appliance migration within a limited time frame
• I learned how to properly reconfigure and refactor my network IP-schema to make more sense, and properly segment DHCP clients from infrastructural static clients

I learned more than just these items, but this is a great overview of some of the highlights of the project! Writing this now, I am left with a network that makes sense, is ordered properly, and runs flawlessly (insofar as current testing has revealed). HomeLab is still the best and most fun way to learn anything in Tech, and it has given me the opportunity to learn a plethora of amazing skills; I can't wait to see what else I can add to it, and I already have some ideas!

What's Coming Next?

I want to wrap up with a list of some ideas for what is coming next in my personal lab here. Now that I have a properly organized network, I am looking to see how far I can push the limits on my legacy server hardware.

1. Adding an Active Directory Lab to my network. I want to do this to better understand the worlds most popular user management directory service! I am interested in using the test Windows VMs that I mentioned earlier as examples to enroll into a Windows domain and play around with
2. Adding Group Policy to my network. This is closely related to the previous topic. I want to use GPOs to perform a few basic management tasks on my tests users and computers. Things such as: Forcing SMB Drive Mappings, Deploying mandated software, Managing Windows Updates, Enforcing a lock-screen / password requirement etc
3. Creating multiple VLANs and Subnets for separated experimental endpoints and for isolating traffic from endpoints that may be exposed to the public internet
4. Creating a Wireguard Tunnel to a cloud VPS to create remote access to my network. This will help allow me to start creating public services
5. Create a TrueNAS device on my network for file sharing and iSCSI services
6. Syslog & SIEM implementation
7. Add a proper Patch Panel to my rack
8. Rackmounted PDU / UPS systems

There are more things that I am looking to do and, very soon, I will be taking my HomeLab ambitions to the cloud! Microsoft Azure and Oracle Cloud are on my to-do list for technologies to learn.

Thank you for reading to the end of this article. This was a much more casual / blog-style post and I like that! I will continue to publish more articles as I learn new things, attempt new experiments, and grow as an Engineer / Technician! Please comment and let me know what things I should try or what I can do better!

Thank you,

Tyler Sell