Classic useless answer but nothing ATM š. Iāve been travelling for a few months and wonāt be able to host anything til I get home mid July. However, I do have plans to host a website or two, maybe even a Bookwyrm, Lemmy or Mastodon instance.
I added a mark as read button to the posts but now patiently waiting till the WebSockets -> REST API transition is complete so it can get merged.
The front end needs a lot of workā¦ Every bit is appreciated and the maintainers are pretty fast at reviewing and providing feedback which is nice to see.
This sounds like a fun project! I recently just ripped out and redid the network segmentation on my 3-node proxmox cluster too.
Originally I had everything in a /16, but that was causing some routing problems because I actually needed to static route a /24 that was in the /16 to a VM for VPN. Anyways, Iām going to try to dig through your post and give some advice. This is all just personal opinion on how Iād set stuff up after over a decade of homelabbing/home infra, so ya know, take and leave what you want.
It sounds like you want to use one of your Proxmox nodes with a VM running OPNSense as your router? Iād highly discourage this. I know you call your setup a lab, but itās running the *arrs and probably a streaming server; and thereās nothing worse than planning a movie night and having your networking be down. Also, itāll make it easier to recover from a power outage or hardware failure, keep your network config much simpler, and provide physical boundaries between machines increasing security.
So, Iād say unless youāre fine with the possibility of extended outages, use dedicated hardware for network. Iām partial to pfsenseās netgate, itās a good price and a lot of bang for your buck, and it comes from an awesome open source project. I use Unifi, though I wouldnāt necessarily recommend it due to some shady stuff the company has done/said over the last few years.
OPNSense looks neat, but the only reason I see to use it over pfsense is the integrated IDS/IPS, which is just a nice gui over suricata and a proofpoints subscription. Personally Iād run suricata in a VM and mirror WAN traffic to it via pfsense. This way a VM isnāt in your critical network path, but IDS is available and easy to manage.
Donāt forget, when you separate stuff into VLANs, it forces the traffic to go up to the router, and back down to the switch. This means any inter-VLAN traffic has a 1gb limit on it. So if you ever upgrade your servers with 10gb nics, if you setup VLANs incorrectly, you wonāt get that performance. Or if you just have a lot of traffic, youāll start getting TCP slow starts and retransmissions and it may play havoc on your network. Thatās why many usually just donāt bother with VLANing, it gives you network isolation, but comes at the cost of increased routing.
As for routing, all VLANs will route between eachother automatically. As obvious as it is, just think of two VLANs as two separate physical switches plugged into the same router. By default those two switches will be able to communicate with each other through the router, but they canāt directly communicate with one an other (which would have higher throughput/bandwidth).
DMZ is interesting, in my mind it basically came from a time when networks had a hard shell and a soft interior (wrt security). I donāt DMZ because I have host level firewalls and network firewalls to do LAN segmentation. But, that isnāt to say itās a bad idea if youāre up for it. Basically a DMZ (demilitarized zone) is a VLAN where youād put stuff like a mail server, DNS servers, and maybe an HTTP server. Stuff youād want to expose to the internet, as well as into your local network. The idea is that if one of those servers were compromised, you wouldnāt want it to have full access to your local network. So instead you split off a DMZ network so if a host in it gets compromised, it wouldnāt provide the attackers a good base to pivot into your local network from. I donāt expose any services to the internet, except a VPN, and that VPN definitionally needs a lot of access to my local network to be useful, so I donāt partition it off into its own DMZ. Iām not a network admin, so thatās just my interpretation of it.
As for structure, this is where I ended up:
10.0.0.0/24 - LAN management - Stuff like Unifi/Pfsense admin panels
10.0.1.0/23 - LAN - Where most of my normal stuff goes, desktops, laptops, phones, etc.
10.99.0.0/24 - OOB Administration - Things like IPMI and BMCs end up here
10.99.1.0/24 - Administration - Things like Proxmox VMs end up here
10.99.2.0/24 - Core network - Things like VPN, DNS, backups, basically important network services.
10.99.3.0/24 - Services - Things like *arr, etc. end up here. I actually run K8s via kubespray in Proxmox, so for me this is actually my metalLB service IP range.
10.99.100.0/24 - VPN IP pool - I static IP my VPN clients instead of masquerade, so they get an IP out of this pool, the VPN instance acts as a router, and I static route from my main router back to the VPN instance.
And I have a separate /24 for my 10gb network for Ceph.
So yeah! I donāt know if that helps at all, feel free to ask questions to clarify. If you still really want to install OPNSense in a VM I can give you some tips on that as well.
Dude this is awesome, thanks for your comment! Itās exactly the type of engagement I was hoping for, someone to help with a few key concepts. Iāll definitely be taking your advice to heart it sounds like youāve been through the ringer with your own setup
I agree with you that running OPNsense in a VM is less than ideal but I figured as it would only affect my lab, I can deal with the occasional outage and this is more so I can learn first before going all in on a solution such as a netgate, I appreciate the suggestion there too. Iām not dead set on OPNsense I just figured a gui would be easier to navigate and it looked nice. I havenāt heard of suricata and mirroring WAN connection to pfsense, that definitely makes sense.
As for the VLANs Iām not dead set on that in particular either, the switch I have supported it and it made the most sense as to how to segment my network in that way. So you are saying you can also segment a LAN connection with just a firewall? It would make sense as you can set rules as to how a device can communicate with other devices on the network, is it rather cumbersome to set it up in that way? And on VLAN performance I would assume that would not be a problem as I am the only user who would be connecting to my media server, unless I had a bunch of services sending and receiving shitloads of data essentially?
I really appreciate all your help! You definitely sound like youāve been doing this for a decade haha, very useful stuff. I might pop back and ask more questions later if you donāt mind
For sure, love talking about my home lab! And itās an easy way to help get a bit of content on Beehaw, albeit a bit scatterbrained haha
First, the OSI model! Yeah, this is a very āfirst you must invent the universeā moment, sorry (not sorry). Basically for you the important bits are layer 2 and layer 3 (though I HIGHLY recommend knowing the OSI model forwards and backwards in its entirety).
Layer 2, the datalink layer, uses MAC addresses and is how to computers on a LAN communicate directly with one an other. Layer 3, the IP layer, uses (you guessed it) IP addresses and is what allows you to communicate between networks (itās where we get the term internet: inter-network). Layer 2 is where switching takes place, layer 3 is where routing takes place.
So, if two computers are in a single LAN (or vLAN), they will communicate through a switch only. If two computers are not in the same LAN/vLAN, they can only have connectivity through a router. Itās really important to understand a switch only sees MAC addresses, so you canāt firewall at the switch level.
If some of this is new to you, I highly recommend reading networking 101 and 201 from this site. It seemed like a good resource.
Okay, with that in mind, next you really need to understand bridges. Any *nix box can be act as a switch and/or router. By default when you install Proxmox it creates a bridge on the machineās default network interface. This bridge interface is effectively a switch. Every VM then āplugsā into this switch, giving it access to your networkās router, and fetches its IP via DHCP. You can use ip link and brctl show to see it. Take a look at the related debian docs too.
Setting up OPN in Proxmox
So with OPNSense, you have a few ways of setting it up, but you always need at least two LANs. One which is shared by OPNSense and your edge router (the ActionTek), and the other is shared by OPNSense and the VMs running in Proxmox. Iād recommend just using two vLANs as the best solution for now. One vLAN will be for your edge router, and the other will be for your OPN router.
First Iād setup your new switch so that every Proxmox host is on a trunk port. This allows tagged traffic from the host, meaning you can specify a vLAN for each VM network interface. Make sure you get this working first, hereās a decent guide, and make sure to read the proxmox network docs too. Now everything should be on the native vLAN (special vlan 1, containing untagged traffic).
Next just add a second interface to your OPNSense with a different vLAN (maybe 100), thisāll be your OPN vLAN. Now you can move VMs to your OPN network by changing their vLAN to the OPN networkās vLAN in Proxmox. Easy peasy!
Finally youāll want to setup proper connectivity between your routers. If you were running a cooler edge router you could setup BGP, but for now static routes will suffice. Setup a static route for your OPN network in your edge router, and point it to your OPN VMās IP address (the IP the edge router assigned it, 192.168.1.X). Finally create the opposing route in your OPN router for the edge router network. This way you donāt need a double NAT.
Okay, really sorry if none of that made sense. I really did try to keep it concise :S
I really appreciate all your input, its really helped me connect a few dots that needed connecting so I really do appreciate it my man. Iām aware of the OSI model and I was hoping my work with my homelab would help me understand better, but Iāve still got a ways to go. I really appreciate your explanations on Layer 2 and Layer 3, I understood the definitions but your explanation helps me connect the idea to real world examples. I also understood bridges somewhat but relating it to a switch makes it simple to understand and configure.
Thanks for the links as well, I will be messing around with this config tonight so I will be using your comments as a resource, so really, thank you! Can I ask for permission to PM you if I have any questions related to homelab/networking? You and I seem very similar in our scatterbrain-ness lol, so I figured it might make things easier if that makes sense. I will definitely be doing more reading and research, the first link you provided seems like an absolute blessing of a resource.
My main wish after P5R soundtrack was releasedā¦ is that I wish there was a definitive playlist of the order once both soundtracks are merged. As of right now, there is a set list for P5 and then a separate one for P5Rā¦
Maybe I need to keep looking, or generate it myself (itās harder than it seems though).
Check out J-music Ensembleās Persona 5 cover album Metagroove! Itās really well made, I couldnāt go back to listening the vanilla soundtrack without thinking of it.
For me one of my favourites are my extractor fan control in my bathrooms. I have a switch on the fan and a humidity sensor close by. Turn on the shower and the extractor will turn on until the humidity is lowered enough, then switch off. In addition I have a static programmable button to press should you need it for other reasons. I just love the world of entities and how they link together to make something functional.
Manly P Hall, The Secret Teachings of all Ages Ted Kaczynski, Industrial Society and its Future
I tend to read 2-3 books in parallel and randomly continue reading based on mood. I love Hallās writing style and the amount of knowledge he put into the work and considering that he wasnāt even 30 when it was first published, itās an incredible read for someone who recently had a spiritual awakening.
Rereading Tedās Manifesto after reading of his death yesterday. Its impressive how he foresaw what technology would do to our society and how disruptive the industrial-capitalist-technocratic trifecta would be for humans.
In my view, having rewatched Voyager again decades after first run, the show not only took successful risks in several episodes like the Demon duology or The Thaw, it has some ābest everā episodes for employing some classic Star Trek tropes.
At the time, I suspect some fans focused on the ānot new ideaā more than ādid it better thanā but at this point itās fairly clear.
For fans who came to Voyager first (including our kids), the original TOS and TNG episodes that Voyager built upon just seem weak by comparison.
More, when SNW does something similar, people are viewing these kind of episodes from the perspective of how well done within a type rather than criticizing them for reworking a trope.
5~ USD a month. Working great for personal use and Iād imagine a handful of users. Hosted in a data center that is very close to me.
Also fwiw: 4 days of lemmy. I am subbed to a bunch of stuff. Iāve only uploaded like three pictures to my instanceā¦ All that space is thumbnails from other instances.
Thereās my current disk usage. Iāve gone wild subscribing to just about every community I come across to see how the storage adds up. Right now Iāve got ~150 communities subbed. Weāll see how it goes and when Iāll need to expand the storage.
Galaxy S10e. I love the small size of it and the fact that it still has a microSD slot and headphone jack. Its starting to show its age when it comes to battery life and the USB C port not working as good.
I like the Samsung android software with the customization that you get with Good Lock so I will probably stick with Samsung flagship for my next phone. The biggest thing I dislike with Samsung is the amount of bloatware they ship, the out of box experience is terrible and I usually have to spend a day researching what apps to disable with adb to get the phone usable. I will probably upgrade to an S23 next year. I usually buy my phones used when the model has been out for about a year. At a year old its still new enough to have plenty of support left, but its no longer the current model so it sells for less. Its a better value IMO then buying a new mid range phone, especially as there isnāt as much advancement year to year in hardware these days. A year old flagship will have better cameras than a new mid range phone typically.
This is what I did as well. Bought an S22 Ultra a few weeks after the S23 line launched. The usual cons I found that I actually didnāt mind. No expandable storage? 256GB is way too much for me; I canāt fill that up with apps and photos. I also use wireless earbuds so Iām not mourning the 3.5 jack. Iāve always been a fan of the Note line so itās perfect for me.
Iām considering an Android smartwatch but Iām not doing research yet. I will also likely buy a previous generation device if ever.
I have never owned an Apple product, not even an iPod.
Thoughts on NixOS? I have heard a lot of positive buzz, but I donāt think I understand it fully. (Iām primarily debian, ubuntu, fedora, arch user, and Iāve adminād a FreeBSD server too).
in my system config (example from Nix manual). It will install lemmy, install caddy, start lemmy backend on port 8536, frontend on 1234, expose it with a caddy reverse proxy to that hostname, and initialize a postgres database. This is also reproducible across systems, so itās pretty much guaranteed to work the same on one PC and on another.
This is very useful, because some programs require some more configuration, and this can remove the need to know where to put their config files, their package names, systemd service names from your head. Itās all in there.
Also, when I fuck something upā¦ when changing the config, it makes a new boot entry with it, so when booting I can just press arrow down when booting to select an older, working config. Magic.
Packages are also nicely separated from each other. I donāt have to install stuff globally, when I need a program one time I can just do nix shell nixpkgs#audacity and have an ephemeral shell with the package installed.
There are (optiona) binary caches, so you practically donāt have to compile anything from source when updating your system.
I have all my configuration on GitHub, like a lot of people, which makes it easy to share information.
A con is that when a program hasnāt been packaged for NixOS (whether itās in nixpkgs or has a flake.nix in the repo), itās not that easy to use it, so learning to write derivations (packages) for NixOS is pretty much a must have.
Also another must have is being in some community that uses NixOS, because it is really hard to learn without someone to help and guide you IMO.
Should work fine. I really recommend installing the nix package manager on your current distro to play with the language and how it works, I did it on arch to get familiar and it has been really helpful.
I really enjoy using NixOS as it is good at what it does, declarative system configuration, but it does have issues that can prevent people from using it. Itās great if you want to put the configuration for all your computers in one git repo but that configuration is in the Nix language so you will eventually need to become familiar with the Nix language. The main issues are that the documentation needs work and understanding the difference between the Nix operating system, the Nix language, and the Nix package collection as the more you use NixOS the more familiar you will need to be with each.
That said, I find it worth learning and recommend some of the following resources for NixOS.
MyNixOS for graphical configuration management. See my configs there.
NixOS Wiki for the best collection of NixOS documentation. Iāve found this collection of peopleās configurations to be very useful for inspiration.
Thank you for your helpful replies! I will put it in my āsomedayā inbox when I feel the need to shave some yaks or when debina/ubuntu/fedora piss me off about something. :)
kbin.life
Newest