Searching the whole Fediverse, literally all of it, 100%, is technically impossible or at least very hard to implement, and if implemented, it'd eat up lots of CPU power and network bandwidth.
It's simply next to impossible for any instance of any Fediverse project, also for any centralised or decentralised dedicated search engine, to know all instances and all content on it without all instances actively pushing their existence, their status and all their content to the search engine in real-time.
A search engine that literally covers all of the Fediverse with no exception has to even know about brand-new instances that have just been started a split-second ago. An instance that's so new doesn't even have any connections into the Fediverse yet, probably no content and only one account, the admin account. (Replace "account" with "channel" on Hubzilla and (streams).)
So if someone spins up a new instance of whatever project, that search feature has to know about that instance immediately before the instance even connects with anything. That is, I'm not sure when that search feature is expected to know about a new Hubzilla hub since ActivityPub is optional per hub and per channel and AFAIK off by default for both: Shall the search feature already know when ActivityPub is still off, and nothing in the Fediverse that isn't Hubzilla or (streams) can connect to it anyway, or shall it only learn about the instance the second that the hub admin turns ActivityPub on?
And when the admin of a new instance puts out a test post to see if it runs as desired, and the instance still isn't connected to any other instance, the search feature would immediately know that test post so you can find it if it's that what you're looking for.
Mind you, Google doesn't know everything on the Internet either.
Sure this is technically true, but it doesn’t really fix the human need to find things. It would be better if some grouping of Fediverse instances came together under a common banner and agreed to certain protocols that helped make things like mass-indexing easier. This would enable a better frontend experience for people trying to find good content. In fact I think building more protocols on top of the existing one would be exactly inline with the philosophical underpinnings of the Fediverse
A search engine that literally covers all of the Fediverse with no exception has to even know about brand-new instances that have just been started a split-second ago. An instance that’s so new doesn’t even have any connections into the Fediverse yet, probably no content and only one account, the admin account. (Replace “account” with “channel” on Hubzilla and (streams).)
So if someone spins up a new instance of whatever project, that search feature has to know about that instance immediately before the instance even connects with anything.
Yes, but who would want a search engine to specifically cover emtpy servers with half a nanosecond lifetime? For all practical intents and purposes, people search for content, which already excludes these theoretical edge cases. More realistically, people will search for quality content, which implies some engagement happened and some upvotes accumulated. There is no value in discovering servers before users discovered them, on the contrary.
If you really care about new and empty servers, you’re rather looking for a fediverse monitoring tool than a search engine. And even for those, it’s questionable what the value of those entries would be. I would prefer if they are filtered out to not bloat the numbers.
I started with the smallest offer available and later upgraded to the second smallest, which now has 4GB RAM. I also have rented additional diskspace, so that I have 30GB now. RAM and CPU are now certainly fine, but I don’t know yet about disk space. I read that Lemmy/Mastodon can eat up space quickly and I have currently used up about half of my disk space.
I really wish the Linux community would do a better job of separating the software updates from the core operating system and user space apps. I feel like most distros do the ‘move fast and break things’ approach, even if that isn’t what they intended to do. I forget which distro it was, but they tried replacing X11 with Wayland way before the other distros, and IIRC, they had to revert everyone back to X11. This type of thing cannot be managed by regular users.
Imagine if you had to understand how 90% of every car part worked in order to drive a car, and if you don’t understand something you ask for help and everyone ridicules you because they are mechanics.
I really wish the Linux community would do a better job of separating the software updates from the core operating system and user space apps.
You can accomplish this with something like Debian stable and Flatpaks. OK, but now you have to explain these concepts to people, too 😆. It works great but it’s not quite user friendly. Ubuntu gets dunked on a lot for Snaps but I think they are actually the one mainstream distro that is trying to make Snaps as transparent for users as possible, thereby achieving the goal of separating the core operating system from user applications. Though I still prefer Flatpaks.
Both are similar, and the very short version is they are sandboxed applications that bundle their own dependencies and can update out of band with your distro’s software repository. With Flatpaks they can share a common runtime environment, but I think with Snaps they bundle everything into the snap (I might be wrong about this).
One key difference is that Snap is basically only on Ubuntu, and Snaps can also bundle CLI applications or server software. Flatpaks are currently really meant for desktop applications.
In both cases you can modify the permissions of the the programs they bundle sort of like how you might expect on iOS or Android. That is to say you can restrict their access to the file system, the network, or other things. So, as an example, I can run a proprietary program as a Flatpak but ensure it cannot access my Bluetooth if for some reason I feel that need.
Problem: Every major distro has its own unique package manager; dpkg/APT, rpm, yum, pacman etc. It’s a nightmare to package apps for Linux, so let’s make one universal standard package management system.
Three or four independent projects: Okay, here you go!
Problem: Every major distro has its own unique package manager, and there’s three different incompatible universal ones and because one of them is made in-house at Canonical none of the three are supported out of the box on every distro.
Flatpak allows you to package your app once and make it available on at least 36 different distros [0] (if not hundreds more if you count their spinoffs). See the list of available packages at flathub [1]. Read more about Flatpak in general here [2].
It is amazing how 3 steps can be challenging for some even though these are explained in flathub (for all major distros) 1- install flatpak which should install a pluging for gui Package manager automatically. 2- add flathub repo. 3- Configure your gui package manager to default to flathub 4- enjoy installing rhe latest software from flathub without even needing root password (except for Opensuse TW)
You can save yourself from reinstalling over and over by using an immutable distribution so at any point you will know what changed in your system and if it breaks you can just roll back to the previous working point and either fix your mistake or wait for a fix from upstream when an issue happens there (this year there were a few kinda major hiccups on Fedora for example).
I suggest you try one of the Fedora immutable spins (Silverblue, Kinoite, Sericea) or Vanilla OS, though I would hold off from it until Orchid comes out.
If you want to go all in you can use NixOS, but it takes a lot of reading
I’ll have to try one out just so I understand how they work, but I don’t personally need something like this. I’ve used Gentoo, Fedora, Slackware, Ubuntu/Debian, Arch, and more for years.
Yeah, actually I don’t know how I ended up responding to you, I have since deleted that comment, I meant it for the OP.
Aside from that, when you’re as experienced as you, you generally don’t end up breaking your system anyway, if one really wanted I think the real good thing to do regardless of distro would be using one of the few packaging solutions that are siloed from the rest system
During a flight to Canada for a two week holiday (plus 3 day work conference for my wife), our son who was four at the time develops a big old chickenpox blister on his back.
Yes, you can. You need a hypervisor that is capable of IOMMU. I know for a fact that you can do it with libvirtd and KVM/qemu. I think you can do it with Proxmox. That much said, I’ve no experience doing this myself.
When we went to Wales when I was 8, the pet hamster we’d inexplicably brought with us escaped and then in an unrelated incident my mam set fire to the holiday cottage. What is it with Wales?!
I believe Jellyfin was a fork of Emby a number of versions back originally. I’ve used Emby for some while and had no issues other than the occasional case where transcribing some ridiculous bitrate file eats up resources. Streaming to Emby (as in the cast icon) via Roku has an issue with series that have more than 100 episodes on them due to an index size issue, that might have carried over to to Jellyfin?
Personally I trust Bitwarden more than myself to keep all my passwords secure AND available. They’ve got a good track record as far as I’m aware.
For general security hardening though…
I use Shodan to help me identify if anything is misconfigured and what is visible from the web. You can pick up an account for usually $1 for life when they run a deal, then you can just monitor your DDNS, domain, and IP address and have it email you when any new services are detected.
Cloudflare Tunnels, to remove the need for a nginx reverse proxy (with the added benefit of easy failover as well as simplifying your stack). Then I’m utilizing Cloudflare’s WAF to handle filtering out known malicious, foreign IP addresses, and other malicious traffic.
Another route you can go is a Nginx/haproxy reverse proxy behind something like Suricata. Then you can utilize something like fail2ban or crowdsec.
Authentik. Get everything behind a SSO experience and don’t expose your backend services to unauthenticated local traffic (utilize http basic auth with header passthrough in authentik). So many people setup auth wrong and then have something like auth.domain.com going through auth but then mistakenly have their external IP address setup to allow traffic in authenticated.
Considering someone arrogant is an opinion. And nuance is hard to convey over text. Maybe you should reflect on why you find a particular person arrogant and what it is about their arrogance that bothers you. Take it as a chance for introspection instead.
Up until this moment, I was CERTAIN these were nsfw communities.
But yeah, fuck car centric infrastructures. People owning cars and driving cars… You can’t blame or hate the individuals who are just trying to survive. But you can absolutely hate how shitty the availability and maintenance of public transport often is (I hear it’s especially bad in the US?) and the car lobbying, which I hear is especially bad here in Germany. Profits are always prioritised, and the car industry is considered more profitable than providing good, affordable public transport.
Yeah, this is perfectly doable. I ran a very similar setup for a while. I’d recommend passing one of the NICs directly through to the VM and using one for the host to keep it simple, but you can also virtualize the networking if you need something more complex. If you do pass through a single NIC, you’ll need a switch capable of handling VLANs and a bit of knowledge on how to set up what’s called a “router on a stick” with everything trunked over one connection and only separated by VLANs.
Keep in mind, while this is a great way to save resources, it also means these systems are sharing resources. If you need to reboot, you’re taking everything down. If you have other users, that might be annoying for everyone involved.
I have a managed switch. I’m a little confused how everything would be hooked up if I’m using a vm for pfsense and another vm for some Linux distro. I want the router and that distro to be isolated from my other vlans. Could I use the onboard nic hooked up to the switch to put the distro on its own vlan?
You can absolutely attach each VM and even the host to separate NICs which each connect back to the switch and has its own VLAN. You can also attach everything to one NIC and just use a virtual bridge(s) on the host to connect everything. Or any combination therein. You have complete freedom on how you want to do it to suit your needs. How this is done depends on what you’re using on the host for a hypervisor though, so I can’t give you exact directions.
One thing I should have thought of before; if two NICs are on one single PCI card, you probably can’t pass them through to the VM independent of one another. So that would limit you to doing virtual networking if you want to split them.
Having tried both, I found it far easier and less troublesome to just add a PCI passthrough than it is to worry about managing the network both on the host and in the VM. As long as FreeBSD supports the driver, I strongly recommend passthrough vs virtualized NICs.
I would strongly disagree. In terms of setting up OPNSense (I use pfSense, but same concept), it’s easier to just do a PCI passthrough. The alternative is to create a virtual network adapter on your hypervisor, bridge it to a physical NIC, and bind the virtual adapter to the VM. The only advantage to be gained from that is being able to switch between physical NICs without reconfiguring the OPNSense installation. For someone with a homelab, when would you ever need to do that?
My Proxmox server uses a 10Gb PCIe adapter for its primary network interface. The onboard NICs are all passed through to pfSense; I’ve never had any need to change that, and it’s been that way for years.
I don’t mean this to sound overly critical, and I’m happy to be proven wrong. I just don’t see a “plethora of reasons” why doing PCI passthrough on a NIC is a bad idea.
I’m happy to discuss it, as I’ve written articles about it.
I live high level routing and firewalling in VMs (60 Gbps+), and there are a couple of realities you need to accept, especially when you involved a *BSD in the mix.
*BSD’s networking drivers and, to a lesser degree, the whole stack SUUUCK. This becomes extra poignant when you involve *pf, which is incredible for hand editing, but also horrible for performance because it’s a straight top-to-bottom list.
We could argue about the whole networking stack sucking all day, but in reality, it’s the driver situtation that really brings it down. That’s why “You must buy Intel” is such a mantra on *BSD. Because they are about the only drivers which don’t make for a completely horrible experience. You can meme about how terrible Realtek is, but really it’s only terrible on *BSD. It’s a first-class linux citizen, and often supports better hardware features than the ancient X520, pre-Connect-4, etc people circle-jerk about. And if you often losing out on cool new features/offloads/abilities.
The virtio drivers are usually more efficient and performant than most physical hardware drivers (on *BSD)
You asked “why would anyone ever need to do that?”. It’s simple. High availability. You can run two router/firewall VMs on two different hosts and have zero downtime. Or, if you only want one, you can migrate the VM either manually or automagically, and only suffer the downtime for a reboot as the VM moves to a different host. You can share the same physical NIC between multiple VMs with SR-IOV for maximum low-latency networking, aka storage. It’s a waste throwing 10Gb at just pfSense when it’ll be idle most of the time, and with older hardware pfSense isn’t going to even be able to hit half of that.
Your VM just works if you ever have to move it to another host. Your main routing and firewall VM is now tied to a single specific host. In a disaster recovery situation, this is going to make you hate yourself as you basically end up needing to either physically pull a card and re-setup passthrough, or setup passthrough on a new card, make sure the VM is bound to those MACs. When it’s fully virtualized, it’s hardware agnostic. Your VM may think it’s 10Gb on a single link, but underneath the links are high availability (aka vSphere vDS), on different VLANs, etc. My example here is a few years ago where I swapped in a Z8350 WYSE 3040 when my main router died with 40Gb uplinks. Sure, I was limping for a few days, but as far as my router is concerned, there is no difference.
NUMA becomes an issue. Even single processors have NUMA nodes now, and it wouldn’t be difficult for someone not knowing was a NUMA node is to create a NUMA issue, where you incur huge penalties going from CPU/Chipset to RAM to NIC and back again, depending on where the items are physically arranged in the system. This is doubly poignant in the *BSD world.
If a 1Gb interface is your bottleneck, your network design is broken. There is no reason for most people in a homelab to try and route >1Gbps on your edge. Don’t packet inspect it, and internally you are up to 10Gbps and beyond. Sure, a >1Gbps link might be a reason in 2023, but what’s your 95th percentile, like 25Mbps if you are lucky. It’s only “hawt” for your speedtest numbers, and an occasional download. And you can do 10Gbps pretty easily with virtio on basically any semi-modern system especially with the large files that most people would want 10Gb for, and not dedicate a PCIe slot to it and make it portable.
I mean, you do you. But I’d much rather to just be able to change the uplink on a vSwitch or bridge to get my router going again instead of having to reboot, passthrough, insert grub cli options, swap cards, etc.
kbin.life
Newest