There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

JovialSodium ,

This doesn’t fit the question exactly but I feel it’s in the same spirit, and a kind of interesting solution, I think.

Back in the early days of scryptcoin mining, I had a few gpu mining rigs running Linux. Occasionally they would hard lock and I’d have to power cycle them.

What I ended up doing is getting some usb to serial adapters, wrote a python script that ran on startup and would send a character over serial at a set interval in a loop. That was hooked up, if I recall correctly, to an attiny85 using softwareserial and some ttl to rs232 conversion. It would listen over serial and if it didn’t receive anything with a reasonable time frame it’d flip a relay that cut mains power to the pc, then flipped it back. A deadman’s switch, of a sort. It worked great!

palordrolap ,

I remember a story about someone who did something similar with a server that kept hanging. They rigged up a second computer to ping it over the local network and if there was no response for a certain amount of time, the computer would eject its CD-ROM tray which had been lined up neatly with the reset button on the server.

Since it couldn't eject fully, it then retracted, having rebooted the server.

I assume that was a temporary fix... and it was probably a Windows server tbh.

The closest I've done is having a job run every 12 hours checking if a process was over a certain memory usage (memory leak) and restarting it if it was. That was also Windows, but the same thing on Linux wouldn't have been difficult... not that the Linux servers ever had that problem.

4am ,

if ping() <> 0 then drinktray.exe

Holy jank, Batman!

But hey; if it’s stupid and it works, it’s not stupid.

kugel7c ,

So I mostly fried the SSD by using it to write and rewrite ML checkpoints and logs, this in turn made the device read only and I somehow managed to migrate to a different SSD probably using clonezilla or something, but it messed up the bootloader so I installed refind in a new partition, configured it and voila it works. It’s scary because you need to do everything without seeing your system even half alive anywhere along the process, but it’s not actually hard, just copying data and installing/configuring a bootloader. But for a then 20year old at his more or less first job my head was on fire for the 1.5 days this took.

By far the most difficult single thing that I’ve ever had to fix that actually had to do with the system.

I now don’t flood my SSDs with data that is constantly rewritten.

federatingIsTooHard ,
@federatingIsTooHard@lemmy.world avatar

is debootstrap considered reinstalling? because i’vedebootstrapped at least 2 systems to fix botched upgrades.

4am ,

I screwed up permissions on an LXC container in Proxmox by converting it from unprivileged to privileged (against recommendations) and had to mount it offline and write a script with find into chown via the execute flag to change all the UIDs and GIDs from the shifted unprivileged ones to the standard host-level ones.

Luckily this was in my own lab so it was a (mostly) harmless learning experience.

0x30507DE ,
@0x30507DE@lemmy.today avatar

Accidentally put grub on the wrong partition on the device, which it was not happy with. Was able to copy some files over, manually boot the OS, and reconfigure grub to be in the right partition, took me about 2 hours? Then I did it again on a different machine, and speedran it lol

bruhbeans ,

I recently managed to recover from a corrupted libstdc .so. Turns out I shouldn’t have bothered because the it was a Pi and, of course, the SD card had shit the bed, but I was pretty happy with myself for like 30 minutes.

shadowintheday2 ,

Used to be messing with kernel arguments and installing/tweaking boot parameters. That was until Grub broke, I learned systemd-boot and chrooting into the system via live USB

Now if I break anything it’s just a matter of “sigh, let me get the USB and type a few commands”

sirico ,
@sirico@feddit.uk avatar

Biggest thing I noticed after switching is forum posts. In Linux ones you usually get a fix where the Windows ones 9/10 they just advise you reformat

Limonene ,

A couple months ago, I made a Palworld server box out of a spare motherboard assembly (mobo, processor, ram) from a computer I had recently upgraded.

I didn’t have any spare drives lying around, so I plugged in 7 USB flash drives and made them into a RAID array. Not a true RAID array, but a BTRFS filesystem with volumes spread onto each flash drive, with the data redundancy set to raid1, and the metadata redundancy set to raid1c3.

It worked… in the sense that I never lost any data. It certainly didn’t work in the sense of having good uptime.

The first problem was getting it to boot right. The boot line in GRUB had “root=UUID=…” instead of a specific drive named. That is normal. However, in BTRFS multi-volume filesystems, all the volumes have the same UUID. So the initrd was only waiting for a single drive matching that UUID, then trying to mount it as the root filesystem. This failed, because the kernel had not yet set up the other 6 USB drives, and this BTRFS filesystem needs all 7 volumes present. Maybe 6, if you used the “degraded” mount option.

The workaround was to wait for this boot process to fail, at which point you get dropped into an initrd shell. Then, you look at all the drives and make sure they’re all there. And then… I don’t exactly remember what happened next. I think it was some black magic that erases your mind in the process. I somehow got it booted from the initrd shell.

Installing Steam and the Palworld server worked ok, and it even ran for a few hours before crashing overnight.

The next morning, I tried rebooting it. Unfortunately, the USB drives weren’t all appearing. Turns out the motherboard had some bad USB ports, some sometimes-bad USB ports, and a maybe-bad PCIe bus, because the PCIe USB expansion card I plugged in had weird problem that it had never had before.

I found the most reliable ports and plugged the drives in there. But you can’t just replug them in the initrd. It doesn’t have USB hotplug support. So each time it tried to boot with not all the drives there, I restarted it again until one time I finally had all the drives.

I changed the GRUB boot line to “root=/dev/sdg1” . This made it wait for all the drives to load, in any order, and whichever one was last would be mounted as the root filesystem (but the kernel would automatically include all the others too, since they were successfully initialized).

The bad USB ports kept bringing down the server every day or two. I bought a cheap NVMe drive and added it to the BTRFS filesystem, and then removed all the USB drives except the largest. That fixed the reliability. It’s been like that since.

Now, to boot the server, all I have to do is change the GRUB boot line to “root=/dev/sdb1” . Since the NVMe drive is much faster than the USB drive, it always initializes first. If the initrd waits for sdb2, then it will always have both drives initialized when it tries to mount the root filesystem.

I could add that to the grub.cfg, or come up with some other more permanent solution, but I’m not planning on rebooting this server ever again. My friends fell off Palworld, and I gave a shutdown date that’s about a week away. And the electricity is pretty reliable here.

Nilz ,

For me it was migrating my Arch install from EXT4 to ZFS. GRUB had to be configured in particular ways to get it to work with ZFS and I didn’t do it properly so it wouldn’t/couldn’t boot.

Then I updated ZFS to a version that wasn’t supported by GRUB yet so I chrooted into my installation to switch to Systemd-boot with Unified Kernel Images. Now I still can’t figure out how to add a boot entry for Windows. I followed the proper steps I think but selecting the Windows entry just reloads Systemd-boot.

folkrav ,

Some of the crap I had to do back in the late 00s to get wifi, sleep and power management even barely working on some machines felt like the hardest thing at the time. I wonder how I’d fare with those issues today, 17 years later, knowing quite a bit more about the underlying OS and working with the OS daily… I don’t know that I’d qualify that as difficult more than it was extremely tedious and a bunch of trial and error of configuration options I didn’t know anything about.

If we’re talking about modern day… not so much honestly. btrfs snapshots saved my ass a couple of times, the rare issue I encounter I just rollback and wait for an upstream fix, and the rest I typically ignore or use something else. Everything tends to run quite smooth for me as a general rule, though.

Samsy ,

My first home server would get lost on the network every week, at different times and without any apparent reason. I performed hard resets by unplugging and plugging it back in.

After several months, I decided to connect a screen to it, and I initially thought it had hung up, but it hadn’t. After some investigation, I discovered that every time my router obtained a new dynamic IP address, the server lost its network connection, requiring a reset. I wrote a script to check the network connection every minute, and if it’s lost again, it will be reset.

bruhduh ,
@bruhduh@lemmy.world avatar

Installed fedora on btrfs and upgraded from 38 to 39 week after installation, everything broke so bad, even ssd which was used for it locked, not just filesystem, ssd was new btw

mariusafa ,

Upgrading the system I removed glibc from the system (Debian). apt wasn’t working, etc. Had to manually fix dependencies and everything. Currently my working OS so all fixed.

pixelscience ,

I’ve generally had good luck with hardware and things just worked under linux. But one day I upgraded a few machines on my network to 2.5G ethernet. Several already had the ports, but my little NUC NAS box didn’t, so I installed a 2.5G usb ethernet dongle. No matter what I did, I couldn’t get it to work. It would show up and NM would act like it was up and there were no errors or anything, but it just wouldn’t actually function.

Eventually, I found out that it has a built in USB data partition that contains the drivers for windows. The card was coming up as a usb disk first when the hardware was assigned and not a network card which it should have been.

I had to write a blacklist the usb modules first, which I had done before, but I had to also write a udev rule to automatically add the network card and driver on boot. It wasn’t that difficult to actually do, but I had just never had to do anything with udev rules before. Took me a good three days of troubleshooting to finally get everything to work correctly on boot.

ACTION==“add”, ATTRS{idVendor}==“20f4”, ATTRS{idProduct}==“e02c”, RUN+=“/sbin/modprobe r8152” RUN+=“/bin/sh -c ‘echo 20f4 e02c > /sys/bus/usb/drivers/r8152/new_id’”

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • random
  • lifeLocal
  • goranko
  • All magazines
    •