There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

This happens after 3-4 days of running the server, then I have to restart it manually.

I bought an Optiplex 5040, with an i5-6500TE, and 8 GB DDR3L RAM.

When I bought it, I installed Fedora Server on it. It got stuck every few days but I could never see the error. The services just stopped working, I couldn’t ssh into it, and connecting it to a monitor showed a black screen.

So, I thought let’s install Ubuntu Server, maybe Fedora isn’t compatible with all of its hardware. The same thing is happening, now, but I can see this error. Even when there’s nothing installed on it, no containers, nothing other than base packages, this happens.

I have updated the bios. I have tried setting nouveau.modeset=0 in the grub config file. I have tried disabling and enabling c-states. No luck till now.

Would really appreciate if anyone helps me with this.

UPDATE:

  • I cleaned everything and reapplied the thermal paste. I did not see any change in the thermals. It never goes over 55°C even under full load.
  • I reset the motherboard by removing that jumper thing.
  • I ran memtest86, which took over 2½ hours. It did not show any errors.
  • I ran a CPU stress test for over 15 hours, and nothing crashed.
  • I also ran the Dell’s diagnostic tool, available in the boot menu of the motherboard. The whole test took over 2 hours but did not show any errors. It tested the memory, CPU, fans, storage drives, etc.
just_another_person , (edited )

Seen this before, and almost always has to do with hardware failure or bad hardware config.

Reset the BIOS/CMOS jumper on the board, go back into BIOS setup and set the proper time. Do not touch the CPU or Memory timings. Boot with the defaults and see if it still happens. Check and update the BIOS if there is a newer version as well.

Next longer steps: test memory, then stress test the CPU. I’d be shocked if it was a storage issue as I haven’t seen that be the culprit, but might was well run the long SMART tests.

seaQueue ,
@seaQueue@lemmy.world avatar

Make sure the microcode package is up to date as well

4am ,

Yeah, always check all of this stuff. Server hardware gets a lot more updates than like gamer board BIOS, companies invest high millions, even low billions in this stuff and they expect problems to be address promptly for that kind of cash.

Check for any peripherals or cards, too. RAID, backplanes, networking cards; drivers, firmware, anything.

possiblylinux127 ,

Poorly supported hardware will also do this

seaQueue ,
@seaQueue@lemmy.world avatar

An Optiplex 5040 should be well and thoroughly supported for 6+y now

nutbutter OP ,

I did reset it. It did not help. I ran memtest86 for over 2 hours and did a CPU stress test for over 15 hours. Nothing crashed during the testing.

Shimitar ,

Had the same issues, it was heat.

Cool down your server, add a fan or a cooler…

I added a usb-powered fan sucking cooler air from outside the server area directly blowing it on the chassis.

That fixed for me.

nutbutter OP ,

I cleaned everything and reapplied the thermal paste. That did not solve the problem. Also, the CPU is only of 35 watts and never goes over 55°C.

nialv7 ,

Time to add a cron job to auto reboot it once a day

tenchiken ,

That server sounds a bit older in the teeth… Has new thermal paste been applied to the cpu? Even if the reported temps are under 90c, you might be getting hot spots causing glitches inside the package.

Worth trying a couple of different generations of kernel as well, both newer and older. You might be hitting a regression somewhere.

nutbutter OP ,

I cleaned everything and reapplied the thermal paste. That did not solve the problem. Also, the CPU is only of 35 watts and never goes over 55°C.

solrize ,

Try running memtest86 for a few days to test memory. That is fairly easy to do though it involves booting from a flash drive. Web search should find info.

nutbutter OP ,

I just ran it. It took over 2 hours to finish. Showed no errors. Is there a benefit of running it for a few days?

solrize ,

If the problem is intermittent then longer run has better chance of catching it, but 2 hours with no errors is a good sign, with regard to the memory.

ikidd ,
@ikidd@lemmy.world avatar

Can you sudo dmesg | grep microcode and see if you have any errors?

If so, I’d be inclined to sudo dnf reinstall microcode_ctl then do a sudo dracut -f to regenerate the initramfs, and reboot. Be sure to have a working fallback kernel like LTS installed so you can recover if need be.

Edit: I just read you changed to Ubuntu. I can’t be arsed to figure out how Ubuntu does this stuff so that’s on you to figure out. Alternatively, install Arch and use intel-ucode package.

lolonaut ,

I had problems with soft locks because somehow the PSU was in corrupt state, maybe through a black out or something. The problem persisted through reboots and power offs, only cutting power helped.

Decronym Bot , (edited )

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

Fewer Letters More Letters
LTS Long Term Support software version
PSU Power Supply Unit
RAID Redundant Array of Independent Disks for mass storage
SATA Serial AT Attachment interface for mass storage

4 acronyms in this thread; the most compressed thread commented on today has 8 acronyms.

[Thread for this sub, first seen 6th Aug 2024, 10:15] [FAQ] [Full list] [Contact] [Source code]

rookbrood ,

It’s a long shot, but I had something similar on one of mine servers once. It was fixed by installing irqbalance and starting that daemon at startup.

bulwark ,

I’ve never seen this particular error, but CPU stall warnings seem like a fairly common thing. I wouldn’t jump straight to hardware fault, but it’s a possibility.

docs.kernel.org/RCU/stallwarn.html

catloaf ,

I’d lean toward bad hardware.

Try stress testing the CPU and RAM. See if you can get it to happen more frequently. Also see if you can disable that CPU core, either in the BIOS or in the OS, to see if the problem goes away.

loganb ,

I’m with catloaf. Consistent CPU soft locks point to a possible bad memory module or CPU.

Clear CMOS.

Try removing one memory module at a time.

See if there is an option to disable hyperthreading in bios.

Another thing to try is to remove the CPU, careful not to damage the LGA pins on the motherboard, and clean the CPU contacts with alcohol. Take care to ground yourself out and the case before handling the CPU out of socket.

possiblylinux127 ,

Don’t try to clean CPU pins. That is a very bad idea

loganb ,

I mean speaking from experience, its resurrected a couple problematic CPUs for me. CPU pins no, pads on an LGA style CPU, sure.

possiblylinux127 ,

It is much harder to mess up pads

admin ,

The CPU in this has no pins, is just contacts on the chip. The pins are in the motherboard, like the new 7000 series Ryzen.

possiblylinux127 ,

Clean pads not pins

possiblylinux127 ,

Does it have a Nvidia GPU? If it doesn’t then the nouveau modest does nothing.

My guess it that you are using a badly support realtek device. However I would need to see the full dmesg

TheBigBrother , (edited )

Set watchdog to reboot every day and juice it until the last drop before it definetly crashes.

Edit: that’s just a workaround if you really want try to fix it my answer would be restoring BIOS defaults, then clean install of the OS and then check it everything works fine, if the error persist try installing another OS if it still fail then go to the first step.

thesporkeffect ,

I’d start here: askubuntu.com/…/watchdog-bug-soft-lockup-cpu6-stu…

slacktoid ,
@slacktoid@lemmy.ml avatar

What kind of drives do you have in your RAID? Is it SMR?

nutbutter OP ,

The boot drive is an SSD, which is not in any RAID. I have another HDD connected via SATA. Another HDD connected via USB.

slacktoid ,
@slacktoid@lemmy.ml avatar

So 2 HDDs one SATA and one via USB in RAID? Can you remove the RAID drives and test it out?

Also what size are the drives and what’s their capacity?

nutbutter OP ,

No, they are not in RAID either.

slacktoid ,
@slacktoid@lemmy.ml avatar

My bad. What are you using the drives for?

nutbutter OP ,

Films and shows, via Jellyfin.

slacktoid ,
@slacktoid@lemmy.ml avatar

Is it just you, any transcoding? Have you tried it without the USB drive?

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • random
  • lifeLocal
  • goranko
  • All magazines
    •