This is, in a lot of ways, impressive. This is CrowdStrike going full “Hold my beer!” about people talking about what bad production deploy fuckups they made.
Applying updates is considered good practice. Auto-applying is the best you can do with the money provided. My critique here is the amount of money provided.
Also, you cannot pull a Boeing and let people die just because you cannot 100% avoid accidents. There are steps in between these two states.
I work at a different company in the same security space as cloudstrike, and we spend a lot of time considering stuff like “if this goes sideways, we need to make sure the hospitals can still get patient information”.
I’m a little more generous giving the downstream entities slack for trusting that their expensive upstream security vendor isn’t shipping them something entirely fucking broken.
Like, I can’t even imagine the procedureal fuck up that results in a bsod getting shipped like that. Even if you have auto updates enabled for our stuff, we’re still slow rolling it and making sure we see things being normal before we make it available to more customers. That’s after our testing and internal deployments.
I can’t put too much blame on our customers for trusting us when we spend a huge amount of energy convincing them we can be trusted to literally protect all their infrastructure and data.
Not who you asked, but did you ever hear of Valiant and their kernel level anti cheat.
This is not a 1:1 comparison but anticheat software running in the kernel has the ability to monitor all other processes due to its permission levels. It can monitor all scheduled tasks and infer from that information.
Drivers need similar access but for different reasons, they need access to os functionality a user would absolutely never be granted. This is because they interface directly with hardware and means when drivers crash, they generally don’t do it gracefully. Hence the BSOD loop and the need for booting windows without drivers (i.e. safe mode) and the deletion of the misconfiguration file.
Whatever CrowdStrike’s “features” are should already be core security features of the kernel itself, or be exposed/extracted into user space.
NT was supposed to be a micro kernel. That this tool injects itself into the kernel immediately compromises the kernel. Edit: I should point out that it seems that CS injects drivers into the Linux kernel too, it might just be that Linux handles a driver crash more elegantly.
No different to the gaming anti-cheat kernel crap.
Having a “security” tool immediately compromise your actual security is absurd.
I’d love to know how you plan to do user mode packet filtering. Keep in mind that on Linux, the designated API is inherently kernel mode. netfilter.org
This isn’t one of the cases where we’re talking about Linux being superior to windows. Any OS will be fucked if you give it a mangled kernel module. In this case, it’s just that only one got one.
Your perception that anything that touches the kernel is an intrinsic security risk is unfounded.
I, too, work in a similar type of company, and can confirm from experience that Linux can get just as absolutely fucked up by a bad kernel module as windows.
And it’s not just changes to the module that can cause things to go wrong.
For example, the kernel released alongside the latest Ubuntu LTS included a change that conflicted with our module behaviour, so machines with that kernel or newer would panic on boot.
It was a super minor change, but when you’re deep in the weeds, it’s really easy for these things to be brittle. But that’s just an inherent consequence of the fact that this sort of stuff is intrinsically low-level interaction with the OS itself.
The kernel is responsible for managing hardware and general low-level system operations. Anything that wants to do those things needs to get itself into kernel mode one way or another.
The typical way you do this is called a “driver” and no one thinks about them as being kernel code. Things like graphics cards and the like.
Things that want to do actions like monitor network traffic or filesystem activity system wide or in a lower level capacity than the normal tools provide also need to be kernel level.
In a security context, that specifically would include things that want to monitor raw packets rather than the parsed content that assumes the packet is well formed in a way that a malicious one might not be.
Cloudstrike does the same thing on Linux, and the typical tools for network management or advanced security are also either compiled in or loadable kernel modules.
It’s easy to forget that ip/ebtables and selinux and friends are kernel level software frequently distributed as kernel modules, in the case of the firewalls, or compiled in with a special framework and not just user mode software.
I can put the blame to your customers. If I make a contract with a bank they are responsible for my money. I don’t care about their choice of infrastructure. They are responsible for this. They have to be sued for this. Same for hospitals. Same for everyone else. Why should they be exempt from punishment for not providing the one service they were trusted to provide? Am I expected to feel for them because they made the “sensible choice” of employing the cheapest tools?
This was a business decision to trust someone external. It should not be tolerated that they point their fingers elsewhere.
Can’t fault you for feeling that way. I definitely don’t think anyone should be exempt from responsibility, I meant blame in the more emotional “ugh, you jerk” sense.
If someone can’t fulfill their responsibilities because someone they depended on failed them, they’re still responsible for that failure to me, but I’m not blaming them if that makes any sense.
Power outage or not, the store owes me an ice cream cake and they need to make things even between us, but I’m not upset with them for the power outage.
You can be reasonable in your choice of words, but there are heads that need to roll. In this case it is not the one pushing the final button, but all those that created this system. Developers, Project Managers, Team Leaders, all the way up to the CEO. If the space to work in is so limited that the possibility of such pushes seems like a tolerable idea, then everything leading to this is broken. And people need to invest to make this right. Therefore there needs to be incentives, good and bad. To steer out of the current course there need to be very unfavorable incentives.
You can mock my argument by giving a ridiculous example. Once people die it will be too late. It’s why there was a time where people thought it to be a good idea to employ giant generators to keep the power in a hospital running even in case of a power outage. Or to have redundant systems in an airplane.
There is a need for adequate standards in the software world. Trusting businesses to create them will evidently kill people. Creating something like certificates for personal skills and products is severely lacking.
I wasn’t mocking your argument, I was agreeing with you and clarifying that my feeling was about who I’m most “irritated” with, not about responsibility or legal culpability.
My example was for simplicity, not mockery.
The power going out is the power companies fault, so I’m most mad at them. The store didn’t have a generator because they trusted the power company, so my cake got ruined. I’m still mad at them but less so because they weren’t the cause of the problem, even though they could have done more to prevent this from impacting me.
Culpability wise, I can only make demands of the store and hope that enough other people do so that they in turn demand answers from the power company.
There are actually a fair number of certifications, including ones from government agencies, relating to software development, deployment, and related practices. That so many organizations didn’t have the ones relating to protection from supply chain issues is distressing, to say nothing of it slipping through quality control in the first place.
Please, if you think we’re in a place in this thread where I’d be mocking you, re-read it with the understanding that I agree with you entirely on legal and structural issues, and at most just have a different opinion about where the balance of "fuck you"s go. I think I put more scorn towards the vendor because doing the thing is worse than failing to prevent the thing. Also, I work at a parallel company and so I’m more familiar with exactly how much you have to be fucking up for this to happen because I spent the last three days dealing with the more minor controls that prevent this from happening. Everyone has outages because you can’t prevent 100% of errors, but it’s on the vendor to build to the spec of their most sensitive customer and ensure that outages don’t keep a doctor from patient records.
I wasn’t mocking your argument, I was agreeing with you and clarifying that my feeling was about who I’m most “irritated” with, not about responsibility or legal culpability.
Okay, sorry for that. It happens to me sometimes to be mocked without me seeing prior cause for this. Thank you for clarifying that.
If a shop can’t sell me cakes, then it’s inconvenient. If a hospital is not able to keep people alive, that’s where things get intolerable. Them not having access to their PCs is a hospital thing. If they cannot use them they should not use them. If it’s a cost saving measure at the cost of people’s lives, then I want heads to roll. Literally, preferably.
For the icecream, yes. If I want icecream and the shop doesn’t have any because of a power grid failure, then I blame the power company more. The generator would be overkill, as it needs constant maintanance and checkups; immense running costs. This would not be justifiable for something like ice cream.
The hospital needs to be way more thorough with their supply chains. This discrepancy of responsibilities towards patients/customers is why I thought I was mocked, sorry again for that.
I called the certification processes “lacking” because they are very often out of date, if at all applied, like you said. The timeframe for product certifications needs to be drastically reduced for software products. I am aware that those checks need time the developers often don’t have, but that doesn’t matter. If that is a crucial issue, then they should stay the fuck away from critical infrastructure.
If you patch a security vulnerability, who’s fault is the vulnerability? If the OS didn’t suck, why does it need a 90 billion dollar operation to unfuck it?
Crowdstrike completely screwed the pooch with this deploy but ideally, Windows wouldn’t get crashed by a bas 3rd party software update. Although, the crashes may be by design in a way. If you don’t want your machine running without the security software running, and if the security software is buggy and won’t start up, maybe the safest thing is to not start up?
CrowdStrike also supports Linux and if they fucked up a Windows patch, they could very well fuck up a linux one too. If they ever pushed a broken update on Linux endpoints, it could very well cause a kernel panic.
Yeah, it’s a crowd strike issue. The software is essentially a kernel module, and a borked kernel module will have a lot of opportunities to ruin stuff, regardless of the OS.
Ideally, you want your failure mode to be configurable, since things like hospitals would often rather a failure with the security system keep the medical record access available. :/. If they’re to the point of touching system files, you’re pretty close to “game over” for most security contexts unfortunately. Some fun things you can do with hardware encryption modules for some cases, but at that point you’re limiting damage more than preventing a breach.
Architecture wise, the windows hybrid kernel model is potentially more stable in the face of the “bad kernel module” sort of thing since a driver or module can fail without taking out the rest of the system. In practice… Not usually since your video card shiting the bed is gonna ruin your day regardless.
Because it isn’t. Their Linux sensor also uses a kernel driver, which means they could have just as easily caused a looping kernel panic on every Linux device it’s installed on.
Security operations being one of the things that is often best done at the kernel level because of the need to monitor network and file operations in a way you can’t in user mode.
Also, it’s less about “their” drivers and more about what a kernel module can do.
Saying “there’s no way to know” doesn’t fit, because we do know that a malformed kernel module can destabilize a linux or mac system.
“Malformed file” isn’t a programming defect or something you can fix by having a better API.
Having the data exposed to userspace via an API would avoid having to have a kernel module at all… Which when malformed wouldn’t compromise the kernel.
I mean, sure. But typically operating systems don’t expose that type of information to user space, instead providing a kernel interface with user mode configuration.
It’s why they use the same basic approach on mac and Linux.
Was a computer repair tech until a few months ago. About 6 months ago this older guy brought in his laptop because he had been hacked and they had changed his password. Was able to change the password to something new using some fancy tools but upon getting in all his files were still missing. Turns out OneDrive was on and ALL of his important files were only on OneDrive and not the computer. Well, Microsoft had changed his password when the hackers changed his computer password so he was locked out and Microsoft didn’t believe he owned the account anymore since he didn’t know the password. After weeks of calls he just gave up trying to get his stuff back.
I get the hate, but what is Microsoft to do in those situations? They have two users claiming to own the account, each with assumably the same level of proof (virtually none) and no backup recovery set. So what, they just believe the first person to call in and say “I was hacked can I have a new password”?
Unless something that links to the owner in a verifiable way exists on the account, which isn’t available to someone logged in (credit card number used for purchase for instance), I don’t really see a way around this.
The same thing happens with game accounts all the time. Two people with the same level of proof claim they own an account? Unfortunately the account gets marked as irreversibly compromised and permanently banned.
I don’t know that I’d consider this their fault. The user handed their info over to someone else. Yeah, it sucks that the end result is losing their files, but you can’t really hold a company responsible for their users doing dumb things.
The root of the problem is that Microsoft deleted his files off of his hard drive, without his understanding/consent. Had they not done that, there would have been no problem.
No? The “root of the problem” is that the cloud service the files were stored in, was deauthed. At that point, I would absolutely expect all files to be deleted.
You can argue that M$ shouldn’t have pushed for that by default, but the problem as described is “user stored their important files in one drive, they gave away their password, password was changed, new password was unknown, one drive removed all local copies of files stored in it, microsoft couldn’t verify who they were when they called.”
Had this been the other way around, where the scammer got file access and the original user reset their password, you’d expect the scammer to have the local copies deleted… would you not?
The issue here is that OneDrive does not make it clear at all that your local files are going away when you enable OneDrive. On Demand is now on by default for everyone. Unless you know this is a thing that happens (or happen to catch weirdness like I did where the Desktop folder seemed to vanish because it was moved) there is no indication this is happening. That’s why this is Microsoft’s fault.
Yeah, that doesn’t really apply to the story I was replying to. The complaint was about Microsoft not believing the user owned the account.
It’s tangentially related to the overall topic, and that could indeed be the root cause, but “they didn’t give him access because he didn’t know the new password” is security 101.
There are almost always ways to verify the correct owner for something like this… None of which it sounds like Microsoft was willing to do, as they only seemed to care about what the current password is.
You are making an assumption that the person can’t provide any way to identify himself as the owner. The story as written states they didn’t care about anything other than the current password.
Almost always != always, and an individual falling for a scam where they hand off their password would typically fall into the category of “unable to prove ownership”.
For your mouse double click issue, I have a g600 and ran into the same thing. It’s due to a teeny tiny copper plate in the switch degrading over time. I’m not confident in my soldering skills to swap out the whole switches, but I was able to buy some new switches for like $5, pop open the little plastic switch box, carefully pull out the little copper plate with tweezers, pop open the switch on my mouse, and carefully replace the little copper plate with the new one. Worked like a charm.
Are you me? I had the same Logitech mouse click issue and fixed it the same way; ordered extra switches online, opened them, and swapped only the copper plate. Mouse click works like a charm, as you said.
I had the same issue a few years ago. After spending forever looking for a solution online, I found a fantastic video that explained the reason for this degradation: www.youtube.com/watch?v=v5BhECVlKJA
TLDW, it has to do with some components (the contact plates) being rated for electronics of the 90s, with higher voltage than today’s devices use. So these components are now subject to below optimal voltages (say, 1.8V or 3.3V), and tiny sparks happen that would not be there at 5V, thus damaging the plate ever so slightly.
Immediately after watching that video, I opened my mouse and scratched the plates with a flat screwdriver. I haven’t had a problem since then (it’s been a couple of years). But if it happens again I know exactly what to do to save my beloved G302.
Just an FYI, Windows likely just moved your files from users[username] to users[username]\OneDrive instead. When OneDrive sets itself up, it basically grabs all of the relevant folders and moves them into a single “OneDrive” folder. Not a huge issue if you’re setting up the PC for the first time. But if you’ve been using the PC for a while, it’ll break everything because now all of your local files have moved and none of your systems are pointing at the right location anymore. For instance, your desktop is likely black because your image file got moved into that OneDrive folder.
There was this mini golden era around 2019 or so where it really seemed like Windows was getting their shit together. I think a lot of places use Macs for development now and Windows was trying to get that market share back. Stuff like the new console and WSL were amazing.
Yeah I swear from when Nadella took over until like 2 years ago, Microsoft really seemed to be on the right route. They were becoming the “good guys” of big tech companies.
WSL, actually being really good stewards of GitHub, Chredge actually (at first) being way better for users than Chrome, the amazing revitalisation of some of their oldest and most loved game franchises like Age of Empires and Flight Simulator.
But then recently we’ve had Microsoft adding shitty AI to everything, from Edge to Windows. We’ve had that AoE revitalisation tarnshined by showing off a really shitty official mobile game with all the makings of a typical pay 2 win time sink. The Age of Mythology remake has obvious AI art featured in it despite them insisting no AI was used (though thankfully the actual gameplay is as good as hoped for, at least). We’ve got large layoffs and other shitty corporate bullshit towards workers.
It isn’t bash, it’s Linux that’s well-embedded with the rest of Windows. You can get most Linux stuff working reasonably well, and you can even get a working GUI of some distros.
It works well - for a Windows subsystem. It is well-integrated but also separate which can be annoying sometimes.
For example, you might code in Python in VSC against a WSL folder but make a script to eventually run in Windows. You need to install and update Python twice then - a Linux and a Windows version (obvious, but can be annoying).
WSL is also really slow, especially for filesystem heavy stuff. You know how on Linux programs sometimes run faster via Wine/Proton than on Windows itself? Yeah, this is the other way around.
Honestly the type of stuff I do works good enough with MSYS through Git for Windows (which is a basic bash environment). There are three ways to get bash on windows,
MSYS/Git for Windows: Lightest choice. Least capable. Very easy to set up.
Cygwin: Only works with Linux stuff made for Cygwin. Pretty useful all in all but really weird to set up. Babun was my favorite way to use it.
WSL: The most Linux like but at the steep cost of being very disconnected from the Windows side. It feels more like a VM than a shell sometimes.
I preferred the simplicity of Git for Windows and Cygwin. Now, if I still had Windows on a work computer I probably would’ve deep dove into WSL and figured it out more.
Most useful things i found in wsl that made it not feel like a vm is knowing the wslpath command, and the fact that it can execute any exe such as explorer.exe (which works for even wsl directories). those two things let you use sed/grep/awk on files in windows and execute any exe on stuff in linux.
We tried to onboard two devs into our project earlier this year with it and it was not good.
We spent 4 days trying to get it to work, and had all kinds of problems from VPN not working, DNS not working and compile times being 20x slower (as I later learned, you’re not supposed to use your Windows NTFS partition inside of it). Partially, this has to do with our corporate environment being annoying, but it simply being different from a normal Linux in this regard is still annoying.
On the fifth day, we set up a Linux VM with them and they were ready to work in an hour.
It was earlier, when they released Windows 7 and it was the first (and only) release, management gave development a largely free hand and they could bring down some technical debt.
But apparently that didn’t work out for Microsoft and now we get one dystopian news after another.
My mouse logitech mouse is suddenly chattering really bad and double clicking everything
Is is a G903 you using? It’s a issue with cheap ass switches if that’s the case. I RMAed one and the replacement did it even faster than the first. Gave up on that one.
I wanna say it’s a G506 or something? It’s that one that like everyone has because it was ~$80 and there was a deal years ago at Best Buy that included a $50 Steam gift card. I don’t remember how long I’ve had it but it’s certainly out of warranty.
It’s sort of always had this problem but suddenly it got A LOT worse. It’s around the same time as a Windows update. Makes me wonder if Windows was filtering out some of the clicks that were insanely close together before.
Every wireless mouse I’ve ever owned starts double clicking after like a year and a half or two years. The only exception is the Razer Basilisk I bought about 3 years ago, that ones still ok so far
programmer_humor
Hot
This magazine is from a federated server and may be incomplete. Browse more on the original instance.