I always wonder where and how these crazy sovcits are after all this time. October 2023 is quite a long time ago. Like I wonder if they’re homeless, if they ever got back on their feet after everything they do fails.
Agreed, but again these updates were done by the Crowdstrike software. Nothing to do with Microsoft or Windows.
In this case it was an update to the security component which is specifically designed to protect against exploits on the endpoint. You’d want your security system to be up to date to protect as much as possible against new exploits. So updating this every day is a normal thing. In a corporate environment you do not want you end users to be able to block or postpone security updates.
With Microsoft updates they get rolled out to different so called rings, which get bigger and bigger with each ring. This means every update is already in use by a smaller population, which reduces the chances of an update destroying the world like this greatly.
I absolutely expect vendors to push out new patterns automatically and as fast as possible.
But in this case, a new system driver was rolled out. And when updating system software, I absolutely expect security vendors to use a staged rollout like everyone else.
100% agreed, Crowdstrike fucked up with this one. I’m very interested to hear what went wrong. I assume they test their device drivers before deploying them to millions of customers, so something must have gone wrong between testing and deployment.
Something like this simply cannot happen and this will cost them customers. Your reputation is everything in the security business, you trust you security provider to protect your systems. If the trust is gone, they are gone.
One time years ago, Sophos provided an update the blocked every updater on the machine. Each computer had to be manually updated. They are still in business. My point is that this isnt the first and wont be the last time it happens.
Yeah, I mean Microsoft can release something like Windows 11 and still be in business, so I don’t expect a lot will change. But if you had any stocks in Crowdstrike, RIP.
We’ll probably never know. Given the impact of this fuck up, the most that crowdstrike will probably publish is a lawyer-corpo-talk how they did an oopsie doopsie, how complicated, unforseen, and absolutely unavoidable this issue has been, and how they are absolutely not responsible for it, but because they are such a great company and such good guys, they will implement measures that this absolutely, never ever again will happen.
If they admit any smallest wrongdoing whatsoever they will be piledrived by more lawyers than even they’d be able to handle. That’s a lot of CEO yachts in compensations if they will be held responsible.
I disagree. That Crowdstrike crashes is one thing; the issue here is that Windows suffers such a widespread crash, whether it is because of Crowdstrike or for any reason.
It’s not specific to Microsoft, but the general idea of letting proprietary software install whatever it wants whenever it wants directly into your kernel is a bad idea regardless. If the user had any control over this update process, organizations could do small scale testing themselves before unleashing the update on their entire userbase. If it were open source software, the code would be reviewed by many more eyes and tested independently by many more teams before release. The core issue is centralizing all trust on one organization, especially when that organization is a business and thus profit-driven above all else which could be an incentive to rush updates.
Agreed on both counts. This happened because Microsoft made adoption easy. And this will be fixed within a day. None of the fundamentals have shifted. Even though it’s stupid, this isn’t going to fundamentally shake anything up.
There will be no consequences for those who made this choice because going with the biggest suppliers is never wrong: they in theory have the highest reliability, and even if they don’t, then it’s not just your problem but everyone else’s too, can’t blame those responsible when the outage is akin to an “act of God”
It’s great to have alternatives. If it was all linux, and linux got hit, then it’d be the entire world in danger. Too bad M$ is just not good enough for it’s second most popular position.
Well, we got to see roughly something play out with the xz thing. In which case only redhat were going to be impacted because they were the only ones to patch ssh that way.
Most examples I can think of only end of affecting one slice or another of the Linux ecosystem. So a Linux based heterogenous market would likely be more diverse than this.
Of course, this was a relative nothing burger for companies that used windows but not crowdstrike. Including my own company. Well except a whole lot fewer emails from clients today compared to typical Fridays…
What amazes me is that so many big companies still use windows in critical core infrastructure.
Windows endpoints is one thing, but anyone using windows servers and MSSQL for mission critical application stacks need to be hit with the modernization hammer.
And then on top of that, they do not have a test rollout of any changes in a test environment, before rolling it out in the production stack.
Good luck to all the engineers in the trenches, having to fix the mistakes of their leadership.
I’ve not used crowdstrike, but looks like a part of the pitch is “cloud managed”, which often implies that the vendor takes care of everything, including updates. Particularly since they market it as a security solution, they weld likely emphasize that they can update rapidly enough to keep up with security attacks that move very quickly because they don’t care about “risk”.
lemmy.world
Active