CrowdStrike blames testing bugs for security update that took down 8.5M Windows PCs

Company says it's improving testing processes to avoid a repeat.

See full article...

 

[steveftoth]

Some analysts online have shown debugging data from crash dumps and minimal reverse engineering. By their account it's a null reference to a pointer in a system driver. That's something unit testing should have easily caught ... if used.

So here is what we know.

• Trivial error in the software, running as a system driver.
• Insufficient testing.
• Insufficient control over large scale rollouts.
• Not previously sharing release notes with customers.
• Not previously allowing customers to control timing of rollouts.
• Not previously allowing customers to use automated staged rollouts.

As someone working with governance in Enterprise IT, I am astonished they got this big without their customers challenging these things.

It's truly a WTF moment for the industry.

TBH I'm not surprised because the whole reason most companies bought their software was because the client didn't want to worry about any of these things. You can bet that the CIOs of these client companies were willing to break their own systems rather than get IN WRITING that CrowdStrike was actually doing best practices.

Software is basically unregulated and this failure is just a taste of how fragile our ecosystem really is.