CrowdStrike-Microsoft IT outage ‘highlights importance of freelance tech staff resource’

One of the largest ever IT outages proves that companies can’t always know who they’ll need from IT, with what tech skills, or when.

Posting just as MS devices went blue-screen early Friday, a developer suggested the outage may therefore lift contract IT staff’s profile.

Affecting Windows v7.11 and above running Falcon Sensor by CrowdStrike, the outage has since gone on to hit 8.5million devices.

'Microsoft outage'

Following the Global Pay Association (among others) describing it a “Microsoft outage,” the software behemoth was quick to step in.

Sarah Armstrong-Smith, Microsoft’s chief security adviser, clarified online that “this was not a Microsoft incident.”

“[But] given it impacts our ecosystem, we want to provide an update on the steps we’ve taken with CrowdStrike.”

Armstrong-Smith then linked to a Microsoft blog entitled “Helping our customers through the CrowdStrike outage.”

'CrowdStrike Falcon agent issue'

A principal PM Manager for MS subtly pointed the finger too, albeit in a post offering IT admins a USB tool to expedite the repair process.

The Microsoft manager, Nir Hendler wrote:

“As a follow-up to the CrowdStrike Falcon agent issue impacting Windows clients/servers, we have released [this].”

CrowdStrike CEO George Kurtz was initially criticised for taking to ‘X’ about the issue “without a hint of contrition for the chaos caused.”

'Quickly identified the issue and deployed a fix'

But the criticism (levied by the Financial Times), was short-lived thanks to a statement posted by Kurtz just moments later.

In an open-letter to CrowdStrike “customers and partners,” Kurtz wrote:

“I want to sincerely apologize directly to all of you for today’s outage.

“All of CrowdStrike understands the gravity and impact of the situation. We quickly identified the issue and deployed a fix, allowing us to focus diligently on restoring customer systems as our highest priority.

“The outage was caused by a defect found in a Falcon content update for Windows hosts. Mac and Linux hosts are not impacted. This was not a cyberattack.

“We are working closely with impacted customers and partners to ensure that all systems are restored, so you can deliver the services your customers rely on.”

'Technical details'

For IT admins, CrowdStrike also posted “technical details” of the corrupted software update, and a FAQs, plus a repeat of the apology.

(Editor’s Note: An account of the IT outage for a non-technical audience has been written by independent digital forensics and cyber incident adviser Andrew Harbison, here)

A technology sector thought-leader, Larry Whiteside Jr, says CrowdStrike’s response is good enough for him.

But it’s clearly not satisfying everyone. “I saw one social media post which said, ‘I wish CrowdStrike would go bankrupt,’” Whiteside Jr began.

“If this were a systemic problem -- and we do know companies who have had systemic problems, I’d get it.

“Or if this were them [CrowdStrike] coming out and lying about something, I’d get it. But that’s not what happened here. 

“I think all companies should hold them [CrowdStrike] accountable to finding a root cause analysis. But to lambast them is one of the problems”.

'Expectationt that technology vendors are going to be perfect'

Whiteside Jr continued: “[Instead why aren’t we] recognising that each of us should be building resilience into our technologies? And into organisations? Because technology will fail at some point.

“I’ve been in this industry 30-plus years, I’ve never seen technology not fail. Some component of it is going to fail. At some point. Somewhere. 

“So -- why do we have this expectation that the vendors that deliver us technology are going to be perfect?”

Technology services sellers will be glad if an upshot of the CrowdStrike-Microsoft IT outage is being regarded more realistically.

'Thankless task'

Or even just less harshly. In a video to his followers, Whiteside Jr said the IT outage showed some hardware and software suppliers have a “thankless task”.

“You can go 364 days without having an issue, and on that 365th day, that one issue does pop up. [And the reaction is like] ‘The world is coming to an end.’ And ‘How could YOU let this happen?’”

The developer who hoped the IT outage might spotlight the importance of external, freelance techies - who can often hit the ground running and at a moment’s notice, posed his own question.

'Companies don't always know when or what tech skills will be needed'

“I wonder if the outages in the last few days will lead to a realisation that companies may need short-term hiring of skilled staff?”

Online, commentators say the outage will at least see disaster recovery and business contingency plans reviewed by companies.

But specialising in Java Spring, the developer maintained in wake of 8.5m devices lighting up with the Blue Screen of Death: “They don’t always know when [skills] -- or what skills -- will be needed.”

'Chaos'

What is needed is rigorous software testing and appropriate governance, says cyber security specialist Justin Bentley.

“Why CrowdStrike broke all the rules and released an update that wasn’t appropriately tested [still hasn’t been explained to me],” Bentley alleged in a LinkedIn post, adding:

“Or if it was [appropriately tested] that makes the situation worse, as they would have knowingly released chaos on the world!

“This should have never happened and its release [should] have been stopped. Either way, appropriate governance can’t have been followed to allow a simple process to break so much. I expected this 20 years ago not today.”

'Global IT outage highlights how essential techies - many freelance - are'

Andy Chamberlain of IPSE says that if anything good comes out of “one of the biggest IT outages we have ever seen” (TechMarketView analyst Simon Baxter), it will likely be the oft-overlooked, but seminal importance of freelance techies who, compared with full-time staff, can be always-on and always on stand-by.

Policy director of the Association of Independent Professionals and the Self-Employed, Mr Chamberlain last night told ContractorUK:

“The global IT outage highlights how essential our tech professionals are to the economy. Many of those professional work as freelancers, offering their services to a range of clients across industries.

“Freelancers are essential part of the resourcing mix, particularly in the tech sector. As technology becomes more embedded in our everyday lives, the availability of flexible skills will become ever-more crucial.”

CrowdStrike has been invited to comment on whether its deployment to fix its software flaw and resolve the outage included technologists who are freelance, contract or temporary and, if so, in what numbers.