TechRx-14th Edition

Inspiring Secure Collaboration - By: John Orosco Jr

If you would like to know more about the potential of Microsoft 365 for improving your organization's collaboration or need assistance with securing an existing Microsoft 365 tenant, know that we can help. For anything from implementing multi-factor authentication to preventing or limiting external sharing—and everything in between—contact our team today.?

John Orosco is a seasoned engineer on the Microsoft team at Prescriptive, bringing over a decade of technology experience in the healthcare industry. With a robust background in application and systems support, John specializes in cybersecurity best practices and has gained invaluable insights through incident response. Beyond his professional expertise, John is a dedicated father, an avid athlete, and a passionate tech enthusiast. He constantly seeks new challenges to push his mental and physical limits, always striving for personal and professional growth.?

Ep. 103: Mastering Market Moves: Insights from Go-to-Market Expert

With Phillip Swan and your host Paul Di Liegro

In this episode, we sit down with Philip Swan, a renowned go-to-market expert, to explore the intricacies of successful market strategies. Join us as we delve into the essential components of sales and marketing, uncover the secrets to exceptional customer experience and engagement, and discover how to create powerful product and operational synergies. Philip also shares his insights on driving business innovation and growth and the importance of responsible AI in sales. Whether you’re a seasoned professional or just starting, this episode is packed with valuable advice to help you navigate and excel in today’s competitive market.

Do you want to be the next guest on Business, Brains, and the Bottom Line?

• Contact Paul Di Liegro for more information!

CrowdStrike – So That Happened. What now??

CrowdStrike is one of the most respected Cyber Security companies in the world.?

For good reason, their products routinely score at the top in any category they participate in.??As evident by the blast radius of their recent misstep, a huge percentage of organizations rely on CrowdStrike to protect digital assets.?Their customers elected to go with CrowdStrike over other options and paid a premium. CrowdStrike isn’t the low-cost leader.??Clearly there are questions about their testing processes and inability to stop updates from continuing to be pushed out once it was clear there was a major problem.??

?

The problem is bigger than CrowdStrike; to name a few:

• Updates – To Auto or not to Auto.??
• For IT departments, updates have always been a challenge.??
• All the software a company uses and how to keep track of all the updates from each one.??
• The problem has become much bigger – Agile, SaaS. More software = more updates.??
• Auto updates seemed like the answer...
• No silver bullet?

This type of problem doesn’t get factored into the plans too often. Normally you worry about single points of failure in your design – failover / clustering for mission critical apps. Multi region.??

I can explain the problem, but I don’t have a good answer for how to solve it (yet).

• Industry holds vendors accountable.??

• Incident Response – Disaster Recovery.??

• If they took a snapshot (of the virtual machine) prior to the update, why couldn’t they just roll it back? Wouldn't they all be in sync???

• Can you update production and not DR???

• Documentation – It’s hard to recover an environment when you don’t know what you’re trying to get back to. People didn’t know how to or could not get to all their systems.? Documentation was on a server that failed. May not be as simple as turning the systems back on? Order matters, validation procedures.??

• ?Problem with centralization.? I understand the concerns about how much of the internet is controlled by a small number of giant corporations but in this case, we’re simply talking about a popular security solution.
• Market share – number of competitors.??
• Problem with the Windows Operating System – you shouldn’t allow 3rd party software this level of access to the Operating System.??
• Automatic updates. IT has a lot less control than ever.??

• You rely on SaaS providers, you’re beholden to their IT.??

In my view, responsibility for this specific incident falls on CrowdStrike and they’re right to take their lumps for it. Their validation and rollout processes are clearly not good enough.??

That being said, CrowdStrike is a well-respected Cyber Security company, and their products are popular for good reason. They provide some of the best cyber security tools on the market and are particularly adept at protecting large organizations. I'm confident that they’ll get these issues cleaned up.??

Exposes bigger issues such as:

• Your IT team or department has a lot less control than they used to. SaaS.??

• Volume of updates.??

• The actual Code?

• Configuration?

• Testing??

• Snapshot / Backups?

• Staggered Roll out?

• Documentation??

?

The irony is that one of the largest IT outages of all time was not caused by hackers but by a cyber security solution intended to protect you from hackers.?Those impacted were the ones that used auto updates, ensuring updates are applied right away, like a lot of the industry says you should do.?You purchased one of the premier cyber security solutions on the market, configured it based on “best” practices and here you are, neck deep in a catastrophe!??

CrowdStrike is justifiably taking their lumps in the press, on their stock price and likely in the courts. This was a big one and there is no question that their processes for validating and rolling out updates are flawed. That said, CrowdStrike still provides some of the best Cyber Security solutions on the market, and I don’t think this incident indicates a bigger problem with their organization.??

It highlights some larger, more complex challenges that the industry is confronting:

• Organizations use a tremendous number of applications, physical and virtual systems and devices. All of which require updates.??
• Updates are coming out faster and faster. 10 years ago, some applications were updated monthly but most were much less frequent, coming out quarterly or yearly.????
• Updates or patches often addressed security issues, but the primary focus was reliability, stability or enhancements. It’s not that the security flaws didn’t exist, it’s that the attacks weren’t as immediate and relentless as they are now.? The pressure to apply the updates right now, this second wasn’t as prevalent unless it specifically addressed a problem you were having.??
• Pressure to apply updates right away. According to Google's Mandiant in 2023,?33% of successful data breaches involve unpatched software or firmware. Cyber Insurance providers want to know that you’re on top of updates, that you’re applying them right away. If you call tech support about an issue you are having, one of the first questions you will get is have you applied the latest updates. If not, get ready to wrestle, regardless of whether the update indicates any relevance to the problem you are having.??

?

What Steps Should an IT Department Take???

Two primary approaches have been adopted: automatic updates and SaaS.??

Turning on automatic updates is the IT equivalent of waving the white flag of surrender. You’re conceding that you cannot manage all the updates, much less properly vet them before rolling them out, and do it fast enough to minimize your exposure to cyber security threats.??

SaaS on the other hand, mostly shifts the burden from your internal IT team to the vendor’s team. SaaS isn’t magic, those vendors have lots of software too, including security solutions, all of which need updates. It may not be “your fault” but if they take an outage, you’re impacted just the same.??

So, what should we do???

Unfortunately, there is no quick fix, no realistic steps you can take to ensure you’ll never experience this type of event. You can take some steps to make it less likely and hopefully recover more quickly if it does.??

• Vet your SaaS vendors. They should be able to provide clear documentation about how they handle updates and the controls they have in place. If they can’t, assume they’re cowboying it. You might think about picking a different vendor.??
• Control the updates where you can – at least the ones with the biggest potential impact.??

• Let someone else go first. If an update has a major issue, word spreads quickly and you’ll know whether to proceed or hold off.???

• Rolling Updates. Vendors should be doing this and most do, but a good practice is to apply the updates to a subset of the systems in your environment, before going whole hog.??
• Snapshots. This isn’t always an option but a lot of systems and virtual environments, whether on prem or in the cloud will let you take a snapshot including the operating system, prior to applying an update. This step won’t necessarily prevent an outage, but it will help you recover more quickly.??
• Documentation. This is not unique to this event, it’s applicable to any potential disaster.? Good documentation can be the difference between being down hours or days vs weeks or months. Do you know where all your systems are, what each one does, how to get to them, how to get authenticated at a lower level???

??

In conclusion, the IT industry is facing a dilemma between applying updates fast enough to avoid security breaches and testing updates thoroughly enough to avoid outages. Both scenarios can have devastating consequences for businesses and customers. Therefore, IT professionals need to adopt best practices such as vetting vendors, controlling updates, rolling updates, taking snapshots, and documenting systems. These steps can help reduce the risk of update-related incidents and improve the recovery process if they occur.?

Best Regards,

Terry Murray and the Prescriptive Data Solutions Team

Unhappy with your current MSP?

Discover how we can enhance the security of your environment, Contact Us today!

What Our Clients Are Saying

Don’t just take our word for it. Hear from real clients about our real, simple and effective solutions tailored to address the challenges they were facing. Interested in learning more about how Prescriptive can help your organization? Contact us today.