Confessions of a Recovering Cybersecurity Creature

This is from a series of articles I wrote at Medium (https://billfordx.medium.com). I thought it would be a bit better here.

Story #1: Watch out for the trusted vendor.

I’ve taken a lot of security training over the years and have mentored nearly countless kids out of college or just breaking into the industry. One thing I’ve noticed, and it is sort of universal, is that no one prepares them for dealing with the heralded Cybersecurity vendors. The vendor is where they’re told to try to get information from; the vendor is who their company will send them to take training, and the vendor is often the only source of security news folks will use. Oftentimes, folks are so overloaded that if a vendor doesn’t tell them the bad news, they’ll never know it. It happens more than anyone would like to admit. I wanted to write a piece to share some of my experiences with vendors. It’s not a pretty piece, but it is the truth, according to me. Who am I? Let’s back up, and I’ll tell you a little of my experience.

I’m old, like really old, and I’ve been doing computer “stuff,” as my Grandma and Mom called it, since about 1981 or so. Security “stuff” didn’t really exist and no one called anything “cyber” yet, not really anyway. I was obsessed with taking my Atari 2600 apart and putting it back together, as well as anything else I could get my hands on. This is a pretty typical security guy story. Fast forward to the 90s, and my computer is getting a virus called Monkey.B (yes, I still remember the name), and I wanted to learn all I could about computer security. There wasn’t a lot available back then, but the library never failed me, so I checked out a bunch of books and copied some magazine articles; I was hooked. As the 90s went on and Linux was released, I learned about vulnerability assessments, web applications, penetration testing, and other security things, but I was still a programmer at heart.

My first “real” job was coding web applications, such as they were in the mid-90s. Then, I moved on to networking and firewalls. Finally, I moved into management but never lost my technolust. So that’s an abbreviated background story on me, which is important for the first post in this series. The series’ main theme is that I’ve grown to detest what is now called Cybersecurity, even though I hate calling it that. I’ve now officially (mostly) retired from the industry due to health issues, so it would help to get this stuff off of my chest. I hope it also helps some of the younger folks just getting into the industry so they’ll be ready for the stuff they don’t teach at school or so-called boot camps. If it gets too negative or overly angry, I apologize to the reader but not to the industry.

That’s enough about me. Now, onto the complaining, I mean, educating. The so-called Cybersecurity industry is peopled with charlatans. They push products that do not perform as they claim, and the really bad ones don’t even try to make up “lab” results for their products; they don’t test them. As the AI bubble starts to burst, you’ll find this exact same behavior. Why? Because a lot of them came from the Cybersecurity business. This is simply how they operate. They see nothing wrong with it; just roll it out to the customer and disclaim yourself from any real liability. Easy money! I’ve seen this so many times that it is pretty much a cliche; we used to open support tickets with a vendor and, after explaining the problem, say, “I guess this will be in the next release?” It was always in the next release. Somehow, it just never showed up. By the way, these are the good vendors, the ones you’d likely trust. Let’s talk about the bad ones.

The bad vendors promise the world and deliver next to nothing. They often show you a souped-up and mostly fake demo, and once you move to the proof of concept phase, the excuses and promises of the “next release” begin. You cannot expect a “next release” from a vendor that is barely in business. You can barely expect it from the bigger ones. These types of vendors are not worth your time and definitely not worth the money. Avoid these vendors at all costs. You’ve been warned.

None of this is to say there aren’t good people in the industry; there are, and some of them even work at vendors, both good and bad. This is mostly a business problem, but it’s a familiar story. Some vendors start out small with maybe a little (relatively speaking) seed capital to get the business going. The business starts hiring, product development, and laying the groundwork for its sales and marketing drives. This is the way of the startup world: get some money, spend the money, get some more money, and then maybe start turning a profit, but not until forced to. This last part is where the problem is.

Investors in startups will start out being patient and claim to be “partners,” but they want a return on their investments. You can’t blame them, but they will start to turn more aggressive about seeing a profit. This forces the vendor to start demanding faster product release schedules, faster support call resolution time, and faster everything. Once the “faster” cycle kicks in, any hopes for a high-quality product slip away. At this point, they start chipping away at the startup's reputation; they’ll start hiring less experienced developers and support folks, all to cut costs and show the investor they’re trying. At this point, the startup vendor will usually just sell to a larger vendor and be done with it. The ones that persevere will go public and then sell to a larger vendor. I’ve simplified the process a little because I’m sure they teach it in business school, but you can read the history of just about every vendor in the last 25 years; they follow this cycle. It’s at the heart of the problem.

Storytime: This is about a real vendor, full of really good people that I genuinely liked. They followed this process to a tee, so let’s see what happened. I won’t name them because I’m not trying to shame anyone here. Alright, here we go.

We were looking for some software that did a very particular thing (I won’t name the thing because the vendor would be apparent), and it had to do it well and fast. We were processing millions of records a minute or so; it was a very niche thing we were doing. We found a, then, small open-source project that did almost exactly what we needed, so we started using that and built a relationship with the author. It worked reasonably well, and then the person who wrote it started a company; that was cool. In the beginning, we partnered with the new company and had a licensing deal in place to use their product, and everything was just great. Then, the investors crept in.

Suddenly, our licensing fees started to creep up; then, they started sprinting up until they became untenable. The former open-source author was replaced at the top by someone the investors liked, prices continued to skyrocket, and all that was left was for them to sell to a larger vendor when all the little deals they had would be cut; their support had already been cut, and their development has stagnated. They have no future but to become ghosts of their former selves, if they survive at all. This is the vendor lifecycle in cybersecurity, so be careful about which vendors you choose. Even if you’re super careful, you might still end up with a terrible experience and have to replace a working solution with a crude piece of code that only you know how to support. It is not optimal.

The vendor problem in Cybersecurity isn’t really a unique problem; it happens to vendors, restaurants, and retail places all over the country. It is largely a problem of scale, I think; vendors grow as much as they can, so then artificial growth is introduced, such as cutting large amounts of employees and scaling back services and support. Suddenly, there’s a profit surge, and the investor is happy until the next time. The problem in cybersecurity is, as I’ve alluded to, a lot of folks depend on these vendors to be of the highest quality and provide them support when they need it. It’s hard to do that when you’ve cut your support staff to the bone, and no one is there to answer the phone when the customer calls. See if you can name three vendors this has happened to. I bet it won’t take long.