Evaluating security software is not a walk in the park. It's more like a trip to the circus: Vendors promise the world — 100 percent accuracy! 100 percent protection! — for a big one-time fee plus an annual subscription. The one thing you can be sure of is that the results won't measure up to the promise.
If it's your job to choose security software, you have my sympathies. To make things a little easier, I offer my seven-point plan for evaluating solutions. The products have changed radically during the two decades I've been using this framework, but it works. That's why I still swear by it.
1. Write down your goals
First, write down the tactical goal you want to meet with a new product. This might seem ridiculously easy at first, but it becomes less so as you get more specific. For example, you might say you "want antivirus software," but at the next level of detail, what you really want is an antimalware program that runs on the server and on clients, with real-time, on-demand, and scheduled scanning. Does the antimalware program also need to offer a host-based firewall, include antiphishing, or send alerts to your help desk?
2. Create a feature list
Here's a sample list of options: What clients and servers must the product protect? What types of servers (Windows, Linux, and so on) must the management and production software run on? You might also want to indicate which database (Microsoft SQL Server, Oracle, MySQL, Hadoop) and Web server (Apache, IIS) technologies you'll accept. Do you want the product or service to protect mobile computers, tablets, and smartphones? Must the protected clients already be managed by your company? Must the clients reside on the corporate network, or can they be located across the Internet? If you put a lot of thought into your feature list, it can easily reach several dozen requirements.
Be sure to indicate which features are deal breakers vs. nice-to-haves. Have the selection stakeholders (hopefully including both management and end-users) review and approve the feature set. Get everyone to agree to the deal breakers.
3. Do your research
I'm a big fan of software reviews. Rarely do I read one and fail to come up with an issue or feature I would have missed had I tested the software before reading the review. Often, flaws noted in review articles will be fixed by the time you personally review the software, but knowing what was problematic and how the vendor fixed it can help in your decision.
4. Create a test environment
I highly recommend using an isolated test environment before performing even limited testing in your production environment. The biggest decision to make is how much effort you want to put into creating a test environment that accurately mimics your production environment.