Stackhackr; Useless for Testing – Good for Marketing

Barkley, a self-proclaimed security company, has revived the 1990’s era Rosenthal Virus Simulator; an application that called false positives good while claiming to test the quality of antivirus products. Some users believed that this simulator indicated if an antivirus product was good at detecting malware. As a result some AV companies wrote detection specifically for Rosenthal’s harmless files. The customers wanted harmless false positives for harmless files and so they got them.

Barkly has come out with a free product they call stackhakr. Stackhackr is a lead generation application that is disguised as a security product testing tool. In reality it is another Rosenthal type program that convinces users that false positives mean better security.

According to Barkly “The malware you create won’t actually cause any harm, but whether it runs or gets blocked will tell you if your system is vulnerable to the real thing.”

Really? If a completely ineffective security product writes detection specifically for this application then you are not vulnerable to the real thing? If a product false positives and detects your harmless files, then the company’s customers are not vulnerable to ransomware? In order to use stackhackr you have to provide your contact information. It is only then that you get something that does not do what it was promised to do. Like I said, stackhackr is a lead generation application, not a test tool.

Stackhakr does not test the ability of a product to detect ransomware, malware, or the ability of a product to effectively deal with any attacks. Due to the security effectiveness of application reputation Barkly specifically calls out this type of detection as a false positive. Barkly claims that detection of their launcher application is a false positive because the launcher file is harmless and not part of the test. Seriously? Detecting a harmless launcher is a false positive but detecting the harmless files it writes is not? Take me to security school, I had no idea that’s how it works. In reality detecting a “harmless” file is not a false positive when it is only ever seen launching malware. Blocking a launcher or a dropper before it delivers its payload is a good thing. If launcher.exe is used to launch the simulator then it is fair game. Blocking the launcher protects users from a false sense of security. The detection is accurate, not a simulation but real protection against deception.

Now for all you AV vendors, Barkly has thrown down the gauntlet, so what are you going to do? If you identify a site delivering ransomware or other malware you block the site. If simulated ransomware or simulated malware creation kits are on https://stackhackr.barkly.com/, then let’s get this simulation off the ground and go block the site. Be sure to mention it is a simulated malware toolkit creation site you are simulating detection of.

I have interacted with major security product testing organizations as an enterprise security professional and as an employee of a security vendor. I have actually worked for a company (NSS Labs) that tests (and breaks) security products. There are no competent testers in the world that would tell you that stackhackr is usable as a security product testing tool.

I recommend against giving Barkly your user information in exchange for stackhackr. You will not receive anything I can deem as even slightly valuable.

Randy Abrams

Independent Security Analyst