As you may know, some of the powerful tools on NirSoft Web site, especially the tools that recover passwords, are constantly targeted by many Antivirus programs.
In order to find out which Antivirus programs cause more troubles with the tools of NirSoft, I decided to generate a report with the number of false positive alerts of every Antivirus program. I have created a small program that downloads the Antivirus scans result of all .exe files of NirSoft from VirusTotal Web site and then processes the collected information and generates the desired report. I have also decided to generate score for every Antivirus program according to their false positive issues.
Before I continue with more information about this report… let me say a few words about the term “False Positive”: There are people who say that I don’t use the term “False Positive” correctly, simply because the alerts about my tools are not a mistake and the Antivirus programs have to display an alert about a program that can be used by hackers for bad purposes (like my password-recovery tools).
So here’s my opinion…. It’s somewhat legitimate that Antivirus program will display a warning about my password-recovery tools, as long as it’s done with full explanation about the alert, which means that the Antivirus program must explain the user that the program is completely legitimate and it’s not bad by itself, but it can be also used by hackers to steal passwords and that’s why the warning is displayed.
Also… the alerts on password-recovery tools should not be detected in VirusTotal Web site, unless this Web site will start to make full separation between Viruses/Trojans/Malwares and non-malicious tools, so people who check the file in VirusTotal will not think that my tool is an horrible Virus.
Unfortunately, Antivirus programs and VirusTotal Web site don’t provide clear explanation about the alerts they display and many people are confused, thinking that my tools are infected with Virus/Trojan, and As long as there are users who think that my programs are infected, I consider it as a “False Positive”. The right definition of “False Positive”, in my opinion, is a situation that a user thinks a file is infected with a Trojan/Virus/Malware according to an alert displayed by Antivirus software, while the file is not infected at all.
It doesn’t really matter that the Antivirus developers only wanted to warn the user about a software that can be used by hacker, if the Antivirus program doesn’t deliver the message to the end user correctly, then it’ still a false positive.
It’s important to say that some of the Antivirus programs imply that my tools are not a Virus by adding “not-a-virus” or “Hacktool” or “Riskware” strings to the alert name, but many
users don’t understand the meaning of these strings and still think that the file is infected. Nevertheless, in my score calculation , Antivirus programs that do it got an higher score.
Explanation about the report
The report contains 6 columns and one line for every Antivirus software/engine, here’s the description of every column:
- AV Name – The name of the Antivirus
- Total Alerts – The total number of NirSoft files that the specified Antivirus display alerts.
- No Virus – Number of alerts that contain the following strings, implying that NirSoft software is not a Virus/Trojan/malware: not-a-virus, tool, pup (potentially unwanted program) , pua (potentially unwanted application) , riskware, unwanted, passwordrevealer, not malicious, passwordviewer
- NO PR – Number of alerts for programs that are not a password recovery tool.
- Trojan Alerts – Number of alerts that contain the following strings, implying that NirSoft software is a Virus/Trojan (So these alerts are severe false positives): trojan, spyware, malware, adware.
- Score – Total score calculated for this Antivirus. Read the ‘How the score is calculated’ for more information.
How the score is calculated
Here’s a full explanation about how the Antivirus score is calculated:
- Every Antivirus engine starts with 100 points.
- For every alert displayed for a password-recovery tool, 1.5 points are reduced from the Antivirus score.
- For every alert displayed for a tool that doesn’t recover passwords, 3 points are reduced from the Antivirus score.
- When one of the following strings appear inside the alert name, 0.5 points are added to the Antivirus score: not-a-virus, tool, pup (potentially unwanted program) , pua (potentially unwanted application) , riskware, unwanted, passwordrevealer, not malicious, passwordviewer
That’s because the Antivirus does a good thing here, implying the my tool is not a Virus/Trojan/Malware. - When one of the following strings appear inside the alert name, 5 points are reduced from the Antivirus score: Trojan, spyware, malware, adware
That’s because the Antivirus does a bad thing here, implying the my tool is a Trojan/malware, which is completely a lie. Comodo, for example, displays ‘UnclassifiedMalware’ alert for 11 NirSoft files, which is totally misleading, because the “Malware” term is mostly used for programs that are designed to be bad , and that’s why they got very low score.
ViRobot and Antiy-AVL also got low score from the same reason.
Example for score calculation
AVG display alerts for 13 files, 12 of them are password recovery tools, so 1.5 * 12 = 18 points are reduced, 1 tool is not password recovery, so additional 3 points are reduced.
All 13 alerts contain ‘hacktool’ and ‘passwordviewer’ strings, so 13 * 0.5 = 6.5 points are added.
100 – 1.5 * 12 – 3 * 1 + 13 * 0.5 = 85.5
Finally… Here’s the report.
The report is based on Virus scanners results downloaded from VirusTotal on October 4, 2015. The NirSoft files taken from NirLauncher package 1.19.53. Be aware that Antivirus signatures changes every day, so it’s possible that if you check the virus alerts from today you’ll get a little different result. You can download a csv file containing all alerts found on this day from here. This file contains the Antivirus Name, the alert name, the NirSoft file that triggered the alert and the SHA-256 hash of this file, and you can optionally view this file with CSVFileView…
The good news in this report is that there are 12 Antivirus engines without any false positive and they got the best score possible (100)
The bad news – There are 2 Antivirus engines that show alerts for more than 100 files of NirSoft (!!) – Bkav and TheHacker, and they got very low negative score…
AV Name | Total Alerts | No Virus | NO PR | Trojan Alerts | Score |
---|---|---|---|---|---|
AegisLab | 0 | 0 | 0 | 0 | 100 |
Alibaba | 0 | 0 | 0 | 0 | 100 |
ALYac | 0 | 0 | 0 | 0 | 100 |
ByteHero | 0 | 0 | 0 | 0 | 100 |
ClamAV | 0 | 0 | 0 | 0 | 100 |
Emsisoft | 0 | 0 | 0 | 0 | 100 |
Panda | 0 | 0 | 0 | 0 | 100 |
Qihoo-360 | 0 | 0 | 0 | 0 | 100 |
Tencent | 0 | 0 | 0 | 0 | 100 |
TotalDefense | 0 | 0 | 0 | 0 | 100 |
VBA32 | 0 | 0 | 0 | 0 | 100 |
Zoner | 0 | 0 | 0 | 0 | 100 |
nProtect | 1 | 0 | 0 | 0 | 98.5 |
Microsoft | 3 | 3 | 0 | 0 | 97 |
F-Prot | 2 | 1 | 1 | 0 | 96 |
Avira | 5 | 1 | 0 | 0 | 93 |
Cyren | 5 | 0 | 1 | 0 | 91 |
Agnitum | 9 | 9 | 0 | 0 | 91 |
AhnLab-V3 | 9 | 9 | 0 | 0 | 91 |
CMC | 6 | 5 | 2 | 0 | 90.5 |
Ikarus | 5 | 4 | 0 | 1 | 89.5 |
Baidu-International | 6 | 6 | 2 | 1 | 86 |
Kingsoft | 8 | 2 | 2 | 0 | 86 |
AVware | 3 | 0 | 0 | 2 | 85.5 |
AVG | 13 | 13 | 1 | 0 | 85.5 |
Ad-Aware | 10 | 0 | 0 | 0 | 85 |
BitDefender | 10 | 0 | 0 | 0 | 85 |
F-Secure | 10 | 0 | 0 | 0 | 85 |
MicroWorld-eScan | 10 | 0 | 0 | 0 | 85 |
Jiangmin | 3 | 1 | 1 | 2 | 84.5 |
Zillya | 10 | 9 | 0 | 1 | 84.5 |
Avast | 14 | 14 | 1 | 0 | 84.5 |
Malwarebytes | 11 | 11 | 4 | 0 | 83 |
Kaspersky | 16 | 16 | 2 | 0 | 81 |
K7AntiVirus | 17 | 16 | 2 | 0 | 79.5 |
K7GW | 18 | 17 | 2 | 0 | 78.5 |
Rising | 6 | 1 | 3 | 2 | 77 |
VIPRE | 10 | 7 | 1 | 2 | 77 |
SUPERAntiSpyware | 15 | 14 | 2 | 1 | 76.5 |
CAT-QuickHeal | 21 | 21 | 3 | 0 | 74.5 |
GData | 16 | 2 | 0 | 1 | 72 |
Fortinet | 22 | 22 | 4 | 0 | 72 |
NANO-Antivirus | 12 | 9 | 0 | 3 | 71.5 |
DrWeb | 16 | 15 | 5 | 1 | 71 |
Symantec | 20 | 14 | 4 | 0 | 71 |
McAfee-GW-Edition | 24 | 21 | 4 | 0 | 68.5 |
McAfee | 21 | 10 | 4 | 0 | 67.5 |
Arcabit | 12 | 0 | 0 | 3 | 67 |
TrendMicro | 24 | 0 | 3 | 0 | 59.5 |
ESET-NOD32 | 26 | 16 | 8 | 0 | 57 |
TrendMicro-HouseCall | 25 | 0 | 5 | 0 | 55 |
ViRobot | 12 | 5 | 2 | 7 | 46.5 |
Sophos | 34 | 32 | 19 | 0 | 36.5 |
Comodo | 13 | 2 | 0 | 11 | 26.5 |
Antiy-AVL | 27 | 19 | 7 | 13 | -6.5 |
TheHacker | 113 | 0 | 104 | 1 | -230.5 |
Bkav | 175 | 0 | 162 | 175 | -1280.5 |
It’s possible that I’ll generate another false positives report within a few months in order to check whether the Antivirus companies improve their software or they are getting worse…