As you may know, some of the powerful tools on NirSoft Web site, especially the tools that recover passwords, are constantly targeted by many Antivirus programs.
In order to find out which Antivirus programs cause more troubles with the tools of NirSoft, I decided to generate a report with the number of false positive alerts of every Antivirus program. I have created a small program that downloads the Antivirus scans result of all .exe files of NirSoft from VirusTotal Web site and then processes the collected information and generates the desired report. I have also decided to generate score for every Antivirus program according to their false positive issues.
Before I continue with more information about this report… let me say a few words about the term “False Positive”: There are people who say that I don’t use the term “False Positive” correctly, simply because the alerts about my tools are not a mistake and the Antivirus programs have to display an alert about a program that can be used by hackers for bad purposes (like my password-recovery tools).
So here’s my opinion…. It’s somewhat legitimate that Antivirus program will display a warning about my password-recovery tools, as long as it’s done with full explanation about the alert, which means that the Antivirus program must explain the user that the program is completely legitimate and it’s not bad by itself, but it can be also used by hackers to steal passwords and that’s why the warning is displayed.
Also… the alerts on password-recovery tools should not be detected in VirusTotal Web site, unless this Web site will start to make full separation between Viruses/Trojans/Malwares and non-malicious tools, so people who check the file in VirusTotal will not think that my tool is an horrible Virus.
Unfortunately, Antivirus programs and VirusTotal Web site don’t provide clear explanation about the alerts they display and many people are confused, thinking that my tools are infected with Virus/Trojan, and As long as there are users who think that my programs are infected, I consider it as a “False Positive”. The right definition of “False Positive”, in my opinion, is a situation that a user thinks a file is infected with a Trojan/Virus/Malware according to an alert displayed by Antivirus software, while the file is not infected at all.
It doesn’t really matter that the Antivirus developers only wanted to warn the user about a software that can be used by hacker, if the Antivirus program doesn’t deliver the message to the end user correctly, then it’ still a false positive.
It’s important to say that some of the Antivirus programs imply that my tools are not a Virus by adding “not-a-virus” or “Hacktool” or “Riskware” strings to the alert name, but many
users don’t understand the meaning of these strings and still think that the file is infected. Nevertheless, in my score calculation , Antivirus programs that do it got an higher score.
Explanation about the report
The report contains 6 columns and one line for every Antivirus software/engine, here’s the description of every column:
- AV Name – The name of the Antivirus
- Total Alerts – The total number of NirSoft files that the specified Antivirus display alerts.
- No Virus – Number of alerts that contain the following strings, implying that NirSoft software is not a Virus/Trojan/malware: not-a-virus, tool, pup (potentially unwanted program) , pua (potentially unwanted application) , riskware, unwanted, passwordrevealer, not malicious, passwordviewer
- NO PR – Number of alerts for programs that are not a password recovery tool.
- Trojan Alerts – Number of alerts that contain the following strings, implying that NirSoft software is a Virus/Trojan (So these alerts are severe false positives): trojan, spyware, malware, adware.
- Score – Total score calculated for this Antivirus. Read the ‘How the score is calculated’ for more information.
How the score is calculated
Here’s a full explanation about how the Antivirus score is calculated:
- Every Antivirus engine starts with 100 points.
- For every alert displayed for a password-recovery tool, 1.5 points are reduced from the Antivirus score.
- For every alert displayed for a tool that doesn’t recover passwords, 3 points are reduced from the Antivirus score.
- When one of the following strings appear inside the alert name, 0.5 points are added to the Antivirus score: not-a-virus, tool, pup (potentially unwanted program) , pua (potentially unwanted application) , riskware, unwanted, passwordrevealer, not malicious, passwordviewer
That’s because the Antivirus does a good thing here, implying the my tool is not a Virus/Trojan/Malware. - When one of the following strings appear inside the alert name, 5 points are reduced from the Antivirus score: Trojan, spyware, malware, adware
That’s because the Antivirus does a bad thing here, implying the my tool is a Trojan/malware, which is completely a lie. Comodo, for example, displays ‘UnclassifiedMalware’ alert for 11 NirSoft files, which is totally misleading, because the “Malware” term is mostly used for programs that are designed to be bad , and that’s why they got very low score.
ViRobot and Antiy-AVL also got low score from the same reason.
Example for score calculation
AVG display alerts for 13 files, 12 of them are password recovery tools, so 1.5 * 12 = 18 points are reduced, 1 tool is not password recovery, so additional 3 points are reduced.
All 13 alerts contain ‘hacktool’ and ‘passwordviewer’ strings, so 13 * 0.5 = 6.5 points are added.
100 – 1.5 * 12 – 3 * 1 + 13 * 0.5 = 85.5
Finally… Here’s the report.
The report is based on Virus scanners results downloaded from VirusTotal on October 4, 2015. The NirSoft files taken from NirLauncher package 1.19.53. Be aware that Antivirus signatures changes every day, so it’s possible that if you check the virus alerts from today you’ll get a little different result. You can download a csv file containing all alerts found on this day from here. This file contains the Antivirus Name, the alert name, the NirSoft file that triggered the alert and the SHA-256 hash of this file, and you can optionally view this file with CSVFileView…
The good news in this report is that there are 12 Antivirus engines without any false positive and they got the best score possible (100)
The bad news – There are 2 Antivirus engines that show alerts for more than 100 files of NirSoft (!!) – Bkav and TheHacker, and they got very low negative score…
| AV Name | Total Alerts | No Virus | NO PR | Trojan Alerts | Score |
|---|---|---|---|---|---|
| AegisLab | 0 | 0 | 0 | 0 | 100 |
| Alibaba | 0 | 0 | 0 | 0 | 100 |
| ALYac | 0 | 0 | 0 | 0 | 100 |
| ByteHero | 0 | 0 | 0 | 0 | 100 |
| ClamAV | 0 | 0 | 0 | 0 | 100 |
| Emsisoft | 0 | 0 | 0 | 0 | 100 |
| Panda | 0 | 0 | 0 | 0 | 100 |
| Qihoo-360 | 0 | 0 | 0 | 0 | 100 |
| Tencent | 0 | 0 | 0 | 0 | 100 |
| TotalDefense | 0 | 0 | 0 | 0 | 100 |
| VBA32 | 0 | 0 | 0 | 0 | 100 |
| Zoner | 0 | 0 | 0 | 0 | 100 |
| nProtect | 1 | 0 | 0 | 0 | 98.5 |
| Microsoft | 3 | 3 | 0 | 0 | 97 |
| F-Prot | 2 | 1 | 1 | 0 | 96 |
| Avira | 5 | 1 | 0 | 0 | 93 |
| Cyren | 5 | 0 | 1 | 0 | 91 |
| Agnitum | 9 | 9 | 0 | 0 | 91 |
| AhnLab-V3 | 9 | 9 | 0 | 0 | 91 |
| CMC | 6 | 5 | 2 | 0 | 90.5 |
| Ikarus | 5 | 4 | 0 | 1 | 89.5 |
| Baidu-International | 6 | 6 | 2 | 1 | 86 |
| Kingsoft | 8 | 2 | 2 | 0 | 86 |
| AVware | 3 | 0 | 0 | 2 | 85.5 |
| AVG | 13 | 13 | 1 | 0 | 85.5 |
| Ad-Aware | 10 | 0 | 0 | 0 | 85 |
| BitDefender | 10 | 0 | 0 | 0 | 85 |
| F-Secure | 10 | 0 | 0 | 0 | 85 |
| MicroWorld-eScan | 10 | 0 | 0 | 0 | 85 |
| Jiangmin | 3 | 1 | 1 | 2 | 84.5 |
| Zillya | 10 | 9 | 0 | 1 | 84.5 |
| Avast | 14 | 14 | 1 | 0 | 84.5 |
| Malwarebytes | 11 | 11 | 4 | 0 | 83 |
| Kaspersky | 16 | 16 | 2 | 0 | 81 |
| K7AntiVirus | 17 | 16 | 2 | 0 | 79.5 |
| K7GW | 18 | 17 | 2 | 0 | 78.5 |
| Rising | 6 | 1 | 3 | 2 | 77 |
| VIPRE | 10 | 7 | 1 | 2 | 77 |
| SUPERAntiSpyware | 15 | 14 | 2 | 1 | 76.5 |
| CAT-QuickHeal | 21 | 21 | 3 | 0 | 74.5 |
| GData | 16 | 2 | 0 | 1 | 72 |
| Fortinet | 22 | 22 | 4 | 0 | 72 |
| NANO-Antivirus | 12 | 9 | 0 | 3 | 71.5 |
| DrWeb | 16 | 15 | 5 | 1 | 71 |
| Symantec | 20 | 14 | 4 | 0 | 71 |
| McAfee-GW-Edition | 24 | 21 | 4 | 0 | 68.5 |
| McAfee | 21 | 10 | 4 | 0 | 67.5 |
| Arcabit | 12 | 0 | 0 | 3 | 67 |
| TrendMicro | 24 | 0 | 3 | 0 | 59.5 |
| ESET-NOD32 | 26 | 16 | 8 | 0 | 57 |
| TrendMicro-HouseCall | 25 | 0 | 5 | 0 | 55 |
| ViRobot | 12 | 5 | 2 | 7 | 46.5 |
| Sophos | 34 | 32 | 19 | 0 | 36.5 |
| Comodo | 13 | 2 | 0 | 11 | 26.5 |
| Antiy-AVL | 27 | 19 | 7 | 13 | -6.5 |
| TheHacker | 113 | 0 | 104 | 1 | -230.5 |
| Bkav | 175 | 0 | 162 | 175 | -1280.5 |
It’s possible that I’ll generate another false positives report within a few months in order to check whether the Antivirus companies improve their software or they are getting worse…
Dennis says:
Interesting, I agree with you that these are false positives and I will be trying to us your list along with one that correctly reports viruses to choose my next Av app.
October 18, 2015, 6:13 pmGuest says:
If you would put some colors to the table that would greatly improve readability.
October 19, 2015, 3:38 amAVi says:
Thanks Nir,
I am soooooooooooooooo frustrated with antiviruses that report hacking tools as actual viruses.
99% of people don’t diffrentiate and think that if they have one of your tools or a keygen or anything that is rightly named by you as false positive is an actual virus.
I try to manually exclude those applications, but it is tedious and frustrating work.
I will use your list with conjuction with other lists in order to choose my antivrus, which probably means I will use Avira, as it gets your highest score and also scores high on general lists.
Thanks Nir,
And please continue creating your great software.
Avi – a long time follower
October 19, 2015, 5:01 amAnderson says:
Hey. You should keep the reports coming. Very useful. 🙂
October 19, 2015, 6:20 amGuest says:
lol get rekd bkav
October 19, 2015, 6:44 amnirsnoozer says:
So Nir tools are not spyware?! 🙂
By the way bluescreenview no longer works in Windows 7 anymore – doesn’t list bsod details!
October 19, 2015, 3:05 pmDHL says:
Most of the detections are NOT False Positives.
October 20, 2015, 1:44 pmThey are detected not because the files are malicious but because the files can be used maliciously.
coch says:
To me yes there are really false positives. If they are infected, flag them as such, if they aren’t, then don’t and do not want me that the tool has the potential for being malicious or has the potential for doing this or that which could perhaps he harmful. It’s infected by a virus/worm/trojan or it is not. Otherwise why not flagging cmd.exe as it can be used maliciously (del *.*, format C:, etc…) or while we’re at it, detect my whole PC as being malicious then, because yes my PC can be used maliciously if I decide to.
October 20, 2015, 5:51 pmAntiviruses should be truthful, otherwise it’s little more than fearmongering for the less savvy users, and annoyance for power users.
coch says:
Correction:
October 20, 2015, 10:07 pm…do not *warn* me…
Really says:
DHL (not thinking or just naive)
“Most of the detections are NOT False Positives.
They are detected not because the files are malicious but because the files can be used maliciously.”
I once installed a program from a ‘Confirmed’ site and it installed MIRC with C:\ shared and pinged to a site.
MIRC isn’t a malware, its chat program with features. Still it was NOT claimed to be a Virus or a Trojan.
But in this instance IT WAS a Trojan.
So here you have a Positive thats False and not the other way around.
A positive that was being used maliciously. And no one says a sh*t.
So Even If you have a AV installed you still have to Use Your Head.
October 28, 2015, 2:56 am(I have a program monitor installed that warned me)
Angel says:
Please the updated version with command line input is needed please. As for the detections its easy just tell the users of your product to turn off their antivirus to avoid flagging because as long as it retrieves passwords there nothing that can be done else you introduce cryptography in coding, or I can help you amend the source code if necessary.
October 28, 2015, 6:02 amUwe says:
Nirsoft programs are ranked as PUA and not as Trojan, malware or spyware.
November 3, 2015, 4:02 pmPotential Unwanted Software, due the possibility to read stored passwords (Firefox, Outlook, OfficeFiles, WLAN Keys)…
Just create in your viurs scanner an exclusion (exception) for the path or processes.
Thats all.
It’s just an “security feature” of your virus scanner, that it warns you about PUA.
Morten Bergfall says:
I might be somewhat egosentric for saying this,
but I’ve always felt that the people who deny themselves NirSoft’s many unique and persistent royalty of utility software
because they panic over alerts they never bother to fully read, from security related software they never bother grasping the workings of, get what they deserve.
A bit hen and egg problem…NirSoft can expand your knowledge a lot, but will it really in the hands of people not adventure-seeking enough to skip a warning or two.
At work you might not have a choice, but few of these tools should be used at work 😉
NirSoft has made my life so much easier, and given me a lot of Window’s internal knowledge,
November 5, 2015, 5:37 amin your debt forever, regards Morten Bergfall
DManitoba says:
Glad to see you have taken this step to address individuals, businesses, and organizations who, by their very actions, are doing their best to undermine or otherwise slander perfectly safe and useful software, and doing all that in order to make it appear as though their software “detects more”. It has been a practice I have been sick of for well over a decade, and quite frankly, a hundred thousand other developers need to step up and start pointing fingers as these B@t4(%s. I will note here that all of the “free” and “popular” programs (avast, et al) that your table notes as having double-digit false positives I have both trialled and un-installed, and all because of this very issue.
If reliable software free of any pain or hanky panky is considered non grata the instant you install their programs, it says all anyone needs to know about these programs. They lie. They profit from lying, and they certainly do not give a ^&@7 about those they harm, and all of the perfectly good software their programs are responsible for uninstalling needlessly from systems world-over.
Thank you, Nir.
November 30, 2015, 3:15 pmIts one more reason why I like you and your software.
Alexej says:
Fastresolver is detected (and blocked) by Sonicwall NSA 2400 as Necurs.SLO Trojan 🙁
January 13, 2016, 5:27 amDaniel Saner says:
This is a nice survey and was a very interesting read. I especially like how you consider the message displayed to the user in your scoring system. As pointed out, the heuristic analysis of software behaviour, and flagging suspicious ones, is not a bad thing in itself. The bad thing many antivirus solutions are guilty of is not being clear about this and telling the user that they didn’t actually detect a signature of a known malware, so there is a high degree of uncertainty in the verdict. Some of the things several of your tools do are quite sensitive, and I would definitely want a good antivirus solution to warn me about such behaviour if it was detected in any software that might be from a source less reputable than NirSoft, for instance.
There are always many conspiracy theories around these false positives and heuristical detections, especially where software is in a bit more of a legal grey area than around here. I don’t think there has ever been any truth to those accusations (as in, major software developers and publishers paying antivirus companies to flag certain tools they find undesirable). For one thing, if it was ever revealed, the antivirus developer would be finished. Trust is an important factor in their business. More simply, most of them probably just don’t care enough. Handling reports of false positives is a laborious and mostly manual process and the return is unclear. What I believe is the most important factor though, is the psychological one for most users of antivirus software. Detections generate trust! If novice users see their antivirus software detecting malware and offering to remove it, they feel like it’s doing a good job protecting them. If there are no detections, they might not trust that it is doing a good job. So even false positives can, on the whole, be beneficial to the producer of the AV software. That’s why you see a lot of those “rogue” anti-malware and PC cleanup utilities always detect dozens of things wrong with the system – to get inexperienced users to become afraid and shell out money for a fix they wouldn’t actually need.
January 31, 2016, 5:27 amgoldie says:
Well, it was about time that somebody with some skills took this issue on…
April 25, 2016, 7:46 pmXiaopang says:
Awesome article and very nice list. Your calculation itself doesn’t help comparability though. It’s good that 100 is the top score, but a user would expect the minimum value to be Zero, not some minus value. Right now, I have no clue what the absolute zero is, so it’s impossible to judge for example how bad the lower ranking software is. Ideally, the better calculation would have been based on a scoring system ranging from zero to one hundred:
100/number of nirsoft applications = x
For every unwarranted mentioning of a nirsoft tool subtract x
For every positive tag (PUA, PUP etc) subtract only 0.5x.
This way the resulting scores would be linear and thus comparable.
That being said, I want to thank you for all your great work. I’ve been using your tools for over a decade now and hope I will be able to enjoy your awesome applications for decades to come 🙂
February 5, 2017, 8:15 amDave Ward says:
I just tried to start “Smart Sniff”, and it was blocked and quarantined by Avast! I have not any problems previously, and it’s been present on my computer, and used many times, for several years. I restored it from the Virus Vault (along with my desktop shortcut) and it works fine. I submitted it to Virus Total and got a score of 5/59, yet Avast! wasn’t one of them! I retested it and now get 6/61, yet Avast! still doesn’t object. Bizarre…
Thanks for all your efforts.
March 16, 2017, 5:54 pmDavid says:
Bkav still seems to be poor regarding false positives. I have a small open source game – provably clean, compiled from source code on a clean system – submitted the release to virustotal and bkav is the sole antivirus out of 60 that flags it as malware. Reported it two weeks ago but no response from them.
July 11, 2017, 5:22 amdanny says:
The problem with antivirus and NIR software is that of course they do not work closely with each other. Otherwise there would be a problem.
The issue with the antivirus software companies are they view that any software that starts to work beneath the hood of the engine is something to be feared. If you know what the program does and know how to monitor how it works, you can quickly determine that the Nir software is NOT harming your computer. You can put on network diagnostics or files update on your computer etc.
To use NIR software one has to really have some sort of computer background to really know HOW TO USE IT.
Understanding the background of the utility is where the clear understanding will come on how it works and if it really is a virus.
Just putting a publicly virus signature found on most antivirus websites into any software will freak out the antirus softwares because they mostly work of the signatures. Programs have signatures with in them and that is how they are caught.
thanks for your software danny
July 12, 2017, 2:40 pmJoep van Steen says:
BKAV is still a useless piece of crap today https://www.virustotal.com/gui/file/a4d2dd73c03031b6f8edfb35c3678a059798306576b56f597793557727aa43db/detection
August 6, 2020, 9:09 am