Evaluating Remote Utilities - Is VirusTotal the Right Tool

What is VirusTotal?

If you're not familiar with VirusTotal, it's an online service (accessible via a website) where you can check files or URLs against a comprehensive array of antivirus/security software engines. VirusTotal generates an aggregated report that displays the results from each engine in alphabetical order, along with the detection name.

The service has become very popular among ordinary users who want to quickly check a program they've never used before. This is especially true for programs that may deal with sensitive data or authentication details.

Does VirusTotal Provide Definitive Results?

Overall, VirusTotal is a great service that aggregates results from multiple antivirus engines. You might think that the more engines it includes, the more accurate the results, right?

Not exactly. The issue isn’t with VirusTotal, which does a commendable job in aggregating results. The concerns lie in the following factors:

• False positive detections.
• The 'potentially unsafe' and 'potentially unwanted' classifications.
• The 'Hasty Generalization' fallacy and the way you perceive the reports.

VirusTotal.com doesn't provide an absolute 'Yes' or 'No' answer as to whether a certain file or URL is malicious. Instead, its approach is deeply scientific, offering a conclusion along these lines: 'This particular engine thinks that this file (or URL) might be malicious.'

False Positive Detections

When we say a detection is a false positive, it means the file has been erroneously identified as a threat (e.g., a virus or a trojan). With over 70 engines used by VirusTotal to aggregate results, there's always a possibility that several engines will mistakenly flag a benign file as such.

As VirusTotal states:

VirusTotal acts simply as an information aggregator, presenting antivirus results, file characterization tool outputs, URL scanning engine results, etc. VirusTotal is not responsible for false positives generated by any of the resources it uses; false positive issues should be addressed directly with the company or individual behind the product under consideration.

It should also be noted that false positive detections may arise due to very strict detection settings. Here's another quote from VirusTotal:

In VirusTotal, desktop-oriented solutions coexist with perimeter-oriented solutions; heuristics in this latter group may be more aggressive and paranoid, since the impact of false positives is less visible in the perimeter.

A general rule is that whenever a false positive detection occurs, it should be reported to the antivirus software company responsible for it.

A slightly less evident takeaway is that you must be prepared for the possibility that the engines used by VirusTotal may apply far stricter detection rules than the same engines in their off-the-shelf versions.

Potentially Unsafe vs. Potentially Unwanted

Some antivirus programs specifically mark certain classes of software as potentially unsafe (or 'grayware'). For example, here is the definition of potentially unsafe software by ESET:

There are many legitimate programs whose function is to simplify the administration of networked computers. However, in the wrong hands, they may be misused for malicious purposes. ESET provides the option to detect such applications.

Potentially unsafe applications is the classification used for commercial, legitimate software. This classification includes programs such as remote access tools, password-cracking applications, and keyloggers (a program that records each keystroke a user types).

Note that this classification refers to perfectly legitimate software, not malware.

However, some antivirus programs mistakenly apply a different classification to remote access software. This classification is called potentially unwanted programs (PUP) or potentially unwanted applications (PUA), but its use in the case of legitimate remote access software is unjustified, if not entirely incorrect. PUPs or PUAs typically refer to adware, junkware, and other kinds of unsolicited 'ware' that users would prefer not to install on their computers, often bundled with other legitimate software during installation. Legitimate remote control software, such as Remote Utilities, does not fall into this category because it is neither bundled with other software nor does it install any adware upon installation.

The Hasty Generalization Fallacy

Hasty Generalization also known as "the fallacy of a lonely fact" is a logical fallacy that involves making a claim based on a sample size that is too small to support definitive conclusions.

This fallacy comes into play when one concludes that a file or URL is malicious based solely on the fact that only a few of the many engines in a VirusTotal report detect it as such. This conclusion is hasty because it overlooks the possibility that these detections could be false positives, or that the majority of other engines did not find any issues.

Here is a screenshot of a Remote Utilities Agent file, signed with an EV Code Signing certificate:

In this VirusTotal report, there are four detections, all marked in red. What does the average person perceive here? They see "DANGER" in big, bold red font. But let's analyze in detail:

• The two detections on the left are classified as potentially unsafe or grayware—a category for legitimate software that might be considered dangerous under specific circumstances or in the wrong hands.
• The remaining two detections are false positives—erroneous results that incorrectly label the file as an outright threat, reflecting on the quality of those engines' algorithms. It's also important to note that VirusTotal does not differentiate between these two broad classifications; it simply marks any detection in red, regardless how benign.
• But most importantly, 65 out of 69 engines deem the file clean.

An essential takeaway here is this: if you're looking for a definitive yes/no answer on whether a certain file is malicious, as in this case, and you conclude that the file is a threat because four engines detect it as such, you must also accept that the other 66 engines are wrong.

Yes, this includes engines from Avast, BitDefender, Malwarebytes, Microsoft, McAfee, TrendMicro, among others. There's no middle ground. The file is either a threat or it isn't. By concluding that the file is a threat (as indicated by the four engines above), you are essentially asserting that the other 65 engines are mistaken in their clean classification.

The Empty File Experiment

To demonstrate the potential biases and unreliability of some antivirus engines towards certain products, we conducted an experiment. We created an empty file, signed it with our EV Code Signing certificate issued by the trusted authority DigiCert, and then uploaded it to VirusTotal (report link).

As of the time of this writing, there are still detections:

You can download both files (the original 'source' file and the signed compiled exe) and try the experiment for yourself.

The FP Report Submission Paradox

By now, you're aware that not all antivirus engines are created equal. Each of them, at times, mistakes benign files for threats. We refer to these erroneous detections as 'false positives', and they need to be reported.

But the question arises: whom do you report these to? Ideally, you should report them to the antivirus software company responsible for the false detection, as only they can rectify the issue by updating their signatures or algorithms.

However, we at Remote Utilities have observed a paradox over many years. Users, including those in corporate environments, tend to report (or complain about) these detections to us. Even when they have invested in expensive security solutions that erroneously flag Remote Utilities as a threat and disrupt their RU installations, they contact us to resolve the issue.

It remains a mystery why they don't first ask the antivirus software vendors to correct their faulty detection algorithms. This is why we refer to it as the false positive report submission paradox.

Conclusion

In conclusion, the challenges presented by utilizing services like VirusTotal for swift assessments of possible threats underscore a broader issue. It's not the tool itself that's at fault; rather, it's the interpretation of its results by the human mind that often leads to error. While one might hope for a report completely free of any red flags, the reality of false positives means this is not always practical. Drawing a hasty conclusion that a file is malicious, while ignoring the consensus of the majority — which often includes highly reputable engines — is to fall prey to a logical fallacy. It's essential to remember that the true value of tools like VirusTotal lies in their use as part of a more comprehensive approach to security, not as the sole arbiter of safety.

For your convenience, we have created a webpage with links to submit a false positive report to major antivirus software vendors.