AV-Comparatives Feb 2009 On-Demand Antivirus Test Results

AV-Comparatives, one of the leading independent security software testing organizations, has published their latest antivirus test report.
All major AV products have been tested in catching trojans, viri and other malware.
As stated in their report, final results are based on a combination of detection rates and the number of false positives i.e. clean files which are marked as (containing) malware.
This time the FP treshold was 15, (0-15 and 16-100).

AV products with the highest detection rates that also produce more than 15 FP's with the test of 1,3 million malware samples, get a lower score.
An example is Avira with 99,7% detection and 24 FP's, rating "2 Stars/Silver" and Kaspersky with 97,1% detection and 14 FP's, rating "3 Stars/Gold".

The winners in the AV-Comparatives test, all with 3 Stars/Gold are;
Symantec AV
Eset NOD32
Kaspersky AV
McAfee VirusScan+

The 3 free AV's listed in the free security software thread, score as follow;
Avira 99,7% detection - 24 FP's 2 Stars/Silver
Avast 98,2% detection - 28 FP's 2 Stars/Silver
AVG 93% detection - 14 FP's. 1 Star/Bronze

Please read the full report yourself, to know what AV versions have been tested, what methodology AV-Comparatives uses (methodology PDF link), the malware sample set used, what scan settings were used and what tests are coming up next with the programs tested and more.

As they write themselves, their test is one of a few done by independent organizations but only one, so these scores should be seen as quality guidelines.
Not as the be-all and end-all answer to what's ' the best'.
It depends on your personal preference; do you like to have the highest detection rate, no matter what, or do you despise FP's (even a few).
Read about it yourself; the Februari 2009 test results can be found here (PDF).

 

Thanks for info Baserk. I guess looking through all of it, depending on how one reads it, the message is that most of the packages they tested can be considered Very Good and all of them can be considered Imperfect, which we already knew.

 

Not sure if I agree with their ranking system.

I.e., Symantec had a lower detection rate but less false positives than Avira, and so it got 3 stars/gold. I'd rather detect viruses and have false positives than miss a virus but not have false positives. Missing a virus that could do damage is a LOT worse than having to check if a virus detection is true. Just my opinion, but I will admit I'm biased for Avira since I love it so much.

 

i disagree, FPs are just as bad, specially in companies, and can harm productivity costing the company a lot of money (the time their offline).
If a AV detects windows components (see AVG, Norton's recent history of FPs) you can end up with a windows that isn't working, and that's actually worse that most malware you would encounter on a regular basis out there.

 

it is a good information about the new anti virus software .

symantec takes the highest ratings for all these software.

avira is also the best software for the threads.

 

I agree. AVG used to have a very serious FP issue; main reason that got me off their bandwagon years ago although they seem better in that respect today.

 

Thanks Baserk, for the Comparatives

I like where Avira is ranked! It really is a grat AV software! (and free)


Cin

 

Gunna have to disagree with you. First point is that having a virus is much more likely to be worse than having a false positive. Second point is that you're much more likely to get a virus than a false positive, as shown given the amount of viruses detected versus the amount of false positives detected. It might be possible, and it might even have happened in the past (and its because of its extraordinary circumstance that you would have heard about it), but it is extremely unlikely that a false positive will render an entire operating system useless like you say. Yes, if a critical system file gets quarantined, it could do that. But any decent antivirus software is not going to mistake a critical system file for a virus, and even if a poorly written program did, then you could just remove it from quarantine if you're not a 60 year old bookkeeper that doesn't know how to use anything other than excel and peachtree. That being said, viruses are often designed to disrupt work in exactly the way you're suggesting a false positive could. And when they're not designed to disrupt work but instead are designed to be undetectable, then their goal will likely be theft of information, which could easily be responsible for much more financial loss than disruption of work.

If you get a FP, you Google it, review it as safe, and hit ignore. Worst case scenario, you have to take the file out of quarantine. If you have a missed virus on the other hand... enjoy either not knowing and losing money/privacy/identity, or trying to figure out why your processes keep messing up, in which case you'll probably just end up downloading a different AV that detects more viruses but gives more FP, when you could have just had it in the first place.

Like I said, I'd much rather have a program that catches more viruses and gives more false positives than one that has no false positives but catches less viruses, so I definitely do not agree with AV-comparatives ranking system.

AVIRA FTW

 

really? so having a vundo/zlob, that only nags you with popups is worse than having user32.dll nuked (see avg, it's its work).

uhm, not necessarily. there are users who will never see an actual alarm from their AV (they aren't going to nasty places and are plain lucky), but a false alarm on a clean file they can get.

see avg user32.dll, see kaspersky when it detected explorer.exe i think, see norton when it nuked the nt kernel exe on chinese pc's. bitdefender detected winlogon.exe, and that was just last month. and that does render your pc useless. (if not, i dare you, delete winlogon.exe from everywhere, including the sfc cache, or the nt kernel exe and see if your pc will boot afterwards)

yes, expect - most AVs today have some sort of silent mode that removes malware automatically, plus the average user doesn't know what user32.dll is, it's just another 6 character file with a 3 letter extension.

good luck when windows doesn't boot.

 

I think you're confused. Viruses =/= spyware/adware. Regardless, I don't know how you try to use the argument that anything "is worse than having user32.dll nuked" as if viruses don't try to "nuke" (lol) files like user32.dll all the time...

Now your argument against AV's that give false alarms is that some people will never see an actual alarm, and ergo, some people will never need AV's. Do you see the contradiction here? You've just proven that your argument is only valid by removing viruses completely from the picture. Incidentally, that also invalidates your argument because if there's no viruses, there's no AV software necessary in the first place.

1) Like I said, it's probably happened before, and because of the extraordinary circumstance, you heard about it. However, is it likely? No.

2) Like I said, in the unlikely event that you get a false positive, and then in the unlikely event that that FP is a system file (just because it happened before doesn't make it likely), it will be unlikely that you will not be able to remove the file from the quarantine.

3) I use Avira, because I know what is a good antivirus and what isn't. Feel free to take a look at the list of FP's that Avira detected in the comparative. Do you see system files?

4) I wouldn't delete winlogon.exe because I'm not stupid.

5) If I deleted winlogon.exe, I would still be able to boot, because I'm not stupid. (Get usb, plug into working computer, download winlogon.exe, plug usb into non-working computer, boot to command prompt, copy winlogon.exe to respective directories).

What AV program does that by default? Show me an AV that will by default delete files without even asking you, let alone alerting you. In exchange, I'll do you a huge favor and give you a link to a real AV program. The only way an AV is going to delete files without asking you is if you set it to do that, and then it's your own fault if it deletes something you didn't want it to.

Lol. Again, Windows doesn't boot probably a few thousand times more often as a result of viruses than it does a result of FP's. You're only helping my argument when you say things like that.

Regardless, there's a little thing called safemode. Of course, many viruses are designed to run even in Safe Mode .

 

i think you're confused too, viruses infect. malware today don't try to nuke anything - apart from AVs. There's no profit, no passwords to steal, no botnet, spam server creation possible, no stuff to get you to buy etc. i think about 0.05% of modern malware try to delete your system files.

Me neither, but i reached the point where AVs are uselss to me because I'm not stupid, I don't even run an AV, a software firewall, or anyother security software, only sometimes, on vmware, for fun (like the preatty interfaces of all products trying to be more intuitive and failing miserably - even AVIRA).
But we're not the average joe which comes home, starts the pc he bought from some retailer, goes into windows media player, internet explorer, maybe a game etc. they don't even know what .exe is because by default windows hides extensions.
(and command prompt, what's that?, not to mention the task of creating a bootable usb stick that reads ntfs partitions). if you want to act smart, i'll give you a simpler way, boot up to the recovery console and expand the deleted file from the xp cd (on vista, i think you can run sfc from somewhere on the dvd), but again, the twit that just had winlogon erased doesn't know how to do that.

and this might come as a shock to you, but in corporations, workstations have automatic policies set up, the administrator sets them up and the people who work at that workstation have no business going into the AV, and these policies often include automatic handling of infections.

norton has it since a long time, i know eset has it and i think kaspersky too. plus others have "game modes" that do exactly that, handle stuff automatically while in game. that's all the AVs i came in contact recently that have it (and I think others have it too). they do alert you, after it's done, and it flashes for 4-5 seconds, i don't think you look at the lower right corner of your screen constantly (or if you're playing a game then it won't show up)

Really, a couple of weeks ago avira deleted a ton of executables because of some problems with a packer. even the more experienced user would think at that, more exe's, it can't be a FP, those are usually isolated, it must be a virus infection i got somewhere. let's reformat (best way to get rid of a file infecting malware since curing in some cases leads to corrupted executables)

ps, another one, eset detected dllhost.exe
The fact that you think that all these AVs are pitiful, shows that you're just an blinded by avira and can't see the forrest for the trees. Don't get me wrong, i am not saying it's cr&p, but it's highly overrated (and AVIRA's technology is cr&p, when you start having a common ground with those aspects, give me a call )
maybe avc should change the rules, avira can have 100000000 fps and not get any penalization because their kewl, but each fp should decrease other AVs scores by 10%.

 

And this is your argument for lower detection rate and lower FP's versus higher detection rate and higher FPs? Talking about malware intentionally destroying system files?

Who said anything about a bootable USB? All you have to do is hit F8 at startup and boot to safe mode with command prompt. It's not like it can't see your USB stick with the winlogon.exe on it from there... And not everyone has a recovery CD. Lots of OEM's still use boot flash recovery partitions.

You still have the same problem, you're starting to blame the problem on human error. "Twits that erased winlogon" or people missing the virus notification in the lower right of the screen. That's human error, not reason to be afraid of FP. Those are mistakes you need to learn not to make, FPs or not.

It's no shock to me that businesses will do this. However, the default action of any modern top of the line AV is to "deny access", not delete. The second level default action is to quarantine, not to delete. And this is all assuming the unlikely event that an FP will occur, and the extraordinarily unlikely event inside of that already unlikely event that the FP will be a system file, which I keep pointing out, and you keep conveniently ignoring.

Go back and reread, I edited it (before you even replied). I pointed out that no AV even does that by default - deletes files. It's only going to do that if you tell it to, in which case you shouldn't be crying when a file get's deleted that you didn't want to, and you don't even know about it.


One more thing.... please stop picking at small parts of my post. If you're going to argue it, please go back and argue the other points that I made as well. Such things as :

to start. (And I'm not saying I just want you to reply to this. Please go back and actually reply to each point I made so I don't keep racking them in here uncontested).

 

92, 93, 98% is not a lower detection rate, it also doesn't mean a thing as those malware are already a couple of months old, they are not recent, some won't actually work - eg trojan downloaders, the file it should download isn't there anymore - suspended account, guy with hacked webserver woke up etc.

that was just an example, fp's can be more serious (norton - nt kernel), does your windows run without the nt kernel - maybe it has a linux kernel hidden somewhere and just uses it as a backup

quarantine = rename file or move file (aka copy and delete), possibly encrypt. guess what, windows won't know that svchost.exe is now svchost.ex0 and use it instead. otherwise quarantine is completely useless (what exactly does it do if the file remains intact and isn't isolated in anyway)

well, that's more something that scares them. a lot of people wouldn't need an av, it will never detect anything, but they just think that it does a good job of keeping stuff out. it's more like common sense for the home user (something he heard from a friend etc): every pc needs an AV and a firewall period.

yes they do...they quarantine (aka delete and backup) files automatically, so the user doesn't have to worry. also the n00b config suggested even by the tech support of the AV company is to set it too "disinfect and quarantine/delete if it fails", so the user doesn't have to worry about a thing (except FPs of course - which he probably thinks are frames per second and wonders what that has to do with anything since crysis runs perfect on his pc)

ps, avira windows component FP, less than a month ago: advapi32.dll

 

92 isn't less than 93 which isn't less than 98? I didn't get it, is that what you're trying to say? You just diverted from talking about destructive malware as your argument for lowerFP/higher detection, to now pointing to the fact that some malware is old? I don't get where you're going with this, but these are all points for my argument, not yours... are you pointing to the unimportance of the comparative since these malware or old or something? I don't understand how you think you're helping your argument here...

You realized I was right about how easy it is to replace winlogon after your dangerous dare, an as a result you point to kernel? It's even easier to fix that. The point is... if you know to use google, it's not hard to fix a system that had a file misplace/deleted. It's a hundred times harder to fix a system that has been ravaged by a virus.

Quarantine does isolate the file, that's pretty much the definition of quarantine. I don't see your point here... are you trying to say that in the instance that svchost.exe returned an FP and was for some reason renamed (that's not how it works btw, you should look up how most AV's work), that windows wouldn't know what to do? What have we been going over for the past few posts?

You're digressing. I made that point about FP's. At first, you could only make an argument for your case by unrealistically removing viruses from the equation all together. Now you're just removing viruses from the equation, and not even talking about why FP's are as bad as viruses, but are just talking about your opinion that every pc needs an AV... but what of the fact in relation to FP's? Like I said, it was a huge contradiction.

You and I both know, even if you're trying to change your argument without me noticing, that you argued that AV's delete files without asking. That's a lot different from quarantine, which simply moves the file. Regardless, I think you're assuming, and even hoping, for the sake of your argument, that "most" AV's do this without asking, which is just not true. Even though the point is moot, since we're talking about higher detection rate and higher FP (and I know that the AV's with higher detection rates don't do what you're saying, which is why the point is moot), I would still like a list of AV's that by default quarantine without asking you, so that you can prove "most" AV's do, like you said.

Thank you, this has been my point all along. 1) Avira did not auto-delete or quarantine the file liek you're absurdly suggesting that "most" AV's do. 2) It took a single forum thread to find that it was a FP. 3) No harm was done, versus a missed virus, which can be devastating.

This is exactly what I suggested should and does happen, and the very reason that high detection rate with high FP is a lot better than lower detection rate with lower FP.


And you're still not replying to most of my points. Please go back and tell me why those arguments are wrong instead of picking at select parts. I sectionally quote your entire post and tell you why I agree or disagree...

 

it seems that everyone knows what advapi32.dll is (just like everyone knows how to use the command line), and won't think it's some new malware, since everyone knows every dll in the windows by the heart (and they can even recite them).


I'm done arguing with you, I made my point, you're beyond help like all avira fan boys. fine avira is the greatest and all other AV sucks, it should get A+++++++++++++++++++++:SLEEP:

 

Funny that I can ruin this entire sarcastic remark by saying one thing: Everyone knows how to use Google.

Lol, because you've gotten into an argument you can't win with an Avira user, there's suddenly a new group of "Avira fanboys" amidst. It's like saying... 'oh, you think mustard is better than mayonnaise? shutup then, you mustard fanboy!' See how ridiculous you sound?

Regardless, my point was not that Avira should be ranked higher, it is that FP's shouldn't be held in the same esteem as detection rate, let alone higher. Had nothing to do with specific AV's, even if I used Avira as an example because I use it. I'm sorry if you thought that I was attacking you or got sour for some other personal reason, my intention was to have a civil debate about it...