Today I was reading a technical papper and dicovered that some virus have funny particularity, they are mutable (their MD5 can change automatically from time to time). This virus are called oligomorphic viruses. This make me wondering if and how an anti-virus could detect this kind of malware and how can I prevent my system from one of this kind of infection. Did someone got any kind of experience with this kind of viruses?
Oligomorphic and polymorphic software (Oligomorphy is a type of polymorphy, if I remember my classes right) can be detected heuristically by using behavioral analysis. Essentially, you can determine whether or not a piece of software is malicious by looking not at what it is, but rather what it does. You may have seen your firewall prompt you whether or not to allow a program to connect to the internet, that's an example of simple behavioral control of software. Most anti-virus programs have a heuristic engine that will observe the software on your computer and look for programs that do suspicious, virus-like things like:
- attempting to read or write memory allocated to other programs
- attempting to read or write files belonging to other programs
- opening spurious network connections, especially using protocols like IRC that are commonly used by botnets for C&C, or SMTP, commonly used by botnets to send spam
- taking actions to hide itself (renaming processes and files frequently, disabling the task manager, running under the same name as other known programs)
Heuristic detection is imperfect, though, because it essentially involves taking an educated guess (something that computers are bad at). So, there's a constant balance between missing viruses and bothering the user too often (a high false positive rate). Good antivirus programs will allow you to adjust the sensitivity of the heuristic engine.
Do you mean polymorphic?
Anti-virus programs that utilize heuristics, or are "behavior-based" are what can be done against these.