After a hiatus of several years where I was using Apple Mail I've gone back to using Thunderbird for my mail. I'm generally pleased with Thunderbird. I like that it works on all OSes (I use several). I like that it can handle multiple accounts (I have several). I like that I find it intuitive. I want to keep using it.
However, I'm having some serious trouble getting the junk mail filters to actually work. This is on Ubuntu with Thunderbird 2.0.0.22, and MacOS 10.4 with Thunderbird 2.0.0.16. There's more than one account. There's also more than one Thunderbird instance but they are sharing Junk folders over IMAP (only way I know to train both of them).
Everything I've ever read suggests that after training on 3000-4000 messages most spam should be caught (barring waves of new varieties which come along periodically). That's been my experience in the past. At first, mostly untrained, I was getting between 20 and 200 messages a day. This didn't slow down much after a couple of days. I have a fairly large body of existing spam, some 20000 messages, and I added it all at once. That improved the true positive rate considerably, though I was still getting a handful of spam messages a day. Frustratingly many of these seemed to be the same message, and Thunderbird seems to have a great deal of trouble recognizing some of these. (I looked at a few to see if they are doing the normal tricks that give filters a hard time: images instead of text and/or random paragraphs of "known good text". In one case that was true but in others the messages appear to be short and mostly empty. No images or embedded text.) And now, in the past week or so, the rate has shot back up again to a dozen or more an hour. It's as if the filtering has just stopped.
Basic procedure and obvious gotchas, which I've already done:
- Enabling junk mail filtering in Thunderbird requires two separate settings:
- In the
Options
(akaPreferences)
selectPrivacy | Junk
and enabled 'When I mark messages as junk`. You then choose to move messages to a Junk folder or delete them. I always do the former to prevent losing mail. - Under
Account Settings
for the account selectJunk Settings
enableEnable adaptive junk mail controls for this account
.
- In the
I know these work because junk mail is being filtered. Just poorly.
Things I've tried:
- I've tried looking at the file
training.dat
. I've peered inside it with things likeod
andstrings
and it is mostly embedded strings. But the format is basically opaque. It does grow, but noticeably it does not grow every time I mark a message as junk. - I've turned on Junk Filter Logging (
Preferences | Privacy | Enable junk filter logging
). This does nothing. There is no log. I seem to remember trying this years ago and it didn't do anything then either. - I've considered tossing
training.dat
and starting over, but (a) it is changing (b) I don't really want to start over and (c) there's no way to mix training files, so you really can't go back to an old file without losing whatever you've gained.
So, questions:
- How can I tell if training is actually happening?
- Should I expect
training.dat
to change every time I mark something as junk? - Why don't the logs appear?
- Do I have to mark each message individually, or can I mark a whole bunch of messages at once?
- Are there any tools for finding out when the filter runs and what it is doing when it does?
- Are there any tools for decoding the training file?
- Overall, why does it seem to have stopped working, and what can I do about it?
There's lots of links to Thunderbird junk mail filtering on Google, but they almost all boil down to basic tutorials. I am looking for more than just basic instructions: I want to know how to debug or diagnose how the filter is working -- or not working.
Update: I wasn't clear about this originally but I have successfully used Thunderbird in the past, for a period of many years. I stopped for a few years because I was using Apple Mail. So it's not a problem of not knowing the basics or not getting any filtering. Filtering is happening just sporadically and poorly.