12

After a hiatus of several years where I was using Apple Mail I've gone back to using Thunderbird for my mail. I'm generally pleased with Thunderbird. I like that it works on all OSes (I use several). I like that it can handle multiple accounts (I have several). I like that I find it intuitive. I want to keep using it.

However, I'm having some serious trouble getting the junk mail filters to actually work. This is on Ubuntu with Thunderbird 2.0.0.22, and MacOS 10.4 with Thunderbird 2.0.0.16. There's more than one account. There's also more than one Thunderbird instance but they are sharing Junk folders over IMAP (only way I know to train both of them).

Everything I've ever read suggests that after training on 3000-4000 messages most spam should be caught (barring waves of new varieties which come along periodically). That's been my experience in the past. At first, mostly untrained, I was getting between 20 and 200 messages a day. This didn't slow down much after a couple of days. I have a fairly large body of existing spam, some 20000 messages, and I added it all at once. That improved the true positive rate considerably, though I was still getting a handful of spam messages a day. Frustratingly many of these seemed to be the same message, and Thunderbird seems to have a great deal of trouble recognizing some of these. (I looked at a few to see if they are doing the normal tricks that give filters a hard time: images instead of text and/or random paragraphs of "known good text". In one case that was true but in others the messages appear to be short and mostly empty. No images or embedded text.) And now, in the past week or so, the rate has shot back up again to a dozen or more an hour. It's as if the filtering has just stopped.

Basic procedure and obvious gotchas, which I've already done:

  • Enabling junk mail filtering in Thunderbird requires two separate settings:
    • In the Options (aka Preferences) select Privacy | Junk and enabled 'When I mark messages as junk`. You then choose to move messages to a Junk folder or delete them. I always do the former to prevent losing mail.
    • Under Account Settings for the account select Junk Settings enable Enable adaptive junk mail controls for this account.

I know these work because junk mail is being filtered. Just poorly.

Things I've tried:

  • I've tried looking at the file training.dat. I've peered inside it with things like od and strings and it is mostly embedded strings. But the format is basically opaque. It does grow, but noticeably it does not grow every time I mark a message as junk.
  • I've turned on Junk Filter Logging (Preferences | Privacy | Enable junk filter logging). This does nothing. There is no log. I seem to remember trying this years ago and it didn't do anything then either.
  • I've considered tossing training.dat and starting over, but (a) it is changing (b) I don't really want to start over and (c) there's no way to mix training files, so you really can't go back to an old file without losing whatever you've gained.

So, questions:

  • How can I tell if training is actually happening?
  • Should I expect training.dat to change every time I mark something as junk?
  • Why don't the logs appear?
  • Do I have to mark each message individually, or can I mark a whole bunch of messages at once?
  • Are there any tools for finding out when the filter runs and what it is doing when it does?
  • Are there any tools for decoding the training file?
  • Overall, why does it seem to have stopped working, and what can I do about it?

There's lots of links to Thunderbird junk mail filtering on Google, but they almost all boil down to basic tutorials. I am looking for more than just basic instructions: I want to know how to debug or diagnose how the filter is working -- or not working.

Update: I wasn't clear about this originally but I have successfully used Thunderbird in the past, for a period of many years. I stopped for a few years because I was using Apple Mail. So it's not a problem of not knowing the basics or not getting any filtering. Filtering is happening just sporadically and poorly.

4
  • Unfortunately for us the question seems to be something the pool of people available for the beta don't know the answer to :/. If anyone does know the answer they tend reply pretty fast.
    – quark
    Aug 11, 2009 at 15:05
  • this post might help
    – user4100
    Sep 3, 2009 at 5:47
  • I had this problem and upgrading to Thunderbird 3 fixed it. Install Junquilla to make it even better.
    – endolith
    Jul 25, 2010 at 22:46
  • I feel like it worked great when I had only marked a few hundred messages as spam, but after marking thousands as spam and thousands as good, it works very poorly. actual spam is marked anywhere from 10% to 99% likely, even when I've tagged very similar messages the day before.
    – endolith
    Oct 22, 2014 at 15:08

8 Answers 8

4

You may be interested in the JunQuilla extension for Thunderbird. It requires Thunderbird 3.0 or later.

It has the ability to show the bayesian junk rating of your messages as a percentage (among other things).

Info here: http://mesquilla.com/extensions/junquilla/

2

I assume you've seen this article: Junk Mail Controls. There's also a link to this somewhat old article: Junk messages not moved. This forum thread may prove helpful to you as well: Enable junk filter logging.

Good luck!

3
  • 1
    "Junk Mail Controls": Correct, I've read it, several times. It's the first hit for "Thunderbird Junk Mail". It has some things I've not tried like using SpamAssassin (I'd prefer to use the builtin junk filtering), and setting the threshold level for "spam". This is treacherous, but I could try it.
    – quark
    Aug 29, 2009 at 8:07
  • "Junk messages not moved". I believe this is out-of-date (version 0.9 was released in 2004). It's interesting, but the problem isn't that files aren't moved: the problem is they aren't recognized as spam. When they are recognized they are usually (but not always) also moved to the Junk folder.
    – quark
    Aug 29, 2009 at 8:08
  • "Enable junk filter logging." As far as logging goes what I describe above is what they suggest. It just doesn't turn on. As far as "in both places" goes: already doing that, but there's no reason you'd know that. I'll edit the question to note that.
    – quark
    Aug 29, 2009 at 8:09
1

See http://kb.mozillazine.org/Junk_Mail_Controls#Tweaking_the_Junk_Mail_Controls

In Thunderbird:
You can change the preference using Tools -> Options -> Advanced -> General -> Config Editor. Enter "junk" in the Filter field to show only the preferences that contain "junk" in their name, and then double-click on mail.adaptivefilters.junk_threshold, enter a value lower than the default 90 in the edit field and press the OK button. Many users report good results with values of 30 or lower.

0

I don't use the junk filter at all. Instead I use MailWasher to intercept the mail before it gets to my mailbox. I don't need to train it, since I simply set up a filter that defines everything as junk, and I add as friends only legitimate addresses.
It's not free, but the trial version used to never expire. I wouldn't know about the current version, since I paid for MailWasherPro (and it was worth it).

1
  • @ITSnuggles: Thanks. If you liked this answer, this one might also be useful.
    – harrymc
    Jan 7, 2018 at 7:50
0

The junk mail controls, at the start, are going to work poorly because it doesn't know what your junk mail looks like. I don't know what Thunderbird has set up for training data at the very beginning.

When you mark mail as junk by clicking on the junk icon, or using one of the menus to Mark as Junk, it will improve the data in the training database. It will get better over time. j is the keyboard shortcut for marking the selected messages as junk.

Also, something that many people don't realize is that you can accelerate the learning process by selecting a large chunk of good emails and then making them as Not Junk. When I see too many junk emails getting missed, and others junked as false positives, I will be sure to select a few hundred of my recent mails and Shift-j, which is the keyboard shortcut for "Mark as Not Junk".

See also Home->Thunderbird->Tips and tricks->Thunderbird and Junk / Spam Messages

-1

If you have webmail that is forwarding to your account, at the very least you could set up filters there to take some of the spam out of your inbox.

1
  • If I had webmail on any of the major providers it would already be filtered. I wouldn't expect to need to do much filtering locally at all. Unfortunately I don't, and I don't want to switch where I'm getting my mail. Thunderbird needs to actually work as advertised.
    – quark
    Aug 20, 2009 at 21:02
-1

Which plugin are you using, spamassassin or bogo filter? Spamassassin should work pretty robust if you have that much spam training!

1
  • 3
    I'm not using a plugin. I'm using Thunderbird's built-in Junk Mail controls. You turn them on by going to Preferences | Privacy | Junk and selecting 'When I mark messages as junk'.
    – quark
    Aug 29, 2009 at 7:19
-1

I would use spam assassin plugin for Tbird, or one of the other spam detectors, and turn off the automatic learning. you can also block it at the email server level (by that i mean by using the gmail,etc web interface to mark it as spam)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .