[FQuest Announce] DomainKeys signing

[Bob]

What: DomainKeys signing of outgoing mail
When: Now
Who: All mail users

Spammers are increasingly "spoofing" the senders of emails, by forging email addresses on domains that they do not own or control. :blah: You may have received notifications of delivery failure for emails that you never sent for addresses on your domain that you never use. This is an example of spammers spoofing your domain as the sender of their emails. :rollpin:

Several models for signing or verifying the senders of messages have been proposed, including SPF, DomainKeys, Sender ID and the like. The idea behind these signing systems, is that the receiving mail server will be able to know whether or not the email originated from the domain owner, or is forged. :sprint:


:QTFQuest:
FutureQuest is pleased to be able to make DomainKeys signing available to all domains using our mail servers! This feature is available to you now at no extra cost. :)

The mail system on all community servers has been enhanced to add the capability to add DomainKeys signatures to outgoing messages. This should also help to resolve issues some site owners were having with mail deliveries to Yahoo, as they have stated validly signed messages will be treated preferentially.

Site owners can enable this feature by going to the Email Manager in their CNC and clicking on the "Enable/Disable DomainKeys" link. This will provide a list of domains managed by the CNC for which DomainKeys may be enabled.

Note: DomainKeys is Not enabled by default.

The DomainKeys signing system in use requires that

the SMTP session is authenticated (either with SMTP AUTH or POP-before-SMTP)
the domain in the From: header must match one of the primary or overlay domains for the authenticated domain, and
the domain in the From: header must have DomainKeys enabled.
If all three of these conditions are met, the system will automatically sign the message with no need for further action from the sender.

DomainKeys is not yet available for use with QuestMail or mailing lists sent from ezmlm-idx however work is continuing to add these areas to the DomainKeys signing process. You must use a local Email Client, such as Outlook/Outlook Express, Eudora, Thunderbird or other similar email client to have DomainKeys sign your email.

DomainKeys is available regardless of which SMTP Port you use.

--
The FutureQuest Team
www.FutureQuest.net

:QTFQuest:

[chernove]

Thanks for this. I suspect this was a problem that was vexing a great number of us.

Once again FQ comes through!:yeah: No surprises there.

I look forward to trying it out.

-Eric

[Judy Doherty]

Would you be so kind as to give us the pros and cons of using this system?
I am not sure what you mean by "signing" - I don't always follow the techspeak and would like to learn more.
THANKS!

[Velimir]

Thanks FQ! I know it was covered few times, but what are chances of bringing SPF in future? Did they do any work to make it more reliable?

·velimir

edit: any reason that I have two times veljco.com and two times veljco.net listed?

edit2: I have reformulated first question to better express what I ment.

[Monty]

Will vB forum based emails outgoing emails be signed as well if this is enabled on the domain they come from?

[Armand]

Sounds like a great idea though so makes me wonder if there's strings attached to this. So I'm with Judy in at least wondering if there are any cons or downsides to doing this?

On top of that... can we get a bit more about this part:

the SMTP session is authenticated (either with SMTP AUTH or POP-before-SMTP)

For instance I use Outlook Express, does having 'server requires authentication' cover that or does the 'log on with secure password authentication' have to also be checked/used. Or is there anything else that needs to be specifically done on our end in such programs?

Oh and what Monty asked also...lol

[tappel]

Outstanding! I've absolutely flooded with this bounced garbage lately.

:clapper:

Tom

[Bruce]

Would you be so kind as to give us the pros and cons of using this system?The only downside to the system that I can think of is that when setting a domain to "sign all", all mail must go through our servers to be signed. Anything else will be considered unauthentic. This is, of course, exactly how it's supposed to work, for good and for bad.

Technically, another downside is that DomainKeys uses some CPU time to compute the signatures when sending mail and to verify the signatures when receivers validate those signatures. On today's CPUs, however, this CPU time is rather small, especially when compared to all the other costs involved. It also adds about 300 bytes to the header of each message (dependant on the size of the key) but, again, compared to the size of the message this is pretty small.

I am not sure what you mean by "signing" - I don't always follow the techspeak and would like to learn more.A digital signature is a cryptographic mechanism by which a sender can add a piece of data to a message that can only be generated by the sender but can be verified by anybody. This involves a pair of keys, one public and one secret. The public key is known to the recipient and is freely distributed to all who need to know it. The secret key is known only to the sender, and is required to generate the signature.

More information about DomainKeys can be found on Yahoo's page (http://antispam.yahoo.com/domainkeys) and in the Wikipedia article (http://en.wikipedia.org/wiki/Domainkeys).

what are chances of bringing SPF in future?We have no plans to add SPF to our mail system.Did they do any work to make it more reliable?I doubt it. The SPF system has a number of fundamental implications that require invasive changes to mail transports to resolve. There is simply no way to do authentication in the way SPF specifies without those problems.

Will vB forum based emails outgoing emails be signed as well if this is enabled on the domain they come from?Not yet, no. If you have such a forum, you will want to only set the domain to "sign some". We are continuing to develop a solution to this issue.

I use Outlook Express, does having 'server requires authentication' cover that or does the 'log on with secure password authentication' have to also be checked/used. Or is there anything else that needs to be specifically done on our end in such programs?Effectively, if you are able to relay mail through our servers to senders outside of your domain, you are already doing all that needs to be done in regards to authentication.

[Bob]

Outstanding! I've absolutely flooded with this bounced garbage lately.

:clapper:

Tom

Tom,

This will not stop someone from Forging your domain in the From: address in spam emails and it will not stop bounces from spam messages from being sent to your domain when forged in this manner.

The best way to reduce the number of bounces you receive when a spammer does Forge your domain in the From: address of an email is to ensure your Catch-All is turned off.
http://Service.FutureQuest.net/kb267

-Bob

[Monty]

Not yet, no. If you have such a forum, you will want to only set the domain to "sign some". We are continuing to develop a solution to this issue.


2nd dumb question for the day...If I have it set to sign all, what are the downfalls?

[jeep]

This is really great, thanks FQ!

Just to clarify, if a server script (PHP or whatever) sends an email, it will not be DomainKey signed unless sent via smtp.mydomainname.com?

[Bob]

Monty,

If you select "Sign All" then any mail sent from your domain not signed would be considered unauthenticated and potentially distrusted by any network using DomainKeys to check inbound mail.

This would include sending an email using one domain as From but through another mail server, such as sending from your ISP but using a From address containing your domain name. At this time it would also include VB generated mail and Questmail generated mail.

If every message sent from a domain are all sent using SMTP through that domain's mail server then "Sign All" would be appropriate.

Hope this helps clarify,
Bob

[KAT]

:bow: Thank you SO MUCH. I'm setting to "sign everthing". It's always seemed unfair to get so many notices of emails sent from my domain not going through, when I NEVER send anything from my domain name because my ISP doesn't allow it.

Sure hope this cuts down on spammers using our domains for their own imoral/illeagl purposes.

[phppete]

Hi FQ,

So just to be clear, I would select 'sign some' for a domain where I use sale@mydomain.com via Outlook or OE but also where sales@mydomain.com is the address I use when sending mail from PHP scripts.

Does this mean that mail sent via PHP (or other similar languages) will NOT be affected by using 'sign some'. Would there be any situation where using 'sign some' would not be advised?

With the above domain example, automated order confirmation emails are actually more 'mission critical' than when we send email via OE or OL.

Thank you

Pete

[Bob]

Pete,

In that case, at this time, Yes "Sign some" would be most appropriate.

I believe the issue is if you use "Sign all" then you are advertising to the world that any email sent From: your domain must be signed or they may consider it invalid or not to be trusted.

I would think that "Sign some" will be a valid and appropriate setting for some time to come...

-Bob

[lynxtrax]

Would you be so kind as to give us the pros and cons of using this system?
I am not sure what you mean by "signing" - I don't always follow the techspeak and would like to learn more.
THANKS!

wikipedia (http://en.wikipedia.org/wiki/DomainKeys) has a pretty good article

[phppete]

Pete,

In that case, at this time, Yes "Sign some" would be most appropriate.

I believe the issue is if you use "Sign all" then you are advertising to the world that any email sent From: your domain must be signed or they may consider it invalid or not to be trusted.

I would think that "Sign some" will be a valid and appropriate setting for some time to come...

-Bob

Thank you for the quick reply Bob, much appreciated :)

[Juan G]

Will vB forum based emails outgoing emails be signed as well if this is enabled on the domain they come from?
Not yet, no. If you have such a forum, you will want to only set the domain to "sign some". We are continuing to develop a solution to this issue.
For forums, etc., I think FQuest uses qmail as a sendmail wrapper. According to Yahoo's page on DomainKeys (http://antispam.yahoo.com/domainkeys), sendmail, qmail and other MTAs support DomainKeys. Indeed, at the DomainKeys (http://domainkeys.sourceforge.net/) site there is a link about "How to set up Qmail to use DomainKeys" (http://jeremy.kister.net/howto/dk.html), using Russ Nelson's qmail-dk (http://www.qmail.org/top.html#200410180) patch. So, it seems doable. Thanks to the FQuest team for working on this.

[Bruce]

According to Yahoo's page on DomainKeys (http://antispam.yahoo.com/domainkeys), sendmail, qmail and other MTAs support DomainKeys.To be clear, there is support for DomainKeys that can be added to qmail. qmail itself has not been updated by the original author for years, and does not include support for DomainKeys in the base package.

Indeed, at the DomainKeys (http://domainkeys.sourceforge.net/) site there is a link about "How to set up Qmail to use DomainKeys" (http://jeremy.kister.net/howto/dk.html), using Russ Nelson's qmail-dk (http://www.qmail.org/top.html#200410180) patch. So, it seems doable.We will be pursuing a similar approach to the qmail-dk patch, but with a different mechanism that takes into account the custom domain structures in place at FutureQuest.

[Josh]

My e-mail accounts are set up such that I have aliases that forward messages to my GMail account. I send messages from my GMail account, but I set the "From" box to one of the alias addresses at my domain. Would setting DomainKeys to "All" cause a problem for me?

[Bob]

Josh,

We do not recommend setting DomainKeys as "Sign All" unless you send all email through the FutureQuest mail servers via SMTP.

Whether it would cause problems would depend on the receiving network and their filtering policies, we can't speak for other networks...

Note that if Gmail ever implemented DomainKeys, then sending using a From: address other then a Gmail address may cause your mail not to be signed, however again this would depend on how Gmail implemented any signing...

-Bob

[songdog]

I just went to my CNC to enable signing, and it raised a couple questions along with an enhancement request:

1. How quickly do changes to the signing mode on a domain show up in that domain's DNS entries?

2. Will FutureQuest be implementing any processes to validate *incoming* email using DomainKeys? For example, maybe you could enhance the mail-handling processes so they subtract 5 or 10 points from the SpamAssassin score of messages having valid DomainKeys.

And finally, an enhancement request: it would be nice if you would show the signing mode for each & every domain on the main Email Manager page, similar to how you show icons for individual mailboxes/aliases indicating the existence of autoresponders and various filters for each address? We would still go to the new DomainKeys page to *change* the signing mode for our domain(s), but we could see the current settings right away. As the # of domains on an account is almost always less than the # of mailboxes/aliases on the account, the added length to the main Email Manager page would be relatively inconsequential.

Thanks! :yeah:

[Juan G]

Site owners can enable this feature by going to the Email Manager in their CNC and clicking on the "Enable/Disable DomainKeys" link. This will provide a list of domains managed by the CNC for which DomainKeys may be enabled.

Note: DomainKeys is Not enabled by default.

The DomainKeys signing system in use requires that

the SMTP session is authenticated (either with SMTP AUTH or POP-before-SMTP)
the domain in the From: header must match one of the primary or overlay domains for the authenticated domain, and
the domain in the From: header must have DomainKeys enabled.
If all three of these conditions are met, the system will automatically sign the message with no need for further action from the sender.

It seems the CNC setting takes some time to go into effect, but now it's working: I can see the DomainKey-Signature header on the test mails. Thank you!

[jelevin]

How would I tell whether an email is signed? Is this something which will eventually be done in my mail reader (thunderbird) or by my pop mail server?

Thanks!

[Bob]

1. How quickly do changes to the signing mode on a domain show up in that domain's DNS entries?

It should be within 60 to 90 seconds however if you should send a message before the time frame is up there is a possibility of a cache situation in which the DNS standards are not clear on how long they will be cached.


2. Will FutureQuest be implementing any processes to validate *incoming* email using DomainKeys? For example, maybe you could enhance the mail-handling processes so they subtract 5 or 10 points from the SpamAssassin score of messages having valid DomainKeys.
At this time there is not any immediate plans to use DomainKeys for validation of incoming email however I am sure that would be something we would look at down the road.

As far as implementing within SA that would have to be an enhancement that the Developers of SA would have to make available.


And finally, an enhancement request: it would be nice if you would show the signing mode for each & every domain on the main Email Manager page, similar to how you show icons for individual mailboxes/aliases indicating the existence of autoresponders and various filters for each address? We would still go to the new DomainKeys page to *change* the signing mode for our domain(s), but we could see the current settings right away. As the # of domains on an account is almost always less than the # of mailboxes/aliases on the account, the added length to the main Email Manager page would be relatively inconsequential.
I will certainly make sure we look at this possibility however at a glance I would think it would take up too much space for many accounts as they may have up to 15 additional domains associated with a main domain and that would place the email account settings much further down the page.

Jelevin,

Here is a sample from a test email I sent...
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
h=X-Originating-IP:Message-ID:Date:From:User-Agent:MIME-Version:To:Subject:Content-Type:Content-Transfer-Encoding;
s=default; d=4example.net;
b=baPCQchAjzTjhgvOwsuJFjsUnYXwh/XBbHkCp0Qj1v52EacFLVGYeHwY/4QQZuAU9KdF9BIHIxtTAhzsP+5w1LQaGAfGnSAlvZ7nyXccVg/Sd7mxzPZh/MwURk3IRXYIZjrwhW1UXtN+leVP54r3Q6m2ddYQMGXzpZQNYeb2rqo=;

This is in the raw email header and would not be visible in your normal email view in an email client.

-Bob

[lenoxb]

... and it will not stop bounces from spam messages from being sent to your domain when forged in this manner.

The best way to reduce the number of bounces you receive when a spammer does Forge your domain in the From: address of an email is to ensure your Catch-All is turned off.

-Bob

I can see how this is a good thing for the greater good of the Internet. I’m trying to understand how it helps me as a domain administrator right now if I turn DomainKeys on.

Scenario:

1. The bad guy spoofs a bogus return address on my domain: asofj@mydomain.com. He can’t sign it because he doesn’t have FQ’s private key.

2. The recipient domain bounces it back to mydomain.com and it winds up in my catchall account for me to deal with. I don’t understand why it can’t be autodeleted if it’s not signed with a correct DomainKey, but you’ve said that’s not what’s going to happen.

So far it sounds just like there's no change from the way things work without turning DomainKeys on. So what’s in it for me? If you’re telling me I have to deactivate the catchall account in order to avoid the blowback problem, well fine, but I don’t need to turn on DomainKeys for that.

[Juan G]

How would I tell whether an email is signed? Is this something which will eventually be done in my mail reader (thunderbird) or by my pop mail server?

Thanks!
Thunderbird menu (to see the DomainKey-Signature header):

View -> Message Source
(or Ctrl+U)

[jelevin]

Thanks for the speedy replies. There is a thunderbird extension which verifies domains. See https://addons.mozilla.org/thunderbird/345/ This appears to work, at least for the messages from futurequest.

[tril]

Does this take awhile to activate? I set it up in CNC a few minutes ago but don't see the header in test mails.

[Juan G]

Does this take awhile to activate?
Yes, I think so.

[Tom]

A couple questions:

Are companies actively blocking messages now that fail the test?

Any guess on what percentage of receiving mail servers that analyze the info? Are the larger systems (AOL, MSN, Netzero, comcast, etc.) running checks on inbound mail?

So if a person clicks on a "mail this" link on an article somewhere (e.g., cnn.com). They enter the recipient and their return mail address and send away. If their domain is set to "sign all", there's a good chance that that message would be blocked by servers that run checks?

[Melissa]

Does this take awhile to activate? I set it up in CNC a few minutes ago but don't see the header in test mails.
It should be within 60 to 90 seconds however if you should send a message before the time frame is up there is a possibility of a cache situation in which the DNS standards are not clear on how long they will be cached.

I would suggest allowing some more time to pass since you've sent a message as that should allow the "negative" results to clear from cache.

[Bob]

I can see how this is a good thing for the greater good of the Internet. I’m trying to understand how it helps me as a domain administrator right now if I turn DomainKeys on.

Scenario:

1. The bad guy spoofs a bogus return address on my domain: asofj@mydomain.com. He can’t sign it because he doesn’t have FQ’s private key.

2. The recipient domain bounces it back to mydomain.com and it winds up in my catchall account for me to deal with. I don’t understand why it can’t be autodeleted if it’s not signed with a correct DomainKey, but you’ve said that’s not what’s going to happen.

So far it sounds just like there's no change from the way things work without turning DomainKeys on. So what’s in it for me? If you’re telling me I have to deactivate the catchall account in order to avoid the blowback problem, well fine, but I don’t need to turn on DomainKeys for that.

FutureQuest has implemented DomainKeys as a possible, and I emphasize possible, assist in ensuring email you send from your domain is accepted by other networks.

Using DomainKeys to block bounces back to your domain is not possible at this time and I personally am not sure it would ever be effective as all bounces do not contain the full original raw headers and the signature would not be available.

Turning OFF your Catch-All is still the best defense against bounces that result from Forged mailings.

While there may not be an immediate benefit to you to enable DomainKeys it is something that costs nothing, requires nothing new from users of your domain for email and the use of DomainKeys would, at this time, have to be considered something you would do for the Greater Good of the state of email as a small step towards making email trusted once again.

-Bob

[tril]

I can see how this is a good thing for the greater good of the Internet. I’m trying to understand how it helps me as a domain administrator right now if I turn DomainKeys on.

It will help you anytime a user on your domain wants to send mail to Yahoo, for starters. Yahoo has been greylisting all unsigned mail for awhile now.

[Bob]

A couple questions:

Are companies actively blocking messages now that fail the test?

Not to our knowledge however there is speculation, possibly more than speculation, that Yahoo greylists unsigned messages as tril indicated...


Any guess on what percentage of receiving mail servers that analyze the info? Are the larger systems (AOL, MSN, Netzero, comcast, etc.) running checks on inbound mail?
My understanding is AOL may use it, Yahoo does and indications Google (Gmail) as well.

Here is a quote I found today...
It's not just Yahoo that's checking e-mails for DomainKeys signing. Some of the other ISPs that already use DomainKeys to rate incoming mail are Earthlink, SBCGlobal, and British Telecom's BTInternet. AOL is widely reported to be implementing checks for both Sender ID and DomainKeys by the end of 2006.
http://itmanagement.earthweb.com/columns/executive_tech/article.php/3604761

A very interesting read, but read the entire article to get all the information...


So if a person clicks on a "mail this" link on an article somewhere (e.g., cnn.com). They enter the recipient and their return mail address and send away. If their domain is set to "sign all", there's a good chance that that message would be blocked by servers that run checks?
It all depends on how the form and script are configured and may or may not be signed which also depends on if that domain is employing any signing system and if so if it encompasses formmail...

-Bob

[tril]

The header is now showing up and my mail is arriving at Yahoo instantly! Thank you, FutureQuest. This has been a thorn in my side for awhile.

:clapper:

[lenoxb]

Using DomainKeys to block bounces back to your domain is not possible at this time and I personally am not sure it would ever be effective as all bounces do not contain the full original raw headers and the signature would not be available.

This makes sense. Will FQ reject mail from domains which mark themselves as "sign all?" If every receiving domain did this (without bouncing the mail back to the spoofed sending domain), then that would seem to be a step toward reducing blowback.

[sheila]

Will FQ reject mail from domains which mark themselves as "sign all?" If every receiving domain did this (without bouncing the mail back to the spoofed sending domain), then that would seem to be a step toward reducing blowback.
I believe Bob addressed this question already (http://www.aota.net/forums/showthread.php?postid=155715#post155715)
At this time there is not any immediate plans to use DomainKeys for validation of incoming email however I am sure that would be something we would look at down the road.

[dank]

lenoxb asked the important question (pg 3) which, in my opinion, should have been addressed in the very first sentence of this thread. For anyone not already familiar with the concept of DomainKeys, it's tough to grasp the benefit which is being sold as immediately obvious. Catch is, there isn't really any measureable benefit just yet. I think I can see how it could be a good thing long term, but I see more questions than answers when I try and think it through. Seems like a "what the hell" solution that can't hurt and might just help.

Dan

[sheila]

Just 4 posts back, Dan, there seems to me to be a measurable benefit?
(from tril)

[garyamort]

So far it sounds just like there's no change from the way things work without turning DomainKeys on. So what’s in it for me? If you’re telling me I have to deactivate the catchall account in order to avoid the blowback problem, well fine, but I don’t need to turn on DomainKeys for that.

I'll propose 2 potential benefits for you:

Scenario 1: You are supplying email addresses to customers, or worse friends and family.

One of them gets an email virus and their system starts sending mail through their mail server(authorized with their userid and password).

Without DomainKeys, you ignore all those bounces because the spammers are using your domain already and your friend, family member, or customer continues to send out spam from YOUR domain, ending up with your entire domain getting blacklisted.

With DomainKeys, you look at those bounces and notice that some of them have a FQ key in them! You get on the horn to that user right away and get him to clean up his system. You find the blacklists which have now listed you and explain that your user got hit with a virus and you have taken measures to stop him from sending out more spam.

Second scenario:
You setup an email account strictly to act as a catch all account so you don't miss any email. Using the PHP IMAP functions, you can write a script to periodically clean out this folder.

So you can clean out all messages over 30 days old, all bounces and rejection notices which do not have a DomainKey-Signature: line in the message. Anything else you want.

Then whenever you want to check that account, you first go to your website and run your script, removing any email programatically that you can determine is garbage. Than you open it.

(obviously, a bogus DomainKey-Signature: line would defeat this simple filter, but my thought is to cut down on on the bad email while not letting any good email get lost)

[Jeff]

(obviously, a bogus DomainKey-Signature: line would defeat this simple filter, but my thought is to cut down on on the bad email while not letting any good email get lost)
Sorry for not being up to speed on DomainKeys, but I take it the idea is that the DomainKey can't be spoofed as easily as the other headers -- you can somehow compare the domainkey in the email to the one generated by the FutureQuest server and verify it is the real key and not a spoofed one?

[garyamort]

Sorry for not being up to speed on DomainKeys, but I take it the idea is that the DomainKey can't be spoofed as easily as the other headers -- you can somehow compare the domainkey in the email to the one generated by the FutureQuest server and verify it is the real key and not a spoofed one?

My brief 15 minute look at the subject is as follows:
A mail message has 2 components, a header and a body. The body has your data. The header has the routing and message data.

The DomainKey is a signature, made with a public/private key pair - and the public key is stored in a DNS entry.
The signature is calculated based on the content of the body of the message, and stored in the header.

So, when the mail reaches it's destination, the destination server can check the DomainKey against the message contents in order to ensure it really came through the domain which signed it(note: there is some fudge factor involved to allow for mailling lists that append signatures to all email messages going through them). If they don't match, the server can then do whatever it does with such email(place it in a low priority mail queue for processing so if spam charectoristics are discovered before the mail is delivered it can be junked, reject the email, pass the email along without comment, note it isn't verified in the subject - whatever).

However, what happens when the email is rejected by the server(user's mailbox is full, user doesn't exist, etc)?

Well, then the server sends a rejection notice consisting of their own header, plus a body which may contain the entire rejected message, only the body of the message, only the header, the message as an attachment, etc.

At this point, the mail is most likekly changed(new content added) such that the domain key is useless for programatic use. It's no longer in the header, and it may no longer match the contents.

So, when your server gets the rejection notice, the best you can do with a quick and dirty approach is check to verify that for bounced email, there is a domainkey in the body of the bounce.

Now, I'm sure given time programmers much smarter than me who understand regular expressions will figure out filters that can extract the original header and body of the email message from the bounce, and then programatically check DNS and do the cryptographic comparison of the data.

And then someone like me can come along and put the little pieces together to build a filter that does all of that.

But right now, I know how to build a filter to look for the domainkey: line in the body of a bounced email, using php and imap(ok, that's a lie - I know how to go about doing it and could figure it out for 75-85% of bounces). But I wouldn't commit to doing the cryptographic stuff at this point.

[garyamort]

Now, I'm sure given time programmers much smarter than me who understand regular expressions will figure out filters that can extract the original header and body of the email message from the bounce, and then programatically check DNS and do the cryptographic comparison of the data.

And then someone like me can come along and put the little pieces together to build a filter that does all of that.


Speaking of which, here is a perl module that you can use to verify a message against the DomainKey in the header:
http://search.cpan.org/~anthonyu/Mail-DomainKeys-0.88/lib/Mail/DomainKeys.pm

What's the point?

Well, by using that module and this one:
http://search.cpan.org/~markov/MailTools-1.74/Mail/Header.pm

You could add a line to the header of an email when it is verified.

For example: X-mydkverification: yes

Than you can add a rule to your spam filter so that any email which was DK verified automatically passes without any other checks. Therefore cutting down on the number of false positives.

[dank]

Just 4 posts back, Dan, there seems to me to be a measurable benefit?
(from tril)
Only if you rely on incorrect, or misleading, information... I can send mail to Yahoo accounts just fine sans DomainKeys. No issue there that I can tell.

Dan

[Yaun]

FutureQuest has implemented DomainKeys as a possible, and I emphasize possible, assist in ensuring email you send from your domain is accepted by other networks.

Using DomainKeys to block bounces back to your domain is not possible at this time and I personally am not sure it would ever be effective as all bounces do not contain the full original raw headers and the signature would not be available.

Turning OFF your Catch-All is still the best defense against bounces that result from Forged mailings...

-Bob

Hi Bob,

A few questions. One was the same one that lenoxb phrased so well, and I hope you don't mind addressing it again.

The first announcement in this thread (same one I received in the email bulletin) gave the impression that this system would reduce the chances of other email systems receiving spoofed headers with our domain names in them. And that, it seemed, was the reason to be excited about this new feature -- because it would improve our lives as users of FQ's email systems.

As lenoxb pointed out, the junk we are receiving as "blowback" has nothing to do with our behavior, nor with the authenticity of our messages. And for me, and others I guess, the blowback is the biggest PITA problem.

(Meanwhile, I'm sure others like me have reasons to keep the catchall account active. I've put filters on mine, but there's only so much you can do, and piles of junk still come through, and turning it off completely seems like an inconvenience to me, not to the spammers!)

So if other email servers are still accepting spoofed headers, and this new feature does not address that problem, then I'm not sure I get the point. I won't argue (much) about the notion of doing this for the "greater good of the internet" except I'm trying to figure out whether the added overhead (300 bytes or so, I think you said) on every single message actually imposes a burden on the pipelines, while not really cutting back on spoofed traffic.

Am I missing something on the logic and math there?

Second point, related to that one: I am not aware of ever having my own, real emails blocked by other systems before this, because the only bounces I've ever received were due to spoofs and fakes. So I'm not anticipating a benefit in that area either. Perhaps this might apply more to other folks who use the SMTP outgoing servers at FQ more than I do, but personally it seems to address a problem I don't have. Please help me out if I'm missing something there, too.

Finally, the simplest question -- for the most part, I am using a truly ancient email client -- a 4.x version of Netscape -- largely because I can turn off most of the features whereby Malware can get in, and because the client is so old that I assume it is not a popular target for hackers. It still works fine for my purposes, so I haven't needed to change.

I am mostly curious whether there is anything about the old Netscape mail client that would prevent it from properly implementing (sending or reading) the security key in the header that you depicted earlier. Everybody always mentions Outlook and Eudora, so I want to make sure that I would not have to change clients just to use this feature.

THANKS for any replies, as well as all your help here already. :smile:

[tril]

Only if you rely on incorrect, or misleading, information... I can send mail to Yahoo accounts just fine sans DomainKeys. No issue there that I can tell.

Dan

Unsigned mail sent to Yahoo can be delayed for up to a day before it is delivered. This counts as a significant annoyance for some of us.

See http://www.ahfx.net/weblog.php?article=107

[tril]

Okay, I'm not a FQ employee, just a humble site owner, but I feel someone ought to clear a few things up. FQ employees feel free to correct me :smile:

- Forget about bounces. This is not about bounces.

- (As I've said a few times in this thread) Yahoo greylists unsigned mail. If you've ever received a "message deferred" mail after sending to a Yahoo account, that's why. Use DomainKeys and your mail to Yahoo will be delivered instantly instead of hours later.

- Various spam filters, including SpamAssassin, have functionality to check DomainKeys. Your users' email will look more "hammy" with this feature enabled.

- You don't need any special support in your mail client.

Bottom line: it's a Good Thing and there's no good reason not to use it (with the "sign some" option).

Tril

[Juan G]

- Various spam filters, including SpamAssassin, have functionality to check DomainKeys. Your users' email will look more "hammy" with this feature enabled.
Yes, there are currently 7 DK SpamAssassin tests (http://spamassassin.apache.org/tests.html) (SA v3.1.x):

DK_SIGNED
DK_VERIFIED
DK_POLICY_TESTING
DK_POLICY_SIGNSOME
DK_POLICY_SIGNALL
USER_IN_DK_WHITELIST
USER_IN_DEF_DK_WL

The last two tests (whitelists) have a large effect.

There are also other 7 similar DKIM tests. (DKIM is a kind of upgrade of Yahoo's DomainKeys, upward compatible with existing DomainKeys DNS records).

[Bob]

Hi Bob,

A few questions. One was the same one that lenoxb phrased so well, and I hope you don't mind addressing it again.

The first announcement in this thread (same one I received in the email bulletin) gave the impression that this system would reduce the chances of other email systems receiving spoofed headers with our domain names in them. And that, it seemed, was the reason to be excited about this new feature -- because it would improve our lives as users of FQ's email systems.

The benefit, at this time, would be as others have posted in regards to some mail networks greylisting (basically delaying) mail not signed and passing through DomainKeys signed mail on the first pass...


So if other email servers are still accepting spoofed headers, and this new feature does not address that problem, then I'm not sure I get the point. I won't argue (much) about the notion of doing this for the "greater good of the internet" except I'm trying to figure out whether the added overhead (300 bytes or so, I think you said) on every single message actually imposes a burden on the pipelines, while not really cutting back on spoofed traffic.
My understanding is that the added overhead is not an issue with today's pipes.

No, at this time, it will not reduce the number of spoofed messages however what if everyone was using DomainKeys then we would potentially see a large drop in the number of spoofed emails as more and more would be able to filter on valid keys.

(Ok maybe a pipe dream I know but if we don't try we will never know, and the spammers will win :( )


Second point, related to that one: I am not aware of ever having my own, real emails blocked by other systems before this, because the only bounces I've ever received were due to spoofs and fakes. So I'm not anticipating a benefit in that area either. Perhaps this might apply more to other folks who use the SMTP outgoing servers at FQ more than I do, but personally it seems to address a problem I don't have. Please help me out if I'm missing something there, too.

It is anticipated that other large email networks, AT&T, AOL, MSN, EarthLink and others will eventually start using DomainKeys (or one of the other solutions) and if the big boys ever all get together on the same page they will require mail reaching their network to be signed or risk being bounced, delayed or just dropped.

Why do I believe this, because they do not want to spend one more dime on backend infrastructure to handle the ever increasing amounts of spam. Adding additional backend resources basically cuts directly into their bottom line profits and at some point they will say "enough is enough".

I personally believe this is already happening as evidenced by Yahoo, AT&T, Comcast and Earthlink who have all, at one time or another, have blocked messages from individual domains and the MX servers as a result of what they see as excessive forwarded mail.


Finally, the simplest question -- for the most part, I am using a truly ancient email client -- a 4.x version of Netscape -- largely because I can turn off most of the features whereby Malware can get in, and because the client is so old that I assume it is not a popular target for hackers. It still works fine for my purposes, so I haven't needed to change.

I am mostly curious whether there is anything about the old Netscape mail client that would prevent it from properly implementing (sending or reading) the security key in the header that you depicted earlier. Everybody always mentions Outlook and Eudora, so I want to make sure that I would not have to change clients just to use this feature.

As tril noted DomainKeys is server side, not email client side and what you use for that purpose should not make any difference however I would have to strongly recommend to drop the Netscape 4.X for any use as you have no way of determining if a new threat issued today would also affect your client and no way to patch if it should be affected.

Just because hackers are not targeting your particular email client that is not to say that a threat released today would not be exploitable in older unsupported software and I would rather take my chances with current software, knowing that if a threat is released I can be warned and also get patched... just my personal opinion here...

I would like to add, and maybe I should have much earlier, DomainKeys is not a Holy Grail, it will not stop spam, will not stop spoofing email, will not reduce the phishing attempts... today...

It may eventually be able to be used to reduce some of the above but I believe that sooner, rather than later, you will find that your mail will be accepted quicker by some networks if you do sign your mail and as I said before, its cost, FREE, requires no changes in your email client, no plugins, just takes throwing a switch, one time, in your CNC, and what can it hurt... Go ahead and give it a spin ;)

-Bob

[jeep]

Sorry if I've missed it but I've not seen clarification on my question... Does email sent by PHP via SMTP server (instead of just using mail() function) get DomainKey signed?

Thanks.

[Arthur]

Does email sent by PHP via SMTP server (instead of just using mail() function) get DomainKey signed? Not by default, because the connection is not authenticated (local connections don't require authentication). If you add authentication (SMTP AUTH or PbS) then the message will get signed.

[EDIT] See: [FQuest Announce] DomainKeys Signing of Locally Generated Email

-Arthur

[kitchin]

I will certainly make sure we look at this possibility however at a glance I would think it would take up too much space for many accounts as they may have up to 15 additional domains associated with a main domain and that would place the email account settings much further down the page.
Anything that would put all CNC email info on one page would be a great enhancement. Clicking around in CNC is slow and makes it difficult to document my settings. Note, by slow, I mean fairly slow over SSL, with images turned off in Firefox, over dialup, and somewhat slow on DSL. I wouldn't say the page's HTML is highly formatted -- it's clean -- but the page loads are fairly slow nonetheless. A nice printable text page, with links to the relevant edit pages, would be great.

[jeep]

Thanks Arthur.

One final Q while we are on the topic - is there a big performance hit in doing SMTP AUTH compared to plain local connection?

[kitchin]

This is good news.

Anyway, here is a PHP function for sending using SMTP instead of mail(). It works and the DomainKeys thing shows up in the header. May be handy for now if you want to use "all".

It even works remotely, from my MS Windows command prompt here. Based on example at:
http://pear.php.net/manual/en/package.mail.mail.send.php

<pre>
<?php
// config the function smtp_auth_send_mail() below before using
// error_reporting(E_ALL);
require('Mail.php');

$error= smtp_auth_send_mail(
'test@example2.com', // to
'test subject', // subject
'test message' // body
);
print $error ? "error=$error" : 'OK, mail sent.';



/////////////////////////////////////////////////////////////
function smtp_auth_send_mail($to, $subject, $body) {
/////////////////////////////////////////////////////////////
// return value is 0 on success or an error string
/////////////////////////////////////////////////////////////
// Based on example at:
// http://pear.php.net/manual/en/package.mail.mail.send.php
/////////////////////////////////////////////////////////////

/////////////////////////////////////////////////////////////
/////// config here (do not accept these from user input) ///
/////////////////////////////////////////////////////////////
$domain= 'example.com';
$from= 'you@'.$domain;
// ANY valid POP user/pass at $domain
// (it does not have to be $from's user/pass):
$username= 'you2';
$password= '********';
/////////////////////////////////////////////////////////////

/////////////////////////////////////////////////////////////
/////// Check $to and $subject are clean /////////////////////
/////////////////////////////////////////////////////////////
// Reject invalid 'To:' email addresses and subjects with line breaks.
// The 'To:' pattern misses some invalids, and does not allow every possible
// valid email address, such as usernames with quotes,
// but, hey, you get what you pay for here.
if (!preg_match('/^[\w\-\!\#\$\%\&\*\+\/\=\?\^\.\{\|\}\~]+@[\w.-]+\.[a-z]+\z/i', $to)) return 'bad to';
if (preg_match('/[\x00-\x1F\x7F`]/', $subject)) return 'bad subject';
/////////////////////////////////////////////////////////////

$recipients = $to;

$headers= array();
$headers['From']= $from;
$headers['To']= $to;
$headers['Subject']= $subject;

$params= array();
$params['host']= "mail.$domain"; // The server to connect. Default is localhost
// Port 1025 needed for remote testing from my ISP:
$params['port']= 1025; // The port to connect. Default is 25
$params['auth']= true; // Whether or not to use SMTP authentication. Default is FALSE
$params['username']= $username; // The username to use for SMTP authentication.
$params['password']= $password; // The password to use for SMTP authentication.
// $params['localhost'] // The value to give when sending EHLO or HELO. Default is localhost
// $params['timeout'] // The SMTP connection timeout. Default is NULL (no timeout)
// $params['verp'] // Whether to use VERP or not. Default is FALSEs
// $params['debug'] // Whether to enable SMTP debug mode or not. Default is FALSE
// $params['persist'] // Indicates whether or not the SMTP connection should persist over multiple calls to the send() method.

// Create the mail object using the Mail::factory method
$mail_object =& Mail::factory('smtp', $params);
// the class of $mail_object is 'mail_smtp' or 'pear_error':
if ('mail_smtp' !== get_class($mail_object)) return 'no smtp';

$result= $mail_object->send($recipients, $headers, $body);
// $result is TRUE or a 'pear_error' object
return $result===true ? 0 : 'cannot send';
}
/////////////////////////////////////////////////////////////

?>
</pre>
Done.

[dank]

Unsigned mail sent to Yahoo can be delayed for up to a day before it is delivered.
Maybe I'm just lucky, but I've experienced zero delay. As in, send email, check Yahoo account, seconds later it's there. I've never received delay messages when sending to other peoples' Yahoo addresses.

Bottom line: it's a Good Thing and there's no good reason not to use it
You're overlooking one aspect, which I think is what Yaun was getting at. To some people, the topic might appear crystal clear, but I'm guessing somewhere around 80-90% of the people reading this are pretty confused by it and wondering what the catch is, i.e. are there potential negative implications to turning on the signing option if using it wrong.

if the big boys ever all get together on the same page they will require mail reaching their network to be signed or risk being bounced, delayed or just dropped. Why do I believe this, because they do not want to spend one more dime on backend infrastructure to handle the ever increasing amounts of spam.
Are you sure about that? I was under the impression that the major ISPs are/were making huge amounts of money on spam delivery, thus their unwillingness to curtail it.

Dan

[Stephen]

OK, i've just started looking into the DomainKeys business, partly based on the fact that a client of a client of mine has noticed they are unable to deliver to AOL and Yahoo addresses using the SMTP mailer in my web application.

so now i'm thinking perhaps they are using DomainKeys and are unaware of that fact (they happen to be a bank). now if it was the case that they were using DomainKeys, and they weren't using SMTP authentication in my web app (can't trust people to do the right thing always), i guess there is a good chance those messages are not being signed, and that that may be the reason for non-delivery.

so now i'm thinking that without using SMTP authentication (which i offer in my web app--but let's assume i didn't, or it remains switched off) i assume i could generate the appropriate DomainKeys headers myself (or rather the app could) if i knew the private key of the client.

by extension, what if i wanted to do the same thing on a domain hosted by FQ? i presume i could get the private key by contacting FQ and asking for it? is that possible? you must have one for each separate domain, right? i realize releasing the private key might sound like a bad idea, but if it can be made available via the secure CNC area on request developers like myself might start incorporating DomainKeys solutions into their apps.

does that seem sensible/reasonable? i'm just thinking out loud here--wondering about how to tackle these deliverability problems in general without assuming the web hosting company will tack on DomainKeys headers themselves as FQ does for authenticated email. of course, if every web host that used DomainKeys implemented them on authenticated SMTP messages i wouldn't even need to think about this--i'd just say to clients, add the SMTP authentication parameters and you'll be OK. because getting that information would be a lot easier for them, i would think, than getting the private key...

[Andilinks]

I wish I had time for this, will DomainKeys 2.0 be any easier? Maybe I'll just wait.

[Bob]

You're overlooking one aspect, which I think is what Yaun was getting at. To some people, the topic might appear crystal clear, but I'm guessing somewhere around 80-90% of the people reading this are pretty confused by it and wondering what the catch is, i.e. are there potential negative implications to turning on the signing option if using it wrong.


If "sign some" is enabled and they send an email using a wrong From: address the email is simply not signed, there is no negative entry. As far as I know there would not be any potential downside to turning on as long as "Sign some" is enabled.

There would be a potential for a negative effect if they turn on "Sign all" and they send an email that is not signed to a provider that filters on DomainKeys and rejects any mail not signed if the DNS entry indicates all mail will be signed.


Are you sure about that? I was under the impression that the major ISPs are/were making huge amounts of money on spam delivery, thus their unwillingness to curtail it.

Dan

Dan I really am under the impression that AOL, AT&T, MSN, Hotmail, Yahoo, Gmail and all the other majors are very much anti-spam (they may not have been at some point in the past) as a result of what it is costing them in terms of backend support and customer care.

If you have evidence other than that I would certainly be interested in seeing it. However given the major ISPs moves to block any IP address that they see as sending spam, whether forwarded or sent from originally I really believe they are on the right side at this time.

For what it is worth I enabled DomainKeys on every personal domain of mine the minute it was announced, not because I have personally experienced any problems but to A: Prevent possible problems down the road and B: To try and do what little I can do to further a signing authentication method that hopefully will one day become a Recognized Standard and then it really could help reduce the amount of spam many of us see.

However the bottom line is DomainKeys is available for those that choose to use it and it is each site owners choice...

-Bob

[dank]

If you have evidence other than that I would certainly be interested in seeing it.
Nothing recent. The vibe I got is that they want the bandwidth usage, as that equates to profit, but they don't want to be seen as doing nothing about it, so they occasionally block IPs based on customer complaints (or automated predictors of such).

Perhaps a FAQ would be of help for DomainKeys, indicating what the feature does and does not do, as well as what the optional settings accomplish?

Dan

[Bob]

Dan,

A Knowledgebase Article is already in the works and will be added soon. :)

-Bob

[Yaun]

... No, at this time, it will not reduce the number of spoofed messages however what if everyone was using DomainKeys then we would potentially see a large drop in the number of spoofed emails as more and more would be able to filter on valid keys.

(Ok maybe a pipe dream I know but if we don't try we will never know, and the spammers will win :( )

Well, that's a fine dream that one can easily agree with. But I have a simpler dream, and that is getting some kind of effective filter that can keep FQ from sending me spam that results from bounced spoofs. This is not the same as filtering regular spam, and the email I received sounded like it was addressing this very problem (see below).

That message had an uncanny timeliness for me. Earlier this week, I received over 300 emails that were all just bounces and various rejections from servers that had been spoofed with an address theoretically belonging to me. It wasn't a FQ address being spoofed, but one that forwards to FQ. The point is, the messages went to the catchall address, and could have been filtered out IF the system were set up to allow it. Let me explain.

Since the messages were mostly plain text replies from postmasters and the like, they did not trigger SpamAssassin, which is of course set pretty low on that alias.

I do NOT want to close off or disable the catch-all alias, like some people suggest. It has a role to play, otherwise why even have it? And I can't set SpamAssassin to bounce just the rogue email address, because that particular SpamAssassin setting is universal across the entire account. Thus, it would likely end up bouncing false-positives (because of the low trigger setting).

Since the junk mail" I received was coming from multiple, various sources, i.e. people who had been spoofed with my domain name, I did not have the option of simply filtering by IP address.

A lot of folks on this forum address such problems by saying, "Well, just write a custom script" or such advice. Well, sad to say that some of us do not know how to do that. Some of us are just running simple HTML websites and using whatever rules Futurequest puts in front of us to clean up email.

Some three or four years ago, I suggested/asked about configuring SpamAssassin (or something like it) to be able to make a double-pass through an email header. Currently, it ignores messages that already contain SA scores in the header. As a result, you only have one crack at the disposition of ALL messages in the main SA filters. The other built-in filters can't address the problem, either.

With a dual-pass SA, one would have at least some protection from Malware in all messages that do pass through, by instructing SA to turn every single one into an attachment, with a new header that includes the Spam flag.

Then, on the second pass, you could bounce others based on some other trigger -- whether it is a slightly higher Spam score, or using the Addressee option, or whatever. That way, you would not risk rejecting false-positives, but your Inbox would still be a lot cleaner. Meanwhile, no suspect message would arrive in its native state, where one might click on it by mistake and invoke a problem.

I may not be explaining this properly, as I'm running out of time, but in any case, I (like most others here) have an immediate need -- which is not solved by the long-term "pipe dream" (as you put it) of changing the Internet gradually over a period of years. We need to address BOTH.

Somebody earlier in this thread told me "this has nothing to do with spoofs." OH? Here is what the email from Futurequest said:

Spammers are increasingly "spoofing" the senders of emails, by forging email addresses on domains that they do not own or control. You may have received notifications of delivery failure for emails that you never sent for addresses on your domain that you never use. This is an example of spammers spoofing your domain as the sender of their emails.

Several models for signing or verifying the senders of messages have
been proposed, including SPF, DomainKeys, Sender ID and the like. The idea behind these signing systems, is that the receiving mail server will be able to know whether or not the email originated from the domain owner, or is forged...

As you can plainly see, this message seemed to be addressing the exact problem that I had earlier in the week. Contrary to that other user's comment, the email from FQ mentioned spoofing in the very first sentence. And that message is what led me to this thread. So I'm sorry if I expected more than the system can do, but I think it's understandable.

... As tril noted DomainKeys is server side, not email client side and what you use for that purpose should not make any difference however I would have to strongly recommend to drop the Netscape 4.X for any use as you have no way of determining if a new threat issued today would also affect your client and no way to patch if it should be affected.

Just because hackers are not targeting your particular email client that is not to say that a threat released today would not be exploitable in older unsupported software and I would rather take my chances with current software, knowing that if a threat is released I can be warned and also get patched... just my personal opinion here...

Well, I may not be able to write custom scripts but I do have some horse sense. This old Netscape version offers a really easy way to disable Java, javascript, images, and stylesheets, that would take forever to fish through in Internet Explorer, for instance. So I run no risk that I can perceive of loading an executable, or script, or whatever that might be hidden in an image, or an attachment, or in some part of the message source code.

It has a plain text viewer of source code, of course, as well as a full header report, and it handles most of what comes down the pike. On top of that, I have Norton AV scanning every message that comes in OR goes out, and I run spyware scans every now and then, which of course look through the message folders in the Netscape directory.

With the way I am using this client, its capabilities and actions are so totally crippled that it's hard to imagine what any kind of malware could accomplish with it. There simply aren't very many processes running. I guess I should mention that all of this is happening behind a hardware firewall, as well.

So I just don't see this as *more* dangerous than, let's say, Firefox, for which half the spyware-killers out there don't work correctly, or IE, which seems to need a new patch about every three days. Presumably I must have been exposed to the same stuff as any other frequent Internet user out there, but the only things that ever got on my PC got there by way of IE -- not Netscape.

That does not mean I don't appreciate the advice or good intentions, though! Thanks for the suggestions, and sorry for the quasi-rant.

... I believe that sooner, rather than later, you will find that your mail will be accepted quicker by some networks if you do sign your mail and as I said before, its cost, FREE, requires no changes in your email client, no plugins, just takes throwing a switch, one time, in your CNC, and what can it hurt... Go ahead and give it a spin ;)

-Bob

That last bit of logic is, of course, pretty much inarguable! One can only hope that, as more messages pass through servers with the DomainKeys code embedded, other ISPs and companies will indeed, as you speculated, take notice and join the club -- sooner rather than later. :smile:

[Bob]

Stephen,

On the subject of obtaining the private key, which I see you have also sent to the Service Desk, that would not be possible. FutureQuest uses a single key, which is a provision for ISPs such as hosting providers, for all domains.

If the private key was provided to anyone there is a possibility that it could be used on a server outside of the FutureQuest network to send Forged mail that also was signed which of course defeats the entire purpose of DomainKeys.

-Bob

[Velimir]

I have just sent myself a message from FQ hosted domain to yahoo; it did say it is signed by domain key; it ended up in spam folder :rasberry:

·velimir

[kitchin]

Yaun, do you want to delete all bounce messages?

[Bruce]

One final Q while we are on the topic - is there a big performance hit in doing SMTP AUTH compared to plain local connection?If by "plain local connection" you mean making a local SMTP connection without issuing the AUTH command, then the only difference is issuing a single command. There is a slight delay internally as the authentication data is processed against the mail backends, but normally that would take on the order of 1/10s or less.

However, there is a big difference between what the built-in PHP mail() function does (invoking sendmail/qmail-inject/qmail-queue) and doing a local SMTP connection. Making a local SMTP connection does almost all the work that the equivalent mail() function call does, but adds loading up several extra programs plus the SMTP protocol overhead.

We are already working on a solution to allow locally generated emails (such as with PHP's mail() function) to be signed by DomainKeys without any modification to your scripts. We will post more information about the solution when it has been completed.

[Yaun]

Yaun, do you want to delete all bounce messages?

No, and I don't think the question can be answered that way. A "bounce" message can be many things.

On some of my mailboxes, selected message types are bounced "aggressively" with a terse message like "Undeliverable" or whatever. On others, messages might be handled more gently, like "Your message has triggered our spam filters. If you feel this reply is in error, please send your message again in plain text form."

There are lots of reasons to do things this way, that I hope I don't have to explain here.

Anyway, I am my own example of a postmaster sending "bounce" messages of diverse types. Therefore, I expect others may be doing the same, and that I may receive varied kinds of "bounce" messages in response to those that I originate.

I don't usually get bounces from legitimate messages that I originate, of course, but it does happen. Perhaps the best examples are when a person's mailbox is simply too full for ANY messages, or an address no longer works because a person has changed ISPs. Obviously you would want to know, in both cases, that your message did not arrive. So, automatically deleting every "bounce" reply, sight-unseen, would be a bad idea.

The kind of bounces I want to block are those sent to an obviously fake address, coming from innocent postmasters or mailbots, which are just responding to messages sent to them with forged address headers. They end up in my Catch-all mailbox -- along with other mail that my legitimate correspondents might send with a typo in the spelling of my name, or some such thing.

Other "bounce" messages might come to my non-catchall mail addresses, of course. So that's another twist.

That's a long answer but it would be easier to answer briefly if your question were more detailed. Anyway, if you had some idea in mind, now you know some of the scenarios that I am dealing with.

[kitchin]

So how would you filter on that? All bounce messages have one characteristic that would be easy filter on (no return path, so bounces don't bounce back), and after that you could filter on the text in the message. What are you saying, the bounce messages you want to block display certain text that can distinguish them from other messages? Or is it something in the addresses in the headers? You've already said you don't want to write your own filter, so I was just wondering how specific you can be about the properties that filter would have if someone else wrote it.

[garyamort]

I don't usually get bounces from legitimate messages that I originate, of course, but it does happen. Perhaps the best examples are when a person's mailbox is simply too full for ANY messages, or an address no longer works because a person has changed ISPs. Obviously you would want to know, in both cases, that your message did not arrive. So, automatically deleting every "bounce" reply, sight-unseen, would be a bad idea.

The kind of bounces I want to block are those sent to an obviously fake address, coming from innocent postmasters or mailbots, which are just responding to messages sent to them with forged address headers. They end up in my Catch-all mailbox -- along with other mail that my legitimate correspondents might send with a typo in the spelling of my name, or some such thing.


Some thoughts:
Bounced emails usually contain a copy of the email message in them. It is only if there is a copy of the email message that you could programatically determine if there is even a chance the bounce message is valid.

So, here is my suggestion for a catch all bounce processing function(which I would be happy to work with you to write because it sounds interesting, send me a pm and expect low priority work - I expect someone else could write it faster)

For the catchall address, set up a set of rules identifying bounced messages. Rules such as "subject line contains the words 'returned email'"

First off, you turn on DomainKeys in your email settings.

Secondly we pull out a list of all the emails with charectoristics you tell me are common of bounced emails.

For all returned email, we will follow the following 2 rules:

1) We scan the BODY of the email for a header line indicating there was a DomainKey in the email sent out. If there was, this email is filed in a folder called "Bounced Email to check" on the server using PHP IMAP functions.

2) We scan the BODY of the email for emails which do not contain a DomainKey line, but do have a valid(for your server) email address that sent it(list to be provided by you in some manner....does FQ support the VRFY SMTP function? Ie can I programatically verify from addresses by querying the SMTP server directly? I'd go with caching valid email addresses for some period of time initially, and invalid ones for a different period. IE no reason to check a valid address for every email, once we know it is valid or invalid we use that response for the next 30 days or so - or maybe setup some sort of periodic recheck function to reload the cached data......) These emails get filed in a folder called "Bounced Emails, may be forged"

3) We scan the BODY of the email bounce for emails that don't match the above 2 rules. These emails get filed in a golder called "Bounced Emails, probably forged"

We set this up as cron job, do some logging to see what the server impact is(anyone willing to contribute that bit of code?) and test to see the impact.

After running this for a few months, if you determine that the probably forged emails are always forged, one or two changes in the code are needed to change the process from file to delete.

Problematic note: if the postmaster bot which rejects your email does not send the headers of the rejected email in the bounce, their email will end up in the rejected folder.

I'm willing to work WITH you to do the above. I can handle much of the code, but I am not familiar with live emails sufficiently to come up with the rules to detect bounced emails.

There may be a more efficient way to do this, I default to PHP because PHP is simple to code in and when I'm doing things for personal use, I base my decision for code on what will take the least time. If we find PHP has a big server impact, I'm willing to try recoding it in perl down the line to see if it is any better.

[garyamort]

Oh, I should note, the above has a limitation in that we are assuming the DomainKey header line is valid. Since it is relatively new, I'm assuming Spammers don't include an invalid line in the header in the hopes of email going through. If they start to do so, than this program would need to be extended to make some attempt at checking the contents of the bounce. It would be out of scope for the first try because I start small and add functionality. If we find it is impossible to reliable detect bounced emails to begin with, than it's a waste of time to work on verifying the content of their keys.

Oh, and in case it's not obvious, except for user identification and server identification, any code developed on this would be Public Domain. As you can see from my last snippet, I expect most of the code to already be written and I'm just putting the pieces together.

[lenoxb]

Even before the availability of DomainKeys, I had managed to develop a filter for my catchall account that can discard 95% of the blowback. The heuristics aren’t terribly sophisticated, and I have developed them by piecewise refinement in response to the evolution of the blowback stream. (It’s not exactly rocket science: rejecting mail with From fields containing “Postmaster” or “MAILER-DAEMON” gets about half of them.) At this point, there’s very little tweaking required. The machine is pretty much built, and I don’t even have to turn the crank for it to discard the blowback automatically.

The latest development in my catchall account is more troubling. We know that spammers use viruses on the PC’s of the unwashed masses to harvest addresses. At this point, it seems that the viruses are harvesting forged addresses from my domain that are in old spam messages. Then those forged addresses are themselves spam targets, and my catchall account may soon get more spam than I do.

There’s some poetic justice in this, of course. Spammers are sending their garbage and aren’t even reaching real people with it due to their own tricks. But the specter of 90% of Internet bandwidth being sucked up by unwanted, unread, machine-generated messages is not one that inspires much hope for the future.

[Bruce]

I'm assuming Spammers don't include an invalid line in the header in the hopes of email going through.Sadly you'd be wrong wrong (http://www.eweek.com/article2/0,1895,1732576,00.asp).

[Buck]

Well, that was quick! ;(

[dank]

Uh, that's a two year old article.

Dan

[Terra]

Uh, that's a two year old article.
Two year head start, and they are refining their techniques daily... :(

--
Terra
sysAdmin
FutureQuest

[Yaun]

So how would you filter on that? All bounce messages have one characteristic that would be easy filter on (no return path, so bounces don't bounce back), and after that you could filter on the text in the message.

I just took a look, and it seems that I receive a lot of legitimate mail that does not have a return path in the header. So, evidently, that would not be a useful screening criterion. Frankly, I would not trust that anyway since, as far as I know, nearly every part of the header could be forged anyhow.

What are you saying, the bounce messages you want to block display certain text that can distinguish them from other messages? Or is it something in the addresses in the headers?

It's pretty simple -- these are messages addressed (as replies) to screwy names or aliases that I have not handed out to others, nor used myself. Like, xssafot@xxxx and so on.

Beyond that, they all say words to the effect of "you sent us something we didn't like, so we're sending it back, or at least, we are telling you what the message title was so you can identify it." Naturally, every mail system has different ways of wording this, so I am not sure one could specify all of them.

I suppose that the words "returned mail" in the subject line, in mail sent to the catchall account, would snag a fair number. But I also get replies to spoofs in cases where people used my "real" email ID. AND every now and then, sure I make my own mistakes sending out a message to an old address, or mistyping something -- so I might very well get "returned mail" that really is mine, that I would want to receive and read! All in all, it's a sticky wicket, and I realize it's starting to sound like a set of needs that can't easily be automated.

You've already said you don't want to write your own filter, so I was just wondering how specific you can be about the properties that filter would have if someone else wrote it.

The reason I can't write the filter is at least in part because I could not specify what properties it would have! This leaves me in a catch-22 situation. I don't want to sound dumber than I am, but the point is that designing a decent filter requires knowing what criteria to specify, that will result in effective trapping without creating false positives or false negatives.

As noted above, it seems that the absence of a return path is not useful. That's not even a critierion I would have thought of, which gives you some idea of how conversant I am (or am not) with the appearance of a legitimate header. I get many messages from legitimate sources with headers that look pretty different. I guess I'm saying, I don't know how to answer the question. But thanks anyway!

[Yaun]

Some thoughts:
Bounced emails usually contain a copy of the email message in them. It is only if there is a copy of the email message that you could programatically determine if there is even a chance the bounce message is valid.

In my experience, legitimate forwarded messages also contain entire copies of messages, with headers.

So, here is my suggestion for a catch all bounce processing function(which I would be happy to work with you to write because it sounds interesting, send me a pm and expect low priority work - I expect someone else could write it faster)

For the catchall address, set up a set of rules identifying bounced messages. Rules such as "subject line contains the words 'returned email'"

As noted in my last message, the rules would have to include many, many variations on that wording -- and might still snag messages that were legitimately sent to me due to my own errors (rare though they may be) <g>.

First off, you turn on DomainKeys in your email settings.

Secondly we pull out a list of all the emails with charectoristics you tell me are common of bounced emails.

For all returned email, we will follow the following 2 rules:

1) We scan the BODY of the email for a header line indicating there was a DomainKey in the email sent out. If there was, this email is filed in a folder called "Bounced Email to check" on the server using PHP IMAP functions.

Problem here is that I do not send all my outgoing mail via the Futurequest SMTP server. Generally I do my correspondence elsewhere and have stuff forwarded there from FQ. I log in to FQ when I want correspondents to get return mail from the accounts on my site. This tends to keep my "real" email account off the internet as much as possible, which is why the FQ domain is the target of the spoofs rather than the account at my ISP. Unfortunately, I still have to look at them, one way or another. Hence, this discussion. Anyhow, I guess a filter that used combined criteria -- DomainKeys present PLUS a FQ server named in the header -- would
do what you describe. Starts to bloat a bit, though.

2) We scan the BODY of the email for emails which do not contain a DomainKey line, but do have a valid(for your server) email address that sent it(list to be provided by you in some manner....does FQ support the VRFY SMTP function? Ie can I programatically verify from addresses by querying the SMTP server directly? I'd go with caching valid email addresses for some period of time initially, and invalid ones for a different period. IE no reason to check a valid address for every email, once we know it is valid or invalid we use that response for the next 30 days or so - or maybe setup some sort of periodic recheck function to reload the cached data......) These emails get filed in a folder called "Bounced Emails, may be forged"

3) We scan the BODY of the email bounce for emails that don't match the above 2 rules. These emails get filed in a golder called "Bounced Emails, probably forged"

We set this up as cron job, do some logging to see what the server impact is(anyone willing to contribute that bit of code?) and test to see the impact.

After running this for a few months, if you determine that the probably forged emails are always forged, one or two changes in the code are needed to change the process from file to delete.

Problematic note: if the postmaster bot which rejects your email does not send the headers of the rejected email in the bounce, their email will end up in the rejected folder.

I'm willing to work WITH you to do the above. I can handle much of the code, but I am not familiar with live emails sufficiently to come up with the rules to detect bounced emails.

That makes two of us. In fact, count me as two extra people, because I don't think I could do most of what you described before that, either.

There may be a more efficient way to do this, I default to PHP because PHP is simple to code in and when I'm doing things for personal use, I base my decision for code on what will take the least time. If we find PHP has a big server impact, I'm willing to try recoding it in perl down the line to see if it is any better.

I have to confess that my needs are probably far more modest than some folks here, and my abilities to tinker with scripts and whatnot are probably even more modest than that. So, it does not make much sense for me to go crazy over this one problem. My site is not (presently) doing any sort of e-commerce or other high-profile stuff. It's more of an ancillary part of my activities, and the central part of my business is not getting enough attention as it is. So, as grateful as I am for the generous replies here, I'm not really able to contribute anything useful as far as coding or the like, to a project as big as you described. Oh, well!

[Matt]

I think the following questions may have been answered indirectly, but a simple yes/ no to each would help considerably:
1. Does mail sent via Questmail include DomainKey signature?
2. Would mail sent through a locally installed SquirrelMail (i.e. on FQ server under user's own directory), or other web-based e-mail client, include DomainKey signature?
3. Will mail sent via PHP's mail() call include DomainKey signature? Does the previously listed sample code (that uses SMTP calls) offer a way to generate DomainKey-signed e-mail from within a PHP script?
4. Are all e-mails sent from a domain signed, or only those that match an existing account under the domain name? For example, if my primary account is main@mydomain.com and I send an e-mail (through FQ's mail servers) with the sender's address set as other@mydomain.com ('other' is not a real account), will this e-mail be properly signed?
5. If I send e-mail using a "spoofed" return address that matches an IRM domain through the FQ mail servers, will it be signed properly?

Based on the answers to these questions, it should be possible to generate a checklist so that FQ customers can evaluate whether or not it makes sense to enable DomainKeys at this time.

I agree that DomainKeys is a good idea if a critical mass is reached. The one BIG concern is this: If the private key was provided to anyone there is a possibility that it could be used on a server outside of the FutureQuest network to send Forged mail that also was signed which of course defeats the entire purpose of DomainKeys. If I understand this correctly, it now means that if an ISP's private key is compromised EVERY customer has the potential to be affected, not just those sharing the same IP address or server. I don't have time to read the details of how DomainKeys is implemented, but it sounds like a better idea would be to supply a private key on a per-domain name basis.

Finally, learning that DomainKeys can tell a recipient whether the sender is valid, but cannot tell the sender whether a bounced e-mail was one that the sender did or did not send, seems like a big limitation. I am very much affected by forged e-mail bounces and an inability to address this sounds like a weak point that spammers may be able to exploit.

-Matt

[lenoxb]

AND every now and then, sure I make my own mistakes sending out a message to an old address, or mistyping something -- so I might very well get "returned mail" that really is mine, that I would want to receive and read!

That's not really a problem, given that the outgoing mis-addressed mail was sent from your account. The bounce will go back to you, not to the catchall.

[Bruce]

1. Does mail sent via Questmail include DomainKey signature?No, QuestMail does not sign messages at this point. We will be working on making that happen soon.

2. Would mail sent through a locally installed SquirrelMail (i.e. on FQ server under user's own directory), or other web-based e-mail client, include DomainKey signature?
3. Will mail sent via PHP's mail() call include DomainKey signature? Does the previously listed sample code (that uses SMTP calls) offer a way to generate DomainKey-signed e-mail from within a PHP script?Both of these will be signed after the recently announced enhancement (http://www.aota.net/forums/showthread.php?t=22519) has been completed.

4. Are all e-mails sent from a domain signed, or only those that match an existing account under the domain name? For example, if my primary account is main@mydomain.com and I send an e-mail (through FQ's mail servers) with the sender's address set as other@mydomain.com ('other' is not a real account), will this e-mail be properly signed?Yes. The signing system only looks at the domain names.

5. If I send e-mail using a "spoofed" return address that matches an IRM domain through the FQ mail servers, will it be signed properly?Yes, if you are authenticated on the primary or IRM/IRO domain. That is, if you authenticate to the primary (or IRM/IRO) domain, the system will sign all messages sent from either the primary or any of its IRx domains.

Based on the answers to these questions, it should be possible to generate a checklist so that FQ customers can evaluate whether or not it makes sense to enable DomainKeys at this time.There is little, if any, reason not to enable "sign some" for all domains. As you indicate, you should be careful about enabling "sign all".

If I understand this correctly, it now means that if an ISP's private key is compromised EVERY customer has the potential to be affected, not just those sharing the same IP address or server. I don't have time to read the details of how DomainKeys is implemented, but it sounds like a better idea would be to supply a private key on a per-domain name basis.You are correct in your understanding, although the impact of such a comprimise would be limited to the ability to forge message signatures (at least until we generate a new key). We will be looking into creating individual keys for each domain once the initial rollout of all the components is completed.

Finally, learning that DomainKeys can tell a recipient whether the sender is valid, but cannot tell the sender whether a bounced e-mail was one that the sender did or did not send, seems like a big limitation.I'm not sure what you're trying to say here. If a message containing a DomainKeys signature is bounced, the bounce will (usually) contain that same signature. If you wanted to, you could validate that signature if the bouncing agent included the entire message in the bounce. Not all mail systems do, though, and there are many different bounce formats, so you would be at the mercy of whatever systems were involved in the transport of your email.

[lenoxb]

Maybe the following data will help some people who want to develop their own filters. I don’t want to give out my code for two reasons: 1) I don’t want to support it, and 2) I have a single script on the catchall account which is rejecting both blowback mail and “first generation” spam to bogus addresses which spammers have collected, thinking they are legitimate.

Anyway, here’s what I do to catch and delete blowback.


Discard email with empty From: or To: fields.
Discard email from containing “futurequest.net” in the From: field and “failure notice” in the Subject: field.
Discard email in which any of the following strings appear in the From: field
admin@
administrator@
badmail@
BADSENDERS@
blackhole@
bounce@mail.
bounce_spam@
bounce-spam@
confirm@
Gateway
GoBid S.A.
mailer@
MAILER-DAEMON
mailerr@
NONDELIVERY@
noreply@
NoSpam@
notify@
<postmaster>
postmaster@
Spam Firewall
webmaster@
Discard email in which any of the following strings appear in the From: field
absence_du_bureau
Abwesenheitsnotiz
Address Incorrect
Automated reply
Autoreply
Auto reply
Autoresponse
Challenge Response
confirm subscribe to
Confirm your subscription to
Delivery Error
Delivery Failure
Delivery Notification
Delivery Status
ECHEC DE DISTRIBUTION
Email not allowed
Error en el sistema de correo - Mensaje Regresado
ezmlm response
Failed mail
failure notice
FES Administrator Notice
hold queue
List Message Rejected
mail blocked
Mail could not be delivered
Mail Delivery Failure
mail rejected
mail storage exceeded
message blocked
message rejected
Out of Office AutoReply
Please confirm your email-address
Please confirm your request to join
Please Verify Your Email Address
Rejected posting to
rejete
RESPONSE REQUIRED: Confirm your subscription to
Returned Mail
SMTP error
SMTP filter
spam
Spam Challenge
Unable to deliver your message
Unable to process your message
Undeliverable
Undeliverable mail
Undelivered
unknown recipient
UNSOLICITED BULK EMAIL
Unsolicited commercial email rejected
vacation
You just sent an email to
Your e-mail could not be delivered
Your email requires verification
Your Message Could Not Be Delivered
Discard mail in which any of the following terms appear in the Precedence: or X-Precedence: fields:
auto_reply
bulk
junk
list
Discard any mail in which the X-AutoRespond: Auto-Responder:, or Auto-submitted: field is not blank.

When discarding mail, the filter writes a message to stderr so that it goes into the ~/../logs_email/filter.log file. That message contains the reason for the rejection as well as the From, To, and Subject fields of the rejected message. I check the log file every few days to see if there were any unpleasant surprises.

[garyamort]

Problem here is that I do not send all my outgoing mail via the Futurequest SMTP server. Generally I do my correspondence elsewhere and have stuff forwarded there from FQ. I log in to FQ when I want correspondents to get return mail from the accounts on my site. This tends to keep my "real" email account off the internet as much as possible, which is why the FQ domain is the target of the spoofs rather than the account at my ISP. Unfortunately, I still have to look at them, one way or another.

Here is a simple solution for you that I just learned of, attacks it from the other end(it was posted on the FQ forums, I apologise to the author as now I can't locate the thread)

Here are "the rules"
Email to be delivered directly to individuals uses their email address. So that goes through automatically.

Now, you also want to be able to just 'make up' and email alias whenever you want and not have to configure FQ to map each one of these.


Soooo, you establish a fixed string to include in ALL your made up aliases. For example, all my email aliases will have "gm-" at the start. Or all my email addresses will contain "-" in the address(so I could use gary-phony, allen-phony, and mort-phony).

Create an address for catch all email, and direct all the catch all email to it.

Now go the CNC, go "Email manager", go to "Built-in filters".

Go alll the way down to the bottom and select "SMTP Recipient Address"

Choose enabled for the first option.
Choose Redirect for what to do with the email.
Fill in your email address in the box next to "if redirect...." so it knows where to redirect the email to.

Enter your simple pattern, for example mine was:
*-*@saplings.us

This means all email with a dash somewhere in the email address. If I wanted to further restrict it to just gm-, it would be gm-*@saplings.us

Save the filter.

Now any email being delivered to(note, that is who the server delivering it says it should go to, not who is in the to or cc lines of the email) the FQ server matching your pattern(which you picked so as not to random gibberish) will go to your email box.

Everything else goes to your catchall junk email box.

Save your filters and run for a while, periodically check yoru junk email box to see if there are any aliases you used which don't match the filter criteria and either update the place using that address with a new address, or manually add an alias for that specific address.

This is not a quick fix, it will take time and discipline for you to get to the point where your doing this regularly, and you have to deal with the backlog of email aliases you have used which do not match this pattern(I find that by chance, gm- is in 50% of my own made up aliases, so it cuts my workload down)

But it is SIMPLE, and simple solutions are good IMHO.

[Slim]

Sadly you'd be wrong wrong (http://www.eweek.com/article2/0,1895,1732576,00.asp).
I do wish the article described what the phishers are actually doing. I can't tell if they are just sending the DK headers with a bad value, or if they found a way to get the private key, or what.

[Snarpy]

But instead of using wellsfargo.com they are using wellfs4rg0.com. People read it too quickly. Sure it is properly authenticated to be from that user, but it doesn’t stop the user from thinking it’s from someone else.
ha.ckers.org (http://ha.ckers.org/blog/20060726/phishing-domainkeys-laundering-oh-my/)

[kitchin]

Two interesting things about DomainKeys, stop me if this is obvious & boring:

1. Once you started to use DK to filter incoming mail, your software would consult a lookup list of domain's public keys to validate the message. The lookup list could or should flag known phishy domains like wellfs4rg0.com (in real time).

2. For fifteen years PGP privacy types have been trying to get everyone to encrypt & sign outgoing email. Never happened. Now the signing part has been automated at the server level! This means legal cases can use the DK sig. to verify the email as evidence, even if the sender deleted his or her outgoing mails. And it means jobs for expert witnesses to say how much proof a domain sig represents. (The PC could have been hijacked, or the private key compromised, or there could always be a weakness in the algorithm.) It does leave the PGP privacy dream on the rocks.

[Yaun]

Here is a simple solution for you that I just learned of, attacks it from the other end(it was posted on the FQ forums, I apologise to the author as now I can't locate the thread)

Here are "the rules"
Email to be delivered directly to individuals uses their email address. So that goes through automatically.

Now, you also want to be able to just 'make up' and email alias whenever you want and not have to configure FQ to map each one of these.

Yes! you correctly understand one of my reasons for wanting the catch-all to be active.

Soooo, you establish a fixed string to include in ALL your made up aliases. For example, all my email aliases will have "gm-" at the start. Or all my email addresses will contain "-" in the address(so I could use gary-phony, allen-phony, and mort-phony).

And so on... This is great. You have figured out the one obvious step that I was missing. If I make up an ad-hoc address now and then, it's usually to identify the sender -- in case that sender shares it with some other source and it becomes a spam magnet.

Using "one-shot" aliases for different correspondents gives me the option of either 1)making them "permanent" once they've proven good behavior, by setting them up formally in the Email manager, or 2)blocking that specific address, if it turns out to have been abused.

And for all those spammers who use a made-up address without using my "secret code" (as you described), blocking them is easy. Sounds swell!

...This is not a quick fix, it will take time and discipline for you to get to the point where your doing this regularly, and you have to deal with the backlog of email aliases you have used which do not match this pattern(I find that by chance, gm- is in 50% of my own made up aliases, so it cuts my workload down)

But it is SIMPLE, and simple solutions are good IMHO.

Again, you've identified the key issues, primarily the necessity of "waiting out" old aliases that I used without inserting a special filter-check string within the name, and then remembering to always use it in the future. The latter part won't be hard, after all this discussion!

And I agree, at first glance it does seem quite simple, and therefore elegant for what I'm trying to do. Thanks!

[garyamort]

And so on... This is great. You have figured out the one obvious step that I was missing. If I make up an ad-hoc address now and then, it's usually to identify the sender -- in case that sender shares it with some other source and it becomes a spam magnet.

I can't take credit for this. Someone else mentioned it on the mail forum and I just about banged my head into my keyboard for not having realised it myself.

Than I scurried off and implemented it. :-)

Now I have setup my mail client(thunderbird) with a second email account(my catch all) and I check it about once a day to find any good email to addresses which don't have my key and either add the address as an alias, or find that individual and update them with a new address(most of it is automated mailshots from online stores).

Hopefully, in 2 months or so I can go in and update my catch all to delete automatically once I've gotten everything updated(as I understand it, you must have catch all enabled for this to work, otherwise FQ will reject email that does not have a know to address. However, you don't have to point it to an account you actually are going to look at!)

[Yaun]

Just occurred to me -- one bit of info that might cool off our current sense of accomplishment over this solution that you have implemented, and that I now plan to implement. (Sorry to be a downer!):hrmm:

In that massive batch of bounced junk that I referred to earlier in the thread... the forger had created an email ID that included my real domain name within the fake user name. It was to that address that all the postmaster/mailbot rejection replies were being sent.

It just so happens that the domain in question includes my initials. And that is, obviously, a two-letter string that I might have been tempted to use as an easy-to-remember "secret key" as we've discussed here. If I had been filtering mail that way when the forger struck, that string would have been included in the email address, pretty much by coincidence alone -- and the messages would not have been filtered out!

So I have to remind myself to make the "code" string something unlikely to be used by chance, but still easy and obvious enough that I can insert it in "ad hoc" email addresses that I might give to somebody on the spur of the moment. Obviously I don't want to compound the problem by giving them something weird, that might be easy for them to mistype or whatever. Their messages would get filtered out and I'd be back to square one.

So, it has to be something easy to remember, and hard to guess, at the same time. Sounds like a paradox brewing... Obviously this is why the "waiting period" is a good idea, as we both had thought about, to test any new system.

Anyway, I just thought I'd mention the specifics of this incident, in case anybody else is following along this thread and this idea. When I devise the "test string" for my little filter, I will have to remind myself of this extra rule:

Don't use any sequential combination of letters that are already part of a legitimate email address, or your name, whether it's in a previously set-up alias, or in the domain name itself.

Considering how many weird, random address strings people stick in all these various forged spam headers, I wonder if I will be able to come up with one that they never duplicate simply by chance. Hm!

[Tom E.]

The person in what I think is the original suffix-filtering thread (http://www.aota.net/forums/showthread.php?t=22216) used a hyphen to separate the common part of the address from the unique suffix.

I think it's a decent choice to reduce the risk of random matches, because hyphens aren't used too often in domain names or email addresses.

[garyamort]

J
It just so happens that the domain in question includes my initials. And that is, obviously, a two-letter string that I might have been tempted to use as an easy-to-remember "secret key" as we've discussed here. If I had been filtering mail that way when the forger struck, that string would have been included in the email address, pretty much by coincidence alone -- and the messages would not have been filtered out!


True, but if you use a hyphen as well, than your fine.


IE, if your domain is gm.saplings.us
And your secret key is gm- at the beginning of any email
Than the filter
gm-*@saplings.us

Will not be fooled by
gm.saplings.us@gm.saplings.us
Nor by gm.saplings.us-junk@saplings.us

However, its not foolproof. There is still the possibility of a spammer entering your secret key by accident, or random generation.

Going from 100 spam rejections to 2 or 3 though is still a massive improvement. :-)

[garyamort]

The person in what I think is the original suffix-filtering thread (http://www.aota.net/forums/showthread.php?t=22216) used a hyphen to separate the common part of the address fro.m the unique suffix.


Thanks you! For the life of me I couldn't find it again even with searches. I hate not crediting someone properly for their help.

[Yaun]

The person in what I think is the original suffix-filtering thread (http://www.aota.net/forums/showthread.php?t=22216) used a hyphen to separate the common part of the address from the unique suffix.

I think it's a decent choice to reduce the risk of random matches, because hyphens aren't used too often in domain names or email addresses.

I see that Gary has also addressed this question, and your point (like Gary's) is well taken: testing on a text string that includes a hyphen would, most likely, work better than a string that is strictly alphabetic.

However, I'd like to observe that one of my sites does have a hyphen in the domain name! I'm not crazy about it and the reason for it is too long to explain. In any case, I personally find the hyphen problematic when giving out the address verbally (as opposed to written form). Since people do not always listen carefully, I feel obliged to take extra pains to make sure that they get the punctuation correct. On occasion, they still mess it up.

But I don't want to get into a long talk about it. I just wanted to mention that hyphens work best (in any part of an email OR web page address) when you do not have to say the word "hyphen" out loud, in the midst of other words or letters.

And one more problem occurs to me -- the same problem that would happen no matter what kind of test string one might use. Let's say one of the people (or companies, or whatever) to which you give a "filter-ready" address is naughty, and that address you gave them ends up on a marketing list somewhere, or otherwise published where it can be harvested.

All of a sudden, the system would be broken, and at the very least you'd end up with a few spam messages requiring manual deletion, until such time as you could get around to setting up a filte to block that specific address (i.e. the entire alias, not just the prefix).

Well, I suppose that's not such a massive problem, compared to the one we face at the outset. It just occurred to me as a potential fly in the ointment. But don't get me wrong, I still think the plan is worth trying. Thanks.

[Velimir]

Can someone help me as it seems I am missing something... I have selected Sign All Checkbox in sCNC and this is what I get over here (http://domainkeys.sourceforge.net/cgi-bin/check_policy?domain=veliscoffee.com&Submit=Submit).

Testing veliscoffee.com
Policy TXT=o=~; r=postmaster@veliscoffee.com

This policy record appears valid.

Tag Value Explanation
o ~ Domain signs some email
r postmaster@veliscoffee.com Domain has a reporting address for invalid verification results

Thanks!

· Velimir

[Bob]

Velimir,

If your domain is set to "Sign some" then the results would be:

Tag Value Explanation
o ~ Domain signs some email
r postmaster@example.net Domain has a reporting address for invalid verification results
t y Domain is in test mode


If "Sign all" is chosen then it is not listed as "Test mode"

-Bob

[Velimir]

Velimir,

If your domain is set to "Sign some" then the results would be:



If "Sign all" is chosen then it is not listed as "Test mode"

-Bob

Oh, easy as that :kewl: ... I guess I could have tried ( actually I did, but I was too impatient to wait for few minutes as I needed to send some emails :smile: )

Thanks for all Bob :vday2: !

·Velimir

[Bruce]

Can someone help me as it seems I am missing something... I have selected Sign All Checkbox in sCNC and this is what I get over here (http://domainkeys.sourceforge.net/cgi-bin/check_policy?domain=veliscoffee.com&Submit=Submit).This was actually an oversight in the deployment of the DNS portion of the DomainKeys implementation. :BPG2: While we were developing it, the "testing" notation was used to prevent problems with potential receivers. Now that we have confirmed it is working properly, we have switched away from using the testing flag to actually indicating that domains will sign all messages.

[Matt]

One question: If we set up a domain to "sign some," are unsigned e-mails more likely to be filtered than if that domain was set to its default "sign none?" In other words, by "signing some," e-mails move from a state of all being medium suspicion to some being low (signed) suspicion and others being high (unsigned) suspicion. -Matt

[Velimir]

This was actually an oversight in the deployment of the DNS portion of the DomainKeys implementation. :BPG2: While we were developing it, the "testing" notation was used to prevent problems with potential receivers. Now that we have confirmed it is working properly, we have switched away from using the testing flag to actually indicating that domains will sign all messages.

Thankk for the explanation :yeah: .

[garyamort]

J
It just so happens that the domain in question includes my initials. And that is, obviously, a two-letter string that I might have been tempted to use as an easy-to-remember "secret key" as we've discussed here. If I had been filtering mail that way when the forger struck, that string would have been included in the email address, pretty much by coincidence alone -- and the messages would not have been filtered out!

BTW, I just ran across something that made me stop a think a bit.

There was a guy who had his email address as nospam@domain.com

He did this because he beleives that spam harvestors are being written smarter now so that they automatically delete the common tricks people use to stop from receiving spam, yet still post their email address(so someone might post nospammyname@domain.com and in the text of the message say to delete the nospam part)

If this is true(can anyone confirm it) you could use this 'trick' as part of your key.

IE if your key is 'nospam-' than your made up addresses would be nospam-myname@domain.com

Now you get the added benefit that if that email address ends up on the net somehow, some of the email spammers will delete 'nospam' and try to send their spam to -myname@domain.com - which will get rejected or deleted or however you have it setup.

[sheila]

One question: If we set up a domain to "sign some," are unsigned e-mails more likely to be filtered than if that domain was set to its default "sign none?" In other words, by "signing some," e-mails move from a state of all being medium suspicion to some being low (signed) suspicion and others being high (unsigned) suspicion. -Matt

This would be entirely up to the mail administrators at the receiving end.

As far as I have heard at this time, "sign some" is not currently resulting in any penalties for unsigned emails any more than an email would be penalized by having no DK record at all. Actually, it seems that at this time, signed messages are receiving preferrential treatment on some mail systems.

This is all subject to change, of course.

[Yaun]

BTW, I just ran across something that made me stop a think a bit.

There was a guy who had his email address as nospam@domain.com

He did this because he beleives that spam harvestors are being written smarter now so that they automatically delete the common tricks people use to stop from receiving spam, yet still post their email address(so someone might post nospammyname@domain.com and in the text of the message say to delete the nospam part)

If this is true(can anyone confirm it) you could use this 'trick' as part of your key.

For starters, yes it's true -- has been for many years. As soon as people started putting obvious spam-block strings in their IDs, the spam-harvesters wrote routines to filter them out and glean the legitimate part of the address.

If you think about it, just imagine the way spammers create email addresses and subject lines, to evade spam filters that we use! If they can write addresses to foil our spam filters, they can overcome our attempts to block their harvesters. Obviously some of them are smarter than others, and not all of them have uber-programmers on staff!

But anyway, yes I remember several years ago when they started being able to overlook obvious strings that people had stuck in their addresses when writing online, like "delete-this-part" or "spamblock" or whatever.

IE if your key is 'nospam-' than your made up addresses would be nospam-myname@domain.com

Now you get the added benefit that if that email address ends up on the net somehow, some of the email spammers will delete 'nospam' and try to send their spam to -myname@domain.com - which will get rejected or deleted or however you have it setup.

Well, we're discussing a very specific example (using the hyphen), which is an obvious text divider for an address-harvesting program to look for and exclude, especially if near "nospam" or the like. So, it might not be as foolproof as one might hope, but who knows -- I suppose the proof would be in the pudding, so to speak. ;-)

[Velimir]

just out of curiosity, any particular reason FQ (http://domainkeys.sourceforge.net/cgi-bin/check_policy?domain=futurequest.net&Submit=Submit) is not using Domain Key?
I assume there is a good reason.

thanks :confuz:

.velimir

[Bruce]

any particular reason FQ (http://domainkeys.sourceforge.net/cgi-bin/check_policy?domain=futurequest.net&Submit=Submit) is not using Domain Key?FutureQuest's internal mail systems are intentionally segmented from those used on the community servers. As such, there are different programs involved in sending the mail and different feature sets. We will be adding DomainKeys signatures to our mail, but have so far been prioritizing giving site owners those features first.

[JSWski]

After enjoying a few months of no bounce messages, last night I was innundated with nearly 2000 bounce messages! What happened? :dunno: Did someone figure out how to work around the DomainKeys? Did I somehow lose my settings? I remember enabling something to get that going, but now I can't find the instructions. :eeww:

Any ideas?

Thank you!

[Bob]

Hello,

As I posted in one of the early posts in this thread...

Tom,

This will not stop someone from Forging your domain in the From: address in spam emails and it will not stop bounces from spam messages from being sent to your domain when forged in this manner.

The best way to reduce the number of bounces you receive when a spammer does Forge your domain in the From: address of an email is to ensure your Catch-All is turned off.
http://Service.FutureQuest.net/kb267

-Bob

[JSWski]

Yeah, I do remember that, and I did close down my catch-all account this morning (although I really didn't want to). I guess I was looking for an explanation of the sudden change, from 5-6 bounce msgs a night to 2000. Just the spammers whim, huh?

[Yaun]

FWIW... I concluded early on -- with the help of people here! -- that the new Domainkeys service would not help in my problem of several weeks ago, which is much like yours. Like you, I suddenly had scads of new junk messages, all resulting from the bounces of forged messages using my domain name in the header.

Being forged with made-up IDs, these were coming back through my catch-all account. Like you, I did not want to close that account down. I have to say, quite candidly, that I do not think that a customer having to turn off a feature of his/her account is a good "solution" to a problem. And the catch-all "alias" is very handy for some of us. It handles misspellings, however rare they might be, and it allows one to make up a new address for one-time use, without having to configure it ahead of time.

So, I chose to adjust the Spamassassin settings instead. I looked at all the junk bounces and found the most common IDs in the "from" headings -- usually "Mailer-DAEMON" or "Postmaster" or whatever. Each such message gets tagged with a string that identifies the message as most likely being a bounced forgery.

When the messages finally reach me, I can sort them together by the subject lines, all of which start out alphabetically identical because of the tag I added in the filter. Then I can scan them quickly, and if I don't recognize the website or domain name it came from, out it goes. And of course, that's the case for 99% of these bounces.

This process is fast enough that it is pretty painless, it does not result in false positives (since I give them a visual scan first, rather than deleting automatically), and it is remarkably thorough.

Once I got it in place, I just put up with a daily cleanup, and after a couple of weeks, the number of such messages REALLY dropped off. Now I probably only get a handful of such junk messages per day, if that.

I'm guessing that system administrators and/or postmasters blocked the spam, filtering either by the email address, or just my domain name, or possibly by adding my mail server to some kind of list. Whatever the reason for the flow of junk stopping, it would have happened regardless of what I did at the receiving end.

SO -- I have been able to keep the catch-all address open and functioning normally. You have to remember, most spammers who want to reach YOU try to use real addresses at your domain, rather than making up random strings. That means most of the junk messages going to the catchall are going to be bounces BACK from a recipient, as we've discussed here. At least, that has been my experience.

Hope this helps!

[JSWski]

Thanks, Yaun :yeah: that does help. I've got some ideas now that I think will help me manage the bounce msgs, and, I suspect you're right that number will go down in time.

Why does someone always have to come in a ruin a good thing?!?! I hope the spammers get spam too!

[songdog]

Is FQ using the enhanced version called "DomainKeys Identified Mail" (i.e. DKIM), or the original DomainKeys version?

I just read that the IETF adopted DKIM as a draft standard two days ago, so I'm wondering if we're already set to go with that.

Thanks!

[Bruce]

Is FQ using the enhanced version called "DomainKeys Identified Mail" (i.e. DKIM), or the original DomainKeys version?We are using DomainKeys, not DKIM. There are additional requirements on DKIM email, and library support isn't quite as good yet.

[frankc]

On the Email Manager page of the CNC, can you change this page so that the status of DomainKeys is shown, vs the current "Enable/disable..."? If it's already been enabled, it should show the state.

[Arthur]

On the Email Manager page of the CNC, can you change this page so that the status of DomainKeys is shown, vs the current "Enable/disable..."? If it's already been enabled, it should show the state. Thanks for the suggestion. This is already on the to-do list, but it depends on some other changes so it may take a little time before it'll appear.

-Arthur

[Tom]

Hi - Any update on QuestMail? It's been some months and I haven't seen an announcement that it uses DK's yet, right?

I think that was the reason I set it to "Sign Some" way back when, and today my sister had a ton of reverse DNS warning/bounces that I suspect might be related.

[Bruce]

Hi - Any update on QuestMail?A DomainKeys module for QuestMail has been developed and is in testing, but does not function properly at the moment. We will post a notice when the bugs are worked out.

[Bruce]

Hi - Any update on QuestMail?This has now been completed (http://www.aota.net/forums/showthread.php?threadid=23026)