Post by Steve AtkinsPost by Paul SmithPost by Steve AtkinsPost by Steve AtkinsThe bigger issue is that you shouldn't care about SPF failing. An SPF pass
provides somewhat useful data, an SPF fail means absolutely nothing.
Wasn't it supposed to mean "reject"? IIRC, that was the difference
between setting -all (as irtf.org does) rather than ~all or ?all.
That's what it's proponents used to claim, yes. But it's unfixably broken for
that purpose. The only reason you don't see serious mail problems for
anyone unwise enough to use -all is that almost nobody pays much attention
to SPF for rejecting email.
How is it 'unfixably' broken? Just interested.
it seems to me that the only reason it's 'unfixably broken' is that
people are using it without really understanding what it's doing. If all
MTAs rejected all mail that failed SPF checks, then they'd soon get the
problems fixed. Remote users having to relay via a specified submission
server is a trivial problem to fix (AFAICS), and incorrectly specified
SPF records (as in the BT/MS case) deserve to cause messages to be
blocked (IMHO)
(SPF is certainly not that useful as an 'anti-spam' mechanism, but it
seems to have the potential to work reasonably well as an anti-spoofing
mechanism, which makes it considerably harder for many spammers)
Mail forwarding is one common cause of failure. If you're sending mail to
you and AOL decide between you to enforce SPF -all then that mail will be
undeliverable - and there's no way either you or AOL can predict that.
I thought most people did return-path rewriting for this case. If you don't
do that, then you get undesirable behaviour anyway - eg if the AOL mailbox
is full, you get a bounce message from AOL, even though you didn't send any
messages to AOL at all.
Yes, that happens. No, return-path rewriting is the exception, not the rule.
People still use Sendmail aliases and ~/.forward mechanisms and functional
work-alikes in other MTA's and it breaks the utility of explicitly
derogatory SPF, both "-" and "~"
However, as often as the classical forwarding model is brought up as an
example of SPF breakage, it really isn't the critical failure mode for
simplistic SPF usage, since classical forwarding is mostly a tool of people
who remember the days before the Eternal September. We're an influential
bunch, yes, but there are not enough of us to break something that would
otherwise work.
Post by Steve AtkinsMailing lists is another similar cause of SPF failures..
Similarly, I thought return-path rewriting was standard for mailing lists.
Especially ones which do VERP, but usually even in ones which don't - eg the
return path on the message I received from you was
correctly passed an SPF test for irtf.org.
Yes, any mailing lists using reasonable list management tools rewrite
messages with their own return-path and cause no trouble for SPF. There are
some things called mailing lists that use 'exploder' features of various
MTA's which don't use their own return-paths or any distinct software, but
those sorts of lists were basically dysfunctional before spam was a
significant problem. Like classical forwarding, that is also not the hardest
obstacle for clean and simple use of derogatory SPF results.
Post by Steve AtkinsA third, which is a lot more common and complex to resolve than you'd
expect, is organizations simply not knowing or not controlling where their
mail is sent from. There've been cases both for SPF and ADSP ("SPF for
DKIM") where one group within a company put out a restrictive policy based
on their beliefs of where mail was sent from, and it all fell to pieces
when employees got kicked off mailing lists when mail appearing to come
from them was delivered from the mailing list manager and rejected due to
auth failure, or where internal monitoring email sent from servers (cron,
even) was silently discarded.
This is the problem of people who should know better not understanding
what's going on.
Yes. That is why this problem is the truly hard one for application of SPF
in derogatory ways.
Deprecating technical mechanisms via "standards" (to whatever degree a
low-grade RFC can be called that) and real-world bad outcomes has a
successful track record of working to extinguish their use or encourage
workarounds.
Deprecating human carelessness and ignorance has no successful track record.
NO SUCCESSFUL TRACK RECORD
In other words: THIS DOES NOT WORK!
If the company had a policy that all mail from their domain
had to go through certain servers, and employees were sending mail through
other servers, then they deserved to get kicked off mailing lists... Just as
they would deserve punishment for watching porn over the work Internet
connection, etc. If they had a policy that mail had to go through certain
servers, and configured SPF for different servers, then they shouldn't be
managing mail systems.
That's nice. The fact is that sysadmins follow Sturgeon's Law just like any
other large population. I'm very sorry about that. I really am.
Beyond that sad fact of life, there are other issues with "-all" (and even
"~all") grounded in forms of dumb that are external to the administrators of
any domain setting SPF records. For example, diverse online service
providers including resume search systems, news clipping agencies, magazine
publishers, and so on (and on and on) will send email on behalf of
registered users with any envelope sender address that they have for the
user. Whether it is a "mail a friend this article" link at the NYT or a
"find me candidates for this job" form at a headhunter agency, there are a
lot of ways that random 3rd parties can end up sending mail claiming to be
from a user of a domain whose SPF maintainers cannot predict them. I don't
like that, I think it is wrong and bad. However, it is my direct experience
that fighting that practice is a consistently losing battle because the
practice rarely fails, is easily accommodated, and is attractive to those
who engage in it because it lets them completely abdicate responsibility for
the mail they generate and send.
(I apologize for the overly complex sentences... )
If 'SPF' and/or DKIM was standardised and rigourously enforced, then
return-path rewriting would be de rigeuer as would people learning how to
configure the mechanisms properly.
This would require universal deployment of people designing and running mail
systems who are almost entirely not stupid, evil, irresponsible, lazy, or
even overworked.
Good luck with that. If you figure out how to find clueful people in a
reliable way, you can rule the world.
In the real world, the result is not (as some have claimed here) that SPF is
completely useless. It's just not a tool you can use without paying
attention to its many failure modes and particularly the ones where
ignorance and hubris are likely to cause trouble. The canonical example is
to counter phishing, where the sender domain is not used by random human
users and legitimate sources are actually known. The next most commonly
cited example is to recognize SPF records that consist of "-all" as the only
element, stating that there is no such thing as legitimate mail from the
domain. Stranger uses can be based on the recognition that there are
spammers who know about SPF and about simple spam filtering techniques;
being "too good" while being "very new" is a sign that a sender is a
spammer, and an affirmative SPF result can be a "tell." (yes, I have SA
meta-rules, no, I'm not sharing) If there is a general rule, it is that
SPF's uses require a sender domain (good or bad) whose admins understand SPF
well.
The bottom line is that SPF is actually useful as a small part of a spam
control system, but not in the simplistic way that it was designed to be
used or in ways that are practical for the giant providers of cheap/free
mailboxes. If you want to use SPF to control spam, then you need to have
someone skilled watching your mail stream. If you want to hand out cheap
mailboxes to anyone who wanders by, you will not find SPF to be useful.