2004-06-23 19:03:54 UTC
From: David Farber <***@farber.net>
Subject: [IP] 4 Rivals Almost United on Ways to Fight Spam
Date: Wed, 23 Jun 2004 05:36:29 -0400
4 Rivals Almost United on Ways to Fight SpamBy SAUL HANSELL
June 23, 2004
June 23, 2004
Four large Internet service providers agreed yesterday to apartial truce in their battle with one another over
potential technology to stop junk e-mail in hopes that they
can devote their united energy to fighting spam.
More than a year ago the four providers - America Online,Yahoo, EarthLink and Microsoft - said that they would work
together to create technical standards that could verify
the identity of the sender of an e-mail message.
The core problem with this approach is that it does NOTHING to prevent the
sending of spam... the only thing it does is to help make sure that the return
address on the spam is valid, and it's not even very good at doing that.
Most spam, and nearly all of the messages in the rapidlygrowing identity-theft fraud known as phishing, is done
with a fake return address.
1) There is NOTHING that requires that spam be sent with fake return
addresses... spammers use phoney return addresses largely AS A KINDNESS so that
complaints and bounces don't converge back on some poor victim's E-mail inbox.
2) Requiring the use of "real" return addresses, besides not preventing the
sending of spam, makes the spam problem WORSE instead of better... suddenly,
victimized ISPs will have to DELIVER (and store!) all these bounce messages and
3) A large (and increasing) percentage of all spam messages originate at
spambot zombies, machines that have been infected by worms or viruses and turned
into willing slaves to send spam as spammer proxies. These worms can be
reprogrammed, LITERALLY overnight, to use "real" return addresses and
authorizations belonging to the infected machine's legitimate owner. Once
that's been done, sender authorization is *useless* other than (hopefully)
rapidly identifying the infected machine. But that's still locking the door
after the horse has escaped, since there's thousands of newly infected machines
every day. You're playing a never-ending (if anything, escalating) game of
whack-a-mole... and like whack-a-mole, the machine can pop up moles in the end
stages of the game faster than you could ever hope to club them down.
Many experts suggest that asystem that could identify and discard such falsely
addressed messages is one of the most potent possible
weapons against spam.
Those "experts" are blowing smoke. I don't know why they're so fixated on these
misguided and ill-conceived "authentication/authorization" approaches, but
ultimately these approaches mostly just hurt numerous legitimate users, and do
not really solve the problem.
"The biggest thing we can do to reduce spam is senderauthentication," said Brian Sullivan, the senior director
for mail operations at America Online.
That's simply *rubbish*. Unless and until they get the spambot zombie problem
under control, they cannot solve the spam problem. And it's relatively easy to
solve the spambot zombie problem by using a finely-grained permissions system,
where each recipient authorizes senders to send them familiar and trusted types
By DEFAULT, anybody could send any recipient plain ASCII text messages, up to
some limited size (say 50K or 100K bytes) and without any attachments. In order
for an E-mail sender to send other types of material and have the recipient
actually see it, they'd have to negotiate with the intended recipient to get the
right to have such mails delivered to that recipient.
For example, I might grant my Aunt Gertrude the right to use fonts and
bold/underline in her E-mails to me, or JPGs of her poodle Fifi, but I wouldn't
E-mails that LOOK like the sort of things I'd expect to get from Gerty would be
delivered to me; even if her machine got turned into a zombie spambot. Stuff
that DOESN'T look like what I expect to receive from Gerty would be summarily
t-canned, even if it (actually and truly) came from her machine and with her
return address. (In practice, most people wouldn't allow ANYBODY AT ALL to send
them executables, PIF files, SCR files, CPL files, VBS files, and the like...
even ZIP files... which would essentially eliminate the ability of infecting
those user machines with zombie spambots! You don't require daily updates of
virus/worm signatures (which, of course, also inevitably LAG the problem) for
Once HTML is denied in E-mails of unapproved senders, most of the tricks and
deceptions that spammers and phishers use are also prevented. This allows a
good content-based antispam filter to very effectively deal with the spam that's
But the Internet providers have supported differenttechnical approaches. Last month, Microsoft agreed to merge
its proposal, called Caller ID, with another, called Sender
Policy Framework, or S.P.F., backed by America Online and
EarthLink. The new name of the combined standard is Sender
Those approaches are ALL fatally flawed for a variety of reasons, but the main
one is simply that (despite that they inconvenience a *LOT* of legitimate users)
they simply DO NOT SOLVE THE PROBLEM.
I think it's awfully telling that AOL (one of the biggest offenders in sending
gratuitously HTML-burdened E-mails, which are a big part of the problem) hasn't
figured out that their use of HTML is a big part of what makes the
spam/virus/worm problem so intractable.
Yahoo had continued to support a very different approach,called Domain Keys, that is more technically powerful but
would take longer to carry out.
The nice thing about a permissions-based system controlled by a recipient is
that it requires NO complex re-engineering of the world's E-mail infrastructure,
can be implemented IMMEDIATELY, is easy to understand and use, and immediately
benefits those users as soon as they install it.
In an announcement yesterday, the two remaining camps agreed to give limitedsupport to test each other's technology.
I think this is very sad, since these folks are taking their eye off the ball.
It's sort of the old story about "once you've decided on the hammer as your
tool, you try to make every problem look like a nail." They've developed this
elaborate system for identifying senders, and trying to pretend that it will
work against spam. Even when it should be clear to the most enlightened folks
that it won't solve the problem it's being presented with as justification.
The parallel to presenting invasion of Iraq as a "war against terrorism" with
vague references to 9/11 is too obvious to not mention.
"Over the last year, we had four gorillas learning how to dance," Mr. Sullivansaid. "Finally we can work from the same choreography."
Meng Wong, the author of the S.P.F. protocol, praised the agreement.Of course, since he's been one of the leaders of the "ignore the facts"
movement. I've presented in the SPF discussion groups many of the failings of
SPF, and gotten only excuses and denial in exchange. I concluded more than six
months ago that SPF was fatally flawed, and consequently left that discussion
group, concluding that pursuing it was a waste of time. Unfortunately, it seems
that other people haven't figured it out yet, so we're embarking on this grand
detour on the way to actually addressing and dealing the spam/virus/worm problem
in a meaningful and effective way.
"It's good news because we now have a road map," he said."We can proceed with S.P.F. and Sender ID now and with
Domain Keys as a second wave."
Their "road map" leads to a non-solution. Perhaps they're still happy because
at least they can be SEEN as "doing something", even if it's worthless. But
it's basically dishonest, since they OUGHT to know better.
Indeed, proponents said the two approaches had thepotential to be complementary. The Internet provider that
sends an e-mail message can use both methods at the same
time to vouch for the veracity of the sender's address.
So they require that the zombie spambot software sends the message using the
victim machine's real E-mail address (or, perhaps, the E-mail address of a
different user who happens to use the same ISP domain name). And there the SPF
solution reaches "end of road". You've adopted this grand scheme, and suddenly
you've reached the end of your rope and you've still got the spam/virus/worm
problem. NOW WHAT? Duh!
And the provider that receives a message can also look toeither approach to help determine whether a message should
be discarded as spam.
The problem is that it doesn't really tell you much. Many legitimate messages
will NOT comply with SPF's "rules". Several examples:
1) You're a travelling salesperson or executive and occasionally need to use
Internet cafes, airport waiting lounge kiosks, cruise ship internet cafes, or
other such places to send your important E-mail. Clearly you want to (and NEED
to) send it using your own company's E-mail address, since you won't be at that
location long enough to receive the needed replies at the temporary location
E-mail address. But you won't be (and often can not be) sending via your
habitual SMTP server associated with your domain name.
2) Yahoogroups-like mailing lists, which might forward individual E-mail
messages to group members, or might consolidate them as a daily digest.
America Online and EarthLink said yesterday that they woulduse Domain Keys by the end of the year. And Yahoo said it
would probably start using both Domain Keys and Sender ID
by the end of the year. Microsoft did not commit itself to
using Domain Keys, saying it was still evaluating it and
some other related approaches, like one recently proposed
All of these authentication/authorization approaches (like the micropayments
schemes too, for that matter) simply don't work when zombie spambots commandeer
authorized/authenticated machines and send out spams using a victim's legitimate
Despite the talk of tests, S.P.F. and the new Sender IDproposal appear to have momentum in being adopted by major
That's very disappointing. They OUGHT to know better.
America Online and EarthLink already use S.P.F. toverify their outgoing e-mail. And Microsoft has said it
will soon use the Sender ID system.
They ought to know better too.
Perhaps more important, America Online has said that by theend of the summer it will look to see whether messages it
receives are verified by S.P.F. and that high-volume
mailers will have to use it if they want their messages to
be delivered to AOL subscribers.
The big gorillas certainly can brutalize most people into adhering to stupid and
non-functional schemes, but ultimately they're going to inconvenience a LOT of
people, and end up just looking stupid themselves.
Several large e-mail senders, including Amazon.com and Google, have alreadytaken the steps necessary to verify their mail using S.P.F.
For some people, SPF isn't a big problem. For others, using it is nearly
impossible for a whole variety of reasons. But E-mail is supposed to be
UNIVERSAL; it's not enough to allow 80% or 90% of users to be able to send
E-mail and ignore the legitimate needs of the remaining 10% or 20%.
S.P.F. and Sender ID have gained a following because theyare the easiest to put in effect. They are based on the
fact that every computer on the Internet has a unique
identifier, called an Internet Protocol number. That number
is much harder to fake than a return e-mail address.
Sender ID allows an organization, like an Internet provideror a company, to designate certain I.P. addresses as the
computers that are authorized to send e-mail on its behalf.
Any e-mail that pretended to be from that organization but
was not from those designated I.P. numbers would be
And that's fine for a little organization with ten or twenty users which only
ever needs to send mail from within their offices.
But bigger organizations (say, Comcast.com/.net, or Earthlink, or AOL) which
have many millions of users and at the very least probably HUNDREDS of mail
servers mean that if a given E-mail comes from a valid ISP E-mail user name and
through ANY of that ISP's mail servers, it will still be "approved" (as indeed,
it would NEED to be... an Earthlink user might be travelling and call in to the
Earthlink access number in a distant city, during a trip). User A's Comcast
E-mail address could be forged by any of six or ten million zombie-infected
Comcast-connected machines in other cities, and still pass these misguided and
ineffective "authorized mail servers for the user's domain" tests with flying
The problem with this approach is that there are legitimatecases of one server's sending e-mail on behalf of another.
That's just ONE of the problems.
For example, online greeting card services often sendmessages with the return address of the person who sent the
message. That way, if the recipient of that message replies
to it, the response is routed back to the original sender.
A much bigger problem is that of mailing lists, whether Majordomo-style or
Frankly, I'd like to see an end to these "online greeting card services" anyhow,
since they're often just a front to collect E-mail addresses. Far better that
the "card company" send the card back to the originator, so they can forward it
themselves. That way, the sender isn't tricked into betraying the recipient by
giving the recipient's E-mail address to a third party. But that's a battle for
The backers of S.P.F. and Sender ID say there are ways towork around these problems, but they may require
adjustments to the procedures of some mail senders.
The fact that those "adjustments" are very often infeasible doesn't seem to
bother Wong in the slightest. He's determined to pursue his misguided scheme,
and to hell with everyone who won't or can't comply with it. It might ALMOST be
worth it if the scheme fixed the problem... but it doesn't, and never will!
The Domain Keys approach tries to verify the actual senderof a message, not the computer used to send it. The author
of an e-mail inserts a short code, known as a digital
signature, into the header of each message. The computer
that receives the message can use the signature to verify
if the message was actually created by the sender in the
"from" line. This method could let one computer send mail
on behalf of another, as in the greeting card example. But
it requires greater changes to the programs that send and
If it's possible for a third party to use the signature to send on behalf of the
sender owning the signature, then obviously that same third party could send
spam using the E-mail address and signature too.
The Internet providers, however, cautioned that both ofthese technical approaches are just part of the solution to
the problem. Once Internet recipients can verify who is
sending them mail, they can start to keep track of who
sends legitimate mail and who sends spam.
And that's about as far as it gets you, if even there. Okay, now you've
identified an infected machine. Now what? So you clean it up. Next week, next
month, it will probably become infected again. Meanwhile, literally MILLIONS of
new infections elsewhere will occur... tens or hundreds of thousands of
infections a day. Again, this is a never-ending game of whack-a-mole, and the
proposed solution will do DAMNED little to put an end to it. The proposed
solutions just divert attention from coming up with a REAL and EFFECTIVE
"I don't think that users will see a reduction in spamright away," said Robert Sanders, chief architect at
EarthLink. "Identity is just the first step."
No, of course they won't see a reduction in spam right away. In fact, based on
these schemes, they NEVER will see a reduction in spam, since these solutions
DON'T SOLVE THE PROBLEM!
Why don't we SOLVE THE PROBLEM, instead, and skip these diversionary and costly
non-solutions that complicate the infrastructure without any truly worthwhile
payback adequate to justify their cost?
By eliminating inappropriate HTML and attachments (via a finegrained permissions
whitelist) coming from unknown/untrusted senders, we can let through the mail we
want and need to receive, while blocking unexpected stuff coming from untrusted
(or even, for that matter, trusted) senders. For most users, who won't have
authorized ANY users to send them executable attachments (even encoded in ZIP or
other archives), we virtually *eliminate* in one fell swoop (and without needing
to download a neverending succession of updated "signature" files, which
*inevitably* lag new malware anyhow) the ability to use E-mail as a vector for
zombie/virus/worm infections... which are the largest single cause of
intractable spam, as well as other nasty problems like DDOS attacks.
Better still, if the problem is solved RIGHT, it requires NO change to the
worldwide E-mail or DNS infrastructure, [thus] NO worldwide consensus on what
needs to be changed, can be made understandable by even unsophisticated users,
and can be implemented IMMEDIATELY and with IMMEDIATE beneficial effect for
early adopters. And it doesn't block ANY legitimate uses of the Net.
-------------------------------------Archives at: http://www.interesting-people.org/archives/interesting-people/
<---- End Forwarded Message ---->
Gordon Peterson http://personal.terabites.com/
1977-2002 Twenty-fifth anniversary year of Local Area Networking!
Support free and fair US elections! http://stickers.defend-democracy.org
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.