Discussion:
asrg - research or die
(too old to reply)
John R. Levine
2013-03-20 13:55:50 UTC
Permalink
For the past couple of days, all I've seen is people rehashing their
favorite FUSSPs, with nothing new, and no indication that anyone plans to
do any implementation, research, or anything else.

If you want to have that kind of discussion, the SDLU list which is the
most recent reincarnation of SPAM-L is here:

http://www.medwayhosting.com/spam-l/index.html

Unless I see some indication of research interest, e.g., my often stated
desire to collect statistics on DNSxL cache behavior so we can try to
predict how it will or won't work with IPv6, I will shut this list down at
the end of the week.

Regards,
John Levine, ***@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

PS: My server, my rules, you know.
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
d***@chaosreigns.com
2013-03-20 15:25:46 UTC
Permalink
Post by John R. Levine
Unless I see some indication of research interest, e.g., my often
stated desire to collect statistics on DNSxL cache behavior so we
can try to predict how it will or won't work with IPv6,
There has been a fair amount of related discussion on the dnswl.org admins
list lately. I was thinking you folks should talk.

There was also previously discussion on the spamassassin dev list, but none
of the black/whitelist providers were interested in talking at the time.
Might be time.
Post by John R. Levine
I will shut
this list down at the end of the week.
I object, but maybe getting everybody into one place (spam-l) is good. I
sent a subscription request. I'd like a sign post left on the asrg web
site of what became of us, either way.
--
"Since everything in life is but an experience perfect in being what
it is, having nothing to do with good or bad, acceptance or rejection,
one may well burst out in laughter." - Long Chen Pa
http://www.ChaosReigns.com
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Barry Shein
2013-03-20 17:58:40 UTC
Permalink
At first I had a predictably disappointed reaction to John's ultimatum
but on reflection I tend to agree with him.

In particular the meta-discussion about list mechanics gets old, fast.

But even focusing on "what would be research? what would be progress
(on the problem at hand)?" would be more productive than "what is
spam?"

For example, modeling tools for email and other messaging behavior
might be useful.

I informally rely on very ad hoc tools here such as just picking out
the top email "user unknown" IPs in a real-time display I put
together. At certain thresholds some get blocked for a period of time.

Some might be amazed how often we get "dictionary attacks" and similar
from $BIGSITE that can go on for days. Right this minute there's one
going on from hotmail.com, mail to gdean1, gdean2, gdean3, ..., gdi1,
gdi2, ... on and on.

But in a sense that's a modeling tool, or an instrumentation anyhow.

Granted it's also an perhaps an example of how differently this
problem looks from the POV of an ISP vs an end-user just trying to
focus their email inbox better.
--
-Barry Shein

The World | ***@TheWorld.com | http://www.TheWorld.com
Purveyors to the Trade | Voice: 800-THE-WRLD | Dial-Up: US, PR, Canada
Software Tool & Die | Public Access Internet | SINCE 1989 *oo*
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Dotzero
2013-03-20 20:21:02 UTC
Permalink
Post by Barry Shein
But even focusing on "what would be research? what would be progress
(on the problem at hand)?" would be more productive than "what is
spam?"
I think it would be interesting to see a project that examines
phishing/malware emails to determine whether email authentication
(DMARC/SPF/DKIM) or other practices would have prevented the malicious
email from reaching endusers.
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Steve Atkins
2013-03-20 20:28:18 UTC
Permalink
Post by Dotzero
Post by Barry Shein
But even focusing on "what would be research? what would be progress
(on the problem at hand)?" would be more productive than "what is
spam?"
I think it would be interesting to see a project that examines
phishing/malware emails to determine whether email authentication
(DMARC/SPF/DKIM) or other practices would have prevented the malicious
email from reaching endusers.
I did that briefly, looking at both phishing attempts for, and legitimate mail from
one particular company, as seen at my inbox.

The results[1] were not what people wanted to hear, so I didn't bother digging
deeper or formalizing the results.

Unless you're in academia, where you justify your existence by publishing
papers, there doesn't seem much benefit to doing research that won't affect
behaviour.

An end goal beyond the (perfectly reasonable) "it'd be interesting" seems
like something to consider.

Cheers,
Steve

[1] Flipping a coin was a better phishing filter, both in terms of false
positives and false negatives, than DKIM+ADSP.

-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Dotzero
2013-03-20 20:42:15 UTC
Permalink
Post by Steve Atkins
Post by Dotzero
Post by Barry Shein
But even focusing on "what would be research? what would be progress
(on the problem at hand)?" would be more productive than "what is
spam?"
I think it would be interesting to see a project that examines
phishing/malware emails to determine whether email authentication
(DMARC/SPF/DKIM) or other practices would have prevented the malicious
email from reaching endusers.
I did that briefly, looking at both phishing attempts for, and legitimate mail from
one particular company, as seen at my inbox.
The results[1] were not what people wanted to hear, so I didn't bother digging
deeper or formalizing the results.
ADSP was an experiment gone wrong.
Post by Steve Atkins
Unless you're in academia, where you justify your existence by publishing
papers, there doesn't seem much benefit to doing research that won't affect
behaviour.
It isn't always easy to determine what will affect behavior.
Post by Steve Atkins
An end goal beyond the (perfectly reasonable) "it'd be interesting" seems
like something to consider.
The obvious goal would be to determine whether domains such as those
belonging to banks would afford endusers some modicum of protection by
adopting the combination of DMARC/DKIM/SPF.I've seen very positive
results for the domains I have implemented DMARC p=reject for.
Unfortunately there aren't mant case studies out there for people to
refer to.

Having said this, DMARC only addresses specific kinds of abuse.
Post by Steve Atkins
Cheers,
Steve
[1] Flipping a coin was a better phishing filter, both in terms of false
positives and false negatives, than DKIM+ADSP.
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Steve Atkins
2013-03-20 20:49:09 UTC
Permalink
Post by Dotzero
Post by Steve Atkins
Post by Dotzero
I think it would be interesting to see a project that examines
phishing/malware emails to determine whether email authentication
(DMARC/SPF/DKIM) or other practices would have prevented the malicious
email from reaching endusers.
I did that briefly, looking at both phishing attempts for, and legitimate mail from
one particular company, as seen at my inbox.
The results[1] were not what people wanted to hear, so I didn't bother digging
deeper or formalizing the results.
ADSP was an experiment gone wrong.
But it's effectiveness at stopping phishing is pretty much the same as DMARC,
so the experience is somewhat relevant.
Post by Dotzero
Post by Steve Atkins
Unless you're in academia, where you justify your existence by publishing
papers, there doesn't seem much benefit to doing research that won't affect
behaviour.
It isn't always easy to determine what will affect behavior.
It's not, no. But you need to consider that when deciding whether to expend
energy and time on research, with no particular goal and no funding.
Post by Dotzero
Post by Steve Atkins
An end goal beyond the (perfectly reasonable) "it'd be interesting" seems
like something to consider.
The obvious goal would be to determine whether domains such as those
belonging to banks would afford endusers some modicum of protection by
adopting the combination of DMARC/DKIM/SPF.I've seen very positive
results for the domains I have implemented DMARC p=reject for.
Unfortunately there aren't mant case studies out there for people to
refer to.
Could you share those positive results? Actually, more importantly than
the actual results is what you decided to measure, and why.
Post by Dotzero
Having said this, DMARC only addresses specific kinds of abuse.
Defining what those specific kinds are would be part of deciding
what to measure, I guess.

Cheers,
Steve

-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Dotzero
2013-03-20 21:21:28 UTC
Permalink
Post by Steve Atkins
Post by Dotzero
Post by Steve Atkins
Post by Dotzero
I think it would be interesting to see a project that examines
phishing/malware emails to determine whether email authentication
(DMARC/SPF/DKIM) or other practices would have prevented the malicious
email from reaching endusers.
I did that briefly, looking at both phishing attempts for, and legitimate mail from
one particular company, as seen at my ix.
The results[1] were not what people wanted to hear, so I didn't bother digging
deeper or formalizing the results.
ADSP was an experiment gone wrong.
But it's effectiveness at stopping phishing is pretty much the same as DMARC,
so the experience is somewhat relevant.
Assertion without facts in evidence, especially when considering false
positive rates. The other factor being that ADSP never garnered much
adoption on the mailbox provider side.
Post by Steve Atkins
Post by Dotzero
Post by Steve Atkins
Unless you're in academia, where you justify your existence by publishing
papers, there doesn't seem much benefit to doing research that won't affect
behaviour.
It isn't always easy to determine what will affect behavior.
It's not, no. But you need to consider that when deciding whether to expend
energy and time on research, with no particular goal and no funding.
Post by Dotzero
Post by Steve Atkins
An end goal beyond the (perfectly reasonable) "it'd be interesting" seems
like something to consider.
The obvious goal would be to determine whether domains such as those
belonging to banks would afford endusers some modicum of protection by
adopting the combination of DMARC/DKIM/SPF.I've seen very positive
results for the domains I have implemented DMARC p=reject for.
Unfortunately there aren't mant case studies out there for people to
refer to.
Could you share those positive results? Actually, more importantly than
the actual results is what you decided to measure, and why.
I have shared those results at MAAWG, OTA, ISOI and other closed
forums. I'm not in a position to make them public without sign off
from my management - not in the cards because it involves disclosing
some details which are considered sensitive.

Knowing you personally I'll reach out to you out-of-band.

As far as the metrics measured, we look at number of direct domain
attacks (abusive mail claiming to be from one of our domains),
(estimated) volume per attack, volume per attack dropped at domains
providing rua and/or ruf reports, volume of submissions/use complaints
to our customer service or security teams, (estimated) volume to
mailbox providers not validating DMARC.

We also look at/model attacks abusing our domains/brands that are not
direct domain abuse. This would include attacks where some other From
email address is used but the display name abuses our domain/brand,
attacks where the abusive content is only in the body of the message,
etc.
Post by Steve Atkins
Post by Dotzero
Having said this, DMARC only addresses specific kinds of abuse.
Defining what those specific kinds are would be part of deciding
what to measure, I guess.
Cheers,
Steve
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Steve Atkins
2013-03-20 21:24:09 UTC
Permalink
Post by Dotzero
Post by Steve Atkins
Could you share those positive results? Actually, more importantly than
the actual results is what you decided to measure, and why.
I have shared those results at MAAWG, OTA, ISOI and other closed
forums. I'm not in a position to make them public without sign off
from my management - not in the cards because it involves disclosing
some details which are considered sensitive.
Knowing you personally I'll reach out to you out-of-band.
No need - it was the choice of metrics that's the really interesting
bit right now.
Post by Dotzero
As far as the metrics measured, we look at number of direct domain
attacks (abusive mail claiming to be from one of our domains),
(estimated) volume per attack, volume per attack dropped at domains
providing rua and/or ruf reports, volume of submissions/use complaints
to our customer service or security teams, (estimated) volume to
mailbox providers not validating DMARC.
We also look at/model attacks abusing our domains/brands that are not
direct domain abuse. This would include attacks where some other From
email address is used but the display name abuses our domain/brand,
attacks where the abusive content is only in the body of the message,
etc.
Seems like a decent basis.

Cheers,
Steve
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org
Murray S. Kucherawy
2013-03-21 04:08:34 UTC
Permalink
OK, I'll bite.

A couple of years ago I started experimenting with DKIM and domain
reputation. I had a bunch of sites giving me DKIM details, but several of
them have faded away by attrition. I now only have a couple of sites other
than my own feeding me data. I've managed to do some interesting things
with it and presented them at MAAWG a couple of times, but I need more data
and more eyes on the algorithms to keep it moving. If you're interested in
giving me either or both, let me know.

I should say up front that I'm doing this under the banner of a research
non-profit I've formed, and not as an individual. Your contributions will
be credited appropriately but you have to agree to assign some IPR rights,
etc. in order to participate. Those of you who have participated in the
IETF are used to that kind of thing by now anyway.

-MSK
Post by John R. Levine
For the past couple of days, all I've seen is people rehashing their
favorite FUSSPs, with nothing new, and no indication that anyone plans to
do any implementation, research, or anything else.
If you want to have that kind of discussion, the SDLU list which is the
http://www.medwayhosting.com/**spam-l/index.html<http://www.medwayhosting.com/spam-l/index.html>
Unless I see some indication of research interest, e.g., my often stated
desire to collect statistics on DNSxL cache behavior so we can try to
predict how it will or won't work with IPv6, I will shut this list down at
the end of the week.
Regards,
Please consider the environment before reading this e-mail. http://jl.ly
PS: My server, my rules, you know.
-
This is the asrg mailing list. To change your subscription settings, see
http://lists.services.net/cgi-**bin/mj_wwwusr/domain=lists.**gurus.org<http://lists.services.net/cgi-bin/mj_wwwusr/domain=lists.gurus.org>
Continue reading on narkive:
Loading...