Are Mailing Lists Toast?

Definitely Toast

 

From the very beginning when IIM (Cisco's email authentication draft) was merged with DK (Domain Keys from Yahoo!) to became DKIM, we both envisioned a sender signing policy module which allowed a domain so say "we sign all of our mail, so if you have unsigned mail or the signature is broken that's purportedly from us, that's not cool". Since we were all experienced with internet standards it was plain as day that there was a serious deployment problem since mailing lists mangle messages and thus broke signatures. Mailing lists would thus throw a wrench into the policy gears. This was 16 years ago.

Our effort at Cisco was driven primarily by phishing, and spear phishing in particular. We had heard tell of some exec at another company falling for a spear phishing attack, and we didn't figure our execs were any more clueful so that was pretty frightening. Since Cisco had exactly no presence in email at all, it also gave some plausible deniability as to what we were up to. We weren't looking to get into the email biz, but we weren't not looking either. 

We formed a group which included Jim Fenton and me, and created a project to sign all of the mail at Cisco with a goal of being able to annotate suspicious email purporting to be coming from Cisco employees. This required finding and signing all legitimate Cisco email. So off we went trying to find all of the sources of unsigned email in the company so that we could route it through the DKIM signing infrastructure. We didn't have the nifty reporting feature of DMARC so it wasn't the easiest thing to figure out. It was made much worse because Cisco had tons of acquisitions so there was a lot of legacy infrastructure left over from them, and who knows whether they were still using their mail servers or not. This was very slow going to say the least.

Most of the DKIM working group was pretty cavalier and dismissive about the mailing list problem. A DKIM signature from the mailing list would somehow solve the problem. We just needed to somehow trust that mailing list. Coming back after a dozen years and a lot of skiing under my belt, it seems that the previously unsolved problem remains unsolved: nobody knows how to "trust" a mailing list.

When I was still at Cisco I used a bunch of heuristics trying answer the question of whether the originating domain signature could actually be reconstructed and verified. It was met with a lot of derision and hysteria from the usual attack poodles, but I didn't care and kept trying to improve my recovery rate ultimately achieving northward of 90% recovery. It was interesting and tantalizing that the false positive rate was close enough to be worth considering marking suspicious mail up with warnings. The next step would have been to differentiate what that broken signature traffic was, where it was coming from, what it was doing that was not reversible and ultimately whether we cared about that case enough. There was no silver bullet that we could find, and we definitely didn't know how to  "trust" mailing lists.

So as we slogged on with hunting down our infrastructure and me still hacking on the heuristics, Cisco decided that it was interested in the email security angle that we had pioneered. We did diligence on a bunch of companies and settled on Ironport just down the peninsula from me. Cisco bought them and my group it was with decided -- without my input -- that they were switching to some wacky telephony thing that I had no interest in. When Ironport wouldn't transfer me, I wandered around a while and then decided it was time to quit and ski. The one thing I regret is that we didn't get a chance to finish off the research side of the problem, especially with mail lists to separate out the enormity of the remaining problem which seemingly nobody knows to this day.

Fast forward 12 years and I found this curious creature called and ARC-Signature and ARC-Seal in my email headers. The signature looked pretty identical to a DKIM signature which I found very odd, and it also had an ARC version of the Authentication-Results header. So what's going on? I found that ARC is an experimental RFC which was seemingly trying to solve the mailing list problem. Again. But using even more machinery in the form of the ARC-Seal. What is the Seal's purpose? I determined it is so that it can bind the ARC auth-res to the ARC signature. Why are they doing that? Because it's supposed to be of some value to the receiving domain to know what the original assessment of the originating DKIM signature was. But DKIM can already do that if a mailing list resigns the email. It was all a mystery.

So wondering whether there was some secret sauce that I had completely overlooked all those years ago, I posted on the DMARC working group list for the first time asking what that secret sauce might be. "It requires the receiver to trust the mailing list" I was told. I said that you can do that right now if the mailing list resigns the mail with DKIM, which I assume most do at this point (and screw them if they don't). Why does this require a different signature than a DKIM signature? They wanted to bind the ARC auth-res to the signature I was told. Why can't you just add a new tag to DKIM assuming that is a problem at all, which I am not very convinced it is? Never got a good answer for that. And most importantly why does the receiver care about the original auth-res in the first place? Never got a good answer for that either.

So this all boils down to trusting a mailing list at its heart. It's not clear to me whether some receivers are using the list's DKIM signature to bind it to some whitelist or reputation service. Somebody as big as Google could easily roll their own and just not tell anybody about it since it's completely proprietary just like the rest of the spam filtering. So on the outside we really don't know whether mailing list reputation is a Thing or not, but the assumption that the working group seems to be operating on is that it is not a Thing and remains a previously unsolved problem. I am willing to believe that assumption since it seemed like a hard problem all those years ago too. That and Google itself is participating with ARC, so that suggests they aren't any better off than anybody else. But who knows? That's part of the problem of being opaque is that nobody on the outside can scrutinize whether there is some magic thinking going, or whether there actually is some there three.

So here we are over a decade and half after DKIM's inception right back where we started. As far as I can tell, ARC doesn't bring much of anything new to the table and the original auth-res doesn't address the fundamental problem of trusting the mailing list or not. Whether the original signature verified on the list or not seems completely beside the point: if I trust the mailing list, why do I even care whether it verified or not? If they are forwarding messages without valid originating signatures, that is a very good reason to not trust them for starters. Any reputation system needs to take into account a lot of factors, and requiring signature verification at the mailing seems like table stakes.

Mailing lists. Again. We are completely wrapped around the axle of being able to provide pretty good forgery protection from mail from malicious domains, but domains can't seem to pull the trigger of asking receiving domains to toss mail without valid signatures because it would cause mailing list traffic to be discarded as well. There have been other drafts beyond my experiment on signature recovery that have not been well received, and have languished as probably they should. The worst part of this problem is that there is no way to determine what success looks like. How many false positives are acceptable? How can we assess quality and quantity? Cisco, for example, would grind to a halt if it's upper level engineers working on IETF standards ceased to work because they implemented a p=reject policy (I just checked, it's p=quarantine 0% percent which is basically p=none with a little bit of attitude). 

Been toast a little too long

So pretty much we're in the same trenches as we were from the beginning with no forward progress, and no forward progress on the horizon. It then caused me to question the unthinkable: are mailing lists actually worth saving? They are ancient technology that never had security constrains from the beginning and they are operating in a world where security is a Must requirement. Likewise, it's not like there aren't alternatives to mailing lists. Web based forums have been around for decades, so it doesn't even require new infrastructure. Lots of things that once were a Thing are now gone. Take Usenet for example. Usenet was revolutionary in many ways and provided the first thing we would recognize as social media on the nascent internet. But it's been sidelined by the new social media companies, but one of the biggest problems is that it couldn't adapt to spam for whatever reason. Probably not technical and more likely neglect, but it is basically dead now. The world moved on, and now it's just a relic of the past. It's not even that the new tech has to be particularly good: Reddit is the most similar to Usenet of the social media platforms and it is a terrible and buggy reimplementation of Usenet, yet it is popular and Usenet is done.

So are mailing lists a relic of the past too? Going by the total volume of email, it sounds like they're down in the noise from what I've heard -- like 1% -- but it would be good if some of the big providers stepped up with some concrete numbers. For many companies, perhaps most, losing access to external mailing lists wouldn't even be missed at all. Us in the internet community are profoundly attached to mailing lists because that's how business is done. But let's be serious here: we are complete outliers. Changing over to some off the shelf forum software while not trivial, is certainly well within the capability of internet community if it needed to. The same for other lists. It quite possible that a hybrid system could even be done in transition where mail is gateway'd to the forums or some such.

In conclusion the bad security properties that mailing lists and the like are causing better security to not be deployed. So yes we should just ignore mailing lists and let the people who run them them adapt however they feel fit. After 15+ years since the advent of DKIM-like technologies and the ability to determine who is sending mail and discover what that domain desires on receipt of mail that doesn't verify, we need to just move on and accept that the desire for verifiable mail from domains is more important than a legacy technology with ample alternatives. 

This is not to say that mailing lists need to be burned to the ground or anything drastic. I've noticed that some lists have taken to re-writing From headers seemingly from domains with a DMARC policy other than "none". This is pretty awful from a security standpoint because it further trains people to rely on the pretty name rather than the email address, but to be fair that is just building on an existing problem and MUA's are the real culprit of a lot of this since they do little to help people know who they are actually talking to. That said, none of this would be necessary if we used a more modern technology.

Sorry mailing lists. But it's come to this.