|Not the solution to everything
I'm fairly certain I had a basic understanding about how certificates for identity worked, though not much about the underlying technology before 1998. But in 1998 all of that had to change really quickly because I opened my mouth about the security problems for the residential voice over IP project I was working on at Cisco and in particular the signaling protocol we were using a called MGCP (nee SGCP). MGCP is a pretty simple command/response protocol where a server tells a home POTS (eg phone) gateway to, say, go off hook, or ring the ringer, etc. Needless to say having some script kiddie being able to ring the ringer or listen in on the microphone would not be ideal. For opening my mouth I got told to solve it. So there I was having to do a crash course on network security and all of its protocols and really how it worked at all.
My group in particular was tasked with creating the residential gateway which was a box that had a couple of POTS ports and was integrated together with either a cable or DSL modem. These needed to be authenticated both ways so that the service providers could prevent rogue gateways getting access to their telephone network. In this case the gateway is the client device in a client/server relationship. Normally clients use passwords but that doesn't seem especially elegant for a box sitting in the corner, though now that I think about it that is exactly what my router does when connecting using PPPoE to my ISP. There was a requirement that the user wouldn't have access to the gateway so that would have made it more difficult, and especially for manufacturers if they had to pre-provision the secret keys.
So it was time to learn about asymmetric keys. Well, rather the first thing to learn about was certificates because that's how they always got couched. Certificates were these magic identity thingies that through some math voodoo allowed the other side of the conversation to know who they were talking to. Once you had a certificate all of that math voodoo became mostly irrelevant so I mostly concentrated on them rather how asymmetric keys actually work. To give an understanding of how clueless I was at the time, I remember asking another engineer whether we could just RSA sign the MGCP packets or something like that. Looking back that seems like a silly question to ask, but as it turns out it was exactly the right question to ask with signing email for DKIM just a few years later.
So everything was in terms of certificates, how to get them onto the box what to do with them once they were there and how this all related to keeping kiddie scripters from ringing my phone in the dead of night. Some of my previous group were working on IPsec so I got up to speed with that and it seemed like a good solution to the crypto needs for our residential gateway security problems. Though TLS (then SSL, I think) was definitely in the air back then, MGCP was a UDP based protocol, and TLS only works on TCP (though now integrated with QUIC). I was persistent on this point in the SIP working group too -- SIP could also be run over UDP -- because I thought IPsec in transport mode was a better choice since it dealt with UDP as well. Instead, others went off and designed DTLS to meet the UDP requirement. Oi. The irony now is that SIP is so bloated that it wouldn't even fit in an MTU packet anymore, so we were both "wrong" in that deprecating UDP would have been the better choice.
Now that we had an underlying crypto mechanism it was back to getting those certs on to the gateway and what were these certs anyway? The general idea was to have a root CA which vouched for approved manufacturers (I was by that time participating with Packetcable, Cablelabs' residential voice standardization project). To me this was still all rather mysterious and something of a black box. I finally started to grok the larger picture when I was in a meeting with Van Jacobson and he said "ah, the enrollment problem". We didn't have a certificate problem, we had an enrollment problem. How do you enroll those devices such that the server knows who is who? That is the basic problem going on, and in the race to solve it with certificates nobody asked why they were needed at all.
That's sort of how it always seems go when people start talking about using certs as if they were some magic incantation and The way you used asymmetric keys. Nobody asked why we needed to bind a name to a public key in the first place. I finally started to understand the underlying math and how IKE worked and especially how RSA signing and encryption worked. Being able to determine who you're talking to doesn't require having a key bound to a name at all. The public key itself is unique and can be used directly as an identifier itself. Certs completely obscure that property. Later when Jim Fenton and I designed IIM which is one of the precursors of DKIM we took advantage of that property and just used the public keys as an identifier itself. It was DK that had a somewhat gratuitous name/key binding in the form of selectors, but it didn't hurt anything and allowed me to have a selector name called "fluffulence".
So why do I like to bag on certificates? Because they confuse getting to the bottom of what you're trying to do. Like I said, since everything is very certificate oriented, nobody asks the obvious question of why do you need a name to key binding? In the Packetcable case I recall us struggling with what exactly the name should be in the cert. That right there says that first principles almost certainly need to be revisited. We didn't have a naming problem, we had an enrollment problem and the name was irrelevant and thus there was no requirement to carry it using an obscure and ossified bag of bits in the form of X.509 and ASN.1. The other part that I dislike about certificates is that they are a
business model. It costs nothing to put a public key into the DNS or in
some database. It costs enough to support lots of CA vendors' bottoms
lines for certificates though. There is one use case that certificates can do that are not easy to reproduce in other ways: offline verification. This was an important use case when they first arrived since expectations of online was a rare beast in the 80's. Today the need for offline verification is niche and the whole world is a connected internet. So we're supporting billions dollar business model for a feature almost nobody uses.
When our residential voip project was going on over 20 years ago it might be somewhat justifiable because a whole lot of us were getting a crash course on network security. However I don't really think much has changed on that front. Everybody proceeds from a cert-first mindset as if it were a given without thinking about what the actual requirements are and then deciding whether a name/key binding is needed at all, and next to determine how that binding is achieved if needed. It's also unfortunate that so many protocols have a built in expectation that certificates must be used, though it's my understanding that TLS and IPsec both allow for naked public use obviating the need for actual valid certificates. I'm not sure if they implement it just by sending a self-signed cert where the server just ignores the CA signature, or whether it truly is a means of just sending a naked public key (the latter would certainly be better since the intent is clear).
In the VoIP/Packetcable use case, client certificates were never needed. The naked public key (or a hash of it) was perfectly serviceable as an identifier for the residential gateway. All that needed to happen was to get it enrolled somehow. There are many ways to do that depending on security requirements. Dispensing with the complex X.509 infrastructure makes the entire problem both easier to administer and much simpler to understand. It should be a dead giveaway for anything that proposes client side certificates to ask why they are needed. In the wild, client side certificates are exceedingly rare, so why is this different?
The reason I decided to write this post is because as of the writing I was having a conversation in which I voiced my dislike of certificates and most especially the certificate-centric view that most people have with authentication with asymmetric keys. I had brought up SSH which along with DKIM are two of the most used tools that use asymmetric keys (TLS being the most), neither of which use or need certificate based identity. Somebody pointed out that SSH allows for client certificates, so I looked it up and it seems that they hacked the protocol to get that to work and that apparently it is used as a replacement for the SSH authorized_keys file on servers which is supposedly better at scale. When I pointed out that it would be easier to just put the SSH public key into the user's profile with a LDAP directory or some such, I got told that it was infinitely easier to create a certificate and put it on the client. Since both have to upload the public key to something, that cancels out. How can putting certificates on a client be easier than to doing nothing at all? Magic, I guess. Or confusion. Lots of confusion.
The moral of this story is to not start with certificates as a given if you are thinking about using asymmetric keys for authentication. That just confuses everything. You need to understand what problems you are trying to solve first and foremost. What are the requirements for authentication? Do those requirements require a key/name binding? Do those requirements need the ability to verify authentication when the verifier is offline? If the answer to both of those is yes, then you should consider using certificates. If the answer to offline is no, then you don't need certificates and it can be designed without them by using naked public keys. Simplicity is always good with security. Certificates are not at all simple and should be used only as necessary.