SIP: what about the From: header? No love?

I posted a while ago with questions about SIP's STIR/SHAKEN stuff (RFC 7340 has a very good problem statement worth reading) that I became aware of. Well, it turns out that it's really trying to shore up the miserable P-Asserted-Identity mess. I actually kind of like saying I told you so, so if that makes me a horrible person I'll own it. For SIP, P-Asserted-Identity is really nothing more than passing on the PSTN identities (caller-id, e.164 addresses). Which begs the question of whether you can trust their contents. The answer now is the same as the answer 15 years ago, and that answer is... no. The only real surprise is how long it took for $EVIL to figure this out.

For many, many reasons trying to give some guarantees about whether somebody is allowed to assert a given e.164 address is a very hard problem. The new standards have had to deal with this and it's not pretty. Not a knock on the work, it's just that the problem is really, really awful and hard: the PSTN never, ever envisioned the sort of trust model that has become common, or the financial incentives to not care about the problem. This is going to take a great amount of effort to roll out and that's just the beginning. $EVIL is not a static thing, and if I understand correctly there are some pretty significant holes that can't really be plugged.

Which got me to thinking. Why in the hell do I even care about e.164 addresses in this day and age? They are, actually, quite a nuisance. I can barely remember my own phone number, let alone anybody else's. SIP from the very beginning didn't really envision co-existing with the PSTN. It was a new way to use internet mechanisms instead of the inadequate PSTN standards. SIP was just like email, in that it had headers, one of which is a From: header that is identical to email addresses. The idea is that if you wanted to email me, you'd use mike@mtcc.com. If you wanted to call me, you'd use mike@mtcc.com. Simple. It was't until telcos started getting interested in SIP that PSTN integration started rear its ugly head. And hence the sorry situation we're in today.

So let's go retro for a moment. Maybe the original idea of using From: addresses wasn't so bad an idea, and is certainly widely in use today. A lot has changed since the telcos have butted into the VoIP world. For one thing, it's practically extinct. If it weren't for cell phones and the last mile it probably would be extinct. From what I can tell, it's IP the second it hits telco equipment. My little provider here in the Sierra has a gadget that terminates POTS and sends it out as SIP and RTP,  has a DSLAM and backhauls IP over fiber and is battery backed up from CO. Pretty nifty that stuff I've worked on is a block or two away. I think it's pretty much the same for the cellular RAN networks. Since POTS is pretty much dead that just leaves cellular. And if you believe the hype about 5G it will be pretty redundant since it supposedly deals jitter, latency and other things that make VoIP a little dodgy on 4G. I'm not sure of the exact details, but I'll take them at their word that VoIP will be pretty acceptable on 5G. Update: found out that VoLTE is a Thing. So PSTN stuff is now almost completely redundant.

So it's a pretty SIP-y world, and it's about to get a lot more. If I have a SIP UA on my phone, I can completely decouple who provides the bits from who provides the rendezvous services. And I can guarantee you that the telcos are not going to be my first choice. So I may well get my wish that the From: address becomes what people expect on an incoming call, not PSTN anachronisms. So all is good, right? Well, no. Not quite. We still have the problem of spoofed addresses, but now it's put on the From: header instead of the P-Asserted-Identity header.

As far as I can tell (and i could be wrong because there's a mountain of SIP RFC's), there's really not a viable end to end or end to middle or middle to middle kind of way of asserting identity. Yes, I know there is an RFC for S/MIME, but client certs have never seen any wide adoption, and probably never will. And S/MIME is really about end to end crypto which while useful, is not exactly problem that SIP's version of the "caller id" problem is trying to solve.

What we learned with email back in the DKIM days is that end-to-end authentication is a hopeless task. Domain based aggregation, on the other hand, seemed quite tractable. That is a domain can claim responsibility for a particular message (email for DKIM) as having come or passed through its infrastructure. The way we characterized is that DKIM is a "blame me" mechanism if something malicious happened with one of its users. The tradeoff that DKIM  made, however, is that you really don't know if the user part of the email address is who they say they are. But for the purposes of reporting abuse that's not necessary: it's really the sending provider's problem to figure that out. As it turned out, a lot of providers and probably all of the major providers nowadays require SMTP auth. I'm not sure if there was any cause and effect from DKIM to adoption of SMTP auth, but it was certainly in the air at the time.

Now back to SIP. Given the spam we're seeing it sure would be nice to have a "blame me" mechanism to see who injected a particular piece of voice spam into the SIP legs of the INVITE. Reputations can be aggregated at a domain level, and signing policies can be advertised for evaluation of the message. While I might not trust my provider on every front, our interests are alined  when dealing with spam and misuse. Even if I can't verify the incoming INVITE directly (say, you're on a 4G phone), I do trust that my provider can verify it on my behalf and they could stuff the verified message's From: into the caller-id, or somesuch. With VoLTE, they're using SIP so you wouldn't even need to do anything heroic: just show the From: address.

A nice property of this is that the unpluggable holes with e.164 address security aren't a problem in a world that is becoming more and more native SIP. We should be looking forward to that future in addition to any backward looking legacy problems. DKIM has been amazingly successful and extremely widely deployed with tremendous volumes. And since email message structure is the template from many protocols including SIP, it should be pretty easily transferable. In fact, in the day I actually wrote a SIP DKIM signer and verifier just for fun, so mechanically there's not any problems.

There are definitely questions to be answered though: Should verbs other than INVITE be signed? Should the replies? I'm not sure of what benefit there would be to signing REGISTER, for example, but it may be just as well to sign everything regardless of whether it's useful. And then there is the every present  problem of B2BUA's (back to back UA's). Honestly, these aren't entirely different than the Mailing List Problem with DKIM. The answer there is that the entity in the middle that breaks the signature should resign it. And it's probably not as bad a problem as with mailing lists because if I understand correctly B2BUA's are mostly being used as session border controllers which are typically in the same domain as the sender which is typically not the case with mailing lists.

In conclusion while it might be worthwhile to solve the E.164 problem, we definitely need to look to a future where it eventually shrivels up and dies. The future is being able to verify the sending domain of SIP messages, and especially knowing whether the From: address checks out which should be the case in a large percentage of signaling traffic. That would greatly help the voice spam problem since we would be able to reliably blame the sending domain.

The Apple App Store is Security Theater

Apples are sometimes rotten

 

Money, $$$, Money

From its inception the Apple App store has been about exactly one thing: separating you from your money. That's not exactly news, but Apple being the good marketers they are, decided to put a nice bow on their money turd by claiming that they were going to curate the apps and be super careful, and look out for you, and send roses to your mother on mother's day, and, and... they may do some of those things, and the people who do that curation may in fact be honestly trying to do their job to stop bad guys. But in the end, it's all a bunch of hooey, aka security theater.

The App Store Treadmill

In order to understand this, you have to understand what the process is for developers. In a nutshell, your iPhone, etc will not run an app whose code has not been signed by an Apple cert/key. This blocks any other store from being able to sell apps for the iPhone. There are a few exceptions to this (obviously one that allows developers to develop, but also enterprise apps these days too I hear), but the main line loop is: developer develops app, developer submits app for approval, Apple either approves it and it goes up to the App Store, or they reject it and the developer has to fix whatever they are complaining about. Rinse, repeat. My experience with their rejection criteria was mostly that it was petty and small things that didn't have much if anything to do with security. Others may have different experiences. The huge downside is that if they reject your app for whatever reason, you have to resubmit the app for approval once you're done, and...Wait. A long time. If you were fixing a critical bug -- including a security bug! -- tough noogies.

Now Apple has always been very secretive about what their testing entails. We weren't trying to build an app to probe the surface of their security testing, so it's really hard to say what it might involve. Maybe they do find both malicious and unintentional problems. Maybe they find that a lot. Who knows? But there is a gigantic hole, like you could sail the Titantic through it sized hole, that makes all of that testing complete useless, especially if you're a bad guy.

Using WebViews

When we were building the Phresheez app, it became pretty scary to think about writing and maintaining two different UI's. A common language between the two would be very useful. And as it happens there is: javascript. Both Android and iOS have what are known as webviews (UIWebView was what we used on the iPhone). Webviews are pretty much what they sound like: embedded web browsers that an app can display on the phone. More importantly, the app can communicate with the code running in the webview bidirectionally. This is very handy: we wrote the parts of the app that needed be written natively (mostly the GPS handling stuff), and all of the UI in the webview. Portability problem solved.

There's one other important property about all of this. Since it's just web stuff (html, js, css, etc) the app can either get it locally from the app's assets, or make a request to some server somewhere, or both. So we bundled up all of this stuff up into a zip ball on our backend servers, and the app would know how to go fetch the zip ball and download it into the webview. Remember about all of that waiting for reviews? Problem solved.

The Hole

There is nothing nefarious about what we did. We had perfectly legitimate reasons for doing this, and Apple does not have a policy against this. I'm not sure they could ever have a policy banning that because that's just the way the web works: you can just add script tags in the html that point to a remote server and that's perfectly legitimate. How does this make all of Apple's so-called vetting security theater? Well if it's legitimate to load javascript code from an external server from a good guy, it's legitimate to load it from an external server from a bad guy. And a clever bad guy could even go to the trouble of cleanup their malware while they are in App Store review, and then switch back to the evil code after.

Wouldn't that be against Apple's policies? You bet. But they would be none the wiser until after the app was deployed. Which is the exact same situation that Apple fanboys love to make fun of Android. You put your evil stuff in javascript, run the webview, PROFIT! Note: the webview doesn't even have to be part of your UI... it's just a vehicle to run javascript if that's all you want to do.

The Pooch, She is Screwed

This feature has been in iOS from the very beginning, and there are tons of hybrid web/native app packages out there. To close this hole would be to break every single one of them, and for that matter probably break every app that uses a webview at all. So they realistically can't do that, and even if they did it would be catastrophic. Apple really doesn't need to spend much if any resources behind the curtain to preemptively root out bad guys. Any bad guy worth their salt would already know this trick. If Apple does spend lots of resources, it's just a marketing expense.

This is why I've been saying that so-called vetting is all a bunch of security theater. This was always a marketing thing and nothing more.

Don't Eat Webauthn, it's Made of HOBA's!

So I recently heard about a new W3C standard called Webauthn. From what I can tell it's very much in the spirit of our HOBA RFC (RFC 7340). Ours was more of a sketch of what could be, and my javascript implementation of one of the alternative ideas in the RFC was a truly horrible hack and I knew it, even if I did try to justify some of the sketchier aspects of it. But it was 2012 and the working group didn't even spin up till 2015, so we were clearly ahead of our time.

It works pretty much like my javascript implementation, but with much better crypto, credential storage, etc, etc. But it also goes on to specify interfaces to signing dongles which is good for what it's trying to do, but I'm seriously worried that people are going to think that webauthn *requires* portable keys, biometric, etc. That would be a shame because the real enemy is not passwords, per se, it's passwords that are transmitted over the wire. And of course, their reuse. A single strong local password which is never put on a wire is perfectly fine to gain access to credentials. That's pretty much what we do today with password managers, browser fill-in-the-blanks, etc.

The reason I think that's bad is because I worry that site owners doing their own authentication will view it as niche if they think it's required to have something other a local password. Why would Epicurious implement this if they thought it was just going to be the slice of people who have access to these things? Remember, not everything is done on phones so counting on, say, biometrics only works for some of the use cases. That would be a very bad result, as webauthn using a local credential store password is a perfectly fine thing to keep others out of my super secret recipe box.

One of the things I did with my part of HOBA is elaborated quite a bit on the enrollment problem. On the tutorial sites I found, nobody seemed to mention it. I can definitely understand why the standard itself considers that out of scope (it is), but it's a little odd that it's not getting talked about much that I've found because it's probably the hardest part of deploying webauthn server-side. Maybe my Google foo failed me, but dealing with enrollment always sucks. Since webauthn is device specific, it's going to take some guidance on how to implement it server side, and users getting used to a new routine. Fortunately on the user side, we've come a long way on enrollment with more and more sites validating that the use of a new device is ok. With webauthn, that would have to be universal.

All in all, I think this is great stuff and about time. The scourge of passwords on the wire has a real chance of going away in time as sites adopt it. Let's just not hitch it to getting rid of passwords altogether. All forms of credentials have their own set of issues and tradeoffs. The important part is keeping them off the wire.

Stir/Shaken Questions

I've been looking over STIR/SHAKEN stuff for dealing with Spam-o-SIP. There's a really nice problem statement (rfc 7340) which outlines all of the interactions with the legacy PSTN and its interactions with SIP gateways and session border controllers (SBC). The gist of the resulting set of RFC's presents a pretty complicated architecture with Policy Administrators (PA) (I assume something like the ITU or some other such creature) which does admission control of who is allowed to join the telephone number bearing club, delegated CA's to sign club member's certs, and then a sort of peculiar over-the-wire signing and verifying service within each provider. From my understanding this is all within a SIP network -- anything beyond a PSTN gateway is terra incognita if the gateway switch is relaying it from a source it does not control. But that's just a previously unsolved technical problem, I think, and the layer 8 solutions from the past may still be effective, re: Caller ID. Now I must fess up that I only recently heard about this, so my characterizations are likely to have flaws or be downright wrong, so take this with a grain of salt.

Plain Old SIP

My first question is whether this is intended to be a solution for plain old SIP URI's as well? If so, it's sort of worrisome to have his PA entity determining who can join the club for an all-IP service. Binding telephone numbers to particular carriers and making certain that the carrier is part of the telephone number club is a much different problem space than SIP-SIP with no telephone numbers involved. Assuming that is not intended to solve for that problem, is there a way to identify either the end user URI, or aggregated at provider boundaries on, say, a domain basis? With DKIM, we chose to allow mail providers to sign the mail with their domain -- whether or not they are the originating domain -- to say "blame me".  This gives the originating provider some incentive not to be blamed by requiring MUA authentication, etc.

RFC 7340 talks about SIP Identity (rfc 4474) and I can certainly understand the issue with B2BUA's which have analogs in the email universe (cf mailing lists). With DKIM we essentially punted and declared that the breaking gateway should sign the mail anew, and that they are now the one to blame. This seems to have worked out ok, as far I can tell. Dealing with spam is all about messy heuristics and I expect that SIP spam will be no different. The general idea is to be able get enough clues about bad actors as part of the full set of clues.

DKIM for SIP?

It has always seemed that a DKIM like approach would work for SIP as well. It has the nice property that it doesn't require a centralized CA. I'm too lazy to look up SIP identity, but I'm guessing that it does require them in some form. The web got lucky with TLS because there were only a very few browsers and the accepted CA root list was controlled by them before things could get out of control. With SIP, it's been around a long time and SIP Identity didn't get much traction from what I read, so I'm not sure if that's part of the problem, or is just academic since it just didn't get deployed.

DKIM for SIP would be fairly trivial to implement. In fact, I hacked together a SIP proxy that signed and verified SIP messages. And, of course, the infrastructure is all there as DKIM at this point is really old and really widely deployed. If SIP Identity doesn't get used for whatever reason, and we don't have a deployed solution for SIP-SIP calls, that seems like a really big hole. Fortunately, we have an existence proof with email which could be retrofitted pretty easily.

Is this Partly a UI Problem?

When I receive a call, my phone does the best it can to tell me who it is rather than giving me just a plain old telephone number. In fact, that's one of the best ways to determine whether it's spam is if you don't recognize the number. At this point, we are all very comfortable using email addresses as identities. So what I'm getting at is why doesn't my phone app take plain old SIP calls and treat them equivalently? In which case, it can use the From: address rather than some Paleolithic telephone number. There's nothing stopping Google or Apple from adding a new SIP protocol head onto their existing call apps, after all. This would at least break the notion that legacy telephone numbers are the only ID I should see when I get an incoming call. Hopefully the legacy PSTN calling will be winding down in the next 10 years or so, but we certainly don't want to carry along legacy telephone anachronisms into the future of an all SIP infrastructure. 

Signer and Verifier Services

This is more of a quibble, but I really don't understand why there needs to be a separate on-the-wire protocol for SIP proxies and authentication. With DKIM, the signing and verifying are done in the MTA, and it demonstrably works just fine. The CPU cost of signing these days is minuscule, and verification was always nothing. Plus, it costs time to send something over the wire and get a response, and with SIP that is a consideration. So I just don't get what motivated this.

Conclusion

While I think this looks like good work, I'm worried that it's putting a lot of time and energy into dealing with a legacy issue, and perhaps either not dealing with the SIP-SIP case, or dealing with it in a way that requires being invited to a club.  Either would be very bad. We really should be trying as hard as possible to make plain old SIP the way people expect to get a "phone call", and SIP itself needs to be able to have a workable solution that doesn't depend in any way on legacy PSTN stuff or we're going to be in the same boat a few years down the line.

Recursively Recursing the Burstorm App

Where I've Been

It's been 3 years since I've written to this blog, but I haven't been asleep again. In fact, I've been working furiously on an web app for a startup called Burstorm. The Burstorm app implements a CAD-like graphical interface for designing IT infrastructure, for both traditional datacenter providers, but most especially for the new-world cloud providers like AWS, Google, Azure, etc. The idea of the app is that you can specify in fairly abstract terms what your requirements are for some deployment (eg, I want 2 Linux boxes with 4 cores, 16GB ram, and 200GB SSD), and the app with provide you with insight about what your choices are to take your model and actually spin it up.

The app has lots and lots of goodies like collaborative editing, importing of cloud provider billing files, the ability to do traditional provider quoting, as well as an entire provider-centric side for creating and managing products both programatically as well as UI-driven.

This post is mainly about me eating my own dog food and telling the world that it was, in fact, tasty. So here's the story. About 3 months ago, I got word that we needed to move providers because a customer was insisting that we run in a ISO-27001 compliant datacenter, and our then-current provider wasn't compliant. When I stopped being annoyed, I decided to use this as opportunity to do something with our app that I'd wanted to do for a long time: use our app to model, build and deploy itself on other cloud providers. Happily, I for other reasons I had almost all of the pieces built in the app to do just that, and this provided the perfect excuse do the last bits and dregs of code so that I could recursively recurse the Bustorm App.

IT Lifecycle

The Burstorm app models the life-cycle of an IT project. 

• Models: when you want to just create a new design of some infrastructure and figure out who can provide it where and for how much, we call that modeling. Here, you lay out what your requirements and constraints are for your infrastructure, and you can then use the Burstorm Designer to get recommendations of providers that meet your requirements, what the cost is, and for many cloud providers even get price/performance specs on their various VM's through our performance benchmarking initiative.
• Builds: when you see a provider's offering in the Designer that looks interesting, you can create a Build from it. A Build in the Burstorm lifecycle is much more specific: it includes pricing from a particular location for a particular provider. And for cloud providers that we support, it even includes information about how your Build can be spun up.
• Contracts: you can also tell the app about infrastructure that you already have deployed with its pricing information, contract terms, etc. The app can give you help to figure out when it might be better to jump ship to another provider, as well as lots of other insights. Additionally, you can copy your current infrastructure into a Model, and start the whole process over again to redesign your current infrastructure with various tweaks you might want to make.

Our App Setup

Enough background, onward! The Burstorm app is a fairly standard web-like app using Ruby-on-Rails, Postgres and Linux on a bunch of different servers: some that are dedicated like our PG master, some that have dual roles like our postgres-slave/webhead, and some that are just webheads. We also use hosted Chef  to provision all of our servers, using Chef roles to determine what pieces of config/packages need to go on what servers. 

"Burstorm App" Model

So in order to achieve our goal of getting the Burstorm app to clone itself, we start by creating a model in the Burstorm app. This illustration shows a simplified view of our infrastructure.

Fig 1. Development and App model

In it, you can see a Design Scenario called "Main App" in the Model which contains the new app. You'll also notice that there's a separate Design Scenario called "Devbox" -- this models a development environment we spin up for our devs. 

By clicking on the Design Scenario "Main App", you can see that we've set its constraints to be within 5000 miles of Dallas, with month-month terms, with flexibility about upfront NRC.

We can now inspect one of our servers. You can see for pgs (our Postgres Slave) that it is set up to be a 2-core, 4GB ram, 200GB storage VM.

Note also, that you can see bunch of tags that are associated with this Objective. In fact, those are the Chef roles that will be used to provision this server should we decide to spin it up!

Here you can see a complete list of provisioning tags which correspond to all of the Chef roles we use for all of our app infrastructure. 

"Main App" Designer

The Burstorm Designer allows you to drill down on a particular design scenario and get all kind of interesting information including costs for various providers and how/where they'd deliver it, price performance information, various charts that show you provider information, histograms, etc. For the Burstorm App cloning the Burstorm App, this is the meat: we're going to use the designer to figure out which providers we are interested in.

In this view, you can see that AWS, and Google look pretty favorable. If we drill down on the details, we can see for both AWS and Google what VM instances they would use to deliver our servers, and how they differ from what we asked for: Google's instances, for example, provide quite a bit more RAM than we asked for, so it's not just the base cost that needs to be considered.

Note also that we're comparing these by Price/Performance. That is the BCU column for each provider. What's cool is that we can change that to sort things by Price alone, CPU performance, IO performance, etc so that we can really get an idea of how these VM's compare on other axes. 

Now that we've figured it all out, we can now move to the next step by copying these solutions to a Build.

Building Our App

We've now created a set of Build Specs which we can use to communicate with the various providers. In this example, we've chosen Amazon and Google, but when I was working on this I actually spun up clones of our app on Azure, Linode, and Digital Ocean too.

asdf

We do need a little setup here to tell the app the credentials to use, how to log into our VM once it's spun up, what provisioning system to use (if any), etc. Here we're spinning up an AWS  t2.medium instance, using Chef to bootstrap the box with the 'benchmark' SSH key to login with.

Spin-Up

So now we're ready to spin up the Burstorm app from within the Burstorm app. Each server objective contains a Run Spec which tells the app all of the relevant parameters, like what the instance name is, the provider's location spec, ssh keys to use, and whether to run Chef after the instance is created. Here we will spin up our AWS Build Spec, and clone the entire setup.

What the app actually does now is hands off a JSON blob describing the Build and Run specs off to a remote web server. This would be running in your infrastructure. This implementation doesn't  require us to have access to your AWS keys since they'd be running on your webserver. Other arrangements could be envisioned though to make this more seamless. Either way is viable, and we're still thinking about what the best way to support this would be (if not both).

Our general strategy has been to define an API endpoint for the Burstorm App that allows programmatic access to our models, builds, product sets, etc. Our Perfrun repo takes that approach, and is the basis of the code to do the actual instance-creation/provisioning. Looking at the repo, you will see that it's just using our API like any other API consumer. In this case, it's creating VM instances, and optionally running our performance benchmark code on them.

If you squint real hard, you can see that we are doing a normal Chef bootstrap. The key is that the Spinner-upper takes the Chef roles we set up with the Burstorm tags above and uses them to provision the servers with each specific environment. This all happens in parallel, by the way, so it only take 5 minutes or so till it's complete.

Todo

There's *tons* more to do. We have the ability to import AWS and other provider's billing files, but it would be interesting to correlate them with what we spun up. It might be fun to map our utilization too against the actual work load. There's all kinds of places this can go, but this is, after all, just a prototype which is intended to spark those kinds of discussions.

C'est fini

Once this is done, we have 4 servers up and running. In the subclassed version I use of the Spinner-upper code, I actually have it communicate with our DNS provider to set up the A and AAAA records, and thus can directly access each server by name. In reality, there are a few more bits and pieces I had to do -- like loading the database -- but almost everything was handled by our Chef.

So there you have it, I used our Burstorm app to clone itself.  The actual process of doing this start to finish probably took an hour or so, if that -- way less time than it took to write this over-long post!

Interviews as Hazing Rituals

Aggro Interviews Making Up for Um...

I started this blog as a tongue in cheek kind of play on the fact that I'd been so out of the web world for so long that when I came back to look at the web, everything was so fresh and minty. Well, it turns out that that's not the only thing that has changed since I took my long snooze. I've been putting myself out again to look for a new job and I've been pretty shocked at how awful the interviewing process has become. It's become a hazing ritual straight out of your local frat house.

At some level this shouldn't be a big surprise because the entire industry -- at least what I'm seeing here in SF -- is being taken over by brogrammers. Call me old school, but I really don't want a keg on every floor and I don't see it as a feature that you use more space for ping pong tables and other toys than you do for space for your employees to work, assuming that they ever do that. And what's up with using 8' banquet tables with two people per table? It makes me want to look for severed fingers in my software because I'm surely looking at a sausage factory instead of software development.

The Greek System

But back to interviewing. One of the biggest changes nowadays is that everybody seems to be using keyword filtering software. This software doesn't appear to be especially bright as it just seems to do an intersection between the job requirements and the words that appear in your resume. If it isn't an exact intersection, you're rejected. It doesn't seem to discriminate between required, strong preference and nice to have either. If there isn't an exact match, then you're dropped. This seems especially prevalent at big companies where they force you to fill out an extensive questionnaire because, I guess, their filtering software isn't even smart enough to do the intersection function using your raw resume text. Worse, I've seen some that require you do that before you can see their job listings. I mean, seriously?

 

The net effect is that it's fairly useless to actually apply for a job because it never gets seen. Because, of course, there does not exist somebody who actually is an expert in the 99 things that you put on your listing as "requirements". So the unfortunately lesson here is that just dropping a resume off in the job inbox is a complete waste of time.


First, drink 10 beers


The biggest cancer -- and the subject of this rant -- are tests in the form of programming tests and puzzles. This is nothing more than a hazing ritual dressed up in all sorts of self-righteous excuses. The justification is that candidates might be lying about their programming prowess, thus it needs to be conclusively proven that you can write code on the spot. Never mind that this is extremely uncomfortable for the interviewee. Never mind that what is invariably given to be written you'd be fired for reinventing rather than just googling for the answer. Never mind the fact that most interviewers can't actually explain the problem they want you to solve. Never mind the fact that once you solve it, they want you "scale it up" whatever that means.

None of this matters because the point isn't to find out if you're a great programmer. The actual point is to convince the interviewers that they're great programmers. And superior to you in every way. Unless, I suppose, you look like them and have an affinity for drunken programming. The best part of this is when they tell you "we just want to see how you think". Call me crazy, but the only part of my thinking process you're going to understand after one of these tests is how good my rote memory is. I mean, when is the last time you've been asked to write a program to enumerate all permutations of a number? When is the last time you coded up a binary tree (no fair if you've been interviewing recently).

The absolutely worst are companies that use their hazing rituals -- like frats -- to prescreen pledges before they even a) talk to you and b) give you any clue whether you want to work in their sausage factory. This is abuse, pure and simple. And the larger joke is that it's been shown that it serves absolutely no purpose other than making the frat boys judge their penises large and manly. This google paper really drives this home, and I felt very vindicated.

We're not sure if you look like a Theta Tau

Every company has a "culture". Every company is completely convinced that their culture is the best. The interviewer's job is to make certain that the wrong element is not let into their restricted and exclusive frat ("exclusively what, and restricted to whom?"). You better be, well, top drawer just like Bunny Bixler and Muriel Puce.

Now there are cultural reasons why you may not be a good fit, like it being a stressful job where the interviewee is looking to destress (and vise versa!). But let's be real here for a second: every company says they only hire the "best and brightest" but not a one of them is located at Lake Wobegon. How, exactly, does that work? Or is "culture" a dog whistle for "looks-like-me"? And didn't Enron pretty much prove that hiring exclusively for "best and brightest" isn't a guarantee of success? Which isn't to say that trying to get good people should be downplayed -- not at all. It's just that looking into your sausage factory I have to assume there must be some third world country where everybody is white, male, 25, and dresses in skinny jeans to escape poverty by being exceptionally smart. At companies who do exceptionally amazing things like making sepia filters for .jpg's, and virtual row crops for e-farmers.

The PI-Thon Frat House

Computer languages are tools. Often complicated tools to be sure, but tools nonetheless. They do not describe your life essence. The ability to use them is not written into base pairs where some people inherit them, and some people don't. You do not need to give blood and do a PCR test to determine whether you are Java negative. If you're worth anything as a programmer you can and will pick up lots of languages over your lifetime. Some of them may actually be pretty awful and will make you apologize (hi PHP!). Some of them are more applicable to some problems than others.

The same thing goes for platforms and packages/frameworks. The current rage is to silo you based on whether you're iOS or Android, but it's no different than when they used to do that between M$ and Unix. Now true enough it takes time to learn a new language or platform or package/framework so it's a valid consideration to judge how much time a candidate might take to come up to speed. And true enough, there isn't much that you could do to convince me to waste my time learning .NET or some other piece of crap language/platform/package that I have no interest in learning. So, yes, thank you for putting down all of the buzz words so that I can filter too.

But where you go wrong is thinking that the lack of 7 years of mobile experience (seriously, I'm not making this up; edit: remember this is in 2013) means a priori that an otherwise competent person can't pick it up. In fact, part of the fun of a new job is getting exposure to new and different ways of approaching problems. Competent people break free of the frat they pledged to. Really.

Will nobody think of the code?

One of the most shocking things I've found interviewing is that -- except for the filters -- nobody seems to read your resume. And because they haven't read you resume, they obviously haven't seen anything online to validate your skill. The great thing about the net is that there are so many ways to show what you've done in the past, and whether it's a piece of art or a piece of crap. And since it's a given that the interviewer is smarter than Donald Knuth (even if they don't know who he is), he should be able to instantly assess whether your code is written well and whether the design choices were appropriate. Right? I've had exactly one interviewer ask me for code.

Now I understand that people lie, but it seems to me that using their own purported code to see how well they really understand what they claim they've written would be an immense help to ferret out the fakers. Much better than a real time rote memory test of coding up 2-3 tree or a lalr parser. Now this isn't failsafe because it can be perfectly legitimate for a person to not have an oeuvre that is publicly available. But to not use it as a resource when it's available? That's craziness. 

I'll pass on the spanking conga line

The Google HR guy's interview (and this one too) which says that the only thing that makes much of a difference in interviews are behavioral questions was quite a relief. I hadn't actually woken up in the alternate world where Spock and Kirk wear beards. Of course their revelation hasn't filtered out to the larger tech world, so there are still plenty of fucked companies with all of their interviewers who are looking for reassurance about their manhood. Oddly enough, it's the way I've always interviewed people too so it's doubly comforting that something I did by instinct was actually the only thing that seemingly works worth a damn. 

The last thing I'd like to say is this: instead of being paralyzed by making a mistake, do the obvious. If they can't program after you hire them, then... just fire them when you figure it out. It usually doesn't take too much time to find out somebody who interviewed well was a complete joke once on the job. And no, don't require they initially "consult" or whatever -- you're going to lose good people who view that as game playing. That and get rid of the kegs. Seriously.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

MVC or Not?

With Phresheez, I've noticed a pattern that comes up over and over and over again given that most Phresheez pages are not web pages, per se, but long lasting apps that auto-refresh content dynamically. I've tried to reconcile this with the MVC (model/view/controller) pattern which is quite common, but I've never been able to tell whether the way I think of things is the same or even close to the MVC pattern.
I'm thinking that they have similarities, but that they're not really the same as MVC as far as I understand it.

So here's what I saw coming up all the time with Phresheez.

1. There's a dynamic piece of data that needs to be fetched from the server
2. There's a piece of HTML content I generate from that data
3. That content needs to be refreshed as new data comes into the server
4. I want to use that content in multiple places, but they may want to be formatted for different uses

Typically, I'd start out with a monolithic piece of code that did all of things, but I'd almost always regret it as I'd have to extend or refactor the code when I wanted to use some or all of the widget in a different context.

The rest of this post is what I'm thinking the architecture should be.

Goals

First off, I guess I should state what I want to provide.

• A data abstraction layer that takes care of all of the grotty details of network errors, retransmissions, refreshing, etc, etc. Before refactoring, this code was ~duplicated all over the place each with its own subtle set of bugs. I want just one set of bugs.
• A cleanish separation between content formatting and container decoration. By container decorations, I mean things like menus/nav, titles, last updated, etc. 
• I want to be able to facilitate mixing and matching different widgets. That is, I want to be able get content from widget A, widget B and widget C and mix and match them into a new super widget D.
• A recognition that there's some interplay between some of the aspects and not go crazy trying to ferret out every soi-disant layering violation
  

Data Layer

The Data Layer is in charge of getting and refreshing data between the client and the server. Other services subscribe to the data layer to get and refresh data from the server. The data layer may or may not auto-refresh depending on settings, but it will only refresh if something is actively subscribed to it.

The Data Layer should also have error callbacks for when something goes wrong. The Data Layer must never affect the DOM in any way: it is up to its subscribers to determine what is displayed.

There may be many Data Layer objects running around with different parameters to the same underlying service. For example, you might have an object that fetches the user information for user1 and another that fetches it for user2. When the Data Layer publishes the data, it must be specific to its instance.

As an optimization, however, it would be good for the Data Layer instances to know about each other such that an instance that has the same url parameters as another will share the server's results with other Data Layer objects so as not to generate useless duplicated calls to the server. I suppose that one way to accomplish that would be for them to subscribe to each other, or maybe a global data cache layer on top of the server callbacks, but that's an implementation detail.

Container Layer

The container layer is the business end of this model. It alone is responsible for inserting/updating elements within the DOM. The normal data flow is that the Container Layer subscribes to some set of data layers (or not!) and upon a published update calls the necessary content binding objects to populate the container (normally a div, but it could be anything's innerHTML really). I'm going to say that it's also its job to deal with layout and general look and feel for the container since a container may contain many objects. So it produces HTML as well, but it's much more oriented at layout than content per se.

The container layer is normally the recipient of the Data Layer's publications. It is responsible for painting the placeholders when the data is not yet available (eg, loading spinners, etc), and coordinating when it's interacting with multiple Data Layers. The Container Layer is also responsible for unsubscribing to the data layer when it has lost focus, for example, to cut down on useless chatter to the server. 

Finally, the Container Layer is responsible for implementing any callbacks (eg, onclick) that the container content requires. Which is another way of saying that the Container Layer owns the interaction with the DOM.

Content Binding Layer

The content binding layer binds data from the server to html layout and content. It never interacts with the  DOM. It merely takes the data, typically provided by the data layer, but not necessarily exclusively and grinds out the content for that data. The Content Binding layer should be thought of as effectively "stateless" even though for efficiency the Content Binding Layer ought to provide one other service: sameness. That is, if it produces the same content that the caller was returned the last time, it should inform the caller. This is needed to eliminate useless redraws at the container layer which are annoying. Note that if the content layer needs to do extensive massaging of the dynamic data, it is not a layering violation to hold onto the dynamic data (or a massaged form of it) if it wants to do delta's of old and new datasets, etc. 

It could be argued that the sameness property ought to be up to the Container Layer, but I don't think so. The Container Layer is the one that knows which bits and piece of dynamic content its using, so it alone is the one that knows if they are the same, modulo comparing blobs of HTML at the Container Layer, which I think is bad practice.

Clicks and other Events

It's not entirely clear to me who should be generating the onclick='s (typically). In the MVC pattern, I'm pretty sure that's the job of the controller, which would be more or less equivalent to my Container Layer. But that would be rather messy in practice: if you're just producing a bunch of html in the Content Binding Layer, it's much easier to insert the onclick's inline with the generated HTML. The downside here is that at the very least the onclicks need to be parameterized with the instance they represent from the Container Layer (ie, this click is associated with this container instance). And of course, the click actions across different container layers may well be very different.

What I think is probably the best compromise is let the Content Binding Layer generate the onclicks, but with input from the Container Layer's state. This still runs afoul if you want something like

<a href=# onclick="st.mywidget.clickHandler('mike')">Click me!</a>

since it is the container layer's job to handle that click, so why should the Content Binding Layer be setting up its callback parameters? Suboptimal, but in practice it haven't noticed many instances where I have one use with a set of parameters and in another use with a different set of parameters... they almost always follow a similar pattern, though you may end up with the union of parameters the various uses require.

My guess is that not stressing about this fuzziness and the apparent layering violations is best. The object here is to get better code reuse and separation of function, but that doesn't mean that we have to solve world hunger as well.

Inserts/Updates

For insert/update type callbacks to the server, I think I would keep it completely as a property of the Container Layer. This is the flip side of the argument about clicks in the previous section: the Content Binding Layer is the one that generates the input's, checkboxes, etc, so why shouldn't it provide the interface to fetch their values? I dunno, because it's sort of fuzzy in the same way that clicks are fuzzy and it would probably be messy to have a grand unified theory here. So some give and take between the Container Layer and Content Binding Layer seems reasonable.

Likewise, I don't see any particular advantage to tasking the Data Layer with the ajax callbacks to the server to save/update state. The point of the Data Layer is to deal with multiple users of data, and keeping it refreshed. It doesn't need to be the singular owner of HttpXMLRequest as well.

So is this MVC?

So have I reinvented MVC or not? Maybe it is, or maybe I've extended it a little, or something. Or maybe it's all different. You tell me. Here's my take on what I've done vs. MVC: