Or “please don’t make my life 100% easier, 80% is enough”

I had an interesting argument recently in a very crowded meeting – maybe we were all getting that purple meeting haze, but it started to take on all the makings of a holy war (so I knew it would make a good blog post).

We were discussing an API for interacting with a server cloud and the API was intentionally very abstracted.  Specifically, you could manage a virtual server but you could not see which host was providing the resources.  The vendor wanted to hide the raw resources from API consumers.  This abstraction was good; it made the API simpler and allowed the provider to flexibility about how it implemented the backend.  The API abstraction made the underlying system opaque.

So far it was all rainbows, unicorns and smiling yellow hatted yard gnomes.

Then I wanted to know if it was possible to relate information between the new API and the existing resource transparent API.  Why would I want to do that?  I was interested in the 5% case where we needed to get information about the specific resources that assigned.  For example, when setting up redundant database replication, we want to make sure that they are not assigned to the same physical hosts.

More importantly, I do not want the vendor to clutter their new abstracted API with stuff to handle these odd ball use cases.  Calling them 5% use-cases is deceptive: they are really in the hugely diverse bucket of use-cases outside of the 95% that are handled nicely the abstractions bucket.  Trying to crow bar in these extra long tail use-cases will make the API unwieldy for the intended audience. 

Someone else in the meeting disagreed with the premise of my question and wanted me to explain it.  In answer, I used the tautology “Abstractions are useful, until they are not.”

The clearest example of this concept is the difference between Rails ActiveRecord and Hibernate.  Both are excellent object-relational (OR) abstractions.   The make the most general cases (select * from foo where ID = bar) quick and easy.   But they are radically different at the edge of the abstraction.  ActiveRecord expects that programmers will write directly to the database’s native SQL to handle the 5% exceptions.  Hibernate added the complex and cumbersome HQL on top of their abstraction layer.  HQL is nearly (in some cases, more) complex than the SQL language that it tries to abstract.  For Hibernate, this is really an anti-abstraction that’s no longer useful.

 Over stretching an abstraction encourages the wrong behaviors and leads to overly complex and difficult to maintain APIs.  When you reach the edge of an abstraction, it’s healthy to peek under the covers.  Chances are that you’re doing something wrong or something unique enough that you’ve outgrown the abstraction.

And that’s enough for now because blog posts are useful, until they are not.