Metaclass's Lament

So after what seemed like an unreasonably-long time, long, perhaps, because my life did not depend upon it, I think I finally figured out what "inversion of control" and "dependency injection" are supposed to be. Some commenters on the topic, whom I have read, have volunteered doubt about the referential transparency, so to speak, of these labels, and I guess I have to agree.

Have you ever felt inconvenienced by hard-coding the use of "new" for some particular class? Do you wish somehow that the kind of thing you get back didn't have to be statically declared like that, but that it could merely satisfy some interface instead?

If so, you might have used a factory pattern.

This approach provides some alleviation, but unless it's built into the language, requires you to create factories for your types, which seems very boilerplatey, which is certainly a great nuisance compared to reaching for "new". Moreover the approach only provides partial separation of interface from implementation. At best, you've pushed the problem of "new" back to factories.

Now, I ran into this issue many, many years ago, but it already had a name. It was the issue of "virtual constructors". If the fresh faces that labored insightfully, and with our gratitude, to provide dependency injection frameworks, have made all due diligence in the engineering history of the matters addressed, and (more to my point) if the notion of the discovery of a new problem, a new solution, and the need for a new nomenclature for the whole business, are all better warranted than is apparent, then I'm somewhat out of my reckoning, and offer amends.

The "dependency injection" solution basically uses language reflection to allow one to provide, let's say in Java, some declaration of dependencies among one's interfaces and classes, and those dependencies can even themselves be dynamically bound. So now you have virtual constructors of a sort, and even virtual virtual-constructor constructors, if you like. It's clearly a contribution worthy of the praise it has garnered, and I don't disparage the engineering achievement in the least -- though I might not rush to use it. My complaints pertain more to history, terminology, and overlooking of opportunities.

Now, I knew there was something in Smalltalk regarding types that Smalltalk fans were very pleased with. After hearing about "dependency injection" and beginning to understand what problem the technology was simultaneously solving so nicely and obscuring so thoroughly (behind a certain amount of, well, "new-speak"), I finally got around to figuring out what was putatively so good about the Smalltalk type system. Typing in Smalltalk sounded horrendous and abstract as far back as I can recall, and I mostly just ran screaming from it. But in fact, my current guess is that it's the minimum amount of abstraction required to artfully address the issue that it addresses, an issue that keeps coming back to haunt the designers of new languages, so long as they ignore it.

And at this point, I kind of wish the designers of Java had done the same thing as the designers of Smalltalk with regard to typing. When you consider all the heavy lifting they *did* do in producing something nearly as useful as C++, but that is about one one-millionth as painful to use, they surely could have done, despite the discrepancy between the dynamic typing of Smalltalk and the static typing in Java.

To get virtual object construction, instead of calling "new", which is surely a static operation scoped by the name of the relevant class, you want to talk to an object-constructor and ask it for an object. But since you want *virtual* object-construction, you really want to talk to an *interface* for an object-constructor, and let the actual object-constructor be whatever it happens to be at the time of construction, late-bound.

Unfortunately it sounds like we are on the precipice of an infinite regress, since we still need to construct object-constructors, and then object-constructor-constructors. But not to worry. A chain of dependencies that does not terminate, but also that is not infinite in length, eventually reaches something that depends only on itself. Note that in Java, classes are objects, and in fact the class Class, which is an object, is a member of itself. This is where Java's type-dependency chain loops back upon itself.

In Smalltalk, as in Java, every object is a member of some class. And when an object gets a message, that message goes up the class hierarchy to find the first (i.e. most-derived) implementor. That's just what it means for methods to be virtual. Classes in Smalltalk are also objects (because everything in Smalltalk is), however they are also something more than that. Each class is a member of an appropriate metaclass, and the metaclasses form their own hierarchy distinct from that of the classes. Perhaps this seems a bit abstruse and even silly; perhaps it seems that we're just abstracting upward arbitrarily as a way of kind of showing off. But stick with me, and hopefully you'll agree that it's the exact right amount of silliness when all is said and done.

When a class gets a message, like "create a thing of your type" (or maybe, "create an array of things of your type", or "create a set of things of your type", if we permit the notion of objects parameterized by collections rather than the usual vice-versa *), then that message goes up the *metaclass* hierarchy. The nicely-balanced insight of the Smalltalk designers was to realize that after these two levels, you can basically stop. Because the higher you go in this fashion, the more unlikely it is that you'll want to change behavior dynamically. At the same time however, if you stop short of this level, as happened in Java, you wind up with no language-native means of having virtual constructors. The Smalltalk entity called metaclass is a singleton (meta-)class which is a member of itself, and this is where Smalltalk's type-dependency chain loops back upon itself.

To try to put this Smalltalk ontology in Java-like terms, consider that one could create new objects not by appealing to the static-"new" method of some class, or even to a factory (a notion for which there is no native-language support in Java, and thus factories are just ordinary Java artefacts that require their own engineering), but to a "metainterface" (hence the type-safety that Smalltalk lacks) that specifies what metaclasses will do for you, without constraining their implementations. Maybe, in the case of construction, they'll provide you objects fresh from RAM each time, maybe they'll come from a pool, maybe they'll be disk-based, or whatever. And consider this capability in the form of native language support so that it is just as convenient as reaching for "new", and avoiding the imposition of boilerplate development or boilerplate frameworks.

The last thing to say here, I think, is whether the extensibility introduced by Java annotations could have been -- or could still be -- applied to produce a virtual-constructor regime as straightforward (all things being relative) as that of Smalltalk, thereby leveraging a long history of thought about the issue, as well as some fine exemplar technologies. One could consider the use of annotations in the creation of "dependency injection" as kind of a constructive proof, maybe a metaclass-Mark-I. I just can't shake the feeling that something about one one-millionth as painful to use is in the offing.

* The introduction of type-parameterization in C++, i.e. templates, somehow managed to fail to notice and extend the fact that the language had already introduced objects parameterized by collections as opposed to the reverse. One parameterizes "int" to be "int[]" in order to say, "type integer, but an array of those". Java carried over from C++ both this feature, and the ignoring of it, when generics appeared. I actually don't have a strong feeling about whether it's better to parameterize objects by collections or the reverse -- the former seems more "objecty" to me, but might ultimately be more awkward -- but the notion makes for interesting examples of the kinds of things that meta-classes might like to do.