Modules vs. dynamic binding

[cph (Chris Hanson)]

   Date: Thu, 7 Dec 89 15:16 PST
   From: Gyro@Reasoning.COM

   Consider this scenario: we have a system function READ-GEN that takes
   arguments like BASE, READTABLE (or whatever we call it), etc. and
   returns a procedure suitable for binding to READ; i.e. one might write
   something like

     (let ((read (read-gen 10 scheme-readtable ...)))
       (read some-stream))

   That is, we make it possible for READ, in different **LEXICAL!!**
   contexts, to read using different parameters.  Simple enough, yes?

   All that remains to make this usable, seems to me, is to give us a way
   to establish bindings, such as this one for READ, outside entire sets of
   top-level definitions.  That, as I understand it, is a primary purpose
   of a module system.  At least, I hope that when we get around to
   designing our module system, we'll support this kind of thing.
   (Actually, I think what we really want is generator procedures that
   generate entire modules from a single list of parameters; but this
   discussion can wait.)

Scott, this technique you propose is a useful one, but I don't believe
it provides a replacement for dynamic binding.  The problem with the
technique is that the procedure `read' cannot be embedded in another
program unless the embedding program is also parameterized in a
similar way (by "embedded" I mean that the variable `read' is bound in
some closing environment of the embedding program).

To properly take advantage of this technique (as you point out), every
embedding program must be parameterized, and furthermore the
parameters of the embedded program must also be parameters of the
embedding program.  That's the only way we can specialize programs
_after_ they've been built and embedded in other programs.  Ultimately
this means that every time a new program is built it must be
specialized with all of the "dynamic" variables of all of the
procedures that it calls before it can be run.  Clearly nobody wants
to write all of that specialization by hand, and yes, all this "late
binding" can be hidden by a module system; but what about the program
that occasionally specializes itself as it runs?  Does the module
system know how to create new programs on the fly, or is it restricted
to describing more static relationships?  (I'm used to thinking of a
"module system" as something that describes the static relationships
of a program's parts, not their dynamic relationships -- although if
there's a strict formal definition of "module system" I'm unaware of
it.)  Ultimately this "module system" becomes an integral part of the
language -- it has introduced a new kind of binding with "dynamic"
characteristics.  What are the semantics of this binding?  Is it
different from what we now think of as "dynamic binding"?  Is this
really a new idea or just a new implementation of an old idea?

Another solution to this problem was invented by John Lamping a few
years ago: he introduced another kind of variable that could be bound
after the program was constructed.  These variables can be considered
to be "holes" in the program, which can be filled in at any time by a
virtual-copy process that produced a new program in which a particular
set of "holes" was filled.  Program fragments can be combined into
larger program fragments, independent of whether they have "holes";
"matching holes" in two program fragments become identified in the
combined fragment.  With this solution, `read' would have such a
variable for its radix, and this variable would be unspecified when
`read' was defined.  You could then embed `read' in some big program,
and just before you were ready to run it, you could specialize the
embedding program with the desired radix.

Lamping's solution has some major advantages over what you propose:

  * The embedding program doesn't have to know anything about what
    "holes" `read' has.

  * The values being supplied to the "holes" can be collected
    together in an environment and supplied all at once.  A "default
    environment" can easily be created, and the user can override
    particular bindings in that environment by shadowing.

  * It is clearly a change to the language itself rather than a
    system that at least nominally is not a fundamental part of the
    language.

The problem with this solution is that it is a _major_ language
change; it affects the semantics at a very basic level.

But are either of these solutions really so different from dynamic
binding?  You claim that dynamic binding is destructive to modularity,
but with the implementation proposed by Pavel the designer must
clearly state the intention that a particular parameter may be
dynamically bound, so the parameter is clearly part of the program's
abstraction boundary (if you can get your hands on it).  The only
other way I can imagine harming modularity is if the user dynamically
binds some procedure's variable, then calls an embedding program which
uses that procedure in two different contexts, only one of which would
like to see the dynamic binding; this would be an inadvertent capture.
But I think this could happen with your solution or with Lamping's
solution as well; the main advantage of your solution here is that the
explicitness of the parameter binding forces the designer to see that
the procedure is being used in two different contexts.  A good
designer will see that anyway, because the abstraction spec of the
procedure will contain the information.

I think this topic is an interesting one that falls between the
cracks: it's a little too dynamic for modules, and not dynamic enough
for procedural arguments.  I'd like to see some work done on
understanding this; I suspect that Lamping's work is a good first step
(implementing his language efficiently is quite challenging).

On the other hand, I have a strong feeling that dynamic binding is the
best solution to this problem for Scheme -- a better solution may
require changes that are fundamental enough that the result may no
longer be "Scheme-like".