Cost of R5RS macros (was: Fyi.)

[jar@cs.cornell.edu (Jonathan Rees)]

   Date: Wed, 7 Apr 93 16:18:21 -0400
   From: "Aubrey Jaffer" <jaffer@martigny.ai.mit.edu>

   Are you referring to the low-level SCM builtins:

     (procedure->syntax <proc>)				procedure
     (procedure->macro <proc>)				procedure
     (procedure->memoizing-macro <proc>)			procedure

   or to the syntactic closures and syntax-case packages in SLIB?

To what's in SLIB.  Sorry if I was looking at the wrong thing.

   As for the SLIB packages, I posted notices more than once asking for
   portable macro packages.

Wish I had more time to work on things like that.

   I got no replies so I had to port syntactic
   closures myself.  I have since then asked if anyone had a macro
   package which didn't expand all the way down to primitive expressions
   and also got no replies.

I don't know what you mean by "expand all the way down to primitive
expressions".

      For your purposes -- assuming you still really really want to stick
      with a pure s-expression interpreter --

   Yes I do.  It is small and fast and it is not really pure (it modifies
   Scheme expressions as it evals).  Your apparent incredulity is the
   sort of comment that motivated my last message.  I think one of the
   attributes that shows a language has expressive power is that it can
   be implemented in various ways.  I hope that some other authors share
   this view and will not write SCM out of R5RS.

No, I agree on philosphical grounds, just not on practical grounds: I
don't really understand your practical motivations for wanting an
s-expression interpreter.  (I feel like pointing out that the fact
that you're passing the code through READ already constitutes an
information-losing pre-pass.  If you're concerned about the
information loss aspect of macro expansion, why not retain pointers to
the source code *text*?  That would be much more useful for debugging,
in any case...)

   As for interleaving expansion with evaluation, that is what I
   tried to do, but I am having problems with internal defines.  In order
   to find out that a macro will return DEFINE or BEGIN of DEFINEs, I
   have to expand it before evaluation.  This requires an expansion
   prepass which replicates the environment tree ...

Why does it have to replicate the environment tree?  You don't need to
do recursive macro-expansion, only single-expression expansion.

   Another possibility is to have internal defines be a SCM primitive
   form.

Probably a good idea.  The internal define problem doesn't have
anything to do with hygiene, does it?  You run into trouble
even with old-style non-hygienic macros, if you allow macro forms
that expand into definitions.

   In order for this to work, I have to be able to augment the
   environment of a body without knowing if this is the first DEFINE in
   the body or not.  I am investigating this but I think there are some
   screw cases like (letrec ((x 3)) (define x 5) x).  If I can get this
   to work, it has the additional benefit that macros can be expand-only;
   I.e. I don't have to combine DEFINEs into a LETREC.

Are you only worried about the overhead of introducing an extra
mutable, initially null environment contour for LETRECs?  That seems
like a pretty small price to pay.
Again, this also has nothing to do with hygiene.

   There is not much difference in the output from the macro expanders
   (syntax-case and syntactic closures) as they both output primitive
   scheme expressions.  The reason this slows down SCM execution 30% is
   that it doesn't allow SCM to execute the derived forms directly.

Isn't that true of any macro system?  Would you prefer to not have
macros at all?

   I have looked through pseudoscheme.  In derive.scm it appears you are
   expanding forms down to the primitive expression types.  If this is
   the case, hygenic macro expansion will not slow down your execution.

Sorry, I'm still baffled by the distinction you're trying to draw.

Pseudoscheme only avoids introducing macro forms in macro output for
the sake of efficient expansion; there is no inherent reason for what
it does.  And it is not completely consistent; e.g. LET* expands into
LET.

   One way that hygenic expansion screws small implementations is the
   large number of symbols it generates.  Each symbol in SCM occupies 3
   cons cells (symbol header + link in hash bucket + value cell= 12 bytes)
   plus a malloc()ed string.  This storage does not get GCed; It stays
   allocated until SCM is exited.  I don't have a figure on this yet, but
   I expect it will make a big difference on a 1Meg IBMPC, where Jacal in
   SCM still fits.

The objects that represent "generated names" in macro expansions do
not need ot be symbols.  Neither Pseudoscheme no Scheme48 generates
any extra symbols.

   The solution I would like to see would be one where the renaming and
   macro expansion take place only when a macro is defined and in the
   application of the macro.

Could you explain what you mean by this?  Sorry if I'm being dense.

Jonathan