Purify'd scheme(s) -- a contribution to the `GEL = SCM + RMS' debate

[Matthias Blume <blume@CS.Princeton.EDU>]

Hi all,

I just got a chance to play with Purify (a commercial product for
`purifying' C programs).

And just for the heck of it I thought -- why don't I try applying it
to SCM...?  So I went and did it.  The `Purify'd' SCM was then used to
run some of the Gabriel benchmarks.  Here is what I found:

CPSTAK  :   Finished  scm                  (11492 errors,    0 leaked bytes)
CTAK    :   Finished  scm                  (1724633 errors,    0 leaked bytes)
DERIV   :   Finished  scm                  (14834 errors,    0 leaked bytes)
DIV-ITER:   Finished  scm                  (47766 errors,    0 leaked bytes)
DIV-REC :   Finished  scm                  (7169 errors,    0 leaked bytes)
FFT     :   Finished  scm                  (   0 errors,    0 leaked bytes)
FPRINT  :   Finished  scm                  (   0 errors,    0 leaked bytes)
FREAD   :   Finished  scm                  (1951 errors,    0 leaked bytes)
TAK     :   Finished  scm                  (10104 errors,    0 leaked bytes)
TAKL    :   Finished  scm                  (78712 errors,    0 leaked bytes)
TAKR    :   Finished  scm                  (13992 errors,    0 leaked bytes)

(no typos -- I used cut&paste from the Purify window...)

I didn't wait for SCM to finish the CTAK test, because I figured that
1.7 million errors are enough to make my point.

The two benchmarks with no errors didn't use call/cc and were too
short to trigger even a single run of the garbage collector.

To give you an idea of what Purify is looking for I give you an
expanded view of what happened to the `puzzle' benchmark (this has
almost the fewest errors, because it only runs for a very short period
of time).

----BEGIN OF PURIFY OUTPUT----
   Finished  scm                  (1951 errors,    0 leaked bytes)
      Purify instrumented scm (pid 6331 at Wed Oct 26 17:10:00 1994)
      UMR: Uninitialized memory read (9 times)
      UMR: Uninitialized memory read
      UMR: Uninitialized memory read (276 times)
      UMR: Uninitialized memory read (45 times)
      SBR: Stack array bounds read (80 times)
      SBR: Stack array bounds read (12 times)
      BSR: Beyond stack read error (5 times)
      SBR: Stack array bounds read (96 times)
      BSR: Beyond stack read error (5 times)
      SBR: Stack array bounds read (96 times)
      UMR: Uninitialized memory read (11 times)
      BSR: Beyond stack read error (55 times)
      SBR: Stack array bounds read (1056 times)
      SBW: Stack array bounds write (166 times)
      This is occurring while in:
            dynthrow       [sys.c:907]
               # endif /* def STACK_GROWS_UP */
                       FLUSH_REGISTER_WINDOWS;
                       src = (STACKITEM *)(CHARS(cont)+sizeof(regs));
            =>         for (j = LENGTH(cont);0 <= --j;) *dst++ = *src++;
               #endif /* ndef CHEAP_CONTINUATIONS */
                       throwval = val;
                       longjmp(JMPBUF(cont),1);
            lthrow         [sys.c:917]
            ceval          [eval.c:883]
            ceval          [eval.c:608]
            ceval          [eval.c:660]
            ceval          [eval.c:583]
      Writing 4 bytes to 0xf7ffdb44.
      Frame pointer 0xf7ffdb40
      Address 0xf7ffdb44 is 4 bytes above stack pointer in function ceval.
      UMR: Uninitialized memory read (9 times)
      UMR: Uninitialized memory read
      SBW: Stack array bounds write (14 times)
      UMR: Uninitialized memory read
      UMR: Uninitialized memory read
      UMR: Uninitialized memory read (12 times)
      Current file descriptors in use: 5
      Memory in-use: 8 bytes (40% of total allocated)
      Memory leaked: 0 bytes (0%); potentially leaked: 0 bytes (0%)
      Program exited with status code 0.
----END OF PURIFY OUTPUT----

Before I started to do these tests on SCM I did them on some other
Scheme implementation (one, which I have a more vital interest in) --
I guess I don't need to say which one that would be.

For this (other) implementation Purify came up with one source code
location that produced an error -- this was due to a genuine bug in
the source.  After fixing this I ran the benchmarks from above,
test.scm, conform.scm, and various large byte-code compilations --
all with the same result:  0 errors, 0 leaked bytes, 0 bytes of
memory in use after completion.

I leave the detailed evaluation of these figures to you.  I myself can
give these recommendations:

	1. If you are a C programmer, then give Purify a try! (I'm not
	   affiliated with Pure Software Inc.)
	2. Don't use SCM for critical applications until it has been
	   CAREFULLY overhauled.  Don't turn it into a (*the*) GEL, before
	   this has been done!
	3. Don't close your eyes in front of other choices!

Disclaimer: I found Purify on the machine I was using to be somewhat
unreliable in some cases.  So take everything I said with a grain of
salt!

Thanks for your attention,