This file documents new and changed features of version 14 of the Scheme Runtime Library.  Parts of this document will eventually be merged into the MIT Scheme Manual.

Copyright © 1988, 1989 Massachusetts Institute of Technology

This material was developed by the Scheme project at the Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science.  Permission to copy this document, to redistribute it, and to use it for any purpose is granted, subject to the following restrictions and understandings.

 1. Any copy made of this document must include this copyright notice in full.

 2. Users of this document agree to make their best efforts (a) to return to the MIT Scheme project any improvements or extensions that they make, so that these may be included in future releases; and (b) to inform MIT of noteworthy uses of this document.

 3. All materials developed as a consequence of the use of this document shall duly acknowledge such use, in accordance with the usual standards of acknowledging credit in academic research.

 4. MIT has made no warrantee or representation that the contents of this document will be error-free, and MIT is under no obligation to provide any services, by way of maintenance, update, or otherwise.

 5. In conjunction with products arising from the use of this material, there shall be no use of the name of the Massachusetts Institute of Technology nor of any adaptation thereof in any advertising, promotional, or sales literature without prior written consent from MIT in each case.

Summary
*******

This document describes the new and changed features for version 14 of the Scheme Runtime Library.  What follows is a summary of the notable changes.  Extensive changes are further documented in the Scheme Reference Manual.

The changes incorporated in this release of the runtime library were prompted by our attempts to compile the previous release using the Liar compiler.  Many of the design decisions, including various strategies for packaging and choices of data structures, caused the resulting compiled code to be less efficient than we thought reasonable.  For example, our implementation of I/O ports as first class environments placed severe restrictions on the compiler's ability to optimize that code.  In some cases the problems went beyond efficiency: occasionally the runtime library reflected on data structures (such as procedure objects) in a way that depended on the interpreter's representations. In these cases compiling the code changed the representations, thus violating assumptions built into the reflective code.

Of course, once we had started making wholesale changes to the runtime library, it became easy to make changes unrelated to the compilation issues.  In fact, most of the changes documented here are in that category.  We have tried to limit the number of incompatible changes, but in general when conflicts arose between compatibility and clean design, the latter won.  Most such changes are in the more obscure parts of the runtime library and hopefully will not affect the average user.

Major Changes
=============

  * The error system has been completely rewritten.  Errors and other exceptional conditions are now signalled through a general mechanism similar to that implemented on the Lisp Machine.  This mechanism provides the programmer with the ability to specify the action to be taken when a particular error occurs.

  * I/O ports have been completely reimplemented.  The new implementation represents ports with vectors for efficiency, and defines a standard interface for defining and selecting custom operations on ports.

  * A new graphics interface provides generic support for two-dimensional line and point graphics, with simple text labelling.  Device-specific support for colors and various other features is implemented.  The devices supported in this release are X windows (version 11) and HP Starbase graphics.

  * The stack parser has been redesigned.  The new parser provides all of the information available on the stack, and makes few assumptions about how the information will be used.  The debugger is being redesigned to take advantage of this.

  * The debugger now understands compiled code, and shows almost exactly the same information for compiled code as it would have for the same code, interpreted.

  * The read-eval-print loop has been redesigned.  From the user's point of view it is very similar to the old one, but the programming model has changed significantly.  The state of a REP loop is now stored in an object, called a repl.  This allows programs to reflect on REP loops more effectively.  Repl objects are built on top of cmdl objects, which are a primitive form of command loop.

  * The global package structure of the runtime library has been changed. An experimental packaging system has been implemented to support the runtime library.

New Features
============

  * New special form default-object?, should be used when testing the value of an optional argument to determine if that argument has been supplied: it returns #F in that case.

     At present this is identical to unassigned?.  In the near future the handling of optional arguments will be changed.  When that happens, there will be a special class of objects representing default values for optional arguments.  Defaulted optional arguments will be bound to one of those values, and default-object? will be a procedure which tests for that case.

  * A new facility for walking over SCode trees has been implemented, to replace the previous type dispatcher facility (see SCode Walker).

  * New 1D weak association tables.  This abstraction is a conveniently packaged weak alist facility.  The table is implemented as an alist of weak pairs.

  * New procedures have been defined to support booleans.

  * New procedures have been defined to support weak pairs.

  * Simple support for multiple values has been implemented.  Current implementation is low-tech but adequate for many purposes.  In the future the performance of this construct will improve considerably, (most likely) without change to the interface (see Multiple Values).

  * Interned symbols are now stored internally using lower case.  This means that they print out in lower case as well.

  * The pretty printer pp will now accept structures defined by define-structure as arguments, and print their component parts out by name.  It also accepts non-negative integers as arguments, treating them as hash numbers and printing the hashed object.

  * The reader syntax #@DIGITS refers to the object whose hash number is DIGITS.

  * Bit strings now read and print using syntax similar to Common Lisp.  The syntax is #* followed by a string of binary digits.  However, the order of the digits is opposite that of Common Lisp (MSB first, making it similar to #b), and a length prefix is not allowed.

     Right now this is provided for user convenience.  There is a good possibility that it will change in the future, probably to be more compatible with Common Lisp.

  * A variety of standard unix system calls are now built in:

          (file-attributes FILENAME)
          (file-directory? FILENAME)
          (file-modes FILENAME)
          (file-modification-time FILENAME)
          (file-symbolic-link? FILENAME)
          (file-touch FILENAME)
          (file-writable? FILENAME)
          (get-environment-variable NAME)
          (get-user-home-directory USER-NAME)
          (set-file-modes! FILENAME MODES)
          (unix/current-file-time)
          (unix/current-gid)
          (unix/current-uid)
          (unix/current-user-name)
          (unix/file-access FILENAME AMODE)
          (unix/file-time->string TIME)
          (unix/gid->string GID)
          (unix/system COMMAND-STRING)
          (unix/uid->string UID)

  * The variable prime-numbers-stream is bound to a stream of prime numbers.  This is useful for a variety of things.

  * New user procedure re is like in (previously %in) except that it re-evaluates the input expression in the current REP loop environment and syntax table.

  * New procedure (exit) asks for confirmation before calling %exit.

  * New procedures allow convenient user prompting which is handled specially by the Emacs interface.  They return (respectively) a character, boolean, and an s-expression.

          (prompt-for-command-char STRING)
          (prompt-for-confirmation STRING)
          (prompt-for-expression STRING)

  * New procedures provide information about whether a given object is a pointer or non-pointer object.  Every object satisfies exactly one of these predicates.

          (object-pointer? OBJECT)
          (object-non-pointer? OBJECT)

  * New procedure (show-time THUNK) executes THUNK, then prints the time it took and returns the value it returned.

  * New variable load/default-types is bound to a list of pathname types.  These types are tried sequentially when the load procedure is invoked on a pathname with no type.  The variable fasload/default-types provides an identical facility for the fasload procedure.

  * New procedures pathname-default-COMPONENT have been defined for all pathname components.  These are like pathname-new-COMPONENT, except that they do not override an existing component.

  * New procedure init-file-truename, which takes no arguments, returns the pathname of the user's init file, if it exists, or #F.  This procedure searches the working directory and the user's home directory, in that order.

  * New variable unspecific is bound to an arbitrary value.  If the usual declarations are used, the compiler assumes that the value of this variable is unimportant, and may substitute any convenient value.  This is useful for declaring that the value of a procedure (or other expression) is not useful, and for improving the efficiency of compiled code.

Incompatibilities
=================

  * The hooks for defining custom unparsers for tagged objects have been changed.  The new implementation passes two arguments to the unparser: an unparser-state object, and the object to be unparsed.  Various new output routines are available for writing unparsers, which use the unparser-state object.  Also there is a standard unparser which handles many common cases.

     Compatibility procedures are available for supporting existing code, but these will eventually be deleted.

     See Custom Unparsers, for more information.

  * The procedures there-exists? and for-all? have had their arguments redefined.  Previously they accepted a predicate and returned a procedure which accepted a list to scan over.  Now they accept two arguments, a predicate and a list, and return the appropriate boolean value.

  * Pathnames now have a host component.  At present this is not used.

  * The procedures pathname-extract and pathname-extract-string have been deleted.

  * init-file-pathname, previously a pathname-valued variable, is now a procedure of no arguments which returns the pathname.

  * The unbound? special form, and the associated SCode type, have been deleted.  In the near future, once the new argument defaulting mechanism is in place, unassigned? will also be deleted.

     Note that no real functionality is lost because of this deletion, since unbound? was just a macro that expanded as follows:

          (unbound? x)    ==>    (lexical-unbound? (the-environment) 'x)

  * The operations on representatives of the unassigned and unbound variable markers have been replaced as follows:

          make-unassigned-object    ==>    make-unassigned-reference-trap
          unassigned-object?        ==>    unassigned-reference-trap?
          make-unbound-object       ==>    make-unbound-reference-trap
          unbound-object?           ==>    unbound-reference-trap?

     In addition, a new operation, unmap-reference-trap, maps a reference-trap representative into a reference-trap object, while acting as an identity procedure for all other objects.

  * The type system has been deleted.  This was an ad-hoc mechanism which was not being used extensively.  At some time in the future we may install a more carefully designed one.  Note that the type dispatcher mechanism (which was used) has been replaced by a simpler code walking facility, scode-walk (see SCode Walker).

  * The procedures date and time have been replaced by the new procedure get-decoded-time which is somewhat like the procedure of the same name in Common Lisp.  This procedure returns a decoded-time object which has the following accessors:

          decoded-time/day
          decoded-time/hour
          decoded-time/minute
          decoded-time/month
          decoded-time/second
          decoded-time/year

     Likewise the procedures date->string and  time->string have been replaced by the following:

          decoded-time/date-string
          decoded-time/time-string

  * The event-distributor datatype has been changed.  Previously it was a procedure which was invoked by application.  Now it is an unspecified datatype which is invoked using the operation event-distributor/invoke!.

  * The names of the following variables have been changed:

          interrupt-mask-gc-ok    ==>    interrupt-mask/gc-ok
          interrupt-mask-none     ==>    interrupt-mask/none
          interrupt-mask-all      ==>    interrupt-mask/all

  * The three different tags used by the syntaxer and unsyntaxer for identifying the three fluid-let special forms have been compressed into a single tag.  The unsyntaxer now figures out which special form was intended from the code.  Thus, the new variable lambda-tag:fluid-let replaces the following three variables:

          lambda-tag:shallow-fluid-let
          lambda-tag:deep-fluid-let
          lambda-tag:common-lisp-fluid-let

  * The procedures breakpoint and read-eval-print have been changed to require that their MESSAGE argument be a cmdl-message object rather than a string.  The cmdl-message datatype is new with this release.

  * The following procedures have been deleted:

          make-null
          make-false
          make-true
          make-interned-symbol
          symbol-print-name
          final-segment
          with-threaded-continuation

  * Various undefined values, including the value of *the-non-printing-object*, now print out as #[undefined-value].  Previously their printed representation was the null string.  The REP loop treats them specially, causing them to print as ;No value when they are the result of a top level evaluation.  The new predicate undefined-value? discriminates objects in this set.

  * The procedure format is no longer defined by default.  The distribution contains a library program to provide this functionality, which can be loaded by evaluating the expression

          (load-option 'format)

     The new version of format has been changed as follows, for compatibility with Common Lisp:

      1. The DESTINATION argument is now required.  It may be #F, indicating the result should be returned as a string, or #T, meaning it should be printed on the current output port.  Otherwise it must be the desired output port.

      2. The ~C format is no longer implemented.

      3. The ~A format agrees with Common Lisp.  It uses display to write the next argument to the output port.  This functionality was previously provided, in a limited way, by ~S.

      4. The ~S format agrees with Common Lisp.  It uses write to write the next argument to the output port.  This functionality was previously provided by ~O.

      5. The meaning of the @ modifier has been reversed to agree with Common Lisp.  The form ~NS means ``pad the argument on the left to size N'', while ~N@S means ``pad the argument on the right to size N''.

  * The following (approximate) renamings have occurred:

          primitive-type        ==>    object-type
          primitive-gc-type     ==>    object-gc-type
          primitive-type?       ==>    object-type?
          primitive-datum       ==>    object-datum
          primitive-set-type    ==>    object-new-type

     The new operations differ in that they all touch their arguments (this matters only in a Scheme with futures) and the object-new-type operation is GC safe.  Also, the operation object-datum is guaranteed to return a non-negative integer; primitive-datum would sometimes return negative integers.

     Note that the previous definition of object-type, which returned a symbol representing the type, has been renamed to user-object-type.

  * The procedure initial-segment has been renamed list-head (corresponding to list-tail).

  * The procedure number->string defaults its FORMAT argument to '(heur).  This means the radix defaults to decimal.  Previously it defaulted to the current unparser radix.

  * The procedure symbol->string now copies the string before returning it, thus preventing accidental side-effects from screwing up the obarray.

  * The procedure purify now (by default) queues its argument until the next garbage collection, and then performs a purification of the queue rather than a GC.  A second optional argument allows forcing of the purification immediately.

  * All user procedures previously called %FOO are now just FOO. These are: in out ge ve gst vst cd pwd.  The old names are still defined for compatibility.

  * The g interrupt no longer resets the top-level REP loop by default. Instead, it aborts whatever computation is in progress and resumes the top-level REP loop.  One consequence of this is that the top-level reader and printer histories (%in and %out) are not destroyed by g.  Another is that, once the top-level environment is visiting an environment, g will not restore the environment to its original value (also the prompt will always be "Visiting->").

     Executing (proceed) in the top-level REP loop will cause it to terminate, and a new one to be created.

     A user option flag, cmdl-interrupt/abort-top-level/reset?, changes the behavior of g to be more like the old behavior.  When true, g will escape from the top-level REP loop and start a new one.  By default, this flag is false.

  * The representation of syntax tables has been changed.  Also the following operations have been renamed:

          syntax-table-copy      ==>    syntax-table/copy
          syntax-table-define    ==>    syntax-table/define
          syntax-table-extend    ==>    syntax-table/extend
          syntax-table-ref       ==>    syntax-table/ref

Miscellaneous
*************

Custom Unparsers
================

The unparser (sometimes called the printer) supports the definition of programmer customizable methods for the unparsing of certain kinds of objects.  Previous implementations of this interface supplied each custom method with the object to be unparsed, and the method was supposed to output the printed representation to the current output port.  Now, the interface supplies the method with two objects: (1) the object to be unparsed, and (2) an object which encapsulates the state of the unparser at that point.  It is the responsibility of the method to output the printed representation onto a port specified by the unparser state object, subject to restrictions made explicit in that state.

Operations are provided for the following:

  * Associating methods with objects.

  * Performing output on characters and strings.

  * Recursive calls into the unparser.

  * Composition and decomposition of unparser state objects.

 -- Procedure: unparser/set-tagged-pair-method! tag method
 -- Procedure: unparser/set-tagged-vector-method! tag method
     Custom unparser methods can be defined for two kinds of objects: tagged pairs and tagged vectors.  Pairs are tagged by placing a unique object in their car, and vectors are tagged by placing a unique object in their zeroth element.  These two operations associate unparser methods with the tags used in constructing tagged objects.  The values of these operations are undefined.

 -- Procedure: unparser/tagged-pair-method tag
 -- Procedure: unparser/tagged-vector-method tag
     These operations return the method associated with their argument, or #F if there is no such method.

 -- Procedure: unparser/standard-method name #!optional unparser
     This procedure returns a standard method which handles many common cases of customization.  NAME is an object which is used to identify the type of object (usually a string), and UNPARSER, if supplied, must be a procedure of two arguments.

     If UNPARSER is not supplied, the generated method uses the following format to print objects:

          #[NAME HASH]

     Here NAME is as above, and HASH is a non-negative integer unique to the particular object being printed.  The expression

          #@HASH

     returns a pointer to the object referred to by HASH if the object still exists (i.e. if its storage has not been reclaimed by the garbage collector), signalling an error if it no longer exists.

     If UNPARSER is supplied, a slightly different format is used:

          #[NAME HASH OUTPUT]

     Here NAME and HASH are as above, and OUTPUT is the output written to the port by UNPARSER.  The printed representation is constructed in three stages:

      1. The first part of the format (up to OUTPUT) is written to the output port specified in the unparser state.

      2. UNPARSER is invoked on two arguments: the unparser state and the object.

      3. The closing bracket is written on the output port.

 -- Procedure: add-unparser-special-pair! tag method
 -- Procedure: add-unparser-special-object! tag method
 -- Procedure: unparse-with-brackets thunk
     These operations are supplied for compatibility with existing code.  If you don't know what they do you don't need to.

 -- Procedure: unparse-char unparser-state char
 -- Procedure: unparse-string unparser-state string
 -- Procedure: unparse-object unparser-state object
     These operations are useful for writing custom unparsers. UNPARSER-STATE must be an unparser state object.  The first two operations write a character and a string, respectively, onto the output port specified by UNPARSER-STATE.  The third operation recursively enters the unparser to print OBJECT.  Note that some care must be taken with recursive unparsing to guarantee that list depth and slashification are handled properly.

 -- Procedure: make-unparser-state port list-depth slashify? unparser-table
 -- Procedure: unparser-state? object
 -- Procedure: guarantee-unparser-state object
 -- Procedure: unparser-state/port unparser-state
 -- Procedure: unparser-state/list-depth unparser-state
 -- Procedure: unparser-state/slashify? unparser-state
 -- Procedure: unparser-state/unparser-table unparser-state
     These operations define the unparser-state datatype.  It is a tagged four-tuple with no side-effects.  In the future, if additional items are added to the unparser's internal state, unparser methods that do not depend on the representation of the state object will be unaffected.

     The meaning of each of the components is as follows:

       * port is the output port to which the unparser is writing.

       * list-depth is the number of lists (or vectors) which enclose the current object.  This is used by the mechanism which limits how deeply the unparser will print a list.

       * slashify? is a flag specifying whether slashification is in effect.  This is true when the unparser is called by the procedure write, and false when the unparser is called by display.  Its meaning is defined by the documentation for those two procedures.

       * unparser-table is a table specifying the syntax used to print objects.  This is rarely anything other than the standard unparser table.

SCode Walker
============

MIT Scheme has an extensive set of procedures, collectively called the SCode abstraction, that are useful for writing programs that analyze other programs.  The SCode abstraction constructs and manipulates the SCode program representation, which is a tagged, tree-structured representation similar to the parse tree of many conventional compilers.

The typical structure of a program-analysis program is a recursive loop, which at its beginning performs an n-way branch based on the type of the expression.  Each of the branches has as its target some code which handles a particular type of expression, and expressions which have arbitrary subexpressions as components do a recursive call to the beginning of the loop to process the subexpression.

The SCode Walker is a simple mechanism which implements an efficient form of the n-way branch of SCode expression types.  This branch could be implemented by means of conditional expressions, but it turns out that by taking advantage of the representation of SCode, the runtime library can supply a much more efficient branching mechanism.

The SCode walking mechanism works as follows.  The procedure make-scode-walker is invoked to create an scode-walker object.  This object represents a mapping from SCode types to arbitrary Scheme objects.  Given such an object, the procedure scode-walk will map an arbitrary SCode expression into the object associated with that expression's type by a given scode-walker.

The following is a list of the SCode types recognized by make-scode-walker:

     access
     assignment
     combination
     comment
     conditional
     declaration
     definition
     delay
     disjunction
     error-combination
     in-package
     lambda
     open-block
     quotation
     sequence
     the-environment
     unassigned?
     variable

When constructing an scode-walker, It is reasonable to specify associations for a subset of the above types.  In that case, expressions of other types are mapped to a default object, which is specified when the scode-walker is constructed.  For certain types, however, the default is the association for another type if that is specified (in other words, these types are considered to be subtypes of other types). Here is a list of those types, along with the types from which their defaults are derived:

     Type               Default
     ----               -------
     declaration        comment
     error-combination  combination
     open-block         sequence
     unassigned?        combination

 -- Procedure: make-scode-walker default alist
     Construct and return a new SCode walker object.  DEFAULT is any Scheme object, and ALIST must be an association list.  Each element of ALIST must be a list of two items; the first item is the name of an SCode type (a symbol), and the second is an arbitrary Scheme object.

     The SCode walker resulting from this call behaves according to the description given above.

 -- Procedure: scode-walker? object
     This predicate is true iff OBJECT is an SCode walker.

 -- Procedure: scode-walk scode-walker expression
     SCODE-WALKER must be an SCode walker, and EXPRESSION may be any SCode expression.  This procedure returns the object associated with EXPRESSION, according to the rules outlined above.

Multiple Values
===============

Multiple values are supported, in a limited way, by the following operations.  The current implementation is trivial, does not signal number of value mismatches correctly, and has poor performance.  Also, first class continuations do not accept multiple values.  Eventually these defects will be fixed without modification to the interface.

 -- Procedure: with-values thunk receiver
     THUNK must be a procedure of no arguments which returns multiple values, and RECEIVER must be a procedure which accepts the number of values returned by THUNK.  THUNK is invoked with a continuation that invokes RECEIVER on THUNK's values; the number of values that this continuation will accept is the same as the number of arguments that RECEIVER will accept.

     In the current implementation, it is an error if THUNK uses

     call-with-current-continuation to reify its continuation.

 -- Procedure: values . objects
     The arguments OBJECTS are returned as multiple values.  It is an error if the current continuation expects a different number of values than are supplied.