;;; -*- Mode:gate; Fonts:(HL12 HL12I HL12B CPTFONTB HL12BI HL12B HL12I ) -*- =Node: Introduction to Subprimitives =Text: 3INTRODUCTION TO SUBPRIMITIVES* 1Subprimitives* are functions which are not intended to be used by the average program, only by system programs. They allow one to manipulate the environment at a level lower than normal Lisp. They are described in this chapter. Subprimitives usually have names starting with a 2%* character. The primitives described in other sections of the manual typically use subprimitives to accomplish their work. To some extent the subprimitives take the place of what in other systems would be individual machine instructions. Subprimitives are normally hand-coded in microcode. There is plenty of stuff in this chapter that is not fully explained; there are terms that are undefined, there are forward references, and so on. Furthermore, most of what is in here is considered subject to change without notice. In fact, this chapter does not exactly belong in this manual, but in some other more low-level manual. Since the latter manual does not exist, it is here for the interim. Subprimitives by their very nature cannot do full checking. Improper use of subprimitives can destroy the environment. Subprimitives come in varying degrees of dangerousness. Generally, those without a 2%* sign in their name are not directly dangerous, whereas those whose names begin with 2%* can ruin the Lisp world just as readily as they can do something useful. The subprimitives are documented here since they need to be documented somewhere, but this manual does not document all the things you need to know in order to use them. Still other subprimitives are not documented here because they are very specialized. Most of these are never used explicitly by a programmer; the compiler inserts them into the program to perform operations which are expressed differently in the source code. The most common problem you can cause using subprimitives, though by no means the only one, is to create illegal pointers: pointers that are, for one reason or another, according to storage conventions, not allowed to exist. The storage conventions are not documented; as we said, you have to be an expert to use a lot of the functions in this chapter correctly. If you create such an illegal pointer, it probably will not be detected immediately, but later on parts of the system may see it, notice that it is illegal, and (probably) halt the Lisp Machine. In a certain sense 2car*, 2cdr*, 2rplaca*, and 2rplacd* are subprimitives. If these are given a locative instead of a list, they access or modify the cell addressed by the locative without regard to what object the cell is inside. Subprimitives can be used to create locatives to strange places. =Node: Data Types =Text: 3DATA TYPES data-type* 1arg* 2data-type* returns a symbol that is the name for the internal data-type of 1arg*. The 2type-of* function (4(PRIMOBJTYPE-1)Testing Types with Type Specifiers*) is a higher-level primitive that is more useful in most cases; normal programs should always use 2type-of* (or, when appropriate, 2typep*) rather than 2data-type*. Note that some types as seen by the user are not distinguished from each other at this level, and some user types may be represented by more than one internal type. For example, 2dtp-extended-number* is the symbol that 2data-type* would return for either a single-float or a bignum, even though those two types are quite different. Some of these type codes occur in memory words but cannot be the type of an actual Lisp object. These include header types such as 2dtp-symbol-header*, which identify the first word of a structure, and forwarding or ``invisible'' pointer types such as 2dtp-one-q-forward*. 2dtp-symbol* The object is a symbol. 2dtp-fix* The object is a fixnum; the numeric value is contained in the address field of the pointer. 2dtp-small-flonum* The object is a short float; the numeric value is contained in the address field of the pointer. 2dtp-extended-number* The object is a single-float, ratio, bignum or complexnum. This value will also be used for future numeric types. 2dtp-character* The object is a character object; the value is contained in the address field of the pointer. 2dtp-list* The object is a cons. 2dtp-locative* The object is a locative pointer. 2dtp-array-pointer* The object is an array. 2dtp-fef-pointer* The object is a compiled function. 2dtp-u-entry* The object is a microcode entry. 2dtp-closure* The object is a closure; see 4(CLOSURES-0)Closures*. 2dtp-stack-closure* The object is a closure which lives inside a stack, and which must be copied if it is stored anywhere but farther down in the same stack. Lexical scoping is implemented using these. 2dtp-instance* The object is an instance of a flavor; see 4(FLAVOR-0)Objects, Message Passing, and Flavors*. 2dtp-entity* The object is an entity; see 4(CLOSURES-1)Entities*. 2dtp-select-method* The object is a select-method; see 4(FUNCTIONS-1)Other Kinds of Functions*. 2dtp-stack-group* The object is a stack-group; see 4(STACKGROUPS-0)Stack Groups*. 2The remaining types are internal only.* 2dtp-header* An internal type used to mark the first word of several kinds of multi-word structure, including single-floats, ratios, bignums and FEFs. 2dtp-array-header* An internal type used to mark the first word of an array. 2dtp-symbol-header* An internal type used to mark the first word of a symbol. The pointer field points to the symbol's print-name, which is a string. 2dtp-instance-header* An internal type used to mark the first word of an instance. The pointer field points to the structure that describes the instance's flavor. 2dtp-null* Nothing to do with 2nil*. This type code identifies a void marker. An attempt to refer to the contents of a cell that contains a 2dtp-null* signals an error. This is how ``unbound variable'' and ``undefined function'' errors are detected. 2dtp-trap* The zero data-type, which is not used. This hopes to detect microcode bugs. 2dtp-free* This type is used to fill free storage, to catch wild references. 2dtp-external-value-cell-pointer* An ``invisible pointer'' used for external value cells, which are part of the closure mechanism (see 4(CLOSURES-0)Closures*), and used by compiled code to address value and function cells. 2dtp-self-ref-pointer* An ``invisible pointer'' used to refer to an instance variable of 2self*. This data type appears in FEFs of flavor methods. 2dtp-header-forward* An ``invisible pointer'' used to indicate that the structure containing it has been moved elsewhere. The ``header word'' of the structure is replaced by one of these invisible pointers. See the function 2structure-forward* (4(SUBPRIMITIVES-1)Forwarding*). 2dtp-body-forward* An ``invisible pointer'' used to indicate that the structure containing it has been moved elsewhere. This points to the word containing the header-forward, which points to the new copy of the structure. 2dtp-one-q-forward* An ``invisible pointer'' used to indicate that the single cell containing it has been moved elsewhere. 2dtp-gc-forward* This is used by the copying garbage collector to flag the obsolete copy of an object; it points to the new copy. 3q-data-types* 1Constant* The value of 2q-data-types* is a list of all of the symbolic names for data types described above under 2data-type*. These are the symbols whose print names begin with `2dtp-*'. The values of these symbols are the internal numeric data-type codes for the various types. 3q-data-types* 1type-code* Given the internal numeric data-type code, returns the corresponding symbolic name. This ``function'' is actually an array. =Node: Forwarding =Text: 3FORWARDING* An 1invisible pointer* or 1forwarding pointer* is a kind of pointer that does not represent a Lisp object, but just resides in memory. There are several kinds of invisible pointer, and there are various rules about where they may or may not appear. The basic property of an invisible pointer is that if the Lisp Machine reads a word of memory and finds an invisible pointer there, instead of seeing the invisible pointer as the result of the read, it does a second read, at the location addressed by the invisible pointer, and returns that as the result instead. Writing behaves in a similar fashion. When the Lisp Machine writes a word of memory it first checks to see if that word contains an invisible pointer; if so it goes to the location pointed to by the invisible pointer and tries to write there instead. Many subprimitives that read and write memory do not do this checking. The simplest kind of invisible pointer has the data type code 2dtp-one-q-forward*. It is used to forward a single word of memory to someplace else. The invisible pointers with data types 2dtp-header-forward* and 2dtp-body-forward* are used for moving whole Lisp objects (such as cons cells or arrays) somewhere else. The 2dtp-external-value-cell-pointer* is very similar to the 2dtp-one-q-forward*; the difference is that it is not ``invisible'' to the operation of binding. If the (internal) value cell of a symbol contains a 2dtp-external-value-cell-pointer* that points to some other word (the external value cell), then 2symeval* or 2set* operations on the symbol consider the pointer to be invisible and use the external value cell, but binding the symbol saves away the 2dtp-external-value-cell-pointer* itself, and stores the new value into the internal value cell of the symbol. This is how closures are implemented. 2dtp-gc-forward* is not an invisible pointer at all; it only appears in ``old spaced'' and can never be seen by any program other than the garbage collector. When an object is found not to be garbage, and the garbage collector moves it from ``old space'' to ``new space'', a 2dtp-gc-forward* is left behind to point to the new copy of the object. This ensures that other references to the same object get the same new copy. 3structure-forward* 1old-object* 1new-object* This causes references to 1old-object* actually to reference 1new-object*, by storing invisible pointers in 1old-object*. It returns 1old-object*. An example of the use of 2structure-forward* is 2adjust-array*. If the array is being made bigger and cannot be expanded in place, a new array is allocated, the contents are copied, and the old array is structure-forwarded to the new one. This forwarding ensures that pointers to the old array, or to cells within it, continue to work. When the garbage collector goes to copy the old array, it notices the forwarding and uses the new array as the copy; thus the overhead of forwarding disappears eventually if garbage collection is in use. 3follow-structure-forwarding* 1object* Normally returns 1object*, but if 1object* has been 2structure-forward*'ed, returns the object at the end of the chain of forwardings. If 1object* is not exactly an object, but a locative to a cell in the middle of an object, a locative to the corresponding cell in the latest copy of the object is returned. 3forward-value-cell* 1from-symbol* 1to-symbol* This alters 1from-symbol* so that it always has the same value as 1to-symbol*, by sharing its value cell. A 2dtp-one-q-forward* invisible pointer is stored into 1from-symbol*'s value cell. Do not do this while 1from-symbol*'s current dynamic binding is not global, as the microcode does not bother to check for that case and something bad will happen when 1from-symbol*'s binding is unbound. The microcode check is omitted to speed up binding and unbinding. This is how synonymous variables (such as 2*terminal-io** and 2terminal-io*) are created. To forward one arbitrary cell to another (rather than specifically one value cell to another), given two locatives, do 3(%p-store-tag-and-pointer 1locative1* dtp-one-q-forward 1locative2*) follow-cell-forwarding* 1loc* 1evcp-p* 1loc* is a locative to a cell. Normally 1loc* is returned, but if the cell has been forwarded, this follows the chain of forwardings and returns a locative to the final cell. If the cell is part of a structure which has been forwarded, the chain of structure forwardings is followed, too. If 1evcp-p* is 2t*, external value cell pointers are followed; if it is 2nil* they are not. =Node: Pointer Manipulation =Text: 3POINTER MANIPULATION* It should again be emphasized that improper use of these functions can damage or destroy the Lisp environment. It is possible to create pointers with illegal data-type, pointers to non-existent objects, and pointers to untyped storage, which will completely confuse the garbage collector. 3%data-type* 1x* Returns the data-type field of 1x*, as a fixnum. 3%pointer* 1x* Returns the pointer field of 1x*, as a fixnum. For most types, this is dangerous since the garbage collector can copy the object and change its address. 3%make-pointer* 1data-type* 1pointer* Makes up a pointer, with 1data-type* in the data-type field and 1pointer* in the pointer field, and returns it. 1data-type* should be an internal numeric data-type code; these are the values of the symbols that start with 2dtp-*. 1pointer* may be any object; its pointer field is used. This is most commonly used for changing the type of a pointer. Do not use this to make pointers which are not allowed to be in the machine, such as 2dtp-null*, invisible pointers, etc. 3%make-pointer-offset* 1data-type* 1pointer* 1offset* Returns a pointer with 1data-type* in the data-type field, and 1pointer* plus 1offset* in the pointer field. The 1data-type* and 1pointer* arguments are like those of 2%make-pointer*; 1offset* may be any object but is usually a fixnum. The types of the arguments are not checked; their pointer fields are simply added together. This is useful for constructing locative pointers into the middle of an object. However, note that it is illegal to have a pointer to untyped data, such as the inside of a FEF or a numeric array. 3%pointer-difference* 1pointer-1* 1pointer-2* Returns a fixnum which is 1pointer-1* minus 1pointer-2*. No type checks are made. For the result to be meaningful, the two pointers must point into the same object, so that their difference cannot change as a result of garbage collection. 3%pointerp* 1object* 2t* if 1object* points to storage. For example, 2(%pointerp "foo")* is 2t*, but 2(%pointerp 5)* is 2nil*. 3%pointer-type-p* 1data-type* 2t* if the specified data type is one which points to storage. For example, 2(%pointer-type-p* 2dtp-fix)* returns 2nil*. =Node: Special Memory Referencing =Text: 3SPECIAL MEMORY REFERENCING %p-pointerp* 1location* 2t* if the contents of the word at 1location* points to storage. This is similar to 2(%pointerp* 2(contects 1location*))*, but the latter may get an error if 1location* contains a forwarding pointer, a header type, or a void marker. In such cases, 2%p-pointerp* correctly tells you whether the header or forward points to storage. 3%p-pointerp-offset* 1location* 1offset* Similar to 2%p-pointerp* but operates on the word 1offset* words beyond 1location*. 3%p-contents-offset* 1base-pointer* 1offset* Returns the contents of the word 1offset* words beyond 1base-pointer*. This first checks the cell pointed to by 1base-pointer* for a forwarding pointer. Having followed forwarding pointers to the real structure pointed to, it adds 1offset* to the resulting forwarded 1base-pointer* and returns the contents of that location. There is no 2%p-contents*, since 2car* performs that operation. 3%p-contents-safe-p* 1location* 2t* if the contents of word 1location* are a valid Lisp object, at least as far as data type is concerned. It is 2nil* if the word contains a header type, a forwarding pointer, or a void marker. If the value of this function is 2t*, you will not get an error from 2(contents 1location*)*. 3%p-contents-safe-p-offset* 1location* 1offset* Similar to 2%p-contents-safe-p* but operates on the word 1offset* words beyond 1location*. 3%p-contents-as-locative* 1pointer* Given a pointer to a memory location containing a pointer that isn't allowed to be ``in the machine'' (typically an invisible pointer) this function returns the contents of the location as a 2dtp-locative*. It changes the disallowed data type to 2dtp-locative* so that you can safely look at it and see what it points to. 3%p-contents-as-locative-offset* 1base-pointer* 1offset* Extracts the contents of a word like 2%p-contents-offset*, but changes it into a locative like 2%p-contents-as-locative*. This can be used, for example, to analyze the 2dtp-external-value-cell-pointer* pointers in a FEF, which are used by the compiled code to reference value cells and function cells of symbols. 3%p-safe-contents-offset* 1location* 1offset* Returns the contents of the word 1offset* words beyond 1location* as accurately as possible without getting an error. If the contents are a valid Lisp object, it is returned exactly. If the contents are not a valid Lisp object but do point to storage, the value returned is a locative which points to the same place in storage. If the contents are not a valid Lisp object and do not point to storage, the value returned is a fixnum with the same pointer field. Forwarding pointers are checked as in 2%p-contents-offset*. 3%p-store-contents* 1pointer* 1value* Stores 1value* into the data-type and pointer fields of the location addressed by 1pointer*, and returns 1value*. The cdr-code field of the location remains unchanged. 3%p-store-contents-offset* 1value* 1base-pointer* 1offset* Stores 1value* in the location 1offset* beyond words beyond 1base-pointer*, then returns 1value*. The cdr-code field remains unchanged. Forwarding pointers in the location at 1base-pointer* are handled as they are in 2%p-contents-offset*. 3%p-store-tag-and-pointer* 1pointer* 1miscfields* 1pointerfield* Stores 1miscfields* and 1pointerfield* into the location addressed by 1pointer*. 25 bits are taken from 1pointerfield* to fill the pointer field of the location, and the low 7 bits of 1miscfields* are used to fill both the data-type and cdr-code fields of the location. The low 5 bits of 1miscfields* become the data-type, and the top two bits become the cdr-code. This is a good way to store a forwarding pointer from one structure to another (for example). 2%p-store-tag-and-pointer* should be used only for storing into `boxed' words, for the same reason as 2%blt-typed*: the microcode could halt if the data stored is not valid boxed data. See 4(SUBPRIMITIVES-2)Copying Data*. 3%p-ldb* 1byte-spec* 1pointer* Extracts a byte according to 1byte-spec* from the contents of the location addressed by 1pointer*, in effect regarding the contents as a 32-bit number and using 2ldb*. The result is always a fixnum. For example, 2(%p-ldb %%q-cdr-code 1loc*)* returns the cdr-code of the location addressed by 1loc*. 3%p-ldb-offset* 1byte-spec* 1base-pointer* 1offset* Extracts a byte according to 1byte-spec* from the contents of the location 1offset* words beyond 1base-pointer*, after handling forwarding pointers like 2%p-contents-offset*. This is the way to reference byte fields within a structure without violating system storage conventions. 3%p-mask-field* 1byte-spec* 1pointer* Like 2%p-ldb*, except that the selected byte is returned in its original position within the word instead of right-aligned. 3%p-mask-field-offset* 1byte-spec* 1base-pointer* 1offset* Like 2%p-ldb-offset*, except that the selected byte is returned in its original position within the word instead of right-aligned. Note: 2%p-dbp*, 2%p-dpb-offset*, 2%p-deposit-field* and 2%p-deposit-field-offset* should never be used to modify the pointer field of a boxed word if the data type is one which actually points to storage, unless you are sure that the new pointer is such as to cause no trouble (such as, if it points to a static area). Likewise, it should never be used to change a data type which does not point to storage into one which does. Either action could confuse the garbage collector. 3%p-dpb* 1value* 1byte-spec* 1pointer* Stores 1value*, a fixnum, into the byte selected by 1byte-spec* in the word addressed by 1pointer*. 2nil* is returned. You can use this to alter data types, cdr-codes, etc., but see the note above for restrictions. 3%p-dpb-offset* 1value* 1byte-spec* 1base-pointer* 1offset* Stores 1value* into the specified byte of the location 1offset* words beyond that addressed by 1base-pointer*, after first handling forwarding pointers in the location addressed by 1base-pointer* as in 2%p-contents-offset*. 2nil* is returned. This is the way to alter unboxed data within a structure without violating system storage conventions. You can use this to alter boxed words too, but see the note above for restrictions. 3%p-deposit-field* 1value* 1byte-spec* 1pointer* Like 2%p-dpb*, except that the selected byte is stored from the corresponding bits of 1value* rather than the right-aligned bits. See the note above 2%p-dpb* for restrictions. 3%p-deposit-field-offset* 1value* 1byte-spec* 1base-pointer* 1offset* Like 2%p-dpb-offset*, except that the selected byte is stored from the corresponding bits of 1value* rather than the right-aligned bits. See the note above 2%p-dpb* for restrictions. 3%p-pointer* 1pointer* Extracts the pointer field of the contents of the location addressed by 1pointer* and returns it as a fixnum. 3%p-data-type* 1pointer* Extracts the data-type field of the contents of the location addressed by 1pointer* and returns it as a fixnum. 3%p-cdr-code* 1pointer* Extracts the cdr-code field of the contents of the location addressed by 1pointer* and returns it as a fixnum. 3%p-store-pointer* 1pointer* 1value* Stores 1value* in the pointer field of the location addressed by 1pointer*, and returns 1value*. 3%p-store-data-type* 1pointer* 1value* Stores 1value* in the data-type field of the location addressed by 1pointer*, and returns 1value*. 3%p-store-cdr-code* 1pointer* 1value* Stores 1value* in the cdr-code field of the location addressed by 1pointer*, and returns 1value*. 3%stack-frame-pointer* Returns a locative pointer to its caller's stack frame. This function is not defined in the interpreted Lisp environment; it only works in compiled code. Since it turns into a ``misc'' instruction, the ``caller's stack frame'' really means ``the frame for the FEF that executed the 2%stack-frame-pointer* instruction''.