The Implementation of Data Structures in Version 5 of Icon* Ralph E. Gr is wo Id TR 85-8

Transcription

1 The Implemetatio of Data Structures i Versio 5 of Ico* Ralph E. Gr is wo Id TR 85-8 April 1, 1985 Departmet of Computer Sciece The Uiversity of Arizoa Tucso. Arizoa This work was supported by the Natioal Sciece Foudatio uder Grats MCS ad DCR

2

3 The Implemetatio of Data Structures i Versio 5 of Ico 1. Itroductio Ico supports a variety of data types, some of which are uusual [1]. There are sophisticated access mechaisms for structures, icludig associative look-up ad lists that ca grow ad shrik. Variables i Ico are utyped ad structures ca be heterogeeous, havig elemets of differet types. For example, a elemet of a list ca be a list eve itself. Storage maagemet is automatic. There are o storage declaratios; objects are created durig program executio, space is allocated implicitly as it is eeded, ad space is reclaimed automatically. These features of Ico have a profoud effect o the way that it is implemeted. This report describes the implemetatio of data structures i Ico i some detail, but it does ot attempt to justify all desig decisios or to explai why every give implemetatio choice is better tha a alterative. I fact, some aspects of the implemetatio were affected by earlier work whose traces are o loger evidet. This report describes data structures as they are implemeted i Versio 5.9 of Ico [2]; there are some implemetatio differeces from earlier versios. The reader should be familiar with Ico, particularly its data structures. This report ca be read aloe or i cojuctio with listigs of the source code to illumiate details. The Ico "tour" [3] is a useful auxiliary referece. Some kowledge of C [4], i which most of Ico is writte, also may be useful. 2. Descriptors All values ad variables i Ico are represeted by descriptors. Descriptors are fixed i size for ay give implemetatio. A descriptor either may cotai a value (as i the case of a Ico iteger) or it may cotai a poiter to a object that cotais the value (as i the case of a Ico real umber). The layout of data withi a descriptor depeds somewhat o the kid of value or variable that it represets. A descriptor ormally cosists of two words (C its): Descriptor Layout t-word v-word The t-word (for "type") typically cotais a type code ad flags that idicate properties of the descriptor. The v-word (for "value") typically cotais a value or a poiter to data represetig the value. The dotted lie betwee the two words of a descriptor is provided for readability ad does ot represet ay real demarcatio i memory. There are 11 source-laguage data types i Ico as well as records (which collectively costitute a sigle type for implemetatio purposes). There are also a umber of iteral data types, such as table elemets, that are o a par with source-laguage data types, but which are ot directly accessible from a sourcelaguage program. There are two forms of variables, ordiary ad trapped. A ordiary variable poits to a descriptor that cotais the correspodig value. A trapped variable is used i situatios i which processig the variable requires some special actio, such as i assigmet to a table referece or to &pos. Some trapped variables poit to blocks of data that cotai iformatio that is eeded whe the variable is accessed. Other trapped This icludes the set data type that is available as a optioal extesio i Versio 5.9 of Ico [2].

4 variables simply cotai a code that idicates the actio that is required". See Sectios 4, 5, ad 1. The descriptors for all of these cases fall ito six categories: the ull value strigs itegers other values ordiary variables trapped variables The ull value has a uique represetatio i which both the t-word ad v-word are zero: The Descriptor for the Null Value For strigs, the t-word cotais the legth of the strig (the umber of characters i it) ad the v-word is a poiter to the first character of the strig: A Descriptor for a Strig legth first character I order to make strig descriptors more itelligible i the diagrams that follow, a poiter to a strig is followed by the strig i quotatio marks rather tha a address. For example, the strig descriptor for "hello" is depicted as A Descriptor for the Strig "hello" - "hello" Sice the empty strig has a legth of zero, its v-word is always ozero to avoid ambiguity with the ull value. The actual value of the v-word of a empty strig has o special sigificace. The empty strig is idicated as follows: A Empty Strig All descriptors except those for the ull value ad strigs cotai flags i the high-order bits of their t- words. I subsequet diagrams, these flags are represeted symbolically as follows: p v t m descriptor is ot the ull value or a strig v-word of the descriptor cotais a poiter to ostrig data descriptor is a variable descriptor is a trapped variable descriptor is marked (used oly durig garbage collectio) Descriptors for values other tha the ull value ad strigs, as well as those for trapped variables, also cotai a type code i the low-order bits of the t-word. There are 2 type codes represeted by small itegers. The actual values of the type codes are ot importat; they are represeted symbolically as show i the followig list: -2-

5 descriptor type type cod iteger iteger log iteger log real umber real cset cset file file procedure proc list Iheader set sheader table theader co-expressio estack record record list elemet set elemet selem table elemet telem co-expressio block eblock substrig trapped variable tvsubs table elemet trapped variable tvtbl &pos trapped variable tvpos &radom trapped variable tvrad &trace trapped variable tvtrac For example, the Ico iteger 1357 is represeted by iteger 1357 Descriptor for the Iteger 1357 Note that the flag distiguishes this descriptor from a strig whose first character might be at the address 1357 ad whose legth might have the same value as the type code for iteger. O the other had, a list is represeted by p Descriptor for a List Iheader - structure for the list where the v-word poits to the structure for the list. The p flag is used by the garbage collector. 3. Blocks Most kids of Ico data are represeted by blocks of words. These blocks have a comparatively uiform structure that is desiged to facilitate their processig durig garbage collectio. The first word of every block cotais a type code the same code that is i the t-word of a descriptor that poits to the block. Some blocks are fixed i size for all values of a give type. For example, all Ico values of type file poit to blocks of the same size. For example, as a result of the value of i is i := ope("log.dat") These are iteral data types. -3-

6 p file file - UNIX file descriptor file status "log.dat" The Value of i The strig ame of the file is used i tracig ad diagostic operatios. Files are discussed i more detail i Sectio 12. Blocks of some other types, such as record blocks, vary i size from value to value, but ay oe block is fixed i size ad ever grows or shriks. If the type itself does ot determie the size of the block, the secod word i the block cotais the size (i bytes). I the diagrams that follow, the sizes of blocks are give for 32- bit computers such as the VAX. For example, give the record declaratio ad the value of j is: record complex(r,i) j := complex(1,3) p record record 28 iteger 1 iteger 3 type size ( record costructor A Record with Two Fields The record costructor cotais iformatio that is eeded to resolve field refereces. See Sectio 13. O the other had, with the declaratio record term(value, code, cout) ad word := term("chair", "ou", 4) the value of word is: p record record 36 type code size of block record costructor - "chair" A Record with Three Fields iteger 4 - ou As illustrated by this example, blocks may cotai descriptors as well as o-descriptor data. Nodescriptor data comes first i the block, followed by ay descriptors, as illustrated by the precedig figure. The -4-

7 locatio of the first descriptor i a block is costat for all blocks of a give type, which facilitates garbage collectio. Some blocks, such as those for the stadard iput ad output files, are compiled as part of the Ico rutime system. Other blocks, such as those for the procedures declared i a program, are created whe the program is traslated, liked, ad loaded ito memory prior to executio. Such blocks are located i regios of memory that are static ad they are ot moved durig program executio. Most blocks, such as those produced durig executio by the creatio of lists ad other structures, are dyamically allocated i a "heap" regio. Blocks i the heap regio may be moved (relocated) durig garbage collectio. All poiters to blocks are i the v-words of descriptors, sice descriptors cotai the iformatio eeded to idetify blocks (the p flag, the type code, ad the address of the block). If a block i the heap is moved as a cosequece of garbage collectio, all poiters to it are chaged accordigly. 4. Variables Variables are represeted by descriptors, just as values are. Variables for idetifiers poit to descriptors for the correspodig values. Ordiary variables always poit to descriptors for values, ever to other variables. The values of local idetifiers are kept o the stack, while the Values of global ad static idetifiers are located at fixed places i memory. Variables that poit to the values of idetifiers are created o the stack by icode istructios [3] that correspod to the use of the idetifiers i the program. A variable that refereces a value i a data structure poits directly to the descriptor for the value. The low-order portio of the t-word for such a variable cotais the offset, i words, of the value descriptor from the top of the block i which the value is cotaied. This offset is used by the garbage collector. The use of words, rather tha bytes, allows larger offsets. For example, the variable word.cout for the record give i the precedig sectio is: v 7 record iteger 4 record costructor "chair" "ou A Field Referece Ulike a ordiary variable, a trapped variable does ot poit to a descriptor for its value, but istead either (1) poits to a block that cotais iformatio that is ecessary to dereferece or assig to the variable, or (2) idicates a computatio that must be performed to dereferece or assig to the variable. The first kid of trapped variable is used for substrig ad table refereces, which are described i Sectios 5 ad 1. The keyword &pos is a example of the secod kid of trapped variable ad is represeted by tv tvpos - value of &pos The Trapped Variable for &pos The type code tvpos determies the computatio to be performed. If &pos is derefereced, as i write (&pos) the curret positio i the &subject, which is stored i a fixed locatio i the ru-time system, is produced. O the other had, i - 5

8 &pos := expr the value of expr is stored at that locatio. Note, however, that this requires type checkig ad possible type coversio, sice the value of expr may ot be a iteger. This is why a ordiary variable caot be used for &pos. 5. Strigs Ico strigs, ulike C strigs, are ot ull-termiated ad may cotai ay of the 256 characters i the Ico character set. The characters of Ico strigs, like C strigs, are i cosecutive locatios (bytes) i memory. As described earlier, a Ico strig is represeted by a descriptor that cotais its legth ad the address of its first character. This address is a byte address ad may ot be at a word boudary. Such descriptors are called strig qualifiers or simply qualifiers. The strig "the" is represeted by the qualifier The Qualifier for "the" - "the" The poiter to "the" is just a otatioal coveiece. A more accurate represetatio is The Qualifier for "the".the. The actual value of the v-word might be 569a (hexadecimal), where the character t is at locatio 569a, the character h is at locatio 569b, ad the character e is at locatio 569c. Strig qualifiers are used to represet all Ico-style strigs (as opposed to C-style strigs that are used i the C computatios). Because a qualifier delimits a strig oly by the address of its first character ad its legth, Ico strigs ca overlap or be cotaied withi other Ico strigs. Note that the computatio of the legth of a Ico strig is simple ad does ot require examiatio of the characters i the strig. Like blocks, some strigs are compiled ito the ru-time system ad others, such as strigs that appear as literals i a program, are loaded ito memory alog with the program. Durig program executio, Ico strigs may be stored i work areas (usually referred to as "buffers"), but most ewly created strigs are allocated i a commo strig regio. As source-laguage operatios costruct ew strigs, their characters are appeded to the ed of those already i the strig regio. The amout of space allocated i the strig regio typically icreases durig program executio util the regio is full, i which poit it is compacted by garbage collectio, squeezig out characters that are o loger eeded. 5.1 Strig Costructio Some operatios that produce strigs require the allocatio of ew strigs, while other operatios just produce ew qualifiers. For example, s := "ab" "cdef" allocates a strig ad produces a qualifier for it: -6-

9 ... a b c d e f The Qualifier for a Newly Allocated Strig This qualifier the becomes the value of S. The strigs "ab" ad "cdef" are literals ad hece are loaded i with the program, istead of beig i the strig regio. O the other had, s[2:5] does ot allocate a strig, but oly produces a qualifier that poits to a substrig of S: s[2:5] A Substrig 7> abco-ef Copies of them are made i the strig regio whe the cocateatio above is performed. Operatios such as cocateatio allocate ew strigs ad add them to the ed of the curret characters i the strig regio. At ay time, the strig regio cotais a sequece of cosecutive characters ito which various qualifiers poit. I the case of a cocateatio such as 81 S2 the strig values of s1 ad s2 are appeded to the ed of the existig strig i the strig regio. Oe optimizatio is performed: I case s1 is the last strig i the strig regio, s2 is simply appeded to the ed of the strig regio. Thus operatios of the form S := S expr perform less allocatio tha operatios of the form S := expr 11 s Except for this optimizatio, o strig costructio operatio attempts to use aother istace of a strig that may occur somewhere else i the strig regio. Thus s1 := "ab" s2 := "a" I "c" "be" produce two distict strigs: s1 s2 A Example of Duplicate Allocatio... abcabc -7-

10 5.2 Substrigs Assigmet to a subscripted strig, as i s1[i] := S2 does ot, i geeral, chage the ith character of si to s2. Istead, it is equivalet to the followig cocateatio: s1 := s1[1:i] s2 s1[i + 1:] Whe s1 [i] is evaluated, the cotext (dereferecig or assigmet) i which the result will be used may ot be kow. For example, if a procedure p cotais the expressio retur s[i] ad s is a global idetifier, the use of the subscripted strig depeds o the cotext of the procedure ivocatio, which might be or write (p()) P() := "" I order to hadle such situatios, whe a strig-valued variable is subscripted, a substrig trapped variable is produced. For example, if s := "abcdef" the result of evaluatig s[2:5] is a substrig trapped variable that has the form tv tvsubs v tvsubs 3 legth offset v - "abcdef" The Substrig Trapped Variable Produced by s[2:5] The legth ad offset of the substrig provide the ecessary iformatio either to produce a qualifier for the substrig i the case of dereferecig, or to costruct a ew strig i the case of assigmet. Note that the strig itself is poited to via a variable, sice if a assigmet is made, it is ecessary to chage the value of the variable that is subscripted. After a assigmet such as the situatio is: s[2:5] := "x" -8-

11 tv tvsubs v tvsubs 1 2 legth offset 4 *"axef" s v i The Substrig Trapped Variable After Assigmet Note that the legth of the subscripted portio of the strig has be chaged to correspod to the legth of the strig assiged to it. This reflects the fact that subscriptig idetifies the portios of the strig before ad after the subscripted portio ("a" ad "ef", i this case). I the case of a multiple assigmet to a subscripted strig, oly the origial subscripted portio is chaged. Thus, i (s[2:5] := "x") := "yyyyy" the fial value of y is "ayyyyyef". 6. Numeric Types Ico itegers may be as large as C logs. O computers for which C its are the same size as C logs, itegers are cotaied i descriptors as illustrated by the diagrams above. O computers for which C its are smaller tha C logs, Ico itegers that do ot fit ito C its are cotaied i blocks that are poited to by descriptors. For example, o the PDP-11 the iteger 1 is represeted by p log log 1 - The Iteger 1 o the PDP-11 Ico real umbers are represeted by C double-precisio/7oar.s (64 bits) ad are cotaied i blocks. For example, the real umber 2. is represeted by p real real 2. - The Real Number 2. Thus a real umber block is 5 words (1 bytes) o a 16-bit computer ad 3 words (12 bytes) o a 32-bit computer. 7. Csets Sice Ico uses 8-bit characters, regardless of the computer o which it is is implemeted, there are 256 differet characters that ca occur i csets. A cset block cosists of a header word cotaiig the cset type code followed by 256 bits, each of which represets the presece or absece of a character i the cset. Thus a cset block, icludig the headig type word, is 17 words (34 bytes) log o a 16-bit computer ad 9 words (36 bytes) log o a 32-bit computer. -9-

12 A bit is 1 if the correspodig character is i the cset ad otherwise. The first 128 bits correspod to the ASCII character set, as illustrated by the value of &ascii: p cset cset liiiiimiiiiiiiiiiiiiiiiiiiiiii llllllllllllllllllllllllllllllll liiiiimmmimiiimmiiii liiiiiiimimimiiiiimiii 3 }()()()() [WOfWWXXKXXWOOOOOOOOOOOOOOOOOOO X)()()O()()()O()()() The Value of &ascii Because of the way that memory is displayed i these diagrams, the rightmost bit i each word has the lowest address. Thus, the cset block for the first character, '\\ is cset 31 XKWOOOOOOOOOOOOOOOOOOOOOOOOOOOO xwooooooooooooooooooooooooooooo XXXX)()()()()()() XXX)()()()()()() )(X)()()()()()()()()()() xwooooooooooooooooooooooooooooo 3 The Cset Block for '\' Similarly, the cset block for the last character, '\xff, is cset 3(X)(>()()()(><)(> XXWOOOOOOOOOOOOOOOOOOOOOOOOOOOO XX)OO()OOOO()OOOO()OOOO()O()O()O()O()OO )()()(K)()(X)() )(X)()()()()(X)()()() XXXJOOOOOOOOOOOOOOOOOOOOOOOOOOOO X>()(><)()<)<) 1 The Cset Block for \xff' Aother example is the cset for 'aeiou': -1

13 cset oooooooooooooooooooooooooooooooo 3 3 OOOOOOOOOO11OOOOOII1 O()()()()() The Cset Block for 'aeiou' 8. Lists Lists ca be accessed by positio, but they also ca grow ad shrik at their eds as the result of stack ad queue operatios. The structures used for lists therefore are more complicated tha those that would be eeded to support oly positioal access. A list cosists of a list header block, which cotais the usual type word, the curret size of the list (the umber of elemets i it), ad descriptors that poit to the first ad last blocks o a doubly-liked chai of list elemet blocks that cotai the actual list elemets. Whe a list is created, either by or by list(i,expr) \expr v expr 2,..., exprj there is oly oe list elemet block. Other list elemet blocks may be added to the chai as the result of pushes or puts. A list elemet block cotais the usual type word, the size of the block (i bytes), three words used to determie the locatios of elemets i the list elemet block, ad descriptors that poit to the ext ad previous list elemet blocks, if ay. (The ull descriptor idicates the absece of a poiter.) Followig this data are slots for elemets (slots always cotai valid descriptors, eve if they are ot used to hold list elemets). The umber of slots i a list elemet block is at least eight (a ad hoc implemetatio costat). This allows some expasio room for addig elemets to lists, such as the empty list, that are small iitially. The data structures for a list are illustrated by the result of evaluatig a := [7,8,9] which produces the followig structures: -11 -

14 p Iheader p Iheader 3 umber of p ^ r iteger 7 iteger 8 iteger 9 6 A size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 Data Structures for the List [7, 8, 9] Note that there is oly oe list elemet block ad that the slot idexig is zero-based. Uused slots cotai ull values that are logically iaccessible. 8.1 Storage of Elemets i List Blocks Elemets i a list elemet block are stored as a circular queue. If a elemet is added to the ed of a list, as i put(a, 1) the result is equivalet to a := [7,8, 9,1] The value is added to the "ed" of the last list elemet block (assumig there is a uused slot). The result is: -12-

15 p p (header umber of elemets curretly i the list iteger 7 iteger 8 iteger 9 iteger 1 size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 The List Elemet Block After a put Note that the icrease i the umber of elemets i the list is reflected i the list header block. If a elemet is added to the begiig of a list, as i push (a, 6) the result is equivalet to a := [6,7,8,9,1] The ew elemet is put at the "begiig" of the first list elemet block (assumig there is a uused slot). For the case above, the result is: -13

16 p p Iheader 5 umber of elemt :ts curretly i th J iteger 7 iteger 8 iteger 9 iteger 1 iteger 6 size of block umber of slots i block first slot used.. umber of slots used poiter to previous list elemet block poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 The List Elemet Block After a push Note that the "begiig", which is before the first physical slot i the list elemet block, is the last physical slot. The locatios of elemets that are i a list elemet block are determied by the three itegers at the head of the list elemet block "removal" of a elemet by a pop, get, or pull does ot shorte the list elemet block or overwrite the elemet the elemet merely becomes iaccessible Addig ad Removig List Blocks If a elemet is added to a list ad o more slots are available i the appropriate list elemet block, a ew list elemet block (with eight slots) is allocated ad liked i. For example. is equivalet to The result is: every push(a,5 to 1 by -1) a := [1,2, 3, 4, 5, 6, 7, 8, 9,1] 'Pathological source-laguage code ca access such elemets, at least trasietly. -14-

17 p p Iheader 1 1 size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block p poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 iteger i* iteger 2 slot 6 slot 7 p size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block The Additio of a List Elemet Block iteger 7 iteger 8 iteger 9 iteger 1 iteger 3 iteger 4 iteger 5 iteger 6 poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 As elemets are removed from a list by get (which is syoymous with pop) or pull, the idices i the appropriate list elemet block are adjusted. Thus if two elemets are popped, as i pop(a); pop(a) the result is equivalet to 15-

18 a := [3,4,5,6,7,8,9,1] ad the structures become: p p Iheader 1 size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block p poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 I p iteger r iteger slot 6 slot 7 iteger sloto 7 iteger slot 1 8 iteger slot 2 9 iteger slot 3 i'o iteger slot 4 3 iteger slot 5 4 iteger slot 6 5 iteger slot 7 6 size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block poiter to ext list elemet block The Result of Removig Elemets from a List Elemet Block -16-

19 Note that the secod list elemet block is still liked i the chai, eve though it o loger cotais ay elemets that are logically accessible. A list elemet block is ot removed from the chai whe it becomes empty, but oly whe a elemet is removed from a list that already has a empty list elemet block. Thus there is always at least oe list elemet block o the chai, eve if the list is empty. Aside from simplifyig the access to list elemet blocks from the list header block, this avoids repeated allocatio i the case that pop/ push pairs occur at the boudary of two list elemet blocks. Cotiuig the example above, pop(a) which is equivalet to a := [4, 5, 6,7, 8, 9,1] causes the empty list elemet block to be removed: p p Iheader 7 umber of elemi :ts curretly i th ) iteger 7 iteger 8 iteger 9 iteger 1 iteger 3 iteger 4 iteger 5 iteger 6 size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block poiter to ext list elemet block sloto slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 The Removal of a Empty List Elemet Block Note that the value 3 is still physically i the list elemet block, although it is logically iaccessible. 8.3 Positioal Refereces to Lists A positioal referece of the form a[i] requires locatig the correct list elemet block (ote that out-ofrage refereces ca be determied by examiig the list header block). If the list has a umber of list elemet blocks, this ivolves likig through the list elemet blocks, keepig track of the cout of elemets i each block util the appropriate oe is reached. Nopositive specificatios are coverted to positive oes before accessig the chai of list elemet blocks. The result of evaluatig a[i] is a variable that poits to the appropriate slot i the list. For the precedig -17

20 example, a[3] produces a variable that poits to the descriptor for the value 6 i [4, 5, 6, 7, 8, 9, 1] iteger sloto 7 iteger slot 1 8 iteger slot 2 9 iteger slot 3 1 iteger slot 4 3 iteger slot 5 4 iteger slot 6 a f 3 ]... v 23 L_ 5 iteger slot 7 6 Referecig a List Elemet Note the offset of 23 words i the t-word of the variable. size of block umber of slots i block first slot used umber of slots used poiter to previous list elemet block poiter to ext list elemet block 8.4 Other List Costructio Operatios The other sources of lists are list sectioig (a[i:j] ad its variats), list cocateatio, copy(a), sort(a), ad sort(t). I the case of a list sectio, a ew list is costructed ad the specified elemets are copied ito it. The ew list has oly oe list elemet block, regardless of the umber of list elemet blocks i the list i which the sectio is specified. As with explicit list creatio, at least eight slots are provided. The fuctio copy(a) is similar oe list elemet block is created ad all the elemets from a are copied ito it. I list cocateatio, two list elemet blocks are created, oe for each of the lists specified i the cocateatio (regardless of their list elemet block structure). These two list elemet blocks are liked together after the elemets are copied ito them. The use of two list elemet blocks rather tha a sigle large oe is primarily a matter of coveiece. The fuctio sort(a) is very similar to copy (a), but sort(t) is more iterestig, sice a list of two-elemet lists is produced. These two-elemet lists have list elemet blocks with oly two slots. This is the oly case i which list elemet blocks have less tha eight elemets. 9. Sets Sice sets ca cotai a arbitrary umber of members of ay type, a hash lookup scheme is used to provide a efficiet way of locatig set members. For every set there is a set header block that cotais a word for the umber of members i the set ad slots that serve as heads for (possibly empty) liked lists ("buckets") of set elemet blocks. The umber of slots is a implemetatio parameter. There are 37 slots i table header blocks o the VAX, but oly 13 slots o the PDP-11, where the address space is more limited. The structure for a empty set, produced by 18

21 s := set([]) is: p sheader sheader umber of members i the set sloto slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 slot 8 A Empty Set Each member of a set is cotaied i a separate set elemet block. Whe a value is a looked up i a set (for example, to add a ew member), a hash umber is computed from this value. The hash umber modulo the umber of slots is used to select a slot. Each set elemet block cotais a descriptor for its value, the correspodig hash umber, ad a poiter to the ext set elemet block, if ay, o the list. For example, the set elemet block for the iteger 37 is: selem 37 iteger 37 hash umber member value poiter to ext set elemet block The Set Elemet Block for the Iteger 37 As illustrated by this figure, the hash umber for a iteger is just the value of the iteger. This member goes i slot o the VAX, sice its remaider o divisio by the umber of slots is zero. See Sectio 11 for a geeral discussio of hash computatios. The structures for the set are: s := set([37, ]) -19

22 p sheader 2 selem p selem iteger selem selem 37 iteger 37 ; A Set with Two Members This example was chose for illustratio, sice both ad 37 go i slot. I searchig the list, the hash umber of the look-up value is compared with the hash umbers i the set elemet blocks. If a match is foud, the value i the set elemet block may or may ot be the same as the value beig looked up (collisios i the hash computatio are ievitable). Thus if the hash umbers are the same, it is ecessary to compare the values. The compariso that is used correspods to the source-laguage operatio To improve the performace of the look-up process, the set elemets i each liked list are ordered by their hash umbers. Whe a chai of set elemet blocks is examied, the search stops if a hash umber of a elemet o the chai is greater tha the hash umber of the value beig looked up. If the value is ot foud ad the lookup is beig performed to isert a ew member, a set elemet block for the ew member is created ad liked ito the list. The word i the set header block that cotais the umber of members is icremeted to reflect the isertio. For example, isert(s, -37) iserts a set elemet block for -37 at the head of the list i slot, sice its hash value is Tables Tables are implemeted i a fashio similar to sets, with a header block cotaiig slots for elemets ordered by hash umbers. A table header block cotais a extra descriptor for the default assiged value. A empty table with the default etry value, produced by t := table(o) illustrates the structure of the table header block: -2

23 p theader theader umber of elemets i the table iteger default assiged value sloto slot 1 slot 2 6 slot 3 slot 4 slot 5 slot 6 slot 7 slot 8 A Empty Table Table lookup is more complicated that set lookup, sice table elemets cotai both a etry value ad a assiged value ad a ew table elemet ca be created by assigmet to a table referece with a etry value that is ot i the table. The structure of a table elemet block is illustrated by the oe produced for a assigmet such as t[37] := 1 telem 37 iteger 37 iteger 1 hash umber poiter to ext table elemet block etry value assiged value I the case of a table referece such as t[x] The Structure of a Table Elemet Block the hash umber for the etry value x is used to select a slot ad the correspodig list is searched for a table elemet block that cotais the same etry value. As for sets, compariso is first made usig hash umbers; values are compared oly if their hash umbers are the same. If a table elemet block with a matchig etry value is foud, a variable that poits to the correspodig assiged value is produced. For example, if 37 is i t as illustrated above, a subsequet referece produces: t[37] -21-

24 v 7 telem 37 iteger 37 iteger 1 Result of a Table Referece If this variable is derefereced, as i write(t[37]) the value 1 is writte. O the other had, if a assigmet is made to this variable, as i t[37] +:= 1 the assiged value i the table elemet block is chaged: telem 37 iteger 37 iteger 2 Result of Assigmet to a Table Referece If a table elemet with a matchig etry value is ot foud, a table elemet trapped variable is created. Suppose, for example, that the etry value 36 is ot i the table t. The t[36] produces the followig result: tv tvtbl p tvtbl 36 hash umber theader iteger 36 table header block for t etry value reserved for subsequet use A Table Elemet Trapped Variable The last descriptor i the table elemet trapped variable block is reserved for subsequet use, as described below. The use of this trapped variable depeds o the cotext i which the referece that produced it is used. If it is derefereced, as i write(t[36]) the default assiged value,, which is i the table header block for X, is produced. It is possible, however, for elemets to be iserted i a table betwee the time the table elemet trapped variable is created ad the time it is derefereced. A example is -22-

25 write(t[36], t[36] := 2) Sice fuctios do ot dereferece their argumets util all the argumets have bee evaluated, the result of dereferecig the first argumet of write should be 2, ot. I order to hadle such cases, whe a table elemet trapped variable is derefereced, its list must be searched agai to determie whether to retur the assiged value of a ewly iserted elemet or to retur the default value. If a assigmet is made to the table referece, as i t[36] +:= 1 the table elemet trapped variable is coverted to a table elemet ad iserted i the appropriate place i the table. Note that the structures of table elemet blocks ad table elemet trapped variable blocks are the same, allowig this coversio without allocatig a ew table elemet block. It is ecessary to search the list for its slot agai to determie the place to isert it. As i the case of dereferecig, elemets may have bee iserted i the table betwee the time that the table elemet trapped variable is created ad the time that a value is assiged to it. Normally, o matchig etry is foud ad the table elemet trapped variable, trasformed ito a table elemet block, is iserted with the ew assiged value. If a matchig etry is foud, its assiged value is simply chaged. Note that referece to a value that is ot i a table requires oly oe computatio of the hash value, but two lookups i the list of table elemet blocks for its slot. 11. Hash Computatios The purpose of usig hash umbers is to reduce the time ecessary to determie if a value is i a set or table. Storig the hash umbers i elemet blocks speeds the search, sice compariso of hash umbers avoids the more time-cosumig compariso of values i most cases. The computatio of hash umbers is a iterestig problem, sice sets ad tables ca cotai values of ay type. As show above, the computatio of the hash umber for a iteger is trivial. I the case of strigs, the hash umber is determied by addig the umerical values of the characters of the strig, up to a maximum of 1. The legth of the strig is added to the result. For csets, the hash umber is the exclusive-or of the words cotaiig the bits for the cset. Structures such as lists ad records pose a coceptual problem, sice the hash umber computatio for a value must produce the same umber regardless of whe the computatio is made, ad hece must be based o some uchageable attribute of the value. However, the elemets i a list may chage, as well as its size. Eve the locatio of a list (the address of its list header block) may chage because of garbage collectio [3]. I fact, the oly uchageable attribute of a list is its type, I header. The situatio is the same for sets ad tables. I these cases, the types are used as hash umbers. This differetiates lists, sets, ad tables, but all values of a give structure type have the same hash umber. For example, all lists go ito the same slot ad their elemets all have the same hash umber. Cosequetly, lookup for structure types is liear. Fortuately, few programs costruct sets or tables that cotai a large umber of elemets that are structures of the same type. For such situatios, better performace could be obtaied if there were a word for a hash umber (such as the time of creatio) i each structure header block. This would icrease the size of every structure, however. 12. Files A value of type file i Ico poits to a block that cotais the UNIX* file descriptor, a status word, ad the strig ame of the file. The file status values are 'UNIX is a trademark of AT&T Bell Laboratories. -23-

26 closed ope for readig ope for writig ope to create ope to apped ope as a pipe For example, the value of &iput is p file file 1 6 # UNIX file descriptor files *"&iput' The Value of &iput while the value of &output is p file file 2 7 * UNIX file descriptor files * "&output" The Value of &output Aother example is out := ope("log", "a") for which the value of out is p file file 1 3 UNIX file descriptor files *"log' The Value of out Note that the file status is 1, correspodig to beig ope for writig ad appedig. Closig a file, as i close(out) merely chages its file status: 24

27 file - UNIX file descriptor file status "log' A Closed File 13. Procedures There are three kids of procedures i Ico: declared procedures, built-i fuctios, ad record costructors. All are source-laguage values. The blocks for the three kids of procedures are similar, differig oly i detail. Declared Procedures For declared procedures, the procedure block cotais the usual type ad block size words, followed by five words that characterize the procedure: address for the procedure i the iterpretable program (icode) the umber of argumets (formal parameters) i the procedure the umber of dyamic local idetifiers i the procedure the umber of static local idetifiers i the procedure the idex of the first static local idetifier i the procedure The remaider of the procedure block cotais descriptors: first the strig ame of the procedure ad the the strig ames of the argumets, dyamic local idetifiers, ad static local idetifiers, i that order. The procedure ame is eeded for tracig. The fuctio display(f, i) uses the idex of the first static local idetifier, the procedure ame, ad the ames of the local idetifiers. Descriptors for the values of argumets ad dyamic local idetifiers are allocated o the stack whe a procedure is called. The values of static local idetifiers for all procedures are cotaied i a sigle array at a fixed locatio i memory. For example, the procedure declaratio procedure p(a1,a2) local i static base, idex ed has the followig procedure block: 25-

28 proc 76 size of block etry poit of the procedure i the icode umber of argumets umber of dyamic local idetifiers umber of static local idetifiers idex of first static local idetifier -*"p' - "al" - "a2' *"!' - "base" - "idex" A Procedure Block The value for the idex i the static idetifier regio idicates that base is the first static local idetifier i the program (the idices of static idetifiers are zero-based). Built-i Fuctios Fuctios are distiguished from declared procedures by the value -1 i the word that otherwise cotais the umber of dyamic local idetifiers. A example is proc 36-1 size of block etry poit of the fuctio i the ru-time system umber of argumets built-i fuctio idicator - "repl" The Procedure Block for the Fuctio repl The etry poit refers to the correspodig C fuctio i the ru-time system. Note that there are o argumet ames. Some fuctios, such as write, allow a arbitrary umber of argumets. This is idicated by the value -1 i place of the umber of argumets: proc 36 size of block etry poit of the fuctio i the ru-time system idicator of a variable umber of argumets built-i fuctio idicator - "write" The Procedure Block for the Fuctio write -26-

29 Record Costructors record term(value, code, cout) produces a record costructor term that creates records of type term, as i x := term ("ou", 5,) Record costructors are distiguished from declared procedures ad built-i fuctios by the value -2 i place of the umber of dyamic local idetifiers. The etry poit refers to the C fuctio that costructs record blocks. Each record declaratio is distiguished by a uique record idetificatio umber. This umber appears i place of the umber of static local idetifiers. For example, if the first record declaratio i a program is record term(value, code, cout) the its record idetificatio umber is 1 ad the procedure block for its record costructor is: proc 36 size of block etry poit for record costructio i the ru-time system umber of argumets record costructor idicator record idetificatio umber - "term' The Procedure Block for the Record Costructor term The record idetificatio umber could be used to distiguish records of differet types for the purpose of hashig, but it is ot. 14. Co-Expressios A co-expressio cosists of a stack o which the evaluatio of the co-expressio takes place, a descriptor that poits to the most recet activator of the co-expressio, poiters related to expressio evaluatio, a refresh block, ad a word cotaiig the umber of times the co-expressio has bee activated. Cosider the followig procedure procedure labge(tag, i, j) retur create tag (i to j) ed ad the co-expressio produced by e := labge("l", 1,1) The value assiged to e is: A record declaratio produces a record costructor, which is aalogous to a built-i fuctio. For example, -27-

30 p estack eblock poiter to the most recet activator + stack base - stack poiter - argumet poiter - boudary umber of results produced refresh block A Co-Expressio Stack O the VAX, the stack grows from large addresses toward smaller oes, so whe the co-expressio is activated the stack grows away from the type word. Prior to the first activatio of e, the most recet activator is ull. The stack base, stack poiter, argumet poiter, ad boudary refer to stack locatios that are related to expressio evaluatio. The data pushed o the stack whe a expressio is evaluated is machie depedet. See [5] for details. The refresh block is used to produce a refreshed copy of the co-expressio, as i e := Ae which restores the values of the local idetifiers i e to the values they had whe e was origially created. The refresh block cotais a poiter to the locatio i the icode where the expressio correspodig to the co-expressio is located, followed by the umber of argumets, the umber of local idetifiers, a descriptor that poits to the procedure block for the procedure i which the co-expressio was created, ad the values of the argumets ad local idetifiers at the time of creatio: The refresh block ad correspodig procedure blocks are: 28-

31 eblock 52 c p 3 proc 1 iteger 1 iteger 1 "I " -> proc 6 size of block etry poit i the icode umber of argumets umber of local idetifiers - "labge' - "tag "*/.*-." A Co-Expressio Refresh Block ad its Procedure Block Whe a co-expressio is activated, its poiter to its most recet activator is filled i by the co-expressio that activated it. Whe it suspeds, the umber of results is icreased by 1. For example, if lab is evaluated i the procedure mai, the co-expressio for e becomes "j- -29-

32 p estack estack -> co-expressio for &mai p eblock Result of a Co-Expressio Activatio Ackowledgemets Dave Haso, Tim Korb, ad Walt Hase desiged the implemetatio of the origial versio of Ico, o which may of the curret structures are based [6, 7]. Cary Coutat ad Steve Wampler desiged the implemetatio of Versio 5 of Ico. Most of the curret data structures were doe by them. Bill Mitchell has doe most of the recet work o this implemetatio. Rob McCoeghy implemeted sets ad revised the earlier implemetatio of tables i collaboratio with the author. Dave Haso, Rob McCoeghy, Bill Mitchell, ad Jaalee O'Bagy provided a umber of useful suggestios o the presetatio of the material i this report. Refereces 1. R. E. Griswold ad M. T. Griswold, The Ico Programmig Laguage, Pretice-Hall, Ic., Eglewood Cliffs, NJ, R. E. Griswold, R. K. McCoeghy ad W. H. Mitchell, Extesios to Versio 5 of the Ico Programmig Laguage, The Uiv. of Arizoa Tech. Rep. Tech. Rep. 84-1, Aug R. E. Griswold, R. K. McCoeghy ad W. H. Mitchell, A Tour Through the C Implemetatio of Ico; Versio 5.9, The Uiv. of Arizoa Tech. Rep , Aug B. W. Kerigha ad D. M. Ritchie, The C Programmig Laguage, Pretice-Hall, Ic., Eglewood Cliffs, NJ, W. H. Mitchell, Portig the UNIX Implemetatio of Ico, The Uiv. of Arizoa Tech. Rep. 83-1d, R. E. Griswold ad D. R. Haso, Referece Maual for the Ico Programmig Laguage; Versio 2, The Uiv. of Arizoa Tech. Rep. 79-la, D. R. Haso ad W. J. Hase, Ico Implemetatio Notes, The Uiv. of Arizoa Tech. Rep a,