Visualizing Complex Notions of Time

Transcription

1 Visualizing Complex Noions of Time Rober Kosara, Silvia Miksch Insiue of Sofware Technology, Vienna Universiy of Technology, Vienna, Ausria Absrac Time plays an imporan role in medicine. Condiions are no jus evaluaed a single insans in ime, bu raced over periods. Medicaions mus be adminisered wihin specified emporal limis, and heir effecs observed wih regard o ime. When planning reamens, he emporal aspec becomes even more complicaed. The planner has o deal wih uncerainy and allowable inervals. A visual represenaion of he informaion would be helpful, bu here are few visualizaions of ime ha are powerful enough. We presen a visualizaion ha graphically represens a complex noion of ime, and has also been implemened in a program ha allows users o direcly specify his informaion. The resuls of a small user sudy are repored. Keywords: Time; Time Managemen; Daa Display; User-Compuer Inerface; Clinical Proocols; Inroducion Treamens in medicine involve many decisions and processes ha are very ime-dependen. The simples example are ime series of paien daa, which form he basis for many decisions. Treamen monioring is anoher example: if a reamen fails o reach is goal in a given amoun of ime, i is abored and anoher one is ried. These wo examples have one hing in common: hey deal wih informaion from he pas. As soon as a daa value is recorded, he exac poin in ime a which he measuremen was aken is known, and he value can be ploed ino a diagram, for example. When planning reamens, his is differen. Acions can never be planned precisely, each paien reacs differenly o medicaion, and here are complicaions ha canno be foreseen for a single paien. Mos graphical represenaions of ime do no provide means for displaying such uncerain informaion. Wih uncerainy, we mean ime inervals where bounds for sar and end imes as well as duraion are a leas parially known. The Asgaard Projec In he Asgaard Projec, we develop mehods for represening clinical guidelines and proocols, and assising he medical saff in implemening hem. In his secion, a shor overview over some of he key feaures of Asbru is given. A horough inroducion is far beyond he scope of his paper; please refer o [1] for more informaion. We are using a plan represenaion language called Asbru [2] for specifying medical proocols. Each proocol is ranslaed ino many Asbru plans herefore, we use he erm plan insead of proocol in his paper. An Asbru plan can have subplans. If i does, is ype specifies how is subplans are arranged in ime. The ype can be sequenial, parallel, any-order or unordered (his includes parly parallel execuion, which any-order does no allow). Depending on heir definiion, subplans do no have o be performed. They can be opional, and heir success or failure can be ignored. By defaul, all plans are non-opional and heir success affecs he success of he conaining plan. If a plan does no have subplans, i is considered an acion (ha is performed by medical saff or a device, like a respiraor). Plans are governed by condiions ha specify when a plan can be applied, when i has o be abored, has compleed (i.e., reached is goal), ec. There are also oher so-called knowledge roles ha specify he effecs of a plan, is inenions, ec. Asbru s Time Annoaion One of he main componens of Asbru is he ime annoaion. I has several uses: he mos common is specifying he emporal exen of a condiion. In Asbru, a condiion does no consis only of a value comparison ( if (body_emperaure > 38) do... ), bu also conains a specificaion of he ime span in which his condiion mus hold ( if ((body_emperaure > 38) for a leas 10 hours bu no longer han 20 hours) do... ). Because medical herapy plans have o deal wih uncerainy, his ime span specificaion does no jus consis of a simple inerval, bu has o have much more power. A condiion wih a ime annoaion is called a emporal paern. Anoher use for ime annoaions is o specify he ime

2 Face 1 Face 2 Even A Even B Even C Even D Even E Figure 1- Time Lines frame in which an acion has o ake place (e.g., a reamen has o be performed wihin cerain limis afer diagnosis). Time annoaions consis of he pars described in he following lis. Any of hese pars (excep he reference poin) can be lef unspecified, o denoe ha his informaion is no imporan. Reference Poin. This is he poin ha all he oher poins in ime are defined relaive o. I can be an absrac poin in ime (e.g., concepion), or refer o a plan sae ransiion (e.g., he poin a which he previous plan was compleed). Earlies Saring Shif (ESS). The smalles offse from he reference poin when he acion or condiion can sar. If i has sared earlier, he ime annoaion is no fulfilled. By leaving his field undefined, i is possible o have he condiion sar a any poin before he laes saring shif. Laes Saring Shif (LSS). The laes poin in ime when he acion mus sar, or he condiion mus be rue. If i has no sared a his poin, he emporal paern fails. Earlies Finishing Shif (EFS). The earlies poin in ime when he acion can end. If i ends earlier, he emporal paern fails. Laes Finishing Shif (LFS). The greaes offse from he reference poin when he acion mus end, or he condiion mus become false for he emporal paern o be rue. If he LFS is specified, he emporal paern can only be decided afer i has passed. Minimum Duraion (MinDu). The minimum amoun of ime he acion or condiion mus las. This is no necessarily idenical wih he inerval beween LSS and EFS. I canno be shorer, however, han his difference, and no be longer han he maximum duraion. Maximum Duraion (MaxDu). The maximum duraion ha he condiion or acion may las. I is bounded by he difference beween LFS and ESS, and he minimum duraion. Informaion Visualizaion Informaion Visualizaion [3] graphically depics informaion ha does no have an inheren spaial srucure. Examples for such daa are file sysems, paien daa (body emperaure, venilaion parameers, ec), general daa bases, and of course emporal informaion. Many echniques exis o make informaion visible so as o ge a beer overview, o undersand correlaions, and o perceive he daa more quickly (looking a an image makes i possible o ge an idea of he srucure of he daa much faser han reading hundreds or housands of numbers). The imporance of informaion visualizaion is only saring o be undersood by praciioners in many fields. Bu everybody is aware of visualizaions such as maps, bar and pie chars, flow diagrams, ec. Visualizaion Requiremens Based on he descripion above, we wan o develop a visualizaion of emporal consrains. These are he requiremens we wan his visualizaion o fulfill. Allen s Relaions. Any visual represenaion of ime mus of course be able o visually represen all possible relaionships beween inervals. There are 13 such relaions described in [4]: A before B (A ends before B sars), A mees B (B sars a he same insan ha A ends), A overlaps B (B sars before A ends), A sars B (A and B sar a he same ime), A conains B (A is shorer han B and sars afer B sars), A finishes B (B ends a he same ime A ends). For each of hese six relaions, here also is a mirrored one plus he commuaive relaion A equals B. Temporal Uncerainy. The represenaion mus be able o deal wih emporal uncerainy as described in he previous secion. Undefined/Unknown Pars. If pars of a ime annoaion are unknown (denoed in Asbru as _, see Figure 3), hey mus be depiced in a way o make his easily recognizable wihou a he same ime being oo dominan. Resoluion. In addiion o he emporal uncerainy of acions (see previous secion), i should also be possible o see o wha precision a poin in ime has been specified relaive o he scale i is currenly being viewed on. Hierarchical Decomposiion. I mus also be possible o communicae he fac ha plans are made up of sub-plans. This hierarchical decomposiion is imporan no only o srucure a plan and o make pars reusable, bu also o be able o selec he amoun of informaion visible by showing more or fewer levels a he same ime. Faces. Differen kinds of informaion abou he same objec should be visible a he same ime. One of he key ideas for his are faces (see Figure 1). Relaed Work This secion describes wo ypes of ime visualizaion. Time Lines are relaively widely used, and have been exended ino LifeLines. SOPOs are a very powerful bu lile inuiive depicion of ime.

3 Time Lines, LifeLines LifeLines [5] are an exension of a raher old concep called ime lines [6], which is a very inuiive way of represening ime spans. On a wo-dimensional, Caresian coordinae sysem, one axis (usually he horizonal one) represens ime, and one axis is divided ino segmens for differen evens (Figure 1). A line (or box) is drawn over he ime span ha he even akes place in. One of he exensions inroduced in [5] are faces. Faces are verical segmens ha group similar evens. They can be opened and closed o provide he user wih informaion on differen aspecs of he same srucures wihou cluering he display wih oo much informaion. They are usually used o show differen aspecs (views) of he same informaion. A similar idea o faces is used in [7] o visualize differen pahs hrough a vaccinaion plan. In addiion o he basic ime line idea, arrows are drawn from he decision poins of he plan o alernaives forking off a his poin. Ses of Possible Occurrences (SOPOs) An ineresing way of looking a ime are ses of possible occurrences (SOPOs) [8]. This diagram uses wo ime axes ha represen he begin and end imes of an inerval, respecively (Figure 2). Any poin in his diagram represens a whole inerval, specified by is sar (x coordinae) and end ime (y coordinae). The area a SOPO covers (black in he figure) conains all inervals ha fi he specificaion given by means of an earlies sar, laes sar, earlies end, laes end, and minimum and maximum duraions. AsbruView s Time Annoaion Glyph This diagram is a par of AsbruView [9], which is a user inerface for he plan represenaion language Asbru. The glyph 1 described here is an exension of LifeLines and was specifically designed o represen Asbru s ime annoaion. Basic Principle I is based parly on a simple meaphor (Figure 3). Four verical pillars along he ime axis represen he earlies and laes saring and ending imes. On op of hese verical bars lies a bar ha is as long as he maximum duraion. On op of he MaxDu bar, suppored by wo diamonds (which lie above he LSS and EFS suppors, see Figure 3), lies anoher bar represening he minimum duraion. Undefined pars of a ime annoaion are displayed in gray, and in addiion he diamonds supporing he MinDu bar become rolls. I is possible o undersand a few simple consrains based on his meaphor. One is ha he minimum duraion can laes end ime finish ime earlies finish ime earlies sar ime laes sar ime begin ime Figure 2- Ses of Possible Occurrences (SOPOs) never be shorer han he difference beween LSS and EFS if i was, he MinDu bar would fall down beween is suppors. If LSS or EFS are undefined, he corresponding diamonds become rolls, which means ha he implicily defined EFS moves if he MinDu changes, for example. On he righ side of Figure 3, he depicion of differen emporal scales can be seen. If he curren scale is coarser han he precision o which he ime poins were defined, a circle appears (similar o he noion of an open inerval in mahemaics, where he end poin of he inerval from 1 o, bu no including, 2 canno be depiced direcly). On a smaller scale, a zigzag line appears ha covers he area of addiional uncerainy due o he lower resoluion of he acual poin in ime. User Sudy minimum duraion maximum duraion We implemened he described mehod in a prooype of a more general user inerface called AsbruView (Figure 4). To assess is usefulness in pracice, we performed a user sudy wih six physicians [9] o es he glyph (and also oher pars of AsbruView). For his sudy, we le he paricipans perform simple asks wih he program, afer having given hem a shor (ca. 30min) inroducion o Asbru and AsbruView. The paricipans were asked o fill ou quesionnaires before and afer he es o find ou how hey judged he sysem. The resuls were quie saisfying. The paricipans immediaely undersood he (raher complex) ime annoaion explained in erms of he meaphor underlying he ime annoaion glyph. They were able o specify ime annoaions for heir own use, and undersood he meaning of undefined values. 1 A glyph is a graphical objec (ofen vaguely represening a real objec, like a face) whose feaures express he values of cerain aribues ha are o be shown [10, 11].

4 Definiion: [[ESS, LSS], [EFS, LFS], [MinDu, MaxDu], Reference] Reference ESS LSS Example: [[2 d, 3 d], [_, 11 d ], [6 d, _], Diagnosis] Diagnosis 2 d 3 d Discussion MinDu MaxDu 6 d undef. EFS undef. LFS 11 d Figure 3- Time Annoaion Glyphs Even if only a par of he funcionaliy of he ime annoaion is used in each emporal paern or acion specificaion, none of is capabiliies can be ignored. Lifelines clearly do no saisfy he requiremens for displaying ime annoaions. They canno represen uncerainy or even undefined values. As Ri poins ou [8], his is simply due o he fac ha hey are one-dimensional, so any informaion exceeding one-dimensional ime resuls in an ambiguous diagram. For recorded paien informaion, however, hey are very useful. This mosly includes informaion abou evens, raher han recorded device readings (bu a combinaion of boh is easily imaginable). SOPOs were designed for he easy graphical propagaion of emporal consrains, no for making a complex noion of ime easy o undersand. Specifically, parallel plans and hierarchical decomposiion are very hard o depic and o work wih (if plans from several levels are drawn ino he same diagram, heir relaionship is no immediaely visible; parallel plans cover he same area in he diagram). A noion of undefined pars is missing in he original design. We exended SOPOs so ha hey me almos all he requiremens described a he beginning of his paper [12]. In a usabiliy sudy we performed wih hese exended SOPOs, one of he resuls was ha his ype of diagram is hard o undersand for people ouside of compuer science (and MinDu and LFS defined o higher precision han ime axis MinDu and LFS defined o lower precision han ime axis even many wihin...). AsbruView s ime annoaion glyph was specifically designed o mee he requiremens specified a he beginning of his paper, so i does saisfy all of hem. The simple meaphor i is based on has proven o be very helpful, wihou i limiing he represenaive power of he glyph (which can happen if a glyph is oo ighly coupled o a meaphor: hen, he consrains of he meaphor become consrains of he glyph. This is of course very undesirable). Oher mehods like Gan and PERT chars are quie common in many areas, including medicine. Gan chars are similar o LifeLines, bu can display hierarchies, and i is easy o imagine how o add faces (similar o LifeLines) o a Gan char display. PERT chars, on he oher hand, do no show any precise emporal informaion, only he order in which acions should occur. This can be useful, of course, especially in early sages of planning. Bu he approach is quie limied, especially when dealing wih such complex hings as reamen proocols. This discussion is summarized in Table 1. Table 1 Comparison of Time Visualizaion Mehods Allen s Relaions Temporal Uncerainy Undefined Pars Resoluion Hierarchical Decomposiion Faces Mehod TimeLines l l Gan l l PERT SOPOs l l Exended SOPOs Time Ann. Glyph l l l l l l l l l l l l Figure 4- A screensho of a par of he AsbruView prooype showing a real herapy plan in erms of ime annoaion glyphs

5 Conclusions and Fuure Work Visualizaion suppors complex asks such as medical reamen planning. Simple one-dimensional ime represenaions are no powerful enough for his ask, however. We presened a mehod for visualizing such complex noions of ime and showed ha i is useful in pracice by performing a small user sudy. A few open problems remain. One is he problem of how o represen cyclical as well as oher ypes of evens. This is made complicaed hrough he fac ha cyclical evens migh be specified o be performed unil a cerain sae is reached. In such a case, he emporal exen of he acion is no known (no even possible inervals), and a way of depicing his would have o be found. Anoher problem is ha of absrac reference poins. If a poin in ime (like dae of birh) is no (ye) known, nohing can be drawn. Only when he reference poin is bound o an absolue poin in ime can he inerval be drawn relaive o oher inervals. The quesion is, how such a ime annoaion could be specified graphically and depiced before his is possible. There are also oher ypes of uncerainy han he one addressed in his paper. If only he se of acions o be performed is known, how can i be made clear ha he order in which hey are depiced does no prejudge heir acual order of execuion? A relaed problem is ha of opional acions or plans. If a plan does no have o be performed (bu can, depending on condiions), i mus be depiced in a differen way han an obligaory one. Acknowledgemens We would like o hank he paricipans in he user sudy (in alphabeic order and ignoring heir iles): Shahram Adel, Sophie Brandseer, Maria Dobner, Gerhard Miksch, Franz Paky, and Chrisian Popow. The Asgaard Projec is suppored by Fonds zur Förderung der wissenschaflichen Forschung (Ausrian Science Fund), gran P12797-INF. UK, Open Universiy, [3] Ware C. Informaion Visualizaion: Percepion for Design. Morgan Kaufmann Publishers, [4] Allen J. Mainaining Knowledge abou Temporal Inervals. CACM 1983: 26(11): [5] Plaisan C, Milash B, Rose A, Widoff S, and Shneiderman B. LifeLines: Visualizing Personal Hisories. In Proceedings of ACM CHI 96 Conference on Human Facors in Compuing Sysems, volume 1 of PAPERS: Ineracive Informaion Rerieval, pages , [6] Tufe ER. The Visual Display of Quaniaive Informaion. Cheshire, CT: Graphics Press, [7] Brand CA, Frawley SJ, Powsner SM, Shiffman RN, and Miller PL. Visualizing he Logic of a Clinical Guideline: A Case Sudy in Childhood Immunizaion. Meh Inform Med 1997: 36: [8] Ri J-F. Propagaing emporal consrains for scheduling. In Proceedings of he Fifh Naional Conference on Arificial Inelligence, pages Morgan Kaufman Publishers, Inc, [9] Kosara R and Miksch S. Meaphors of Movemen: A Visualizaion and User Inerface for Time-Oriened, Skeleal Plans. Ar In Med, special issue on Informaion Visualizaion in Medicine. Forhcoming, [10] Chernoff H. The use of faces o represen poins in k- dimensional space graphically. J Am Sa Ass 1973: 68: [11] Chuah MC and Eick SG. Glyphs for Sofware Visualizaion. In 5h Inernaional Workshop on Program Comprehension (IWPC 97) Proceedings, pages Dearborn, Michigan: IEEE Compuer Sociey Press, [12] Messner P. Time Squares: A Two-Dimensional Represenaion of Temporal Aspecs in Skeleal Plans Evaluaion of Differen Approaches. Maser's hesis, Vienna Universiy of Technology, Vienna, Ausria, References [1] Seyfang S, Kosara R, and Miksch S. Asbru Reference Manual. Vienna Universiy of Technology, Insiue of Sofware Technology, Vienna, Technical Repor, Asgaard-TR , [2] Miksch S, Shahar Y, and Johnson P. Asbru: A Task- Specific, Inenion-Based, and Time-Oriened Language for Represening Skeleal Plans. In Proceedings of he 7h Workshop on Knowledge Engineering: Mehods & Languages (KEML-97). Milon Keynes, Address for correspondence Rober Kosara, Insiue for Sofware Technology, Vienna Universiy of Technology, Favoriensraße 9 11/E188, A Vienna, Ausria. rkosara@ifs.uwien.ac.a. hp:// hp://