An Integrated Simulation System for Human Factors Study

Transcription

1 An Integrted Simultion System for Humn Fctors Study Ying Wng, Wei Zhng Deprtment of Industril Engineering, Tsinghu University, Beijing , Chin Foud Bennis, Dmien Chblt IRCCyN, Ecole Centrle de Nntes, F44321 Nntes Cedex 3, Frnce Abstrct It hs been reported tht virtul relity cn be useful tool for ergonomics study. The proposed integrted simultion system ims t mesuring opertor s performnce in n interctive wy for 2D control pnel design. By incorporting some sophisticted virtul relity hrdwre/softwre, the system llows nturl humn-system nd/or humn-humn interction in simulted virtul environment; enbles dynmic objective mesurement of humn performnce; nd evlutes the qulity of the system design in humn fctors perspective bsed on the mesurement. It cn lso be for opertion trining for some 2D control pnels. Keywords: virtul relity, humn fctors, ergonomics, simultion, mnikin 1. Introduction Humn fctors engineering requires tht the design of system should consider the humn s spects. In other word, ny designed system should be user-friendly. However, in mny cses, it is not esy to stisfy this requirement becuse it is difficult to quntify the humn s requirement or the humn s performnce. Sometimes, itertive design t the cost of time nd money is necessry. In order to reduce the itertions for time nd cost concern, designers my use one of the following three pproches, or combintion of them. The first pproch is tht the designers hve the required quntittive dt or enough cler understnding bout the users. The dt my include the nthropometricl dt bout the users, the cpbility of the users, nd the specific scenrios of the system use. For exmple, by hving the nthropometricl dt nd the scenrio of use, one cn design chir tht is suitble for specific percentge users nd the specific use scenrio. However, in mny situtions, especilly for new pplictions, hving these dt is not possible. As n lterntive pproch, one cn use physicl prototypes to test the design. Now with rpid prototyping technology, one cn obtin physicl prototype directly from its CAD model within one dy or only hours. By testing the interction with the user, the designer cn esily find the problems nd then quickly modify the design nd test gin. Mny pplictions hve shown tht the use of rpid prototyping technology cn effectively reduce the itertion time nd improve the design qulity [1]. However, in some cses, the designed system cn be so complicted tht hving its prototype cn be time consuming nd costly. In some cses, it is even impossible. The third pproch is using virtul prototype insted of physicl one. Arthur et l. hve demonstrted tht the

2 perception, nd sptil knowledge cquisition in the virtul world ws indistinguishble from tht of the ctul physicl representtion of the rel-world lyout for certin tsks [2]. According to Buck (1998), ergonomics studies in virtul prototypes llow in erly design stges to verify whether the user interfce of product is user friendly or not. The typicl opertions nd procedures cn be crried out in wy, which is close to relity. The design of the virtul environment nd the provided virtul relity interction tools should llow the user to mke ll the necessry movements. In this wy, ergonomic flws re very probble to be detected in n immersive virtul environment, nd cn be corrected t n erly design stge, which sves lot of effort [3]. Wilson (1999) hs demonstrted tht virtul relity cn be potentil tool to support mny types of ergonomics contribution, including ssessments of office nd workplce lyouts, testing consequences for rech nd ccess, reconfiguring nd evlution of lterntive interfce designs, checking operting or emergency procedures, nd trining for industril nd commercil tsks [4]. Sonod et l. hve developed power plnt trining simultor bsed on virtul relity. The developed trining simultor cn relize field spce s virtul spce nd cn hve fst disply of lrge-scle 3D models, 3D user-interfces, nd the stereoscopic vision nd sound effects. The trinee cn trin in field opertions similr to the rel world [5]. The objective of our reserch is to develop n integrted system tht incorportes virtul relity techniques for humn fctors study. More specificlly t present, this system is for 2D control pnel design evlution nd improvement, nd for trining. However, by hving stereo projection, it is potentilly possible for 3D system simultions. 2. System Structure For humn fctors study, the developed simultion system should be ble to cpture the user s opertion, such s the opertor s hnd position, gesture, nd be ble to recognize the specific opertions (e.g. pushing button, tuning knob, or turning on switch etc.). On the other hnd, in order to simulte 2D control pnels of different size in n ctul scle, the system is required to hve screen with enough size for most pplictions. For ccurte mesurement of user s opertion, the system should hve quick responding fetures. The min components of our hrdwre system re shown in Figure 1. There re four CCD cmers to detect the opertor s hnd position. A pir of 5DT Dt Gloves is used to cpture the opertor s finger gesture. By combining the hnd position nd gesture informtion, the detection computer cn recognize the opertor s opertion, such s pushing button or tuning knob, etc. The recognized opertion is then sent to the simultion computers vi LAN network. The simultion computers, depending on the screen size (width: height), my be ny number. In our system, the screen size is 7.6 meters wide nd 2 meters high. So we used three synchronized chnnels, which mens there re three simultion computers nd three projectors. The simultions computers work together, run the simultion scenrio nd the logic nd displys the virtul pnel to the wide screen through three 3600 lumens projectors. Then the opertor cn reltimely interct with the virtul control pnel, such turning on light by pushing certin button, or chnging quntity vlue by tuning certin knob. For simultion purpose, it is needed to hve designed control pnel model. The model includes the geometric model nd the control logic model. The geometric model defines the geometry of the pnel s well s the textures to mke it intuitive. It my be designed using ny modeling softwre pckges, such s Multigen Cretor TM, 3D MAX, etc. Figure 2 shows one simplified geometric model of control pnel, with six meters, six lights, six buttons, nd

3 two disply screens. The logic model defines the control logics, for exmple, in order to turn on certin light, which button or buttons should be pushed. Tble 1 lists prt of the logics using reltion mtrix. According to Tble 1, when the opertor pushes the Red button, the control logics will mke R-1 nd R-2 lights on. At the sme time, he/she will her n udio feedbck, which is n lrming udio. The logics will be reflected in the developed progrm. 4 CCD Cmers Video imges 3 Projectors Dt glove Gesture Detection Computer Spekers Screen Recognized Opertions Simultion Computers Figure 1. Hrdwre components of the simultion system Figure 2. Geometric model of simplified control pnel 3. Motion Trcking nd Recognition 3.1 Finger Position Trcking Finger position trcking is relized by four CCD cmers. A smll ctive light, s the position mrker, is mounted on the opertor s index finger. CCD pertures re reduced such tht only the ctive lights re visible in the imge, ll

4 the other objects in the environment, including the screen with projection, re not visible. This cn insure enough contrst between the mrker nd the other objects. Before simultion, the screen re must be clibrted in order to estblish the mpping reltionship between the globl coordinte system nd the pixel coordinte system in the four CCD cmer imges. In other words, the mrker in ech specific position in the globl coordinte system hs corresponding pixel coordintes in the four CCD imges. In this wy, during simultion, the detection computer cquires the CCD imges every 40 milliseconds nd computes the mrker s position ccording to the clibrted reltionship. This cn be expressed s follows: x y = z i j where x, y, z re mrker s globl coordintes, 11, 12,, 32 re trnsformtion mtrix which is obtined by clibrtion, i nd j re mrker s pixel coordintes in ech CCD imge [6]. (1) Tble 1. Prt of the control logics Button Audio Lights Screens Meters Effect R-1 R-2 R-3 Y-1 Y-2 Y-3 G-1 G-2 Left Right Red Alrm On On Yellow Cution On On White Audio On On On On On A file Blck Audio Off Off Off Off Off Off Off Off Off B file Left Audio C file On Next Slide Right Audio D file On Next Slide Defult Env Flk Flk * R: Red Light Y: Yellow Lights G: Green Lights Env: Environment sound Flk: Flicker 3.2 Finger Gesture Trcking Finger gesture informtion is provided by the dt glove. The glove detects ech finger s gesture nd expresses the gesture using number between 0 nd 255. By reding the number for ech finger, the detection computer cn obtin the gesture of ech finger. 3.3 Opertion Recognition Currently, we defined three kinds of opertions: pushing button, tuning knob, nd turning on switch. All these three opertions cn be recognized by nlyzing the finger s position chnge nd the gesture. They re defined s follows:

5 Pushing button is TRUE when: 1) Index finger (with mrker) is in vlid button zone, nd 2) Index finger is open (gesture vlue over certin threshold), nd 3) The finger ccelertion nd decelertion re over certin threshold (i.e. pushing ). Tuning knob is TRUE when: 1) Index finger (with mrker) is in vlid knob zone, nd 2) Thumb finger nd the other four fingers form hold gesture, nd 3) The reltive gesture chnge of the fingers forms clockwise or counterclockwise tuning Turning switch is TRUE when: 1) Index finger (with mrker) is in vlid switch zone, nd 2) Thumb finger nd the other four fingers form hold gesture, nd 3) The finger ccelertion nd decelertion re over certin threshold (i.e. push or pull ). 3.4 Opertion Acquiring The recognized opertions bove re cquired in rte of 25 times per second. The cquired dt is sved both in the detection computer nd in the simultion computer (server computer only, slve computers don t sve). After testing, the reserchers cn retrieve the dt nd nlyze. For exmple, how mny errors did the opertor mke, in wht kind circumstnces did the opertor mke the errors, wht is the shortest, verge, nd longest response time between certin events (e.g. when n lrm signl is given by light, how long did it tke for the opertor to respond to the lrm). 4. Multi-Chnnel Synchroniztion The simultion softwre is developed on the pltform of Multigen Veg TM. Veg hs module of Distributed Veg, which enbles multi-computer synchroniztion [7]. The synchronized computers will not compute nd updte the next frme if ny synchronized computer hs not finished the current frme. In our system, we used three simultion computers. One of the computers is defined s server, the other two s slves. The server computer reds dt trnsmitted from the detection computer vi the network, nd then distributes the dt to the slves through Distributed Veg module. Ech of the three computers drives one projector for one disply chnnel on the wide screen. Besides synchroniztion, it is lso needed to overcome the chnnel overlp problem. Otherwise, the overlpped zone will hve higher brightness thn the other res. This problem is solved by chnnel overlp blending techniques. By pplying series of continuously chnging gry brs in the overlp zone, continuous brightness is chieved long the three chnnels. Figure 3 shows the blended result of the simultion system. In this simultion, we used the geometric model shown in Figure 2 nd the logics listed in Tble 1. Due to the CCD specifictions (25Hz, 768*576 pixels), currently, this system hs time resolution of 40 milliseconds (25Hz), nd sptil resolution of cm on the screen, depending on the loction of the screen. This performnce cn be further improved if high-speed CCD cmers with higher imge resolutions re used. However, it is lso possible to use more CCDs to increse the sptil resolutions. Time resolution seems enough for most testing pplictions.

6 Figure 3. Multi-chnnel blending nd simultion results 5. Conclusions In this reserch, wide-screen simultion system ws developed. It is especilly suitble for 2D control pnel testing nd trining. Opticl motion trcking nd wireless dt gloves eliminte the requirement of cbles between the opertor nd computer. Distributed Veg llows synchroniztion of multi-computers, which enbles virtully unlimited number of computers, s well s unlimited number of projectors for lrge screen simultion. The current stge of this reserch hs demonstrted the fesibility of using virtul relity for humn fctors study. In the future, prcticl control pnel design evlution nd trining ppliction will be tested on this system. More mrkers will be trcked in order to obtin more informtion bout the whole body movement. Acknowledgements The uthors would like to cknowledge the finncil support from the Ntionl Nturl Science Foundtion of Chin under grnt number We would lso like to cknowledge the resercher exchnge progrm in Ecole Centrle de Nntes, Frnce, for mking the collborted study possible. References [1] Di, F. (ed.), 1998, Virtul Relity for Industril Applictions, Springer, Berlin, 2-3. [2] Arthur, E., Hncock, P. A., Chrysler, S. T., 1997, The Perception of Sptil Lyout in Rel nd Virtul Worlds, Ergonomics, 40 (1), [3] Buck, M., 1998, Chpter 2: Immersive User Interction within Industril Virtul Environment, ppers in Virtul Relity for Industril Applictions, Dn, F. (ed.), Springer, Berlin, [4] Wilson, J.R., 1999, Virtul Environments Applictions nd Applied Ergonomics, Applied Ergonomics, 30 (1), 3-9. [5] Sonod, Y., Yoshimoto, N., Morimur, K., Nktni, T., Mtsuzki, S., 2000, Development of Plnt Trining Simultor bsed on Virtul Relity, Technicl Review - Mitsubishi Hevy Industry, 37(1), [6] Bnerjee, P., Zetu, D., Virtul Mnufcturing, John Wiley & Sons, [7]