OLE Coordinate System: Subjective Motion for Interactive Optical Illusion

Transcription

1 OLE Coordinate System: Subjective Motion for Interactive Optical Illusion Jun Fujiki Graduate School of Design Taketoshi Ushiama Faculty of Design Reiji Tsuruno Faculty of Design Kiyoshi Tomimatsu Faculty of Design Figure 1: The Screen of OLE Coordinate System Abstract We propose animation expression of a character in an illusionary picture that a user can control by arrangement of an object or a change of viewpoint. Examples of such animation expressions are: character movements based on a 2D interpretation of attached blocks which are not contiguous in three-dimensional space; falling motions on a single plane, etc. Furthermore, we propose a simple technique to implement these animation expressions using a program. Here, we use a technique with a uniquely defined space around the character, a technique using depth information, and a technique for seeking an object. We used these techniques to produce an OLE Coordinate System prototype and then confirmed the motion. CR Categories: I.3.8 [Computer Graphics]: Applications; J.5 [Arts and Humanities]: Fine arts. Keywords: optical illusion, interactive, motion, Escher 1 Introduction We are interested in an illusionary picture expression that makes a reader recognize impossibility. It is expected that illusionary pic- fujiki@gsd.design.kyushu-u.ac.jp ushiama@design.kyushu-u.ac.jp tsuruno@design.kyushu-u.ac.jp tomimatu@design.kyushu-u.ac.jp tures that can be explored and manipulated interactively using a computer can draw users into a more fantastic virtual milieu. In our previous study, we specifically examined the degree of user satisfaction, and developed a prototype of a block-based modeling system called Incompatible Block [Fujiki et al. 2006a], which was intended to give users a rich operative experience using a modeling interface as an illusionary picture. Incompatible Block uses a technique by which the block depth in a 2D movement was determined from the positional relation of an object for the screen [2006b]. It was confirmed that Incompatible Block piqued users curiosity and presented the possibility of enhancing the degree of user satisfaction. In this study, we focus expressions as illusionary pictures based on a drawn image that a user recognizes as impossible in the real world, and we propose animation expression of a character in an illusionary picture. A user can control the character by arrangement of an object or change of viewpoint. A character can move between blocks that are not connected in three-dimensional 3D space, or can perform a falling motion on the same plane, based on a two-dimensional (2D) interpretation. In this paper, we introduce such animation expressions using five illusionary pictures and describe an algorithm for performing the expressions. 2 Motivation We might sometimes want to express something that is not realistic. Many techniques by which images of an illustration tone and a picture tone are generated based on a 2D or 3D figure are also proposed [Hertzmann 1998] [Saito and Takahashi 1990] [Sousa and Buchanan 1999]. This report includes proposals for a technique for producing a surrealistic drawing method for application to animation [Meier 1996] [Kaplan et al. 2000], along with a technique to exaggerate the character motion [Kobayashi et al. 1998]. Also, some techniques for unreal perspective or projection are proposed [Vallance and Calder 2001] [Coleman and Singh 2004]. An illusionary picture shares commonality with such a drawing method

2 in the point that the illusionary picture is not realistic; that dilemma - the illusionary picture cannot actually exist - stimulates the viewers and users curiosity. We expect that such illusionary pictures, with which interactive operations are possible, would further enhance users curiosity. As an illustrative work in which illusionary pictures are used as a motif, M.C. Escher s works are famous in the field of art, but works that also use illusionary picture expressions are used in moving picture images. In Tsuruno s Animation of M.C. Escher s Belvedere [1997], the image of Belvedere of M.C. Escher rotates. In Yee s 3D Illusion in Motion [2006], a falling block is rendered as an impossible object. For animation work, development of a technique is not necessarily required, but an interactive illusionary picture requires a technique by which motion is enabled in real time. Among the techniques realizing expressions found in Escher s works, there is a technique that changes the original figure of Kaplan et al. to a plane-filling figure in the range maintained to some extent [Kaplan and Salesin 2000]. There is also a technique of Scott to change images such as a winding staircase so that they might be visible from every viewpoint [Scott 2002]. Our illusionary picture expression differs from those expressions on the important point that, to date, viewers of such images are made to recognize the relation of a figure and a character position because of some point: that point is inconsistent by a character s motion depending on the viewer s perspective. In recent years, prosperous development of this area of rendering has produced techniques by which a 3D figure is obtained from an image drawn like a sketch [Zeleznik et al. 1996] [Igarashi et al. 1999]. These generate or transform a figure based on a 2D strokes, but the output figure should preferably resemble a figure that a user expects. We need an algorithm to betray the anticipation of a user. In addition, a technique by which a figure is changed depending on a viewpoint is also proposed [Rademacher 1996] [Martin et al. 2000], but we must change the character motion depending on the viewpoint. We developed a simple, inexpensive technique by which our expression is performed in real time without using an vision technique technique requiring processing time [Sugihara 1986] [Huffman 1977] [Imai 1985]. We subsequently confirmed the obtained motion using a prototype system using an OLE Coordinate System. 3 User Interface of OLE Coordinate System A user can control the motion of two or more characters by arranging an object and changing a viewpoint. An example of a display screen of the OLE Coordinate System is shown in Figure 1. On the right-hand side in the canvas, the following tools stand in a line: a block tool to arrange a block (Figure 2(a)); a character tool to arrange a character (Figure 2(b)); a trapdoor tool to arrange a trapdoor (Figure 2(c)); a stair tool to arrange a stair (Figure 2(d)); a delete tool to delete an arranged object; and a rotation tool to change a perspective. The trapdoor tool might be switched to a platform tool for jumping to arrange a platform for jumping (Figure 2(e)); by clicking an icon using the right mouse button. The session is completed using the door tool located on the left-hand side of the canvas. Like inpute method used of some block-based modeling system [Jonathan 2002], For each object, the arrangement position is specified by a mouse click on the canvas. A block exists at the mouse click position; a block and stairs are arranged to touch the side of this block, along with an operation method for the existing block. A character, a trapdoor and a platform for jump are arranged so that they might be placed on the upper surface of a block when it Figure 2: Appearance objects is clicked with a mouse. One exception to the character, trapdoor and platform for the jump described above is placed on a point of intersection between a plane passing through the starting point and perpendicular to the view direction vector and the vector going to the direction of view line from the mouse-click position. Each object, aside from a character, is arranged along grid lines. The character arranged by a user advances automatically aiming at a previous block by one block and a central position of the upper surface of stairs. A character at the time of generation aims at the object nearest to the character. A character begins to walk in the case where it exists on a block, ascends and descends in the case where it exists on the stairs, falls in the case that a trapdoor exists on the block, and jumps up and down in the case where a jumping platform exists. When no path exists in a course or when a character approaches a branched path or a block, which becomes an obstacle, the direction of advancement is changed. A new target is determined among movable objects in four directions around the character when a character arrives at the target. Adding to such a motion imitates the real world, in an OLE Coordinate System, a character performs motions that can exist in the image drawn in 2D, although such a motion is impossible in the real world. A user can change these motions by arrangement of an object and changing of the viewpoint. 4 Design of Interactive Illusinary Picture Our illusionary picture expression visualizes the 3D figure according to a viewpoint change between the perspectives of two or more interpreters. The character motion is determined based on the 3D figure estimated from the 2D image drawn on the screen. The apparent size does not change according to depth when the 3D figure is drawn by parallel projection. Consequently, the 3D figure, as estimated from a 2D image, is not yet uniquely settled. As a result, the motion might sometimes produce some sequence of motion that cannot exist in the real world. Using this principle, we introduce such animation expressions as illusionary images with five characters. Furthermore, in the OLE Coordinate System, to emphasize such an ambiguous view of the world, objects were assumed as symbolic figures and only object outlines were drawn[gooch and Gooch 2001]. 4.1 Subjective Translation The character that is controlled by a user can move between discontinuous objects. Even if a block and stairs are discontinuous in 3D space in practice, they perform a continuous motion in the case that a viewpoint is changed, so that they can thereby be connected in the image drawn in 2D (Figure 3). Therefore, the movement motion by which a user can recognize the possible existence from the drawn image, although it cannot exist in practice, will be called Subjective Translation. Using such Subjective Translation, a user might sometimes recognize that the figure to which a character moves is an

3 impossible object. After the Subjective Translation is performed, when a viewpoint is rotated and moved, it can be confirmed that the viewpoint moves onto the actually separate block. Motion for the normal 3D space is performed when the Subjective Translation stops materializing during movement. Figure 5: Character acting as if there is a block 4.4 Subjective Absence Figure 3: Character crossing between non-contiguous blocks 4.2 Subjective Landing When a character arrives at a designated trapdoor, or a block, on which a character walked, the feature is deleted and the character falls. The character then lands on a block or stairs. Even if a character is not actually placed in 3D space, when the character exists at a position overlapping with a block or stairs in 2D image, the character can be actually placed on these objects so that a user recognizes that the character is placed on this (Figure 4). Here, the landing motion, which can be interpreted as possible from the 2D image, although it is impossible, will be called subjective landing. For the subjective landing, not only the falling position is on the same plane, but also falling on a plane with different height is possible. Landing onto a higher position than the falling start position also becomes possible through this arrangement. Furthermore, a character that came from outside the screen by falling disappears. After the subjective landing materialized also, it can be confirmed by viewpoint movement that a character is actually arranged on the object upon which the character landed. A character falls when it arrives at a trapdoor, and jumps on a platform for a jump. If the character arrived at a trapdoor and a platform for the jump in the 3D space in practice, however, when these exist in a viewpoint in which these are not drawn, the character moves so that a user recognizes that these objects do not exist (Figure 6). Thus, the movement motion disregarding influence by what is not drawn is called subjective absence. As for a situation in which a trapdoor and a platform for a jump are not visible, there are cases in which they are placed at a viewpoint from the down direction, in addition to those in which they are hiding behind a block and stairs or other obscuring object. This is because a trapdoor and a platform for jump can be arranged only on the upper surface of a block as an object. The normal motion for 3D space is performed when the subjective absence stops materializing during movement. Figure 6: Character disregarding invisible trap 4.5 Subjective Jump Figure 4: Character landing on a block that uses no steps When a character arrives at a platform for jump, it flies up so that a parabola of movement might be drawn. At that time, in the case of a viewpoint in which an object exists at the back of a character, the character is drawn so that it is positioned in front of the object (Figure 7). which implies depth forward and backward, depending on the position relation with the object. Therefore, a jump motion that imparts on the viewer the perception from the position relation with the object, is called a subjective jump. 4.3 Subjective Existence Even if a block were discontinuous, in the case that other objects are drawn in overlapping with a discontinuous portion in the 2D image, a character moves so that a user recognizes that a block exists in a place that is not visible (Figure 5). Movement motion reflects such recognition, in which a character is inferred to exist from the drawn image even though it does not exist in practice: this is called subjective existence. To distinguish it from Subjective Translation, the Subjective Translation uses continuity of appearance by two objects at a movement origin and a moving destination. On the other hand, in subjective existence, it is described that continuity of appearance is formed between the movement origin and moving destination by the effect of the third object. The normal motion for a 3D space is performed when the subjective existence stops materializing during movement. Figure 7: Character performing jump, disregarding distance 5 Technical Trick We describe our technique by which a program realizes an expression like an illusionary picture. In our expression like an illusion-

4 ary picture, the character motion is determined based on the image drawn on the screen. Furthermore, a block in our field is a cube, and is arranged along grid lines. Our expression of an illusionary picture is realized simply and rapidly using these features. 5.1 Subjective Translation A character advances one block at a time toward a previous block and a central position of the upper surface of the stairs. When a character arrives at the target, the direction of a course is determined from four directions around the character, and then a target object is selected. A left-hand method is used for determining the direction of a course for a character. The left-hand method is often used in robot control, and can be described in the following conditional branch. if leftward advance is possible then Turn to the left else if forward advance is possible then Advance forward else if rightward advance is possible then Turn the right As a situation in which a character cannot progress, in some cases, impediments exist on the course, and a path that the character used as a way of advancement does not exist as an option for regress. In such a situation, for objects that are apparently adjoining also, determination of a course is performed, and movement is made possible. A concrete sequential process is described below. Subjectivity buffer The central position of subjectivity buffer Figure 8: Subjective buffer Figure 8 shows a space for which object information is prepared for four blocks in height can be stored in the front and rear, right and left four directions. The space in which this object information is stored, is called a subjectivity buffer. The first step of the subjectivity buffer contains information for the point at which a character went down the stairs; the second step is object information around the block on which a character is located. In the OLE Coordinate System, because the character height is assumed as two blocks, the third and fourth steps are used for circumference information of the character. The initialization state stores information on an object, for which the 3D position corresponds with each subjectivity buffer. A screen coordinate for the central position of each subjectivity buffer is calculated. The black circle in Figure 8 shows a screen coordinate for the central position of the subjectivity buffer. A screen coordinate for the central position of a visible object around a character is compared with a screen coordinate for the central position of the calculated subjectivity buffer; then, in the case that these positions are approximately corresponding, object information on the subjectivity buffer is rewritten to information on the corresponding object. That is, when the following conditions are satisfied, it is considered that it corresponds. (B sx O sx ) 2 + +(B sy O sy ) 2 < L Here, Bsx and Bsy respectively designate an x coordinate and a y coordinate for the screen coordinate at the central position of the subjectivity buffer. Also, Osx and Osy respectively denote an x coordinate and a y coordinate for the screen coordinate at the central position of a visible object; L is the maximum distance, which is assumed to be corresponding. Course direction is determined applying the left-hand method for the subjectivity buffer, in which the above processing was performed for objects of all the character circumferences. When the course direction is determined, an object that is left one block in the course direction is specified as the next target. Screen Vc T Pc P1 Figure 9: Movement to discontinuous object When a character leaves from an object on which a character had been placed until then, the character is moved onto the target object so that the position visible from the screen is not changed (Figure 9). This position is an intersection point Pc between view direction vector Vc of a visual line, which passes along the step position P1 of a character, and the upper surface T of the object used as the movement goal target. Using only this processing, however, drawing might be interrupted in part by the object used as the movement purpose target, which is in this side immediately before a character enters the subjectivity buffer, as shown in Figure 10(a)). To avoid that situation, in addition to conditions associated with an object up to that point, drawn images are compared between a character and the object directly ahead of a character. In the case of another intersection also, the character is moved to the target object (Figure 10(b)). In the case of an angle from a lower direction, because it is preferred that the character be interrupted by the back object, it is compared with the back object. It is moved to the direction of the back object when it crosses. (a) Character of drawing interrupted by a front cube (b) Intersection judgment with front cube Figure 10: Incorrect drawing and its corrected version

5 5.2 Subjective Landing Regarding subjective landing, it should be viewed with the notion that a character falls near the center of an object as much as possible. A procedure is explained below by which a character lands near the center of the object that enables the fall to that point. First, it is assumed that a block exists in a vacant space, in the case that an object does not exist one block beyond from a block on which a character is placed, and a block exists in one block beyond. Next, the drawing process of a virtual block is tried. This processing is not drawn on the screen in practice. Information on the virtual block is provided to the subjectivity buffer corresponding to a position when this virtual block has no drawing pixels. A virtual block also is processed as a normal block when the course of a character is determined and a character is moved. B1 Pixel 5.4 Subjective Absence D1 D2 Figure 11: Selection of ideal landing object Several pixels are scanned from the screen coordinate of the character step to the lower direction, and objects drawn with pixels are found. Figure 11 shows that two blocks, B1 and B2, were identified by this scan. Merely using this processing, however, a character might not fall sometimes because the objects placed immediately after and before falling are also selected as candidates. Accordingly, in the case where a drawing of a character is interrupted by obstacles, the character falls normally. As for objects that should land, the following object is selected: the object for which the distance D from the x coordinate of the screen coordinate of the central position of an object to an x coordinate of the screen coordinate of the step position of a character is shortest. In Figure 11, because the value calculated for object D2 is less than that of object D1, object B2 becomes the object that lands. In Figure 12, Vc designates the view direction vector of a visual line, which passes along the step position P1 of a character. P2 denotes the center position of the landing target object and T is the plane, which is vertical to the block and turned to the direction of a camera and includes P2. After the character moved to an intersection point between Vc and T, the normal falling, landing and moving are applied. Screen Vc Pc B2 P2 Figure 12: Movement to object for landing 5.3 Subjective Existence Subjective existence specifies the place recognized by a user. Using subjective existence a discontinuous part that is invisible from a user exists at the back of an object, and forces the character to perform motion under the assumption that the discontinuous part is continuous. Here, we solved the problem by limiting the separation distance to one block and a target to a block. T P1 The visual state of a trapdoor and a platform for jumping can be judged by checking the existence of pixels drawn for the object. Processing is disregarded for the trapdoor or the platform for jumping, which have no drawn pixels. 5.5 Subjective Jump For a subjective jump, a character is always drawn before and after a rise regardless of depth judgment. It is not preferable, however, that the character that is hidden behind other objects appears suddenly in front immediately before starting the jump. In consideration of this, when even a part of the object that is drawn exists in the front of a character at the start of the jump, this processing is not performed until the state is prepared in which the whole character is drawn. 6 Conclusions and Discussion We introduced five animation expressions as illusionary pictures as milieus for a character that can be controlled by arrangement of an object and change of a viewpoint by a user. Furthermore, we described the algorithm simply and at high-speed performance these expressions. We confirmed the effectiveness of these algorithms for motion using a prototype system OLE Coordinate System. Nevertheless, some problems and points remain in these algorithms, which should be improved. They are described below. In Subjective Translation, even if the screen position of the central position of the subjectivity buffer, as shown in Figure 13, does not correspond with the screen position of the center of an object, a situation that should be materialized, exists. The above case, however, cannot be handled by this technique. In subjective landing, it would be better to enable landing on the central position of an object using a higher-precision method. For subjective existence, in this Figure 13: Impossible case for Subjective Translation stage, only the one inter-block left block corresponds. We will improve the technique to show correct correspondence to greater distances and stairs also. The most important points in our study are freshness as illusionary expression. However, we believe that a technique by which these expressions can be handled systematically for an illusionary picture might engender discovery of a newer expression. Finally, two motions like Escher s works produced using the OLE Coordinate System are shown in Figure 14.

6 (a) Ascending and Descending (b) Waterfall Figure 14: Two motions created using OLE Coordinate System References COLEMAN, P., AND SINGH, K Ryan: rendering your animation nonlineary projected. In Proceedings of NPAR 2004, FUJIKI, J., USHIAMA, T., AND TOMIMATSU, K., Incompatible block: Wonders accompanied interface. ACM SIGGRAPH 2006 Emerging Technologies #16, July. FUJIKI, J., USHIAMA, T., AND TOMIMATSU, K., A vieworiented interface for block-based modeling. ACM SIGGRAPH 2006 Sketches #164, July. GOOCH, A., AND GOOCH, B Non-Photorealistic Rendering. A. K. Peters. HERTZMANN, A Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of ACM SIGGRAPH 98, ACM, HUFFMAN, D. A Realizable configurations of lines in pictures of polyhedra. Machine Intelligence 8, IGARASHI, T., MATSUOKA, S., AND TANAKA, H Teddy: a sketching interface for 3d freeform design. In Proceedings of ACM SIGGRAPH 99, ACM, IMAI, H On combinatatorial structures of line drawings of polyhedra. Discrete Applied Mathematics 10, JONATHAN, T. B., Gollygee blocks : A 3d modeler for children. ACM SIGGRAPH 2002 Sketches & Applications #127, July. KAPLAN, C. S., AND SALESIN, D. H Escherization. In Proceeding of SIGGRAPH 2000, ACM, KAPLAN, M., GOOCH, B., AND COHEN, E Interactive artistic rendering. In Proceedings of NPAR 2000, KOBAYASHI, M., KONDO, K., AND SATO, H Emphasized expressions using motion filter in creating animation. In Proceedings of the 8th ICECGDG Conference, vol. 4, MARTIN, D., GARCIA, S., AND TORRES, J. C Observer dependent deformations in illustration. In Proceedings of NPAR 2000, MEIER, B. J Painterly rendering for animation. In Proceedings of ACM SIGGRAPH 90, ACM, RADEMACHER, P View-dependent geometry. In Proceedings of ACM SIGGRAPH 99, ACM, SAITO, T., AND TAKAHASHI, T Comprehensible rendering of 3d shapes. In Proceedings of ACM SIGGRAPH 90, ACM, SCOTT, M. W., Implementing the continuous staircase illusion in opengl. ACM SIGGRAPH 2002 Sketches & Applications #200, July. SOUSA, M. C., AND BUCHANAN, J. W Computergenerated graphite pencil rendering of 3d polygonal model. In EUROGRAPHICS 99 Conference Proceedings, SUGIHARA, K Machine Interpretation of Line Drawings. MIT Press. TSURUNO, S The animation of m.c. escher s belvedere. In ACM SIGGRAPH 97 Visual Proceedings: The art and interdisciplinary programs of SIGGRAPH 97, 237. VALLANCE, S., AND CALDER, P Multi-perspective images for visualization. In Proceedings of the Pan-Sydney area workshop on Visual Information Processing, vol. 11, YEE, L. S., d illusion in motion. ACM SIGGRAPH 2006 Animation Theater #180, July. ZELEZNIK, R. C., HEMDON, K. P., F., J., AND HUGHES Sketch: an interface for sketching 3d scenes. In Proceedings of ACM SIGGRAPH 96, ACM,