Implementation of Kinetic Typography by Motion Recognition Sensor Sooyeon Lim, Sangwook Kim Department of Digital Media Art, Kyungpook National University, Korea School of Computer Science and Engineering, Kyungpook National University, Korea {sylim,kimsw}@knu.ac.kr Abstract. Kinetic typography is a technology based on the changes of the color of the character, size, and location that delivers appreciation of the beautiful and creativity. This research's main purpose is in to use the depth-camera to make natural interaction between users and kinetic typography. We used motion recognition sensor in order to track user's skeleton informations to synchronize with typography. Kinetic typography does not only deliver simple information, it also allows high dimensional communication such as emotional expression. Keywords: Kinetic Typography, Motion Recognition, Kinect, Interactive Typography 1 Introduction Recently, the development of IT led various methods of communication, especially using characters to deliver information is called Typography. Typography is the art and technique of arranging type in order to make language visible. The arrangement of type involves the selection of typefaces, point size, line length, leading (line spacing), adjusting the spaces between groups of letters and adjusting the space between pairs of letter[8]. In offline, typography shows two-dimensional characteristics whereas kinetic typography shows three-dimensional characteristics which are practical movement in the monitor or screen and sound effects. The application extent of Kinetic typography has been extended by development of audiovisual medium and the internet. Now typography is in evolution becoming dynamic, complex kinetic typography put together image and sound[3]. Kinetic typography provides time and motility in the 3D space and continuously shows through traffic line. This premises communication and a user who approached to kinetic typography will concentrate and aim for interaction actively with great interest. Currently, most of kinetic typography are made by professional motion program such as After Effects, Premiere, and Flash etc. which are animated by designer s sense and story. This research is to make flexible stories that can change by user's intention. To do this, motion recognition sensor is used to tract user's skeleton information and synchro-
nize gained information to the typography. Typography, which shows interaction between user, the new word 헐 [heol] is used because it is the best way to express modern people's psychology. 2 Backgrounds 2.1 Natural User Interface The development of sensors, there is new interface which can replace keyboards or mouse. New interface called NUI(Natural User Interface) is attracting attentions. NUI is a technology that performs based on the natural movement. Therefore, it is extremely cheap. In addition, users cannot see the interface and without any extra equipment, sensors or camera can directly use human body as an input device and control the contents. Thus, users feel they are truly governing the technology this will increase user experience. However, though gesture occupies a large portion of NUI, there is a problem its meaning changes according to the cultural background. NUI which is based on the user's natural movement, it materialize sensors: multitouch, motion recognition, and computer vision, etc. Motion and voice recognition technology is applied to game interface to popularize existing special field called microsoft's kinect sensor. 2.2 Korean New word- 헐 [heol] In the respective countries, hundreds of new words are created according to the change of the period. When many people use them for a long time, they will register as a real word. 헐 [heol] is an example of a Korean new words. It is an exclamation used to express absurd situation and it simplifies complicated psychology of young's modern tendency. The reason our research topic is 헐 [heol] is, it matches with our purpose of changing the image of the expression. Our purpose is to input various expressions and movements to the word 헐 [heol] and feel the freedom of the soul of oneself and watching his reality in the modern society. 3 An Implementation of a Kinetic Typography 3.1 Design composition This research uses Kinect sensor in order to create interacting user and kinetic typography. Kinect uses infrared ray and camera to recognize 20 joints from human body and find coordinate value from a person. We use 15 joints information among 20 joints recognized by the Kinect.
Figure 1 shows a connection of 15 joints coordinate value to form a shape of a character. When we imaginary a character, we considered about esthetic factor and gave offset value to a real coordinate value in each individual joints. Fig. 1. Character imagery using user's skeleton information 3.2 System Configuration Once kinetic typography system recognizes the user skeleton information, it activates and start to draw a shape following the skeleton information in the interaction time. We set up interaction time to 20 seconds considering user's fun. After 20 seconds, RGB camera in kinect is used to film the users figure. Filmed image of the user is saved in resolution of 640*480 and play when it is in idle state. Figure 2 shows visual transformation of kinetic typography by user's motion. Fig. 2. Visual transformation of kinetic typography
Fig. 3. Data flow in kinetic typography system Kinect is able to recognize six people but it has a characteristic only two people s skeleton informations are tracked completely at the maximum. So our proposed system is for a one or two user. We developed our system with OpenCV 2.3.5, Microsoft's Kinect SDK 1.7 in integrated development environment Visual Studio 2010. OpenCV is one of open source
vision C library for real-time image processing. We used it to draw shapes such as line, rectangle, circle, ellipse for typography. 4 Conclusions Kinetic typography is a multimedia contents which both contains artistry and industrialization. Currently, based on the media of the image, it did not only apply movement, advertisement, and motion graphic but also applied media art related to smart devices. This research proposed natural interactive kinetic typography using user's skeleton information. User s skeleton information is tracked by the motion recognition sensor. The proposed method is not based on a designer predefined moving story. It synchronized kinetic typography with user's motion information. Our system has been released to the public through exhibitions. As a result, users was actively interact with great curiosity. And we can maximize the user's emotional effects. Kinetic typography does not only deliver simple information, it also allows high dimensional communication such as emotional expression. Therefore, there should be a development of library which can expand to express various words not only expressing specific words. 5 Acknowledgments. This research is supported by Ministry of Culture, Sports and Tourism(MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology(CT) Research & Development Program(Immersive Game Contents CT Co-Research Center). 6 References 1. A. Barmpoutis: Tensor Body: Real-Time Reconstruction of the Human Body and Avatar Synthesis From RGB-D. IEEE Transactions on Cybernetics, Vol. 43, Issue 5, pp. 1347-1356 (2013) 2. H. Kim. Y. Kim. I. Lee: Interactive Installations Using Kinect - Case Studies. In: Proceedings of Korea Information Science Society, Vol. 38, No. 2, pp. 245-248 (2011) 3. H. Lee. S. Park: A Study on Experimental Artwork related to Sound, Typography and Movement Focused on Making Music Video with Kinetic Typography. Info DESIGN ISSUE, Vol. 10, pp. 17-28 (2006) 4. S. Ford, J. Forlizzi, S. Ishizaki: Kinetic typography: issues in time-based presentation of text. In: Proceedings of CHI '97 Extended Abstracts on Human Factors in Computing Systems, pp. 269-270 (1997) 5. T. Pimentel, V. Branco: Dynamic and interactive typography in digital art. Computers & Graphics, Vol. 29, Issue 6, pp. 882-889 (2005) 6. T. Xiaolong, X. Pin, Y. Xing: Research on Skeleton Animation Motion Data Based on Kinect. In: Proceedings of 2012 Fifth International Symposium on Computational Intelligence and Design, Vol. 2, pp. 347-350 (2012)
7. Z. Yeo, S. Hudson: KTE2: an engine for kinetic typography. In: Proceedings of CHI '09 Extended Abstracts on Human Factors in Computing Systems, pp. 3413-3418 (2009) 8. http://en.wikipedia.org/wiki/typography