2016 International Conference on Artificial Intelligence and Computer Science (AICS 2016) ISBN: 978-1-60595-411-0 Natural User Interface Design for Digital Painting Based on User Behavior Patterns Jing SHEN 1,*, Liang CHEN 2, Zhi-nan QIN 3 and Wei-jing HE 4 1-4 School of Design, South China University of Technology, Guangzhou, 510006, China *Corresponding author Keywords: Interaction design, Behavior patterns, Natural user interface. Abstract. This research aims to improve users experience by changing existed interaction systems with NUI-assisted. We first present a new NUI design procedure based on user behavior patterns. Then, we analyze the user group behavior patterns in digital painting, and develop a gesture interaction prototype by Leap Motion. According to results of evaluation experiment, the usability and user experience for this interaction system are improved obviously. Introduction The concept of nature interaction promotes the development of NUI (Natural User Interface) which is an invisible interface.it benefited from the diversification of input devices and the development of information recognition[1]. Recent researches show that users prefer to use familiar or intuitive interface, rather than a new one requiring additional knowledge [2], since they usually have no enough patience of study in digital environment. In GUI (Graphical User Interface) mode, they need to learn the operation methods of a mouse or a keyboard, and make themselves adapt to the rules of input devices before start working. The communication between the people and the system here is an unnatural interaction which is sense missed and behavior restricted [3]. In the natural interaction systems, interactions are more like instinctive behaviors in daily life. Users can interact with the system naturally without extra learning. As NUI interaction mode is significantly different from GUI, it is possible to avoid visual distractions, allowing users to focus more on the task itself. And it restores the way that people interact with real objects in digital environment, so it has been widely used in virtual reality technology, digital control, 3D design and display, etc. It is also the hot topic in the field of novel interaction researches [4-6]. On the other hand, since there are many differences between these two modes, users who used to use GUI systems feel very hard to follow the transition process or even refuse to change. Currently there are a lot of human-centered methods for interaction design, such as unconscious cognition, activity-based context awareness, feedback and habits training, etc. [7-11]. Compared to GUI, NUI is a new concept, and there is no standard principles or assessment methods for it yet. In this paper, we propose a new NUI design method for digital painting based on User Behavior Patterns. This method focus on user group behavior patterns, explores design chances from the dynamic evolution of user habits. In the same time, stabilized habits can guide decisions making of design, reduce cognitive conflicts and achieve stable work-flow in new systems. User Behavior Patterns and Interaction Design The design procedure is showed in Figure 1. First, in User Research phase, users' working hours will be investigated in order to define persona and goals, make sure all design works are focus on the target users. Further, users' goals are divided into several sub-goals and some sub-set are built up. All these data are useful for defining new system functions. The most important is that we define user group behavior patterns analysis to help decision-making. Here is introduction to behavior patterns: 183
Users usually take different ways to achieve sub-goal. If one user always finishes his task in one way, his behavior can be defined as Stabilized Behavior Pattern f (. Similarly, Dynamic Behavior Pattern f '( can be defined as changing behavior. And a dynamic behavior pattern can be seen as consisting of a number of stabilized behavior patterns, namely: 1 f1( 2 f2( 3 f3( ' n f f ( 1 2 3 n 1 is behavior pattern weight, may be referred to the Evolution Coefficient in this habit evolutionary process. If only one factor is not 0, this formula shows stable behavior pattern. For each sub-goal, collecting all evolution coefficients can build Behavior Pattern Matrix, as shown as formula (2). 11 21 i 1 12 22 i2 1 n 2n in If most of the value coefficients are zero, that means the interaction pattern has high recognition in users group: including human subconscious wareness of natural things and common senses in digital environment, facilitating the idea of natural interaction strategies in the new system. Evolution of the group behavior states are described by the complex matrix. It is easy to find that the more non-zero coefficients in the matrix, the more complex for the evolution, namely, this interaction mode has a high cognitive complexity and need much time to learn. So the user behavior pattern matrix, mental model are the basic fators for a new natural interation system design. The status of the break downs of work-flow, cognitive efficiency and SUS can be used to evaluate the usability of the system, in order to achieve the requirements of low cognitive load, high efficiency and low interference. n ( (1) (2) Figure 1. Design procedure based on behavior patterns. Digital Painting System Design with User Behavior Patterns Painting software is very useful for a painter. But in digital environment, tools and effects are not always the same as those in real world. When people turn to draw in digital environment, he need to 184
learn a lot of things about the software. We apply our method in a digital panting system design, in order to lower users' mental workload in a novel painting interaction system. User Behavior Patterns Study 50 painters were selected from some websites, including 20% beginners and 12% experts. All these users do their artworks with painting software SAI in usual situation. Their working process was recorded for further analysis. Sub-goals appeared differently because of their various painting habits. The times of each sub-goal appeared in these users' working process was calculated. System functions along with operation frequency are listed in Table 1. Table 1. High frequency part of sub-goals. Sub-goal Times Sub-goal Times Sub-goal Times Brush size 50 Pencil 38 Flip horizontally 28 Zoom out/zoom in 50 Show/Hide layer 37 Merge down layer 28 Color wheel 50 Free deformation 36 Lasso 27 New layer 50 Composition mode 36 Selection pen 26 Pick layer 50 Pick color 34 Set deformation mode 26 Move canvas 48 Pen 34 Filter 25 Undo 48 Magic wand 34 Selection 23 Eraser 47 Air brush 33 Clipping group 22 Clear selection 43 Fill 32 Set layer opacity 39 Bucket 29 In this context, users' interaction behaviors can be classified as WIMP (Windows, Icons, Menus, Pointer) behavior and Shortcuts behavior.these behavior patterns can be shown on a scatter diagram(see Figure 2), a value of 0 indicates WIMP behavior and a value of 1 indicates Shortcuts behavior. Behavior Patterns Analysis Figure 2. Scatter diagram shows behaviors patterns. Most of the behavior patterns shown in Figure 2 biased toward WIMP, which means using pointer can complete most operations in the painting process. But observation found that stylus is not suitable for click operation. When users click icon with a stylus, a pen-down shaking often cause that action be recognized as double-click even drag event. There are some stable behavior patterns belong to 185
WIMP, but shortcuts behavior patterns are mainly dynamic. Enormous cognitive load comes from recalling keystroke combinations and the impact of cognitive transfer. When similar functions have the same shortcuts, they can be quickly accepted by users in other software. Otherwise the shortcuts will conflict with those behavior patterns user formed, definitely produce negative effect on work-flow. Some of the sub-goals have no obvious tendency, like those in the vicinity of 0.5 values in Figure 2. For these sub-goals, people are adapting to shortcuts in order to improve painting efficiency, but when they forget keystroke combinations or current icon position, their behavior pattern will still be WIMP. Prototype and Testing To deal with these existing problem, we have some design principles here: Define nature painting behavior from basic holding pen posture, reducing redundant action. Redesign those interactions in state of dynamic behavior patterns, to reduce the negative impact of cognitive migration. Rational use of cognitive migration phenomenon. Those widely accepted shortcuts (such as Undo) ensure working efficiency, changing may increase unnecessary cognitive load. Then, we had a usability test on interaction system prototype using Leap Motion. In this system, users can use the stylus with three natural postures to switch corresponding drawing tools, such as pen, brush and air bush. They can also flip canvas horizontally with flip gesture. The feedback styles are icon highlighted still. Each participator had a completely paint work in new system prototype and original GUI system. Testing scale is made based on nine of statements in SUS [12], the average scores of results are shown in Table 2. Testing results show: The new system achieve higher score in the statements of simplicity, ease of learning and feedback reasonably. The work-flow in new system is significantly stable than original one. When using those paint tools in new system, users can react naturally without any hesitation. This process is faster than the original system which required visual search process. Conclusion Table 2. High frequency part of sub-goals. SUS statements New NUI Original system system I think that I would like to use this system frequently. 3.60 3.72 I found the system very simple. 3.60 3.52 I thought the system was easy to use. 3.40 3.48 I think that I wouldn't need the support of a technical person to be able to use this system. 2.80 2.96 I thought there was no inconsistency in this system. 2.80 2.76 I would imagine that most people would learn to use this system very quickly. 3.80 3.76 I found the system very intuition. 3.20 3.44 I felt very confident using the system. 3.20 3.44 I don't need to learn before I could get going with this system. 3.40 3.28 Behavior patterns researches can help built new NUI interaction which GUI user can quickly become accustomed to. This paper presents a NUI design method based on behavior patterns study, proves its value in the application of digital painting system design. Users can enjoy new interaction mode even it is far different form the one before. Today a large GUI users are so familiar to those visual elements basic interaction. They may refuse to spend extra time to adjust to significant changes that NUI brings to, resulting in NUI is difficult to widely used in the existing systems. Behavior patterns keep changing. Its development analysis may can be implied to assess the efficiency of guidance in the system. That will be part of the future work. 186
References [1] Norman Donald A. Natural user interfaces are not natural [J]. Interactions, 2010, 17(3): 6-10 [2] Steane Jamie. The Principles and Processes of Interactive Design [M]. A&C Black, 2014. [3] Malizia Alessio, Bellucci Andrea. The artificiality of natural user interfaces[j]. Communications of the ACM, 2012, 55(3): 36. [4] Wigdor Daniel, Wixon Dennis. Brave NUI world: designing natural user interfaces for touch and gesture [M]. Elsevier, 2011. [5] Tan Hao, Li Wei, Tan Zheng-yu. 3D Gesture Interaction Product Design of Vehicle Information System [J]. Packaging Engineering, 2015, (18): 45-48+53. [6] Fu Xin-sha, Long Li-dun, Li Hai-feng, et al. Method and System Architecture of True Three-Dimensional Highway Alignment Design Based on Motion Sensing Interaction [J]. Journal of South China University of Technology (Natural Science Edition), 2014, (08): 91-96. [7] Xie Wei, Xin Xiangyang. Research on interaction design based on unconscious cognition [J]. Modern Electronics Technique, 2016, (12): 22-25+31. [8] Li Xue-yuan, Ji Tie, Wang Wei. The Methods of Developing User Habits Based on Feedback [J]. Packaging Engineering, 2015, (24): 79-82. [9] Fu Jie, Zhao Jiang-hong, Tan Hao. Affordance in Interaction Design Based on Unconscious and Behavior [J]. Packaging Engineering, 2013, (02): 50-52+82. [10] Chen Yuanyuan, Liu Zhengjie. Activity-based context awareness interaction design. Computer Engineering and Applications, 2013, 49(20): 23-28. [11] Jetter Christian, Gerken Jens, Reiterer Harald. Natural User Interfaces: Why We Need Better Model-Worlds, Not Better Gestures [J]. 2010. [12] Brooke John M. Sus: A 'Quick and Dirty' Usability Scale [M]. 1996: 187