Interactive Computational Intelligence: A Framework and Applications

Transcription

1 1 Interactive Computational Intelligence: A Framework and Applications

2 Agenda Overview Methodology IGA Knowledge-based encoding Partial evaluation based on clustering Direct manipulation of evolution Applications Media retrieval Media design Concluding remarks 2

3 Overview Interactive Computational Intelligence Interaction/Evolution Human Computer Interface (Emotion) User Modeling AI (Soft Computing) (Efficiency) FL GA 3 NN

4 Interactive Evolutionary Computation (IEC) Overview Difficulty of optimization problem in interactive system Outputs must be subjectively evaluated (graphics or music) Definition Evolutionary computation that optimizes systems based on subjective human evaluation Fitness function is replaced by a human user Target system f(p1,p2,,pn) System output My goal is Interactive EC Subjective evaluation From Takagi (2001) 4

5 Overview Technical Problems in IEC Human fatigue problem Common to all human-machine interaction systems IEC Acceleration of EC convergence with a small population size and a few generation numbers Usually 10 or 20 search generations Limitation of the individuals simultaneously displayed on a monitor Limitation of the human capacity to memorize the timesequentially displayed individuals Requirement to minimize human fatigue I m tired 5

6 Overview Framework Direct Manipulation Media Retrieval & Design Media Retrieval & Design IEC IEC Partial Evaluation Domain Knowledge Conventional Proposed 6

7 Methodology Interactive Genetic Algorithm Initial Population GA Fitness Function IGA User Selection Crossover / Mutation Fitness Evaluation Population Reproduction 7

8 Methodology Knowledge-based Encoding New Search Space Effective encoding? Domain Knowledge Removing impractical solutions Using partially known information about the final solution Focusing on a specific search space Encoding in a modular manner Search Space 8

9 Partial Evaluation Based on Clustering Methodology Issues Clustering algorithm selection, indirect fitness allocation strategy 9

10 Methodology Direct Manipulation of Evolution It is very similar to the final solution but a bit different. IGA Direct Manipulation Proof of usefulness of the direct manipulation N-K Landscape 10

11 Applications Overview Humanized Media Retrieval & Design Media Retrieval Media Design Image Retrieval Video Retrieval Music Retrieval Fashion Design Flower Design Intrusion Pattern Design 11

12 Content-based Image Retrieval Applications: Image Retrieval Keywords-based method Much time and labor to construct indexes in a large databases Performance decrease when index constructor and user have different point of view Inherent difficulty to describe image as a keyword Content-based method Image contents as a set of features extracted from image Specifying queries using the features 12

13 Applications: Image Retrieval Motivation System for content-based image retrieval QBIC (IBM, 1993) QVE (Hirata and Kato, 1993) Chabot (Berkeley, 1994) Needs for image retrieval based on human intuition Difficult to get perfect expression by queries Lack of expression capability 13

14 System Structure Applications: Image Retrieval 14

15 Chromosome Construction Applications: Image Retrieval R G B Original image Transformed image Chromosome 15

16 Convergence Test Applications: Image Retrieval The case of gloomy image retrieval Initial population The 8th population 16

17 Applications: Image Retrieval Subjective Test by Sheffe s Method Confidence interval 95% 99% 17

18 Applications: Video Retrieval Motivation Evolution of computing technology (Huge computing storage, networking) Necessary for management of video data transfer, processing and retrieval Change of view point Query by keyword Query by content Difficult to represent human s semantic information Query by semantics Emotion-based retrieval 18

19 Hierarchical Structure of Video Applications: Video Retrieval 19

20 System Architecture Applications: Video Retrieval 20

21 Chromosome Construction Applications: Video Retrieval 21

22 System Interface Applications: Video Retrieval 22

23 User s Satisfaction Applications: Video Retrieval 23

24 Emotional Music Retrieval Applications: Music Retrieval Music information retrieval Immature field Query by singing Query by content Uploading pre-recording humming via web browser Typing the string to represent the melody contour Preprocessing Query by humming transforms user s humming into symbolized representation Conventional music retrieval system Fast and effective extraction of musical patterns and transformation of user s humming into systematic notes Problem : User cannot remember some parts of melody 24

25 Power Spectrum Analysis Applications: Music Retrieval 1/f )] log 10 [ ( f S Z a b a : classic b : jazz c : rock d : news channel c d ( log 10 f ) 25

26 System Architecture Applications: Music Retrieval 3Retrieved music IGA : Music Database 4 User evaluates the music retrieved 5 Generation of new offspring 1Query 2Retrieval of music by the query 26

27 Chromosome Construction Applications: Music Retrieval 27

28 Convergence Test Applications: Music Retrieval Distance m=0.5 c=0.5 2 m=0.2 c=0.5 3 m=0.5 c=0.2 4 m=0.2 c= generation 28

29 Convergence Test (2) Applications: Music Retrieval 29

30 Change of Consumer Economy Applications: Fashion Design Manufacturer Oriented Before the Industrial Revolution : Customers have few choices on buying their clothes After the Industrial Revolution : Customers can make their choices with very large variety Consumer Oriented Near Future : Customers can order and get clothes of their favorite design 30

31 Applications: Fashion Design Need for Interaction-based System Almost all consumers are non-professional at design To make designers contact all consumers is not effective Need for the design system that can be used by non-professionals 31

32 Applications: Fashion Design Fashion Design Definition To make a choice within various styles that clothes can take Three shape part of fashion design Silhouette Detail Trimming 32

33 System Architecture Applications: Fashion Design OpenGL Program Models of each part Combine Display Decode Interactive Genetic Algorithm GA operation Reproduce User Fitness 33

34 Knowledge-based Encoding Applications: Fashion Design A B C D E F Total 23 bits A : Neck and body style(34) B : Color(8) E : Skirt and waistline style(9) F : Color(8) C : Arm and sleeve style(11) D : Color(8) Search space size =34*8*11*8*9*8 =1,880,064 34

35 Direct Manipulation Interface Applications: Fashion Design 35

36 Applications: Fashion Design Fitness Changes for Each Encoding Scheme Average Fitness Knowledge-based Encoding Sequential Encoding Generation 36

37 Hypercube Analysis Applications: Fashion Design mutton 0111 pagoda No mapped design 1110 No mapped design mutton 0111 pagoda No mapped design No mapped design 0010 flare tucked 0010 flare tucked 0011 french 1011 no sleeves 0011 french DM 1011 no sleeves 0001 china melon No mapped design 1100 No mapped design 0001 china melon No mapped design 1100 No mapped design mandarin 1001 tight mandarin 1001 tight bishop 1000 poncho 0000 bishop 1000 poncho One-Mutant Neighbors using Genetic Operator Shortest Path Using DM Method 37

38 Applications: Flower Design Motivation Proposed Method Representation of knowledge-based genotype using the structure of real flower Automatic generation of creatures Process Directed graph: Define the genotype of flower structure IGA: Evaluation of created individuals and automatic creation Math Engine: Representation of phenotype Objective Creation of character morphology similar to reality Find optimal solution in small solution space using knowledgebased genotype representation such as directed graph Automatic creation of character using IGA 38

39 Applications: Flower Design Related Works Tree, flower and feather modeling L-System Box-based artificial characters (Karl Sims) evolution Graph model representation Golem Robotic form and controller evolution Graph-based data structure 39

40 Overall Procedure Applications: Flower Design Create initial edges between nodes Set dimension and color of each part Set parameters of parts and edges Create 3D world in simulation Create rigid parts in genotype Perform 3D rendering Evaluate fitness of each individual No Satisfied individual? Yes Selection Stop Crossover/Mutation Create individuals of next generation 40

41 Chromosome Construction Applications: Flower Design 41

42 Convergence Test Applications: Flower Design 10 Fitness value Generation Average fitness value Best fitness value 42

43 Analysis Using Sheffe s Method Applications: Flower Design 99% 95%

44 Motivation Applications: Intrusion Pattern Design Various intrusion detection system For evaluating, IDS uses known intrusions Possibility of having different patterns within identical intrusion type Difficult to detect transformed intrusions Problems For better evaluation, intrusion patterns are needed more Difficult to generate all possible proper intrusion patterns manually Automatic generation is needed 44

45 Motivation (2) Applications: Intrusion Pattern Design Evolutionary pattern generation Needs a variety of intrusion patterns It is possible to generate new patterns by combining primitives Generation of transformed intrusion patterns using simple genetic algorithm IGA-based intrusion pattern generation Difficult to estimate the fitness of patterns automatically Using interactive genetic algorithm for the estimation 45

46 Applications: Intrusion Pattern Design Method Intrusive behavior can be divided into 5 steps Proposed by DARPA Possible path of U2R attack TRANSPORT ENCODING EXECUTION ACTIONS CLEAN UP web download archive shell variables change file permissions editor octal character shell script display files remove files ftp simple encryption encrypted shell interaction delete files restore permissions mail character stuffing generate chaft output in bg transfer files edit audit logs floopy encoding delayted execution alter files 46

47 System Architecture Applications: Intrusion Pattern Design Known Intrusion Patterns Initial Population IGA GA Fitness Evaluation Intrusion Patterns Generate Audit Data Exploit Code 47

48 Concluding Remarks Humanized computing framework using IEC Knowledge-based encoding Partial evaluation based on clustering Direct manipulation of evolution Applications in media retrieval & design Image, video and music retrieval Fashion, flower and intrusion pattern design 48