HW #4. Hierarchical Temporal Memory

Transcription

1 HW #4. Hierarchical Temporal Memory Implement episodic memory using an HTM network. Skeleton codes are available from Using the code, you are required to store and predict task episodes. After the learning process, confirm if the HTM can predict the next event for the following input cue: The same events learned (consecutively applied to the HTM) A first event of an episode An event in the middle of an episode An incomplete event (either an action or an object is missing) 1

2 In encoding, Discuss the advantage of using 3 bits for an object, instead of using one bit. In spatial pooling, Try with the various values of NUM_COL and NUM_ACTIVATED_COL, and find the required minimum number of NUM_COL. Discuss what happens if the SP process is skipped, i.e. Encoding Temporal pooling Decoding In temporal pooling, Try with NUM_CELL from 1 to 20 and discuss the role of cells. In anomaly detection Plot the cell status when an anomaly occurs. Explain why all the cells within the activated columns are activated. Compare the performance of HTM to that of EM (Deep) ART 2

3 Using the skeleton codes, fill the following functions in Fill folder: anomalydetection.py Submit the report along with your codes with a zip file name: HW4_yourname.zip Due date: May 9, 2017 Send to: yhyoo@rit.kaist.ac.kr 3

4 Nupic: A python library developed by Numenta, Inc., implementing HTM Installation of Nupic in Ubuntu Install python $ sudo apt-get install python-pip $ sudo apt-get install python-dev Install nupic bindings $ pip install 2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindin gs/nupic.bindings cp27-none-linux_x86_64.whl--user Install nupic $ pip install nupic --user Export environment variable $ sudo vi /etc/bash.bashrc Register your nupic path, e.g., - export NUPIC=/home/RITL/nupic Save and quit $ source /etc/bash.bashrc Check your environment variable $ echo $NUPIC 4

5 Verification of Nupic installation Install pythest and git for testing nupic $ sudo apt-get install python-pytest $ sudo apt-get install git Download nupic example files from the server $ git clone $ cd nupic Run nupic installation tests $./scripts/run_nupic_tests u Tests should be PASSED or SKIPPED For more information on Nupic installation, 5

6 PyCharm Python IDE for developers, developed by Jetbrain, Inc. Installation of Pycharm 5 Copy the attached file pycharm-5.04.tar.gz to the desired installation location Unpack the pycharm-5.04.tar.gz using the following command: $ tar xfz pycharm-5.04.tar.gz Remove the pycharm-5.04.tar.gz to save disk space (optional) $ rm pycharm-5.04.tar.gz Run pycharm.sh from the bin directory $ cd pycharm-5.04-community/bin $./pycharm.sh For more information on installing PyCharm, 6

7 HW #?. Hierarchical Temporal Memory Matplotlib Python library for visualization Installation of matplotlib $ sudo apt-get install python-matplotlib 7

8 Episodic memory for task intelligence Episode #1: Water the flowers Episode #2: Bring a book in a drawer Episode #3: Pour the contents of a bottle Episode #4: Sort the toys Episode #5: Toast a slice of bread 8

9 HTM as an episodic memory model 9

10 HTM procedure: 1. Encoding 2. Spatial Pooling 3. Temporal Pooling 4. Decoding (for classification) 1. Encoding To change sensory data into a binary string understood by HTM. Make sure the semantics of the particular data type should be captured. Encoder types: Category: actions, objects, etc. Scalar: real numbers, indoor temperature change, etc. Coordinate: GPS log, location, etc. Date 10

11 5 task episodes Sort toys Water flowers Bring an object Toast a bread Pour the contents Action Object Action Object Action Object Action Object Action Object Grasp RS Walk to WP Walk to Drawer Grasp Bread Grasp Vinegar Move to Box_1 Bend Grasp to Handle Move to Toaster Move to Bowl Release Grasp WP Pull Put into Toaster Tilt Grasp RC Raise Release Move to Button Stand Move to Box_1 Straight Grasp Red arch Press Button Move to Table Release Walk to FP Raise Bread Grasp Release Grasp RT Move to FP Grasp Handle Move to Dish Grasp Mustard Move to Box_1 Tilt Push Release Move to Bowl Release Stand Release Squeeze Grasp GS Move to Ground Walk to Box_1 Move to Table Move to Box_2 Release Move to Box_1 Release Release Release Grasp Sesame Grasp GC Move to Bowl RS = red square, RC = red cylinder, RT = red triangle, GS = Green square, GC = Green cylinder WP = watering pot, FP = flower pot *Some part of sort toys scenario are skipped for the lack of the space of the slide Shake Move to Release Table 11

12 68 encoded inputs (total # of events) Total 138 bits for each encoded input of which element by 3 bits: 15 actions: Grasp, move to, release, tilt, stand, push, pull, put into, shake, squeeze, walk, bend, press, straight, raise. 28 objects: Red-square, re-cylinder, red-arch, red-triangle, green-triangle, green-cylinder, green-arch, green-triangle, blue-square, blue-cylinder, blue-arch, blue-triangle, watering-pot, dish, bread, handle, red-vinegar, mustard, sesame, box-1, box-2, box-3, flower pot, toaster, button, drawer, bowl, table 2 human detection information: Detected (appear), not detected (disappear) 1 end of a sequence: Completed 12

13 Examples of encoded inputs Grasp Red square : [ ] Move Box _ 1 : [ ] Release : [ ] The structure of an encoded input Actions Objects Human detection End 13

14 Code In five_scenarios.py: : output encodedinput = five_scenarios_input.input_generator_total(input_dim=input_dim) # total 68 encoded inputs In five_scenarios_input.py: def input_generator_total(input_dim): encodedinput = numpy.zeros((68, INPUT_DIM), dtype='int') encodedinput[0] = input_generator(0, 0, 99, 99) # grasp, red square, none, none encodedinput[1] = input_generator(0, 1, 99, 99) # grasp, red cylinder, none, none encodedinput[67] = input_generator(99, 99, 99, 1) # none, none, none, finish return encodedinput def input_generator(val_1, val_2, val_3, val_4): ''' Generate HTM inputs (binary vectors) given four kinds of symbols. Parameters val_1 : Action type ( grasp, move, release ) from 0 to 14 val_2 : Object type ( red square, red cylinder, table ) from 0 to 27 val_3 : Human type ( disappear, appear ) 0 and 1 val_4 : End type ( finish ) 0 Returns encoded binary vector ''' 14

15 2. Spatial pooling To change input into sparse distributed representation (SDR). The dimension of SDR equals the column dimension of the HTM. In this homework, the dimension of SDR = 200 (NUM_COL) 10 columns (NUM_ACTIVATED_COL) are activated out of Inputs Spatial pooler results (binary string) (represented as the indices of the activated columns) Grasp, Red square [40, 92, 168, 172, 178, 181, 182, 183, 184, 187] Grasp, Red cylinder [16, 20, 27, 68, 85, 114, 126, 174, 177, 178] Grasp, Red arch [27, 55, 90, 100, 140, 177, 182, 186, 187, 193] 15

16 : output Code In five_scenarios.py # Initialize spatial pooler sp = SpatialPooler(inputDimensions=(INPUT_DIM,), columndimensions=(num_col,), potentialradius=120, numactivecolumnsperinharea=active_bit, globalinhibition=true, synpermactiveinc=0.04, potentialpct=1.0) # Start Learning SP SP_output = numpy.zeros((len(encodedinput), NUM_COL), dtype="int") for i in range(sp_iter): # repeat 200 times for j in range(len(encodedinput)): sp.compute(encodedinput[j], learn=true, activearray=sp_output[j]) 16

17 3. Temporal pooling Core part of HTM To change the SDR (the pattern of the activation of the columns) into a pattern of activation of cells, and Also to make a prediction through a pattern of predicted cells. A pattern of predicted cells represents a prediction of a next-coming input. In this homework, the number of cells per a columns = 20 (NUM_CELL) 17

18 Code In five_scenarios.py # Initialize temporal pooler tp = TP(numberOfCols=NUM_COL, cellspercolumn=num_cell, initialperm=0.5, connectedperm=0.5, minthreshold=5, newsynapsecount=10, permanenceinc=0.05, permanencedec=0.01, activationthreshold=5, globaldecay=0.01, burnin=1, checksynapseconsistency=false, pamlength=10) # Start Learning TP for i in range(tp_iter): # learning for 40 times # Through temporal pooling, HTM learns the temporal sequence of spatial patterns (SDR). # When a temporal sequence ends, use tp.reset() so that HTM knows that the temporal sequence has ended. # When enablelearn=true, tp learns the temporal sequence between inputs. # When computeinfoutput=true, tp infers the next-coming input through a pattern of predicted cells. ##### Sorting toys ##### tp.compute(sp_output[0], enablelearn=true, computeinfoutput=false) # grasp, red square, tp.compute(sp_output[67], enablelearn=true, computeinfoutput=false) # finish tp.reset() #### taking a contents #### tp.compute(sp_output[16], enablelearn=true, computeinfoutput=false) # grasp, red-vinegar, tp.compute(sp_output[67], enablelearn=true, computeinfoutput=false) # finish tp.reset() # Learning process is finished. Decoding is not needed during learning. 18

19 4. Decoding To decode the pattern of predicted columns, so that users can easily understand the meaning of the pattern. In this homework, decoding is implemented by the following function: five_scenarios_input.find_pattern(columns,num_col,sp_output) This function finds the meaning of Columns. The function compares the current input with all learned spatial pattern (SP_output), by counting the number of overlapping bits between the current input and a learned spatial pattern. The one with the largest number of overlapping bits is selected. 19

20 Code In five_scenarios.py : output of TP : output of decoding for t in range(1000): #This is NOT a process for learning; this is a process for Decoding. # Encodeing : get an input from a user and transform it as a form of a binary string. encodedinput_for_simulation= five_scenarios_input.input_generator_2(t, NUM_COL, INPUT_DIM) # Spatial pooling : get a encodedinput and put out the corresponding SDR (SP_output_for_simulation) sp.compute(encodedinput_for_simulation, learn=false, activearray=sp_output_for_simulation) # Temporal pooling : get a SDR and put out a pattern of predicted cells (predictedcells) tp.compute(sp_output_for_simulation, enablelearn=false, computeinfoutput=true) predictedcells = tp.getpredictedstate() # Decoding : get a pattern of predicted columns # put out the predicted next-coming input(procedural[k+1], character string) # Change the pattern of predicted cells into a pattern of predicted columns. predictedcolumns = five_scenarios_input.cells2columns(cells=predictedcells, NUM_COL=NUM_COL) # Compare the next-predicted SDR (predictedcolumns) with all previously leaned SDR patterns (SP_output) # The best-mating event becomes the output, as a form of a character string. procedural.append(five_scenarios_input.find_pattern(columns=predictedcolumns, NUM_COL=NUM_COL, SP_output=SP_output, INPUT_DIM=INPUT_DIM,encodedInput=encodedInput)[0]) 20

21 5. Anomaly detection HTM can easily detect anomalies, by comparing predicted columns at time t-1 and activated columns at time t. Anomaly score is the fraction of active columns that were not predicted. Anomaly detection is implemented by using the following equation: where Predicted columns at time 1 Active columns at time The number of columns 21

22 Code In five_scenarios.py: : output # Anomaly Detection # # except first loop (k=0), calculate anomaly value if k > 0: anomalyscore = anomalydetection.anomalyscore(activecolumns=sp_output_for_simulation, predictedcolumns=predictedcolumns) # If the anomaly score exceeds a threshold (ANOMALY_THRESHOLD=0.5) it is considered as an anomaly. if anomalyscore > ANOMALY_THRESHOLD: print "anomaly detected In Fill/anomalyDetection.py: def anomalyscore(activecolumns, predictedcolumns): # You are required to implement anomalydetection. return anomalyscore 22

23 Pseudo code of the overall HTM procedure FOR (i = 0; i<num_dataset; i++) DO : EncodedInput[i] = Encode(SensoryInput[i]); FOR (i = 0; i<num_dataset; i++) DO : SP_output[i] = SpatialPooling(EncodedInput[i], Learn=True); FOR (i = 0; i<num_dataset; i++) DO : TemporalPooling(SP_output[i], Learn=True, Prediction = False); Learning process t = 0; WHILE(TRUE) DO: oneencodedinput = Encode(SensoryInputFromUser); onesp_output = SpatialPooling(oneEncodedInput, Learn = False); onetp_output = TemporalPooling(oneSP_output, Learn = False, Prediction = True); Procedural[t+1] = Decode(oneTP_output) PRINT Next Predicted Event =, Procedural[t+1] t++; Prediction & decoding process 23

24 Results pouring the contents of a bottle scenario At 0, the first input of the scenario, (Grasp Red-vinegar) to HTM Current state: Grasp a red vinegar bottle Next predicted state: Move to a bowl 24

25 At 1, the second input of the scenario, (Move-to Bowl) to HTM Current state: Move to a bowl Next predicted state: Tilt 25

26 At 1, an unexpected input, (Move to Table) to HTM HTM is expecting (Move to Bowl ) Current state: Move to the table Next predicted state: Release 26