The Problem Space Theory A problem space States o consists of states contain partial knowledge about the problem and about the solution Operators o transform states Search control knowledge o guides selection of operators o guides selection of states 11
Design Methods Observed. Design occurs mainly in algorithm design space. Schematic kernel idea quickly selected I. Successive refinement. Adapt very general operators o specific knowledge o means ends analysis. Symbolic and test-case execution of algorithms (in the absence of knowledge) Discovery and problem-solving in task space 13
Lessons about Design. A data flow space for representing partially specified algorithms. A variety of design schema (generate and test, divide and conquer) General operators instantiated with knowledge or by means-ends analysis Symbolic and test-case execution to expose problems and opportunities. A task space for test-case execution and discoveries. Discovery involves a prepared problem state Discovery requires recognition in the task space 27
Goals for Algorithm Design. Design algorithm sketches from semi-formal specifications Exploit knowledge if available Flexible in the absence of knowledge general problem solving and search
Alternative Automation Methods. Expert systems o successive refinement o mostly data and control structures o detailed and brittle o no problem solving or learning Formal derivation o small set of transformations o guarantee correctness o good local optimizations o lack focus, choice, transparency Inductive learning o input-output pairs or traces o schema matching and heuristic search o methods have not evolved much o possibilities if language is logic-based
Why Study Human Design? human thinking is flexible and robust programming details learned, not hand coded. people are resources to start and increment design knowledge. interfaces (automatic programmers must communicate with people)
Scientific Issues algorithm design (conquer new Al territory) formalize algorithm design principles, optimization, analysis program synthesis (do better than existing systems). discovery (conquer new Al phenomenon). - visual reasoning (critical-path Al phenomenon) shed light en kno Isdge/soarch Issues in expert systems * interaction off domain and programming knowledge in program synthesis how humans design and dfecover (confirm cognitive psychology theory)
The Protocol Analysis Approach Select a specific task (convexhull construction) Take protocols from appropriate subjects o subjects: faculty and graduate students in computer science o transcripts: complete text from audio tapes (videotaping would be better) Analyze protocols o need theory as basis o very detailed, phrase by phrase o refine theory as analysis proceeds
After the Analyses A simulation system (to test the theory) An automated design system An algorithm design assistant A protocol analysis aid Other programming tasks 6
A Task A point set and its convex hull
Sample of Protocol Episode 2 f L21 [»Minute 2«] Let's start with some point. L25 Either a point is on the hull or its not, right? L27 And the question is how to make this decision. Episode 3 Episode 3.1 L28 Let's take a few points here. (Draws 4 points.) L29 Well, that's not a good example, L30 because all four of them are on the convex hull. [S draws figure with 5 points not all in hull.] L35 OK, let's suppose I start with a point here. L36 And I'll just draw a line to some other point, right. L42 Now I can go in three directions from this point. L43 [»Minute 3«] I conjecture that L44 if it's the case that I can choose two points, L45 such that I can go on either side of the given line, L46 then this line can't be on the convex hull. L47 And I had better retreat. Episode 3.2 L63 Let's retreat, uh, back... back to A. L65 And choose some other point. L66 And this time we'll chose C. L67 Right? So now I have a line from A to C. 8
Protocol Sample continued Episode 3.4 L113 [»Minute 6«] And I see that, urn, L114 all the points are to one side of the line AC. L115 So I've got a candidate. L116 Now I'm at C and now I'll go again. L117 Choose some other point. L118 Suppose I choose B. [pause] LI19 A goes to C goes to B. L120 Urn, now I see that uh, [pause] L121 there are points on either side of the line CB, right, L122 there's E and there's A. L123 [»Minute 7«] I guess I have to look at A L124 even though I've already got a line segment from it. L125 So I know that the line CB can't be on the hull. L126 So I have to retreat back to C. L128 It looks like I'm not going to come up with a linear algorithm to do this.
Selected Episodes Episode El E2 E2.1 E3 E3.1 E3.1.1 E3.2 E3.3 E3.4 E3.4.1 E3.4.2 E3.4.3 E3.6 E3.6 E4 E6 Lines L1-L15 L16-L27 L22-L24 L28-L179 L28-L61 L28-L32 L62-L67 L68-L86 L86-L140 L86-L92 L93-L108 L127-L128 L141-L160 L161-L179 L180-L256 L256-L261 Description Acquire problem Design generate-and-test schema Interrupt(E2): specification of points Develop algorithm Find test Get example figure Decide how to handle test failure Find can discard interior start point Push algorithm all the way to find CH Return to previous state after E3.2 Interrupt(S): Exclude segment not point Interrupt(S): Greater than linear Develop initialization Recap algorithm Analyze complexity Termination (algorithm is first try)
A Problem-Space Model. What are the problem spaces? o an algorithm description space o a task domain space (geometry). What design methods are used? What are the operators and representations in the design space?. What is the task-domain knowledge?
Design Methods Observed. Design occurs mainly in algorithm design space Schematic kernel idea quickly selected. Successive refinement. Adapt very general operators o specific knowledge o means ends analysis. Symbolic and test-case execution of algorithms (in the absence of knowledge) Problem-solving in task space. Discovery 8
Basic Design Steps (occur as a result of productions firing) Select a problem (exposed during symbolic execution) Find kernel idea or solution plan Lay down basic structure (components) Elaborate details of structure Verify solution (optional, symbolic execution) Evaluate solution (e.g. time complexity) 14
Representation in DFS. Data-flow space (DFS) is main problem-solving space. represents partial algorithm designs (incomplete knowledge).o new information comes in small increments o postpone commitments. small vocabulary of basic components o arbitrary symbols and assertions (partially specified or alternative concepts) o expert vocabulary built up on top. components characterized by functional properties (not formal input-output relationships) o stable data dependencies in initial connections o internal representation by assertions o add assertions and additional inputs as needed o component can be refined to configuration 10
DFS Descriptions of Algorithm [pt],true >Generate >Test '. Assertions:, on {x}: on Test: elements are points predicate = is-on-hull(pt) +------------------------" ----------------------------+ delete V [x] [yx] true {x} >Generate >Draw >{z hull-so-far} >Test t t false y delete + + + + Assertions: on {x}: on Generate: on {z}: on Test: elements are points ordering = random elements are segments predicate = not(points-both-sides(segment,{x})) 16
DFS Descriptions of Algorithm (new-component 'aeex>ry) (add-coaponent 'generator 'point-gen) (add-cooponent 'test 'side-test) (add-component 'memory nil 'add-elen) (arid-assertion 'point-gen '(Input-order-next)) (add-assert1on '(test-predicate (on-s1de test-in x-ax1s left)) 'side-test) (display-configuration 'subset) ************* * i MEHORY-1 > * a f f f > foimt-geii > f f *********#*** a has symbolic Hen S and has description Item S b has symbolic 1te«POINT-4 c has symbolic 1te«POINT-SET-3 _ -"> SIDE-TEST 4- f f Component GENERATOR-1 has assertions: INPUT-ORDER-NEXT Component TEST-1 has assertions: TEST-PREDICATE-1: (TEST-PREDICATE (ON-SIDE MAIN-INPUT X-AXIS LEFT)) ************ * *a MEMORY-2 f * c f 16
Applying Operators in DPS Edit DPS configuration - add,-modify or remove: o process components links input and output ports assertions items Map the operator onto the situation o simple instantiation by specific task-space or component rules o means ends analysis with subgoaling 18
I I Review of Design Principles. A data flow space for representing partially specified algorithms. A variety of design schema (generate and test, divide and conquer) General operators instantiated with knowledge or by means-ends analysis. Symbolic and test-case execution to expose problems and opportunities. A task space for test-case execution and discoveries. Discovery involves a prepared problem state Discovery requires recognition in the task space 23
I I Future Directions. discoveries example construction low level MEA adaptation. representations. algorithm analysis code generation/ program synthesis algorithm memory. analogy learning interactive assistant protocol analysis aid collect more protocols 24