Midterm Review. CS230 Fall 2018

Transcription

1 Midterm Review CS23 Fall 28

2 Broadcasting

3 Calculating Means How would you calculate the means across the rows of the following matrix? How about the columns?

4 Calculating Means How would you calculate the means across the rows of the following matrix? How about the columns? Rows: row_mu = np.sum(m, axis=) / M.shape[]

5 Calculating Means How would you calculate the means across the rows of the following matrix? How about the columns? Rows: Cols: row_mu = np.sum(m, axis=) / M.shape[] col_mu = np.sum(m, axis=) / M.shape[]

6 Computing Softmax How would you compute the softmax across the columns of the following matrix?

7 Computing Softmax How would you compute the softmax across the columns of the following matrix? exp = np.exp(m)

8 Computing Softmax How would you compute the softmax across the columns of the following matrix? exp = np.exp(m) smx = exp / np.sum(exp, axis=)

9 Computing Distances How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?

10

11 Computing Distances How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?

12 Computing Distances How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)? sq_diff = np.square(x-v)

13 Computing Distances How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)? sq_diff = np.square(x-v) dists = np.sqrt(np.sum(sq_diff, axis=))

14 Computing Distances How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)? sq_diff = np.square(x-v) dists = np.sqrt(np.sum(sq_diff, axis=)) nearest = np.argmin(dists)

15 L/L2 Regularization

16 Logistic Regression and Separable Data What s the issue with training a logistic regression model on the following data?

17

18 Logistic Regression and Separable Data What s the issue with training a logistic regression model on the following data?

19 Logistic Regression and Separable Data What s the issue with training a logistic regression model on the following data? The parameters will tend to plus/minus infinity! So, it will never converge.

20 Solving the Exploding Weights Issue What modification of the loss function can you implement so solve this issue? Write out the new loss function.

21 Solving the Exploding Weights Issue What modification of the loss function can you implement so solve this issue? Write out the new loss function. Add L2 Regularization Lkj This new loss function will keep the magnitude of the weights from exploding!

22 Gradient of the New Loss Compute the gradient of the weight vector with respect to this new loss function.

23 Gradient of the New Loss Compute the gradient of the new loss function with respect to the weight vector.

24 Another Solution... What is another, similar modification to the loss function that could help with this issue? Compute its gradient?

25 Another Solution... What is another, similar modification to the loss function that could help with this issue? Compute its gradient? Add L Regularization:

26 Backprop

27

28

29 CNN Input/Output Sizes

30 Basic (no padding, stride ) Input Filter

31 Basic

32 Basic

33 Basic

34 Basic

35 Basic

36 Basic

37 Basic

38 Basic

39 Basic Input Filter Conv Output

40 Basic Shape = n - f + Input Filter Conv Output

41 Padding Input Filter

42 Padding Input Filter Conv Output

43 Padding Shape = n + 2p- f + Input Filter Conv Output

44 Valid and Same Convolutions Valid No padding Output shape -> n - f + Same Pad so that input is same as output size Output shape -> n + 2p - f +

45 Stride Input Filter

46 Stride

47 Stride

48 Stride

49 Stride

50 Basic Shape = (n - f)/s + Input Filter Conv Output

51 With Stride n x n image f x f filter p padding s stride Output size -> (n + 2p - f)/s +

52 Maxpool

53 Forward prop 2 x 2 Pooling layer with stride Input

54 Forward prop 2 x 2 Pooling layer with stride Input Size of output (n-f)/s + Output

55 Forward prop 2 x 2 Pooling layer with stride Input Output

56 Forward prop 2 x 2 Pooling layer with stride Input Output 5

57 Forward prop 2 x 2 Pooling layer with stride Input Output 5

58 Forward prop 2 x 2 Pooling layer with stride Input Output

59 Forward prop 2 x 2 Pooling layer with stride Input Output

60 Backprop Input to maxpool layer Output of maxpool layer Gradient w.r.t output

61 Backprop??????? 7 5-6????????? Gradient w.r.t input -4 Gradient w.r.t output

62 ReLU

63 Maxpool Maxpool(m Maxpool(mijij)) Maxpool(mij) if mij max(m) if mij = max(m) if mij max(m) if mij = max(m)

64 Backprop Keep track of where the maximum value is Input Output Mask

65 Backprop -4 Mask Gradient w.r.t output Gradient w.r.t input

66 Error Analysis

67 Dog Classifier Trying to predict dog vs not dog.

68 Improving performance Two kinds of errors - misclassification on muffins and fried chicken puter-vision-api-cbda4d6b425d

69 Error analysis - Get examples on dev set Image number Classified as muffin Classified as chicken Y Y Y 5% 5%... Comments

70 Error Analysis

71 Strategic Data Acquisition

72 Trigger Word Detection

73 Classification

74 Dropout

75

76

77 Batchnorm

78