Learning to Remove Soft Shadows Supplementary Material

Transcription

1 Learning to Remove Soft Shadows Supplementary Material

2 Appendix 1: Per-Image Quantitative Comparisons Here we present more quantitative evaluation of our and related methods as an addition to Table I in the main paper. RMSE from ground truth our Arbel and Hel-Or 2011 Guo et al Guo et al (automatic detection) 0 p10_1 p11_1 p12_1 p12_2 p13_1 p14_1 p14_2 p15_1 p16_1 p16_2 p17_1 p17_2 p18_1 p18_2 p19_1 p1_1 p1_2 p1_3 p20_1 p21_1 p21_2 p2_1 p2_2 p3_1 p4_1 p6_1 p6_2 p7_1 p7_2 p8_1 p9_1 70 RMSE from ground truth DSC_0394 DSC_0400 DSC_0445 DSC_0531 DSC_0552 DSC_0559 DSC_0569 DSC_0577 DSC_0587 DSC_0594 DSC_0599 DSC_0607 real106 real108 real109 real111 real116 real125 real128 real138 real151 real152 real39 real42 real43 real60 real79 real82 real92 real97 Figure 1: RMSE between results of different shadow removal methods and the ground truth shadow-free images. Ushadowing results for real151 and p4 1 can be found in figures 8 and 17 in the main paper respectively. 1

3 Appendix 2: User Study Interface Figure 2: Screen capture of Task 1 of the user study. The participants were asked to decide which of the presented images seemed more natural and arrange the presented results by click-and-drag. Figure 3: Screen capture of Task 2 in the user study. The participants were presented with results of shadow removal using either our method, Guo et al. s or Arbel and Hel-Or s and asked to indicate how successful the shadow removal was in the marked region. 2

4 Appendix 3: Additional User Study Data Figure 4: Maximum penumbra widths for images in ours and Guo et al. s datasets. Note that the average penumbra in our dataset is significantly wider than in Guo et al. s (meaning softer shadows). Figure 5: We have measured the standard deviations of scores that the study participants assigned to individual images. The users seemed to mostly agree in their evaluations of the results. 3

5 Appendix 4: Algorithm Design and Exploration Comparison of Different Initialization Techniques plane fitting texture-guided PatchMatch off-the-shelf inpainting Figure 6: Comparison of results using different inpainting approaches. Note that in the case of texture-guided inpainting there are often out-of-shadow parts of the image that the algorithm chooses as the source. This is a failure of the distance function used as described below, however, we were not able to find a better solution. Impact of the Training Set Size We have evaluated how the performance of our algorithm varies with the amount of training data supplied. First, we have kept the number of patches sampled per image constant and set the number of training images rendered to 1000, 5000, and We have empirically determined that using up to images provides a good balance between the training time and performance, while beyond the trade-off becomes less attractive. Secondly, we have used similar procedure to arrive at the appropriate number of patches sampled per image. Feature Vector Here we present a short exploration of the relative importance of features in our feature vector. While feature importance is a very informative measure, it would not be feasible to compute using our full dataset. Instead below we present feature frequency, a number of times each feature was used as a split in the entire forest. We have also explored other features, such as explicit pixel-wise gradient orientation and magnitude, and the angular position of the patch within the masked region. However, none of them offered substantial improvements in performance. 4

6 Figure 7: Number of times each part of our feature vector was used as a split dimension at a node. Note the spike at position 0 indicating that the distance from the mask edge is an important feature. 5

7 Appendix 5: Impact of User Input on Results The results below show unshadowing results obtained by three different people for five scenes, as described in Section 6 in the paper. We conclude that small differences in the provided masks do not contribute significantly to the quality of final results as long as all the penumbrae are selected. The exception is the bottom row in most of the scenes, which shows penumbrae parts being left untouched, which is caused by this user s tendency to underestimate the extent of the shadow. Recall that we intentionally constrain the unselected regions to be considered out-ofshadow and therefore unquestioningly trust the users judgment. This design decision could be revisited e.g. by automatically dilating the masks by a certain amount. real22 real26 6

8 real138 real168 real249 Figure 8: Unshadowed results ae robust to variations in the user-provided binary mask. 7

9 Appendix 6: Converting RAW images to linear PNGs As explained in Section 7 of the paper, we have used linear PNG photographs to evaluate our method. Specifically, we have captured 14-bit RAW images using Canon 550D camera and used the dcraw and pnmtopng command line tools to obtain 8-bit linear PNG images as follows: dcraw -c -q 3 -b 2.0 -W -w -g 1 1 input.cr2 pnmtopng -gamma 1 -force > output.png 8