arxiv: v1 [cs.cv] 11 Apr 2018
|
|
- Hugo Lambert
- 5 years ago
- Views:
Transcription
1 arxiv: v1 [cs.cv] 11 Apr 2018 ExFuse: Enhancing Feature Fusion for Semantic Segmentation Zhenli Zhang 1, Xiangyu Zhang 2, Chao Peng 2, Dazhi Cheng 3, Jian Sun 2 1 Fudan University, 2 Megvii Inc. (Face++), 3 Beijing Institute of Technology zhenlizhang14@fudan.edu.cn, {zhangxiangyu, pengchao}@megvii.com, @bit.edu.com, sunjian@megvii.com Abstract. Modern semantic segmentation frameworks usually combine low-level and high-level features from pre-trained backbone convolutional models to boost performance. In this paper, we first point out that a simple fusion of low-level and high-level features could be less effective because of the gap in semantic levels and spatial resolution. We find that introducing semantic information into low-level features and highresolution details into high-level features is more effective for the later fusion. Based on this observation, we propose a new framework, named ExFuse, to bridge the gap between low-level and high-level features thus significantly improve the segmentation quality by 4.0% in total. Furthermore, we evaluate our approach on the challenging PASCAL VOC 2012 segmentation benchmark and achieve 87.9% mean IoU, which outperforms the previous state-of-the-art results. Keywords: Semantic Segmentation, Convolutional Neural Networks 1 Introduction Most state-of-the-art semantic segmentation frameworks [1,2,3,4,5,6,7,8,9,10,11,12] follow the design of Fully Convolutional Network (FCN) [13]. FCN has a typical encoder-decoder structure semantic information is firstly embedded into the feature maps via encoder then the decoder takes responsibility for generating segmentation results. Usually the encoder is the pre-trained convolutional model to extract image features and the decoder contains multiple upsampling components to recover resolution. Although the top-most feature maps of the encoder could be highly semantic, its ability to reconstruct precise details in segmentation maps is limited due to insufficient resolution, which is very common in modern backbone models such as [14,15,16,17,18,19]. To address this, an U-Net architecture is proposed [7] and adopted in many recent work [2,6,7,8,13,11]. The core idea of U-Net is to gradually fuse high-level low-resolution features from top layers with low-level but high-resolution features from bottom layers, which is expected to be helpful for the decoder to generate high-resolution semantic results. Though the great success of U-Net, the working mechanism is still unknown and worth further investigating. Low-level and high-level features are complementary by nature, where low-level features are rich in spatial details but lack
2 2 Zhang et al. Fig. 1. Fusion of low-level and high-level features. a) Pure low-level high-resolution and pure high-level low-resolution features are difficult to be fused because of the significant semantic and resolution gaps. b) Introducing semantic information into lowlevel features or spatial information into high-level features benefits the feature fusion. dn and up blocks represent abstract up/down-sampling feature embedding. semantic information and vice versa. Consider the extreme case that pure low-level features only encode low-level concepts such as points, lines or edges. Intuitively, the fusion of high-level features with such pure low-level features helps little, because low-level features are too noisy to provide sufficient highresolution semantic guidance. In contrast, if low-level features include more semantic information, for example, encode relatively clearer semantic boundaries, then the fusion becomes easy fine segmentation results could be obtained by aligning high-level feature maps to the boundary. Similarly, pure high-level features with little spatial information cannot take full advantage of low-level features; however, with additional high-resolution features embedded, high-level features may have chance to refine itself by aligning to the nearest low-level boundary. Fig 1 illustrates the above concepts. Empirically, the semantic and resolution overlap between low-level and high-level features plays an important role in the effectiveness of feature fusion. In other words, feature fusion could be enhanced by introducing more semantic concepts into low-level features or by embedding more spatial information into high-level features. Motivated by the above observation, we propose to boost the feature fusion by bridging the semantic and resolution gap between low-level and high-level feature maps. We propose a framework named ExFuse, which addresses the gap from the following two aspects: 1) to introduce more semantic information into low-level features, we suggest three solutions layer rearrangement, semantic supervision and semantic embedding branch; 2) to embed more spatial infor-
3 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 3 mation into high-level features, we propose two novel methods: explicit channel resolution embedding and densely adjacent prediction. Significant improvements are obtained by either approach and a total increase of 4% is obtained by the combination. Furthermore, we evaluate our method on the challenging PASCAL VOC 2012 [20] semantic segmentation task. In the test dataset, we achieve the score of 87.9% mean IoU, surpassing the previous state-of-the-art methods. Our contributions can be summerized as follows: We suggest a new perspective to boost semantic segmentation performance, i.e. bridging the semantic and resolution gap between low-level and high-level features by more effective feature fusion. We propose a novel framework named ExFuse, which introduces more semantic information into low-level features and more spatial high-resolution information into high-level features. Significant improvements are obtained from the enhanced feature fusion. Our fully-equipped model achieves the new state-of-the-art result on the test set of PASCAL VOC 2012 segmentation benchmark. 2 Related Work Feature fusion in semantic segmentation. Feature fusion is frequently employed in semantic segmentation for different purposes and concepts. A lot of methods fuse low-level but high-resolution features and high-level low-resolution features together [2,6,7,8,13,11]. Besides, ASPP module is proposed in DeepLab [1,5,12] to fuse multi-scale features to tackle objects of different size. Pyramid pooling module in PSPNet [10] serves the same purpose through different implementation. BoxSup [21] empirically fuses feature maps of bounding boxes and segmentation maps to further enhance segmentation. Deeply supervised learning. To the best of our knowledge, deeply supervised training is initially proposed in [22], which aims to ease the training process of very deep neural networks since depth is the key limitation for training modern neural networks until batch normalization [23] and residual networks [14] are proposed. Extra losses are utilized in GoogleNet [15] for the same purpose. Recently, PSPNet [10] also employs this method to ease the optimization when training deeper networks. Upsampling. There are mainly three approaches to upsample a feature map. The first one is bilinear interpolation, which is widely used in [1,5,10,12]. The second method is deconvolution, which is initially proposed in FCN [13] and utilized in later work such as [3,6,7,8,11]. The third one is called sub-pixel convolution, which derives from [24,25] in super resolution task and is widely broadcast to other tasks such as semantic segmentation. For instance, [9] employs it to replace the traditional deconvolution operation.
4 4 Zhang et al. Fig. 2. Overall architecture of our approach. Components with solid boxes belong to the backbone GCN framework [8], while others with dashed lines are proposed in this work. Similar to [8], Boundary Refinement blocks are actually used but omitted in the figure. Numbers (H W C) in blocks specify the output dimension of each component. SS semantic supervision. ECRE explicit channel resolution embedding. SEB semantic embedding branch. DAP densely adjacent prediction. 3 Approach In this work we mainly focus on the feature fusion problem in U-Net segmentation frameworks [2,6,7,8,13,11]. In general, U-Net have an encoder-decoder structure as shown in Fig 1. Usually the encoder part is based on a convolutional model pretrained on large-scale classification dataset (e.g. ImageNet [26]), which generates low-level but high-resolution features from the bottom layers and highlevel low-resolution features from the top layers. Then the decoder part mixes up the features to predict segmentation results. A common way of feature fusion [27,2,28,6,7,8,11] is to formulate as a residual form: y l = Upsample(y l+1 ) + F(x l ) (1) where y l is the fused feature at l-th level; x l stands for the l-th feature generated by the encoder. Features with larger l have higher semantic level but lower spatial resolution and vice versa (see Fig 2). In Sec 1 we argue that feature fusion could become less effective if there is a large semantic or resolution gap between low-level and high-level features. To study and verify the impact, we choose one of the start-of-the-art U-Net frameworks Global Convolutional Network (GCN) [8] as our backbone segmentation architecture (see Fig 2 for details). In GCN, 4 different semantic levels
5 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 5 of feature maps are extracted from the encoder network, whose spatial resolutions, given the input, are {128, 64, 32, 16} respectively. To examine the effectiveness of feature fusion, we select several subsets of feature levels and use them to retrain the whole system. Results are shown in Table 1. It is clear that even though the segmentation quality increases with the fusion of more feature levels, the performance tends to saturate quickly. Especially, the lowest two feature levels (1 and 2) only contribute marginal improvements (0.24% for ResNet 50 and 0.05% for ResNeXt 101), which implies the fusion of low-level and high-level features is rather ineffective in this framework. In the following subsections we will introduce our solutions to bridge the gap between low-level and high-level features embedding more semantic information into low-level features and more spatial resolution clues into high-level features. First of all, we introduce our baseline settings: Feature Levels ResNet 50 (%) ResNeXt 101 (%) {4} {3, 4} {2, 3, 4} {1, 2, 3, 4} Table 1. GCN [8] segmentation results using given feature levels. Performances are evaluated by standard mean IoU(%) on PASCAL VOC 2012 validation set. Lower feature level involves less semantic but higher-resolution features and vice versa (see Fig 2). The feature extractor is based on pretrained ResNet50 [14] and ResNeXt101[18] model. Performance is evaluated in miou. Baseline Settings. The overall semantic segmentation framework follows the fully-equipped GCN [8] architecture, as shown in Fig 2. For the backbone encoder network, we use ResNeXt 101 [18] model pretrained on ImageNet by default 1 unless otherwise mentioned. We use two public-available semantic segmentation benchmarks PASCAL VOC 2012 [20] and Semantic Boundaries Dataset [29] for training and evaluate performances on PASCAL VOC 2012 validation set, which is consistent with many previous work [1,2,3,4,5,6,8,9,10,13,11,12,27]. The performance is measured by standard mean intersection-over-union (mean IoU). Other training and test details or hyper-parameters are exactly the same as [8]. Our reproduced GCN baseline score is 76.0%, shown in Table 3 (#1). 3.1 Introducing More Semantic Information into Low-level Features Our solutions are inspired by the fact: for convolutional neural networks, feature maps close to semantic supervisions (e.g. classification loss) tend to encode more 1 Though ResNeXt 101 performs much better than ResNet 101[14] on ImageNet classification task (21.2% vs. 23.6% in top-1 error), we find there are no significant differences on the semantic segmentation results (both are 76.0% miou).
6 6 Zhang et al. semantic information, which has been confirmed by some visualization work [30]. We propose three methods as follows: Layer Rearrangement In our framework, features are extracted from the tail of each stage in the encoder part (res-2 to res-5 in Fig 2). To make lowlevel features (res-2 or res-3) closer to the supervisions, one straight-forward approach is to arrange more layers in the early stages rather than the latter. For example, ResNeXt 101 [18] model has {3, 4, 23, 3} building blocks for Stage 2-5 respectively; we rearrange the assignment into {8, 8, 9, 8} and adjust the number of channels to ensure the same overall computational complexity. Experiment shows that even though the ImageNet classification score of the newly designed model is almost unchanged, its segmentation performance increases by 0.8% (Table 3, compare #2 with #3), which implies the quality of low-level feature might be improved. Semantic Supervision We come up with another way to improve low-level features, named Semantic Supervision (SS), by assigning auxiliary supervisions directly to the early stages of the encoder network (see Fig 2). To generate semantic outputs in the auxiliary branches, low-level features are forced to encode more semantic concepts, which is expected to be helpful for later feature fusion. Such methodology is inspired by Deeply Supervised Learning used in some old classification networks [22,15] to ease the training of deep networks. However, more sophisticated classification models [31,32,14,17,33,18] suggest end-to-end training without auxiliary losses, which is proved to have no convergence issue even for models over 100 layers. Our experiment also shows that for ResNet or ResNeXt models deeply supervised training is useless or even harms the classification accuracy (see Table 2). Therefore, our Semantic Supervision approach mainly focuses on improving the quality of low-level features, rather than boosting the backbone model itself. Model Cls err (top-1, %) Seg miou (%) Res SS Res Table 2. Effects of Semantic Supervision (SS). Classification scores are evaluated on ImageNet 2012 validation set. Fig 3 shows the detailed structure of our Semantic Supervision block. When pretraining the backbone encoder network, the components are attached to the tail of each stage as auxiliary supervisions (see Fig 2). The overall classification loss equals to a weighted summation of all auxiliary branches. Then after pretraining, we remove these branches and use the remaining part for fine tuning.
7 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 7 Experiment shows the method boosts the segmentation result by 1.1%. Moreover, we find that if features are extracted from the second convolutional layer in the auxiliary module for fine tuning (Fig 3), more improvement (1.5%) is obtained (see Table 3, compare #1 with #2), which supports our intuition that feature maps closer to the supervision tend to encode more semantic information. Fig. 3. Details of Semantic Supervision (SS) component in our pipeline. It is worth noting that the recent semantic segmentation work PSPNet [10] also employs deeply supervised learning and reports the improvements. Different from ours, the architecture of [10] do not extract feature maps supervised by the auxiliary explicitly; and their main purpose is to ease the optimization during training. However, in our framework we find the improvements may result from different reasons. For instance, we choose a relatively shallower network ResNet 50 [14] and pretrain with or without semantic supervision. From Table 2, we find the auxiliary losses do not improve the classification score, which implies ResNet 50 is unlikely to suffer from optimization difficulty. However, it still boosts the segmentation result by 1.1%, which is comparable to the deeper case of ResNeXt 101 (1.0%). We believe the enhancement in our framework mainly results from more semantic low-level features. Semantic Embedding Branch As mentioned above, many U-Net structures involve low-level feature as the residue to the upsampled high-level feature. In Equ 1 the residual term F(x l ) is a function of low-level but high-resolution feature, which is used to fill the spatial details. However, if the low-level feature contains little semantic information, it is insufficient to recover the semantic resolution. To address the drawback, we generalize the fusion as follows: y l = Upsample (y l+1 ) + F(x l, x l+1,..., x L ) (2)
8 8 Zhang et al. where L is the number of feature levels. Our insight is to involve more semantic information from high-level features to guide the resolution fusion. The detailed design of function F ( ) is illustrated in Fig 4, named Semantic Embedding Branch, (SEB). We use the component for features of Level 1-3 (see Fig 2). In our experiment SEB improves the performance by 0.7% (Table 3, compare #3 with #5). Fig. 4. Design of the Semantic Embedding Branch in Fig 2. The sign means element-wise multiplication. If there are more than one groups of high-level features, the component outputs the production of each feature map after upsampling. 3.2 Embedding More Spatial Resolution into High-level Features For most backbone feature extractor networks, high-level features have very limited spatial resolution. For example, the spatial size of top-most feature map in ResNet or ResNeXt is 7 7 for input size. To encode more spatial details, a widely used approach is dilated strategy [4,1,5,9,10,12], which is able to enlarge feature resolution without retraining the backbone network. However, since high-level feature maps involve a lot of channels, larger spatial size significantly increases the computational cost. So in this work we mainly consider another direction we do not try to increase the physical resolution of the feature maps; instead, we expect more resolution information encoded within channels. We propose the following two methods: Explicit Channel Resolution Embedding In our overall framework, segmentation loss is only connected to the output of decoder network (see Fig 2), which is considered to have less impact on the spatial information of high-level features by intuition. One straight-forward solution is to borrow the idea of Semantic Supervision (Sec 3.1) we could add an auxiliary supervision branch to the high-level feature map, upsample and force it to learn fine segmentation map. Following the insight, firstly we try adding an extra segmentation loss to the first deconvolution module (the light-blue component in Fig 2), however, no improvements are obtained (Table 4, #2).
9 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 9 Fig. 5. Illustration of the design of Explicit Channel Resolution Embedding (ECRE) module in Fig 2. Why does the auxiliary loss fail to work? Note that the purpose of the supervision is to embed high resolution information explicitly into feature map channels. However, since deconvolution layer includes weights, the embedding becomes implicit. To overcome this issue, we adopt a parameter-free upsampling method Sub-pixel Upsample [24,25] to replace the original deconvolution. Since sub-pixel upsample enlarge the feature map just by reshaping the spatial and channel dimensions, the auxiliary supervision is able to explicitly impact the features. Details of the component are shown in Fig 5. Experiment shows that it enhances the performance by 0.5% (see Table 4 and Table 3). Moreover, to demonstrate that the improvement is brought by explicit resolution embedding rather than sub-pixel upsampling itself, we also try to replace the deconvolution layer only without auxiliary supervision. Table 4 (#3) shows the result, which is even worse than the baseline. Index Baseline SS LR ECRE SEB DAP miou (%) Table 3. Ablation experiments of the methods in Sec 3. Performances are evaluated by standard mean IoU(%) on PASCAL VOC 2012 validation set. The baseline model is [8] (our impl.) SS semantic supervision. LR layer rearrangement. ECRE explicit channel resolution embedding. SEB semantic embedding branch. DAP densely adjacent prediction.
10 10 Zhang et al. Index Method miou (%) 1 Baseline Deconv + Supervised Sub-pixel Upsample Only ECRE (Fig 5) 78.8 Table 4. Ablation study on the design of Explicit Channel Resolution Embedding, (ECRE). The baseline model is in Table 3 (#3) Densely Adjacent Prediction In the decoder upstream of the original architecture (Fig 2), feature point at the spatial location (i, j) mainly takes responsibility for the semantic information at the same place. To encode as much spatial information into channels, we propose a novel mechanism named Densely Adjacent Prediction (DAP), which allows to predict results at the adjacent position, e.g. (i 1, j + 1). Then to get the final segmentation map, result at the position (i, j) can be generated by averaging the associated scores. Formally, given the window size k k, we divide the feature channels into k k groups, then DAP works as follows: r i,j = 1 k k 0 l,m<k x (l k+m) i+l k/2,j+m k/2 (3) where r i,j denotes the result at the position (i, j) and x (c) i,j stands for the features at the position (i, j) belonging to channel group c. In Fig 6 we illustrate the concept of DAP. Fig. 6. Illustration of Densely Adjacent Prediction (DAP) component in Fig 2. We use DAP on the output of our decoder (see Fig 2). In our experiment we set k = 3. Note that DAP requires the number of feature channels increased by k k times, so we increase the output channels of each deconvolution block to 189 (21 3 3). For fair comparison, we also evaluate the baseline model with the same number of channels. Results are shown in Table 5. It is clear that DAP
11 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 11 improves the performance by 0.6% while the counterpart model without DAP only obtains marginal gain, which implies DAP may be helpful for feature maps to embed more spatial information. Index Method miou (%) 1 Baseline Baseline (more channels) DAP (Fig 6) 79.6 Table 5. Ablation study on the effect of Densely Adjacent Prediction (DAP). The baseline model is in Table 3 (#5) 3.3 Discussions Is Feature Fusion Enhanced? At the beginning of Sec 3 we demonstrate that feature fusion in our baseline architecture (GCN [8]) is ineffective. Only marginal improvements are obtained by fusing low-level features (Level 1 and 2), as shown in Table 1. We attribute the issue to the semantic and resolution gap between low-level and high-level features. In Sec 3.1 and Sec 3.2, we propose a series of solutions to introduce more semantic information into low-level features and more spatial details into high-level features. Despite the improved performance, a question raises: is feature fusion in the framework really improved? To justify this, similar to Table 1 we compare several subsets of different feature levels and use them to train original baseline (GCN) and our proposed model (ExFuse) respectively. For the ExFuse model, all the 5 approaches in Sec 3.1 and Sec 3.2 are used. Table 6 shows the results. We find that combined with low-level feature maps (Level 1 and 2) the proposed ExFuse still achieves considerable performance gain ( 1.3%), while the baseline model cannot benefit from them. The comparison implies our insights and methodology enhance the feature fusion indeed. Table 6 also shows that the proposed model is much better than the baseline in the case that only top-most feature maps (Level 4) are used, which implies the superior high-level feature quality to the original model. Our further study shows that methods in Sec 3.2 contribute most of the improvement. Empirically we conclude that boosting high-level features not only benefits feature fusion, but also contributes directly to the segmentation performance. Do techniques work in a vanilla U-Net? Previously we would like to demonstrate that the proposed perspective and techniques are able to improve one of the state-of-the-art U-Net structure GCN. To prove the good generalization of this paper, we apply techniques illustrated above to a vanilla U-Net without GCN module. Performance on PASCAL VOC 2012 is boosted from 72.7 to 79.6
12 12 Zhang et al. Feature Levels Original GCN [8] (%) ExFuse (%) {4} {3, 4} {2, 3, 4} {1, 2, 3, 4} Table 6. Comparison of Original GCN [8] and ExFuse on segmentation results using given feature levels. The backbone feature extractor networks are both ResNeXt 101. in miou. We see that the gap is even bigger (6.9 instead of 4.0), which shows that techniques illustrated above generalize well. Could the perspective and techniques generalize to other computer vision tasks? Since U-Net structure is widely applied to other vision tasks such as low-level vision [34] and detection [35], a question raises naturally: could the proposed perspective and techniques generalize to other tasks? We carefully conducted ablation experiments and observe positive results. We leave detailed discussion for future work. 4 PASCAL VOC 2012 Experiment In the last section we introduce our methodology and evaluate their effectiveness via ablation experiments. In this section we investigate the fully-equipped system and report benchmark results on PASCAL VOC 2012 test set. To further improve the feature quality, we use deeper ResNeXt 131 as our backbone feature extractor, in which Squeeze-and-excitation modules [33] are also involved. The number of building blocks for Stage 2-5 is {8, 8, 19, 8} respectively, which follows the idea of Sec 3.1. With ResNeXt 131, we get 0.8% performance gain and achieve 80.8% miou when training with images from PASCAL VOC 2012 [20] and Semantic Boundaries Dataset (SBD) [29], which is 2.3% better than DeepLabv3 [5] at the same settings. Following the same procedure as [1,2,5,6,8,9,10,11,12], we employ Microsoft COCO dataset [36] to pretrain our model. COCO has 80 classes and we only retain images including the same 20 classes in PASCAL VOC 2012 and all other Index ResNeXt 131 COCO Flip miou (%) 1 (ResNeXt 101) Table 7. Strategies and results on PASCAL VOC 2012 validation set
13 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 13 classes are regarded as background. Training process has 3 stages. In stage- 1, we mix up all images in COCO, SBD and standard PASCAL VOC 2012 images, resulting in images for training in total. In stage-2, we utilize SBD and PASCAL VOC 2012 training images. Finally for stage-3, we only employ standard PASCAL VOC 2012 training set. We keep image crop size unchanged during the whole training procedure and all other settings are exactly the same as [8]. COCO pretraining brings about another 4.6% increase in performance, as shown in Table 7 (#2 and #3). We further average the score map of an image with its horizontal flipped version and eventually get a 85.8% miou on PASCAL VOC 2012 validation set, which is 2.3% better than DeepLabv3+ [37] (Table 7 #4). Resembling [5], we then freeze the batch normalization parameters and fine tune our model on official PASCAL VOC 2012 trainval set. In particular, we duplicate the images that contain hard classes (namely bicycle, chair, dining table, potted plant and sofa). Finally, our ExFuse framework achieves 87.9% miou on PASCAL VOC 2012 test set without any DenseCRF [38] post-processing, which surpasses previous state-of-the-art results, as shown in Table 8. For fair comparison, we also evaluate our model using a standard ResNet101 and it achieves 86.2% miou, which is better than DeepLabv3 at the same setting. Method miou Tusimple [9] 83.1 Large Kernel Matters [8] 83.6 Multipath RefineNet [11] 84.2 ResNet 38 MS COCO [39] 84.9 PSPNet [10] 85.4 DeepLabv3 [5] 85.7 SDN [40] 86.6 DeepLabv3+ (Xception) [37] 87.8 ExFuse ResNet101 (ours) 86.2 ExFuse ResNeXt131 (ours) 87.9 Table 8. Performance on PASCAL VOC 2012 test set Fig 7 visualizes some representative results of the GCN [8] baseline and our proposed ExFuse framework. It is clear that the visualization quality of our method is much better than the baseline. For example, the boundary in ExFuse is more precise than GCN. 5 Conclusions In this work, we first point out the ineffective feature fusion problem in current U-Net structure. Then, we propose our ExFuse framework to tackle this problem via bridging the gap between high-level low-resolution and low-level
14 14 Zhang et al. Fig. 7. Examples of semantic segmentation results on PASCAL VOC 2012 validation set. (b) is our GCN [8] baseline which achieves 81.0% miou on val set. (c) is our method which achieves 85.4% on val set, as shown in Table 7 #3.
15 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 15 high-resolution features. Eventually, better feature fusion is demonstrated by the performance boost when fusing with original low-level features and the overall segmentation performance is improved by a large margin. Our ExFuse framework also achieves new state-of-the-art performance on PASCAL VOC 2012 benchmark. References 1. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence PP(99) (2016) Ghiasi, G., Fowlkes, C.C.: Laplacian pyramid reconstruction and refinement for semantic segmentation. In: European Conference on Computer Vision. (2016) Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for scene segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence PP(99) (2017) Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. (2015) 5. Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. (2017) 6. Amirul Islam, M., Rochan, M., Bruce, N.D.B., Wang, Y.: Gated feedback refinement network for dense image labeling. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). (July 2017) 7. Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. Springer International Publishing (2015) 8. Peng, C., Zhang, X., Yu, G., Luo, G., Sun, J.: Large kernel matters improve semantic segmentation by global convolutional network. (2017) 9. Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., Cottrell, G.: Understanding convolution for semantic segmentation. (2017) 10. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. (2016) 11. Lin, G., Milan, A., Shen, C., Reid, I.: Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. (2016) 12. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected crfs. Computer Science (4) (2014) Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition. (2015) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition. (2016) Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition. (2015) Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems. (2012) He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: European Conference on Computer Vision. (2016)
16 16 Zhang et al. 18. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. (2016) 19. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Computer Science (2014) 20. Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88(2) (2010) Dai, J., He, K., Sun, J.: Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In: IEEE International Conference on Computer Vision. (2015) Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets. Eprint Arxiv (2014) Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. (2015) Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. (2016) Aitken, A., Ledig, C., Theis, L., Caballero, J., Wang, Z., Shi, W.: Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize. (2017) 26. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: Imagenet: A large-scale hierarchical image database. In: Computer Vision and Pattern Recognition, CVPR IEEE Conference on. (2009) Pohlen, T., Hermans, A., Mathias, M., Leibe, B.: Full-resolution residual networks for semantic segmentation in street scenes. (2016) 28. Hariharan, B., Arbelaez, P., Girshick, R., Malik, J.: Hypercolumns for object segmentation and fine-grained localization. (2014) Hariharan, B., Arbelaez, P., Bourdev, L., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: International Conference on Computer Vision. (2011) Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: European conference on computer vision, Springer (2014) Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (2016) Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI. (2017) Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. arxiv preprint arxiv: (2017) 34. Shen, X., Chen, Y.C., Tao, X., Jia, J.: Convolutional neural pyramid for image processing. (2017) 35. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. (2016) 36. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision. (2014) Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. (2018) 38. Krähenbühl, P., Koltun, V.: Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in neural information processing systems. (2011)
17 ExFuse: Enhancing Feature Fusion for Semantic Segmentation Wu, Z., Shen, C., Hengel, A.V.D.: Wider or deeper: Revisiting the resnet model for visual recognition. (2016) 40. Fu, J., Liu, J., Wang, Y., Lu, H.: Stacked deconvolutional network for semantic segmentation. (2017)
R-FCN++: Towards Accurate Region-Based Fully Convolutional Networks for Object Detection
The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) R-FCN++: Towards Accurate Region-Based Fully Convolutional Networks for Object Detection Zeming Li, 1 Yilun Chen, 2 Gang Yu, 2 Yangdong
Channel Locality Block: A Variant of Squeeze-and-Excitation
Channel Locality Block: A Variant of Squeeze-and-Excitation 1 st Huayu Li Northern Arizona University Flagstaff, United State Northern Arizona University hl459@nau.edu arxiv:1901.01493v1 [cs.lg] 6 Jan
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION
REGION AVERAGE POOLING FOR CONTEXT-AWARE OBJECT DETECTION Kingsley Kuan 1, Gaurav Manek 1, Jie Lin 1, Yuan Fang 1, Vijay Chandrasekhar 1,2 Institute for Infocomm Research, A*STAR, Singapore 1 Nanyang Technological
Lecture 7: Semantic Segmentation
Semantic Segmentation CSED703R: Deep Learning for Visual Recognition (207F) Segmenting images based on its semantic notion Lecture 7: Semantic Segmentation Bohyung Han Computer Vision Lab. bhhanpostech.ac.kr
arxiv: v1 [cs.cv] 31 Mar 2016
Object Boundary Guided Semantic Segmentation Qin Huang, Chunyang Xia, Wenchao Zheng, Yuhang Song, Hao Xu and C.-C. Jay Kuo arxiv:1603.09742v1 [cs.cv] 31 Mar 2016 University of Southern California Abstract.
Learning Fully Dense Neural Networks for Image Semantic Segmentation
Learning Fully Dense Neural Networks for Image Semantic Segmentation Mingmin Zhen 1, Jinglu Wang 2, Lei Zhou 1, Tian Fang 3, Long Quan 1 1 Hong Kong University of Science and Technology, 2 Microsoft Research
arxiv: v1 [cs.cv] 8 Mar 2017 Abstract
Large Kernel Matters Improve Semantic Segmentation by Global Convolutional Network Chao Peng Xiangyu Zhang Gang Yu Guiming Luo Jian Sun School of Software, Tsinghua University, {pengc14@mails.tsinghua.edu.cn,
Content-Based Image Recovery
Content-Based Image Recovery Hong-Yu Zhou and Jianxin Wu National Key Laboratory for Novel Software Technology Nanjing University, China zhouhy@lamda.nju.edu.cn wujx2001@nju.edu.cn Abstract. We propose
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS. Kuan-Chuan Peng and Tsuhan Chen
A FRAMEWORK OF EXTRACTING MULTI-SCALE FEATURES USING MULTIPLE CONVOLUTIONAL NEURAL NETWORKS Kuan-Chuan Peng and Tsuhan Chen School of Electrical and Computer Engineering, Cornell University, Ithaca, NY
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs Zhipeng Yan, Moyuan Huang, Hao Jiang 5/1/2017 1 Outline Background semantic segmentation Objective,
Person Part Segmentation based on Weak Supervision
JIANG, CHI: PERSON PART SEGMENTATION BASED ON WEAK SUPERVISION 1 Person Part Segmentation based on Weak Supervision Yalong Jiang 1 yalong.jiang@connect.polyu.hk Zheru Chi 1 chi.zheru@polyu.edu.hk 1 Department
arxiv: v1 [cs.cv] 24 May 2016
Dense CNN Learning with Equivalent Mappings arxiv:1605.07251v1 [cs.cv] 24 May 2016 Jianxin Wu Chen-Wei Xie Jian-Hao Luo National Key Laboratory for Novel Software Technology, Nanjing University 163 Xianlin
Feature-Fused SSD: Fast Detection for Small Objects
Feature-Fused SSD: Fast Detection for Small Objects Guimei Cao, Xuemei Xie, Wenzhe Yang, Quan Liao, Guangming Shi, Jinjian Wu School of Electronic Engineering, Xidian University, China xmxie@mail.xidian.edu.cn
HIERARCHICAL JOINT-GUIDED NETWORKS FOR SEMANTIC IMAGE SEGMENTATION
HIERARCHICAL JOINT-GUIDED NETWORKS FOR SEMANTIC IMAGE SEGMENTATION Chien-Yao Wang, Jyun-Hong Li, Seksan Mathulaprangsan, Chin-Chin Chiang, and Jia-Ching Wang Department of Computer Science and Information
arxiv: v4 [cs.cv] 6 Jul 2016
Object Boundary Guided Semantic Segmentation Qin Huang, Chunyang Xia, Wenchao Zheng, Yuhang Song, Hao Xu, C.-C. Jay Kuo (qinhuang@usc.edu) arxiv:1603.09742v4 [cs.cv] 6 Jul 2016 Abstract. Semantic segmentation
arxiv: v2 [cs.cv] 19 Apr 2018
arxiv:1804.06215v2 [cs.cv] 19 Apr 2018 DetNet: A Backbone network for Object Detection Zeming Li 1, Chao Peng 2, Gang Yu 2, Xiangyu Zhang 2, Yangdong Deng 1, Jian Sun 2 1 School of Software, Tsinghua University,
arxiv: v2 [cs.cv] 18 Jul 2017
PHAM, ITO, KOZAKAYA: BISEG 1 arxiv:1706.02135v2 [cs.cv] 18 Jul 2017 BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks Viet-Quoc Pham quocviet.pham@toshiba.co.jp
Encoder-Decoder Networks for Semantic Segmentation. Sachin Mehta
Encoder-Decoder Networks for Semantic Segmentation Sachin Mehta Outline > Overview of Semantic Segmentation > Encoder-Decoder Networks > Results What is Semantic Segmentation? Input: RGB Image Output:
Mask R-CNN. Kaiming He, Georgia, Gkioxari, Piotr Dollar, Ross Girshick Presenters: Xiaokang Wang, Mengyao Shi Feb. 13, 2018
Mask R-CNN Kaiming He, Georgia, Gkioxari, Piotr Dollar, Ross Girshick Presenters: Xiaokang Wang, Mengyao Shi Feb. 13, 2018 1 Common computer vision tasks Image Classification: one label is generated for
arxiv: v1 [cs.cv] 2 Aug 2018
arxiv:1808.00897v1 [cs.cv] 2 Aug 2018 BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation Changqian Yu 1[0000 0002 4488 4157], Jingbo Wang 2[0000 0001 9700 6262], Chao Peng 3[0000
arxiv: v1 [cs.cv] 15 Oct 2018
Instance Segmentation and Object Detection with Bounding Shape Masks Ha Young Kim 1,2,*, Ba Rom Kang 2 1 Department of Financial Engineering, Ajou University Worldcupro 206, Yeongtong-gu, Suwon, 16499,
RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation Guosheng Lin 1 Anton Milan 2 Chunhua Shen 2,3 Ian Reid 2,3 1 Nanyang Technological University 2 University of Adelaide 3 Australian
Supplementary Material: Pixelwise Instance Segmentation with a Dynamically Instantiated Network
Supplementary Material: Pixelwise Instance Segmentation with a Dynamically Instantiated Network Anurag Arnab and Philip H.S. Torr University of Oxford {anurag.arnab, philip.torr}@eng.ox.ac.uk 1. Introduction
RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation Guosheng Lin 1,2, Anton Milan 1, Chunhua Shen 1,2, Ian Reid 1,2 1 The University of Adelaide, 2 Australian Centre for Robotic
arxiv: v2 [cs.cv] 19 Nov 2018
OCNet: Object Context Network for Scene Parsing Yuhui Yuan Jingdong Wang Microsoft Research {yuyua,jingdw}@microsoft.com arxiv:1809.00916v2 [cs.cv] 19 Nov 2018 Abstract In this paper, we address the problem
Computer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period starts
FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction
FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction Shuyang Sun 1, Jiangmiao Pang 3, Jianping Shi 2, Shuai Yi 2, Wanli Ouyang 1 1 The University of Sydney 2 SenseTime Research 3
Structured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
[Supplementary Material] Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors
[Supplementary Material] Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors Junhyug Noh Soochan Lee Beomsu Kim Gunhee Kim Department of Computer Science and Engineering
Computer Vision Lecture 16
Announcements Computer Vision Lecture 16 Deep Learning Applications 11.01.2017 Seminar registration period starts on Friday We will offer a lab course in the summer semester Deep Robot Learning Topic:
Elastic Neural Networks for Classification
Elastic Neural Networks for Classification Yi Zhou 1, Yue Bai 1, Shuvra S. Bhattacharyya 1, 2 and Heikki Huttunen 1 1 Tampere University of Technology, Finland, 2 University of Maryland, USA arxiv:1810.00589v3
Proceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
Efficient Segmentation-Aided Text Detection For Intelligent Robots
Efficient Segmentation-Aided Text Detection For Intelligent Robots Junting Zhang, Yuewei Na, Siyang Li, C.-C. Jay Kuo University of Southern California Outline Problem Definition and Motivation Related
Classifying a specific image region using convolutional nets with an ROI mask as input
Classifying a specific image region using convolutional nets with an ROI mask as input 1 Sagi Eppel Abstract Convolutional neural nets (CNN) are the leading computer vision method for classifying images.
Dataset Augmentation with Synthetic Images Improves Semantic Segmentation
Dataset Augmentation with Synthetic Images Improves Semantic Segmentation P. S. Rajpura IIT Gandhinagar param.rajpura@iitgn.ac.in M. Goyal IIT Varanasi manik.goyal.cse15@iitbhu.ac.in H. Bojinov Innit Inc.
Instance-aware Semantic Segmentation via Multi-task Network Cascades
Instance-aware Semantic Segmentation via Multi-task Network Cascades Jifeng Dai, Kaiming He, Jian Sun Microsoft research 2016 Yotam Gil Amit Nativ Agenda Introduction Highlights Implementation Further
Mask R-CNN. presented by Jiageng Zhang, Jingyao Zhan, Yunhan Ma
Mask R-CNN presented by Jiageng Zhang, Jingyao Zhan, Yunhan Ma Mask R-CNN Background Related Work Architecture Experiment Mask R-CNN Background Related Work Architecture Experiment Background From left
Extend the shallow part of Single Shot MultiBox Detector via Convolutional Neural Network
Extend the shallow part of Single Shot MultiBox Detector via Convolutional Neural Network Liwen Zheng, Canmiao Fu, Yong Zhao * School of Electronic and Computer Engineering, Shenzhen Graduate School of
arxiv: v2 [cs.cv] 10 Oct 2018
Net: An Accurate and Efficient Single-Shot Object Detector for Autonomous Driving Qijie Zhao 1, Tao Sheng 1, Yongtao Wang 1, Feng Ni 1, Ling Cai 2 1 Institute of Computer Science and Technology, Peking
Deconvolutions in Convolutional Neural Networks
Overview Deconvolutions in Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Deconvolutions in CNNs Applications Network visualization
Deep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Supplementary Material
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Supplementary Material Xintao Wang 1 Ke Yu 1 Chao Dong 2 Chen Change Loy 1 1 CUHK - SenseTime Joint Lab, The Chinese
Unified, real-time object detection
Unified, real-time object detection Final Project Report, Group 02, 8 Nov 2016 Akshat Agarwal (13068), Siddharth Tanwar (13699) CS698N: Recent Advances in Computer Vision, Jul Nov 2016 Instructor: Gaurav
arxiv: v1 [cs.cv] 9 Aug 2017
BlitzNet: A Real-Time Deep Network for Scene Understanding Nikita Dvornik Konstantin Shmelkov Julien Mairal Cordelia Schmid Inria arxiv:1708.02813v1 [cs.cv] 9 Aug 2017 Abstract Real-time scene understanding
Object detection with CNNs
Object detection with CNNs 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before CNNs After CNNs 0% 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 year Region proposals
MULTI-SCALE OBJECT DETECTION WITH FEATURE FUSION AND REGION OBJECTNESS NETWORK. Wenjie Guan, YueXian Zou*, Xiaoqun Zhou
MULTI-SCALE OBJECT DETECTION WITH FEATURE FUSION AND REGION OBJECTNESS NETWORK Wenjie Guan, YueXian Zou*, Xiaoqun Zhou ADSPLAB/Intelligent Lab, School of ECE, Peking University, Shenzhen,518055, China
Ladder-style DenseNets for Semantic Segmentation of Large Natural Images
Ladder-style DenseNets for Semantic Segmentation of Large Natural Images Ivan Krešo Siniša Šegvić Faculty of Electrical Engineering and Computing University of Zagreb, Croatia name.surname@fer.hr Josip
arxiv: v1 [cs.cv] 1 Feb 2018
Learning Semantic Segmentation with Diverse Supervision Linwei Ye University of Manitoba yel3@cs.umanitoba.ca Zhi Liu Shanghai University liuzhi@staff.shu.edu.cn Yang Wang University of Manitoba ywang@cs.umanitoba.ca
arxiv: v2 [cs.cv] 8 Apr 2018
Single-Shot Object Detection with Enriched Semantics Zhishuai Zhang 1 Siyuan Qiao 1 Cihang Xie 1 Wei Shen 1,2 Bo Wang 3 Alan L. Yuille 1 Johns Hopkins University 1 Shanghai University 2 Hikvision Research
Fully Convolutional Networks for Semantic Segmentation
Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer* Trevor Darrell UC Berkeley Chaim Ginzburg for Deep Learning seminar 1 Semantic Segmentation Define a pixel-wise labeling
In-Place Activated BatchNorm for Memory- Optimized Training of DNNs
In-Place Activated BatchNorm for Memory- Optimized Training of DNNs Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder Mapillary Research Paper: https://arxiv.org/abs/1712.02616 Code: https://github.com/mapillary/inplace_abn
arxiv: v2 [cs.cv] 23 Nov 2017
Light-Head R-CNN: In Defense of Two-Stage Object Detector Zeming Li 1, Chao Peng 2, Gang Yu 2, Xiangyu Zhang 2, Yangdong Deng 1, Jian Sun 2 1 School of Software, Tsinghua University, {lizm15@mails.tsinghua.edu.cn,
Dynamic Video Segmentation Network
Dynamic Video Segmentation Network Yu-Syuan Xu, Tsu-Jui Fu, Hsuan-Kung Yang, Student Member, IEEE and Chun-Yi Lee, Member, IEEE Elsa Lab, Department of Computer Science, National Tsing Hua Uiversity {yusean0118,
3D Densely Convolutional Networks for Volumetric Segmentation. Toan Duc Bui, Jitae Shin, and Taesup Moon
3D Densely Convolutional Networks for Volumetric Segmentation Toan Duc Bui, Jitae Shin, and Taesup Moon School of Electronic and Electrical Engineering, Sungkyunkwan University, Republic of Korea arxiv:1709.03199v2
Deep learning for object detection. Slides from Svetlana Lazebnik and many others
Deep learning for object detection Slides from Svetlana Lazebnik and many others Recent developments in object detection 80% PASCAL VOC mean0average0precision0(map) 70% 60% 50% 40% 30% 20% 10% Before deep
Show, Discriminate, and Tell: A Discriminatory Image Captioning Model with Deep Neural Networks
Show, Discriminate, and Tell: A Discriminatory Image Captioning Model with Deep Neural Networks Zelun Luo Department of Computer Science Stanford University zelunluo@stanford.edu Te-Lin Wu Department of
arxiv: v2 [cs.cv] 10 Apr 2017
Fully Convolutional Instance-aware Semantic Segmentation Yi Li 1,2 Haozhi Qi 2 Jifeng Dai 2 Xiangyang Ji 1 Yichen Wei 2 1 Tsinghua University 2 Microsoft Research Asia {liyi14,xyji}@tsinghua.edu.cn, {v-haoq,jifdai,yichenw}@microsoft.com
Parallel Feature Pyramid Network for Object Detection
Parallel Feature Pyramid Network for Object Detection Seung-Wook Kim [0000 0002 6004 4086], Hyong-Keun Kook, Jee-Young Sun, Mun-Cheon Kang, and Sung-Jea Ko School of Electrical Engineering, Korea University,
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation Jifeng Dai Kaiming He Jian Sun Microsoft Research {jifdai,kahe,jiansun}@microsoft.com Abstract Recent leading
An Analysis of Scale Invariance in Object Detection SNIP
An Analysis of Scale Invariance in Object Detection SNIP Bharat Singh Larry S. Davis University of Maryland, College Park {bharat,lsd}@cs.umd.edu Abstract An analysis of different techniques for recognizing
Presentation Outline. Semantic Segmentation. Overview. Presentation Outline CNN. Learning Deconvolution Network for Semantic Segmentation 6/6/16
6/6/16 Learning Deconvolution Network for Semantic Segmentation Hyeonwoo Noh, Seunghoon Hong,Bohyung Han Department of Computer Science and Engineering, POSTECH, Korea Shai Rozenberg 6/6/2016 1 2 Semantic
Pixel Offset Regression (POR) for Single-shot Instance Segmentation
Pixel Offset Regression (POR) for Single-shot Instance Segmentation Yuezun Li 1, Xiao Bian 2, Ming-ching Chang 1, Longyin Wen 2 and Siwei Lyu 1 1 University at Albany, State University of New York, NY,
arxiv: v1 [cs.cv] 5 Oct 2015
Efficient Object Detection for High Resolution Images Yongxi Lu 1 and Tara Javidi 1 arxiv:1510.01257v1 [cs.cv] 5 Oct 2015 Abstract Efficient generation of high-quality object proposals is an essential
arxiv: v1 [cs.cv] 19 Mar 2018
arxiv:1803.07066v1 [cs.cv] 19 Mar 2018 Learning Region Features for Object Detection Jiayuan Gu 1, Han Hu 2, Liwei Wang 1, Yichen Wei 2 and Jifeng Dai 2 1 Key Laboratory of Machine Perception, School of
A MultiPath Network for Object Detection
ZAGORUYKO et al.: A MULTIPATH NETWORK FOR OBJECT DETECTION 1 A MultiPath Network for Object Detection Sergey Zagoruyko, Adam Lerer, Tsung-Yi Lin, Pedro O. Pinheiro, Sam Gross, Soumith Chintala, Piotr Dollár
Computer Vision Lecture 16
Computer Vision Lecture 16 Deep Learning for Object Categorization 14.01.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar registration period
JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS. Zhao Chen Machine Learning Intern, NVIDIA
JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS Zhao Chen Machine Learning Intern, NVIDIA ABOUT ME 5th year PhD student in physics @ Stanford by day, deep learning computer vision scientist
Geometry-aware Traffic Flow Analysis by Detection and Tracking
Geometry-aware Traffic Flow Analysis by Detection and Tracking 1,2 Honghui Shi, 1 Zhonghao Wang, 1,2 Yang Zhang, 1,3 Xinchao Wang, 1 Thomas Huang 1 IFP Group, Beckman Institute at UIUC, 2 IBM Research,
Supplementary Material for Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains
Supplementary Material for Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains Jiahao Pang 1 Wenxiu Sun 1 Chengxi Yang 1 Jimmy Ren 1 Ruichao Xiao 1 Jin Zeng 1 Liang Lin 1,2 1 SenseTime Research
RSRN: Rich Side-output Residual Network for Medial Axis Detection
RSRN: Rich Side-output Residual Network for Medial Axis Detection Chang Liu, Wei Ke, Jianbin Jiao, and Qixiang Ye University of Chinese Academy of Sciences, Beijing, China {liuchang615, kewei11}@mails.ucas.ac.cn,
arxiv: v3 [cs.cv] 18 Oct 2017
SSH: Single Stage Headless Face Detector Mahyar Najibi* Pouya Samangouei* Rama Chellappa University of Maryland arxiv:78.3979v3 [cs.cv] 8 Oct 27 najibi@cs.umd.edu Larry S. Davis {pouya,rama,lsd}@umiacs.umd.edu
arxiv: v2 [cs.cv] 16 Sep 2018
Dual Attention Network for Scene Segmentation Jun Fu, Jing Liu, Haijie Tian, Zhiwei Fang, Hanqing Lu CASIA IVA {jun.fu, jliu, zhiwei.fang, luhq}@nlpr.ia.ac.cn,{hjtian bit}@63.com arxiv:809.0983v [cs.cv]
DifNet: Semantic Segmentation by Diffusion Networks
DifNet: Semantic Segmentation by Diffusion Networks Peng Jiang 1 Fanglin Gu 1 Yunhai Wang 1 Changhe Tu 1 Baoquan Chen 2,1 1 Shandong University, China 2 Peking University, China sdujump@gmail.com, fanglin.gu@gmail.com,
PSANet: Point-wise Spatial Attention Network for Scene Parsing
PSANet: Point-wise Spatial Attention Network for Scene Parsing engshuang Zhao 1, Yi Zhang 2, Shu Liu 1, Jianping Shi 3, Chen Change Loy 4, Dahua Lin 2, and Jiaya Jia 1,5 1 The Chinese University of ong
Mimicking Very Efficient Network for Object Detection
Mimicking Very Efficient Network for Object Detection Quanquan Li 1, Shengying Jin 2, Junjie Yan 1 1 SenseTime 2 Beihang University liquanquan@sensetime.com, jsychffy@gmail.com, yanjunjie@outlook.com Abstract
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun Presented by Tushar Bansal Objective 1. Get bounding box for all objects
arxiv: v1 [cs.cv] 30 Jul 2018
arxiv:1807.11164v1 [cs.cv] 30 Jul 2018 ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma 1,2 Xiangyu Zhang 1 Hai-Tao Zheng 2 Jian Sun 1 1 Megvii Inc (Face++) 2 Tsinghua
FCHD: A fast and accurate head detector
JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1 FCHD: A fast and accurate head detector Aditya Vora, Johnson Controls Inc. arxiv:1809.08766v2 [cs.cv] 26 Sep 2018 Abstract In this paper, we
Supplementary Material: Unconstrained Salient Object Detection via Proposal Subset Optimization
Supplementary Material: Unconstrained Salient Object via Proposal Subset Optimization 1. Proof of the Submodularity According to Eqns. 10-12 in our paper, the objective function of the proposed optimization
Semantic Segmentation with Scarce Data
Semantic Segmentation with Scarce Data Isay Katsman * 1 Rohun Tripathi * 1 Andreas Veit 1 Serge Belongie 1 Abstract Semantic segmentation is a challenging vision problem that usually necessitates the collection
Object Detection Based on Deep Learning
Object Detection Based on Deep Learning Yurii Pashchenko AI Ukraine 2016, Kharkiv, 2016 Image classification (mostly what you ve seen) http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-detection.pdf
arxiv: v2 [cs.cv] 13 Mar 2019
An End-to-End Network for Panoptic Segmentation Huanyu Liu 1, Chao Peng 2, Changqian Yu 3, Jingbo Wang 4, Xu Liu 5, Gang Yu 2, Wei Jiang 1 1 Zhejiang University, 2 Megvii Inc. (Face++), 3 Huazhong University
Gated Bi-directional CNN for Object Detection
Gated Bi-directional CNN for Object Detection Xingyu Zeng,, Wanli Ouyang, Bin Yang, Junjie Yan, Xiaogang Wang The Chinese University of Hong Kong, Sensetime Group Limited {xyzeng,wlouyang}@ee.cuhk.edu.hk,
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation Golnaz Ghiasi (B) and Charless C. Fowlkes Department of Computer Science, University of California, Irvine, USA {gghiasi,fowlkes}@ics.uci.edu
An Analysis of Scale Invariance in Object Detection SNIP
An Analysis of Scale Invariance in Object Detection SNIP Bharat Singh Larry S. Davis University of Maryland, College Park {bharat,lsd}@cs.umd.edu Abstract An analysis of different techniques for recognizing
Error Correction for Dense Semantic Image Labeling
Error Correction for Dense Semantic Image Labeling Yu-Hui Huang 1 Xu Jia 2 Stamatios Georgoulis 1 Tinne Tuytelaars 2 Luc Van Gool 1,3 1 KU-Leuven/ESAT-PSI, Toyota Motor Europe (TRACE) 2 KU-Leuven/ESAT-PSI,
Places Challenge 2017
Places Challenge 2017 Scene Parsing Task CASIA_IVA_JD Jun Fu, Jing Liu, Longteng Guo, Haijie Tian, Fei Liu, Hanqing Lu Yong Li, Yongjun Bao, Weipeng Yan National Laboratory of Pattern Recognition, Institute
DFT-based Transformation Invariant Pooling Layer for Visual Classification
DFT-based Transformation Invariant Pooling Layer for Visual Classification Jongbin Ryu 1, Ming-Hsuan Yang 2, and Jongwoo Lim 1 1 Hanyang University 2 University of California, Merced Abstract. We propose
SEMANTIC SEGMENTATION AVIRAM BAR HAIM & IRIS TAL
SEMANTIC SEGMENTATION AVIRAM BAR HAIM & IRIS TAL IMAGE DESCRIPTIONS IN THE WILD (IDW-CNN) LARGE KERNEL MATTERS (GCN) DEEP LEARNING SEMINAR, TAU NOVEMBER 2017 TOPICS IDW-CNN: Improving Semantic Segmentation
Traffic Multiple Target Detection on YOLOv2
Traffic Multiple Target Detection on YOLOv2 Junhong Li, Huibin Ge, Ziyang Zhang, Weiqin Wang, Yi Yang Taiyuan University of Technology, Shanxi, 030600, China wangweiqin1609@link.tyut.edu.cn Abstract Background
Team G-RMI: Google Research & Machine Intelligence
Team G-RMI: Google Research & Machine Intelligence Alireza Fathi (alirezafathi@google.com) Nori Kanazawa, Kai Yang, George Papandreou, Tyler Zhu, Jonathan Huang, Vivek Rathod, Chen Sun, Kevin Murphy, et
HENet: A Highly Efficient Convolutional Neural. Networks Optimized for Accuracy, Speed and Storage
HENet: A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage Qiuyu Zhu Shanghai University zhuqiuyu@staff.shu.edu.cn Ruixin Zhang Shanghai University chriszhang96@shu.edu.cn
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma 1,2[0000 0003 4628 8831], Xiangyu Zhang 1[0000 0003 2138 4608], Hai-Tao Zheng 2[0000 0001 5128 5649], and Jian Sun
A Bi-directional Message Passing Model for Salient Object Detection
A Bi-directional Message Passing Model for Salient Object Detection Lu Zhang, Ju Dai, Huchuan Lu, You He 2, ang Wang 3 Dalian University of Technology, China 2 Naval Aviation University, China 3 Alibaba
PASCAL Boundaries: A Semantic Boundary Dataset with A Deep Semantic Boundary Detector
PASCAL Boundaries: A Semantic Boundary Dataset with A Deep Semantic Boundary Detector Vittal Premachandran 1, Boyan Bonev 2, Xiaochen Lian 2, and Alan Yuille 1 1 Department of Computer Science, Johns Hopkins
arxiv: v1 [cs.cv] 26 May 2017
arxiv:1705.09587v1 [cs.cv] 26 May 2017 J. JEONG, H. PARK AND N. KWAK: UNDER REVIEW IN BMVC 2017 1 Enhancement of SSD by concatenating feature maps for object detection Jisoo Jeong soo3553@snu.ac.kr Hyojin
Detecting Heads using Feature Refine Net and Cascaded Multi-scale Architecture
Detecting Heads using Feature Refine Net and Cascaded Multi-scale Architecture Dezhi Peng, Zikai Sun, Zirong Chen, Zirui Cai, Lele Xie, Lianwen Jin School of Electronic and Information Engineering South
Semantic Segmentation
Semantic Segmentation UCLA:https://goo.gl/images/I0VTi2 OUTLINE Semantic Segmentation Why? Paper to talk about: Fully Convolutional Networks for Semantic Segmentation. J. Long, E. Shelhamer, and T. Darrell,
Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation
Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation Md Atiqur Rahman and Yang Wang Department of Computer Science, University of Manitoba, Canada {atique, ywang}@cs.umanitoba.ca
arxiv: v1 [cs.cv] 19 Feb 2019
Detector-in-Detector: Multi-Level Analysis for Human-Parts Xiaojie Li 1[0000 0001 6449 2727], Lu Yang 2[0000 0003 3857 3982], Qing Song 2[0000000346162200], and Fuqiang Zhou 1[0000 0001 9341 9342] arxiv:1902.07017v1