Comparing Boosted Cascades to Deep Learning Architectures for Fast and Robust Coconut Tree Detection in Aerial Images

Size: px

Start display at page:

Download "Comparing Boosted Cascades to Deep Learning Architectures for Fast and Robust Coconut Tree Detection in Aerial Images"

Tracy Peters
5 years ago
Views:

1 Cmparing Bsted Cascades t Deep Learning Architectures fr Fast and Rbust Ccnut Tree Detectin in Aerial Images VISAPP2018, January 2018 Steven Puttemans*, Kristf Van Beeck* and Tn Gedemé

2 Intrductin Prject in cperatin with Dutch cmpany Airbrne mapping and surveying Farm and crp inspectin Crp cunting, predict crp prductivity Crp perfrmance, early detectin f health prblems Land use Lcatins fr expansin Planning f land use, planting pattern, height differences Envirnmental analytics (predict ersin, fld risks, ) 2

3 Intrductin Our gal: generate statistics n the number f ccnut trees frm these aerial images 3

Intrductin Currently, this is dne manually Human anntatrs click ccnut tree centers Circle with predefined average diameter (fixed flying height)

4 Intrductin Currently, this is dne manually Human anntatrs click ccnut tree centers Circle with predefined average diameter (fixed flying height) Cumbersme, time-cnsuming and expensive Avid errr and anntatin bias: label same image with multiple anntatrs Mistakes (frget trees, select wrng lcatins, ) 4

vegetatin, ccluded under trees, nt always

5 Challenges Perfect task t autmate! Simple bject detectin task? Challenges: Different vegetatins, ccnut trees in between ther very similar vegetatin, ccluded under trees, nt always strict pattern, different stages f grwth, anntatins 5 N ccnut trees (lk similar!)

6 Apprach Gal f this wrk: cmpare different bject detectin methdlgies fr reliable ccnut tree cunting Tailred twards ease-f-use fr cmpanies Accuracy, runtime, training time, number f training images, We cmpare: Mre traditinal cascade classifier bject detectrs With deep-learned bject detectrs 6

7 Related wrk Bsted cascade f weak classifiers Vila & Jnes (2001): Haar wavelets + AdaBst Early rejectin f nn-bject patches, integral images +: Simple, fast -: n clr, lw accuracy? Often imprved with scene cnstraints and applicatin specific cnstraints ICF (Dllar et al., 2009) Multiple features & clr Extensin t ACF (2014): rectangles + apprx. features +: Higher accuracy -: slwer? 7

Related wrk New trend since 2015: deep learning Enrmus datasets, drp in

(2016) tp accuracy n ImageNet Frm classificatin nets t detectin:

parts which need t be tuned Current trend: single-pass detectrs SSD

8 Related wrk New trend since 2015: deep learning Enrmus datasets, drp in GPU hardware cst Pre-trained nets AlexNet (2012), DenseNet (2014), ResNet (2016) tp accuracy n ImageNet Frm classificatin nets t detectin: multi-scale sliding windw cmputatinally expensive Regin prpsal netwrks tw parts which need t be tuned Current trend: single-pass detectrs SSD (2016), Yl9000 (2017) Real-time perfrmance: 120 VGA reslutin Are V&J and ACF dead? 8

Dataset and framewrks A single 10.000 x 10.

9 Dataset and framewrks A single x pixel image, RGB frmat Ccnut trees: 100 x 100 pixels 3798 anntatins Framewrks: V&J: OpenCV3.2 ACF: internal C++ framewrk InceptinV3: Tensrflw C/CUDA darknet framewrk Darknet19 & Densenet201 9

10 Appraches with bsted cascades First apprach: V&J, 2001 Using LBP (Ahnen et al., 2004) N clr infrmatin (cnvert t grayscale images) N bvius separatin between ccnut and backgrund therwise first clr transfrmatin (e.g. slar panels) Training: split image in fur parts, train n tp left, test thers parts Increase number f ps/neg samples fr each mdel Data augmentatin: randmly flipping patches arund vertical/hrizntal axes Single depth binary decisin trees 10

11 Appraches with bsted cascades Secnd apprach: ACF, 2014 Add multiple channels and clr Initially trained n tp left crner ACF uses a lt mre negatives Nt able t sample enugh frm tp left crner Split dataset: upper (1.741 psitives) and lwer half (1.914 psitives) Up t negative patches 11

12 Appraches with deep learning Third apprach: Deep learning, 2014 Mst likely better accuracy At which cst? Training time? Ease-f-use? Training with limited psitives in three manners: Learn a cmplete new deep netwrk Nt advised, try t see what s pssible Freezing (n-1) layers, nly retrain final layer Transfer learning, nly limited data required Only wrks if new data relates t data f which initial mdel was trained Fine-tuning weights f all layers Again, limited training data needed Mre flexible, new fine-tuned features fr specific task 12

able t cpe with dense bject packed scenes In ur case, bjects clse tgether

13 Appraches with deep learning We als tried a single-pass netwrk (YlV2) Much faster than multi-scale sliding windw Carse grid-based regin prpsals Nt able t cpe with dense bject packed scenes In ur case, bjects clse tgether and slightly verlapping Final utput detectins cver multiple bject instances 13

14 Results V&J Nt pssible t generate mre pints with OpenCV Even with limited training examples, still gd accuracy (P=90%, R=80%) Influence f amunt f training data Training time: 2 hurs CPU nly, evaluatin: 10 minutes ( x , Intel Xen E5-2687W 3.10 GHz) 14

15 Results ACF Mdel nt ptimal, trained n tp left crner Uses clr infrmatin, already much better (P=96%, R=90%) Influence f training/test data Training time: 30 minutes CPU nly, evaluatin: 5 minutes ( x , same hardware) 15

16 Results V&J versus ACF, bth trained n tp left crner Fr same precisin, recall imprves +- 12% 16

17 Results Deep learning: classificatin netwrks Train cmplete mdel frm scratch Mdel seems t cnverge (lss rate lwers) Tp-1 accuracy f 33% (tw classes: ccnut / backgrund) Transfer learning with frzen layers InceptinV3 in TensrFlw, 75 psitive examples / 75 backgrund examples Tp-1 accuracy f 77% Cmpare with bsted cascade: evaluatin at pixel level: P=75%, R=52% Transfer learning by fine tuning layers Darknet19 and Densenet201 Trade-ff between accuracy and inference time 17

Lss-rate Lss-rate Results Transfer learning by fine tuning layers Darknet19: 10.000 iteratins, Tp-1 accuracy f 95.

18 Lss-rate Lss-rate Results Transfer learning by fine tuning layers Darknet19: iteratins, Tp-1 accuracy f 95.2% Densenet201: iteratins, Tp-1 accuracy f 97.4% Training takes multiple hurs (24h fr Darknet19) Darknet19 Densenet201 Iteratins Iteratins 18

Results Deep learning: executin speeds Classificatin n NVIDIA TitanX Darknet19: 100x100 pixel patches: 265 FPS Densenet201: 52 FPS Memry ftprint nly 400MB

19 Results Deep learning: executin speeds Classificatin n NVIDIA TitanX Darknet19: 100x100 pixel patches: 265 FPS Densenet201: 52 FPS Memry ftprint nly 400MB Detectin: multi-scale nt needed Sliding windw evaluated ver different step sizes Achieves excellent accuracy f P=97.31%, R=88.85% V&J: 10 min ACF: 5 min 19

20 Visual results: V&J Green, TP Red, FP Magenta, FN High FP rate, especially n shadws (n clr infrmatin) Several FN (smaller trees) 20

21 Visual results: ACF Green, TP Red, FP Magenta, FN Abut equal amunt f FP: n shadws but in between trees Higher recall (less FN) FN again n smaller trees train separate mdel? 21

22 Visual results: DL Green, TP Red, FP Magenta, FN Almst n FP Again FNs: train separate mdel? reduce step size (50px here)? 22

Accuracy f ACF slightly lwer than DL Evaluatin time: depends n step size Training time and required hardware BIG

23 Cnclusin Evaluated the capability f lder bsted cascaded bject detectrs and deep learning fr ccnut tree detectin Best cascaded: 94.56% AP, 5-10 min evaluatin Best deep learning: 97.4% Tp-1 accuracy, 2m30 4h evaluatin Are VJ & ACF dead? Accuracy f ACF slightly lwer than DL Evaluatin time: depends n step size Training time and required hardware BIG difference (ACF wins) ACF Otherwise: dead, ACF lng is live dead, ACF! lng (if hardware life deep learning! is an issue) 23

24 Future wrk Cmbine regin prpsal netwrks with deep learning Lwer number f candidate patches Cmbine bth deep learning and bsted cascades Use principle f bsted cascaded where the weak classifiers are built using small cnvlutinal neural netwrks 24

25 Questins? Thank yu fr yur attentin! Cntact: 25

IWT TETRA TOBCAT Industrial applications of object categorization techniques. Steven Puttemans

IWT TETRA TOBCAT Industrial applications of object categorization techniques. Steven Puttemans Prgram 12:30 Registratin & cffee 13:00 Welcme and intrductin (Tn Gedemé) 13:15 Results frm the TOBCAT prject (Steven Puttemans) 14:00 Practical applicatins that use cmputer visin algrithms and their internal