STEREO VISION WITH COGNIMEM

Size: px

Start display at page:

Download "STEREO VISION WITH COGNIMEM"

Baldric Wade
6 years ago
Views:

Applicatin Nte STEREO VISION WITH COGNIMEM THE APPLICATION Stere Visin is critical fr the evaluatin f the prximity f an bject and is the starting pint fr many machine visin applicatins.

1 Applicatin Nte STEREO VISION WITH COGNIMEM THE APPLICATION Stere Visin is critical fr the evaluatin f the prximity f an bject and is the starting pint fr many machine visin applicatins. Several cmmercial systems are available and prduce accurate results, but they are bulky and cstly which prevents their use as a cmmdity in eye-perated cmmunicatin devices, smart mtin sensrs, rbtic guidance, etc. The CgniMem pattern recgnitin chip can be an enabler fr a miniature and lw-cst stere tracker due t its recgnitin speed, lw-pwer cnsumptin and abilities t match fuzzy and ill-defined bjects and perfrm real-time learning. This applicatin nte describes hw t implement a stere tracking system based n the CgniMem technlgy. THE METHOD A Stere Tracking system is cmpsed at the minimum f tw sensrs placed at a knwn distance frm ne anther, a target recgnitin and tracking algrithm applied synchrnusly t the vide images f each sensr and a triangulatin frmula t estimate the distance f the target t the plane cmmn t the sensrs axes. Example f a setup with 1-axis mtin rev

2 Applicatin Nte The stere visin algrithm can be decmpsed int three phases as described belw and the CgniMem (CM1K) chip ffers essential benefits in each phase. PHASE 1: IDENTIFICATION OF THE INITIAL TARGET (REAL-TIME OR DEFERRED) - Depending n the cntext f the applicatin, the initial target identificatin can be made manually by an peratr r autmatically by a target identificatin engine. - In the first case, assuming that the peratr can select the target n a screen, the neurns f the CM1K are immediately put t wrk and autmatically learn the features f the target. The prcess f knwledge building has started and recgnitin is enabled. Nte that the manual learning can be executed n ne f the tw cameras nly. As sn as the initial target is detected then it must transmit its lcatin t the secnd camera. - In the secnd case, the features f the target t detect can be retrieved frm a knwledge base built ahead f time and saved t a file r stred n bard a Flash memry r else. The neurns f the CM1K can lad this knwledge in their memry and start recgnitin immediately. If the final system invlves tw identical visin bards, the CM1K f bth bards have t be laded with the same initial knwledge. - The feature vectrs t extract frm the lcatin f the target can be multiple and cntain infrmatin describing t the shape, texture and/r clr f the target learned as a whle r in parts. PHASE2: REAL-TIME TRACKING AND ADAPTIVE LEARNING Use f the high-speed recgnitin capabilities f the neurns t find the target and calculate the smallest pssible regin f search arund it in the next frame. If the visin sensr is munted n a mtin cntrller, the regin f search is changed mechanically by mving the sensr s the target is re-centered in the field f view. If the visin sensr is munted n a fixture, the regin f search is changed prgrammatically. In the event that the cnfidence f the neurns decreases because the target is changing and its initial features becme less and less relevant, the neurns will autmatically learn the latest features f the target t keep tracking crrectly. This is called real-time reinfrced learning and will ccur if the target gets clser r farther, rtates, gets partially hidden r in the shade, etc. Since the tw cameras d nt see the target in the same fcal plane, they can start expanding their cmmn initial knwledge base with different feature vectrs r the applicatin cntrller can ensure that the latest features are learned by bth cameras. rev

Applicatin Nte PHASE 3: DISTANCE EVALUATION Fr accurate calculatin f the distance f the target t the plane defined by the tw cameras, bth visin sensrs must be triggered t grab a new frame at the same

3 Applicatin Nte PHASE 3: DISTANCE EVALUATION Fr accurate calculatin f the distance f the target t the plane defined by the tw cameras, bth visin sensrs must be triggered t grab a new frame at the same time. Prvided that the target has been recgnized by bth cameras, its lcatin in the tw fields f view is submitted t a simple triangulatin frmula returning the distance f the target t the cameras. The resulting distance can be used t clse several lps simplifying the recgnitin and cnsequently the tracking mechanism: Lad f a different primitive knwledge base based n the scale f the target in the field f view and ptimizatin f the regin f search fr the new scale. Adjustment f an aut-zm lens rev

Applicatin Nte THE CHALLENGE The challenging aspects f a stere visin system depend n the type f target t identify, the scenery in which it will evlve, its speed and behavir.

4 Applicatin Nte THE CHALLENGE The challenging aspects f a stere visin system depend n the type f target t identify, the scenery in which it will evlve, its speed and behavir. They can be summarized with the fllwing tw actin items: - Build an initial primitive knwledge capable f detecting the target when it enters the field f view f a camera knwing that its scale, rientatin and clr intensity are unknwn. Nte that this primitive knwledge can be a set f multiple knwledge built n a variety f features and cntexts. - Define a tracking algrithm minimizing the regin f search arund the target withut risking lsing it. Indeed, the larger the regin f search, the slwer the respnse time and higher the prbability that the target exists the regin at the next frame. THE PROPOSED SOLUTION - A miniature system with the fllwing cmpnents: CMOS sensr with glbal shutter, external trigger Lens (ptinal aut-zm) CM1K chip with 1024 neurns (enugh t hld a primitive knwledge and expand it during reinfrced learning) rev

mtrs). Als establish cmmunicatin with the ther camera and utside wrld.

5 Applicatin Nte Prcessr r FPGA t synchrnize the peratins between the sensr, the neurns and the mtin cntrllers (whether prgrammable regins f search r external mtrs). Als establish cmmunicatin with the ther camera and utside wrld. Cmmunicatin prt (chice f Wi-Fi, USB, Ethernet, etc) Miniature mtin cntrllers rev

It has hardware. It has application software.

It has hardware. It has application software. Q.1 What is System? Explain with an example A system is an arrangement in which all its unit assemble wrk tgether accrding t a set f rules. It can als be defined as a way f wrking, rganizing r ding ne