System Block Diagram Tracking Trajectories of Migrating Birds Around a Skyscraper Brian Crombie Matt Zivney Project Advisors Dr. Huggins Dr. Stewart Dr. Malinowski
System Level Block Diagram The goal of this project is to develop a system to calculate, store, and display trajectories of birds flying within 100 meters of a skyscraper. Birds are identified and their position found through stereoscopic imaging. The system displays bird trajectories in three dimensions and provides a confidence indicator for each trajectory. Although the calculations for the confidence indicator have not been determined quantitatively, the confidence indicator will give a measure of the certainty associated with each trajectory. Uncertainty in the calculations may come about when correlating pixels between two images and when correlating bird locations between frames. The system level block diagram is shown in figure 1. Power Stepper Motor Encoder Stepper Motor Ctrl PC Camera Displays Photons Boom Camera 1 Image 1 Data Reduction Display Boom Angle Display Photons Camera 2 Image 2 Raw Data File Figure 1: System Level Block Diagram Camera Subsystem The two cameras convert photons into usable images for the computer. The user will set the cameras some distance apart and parallel to each other. By using two cameras, objects can be located in three dimensions. Camera Image to PC Output Image Figure 2: Camera Subsystem Block Diagram The Image output is a digital image to the computer or analog video to a frame grabber interfaced with the computer. The optimum interface is IEEE 1394 or USB 2.0, which allows image captures at resolutions of 640x480 and beyond, in addition to control over when each frame begins.
With analog video cameras there is no way to synchronize the frames from each camera, which could be a limitation. Operation by Mode Setup In setup mode the cameras may capture images to verify a connection with the PC. In monitoring mode the cameras capture images which are processed by the PC. Data Acquisition In data acquisition mode the cameras capture images which are processed by the PC. Data Reduction The cameras are not used in data reduction mode. Boom Subsystem The boom consists of a mounting structure for two cameras and a stepper motor. The boom can be turned, panning the cameras, using the stepper motor. The position of the boom is determined from the output of an encoder. Power Stepper Motor Encoder to PC Stepper Motor Ctrl from PC Boom Inputs Figure 3: Boom Subsystem Block Diagram Stepper Motor Control The Stepper Motor Control input is a control line from the PC to the stepper motor to turn the boom. It uses a digital signal to control the motor. Power
The Power input is the power required for the stepper motor. Outputs Encoder The Encoder output can be used by the PC to determine the current angle of the boom. It uses a pulse train the PC can analyze to determine the rotation of the boom. Operation by Mode Setup The boom subsystem may be used to pan the cameras to a position specified by the user. The boom is not used in monitoring mode. Data Acquisition The boom is not used in data acquisition mode Data Reduction The boom is not used in data reduction mode. PC Subsystem The PC subsystem uses custom image processing software to find moving birds, identify their positions, and calculate their trajectories. Encoder from Boom Stepper Motor Ctrl to Boom Image from Camera 1 Image from Camera 2 PC Camera Displays to User Data Reduction Display to User Boom Angle Display to User Raw Data File to Disk Figure 4: PC Subsystem Block Diagram Inputs Camera Images The Camera Image inputs are the primary inputs to the object identification, locating, and tracking algorithms. Boom Encoder The Boom Encoder input is used to determine the angle of the Boom. Outputs Camera Displays
The Camera Displays show real-time images from the Camera L and Camera R. Data Reduction Display The Data Reduction Display shows calculated locations and trajectories in 3D along with confidence indicators. Boom Angle Display The Boom Angle Display shows the current angle of the boom. Raw Data File The Raw Data File output stores all calculated values for later display. Operation by Mode Setup In setup mode the PC requests input from the user, and may turn the boom to a position determined from user input. The camera images may also be displayed so the user can verify that the cameras are operating correctly, and to verify that the cameras are focused on the desired volume of space. In monitoring mode the PC acquires images and process them to detect movement. If movement is detected in a number of consecutive frames then the system enters data acquisition mode. Data Acquisition In data acquisition mode the PC acquires images from both cameras and process them to detect moving objects. The moving objects in each image are correlated to moving objects in the other image. From the location of each object in their respective images the location of the object is determined in three dimensions. Finally, the moving objects are correlated with tracking information from past frames to determine the trajectory of each object. If a number of consecutive frames are captured without movement then the system enters monitoring mode. Data Reduction In data reduction mode the PC uses stored or real-time acquired data to generate a 3D plot of bird trajectories around the skyscraper.
Software Flowcharts The following are software flow charts for the four modes of operation for the project: Setup,, Data Acquisition, and Data Display. Setup Mode Flow Chart Setup Mode allows system parameters to be initialized or adjusted by the user. At this point the only parameter that will be entered by the user in Setup Mode is the camera position. Obtain system initialization parameters from user including camera position input Is user input valid? Go to Mode Figure 5 Setup Mode Software Flowchart
Mode Flow Chart After Setup Mode, the system enters Mode, where input from the cameras is processed to detect movement of birds. When moving birds have been detected for a specified number of frames, the system enters Data Acquisition Mode. The number of frames with movement found before Data Acquisition Mode is entered has not been established at this point; it will be determined once experimental data is gathered. Obtain Images from Cameras Identify moving objects Is movement shown in images? Increment number of frames with movement Have moving birds been detected for a specified number of frames? Enter Data Acquisition Mode Figure 6 Mode Software Flowchart
Data Acquisition Mode Flow Chart In Data Acquisition Mode the trajectories of birds flying through the field of view of the camera are calculated and stored, and for each trajectory calculation a confidence indicator is generated. The method for determining the confidence indicator has yet to be determined quantitatively, but it will be a measure of the confidence in the correlation of images. The method for correlating images has also not been determined at this point. In Data Acquisition Mode, images are obtained from the cameras for processing until no moving birds have been detected for a specified number of frames. This specified number of frames will be established once experimental data has been gathered. Obtain Images from Cameras Identify moving objects Is movement shown in images? Has number of frames allowed with no moving objects found been exceeded? Go to Mode Correlate objects in images and generate confidence indicator Calculate positions of objects Correlate objects from previous pair of images with current pair of images Update trajectories Store data Figure 7 Data Acquisition Mode Software Flowchart
Data Display Mode Flow Chart Data Display Mode displays the bird trajectories in 3-D using real time or stored data, and also displays a confidence indicator for each trajectory. The point of view of the 3-D image can be changed by the user so the trajectories can be seen from different angles. Read stored or real-time data Display 3-D trajectories and confidence indicator Is there user input to change the 3-D point of view? Change 3-D point of view Is data in real time? Generate delay to control frame rate Figure 8 Data Display Mode Software Flowchart