Camera calibration for Studierstube

Transcription

1 Camera calibration for Studierstube If you are using ArToolKitPlus and a video background for your AR application, you need to do the following steps: Create or use existing camera calibration data (intrinsic camera parameters and radial distortion). If not already available, provide these parameters in the form as required by ArToolKitPlus. Run a Matlab tool provided in the current Studierstube 4 distribution to calculate the SoOffAxisCamera parameters and set your camera according to these parameters. Set the OpenVideo information for the used camera. ArToolKitPlus will need the radial / tangential distortion parameters in order to undistort the incoming camera image to achieve better marker detection. The SoOffAxisCamera is needed to set the camera parameters of the virtual camera according to the real camera (taking into account the focal length and the principal point offset). The OpenVideo needs information about the camera image resolution. Camera calibration procedure Existing calibration files In many cases, you will not need to calibrate your camera yourself. Many camera calibration files are available at Erick Mendez website: However, when using a camera with adjustable zoom you will need to calibrate your camera yourself. MATLab calibration procedure If you are using a camera not listed on this website, you have to run a calibration procedure yourself. We recommend Jean-Yves Bouguet s Matlab toolbox: Or the improved version: On the first link, there are also good tutorials how to use this toolbox (works for the improved version as well). Especially the following two tutorials are highly recommendable for calibrating the intrinsic parameters: Tutorial 1: Corner extraction, calibration: Tutorial 2: Doing your own calibration: Summarising these tutorials, these steps are necessary for the Studierstube camera calibration: 1

2 First take a decent amount of pictures of a calibration pattern (checkerboard). Checkerboard images can be downloaded directly from the toolbox website (second tutorial mentioned above). Best results were achieved when images were taken with the checkerboard at approximately the same distance to the camera as the markers will be in your AR application. Save your images according to the conventions described in Tutorial 2. Now start Matlab and step into the toolbox folder ( Current directory in the above tool bar). Type calib_gui into the console and select Standard from the appearing menu. Now the main menu appears: For the Studierstube camera calibration only the upper four menu items are needed. Now select Image names or Read images and type the image names (without suffix or number, as stated in Tutorial 2) into the console when prompted. Then add the image format (e.g. jpg ). All images according to the naming convention will now be loaded and displayed: As next step, click Extract grid corners from the GUI menu. Follow the commands in the console (select all images, keep the window size small default value 3 is ok). If the images have high radial distortion (as for example in the checker images above), we recommend not to use automatic object detection as with radial distortion the checker corners will not be found properly. Then the first image will be shown. Click the corners in the order shown in Tutorial 1 (from left upper corner to left lower corner). 2

3 Then enter the number of squares in x and y direction. In the image shown above, this will be 7 for x and 9 for y. Also measure the width and height of the printed checker squares and give the measures when asked for dx and dy in millimetres. Due to the strong radial distortion, the corner detection will not work properly, so a guess for a radial distortion factor will be needed: Play around with the parameter until the result is satisfying. In this example, we chose -0.6: 3

4 This obviously is sufficient for proper corner detection: Repeat this step for all images. When finished, press the Calibration button in the gui and Matlab will print the calculated parameters in the console. Our example produced the following output: [ ] Calibration results after optimization (with uncertainties): Focal Length: fc = [ ] ± [ ] Principal point: cc = [ ] ± [ ] Skew: alpha_c = [ ] ± [ ] => angle of pixel axes = ± degrees 4

5 Distortion: kc = [ ] ± [ ] Pixel error: err = [ ] Note: The numerical errors are approximately three times the standard deviations (for reference). Camera calibration for ArToolKitPlus ArToolKitPlus calibration file format ArToolKitPlus needs these parameters in a certain format, namely: [line1]: ARToolKitPlus_CamCal_Rev02 [line2]: xsize ysize cc_x cc_y fc_x fc_y kc1 kc2 kc3 kc3 kc5 kc6 iter See also the website of Daniel Wagner for more information: Xsize and ysize is the camera resolution during the calibration procedure. When running the Studierstube application, another camera resolution might be chosen as well without having to change the calibration data. Cc_x and cc_y represent the principal point offset in x and y direction. This corresponds to the cc value in the Matlab output. Fc_x and fc_y represent the focal lengths. This corresponds to the fc value in the Matlab output. The parameters kc1 to kc6 represent the radial / tangential distortion parameters. Matlab only has 5 values as output. Just add a 0.0 as k6 it will be ignored by ArToolKitPlus anyway. Iter is the number of iterations used to undistort the image. The Matlab toolkit uses 20 iterations default in their inverse distortion (normalize.m, comp_distortion_oulu.m), so this value seems to be reasonable. In our example, we created the following calibration file ( Pointgray.cal ): ARToolKitPlus_CamCal_Rev \ Opentracker.xml This file has to be referenced in the opentracker.xml file: <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE OpenTracker SYSTEM "opentracker.dtd"> <OpenTracker> <configuration> <!-- Here we place all the necessary configurations --> <ARToolKitPlusConfig camera-parameter="cameracal/pointgray.cal" undist-mode="lut" marker-mode="idbased" border-width="0.250" treshold="auto" pose-estimator="rpp" ov-config="openvideo.xml" ov-sink="artoolkitplusink" /> <ConsoleConfig headerline="sample Tracking Input" interval="1" display="off" /> <EventConfig keyevents="on" mouseevents="on"/> </configuration> 5

6 <!-- With a console sink we send the output to the shell --> <ConsoleSink comment="artk+" active="on"> <!-- This sink is in charge of sending data to stb4 --> <EventSink tracking="ottrack"> <!-- We add an extra tranformation --> <EventTransform scale="1 1 1" rotationtype="euler" rotation=" " translation="0 0 0"> <!-- This source receives data from ARToolKit Plus --> <ARToolKitPlusSource center="0 0" size=" " tag-id="30"/> </EventTransform> </EventSink> </ConsoleSink> </OpenTracker> Create SoOffAxisCamera The SoOffAxisCamera models a virtual camera from the given camera parameters of your real camera. You have to set these parameters in order to create a nice overlap between the shown video background and the virtual scene. SoOffAxisCamera Matlab tool The Studierstube installer provides a Matlab tool to calculate the needed parameters. The Matlab tool can be found in <Studierstube folder>\ ICG-Labs\Studierstube\stb4\tools\artoolkit2offaxis The tool is called artoolkit2offaxis.m. Start Matlab and step into the above folder using the Current Directory menu in the upper toolbar. The tool has 3 input parameters: the camera intrinsics matrix (3x3), the x resolution, and the y resolution. First, use the calibration parameters to create the following matrix: [ fcx 0.0 ccx ; 0.0 fcy (height-ccy) ; ] The height parameter represents the image height of the camera image during calibration process. Also refer to the README file in the artoolkit2offaxis folder. This matrix can be typed directly into the Matlab console. For our example, that would be: >> cam = [ ; ; ] Then press Return and type into the console: artoolkit2offaxis(cam, 320, 240) The two last parameters have to be changed according to your camera resolution used during the calibration process. The output, in our example, would be: position = size =

7 SoOffAxisCamera in Viewer Config File Add your SoOffAxisCamera to the viewer file in your Studierstube application folder. The position and size parameter can be retrieved from the Matlab tool. The eyepointposition has to be set to (0 0 0). For our example, the SoOffAxisCamera definition looks like that: DEF pointgraycamera SoOffAxisCamera position size eyepointposition viewportmapping ADJUST_CAMERA neardistance 0.01 fardistance 100 Your entire viewer.iv file could look something like that in the end: #Inventor V2.1 ascii SoDisplay # Specify position (upper-left corner) xoffset 0 yoffset 0 # and size of the viewer-window (in pixels) width 640 height 480 #SoDisplay #headlight TRUE #headlightintensity 1.0 backgroundcolor transparencytype BLEND clearbackground TRUE #showmouse TRUE windowborder FALSE #windowontop FALSE decoration TRUE #stencilbuffer FALSE showframerate TRUE userefcamera TRUE scenegraph SoSeparator SoSeparator SoViewport origin 0 0 size SoVideoBackground DEF pointgraycamera SoOffAxisCamera position size eyepointposition viewportmapping ADJUST_CAMERA neardistance 0.01 fardistance 100 SoStbScene #scenegraph SoSeparator Note: The file does not need to be named viewer.iv. The actual viewer file loaded has to be specified in kernel.xml, for example: 7

8 <studierstube> <kernel logmode="off" logfilename="kernellog.txt" guibinding="sowin" updatemode="timer" updaterate=" "/> <!-- This is the Event Component. It is in charge of providing us with Tracking data --> <Component name="event" availability="onload" lib="stbevent"> <Param key="configfile" value ="opentracker.xml"/> </Component> <!-- This is the Viewer Component. It is in charge of providing us with a Viewer window --> <Component name="viewer" availability="onload" lib="stbviewer_win"> <Param key="configfile" value="viewer.iv"/> </Component> <!-- This is the Starlight Component. It is in charge of providing us with extra nodes with stb3 functionalities --> <Component name="starlight" availability="onload" lib="stbstarlight"> </Component> <!-- This is the Python Binding component. It is in charge of providing procedural scripting facilities --> <Component name="bpython" availability="onload" lib="stbbpython"> </Component> <!-- This is the Video Component. It is in charge of providing us with Video Background in our scene --> <Component name="video" availability="onload" lib="stbvideo"> <Param key="configfile" value="openvideo.xml" /> <Param key="ovsinkname" value="stb4videosink" /> </Component> <!-- This is the Video Component. It is in charge of providing us with Video Background in our scene <Component name="video" availability="onload" lib="stbvideo"> <Param key="configfile" value="openvideo.xml"/> </Component> --> <!-- This is the Application --> <Application name="simpleapp" availability="onload" lib="simpleapp"> <Param key="scenefile" value ="scene.iv"/> <Param key="needviewer" value ="true"/> <Param key="needevent" value ="true"/> </Application> </studierstube> OpenVideo parameters In the OpenVideo xml file, a config file for video parameters is referenced (usually dsvl.xml ): <?xml version="1.0" encoding="utf-8"?> <openvideo schedulemode="idle" updaterate="15"> <DSVLSrc config-file="dsvl.xml" pixelformat="b8g8r8" num-buffers="4" flip-v="true"> <!-- This will deliver video to ARToolKitPlus --> <VideoSink name="stb4videosink" pixelformat="b8g8r8"/> </DSVLSrc> </openvideo> This xml file sets the parameter for OpenVideo. Set the desired resolution, framerate, pixel format, and, if known that the camera is supported, set the camera name and ask OpenVideo to open a format dialog when starting the Studierstube application: 8

9 <dsvl_input> <camera show_format_dialog="true" friendly_name="quickcam" frame_width="320" frame_height="240" frame_rate="30.0"> <pixel_format> <RGB24 /> </pixel_format> </camera> </dsvl_input> If you are unsure if a dialog box can be opened, set something like that: <dsvl_input> <camera show_format_dialog="false" frame_width="320" frame_height="240" frame_rate="15.0"> <pixel_format> <RGB24/> </pixel_format> </camera> </dsvl_input> SoUndistortedVideoBackground This node is not yet available in the current Studierstube version! In order to undistort the shown video background in the same way as ArToolKitPlus does when detecting the markers, a new video background was created. This node gets a camera calibration file as input (usually the same one as ArToolKitPlus, if used) to retrieve the camera calibration parameters. The node can also be used without ArToolKitPlus to reduce the visual radial distortion effect. Additionally, this node is capable of setting the parameters of a given SoOffAxisCamera, so the offaxis camera does not need to be calculated. However, at least one named SoOffAxisCamera has to be in the scene. The name has to be given to the SoUndistortedVideoBackground: Just exchange the SoVideoBackground by SoUndistortedVideoBackground. Your viewer.iv could thus look like that: #Inventor V2.1 ascii SoDisplay # Specify position (upper-left corner) xoffset 0 yoffset 0 # and size of the viewer-window (in pixels) width 640 height 480 #headlight TRUE #headlightintensity 1.0 backgroundcolor transparencytype BLEND clearbackground TRUE #showmouse TRUE windowborder FALSE #windowontop FALSE decoration TRUE #stencilbuffer FALSE showframerate TRUE userefcamera TRUE scenegraph SoSeparator SoSeparator SoViewport 9

10 #SoDisplay origin 0 0 size SoUndistortedVideoBackground calibfile "cameracal/pointgray.cal" #camera calibration file xresolution 10 #optional: resolution in x direction yresolution 10 #optional: resolution in y direction cameraname "cam" #Name of scene SoOffAxisCamera DEF cam SoOffAxisCamera position #will be overwritten by video bg size #will be overwritten by video bg eyepointposition #will be overwritten by video bg viewportmapping ADJUST_CAMERA neardistance 0.01 fardistance 100 SoStbScene #scenegraph SoSeparator 10