UFG-10 MC. Frame Grabbers LINUX SOFTWARE PROGRAMMING GUIDE. Customized Property

Transcription

1 UFG-10 MC Frame Grabbers LINUX SOFTWARE PROGRAMMING GUIDE Customized Property

2 General Copyright This manual, Copyright 2014 Unigraf Oy. All rights reserved Reproduction of this manual in whole or in part without written permission of Unigraf Oy is prohibited. Notice The information given in this manual is verified in the correctness on the date of issue. The authors reserve the rights to make any changes to this product and to revise the information about the products contained in this manual without an obligation to notify any persons about such revisions or changes. Edition UFG-10 MC Linux Software Programming Guide, Rev 1 Document identifier: Date: 02 October 2014 Trademarks Unigraf and UFG are trademarks of Unigraf Oy. All other trademarks are properties of their respective owners. Company information Unigraf Oy Piispantilankuja 4 FI ESPOO Finland info@unigraf.fi 2

3 General Table of Contents 1. General Driver Architecture... 5 Dual-Stream Module Dependency Module Installation and Un-installation... 8 Module Installation... 8 Module Un-installation... 8 System adjustment: Encoder Header Parser Encoder Property Setting Resolution & Audio Detection Scaling down preview video Scaling down H264 video Input Selection UFG-10 MC sample program H.264 Recorded File Playback with MPlayer Audio GPIO setting

4 General 1. GENERAL UFG-10 Linux driver is built based on Video 4 Linux 2 (V4L2) video and Advanced Linux Sound Architecture (ALSA) audio. Application development is straightforward by adopting the standard APIs. In addition, this document describes some customized properties for the user to control some special features of the UFG-10 boards. UFG-10 MC capture board features on board hardware compression. This manual describes the hardware compression functions and parameters available and the way they are recommended to use. Please find the full V4L2 and ALSA resources in the link below. V4L2 resource: ALSA resource: Note Please make sure that your Linux kernel compilation includes the V4L2 (video) and ALSA (audio) modules built in. Note If you are using a customized Linux kernel, please have the Virtual Video Driver built in your kernel. It is located at Device Drivers -> Multimedia Devices -> Video capture adapters -> Virtual Video Driver. 4

5 Driver Architecture 2. DRIVER ARCHITECTURE Dual-Stream UFG-10 MC sends dual-streams. It transmits raw data (YV12) stream and H.264 main stream and H.264 sub stream simultaneously. In this way, users can use raw data to preview image or do some intelligent functions, and can save the H.264 streams into storage and transmit it through network at the same time. Raw Data Stream YV12 format, SD and HD resolution. Only one video device.: /dev/video0, /dev/video3, /dev/video6, /dev/video9. H.264 Stream Main Stream Sub stream By default, the chipset on UFG-10 MC transmits two H.264 streams. The first four bytes of each H.264 data contain UFG-10 custom information about the H.264 data. The remaining payload is the real H.264 data. Please refer Section Encoder Header Parser later in this document for detailed information about this subject. The UFG-10 driver implements two video devices for each chip. Therefore, you can find two consecutive H.264 video devices in your system. Note The H.264 video devices are enumerated after live video devices, and the enumeration rule is one chipset after another. 5

6 Driver Architecture Example ls /dev/video* /dev/video0(live video device) /dev/video1(h.264 main video device) /dev/video2(h.264 sub video device) /dev/video3(live video device) /dev/video4(h.264 main video device) /dev/video5(h.264 sub video device) /dev/video6(live video device) /dev/video7(h.264 main video device) /dev/video8(h.264 sub video device) /dev/video9(live video device) /dev/video10(h.264 main video device) /dev/video11(h.264 sub video device) 6

7 Module Dependency 3. MODULE DEPENDENCY Note Please make sure that your Linux kernel compilation includes the V4L2 (video) and ALSA (audio) modules built in. Note If you are using a customized Linux kernel, please have the Virtual Video Driver and BT848 Video for Linux (V4L) built in your kernel. It is located at Device Drivers -> Multimedia Devices -> Video capture adapters category. 7

8 Un-installation Module Installation and 4. MODULE INSTALLATION AND UN-INSTALLATION UFG-10 MC Linux driver module name is LXV4L2D.ko. In order to prevent the possibility of installation failures, Unigraf will be glad to compile UFG-10 driver in an environment used by the customer. Please contact Unigraf for details. Module Installation Copy LXV4L2D.ko to your module directory. cp LXV4L2D.ko /lib/modules/`uname -r`/ Add LXV4L2D.ko to modules.dep. depmod a Load LXV4L2D.ko module. Modprobe LXV4L2D If invalid module format error occurs, try to use f parameter. Modprobe f LXV4L2D Module Un-installation Modprobe r LXV4L2D If you still have error message when loading the module, please provide your configuration file to Unigraf. It can be found in the following path: /usr/src/`uname -r`/.config. 8

9 Un-installation Module Installation and System adjustment: Enlarge memory allocation at system boot: Fedora: Update menu.lst, add "vmalloc=512m" after "quiet" Ubuntu Find "/etc/default/grub" and edit the line "GRUB_CMDLINE_LINUX_DEFAULT". Add the option: vmalloc=512mb GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vmalloc=512mb" sudo update-grub 9

10 Encoder Header Parser 5. ENCODER HEADER PARSER UFG-10 MC capture cards transmit H.264 bit-stream by means of standard V4L2 stream capture interface. For each video frame, the driver sends the bit-stream. Additionally one customized 4 bytes stream header data structure is embedded before the H.264 bit-stream payload. The data structure is called _AH8400_H264_STREAM_HEADER- That means that the 1028 bytes encoded data sent from the driver consists of a 4 bytes customized header and 1024 bytes for the real H.264 bit-stream payload. [4 BYTES HEADER] [H.264 BITSTREAM PAYLOAD] // FRAME #00 [4 BYTES HEADER] [H.264 BITSTREAM PAYLOAD] // FRAME #01 [4 BYTES HEADER] [H.264 BITSTREAM PAYLOAD] // FRAME #02... [4 BYTES HEADER] [H.264 BITSTREAM PAYLOAD] // FRAME #XX The data structure _AH8400_H264_STREAM_HEADER is defined as below: typedef struct _AH8400_H264_STREAM_HEADER { DWORD nframebuffersize :24; // FRAME BUFFER SIZE (H.264 BITSTREAM LENGTH) DWORD nframetype :1; // 0 = I FRAME / 1 = P FRAME DWORD nmotionstatus :1; // 0 = STATIC / 1 = MOTION DWORD nlockstatus :1; // 0 = UNLOCK / 1 = LOCK DWORD nreserved :5; } AH8400_H264_STREAM_HEADER; In data structure AH8400_H264_STREAM_HEADER, the H.264 frame buffer size and frame type (I/P frame) are separately stored in nframebuffersize and nframetype fields. The nmotionstatus field, is used to detect the video motion status while the system runs the motion detection function. The nlockstatus field is used to transmit current video signal status to the user. Example 1: Get Stream Header. struct MyStreamBuffer { void * pbuffer; size_t nbuffersize; }; struct MyStreamBuffer * p_stream_buffers = NULL; // to do memory mapping of buffer... // to queue buffer to driver... // to wait frame complete... // to dequeue buffer from driver 10

11 Encoder Header Parser // struct v4l2_buffer s_v4l2_buffer; s_v4l2_buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; s_v4l2_buffer.memory = V4L2_MEMORY_MMAP; ioctl( fd, VIDIOC_DQBUF, & s_v4l2_buffer ); // to get the first 4 bytes of the H.264 bit-stream buffer // DWORD dwstreamheader = (p_stream_buffers[ s_v4l2_buffer.index ]->pbuffer); ULONG nbuffersize = (dwstreamheader & 0x00FFFFFF) >> 0; UINT bframetype = (dwstreamheader & 0x ) >> 24; UINT nmotionstatus = (dwstreamheader & 0x ) >> 25; UINT nlockstatus = (dwstreamheader & 0x ) >> 26; //... 11

12 Encoder Property Setting 6. ENCODER PROPERTY SETTING V4L2_CID_MPEG_VIDEO_BITRATE_MODE V4L2_CID_MPEG_VIDEO_BITRATE V4L2_CID_MPEG_VIDEO_GOP_SIZE #define V4L2_CID_MPEG_VIDEO_QUALITY (V4L2_CID_MPEG_BASE + 250) #define V4L2_CID_MPEG_VIDEO_FRAMERATE (V4L2_CID_MPEG_BASE + 251) #define V4L2_CID_MPEG_VIDEO_OSD_ENABLE (V4L2_CID_MPEG_BASE + 256) #define V4L2_CID_MPEG_VIDEO_OSD (V4L2_CID_MPEG_BASE + 253) #define V4L2_CID_MPEG_SET_KEY_FRAME (V4L2_CID_MPEG_BASE + 255) #define V4L2_CID_ENC_DEINTERLACE (V4L2_CID_BASE + 298) #define V4L2_CID_RESET_I2S (V4L2_CID_BASE + 299) V4L2_CID_MPEG_VIDEO_H264_PROFILE V4L2_CID_MPEG_VIDEO_H264_LEVEL UFG-10 MC capture cards includes ioctl interfaces described below to allow the user to control the hardware encoder setting. vidioc_g_ext_ctrls vidioc_s_ext_ctrls Note These interfaces are based on your kernel version. Currently the property controls listed below are implemented for the UFG-10 MC: 1. Bit Rate Mode: CBR or VBR 2. Bit Rate: In CBR mode, you can set the target bit rate. RANGE: 128K ~ 12 * 1024 * 1024 BITS / SECOND 3. GOP: 0 ~ Quality: RANGE: 0 (WORST) ~ (BEST) 5. H264 frame rate: 1/10~1/2(1080P60) 6. osd(m/s both affected) 7. deinterlace type: 0 (progressive), 1~7(interleave) (M/S both affected) 8. force I frame 9. H264 profile, main frame or baseline (M/S both affected) 10. H264 level: from V4L2_MPEG_VIDEO_H264_LEVEL_1_0 to V4L2_MPEG_VIDEO_H264_LEVEL_4_1 (M/S both affected) 12

13 Encoder Property Setting Properties Available value Default Get / set SC3A0 / SC580 Stream Bit Rate Mode optional CBR, VBR VBR both both M/S Bit Rate optional 128K~12M 4M both both M/S Quality optional 0~10000(BEST) 6000 both both M/S GOP optional 0~ both both M/S H264 frame rate optional 2500, 3000, 5000, 6000, 6250, 7500, 12500,15000, 25000, both both M/S OSD enable optional 0, 1 0 both both M/S OSD string optional 63 chars timer both both M/S De-interlace must 0 (progressive), 1~7 (interleave) 1 both both M/S H264 profile optional main, baseline main both both M/S H264 level optional 1.0 ~ both both M/S The properties vidioc_g_ext_ctrls/ vidioc_s_ext_ctrls ioctlsare categorized in V4L2_CTRL_CLASS_MPEG class. Please fill ctrl_class field of structure v4l2_ext_controls with this macro. After that you can get/set the compression properties you request. Record Mode, Bit Rate, and GOP are readily supported in V4L2, so you can use the V4L2_CID_MPEG_VIDEO_BITRATE_MODE, V4L2_CID_MPEG_VIDEO_BITRATE and V4L2_CID_MPEG_VIDEO_GOP_SIZE macros to specify these properties. UFG-10 MC defines V4L2_CID_MPEG_VIDEO_QUALITY for video quality. Please implement this macro into your application. Example 1. Set compression recording mode to CBR and bit rate is 4 MB/s. struct v4l2_ext_controls sv4l2_ext_controls; struct v4l2_ext_control sv4l2_ext_control; memset( &sv4l2_ext_control, 0x0, sizeof(struct v4l2_ext_control) ); sv4l2_ext_controls.controls = &sv4l2_ext_control; sv4l2_ext_controls.ctrl_class = V4L2_CTRL_CLASS_MPEG; sv4l2_ext_controls.count = 1; sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE; sv4l2_ext_control.value = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR; ioctl ( fd, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_BITRATE; sv4l2_ext_control.value = 4 * 1024 * 1024; ioctl ( fd, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); 13

14 Encoder Property Setting Example 2: De-interlace type struct v4l2_control g_ctrl; g_ctrl.id = V4L2_CID_INTERLEAVE_DETECTED; ioctl( pdevice->m_hhwdevice, VIDIOC_G_CTRL, &g_ctrl ); if(g_ctrl.value == 0)//progressive { struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_ENC_DEINTERLACE; s_ctrl.value = 0x0000; ioctl( pdevice->m_hhwdevice, VIDIOC_S_CTRL, &s_ctrl ); //do only once } if(g_ctrl.value == 1)//interleave { struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_ENC_DEINTERLACE; if( width > 720 ) s_ctrl.value = 0x0007;//HD else s_ctrl.value = 0x0003;//SD } ioctl( pdevice->m_hhwdevice, VIDIOC_S_CTRL, &s_ctrl ); //do only once Example 3: Frame rate (based on 1080p60) sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_FRAMERATE; sv4l2_ext_control.value = 30000; //sv4l2_ext_control.value = 12500;//1/2 //sv4l2_ext_control.value = 15000;//1/2 //sv4l2_ext_control.value = 6250;//1/4 //sv4l2_ext_control.value = 7500;//1/4 //sv4l2_ext_control.value = 5000;//1/5 //sv4l2_ext_control.value = 6000;//1/5 //sv4l2_ext_control.value = 2500;//1/10 //sv4l2_ext_control.value = 3000;//1/10 ioctl(pdevice->m_hhwdevice, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); 14

15 Encoder Property Setting Example 4: Set key frame sv4l2_ext_control.id = V4L2_CID_SET_KEY_FRAME; sv4l2_ext_control.value = 1; ioctl(pdevice->m_hhwdevice, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); Example 5 SET KEY FRAME, separate by \n, maximum length 64 sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_OSD_ENABLE; sv4l2_ext_control. value = 1;// 0 is disable ioctl(pdevice->m_hhwdevice, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); //without below, default OSD timer will display CHAR OSD[64] ; memset(osd, 0x00, sizeof(osd)); memcpy(osd, " $$$Y.$M.$d $h.$n.$s $$$f\n Unigraf UFG-10 MC\n", 50); sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_OSD; sv4l2_ext_control.string = OSD; ioctl(pdevice->m_hhwdevice, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); Example 6 Profile sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE; sv4l2_ext_control.value = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN; ioctl(pdevice->m_hhwdevice, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); sv4l2_ext_control.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL; sv4l2_ext_control.value = V4L2_MPEG_VIDEO_H264_LEVEL_4_1; ioctl(pdevice->m_hhwdevice, VIDIOC_S_EXT_CTRLS, &sv4l2_ext_controls ); Example 7 Reset I2S struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_RESET_I2S; ioctl( pdevice->m_hhwdevice, VIDIOC_S_CTRL, &s_ctrl ); 15

16 Resolution & Audio Detection 7. RESOLUTION & AUDIO DETECTION #define V4L2_CID_X_RES_DETECTED (V4L2_CID_BASE + 294) #define V4L2_CID_Y_RES_DETECTED (V4L2_CID_BASE + 295) #define V4L2_CID_FPS_DETECTED (V4L2_CID_BASE + 296) #define V4L2_CID_AUDIO_SAMPLING_FREQ (V4L2_CID_BASE + 211) User can use above properties to get correct resolution detected by decoder. Available value Default Get / set Stream optional get RAW Example struct v4l2_control g_ctrl; g_ctrl.id = V4L2_CID_X_RES_DETECTED; ioctl( fd, VIDIOC_G_CTRL, &g_ctrl ); LINUXXWIN_DEBUG( "( X_RES = %d ) (2)\n", g_ctrl.value ); g_ctrl.id = V4L2_CID_Y_RES_DETECTED; ioctl( fd, VIDIOC_G_CTRL, &g_ctrl ); LINUXXWIN_DEBUG( "( Y_RES = %d ) (2)\n", g_ctrl.value ); g_ctrl.id = V4L2_CID_FPS_DETECTED; ioctl( fd, VIDIOC_G_CTRL, &g_ctrl ); LINUXXWIN_DEBUG( "( FPS_RES = %d ) (2)\n", g_ctrl.value g_ctrl.id = V4L2_CID_AUDIO_SAMPLING_FREQ; ioctl( pdevice->m_hhwdevice, VIDIOC_G_CTRL, &g_ctrl ); LINUXXWIN_DEBUG( "FREQ = %d \n", g_ctrl.value ); 16

17 Scaling down preview video 8. SCALING DOWN PREVIEW VIDEO #define V4L2_CID_HV_SCALE_FACTOR (V4L2_CID_BASE + 204) This function is to down scale the preview video resolution. It can be applied for and resolutions. If scale value is 0x will become will become If scale value is 0x will become Available value Default Get / set Stream optional 0x for 1280*720 0x / 0x for 1920*1080 Example: 0x get/set RAW struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_HV_SCALE_FACTOR; s_ctrl.value = 0x ;//not scaled ioctl( pdevice->m_hhwdevice, VIDIOC_S_CTRL, &s_ctrl ); 17

18 Scaling down H264 video 9. SCALING DOWN H264 VIDEO #define V4L2_CID_H264_HV_SCALE_FACTOR (V4L2_CID_BASE + 205) This function is to down scale the H264 video resolution. This setting will effect both main and sub H264 stream. optional 0x , 0x , 0x Available value Default Get / set Stream 0x get/set M/S Example: struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_H264_HV_SCALE_FACTOR; //s_ctrl.value = 0x ;// 1/2 of original resolution(m&s) //s_ctrl.value = 0x ;// 1/4 of original resolution(m&s) //s_ctrl.value = 0x ;//main original resolution, //sub 1/2 of original resolution ioctl( pdevice->m_hhwdevice, VIDIOC_S_CTRL, &s_ctrl ); Note The resolution of sub stream must be smaller than main stream 18

19 Input Selection 10. INPUT SELECTION You can use V4L VIDIOC_S_INPUT for video. You can use V4L VIDIOC_S_AUDIO for audio. UFG-10 MC supports multiple inputs. User has to set the correct input. //HDMI 0 //DVI.DIGITAL (TMDS.A) 1 //COMPONENT 2 //DVI.ANALOG 3 //SDI 4 //COMPOSITE 5 //S 6 Available value Default Get / set Stream optional 0~6 4 get/set RAW M/S Example: Set video input signal to SDI ULONG input = 4;//SDI ioctl(fd, VIDIOC_S_INPUT, &input ); Example: Set audio input signal to SDI embedded struct v4l2_audio v4l_a; v4l_a.index = 0;//embedded audio //v4l_a.index = 1;//exernal line in ioctl( pdevice->m_hhwdevice, VIDIOC_S_AUDIO, &v4l_a ); 19

20 UFG-10 MC sample program 11. UFG-10 MC SAMPLE PROGRAM Installation The included UFG-10 MC sample program will display one live channel and record two H.264 channels at the same time. Recorded file names are the same as their node name. Press 'q' to exit the program. Step 1. If you want to change video standard (NTSC/PAL) or video resolutions, please modify line 16~20 in LINUXXWIN.c. line16: #define DEFAULT_VIDEO_STANDARD V4L2_STD_NTSC_M line17:// #define DEFAULT_VIDEO_STANDARD V4L2_STD_PAL_B line19: int width = 720, height = 480; // NTSC line20:// int width = 720, height = 576; // PAL Step 2. Type make. Step 3:./LINUXXWIN 20

21 UFG-10 MC sample program H.264 Recorded File Playback with MPlayer Playback The recorded H.264 file can be playback with MPlayer ( After installing the codecs and MPlayer, you can type the command to playback the H.264 file. mplayer fps 30 H.264_FILE Playback with de-interlacing filter You can get a list of de-interlacing options by typing mplayer -pphelp at the command line. Find the option that you want to use, and then use the -vf pp=<option> syntax to activate the option. The command shown below is using ffmpeg de-interlacer to process the h.264 file. mplayer vf pp=fd -fps 30 H.264_FILE 21

22 UFG-10 MC sample program Audio UFG-10 MC will output one stereo audio data. UFG-10 MC audio driver is designed based on ALSA (Advanced Linux Sound Architecture). You can use arecord to record sound and aplay to play the recorded sound. List your audio device. arecord -l Record audio of capture card and play the sound instantly. arecord -D hw:1,0 -r c 2 -f S16_LE aplay (Usually hw:0,0 is your sound card device, is audio sampling frequency, it could be 48000, 44100, User should use V4L2_CID_AUDIO_SAMPLING_FREQ property to get correct value) 22

23 GPIO setting 12. GPIO SETTING #define V4L2_CID_GPIO_DIRECTION (V4L2_CID_BASE + 270) #define V4L2_CID_GPIO_DATA (V4L2_CID_BASE + 271) The property allows you to access the GPIO interface. The property V4L2_CID_GPIO_DIRECTION allows you to control its direction. Writing 1 to bit enables this pin as output pin. GPIO 1~16 is controlled by the first chipset in one board. SUPPORT VALUE: 0 ~ 1 INPUT ~ OUTPUT The property V4L2_CID_GPIO_DATA allows you to access GPIO s data. SUPPORT VALUE: 0 ~ 1 LOW ~ HIGH EXAMPLE 1 To define GPIO pins [1:16] to output and at high voltage Use /dev/video0 struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_GPIO_DIRECTION; s_ctrl.value = 0xFFFF; ioctl( fd, VIDIOC_S_CTRL, &s_ctrl ); s_ctrl.id = V4L2_CID_GPIO_DATA; s_ctrl.value = 0xFFFF; ioctl( fd, VIDIOC_S_CTRL, &s_ctrl ); EXAMPLE 2: To define even pin to output, odd pin to input Use /dev/video0 struct v4l2_control s_ctrl; s_ctrl.id = V4L2_CID_GPIO_DIRECTION; s_ctrl.value = 0xAAAA; ioctl( fd, VIDIOC_S_CTRL, &s_ctrl ); 23