Extracting Sound Information from High-speed Video Using Three-dimensional Shape Measurement Method

Size: px

Start display at page:

Download "Extracting Sound Information from High-speed Video Using Three-dimensional Shape Measurement Method"

Janel Grant
5 years ago
Views:

1 Extracting Sound Information from High-speed Video Using Three-dimensional Shape Measurement Method Yusei Yamanaka, Kohei Yatabe, Ayumi Nakamura, Yusuke Ikeda and Yasuhiro Oikawa Department of Intermedia Art and Science, Waseda University, Tokyo, Japan Paper Number: O1225 Abstract A dynamic three-dimensional shape measurement technique using a high-speed camera and a projector recovers vibrating object s shapes. Sound information of measured vibrating objects can be extracted from displacement of it s shape by this technique, through pattern projection, Fourier transform, filtering, inverse Fourier transform and phase unwrapping. It achieves full-field, direct and non-contact; those advantages are useful in vibration measurement for studying on characteristics of sound sources. In our experiment, usefulness of a dynamic three-dimensional shape measurement technique for vibration measurement was confirmed by measuring the vibrating loudspeaker s cone. The high-speed images were captured using resolution at 2 fps speed, and input signal to measured loudspeaker was a sine wave whose frequency is 5 Hz. As a result, the wave similar to input sine wave signal could be seen from recovered cone s vibration, and it s spectrum peak could be seen at the neighborhood of 5 Hz similar to the input signal. Index Terms Vibration measurement, Fourier transform profilometry, Phase unwrapping, Highspeed-camera. 1. Introduction It is important to investigate sound characteristic for noise source exploration, however sound is invisible and easy to spread. Therefore, it is necessary to measure vibration of a sound source object in itself for researching emitted sound. There are several methods for that, for example, using vibration pickup and laser-doppler velocimeter (LDV). Vibration pickup may affects measured vibration, because it is a contact method that attaches some sensors to measured objects. LDV is an equipment that can measure vibrations at each one-point of object s surface with noncontact. However, both methods require multipoint measurement to realize full-field measurement. On the other hand, near-field acoustical holography can measure full-field vibration as a non-contact method [1, 2]. It is a indirect method and has limitation of resolution. There is no method which achieves full-field, direct and non-contact observation. It is beneficial that a full-field vibration measurement method which is non-contact and direct is realized for studying on characteristics of sound sources. This is because nonstationary vibration can be measured without external influence at once thanks to these advantages. A dynamic three-dimensional shape measurement technique using a high-speed camera and a projector, as vibration measurement method for extracting sound information from vibrating objects, has possibility of fulfilling the above requirements [3, 4]. The method can obtain full three-dimensional information of vibration, including depth direction, using a single highspeed camera thanks to projection pattern. It utilizes one kinds of fringe projection method which called Fourier transform profilometry (FTP) which is introduced by Takeda and Mutoh. They project a Ronchi grating or sinusoidal grating onto the surface of object, records one-shot deformed fringe image, and recovers object s shape via the Fourier transform and threedimensional phase unwrapping [5]. Information of sound can be recovered from calculated displacements of each point of dynamic object s surface. Zhang et al. measured Loudspeaker s cone vibration for distortion measurement using a dynamic three-dimensional shape measurement technique with FTP and a highspeed camera [6,7]. Since their purpose was distortion measurement of a loudspeaker, they didn t focus on and mention frequency analyzing of vibration of measured loudspeaker s cone. In this paper, we recorded stational vibration of a loudspeaker s cone to confirm the validity of this technique, and analyzed including frequency analysis to confirm whether a dynamic three-dimensional shape measurement technique using a high-speed camera and a projector is suitable for vibration measurement. 2. Fringe projection method [3 5] A fringe projection method is used for threedimensional shape measurement of several objects. A fringe pattern projected on a measuring object s surface deforms depending on its shape. Since a threedimensional shape of a measuring object is obtained 12th Western Pacific Acoustics Conference 215. Edited by Lim Kian Meng Copyright c 216 WESPAC 215 Organisers :: Published by Research Publishing, Singapore ISBN: doi: 1.385/ O1225 3

2 by analyzing an one-shot image which grabs that deformed pattern, a vibration of a measuring object s surface can be captured as time variation of its displacement by recording changes of deformed patterns as movie with enough frame rate and analyzing. In this paper, we used the FTP which is the one kind of fringe projection method using sinusoidal grating as a fringe pattern Fourier Transform Profilometry Figure 1 shows an optical geometry of fringe projection method. A three-dimensional shape, as height distribution Z(x, y, t), is recovered from information of deformed fringe pattern. It requires a reference plane set vertically form an optical axis of a camera. Ca and Pr are the exit pupils of a camera and a projector, respectively. If Pr is at infinity, a ray E from the exit pupil is parallel to the optical axis PrO. A sinusoidal grating image on a reference plane is expressed as g T (x, y) = A n exp(i 2πnf x), (1) where f is a spatial frequency of the grating image. However, if Pr is at finite distance, a period of fringe pattern on a reference plane is not constant. A ray from Pr toward point A strikes point C on a reference plane. On the other hand, E toward point A strikes point B on the reference plane. Therefore, actual grating image on a reference plane grabbed by camera is expressed as Fourier series g (x, y) = r (x, y)a n exp [ i 2πnf {x + s (x)} ], (2) where s (x) = BC is a function of x and r (x, y) is a nonuniform distribution of refractivity on a reference plane. Equation (2) can be expressed as a phasemodulated signal g (x, y) = r (x, y)a n exp [ i{2πnf x + nφ (x)} ], also, where (3) φ (x) =2πf s (x) =2πf BC (4) is a phase shift due to a projection angle of a projector. On the other hand, when a measured object is projected, a ray passing through point C strikes point H Reference plane Pr G Pr Measuring object B C D H Camera S Ca Ca Fig. 1. Optical geometry of fringe projection method on the object s surface and is observed at point D on a reference plane through Ca. Hence, a grating image on the object s surface is expressed as g(x, y, t) = r(x, y, t) = r(x, y, t) where A O A n exp [ i 2πnf {x + s(x, y, t)} ] A n exp [ i{2πnf x + nφ(x, y, t)} ], (5) φ(x, y, t) =2πf s(x, y, t) =2πf BD (6) is a phase shift due to a height distribution Z(x, y, t) and a projection angle of a projector, and r(x, y, t) is a nonuniform distribution of refractivity on the object s surface. Complex signals which contain fundamental components at each frame can be obtained by FFT, filtering and IFFT, and ĝ(x, y, t) =A 1 r(x, y, t)exp [ i{2πf x + φ(x, y, t)} ]. (7) A complex signal on the reference plane is calculated by applying the same operation. ĝ (x, y) =A 1 r (x, y)exp [ i{2πf x + φ (x)} ]. (8) φ(x, y, t) and φ (x) can be calculated by Eq. (7) and Eq. (8), respectively. A phase unwrapping operation is basically required in actual processing whose details are described later. Δφ(x, y, t), a phase shift due to only a height distribution, can be calculated by φ (x) 31

3 Table 1. Experimental devices Camera MEMRECAM HX-3 (nac IMAGE TECHNOLOGY) EB1735W (EPSON) Louspeaker MSP1 (YAMAHA) High-speed camera [mm] [mm] and φ(x, y, t) Δφ(x, y, t) =φ(x, y, t) φ (x) =2πf (BD BC) =2πf CD. (9) Loudspeaker Noting that PrHCa CHD in Fig. 1, it is possible to write that Fig. 2. Experimental setup dz(x, y, t) CD = L Z(x, y, t). (1) From Eq. (9) and Eq. (1), L Δφ(x, y, t) Z(x, y, t) = Δφ(x, y, t) 2πf d, (11) Therefore, a three-dimensional shape of an object can be measured by recovering a height distribution from a phase of a sinusoidal grating image Phase unwrapping Since the phase calculation, φ (x) and φ(x, y, t), by a computer provides the principal values ranging from π to π, phase values have discontinuities with 2π phase jumps. Methods which remove phase jumps are called phase unwrapping procedure. The three-dimensional phase unwrapping can be applied to this paper, because a video has not only x- and y-directions but time-direction information. A phase difference between two neighboring pixels which have same image coordinates is less than π in time-direction, when frame rate is enough high and a time interval between two grabbed images is small. The phase difference between two neighboring frames is expressed as Δφ t (x, y) =φ(x, y, t) φ(x, y, t 1), (12) where Δφ t (x, y) is a phase difference along timedirection. The unwrapped phase in t = T can be calculated by using Δφ t (x, y) as T 1 Φ uw (x, y, T )=Φ uw (x, y, 1) + Δφ k (x, y), (13) k=1 where Φ uw (x, y, 1) is the criteria frame which is required applying two-dimensional phase unwrapping. Fig. 3. Recorded image of loudspeaker s cone, the projected pattern can be seen 3. Experiment and result Table 1 shows experimental devices. The measured object was a loudspeaker s cone as a known vibrating source. The input signal was a sine wave whose frequency is 5 Hz. Images were captured using resolution at 2 fps speed. The experimental setup of the high-speed camera, projector and measured object is shown in Fig. 2. The distance between the exit pupil of the projector and the high-speed camera was d = 5 mm. The white paper sticked on front side of the loudspeaker was a reference plane which was located at a distance of L = 576 mm. We recovered the three-dimensional shape of the loudspeaker s cone with recorded video, using the method that we introduced in Sec. 2. One of the grabbed images is shown in Fig. 3. The surface of the loudspeaker s cone is projected the sinusoidal grating whose spatial frequency was f =.33 line/mm. One of the unwrapped phase images is shown in Fig. 4. Figure 5 shows the unwrapped phase image we got by subtracting its liner component in or- 32

Fig. 7. Cross section of recovered cone s shape 1 Fig. 4. Unwrapped phase before removing liner component Displacement [mm].5.1 Time [s] Fig. 8.

Unwrapped phase after removing liner component Height [mm] -2 16 12 8 Position [mm] 4 4 16 12 8 Position [mm] Fig. 6.

The cone s shape can be seen, although there are unwrapping errors on the circumference. Figure 6 indi- -1-2 -3 Power [db] 5 1 Frequency [Hz] Fig. 9.

4 Fig. 7. Cross section of recovered cone s shape 1 Fig. 4. Unwrapped phase before removing liner component Displacement [mm].5.1 Time [s] Fig. 8. Waveform of the displacement of the center point of recovered loudspeaker s cone Fig. 5. Unwrapped phase after removing liner component Height [mm] Position [mm] Position [mm] Fig. 6. Recovered three-dimensional shape of loudspeaker s cone through process of averaging the values on the circumference for smoothing der to remove the effect of orientation of the camera. The cone s shape can be seen, although there are unwrapping errors on the circumference. Figure 6 indi Power [db] 5 1 Frequency [Hz] Fig. 9. Power spectrum of recovered cone s vibration cates the recovered three-dimensional shape of measured loudspeaker s cone. It was calculated from Fig. 5 through process of averaging the values on the circumference for smoothing. Figure 7 shows the sectional view of the recovered shape. The displacement of the center point of measured loudspeaker s cone, as time wave, is shown in Fig. 8. The wave similar to input sine wave signal can be seen. Figure 9 shows frequency component. The spectrum peak can be seen at 33

5 the neighborhood of 5 Hz. According to the above results, it is confirmed that sound information, including frequency, can be extracted from the high-speed video by using the threedimensional shape measurement method. 4. Conclusion In order to realize the method which achieves fullfield, direct and non-contact observation, we proposed the method using a dynamic three-dimensional shape measurement technique with high-speed camera and a projector. we also confirmed its usefulness by recording stationary vibration of a loudspeaker s cone and analyzing it including frequency. As a result, the same wave as input signal was obtained. It is expected that this technique will be one of the vibration measurement for studying on characteristics of sound sources. As future work, we will evaluate measurement accuracy and characteristic of the method by comparative experiments using LDV, and apply this technique to actual noise source exploration. References [1] J. D. Maynard, E. G. Williams, and Y. Lee, Theory of generalized holography and the development of NAH, J. Acoust. Soc. Am. Vol.78, 1395, [2] W. A. Veronesi, and J. D. Maynard, Holographic reconstruction algorithms and computer implementation, J. Acoust. Soc. Am. Vol.81, 137, [3] X. Su, W. Chen, Q. Zhang and Y. Chao, Dynamic three-dimensional shape measurement method based on FTP, Opt. Lasers Eng., Vol.36, No.1, pp.49 64, Jul. 21. [4] X. Su and Q. Zhang, Dynamic threedimensional shape measurement method: A review, Opt. Lasers Eng., Vol.48, No.2, pp , Feb. 21. [5] M. Takeda and K. Mutoh, Fourier transform profilometry for the automatic measurement of three-dimensional object shapes, Appl Opt., Vol.22, No.24, pp , [6] Q. Zhang, X. Su and L. Xiang, Whole-field Vibration Analysis of a Woofer s Cone using a High-speed Camera, Proc. SPIE 6279, 6279O, 5 pages, 27. [7] Q. Zhang, Y. Liu and P. Lehtonen, Shape measurement and vibration analysis of moving speaker cone, Proc. SPIE 9234, 9234T, 6 pages,

Dynamic three-dimensional sensing for specular surface with monoscopic fringe reflectometry

Dynamic three-dimensional sensing for specular surface with monoscopic fringe reflectometry Lei Huang,* Chi Seng Ng, and Anand Krishna Asundi School of Mechanical and Aerospace Engineering, Nanyang Technological