Adapting Chronotron Speed Changer to Windows Phone 8.1

Transcription

1 Adapting Chronotron Speed Changer to Windows Phone 8.1

2 Meet the App Application Architecture Outlook After a Quick Code Review Universal Apps with Visual Studio Implementation Approach User Interface and Navigation Custom Controls ViewModel Native Components Project Ne10 Other Considerations Conclusion

3 Chronotron Speed Changer is an audio/video speed changer app. It allows you to manipulate tempo and pitch independently during playback, i.e. change the tempo of audio and video files without affecting pitch (time stretching) and/or change pitch without affecting tempo (key change).

4 The implementation is based on the MVVM pattern. The audio/video processing algorithms are implemented using native Media Foundation components. User Interface ViewModel Native MF Components

5 The easy Simple UI: one XAML page, one Settings flyout The challenging Three custom controls Native components

6 Each project in the solution is split in three: A Windows Store Project containing code specific to the Windows Store app A Windows Phone Project containing code specific to the Windows Phone app A Shared Project containing code shared across the above two projects

7 Specific or Shared code? User Interface and Navigation Custom Controls ViewModel Native Components Other

8 Implementation Approach: Specific Code Limited screen real state Major differences in navigation approach and UI behavior (e.g. FileOpenPicker) Some XAML code could be copied over from the Windows Store project, though

9 Implementation Approach: Shared Code Minor differences in XAML templates - not all theme resources are available Change the control templates to use other resources and styles Minor differences in call sequence e.g. on Windows Phone property binding may happen before OnApplyTemplate Program defensively to accommodate for both XAML implementations No #ifdef were required

10 Implementation Approach: Shared Code Written in C# Only one #ifdef construct was needed StorageFile.GetThumbnailAsync always fails on Windows Phone 8.1 Part of the application settings viewmodel is specific in order to allow different default settings on WP

11 Implementation Approach: Shared Code Compiling the native code for Windows Phone was relatively straightforward The actual processing algorithms were already developed in portable C++; x86-specific optimizations had been implemented as an extension Media Foundation is available on WP 8.1 A few memory management APIs aren't available on WP (e.g. CreateFileMappingFromApp), so a few #ifdefs were required

12 Real-time performance was the main area of work The good VC++ does a good job at vectorizing simple loops The not-so-good Memory constraints Solution: Limit memory-hungry functionality, like reverse playback Trigonometric functions in the MSVC Runtime Library are slow on ARM Solution: Use lookup tables and other techniques Video processing on the CPU is prohibitively slow Solution: Use DXGI for video processing (also for the benefit of the Windows Store app!) The Windows Store version relies heavily on x86 SIMD optimization, in particular for the FFT Solution: Rely on Project Ne10. What is it?

13 A library of well-known algorithms optimized for the ARM architecture Developed primarily for Android, but can be compiled with VC++ with minimal changes Free for non-commercial and commercial usage Available at Example: 32-bit floating point real FFT/IFFT runtime on Qualcomm Snapdragon 400, average over 10K calls: Number of Samples My own FFT written in C Ne10 FFT using NEON intrinsics ms 0.159ms ms 0.349ms The 10-12% improvement is very welcome considering the app performs up to 3 FFT per channel per frame!

14 The Windows Store does not support specifying a trial period for Windows Phone apps A trial mode offering limited functionality had to be implemented Testing Tested on a Nokia Lumia 630

15 The Windows Store app could be effectively used as starting point to develop the Windows Phone app Most of the code could be shared (with a few caveats), even native code! Meet the new app

16