IoT Sensor Connectivity and Processing with Ultra-Low Power, Small Form-Factor FPGAs

Similar documents
THE INDUSTRY CASE FOR DISTRIBUTED HETEROGENEOUS PROCESSING

Accelerating Implementation of Low Power Artificial Intelligence at the Edge

VIDEO BRIDGING SOLUTION PROMISES NEW LEVEL OF DESIGN FLEXIBILITY AND INNOVATION

NEW USE CASES HIGHLIGHT CROSSLINK S BROAD APPLICABILITY

NEW APPROACHES TO HARDWARE ACCELERATION USING ULTRA LOW DENSITY FPGAs

Solving Today s Interface Challenge With Ultra-Low-Density FPGA Bridging Solutions

DesignWare IP for IoT SoC Designs

ice40 UltraPlus Image Sensor Elliptical Crypto Engine (ECC) Demo - Radiant Software User Guide

TINY System Ultra-Low Power Sensor Hub for Always-on Context Features

USING LOW COST, NON-VOLATILE PLDs IN SYSTEM APPLICATIONS

Cypress PSoC 6 Microcontrollers

Advantages of MIPI Interfaces in IoT Applications

HEXIWEAR COMPLETE IOT DEVELOPMENT SOLUTION

ENABLING MOBILE INTERFACE BRIDGING IN ADAS AND INFOTAINMENT APPLICATIONS

Renesas Synergy MCUs Build a Foundation for Groundbreaking Integrated Embedded Platform Development

ice40 UltraPlus Display Frame Buffer User Guide

THE GROWING USE OF PROGRAMMABLE LOGIC DEVICES IN MOBILE HANDSETS

BASICS OF THE RENESAS SYNERGY PLATFORM

Cypress PSoC 4 Microcontrollers

Cypress PSoC 4 Microcontrollers

ice40 UltraPlus 8:1 Mic Aggregation Demo User Guide

Intelop. *As new IP blocks become available, please contact the factory for the latest updated info.

D Demonstration of disturbance recording functions for PQ monitoring

Open Sesame. Grant Apodaca Jeffrey Bolin Eric Taba Richie Agpaoa Evin Sellin

Simplify System Complexity

New STM32WB Series MCU with Built-in BLE 5 and IEEE

Key Phrase Detection Using Compact CNN Accelerator IP Reference Design

MDP Based Key Phrase Detection Demonstration User Guide

STM32L4+ MCU series Excellence in ultra-low-power with more performance

The S6000 Family of Processors

User Manual Rev. 0. Freescale Semiconductor Inc. FRDMKL02ZUM

2:1 HDMI Switch Demo User Guide

Ubiquitous IoT Perspectives The Power of Connected Sensors and Actuators

Raspberry Pi - I/O Interfaces

ARDUINO MEGA INTRODUCTION

EMBEDDED SYSTEMS WITH ROBOTICS AND SENSORS USING ERLANG

Simplify System Complexity

MYD-IMX28X Development Board

Select Your Sensor. Select Your Interface. Get The Perfect Fit

T he key to building a presence in a new market

MAX II CPLD Applications Brochure

AT-501 Cortex-A5 System On Module Product Brief

MDP Based Face Detection Demonstration User Guide

Qualcomm Wi-Fi Connectivity Selector Guide

New MONITOUCH V8 Series of Programmable Operation Displays

Full Linux on FPGA. Sven Gregori

HomePlug AV2 Technology

Microsemi Secured Connectivity FPGAs

MYD-IMX28X Development Board

Hugo Cunha. Senior Firmware Developer Globaltronics

White Paper. The advantages of using a combination of DSP s and FPGA s. Version: 1.0. Author: Louis N. Bélanger. Date: May, 2004.

ECE 1160/2160 Embedded Systems Design. Midterm Review. Wei Gao. ECE 1160/2160 Embedded Systems Design

Embedded Vision Solutions

Designing for Low Power with Programmable System Solutions Dr. Yankin Tanurhan, Vice President, System Solutions and Advanced Applications

ST Enables The Wireless Charging World

Title: Using low-power dual-port for inter processor communication in next generation mobile handsets

Let s first take a look at power consumption and its relationship to voltage and frequency. The equation for power consumption of the MCU as it

Overcoming Hurdles in Wearable Device Designs. John Logan Atmel

Zatara Series ARM ASSP High-Performance 32-bit Solution for Secure Transactions

ARDUINO PRIMO. Code: A000135

SmartBond DA Smallest, lowest power and most integrated Bluetooth 5 SoC. Applications DA14585

Implementing the Top Five Control-Path Applications with Low-Cost, Low-Power CPLDs

STM32F429 Overview. Steve Miller STMicroelectronics, MMS Applications Team October 26 th 2015

StreetWise IoT Solutions

The challenge with IoT

Lesson 6 Intel Galileo and Edison Prototype Development Platforms. Chapter-8 L06: "Internet of Things ", Raj Kamal, Publs.: McGraw-Hill Education

IO Aggregation/De-Aggregation in Mobile & Mobile Influenced Systems to Improve Routing Congestion MIPI Alliance, Inc.

MIPI Alliance Introduction & MIPI Camera Serial Interface Overview

Edge Computing and the Next Generation of IoT Sensors. Alex Raimondi

MYD-JA5D2X Development Board

IoT Market: Three Classes of Devices

Typical Applications: GHz Bluetooth low energy systems - Proprietary 2.4 GHz systems - Sports and leisure equipment - Mobile phone accessories,

Quick Start Guide. STM32 ODE Function Pack for connecting 6LoWPAN IoT Nodes to smartphone through a BLE interface (FP-NET-6LPBLE1)

LOW LATENCY DATA DISTRIBUTION IN CAPITAL MARKETS: GETTING IT RIGHT

New STM32 F7 Series. World s 1 st to market, ARM Cortex -M7 based 32-bit MCU

Intel CoFluent Studio in Digital Imaging

Unlocking the Potential of Your Microcontroller

PBLN52832 DataSheet V Copyright c 2017 Prochild.

User Manual Rev. 0. Freescale Semiconductor Inc. FRDMKL02ZUM

SD Express Cards with PCIe and NVMeTM Interfaces

MT2 Introduction Embedded Systems. MT2.1 Mechatronic systems

STM32F7 series ARM Cortex -M7 powered Releasing your creativity

ESPRESSIF SYSTEMS. The Complete IoT Solutions Provider

Application Strategic Focus

MiCOKit-3166 Development Kit Hardware Manual

LBAT90USB162 Atmel. LBAT90USB162 Development Board User s Manual

Discovering Computers 2012

Xynergy It really makes the difference!

Five Ways to Build Flexibility into Industrial Applications with FPGAs

Zynq-7000 All Programmable SoC Product Overview

MeshConnect. Voice over

Reminder. Course project team forming deadline. Course project ideas. Friday 9/8 11:59pm You will be randomly assigned to a team after the deadline

If It s Electronic, It Needs a Clock

Connectivity and Audio

Developing a simple UVC device based on i.mx RT1050

Field Programmable Gate Array (FPGA) Devices

SBAT90USB162 Atmel. SBAT90USB162 Development Board User s Manual

ARDUINO M0 PRO Code: A000111

STM32F7 series ARM Cortex -M7 powered Releasing your creativity

SMART DELTA: SECURE OVER-THE-AIR (OTA) TECHNOLOGY FOR REMOTE MANAGEMENT OF IOT SOLUTIONS

Transcription:

IoT Sensor Connectivity and Processing with Ultra-Low Power, Small Form-Factor FPGAs Applications Run the Gamut from Sensor Buffers, Signal Aggregation to Embedded Camera, Audio and Display Solutions A Lattice Semiconductor White Paper March 2018 Lattice Semiconductor 111 5 th Ave., Suite 700 Portland, Oregon 97204 USA Telephone: (503) 268-8000 www.latticesemi.com

Introduction The rising number of sensors being deployed into smart homes, smart factories and smart cities are posing increased challenges for designers. On one hand, these new IoT applications require improved connectivity to efficiently communicate across a wide array of new and legacy interfaces and protocols. On the other hand, as users demand higher levels of intelligence, designers will need more computational resources for realtime processing of data close to the sensors. What do designers need to address these challenges? In a growing number of applications, they need a programmable solution that combines flexible I/Os, a configurable architecture, ultra-low power operation in a small form-factor, and priced for high-volume production. They need a solution that offers the logic resources of a high performance, best-in-class co-processor capable of highly parallel computing while simultaneously adding high levels of connectivity and support for a wide range of I/O standards and protocols. Finally, designers need a programmable solution that is production-ready for today s systems and is supported by intellectual properties (IPs), reference designs, development boards and software tools needed to get to market faster. In many of these cases, Lattice s ice40 UltraPlus family of FPGAs offers the features best suited to address these needs. Designed specifically to enable designers to quickly build energy-efficient solutions that meet the processing demands of next-generation Internet of Things (IoT), industrial and automotive products, these devices offer eight times the memory and twice the number of DSPs as previous generation products. Despite adding extensive functionality, including multiple 16 x 16 multiplier blocks, the ice40 UltraPlus device only draws 75 µa of static current. Available in densities of 2800 or 5280 LUTs, the ice40 UltraPlus FPGA features 80 Kb or 120 Kb of block RAM, four or eight 16 x 16 Multiply or 16-bit Accumulate DSP blocks for signal processing and 1 Mb of RAM in four 256Kb single port memory blocks. Fig. 1: Potential ice40 UltraPlus applications range from sensor fusion and signal aggregation, to bridging and graphics acceleration

The ice40 UltraPlus device is the ideal foundation for designers building always-on sensor buffers and compute-heavy applications because of its extensive FPGA fabric for custom logic, large amounts of low power non-volatile configuration memory for instant-on applications, and up to 8 multiply/accumulate blocks for signal processing. It also offers a highly attractive blueprint for bridging, buffering and display applications for next-generation mobile and industrial solutions. To help simplify signal aggregation and optimize layout, the ice40 UltraPlus FPGA adds programmable I/Os, a high-bandwidth I3C interface for sensors, and integrated oscillators to minimize power and bill of materials (BOM) cost. Finally, to meet the aggressive physical constraints of current generation consumer and industrial applications, this family of devices supports packages as small as 2.2mm x 2.6 mm, as well as low-cost QFNs (Quad Flat No- Leads). Solving IoT Sensor Connectivity Challenges The rapidly rising number of sensors in today s systems is driving the demand for new and innovative sensor fusion and buffering solutions. Designers developing a wide range of factory automation, industrial test equipment and security camera applications need to capture, aggregate, encrypt, buffer, pre-process and timestamp data. The ice40 UltraPlus extensive new features are designed to serve this purpose. The 1-mbit of on-chip SRAM allows the system to buffer data longer in low power states. The higher number of on-chip DSP blocks helps designers use the ice40 UltraPlus device as a co-processor to the application processor (AP) allowing it to process and analyze data while the system processor is off-line. Fig.2 represents an example of how the ice40 UltraPlus FPGA can resolve system bottlenecks. The upper block diagram depicts a traditional implementation where continuous sensor polling by the system processor drives up overhead. The lower block diagram illustrates how the introduction of an ice40 UltraPlus FPGA between the system processor and the sensor (in this example a gyroscope) can dramatically reduce computational demands on the system processor. Here, the ultra-low density FPGA performs auto polling and data filtering, and collects data in a first-in first-out (FIFO) buffer. This approach conserves power by only waking the processor at infrequent intervals.

Fig. 2: In this example, the use of the ice40 UltraPlus FPGA can significantly reduce overhead on the baseband processor Another common use for the ice40 UltraPlus FPGA lies in sensor data aggregation and processing. By using the ice40 UltraPlus device to aggregate multiple data streams or bridge from slower to faster interfaces, designers can simplify PCB layout and relieve system bottlenecks. The ice40 UltraPlus FPGA can also enable smart processing, by including concurrent data capture and creating flags based on data from multiple sensors. Issues like this crop up frequently in product design. The evolution of always-on monitoring and the proliferation of a new generation of low-cost sensors presents new challenges from a layout perspective. Every sensor must communicate with the AP. Typically, these devices employ a variety of interfaces. In Fig. 3, for example, data coming into the AP from five different sensors creates a bottleneck. By bridging the signals from the five sensors to a low-density FPGA and using the FPGA to aggregate the incoming data and transmit it to the AP via SPI or I3C, designers can eliminate the bottleneck and improve system performance.

Fig. 3: Bridging slow I 2 C devices to a higher speed bus can eliminate system bottlenecks Fig. 4 illustrates the advantages of the embedded SPI block on the ice40 UltraPlus device. Here, designers can use on-chip SPI resources to continue to run the system at low power while all of the sensors continue to collect data.

Fig 4: A low density FPGA can support the operation of multiple SPI peripherals One of the more common applications for the ice40 UltraPlus FPGA is signal aggregation and deaggregation. Many systems use multiple interfaces, including I 2 C, SPI, GPIO and I 2 S, to collect data. In a growing number of applications, such as 2:1 systems, laptops, systems with multiple PCBs, designers often need to minimize the number of wires routed between boards and reduce the number of pins on external connectors. In some cases, designers may have to manage up to 40 signals coming from the device s sensors. The medical instrument market offers an excellent example. Designers building instruments are constantly seeking new ways to reduce pin-count between systems so they can employ smaller connectors that are less prone to damage or contamination by water or dust. In some cases, the resources in the ice40 UltraPlus devices can be used to simplify PCB layout. The diagram below illustrates how designers can aggregate data in the AP

to clean up routing and then use the FPGA s flexible I/Os to deaggregate and meet the application s needs. This type of solution can be implemented in as little as 1500 LUTs. Fig. 5: This Lattice reference design can save pins in the AP by aggregating data Bridge to MIPI DSI Graphics acceleration has increasingly become a high priority in white goods and virtually any system using displays. Today, designers can use the FPGAs to bridge from any of a variety of legacy busses and drive a MIPI DSI display without any additional circuitry. Moreover, designers can load images directly into the SRAM embedded in the FPGA for frame buffering and graphics acceleration. To simplify this task, Lattice offers reference designs for bridging from SPI to MIPI DSI and for delivering 1-layer Gfx acceleration. Bridge Image Sensors to SPI or UART As embedded cameras find their way into a growing number of applications, designers are looking for more efficient ways to bridge camera output to popular system interfaces. With its customizable I/Os, designers of camera-based security systems and factory automation systems can use the ice40 UltraPlus devices to bridge from QVGA image sensors to interfaces such as SPI or UART using minimal resources. Improving Audio Performance Over the last few years, designers have begun using multiple microphones in personal assistant devices for audio beamforming. These microphone arrays can be used as a spatial filter which can extract a signal from a specific direction and reduce the contamination of signals from other directions. Pulse Density Modulation (PDM) is a popular way to deliver audio from microphones to the signal or application processor. This technology brings the benefit of digital, including low noise and freedom from interfering signals, at a low cost. However, signal processors often do not feature enough PDM inputs to support multiple microphone arrays.

To address this problem, developers are turning to microphone aggregation techniques. Lattice has developed a reference design that aggregates up to eight microphones to the processor over the I 2 S bus. Designed for any 48 KHz PDM microphone, the demo transfers data from up to eight PDM microphones on a single data wire using an I 2 S 8- ch aggregation format. Today, this technology is being used in a rapidly growing number of voice-activated systems such as the personal assistant devices, automotive audio systems, remote controls, AV receivers and headphones. LoRa Transmission Along with always-on sensors and aggregation of data comes escalating processing and buffering requirements. In many applications designers need a low power wireless solution, such as LoRa (Long Range) to connect edge devices to the cloud. By allowing users to continuously collect data and analyze it, LoRa offers users an attractive design option for low-power, wireless connectivity. IoT applications can take advantage of sensors connected to a LoRa device to monitor and track assets, measure resource consumption, monitor temperature, pressure, pollution levels and other environmental issues. The technology has been used in a variety of applications including cars, street lights, industrial equipment and homes. Designers can use the ice40 UltraPlus FPGA to implement a LoRa-compliant device featuring a RISC-V processor and accelerators, or a device which supports multiple connections through flexible I/Os and integrated peripheral blocks. LoRa can also be used with machine learning and artificial intelligence applications to create smart edge devices. Fig. 6: LoRa offers designers a low power solution for always-on monitoring and data aggregation

Real-time, Low-Power, Sensor Data Processing At the network edge, there is also demand for highly compact, power-efficient solutions that can deliver additional computational resources using neural networks. One example is the development of solutions that can deliver always-on facial detection capabilities for applications like smart door bells, security systems, factory automation and logo recognition. The block diagram below illustrates the implementation of an always-on facial detection system using a 5K LUTs ice40 UltraPlus FPGA. In this application, designers took an untrained neural network model and trained it using a set of 600,000 images. Once training was complete, the model was converted to weights and activations, and loaded into the FPGA. When a new image is presented to the image sensor, the network compares the new image to its existing data set and decides whether the new image is a human face. The same design techniques can be used to detect other objects such as cars, dogs, cats, or a specific sign or license plate. This application uses 4670 LUTs, four DSP blocks, 30x 4 Kbits of BRAM, four times 32 Kbytes of SPRAM, and draws < 5mW of power. A RISC-V soft processor with accelerators implemented in the FPGA accelerates processing. Fig. 7: Facial detection subsystems can now be implemented in less than 5K LUTs Motor Control Motor control applications in the industrial setting can benefit from FPGA-based motor control drives. In high-speed, high-torque applications that involve dynamic load changes, these drives typically require a feedback loop where a sensor can determine the position of the motor, what speed it is operating at, and how much torque it is applying. Motor control devices are normally used in applications requiring high precision positioning, such as robots, instrumentation, machines and process control. An FPGA with integrated DSP blocks can perform the feedback loop functions needed in these applications. The ice40 UltraPlus family of low density FPGAs features embedded DSP blocks, offering an efficient solution for designers to deliver more power and control to electric motors at a lower cost. Additionally, by using FPGAs with custom

IP, system designers can easily create motor control solutions specific to their needs. The parallel architecture of the FPGA can ensure deterministic behavior by simply replicating the functional blocks during the boot process. By turning to FPGAs, designers can now deliver more compact, integrated control solutions that offer easier feature integration, deterministic performance, improved control bandwidth, reduced component count and higher reliability. Sensor Encryption The embedded capabilities of the ice40 UltraPlus FPGA, particularly its parallel architecture and its ability to interface to a wide array of sensors, can also be used to create more secure IoT edge devices. To address this growing need, Lattice has developed a demo that implements a soft Elliptic Curve Cryptography (ECC) block to secure data targeted for IoT devices. Implemented in the ice40 UltraPlus FPGAs, the block uses 128 encryption, generates private and public keys, and stores them in secure memory. As a result, data captured by the FPGA from sensors can be securely transferred to a server or the cloud. Fig. 8: This encryption example showcases how the ice40 UltraPlus device can ensure safe transmission of data from sensors to the cloud In this application the FPGA takes image data from a sensor, encrypts it using AES 128 encryption and sends it out. The host then decrypts it using keys embedded in the demo.

Design Resources To simplify and accelerate the development with ice40 UltraPlus devices for broad market, low power applications Lattice is offering a wide range of design tools and support. Its most recent addition is the Lattice Radiant software, an FPGA software package that enables predictable design convergence with unmatched ease-of-use. The software s use of a unified database, design constraint flow, and timing analysis ensures predictability of convergence. And use of the industry standard Synopsys Design Constraint (SDC) language ensures maximum interoperability. Lattice Radiant s robust design environment maximizes design predictability with a long list of user-friendly functions, including an updated GUI that offers simple, intuitive and efficient user operation with a modern look and a new design constraint editor that simplifies both logical and physical design constraint editing. The new software also features a new messaging console, filtering functions and adds physical to logical design implementation probing. A comprehensive development ecosystem adds industry standard IEEE 1735 encryption support for IP protection and easy access to IP cores via an IP catalog. Development Boards Fig. 9: Lattice s ice40 UltraPlus breakout board Designers that need to quickly evaluate the key connectivity features of the ice40 UltraPlus FPGA should consider Lattice s ice40 UltraPlus breakout board. Featuring the ice40 UltraPlus 5K FPGA in a 48-pin QFN package, the board is programmed and powered via a USB mini cable, and comes pre-loaded with an RGB LED demonstration code. Designers can use Lattice s Diamond Programmer software to re-program the onboard SPI flash with customized code.

Fig 10: Gnarly Grey s UPduino board Designers that want to take advantage of the popular Arduino Nano and Pro Mini boards can use the UPduino board developed and supported by Lattice partner, Gnarly Grey. The board simplifies integration by plugging right into the Arduino board. The board features an ice40 UltraPlus FPGA with 5.3K LUTs, 1 Mbits SPRAM, 120 Kbits DPRAM and eight multipliers. It also features 34 GPIO on 0.1 inch headers. Mobile Development Platform For designers that need to evaluate crucial connectivity features of their ice40 UltraPlus FPGA designs, Lattice s ice40 UltraPlus Mobile Development Platform (MDP) offers a quicker path to rapid prototyping. The platform supports virtually all standard mobile interfaces, including MIPI DSI, MIPI CSI-2, I 2 C and UART. Designers can also use the MDP to evaluate processing capabilities using multiple DSPs integrated RAM and the ice40 UltraPlus FPGA fabric. The MDP s features include a MIPI DSI interface up to 108 Mbps, four microphone bridging (two I 2 C and two PDM microphones), compass (LSM303), pressure (BMP180), gyro (LSM330) sensors and an accelerometer (LIS2D12). The platform also adds a 640 x 480 (OVM7692) image sensor and a BLE module to transfer captured data wirelessly from the ice40 UltraPlus device. The FPGA can be programmed via an on-board SPI Flash or the USB port. Fig. 11: Lattice s ice40 UltraPlus Mobile Development Platform Lattice also offers a wide range of demos, reference designs and IP blocks for the ice40 UltraPlus products to simplify prototyping and accelerate product development.

Conclusion With the explosion of IoT sensors enabling smart homes, cities, and factories, the demand for a new generation of ultra-low power, small-footprint FPGAs is growing. In this new network landscape, designers building a variety of solutions, ranging from sensor fusion/buffering and signal aggregation to embedded camera, audio and display applications, need a production-ready, programmable device that offers flexible I/Os, a configurable architecture and low-power operation in a highly compact footprint. Designers can look to Lattice s ice40 UltraPlus FPGA, for its highly unique characteristics to solve these IoT sensor connectivity and processing challenges. Accelerate your ice40 UltraPlus designs with our new development board: http://www.latticesemi.com/products/developmentboardsandkits/ice40ultraplusbreakoutboard