A software platform to support dynamically reconfigurable Systems-on-Chip under the GNU/Linux operating system

Transcription

1 POLITECNICO DI MILANO Facoltà di Ingegneria dell Informazione Corso di laurea in Ingegneria Informatica A software platform to support dynamically reconfigurable Systems-on-Chip under the GNU/Linux operating system Relatore: Prof. Fabrizio FERRANDI Correlatore: Ing. Marco Domenico SANTAMBROGIO Tesi di laurea di Alberto DONATO Matr Anno Accademico 2004/2005

2 To my family

3 Riassunto della tesi I sistemi embedded sono attualmente molto diffusi in diversi ambiti operativi, poiché offrono la possibilità di creare architetture hardware/software specifiche anche molto complesse per particolari esigenze, mantenendo tuttavia dimensioni fisiche molto ridotte. L avvento dei dispositivi riconfigurabili ha contribuito notevolmente a questa diffusione, poiché estende ancora di più le possibilità offerte da questo tipo di sistemi. I componenti riconfigurabili oggi più utilizzati sono le FPGA, Field Programmable Gate Arrays, basate su RAM che stanno progressivamente prendendo il posto degli ASIC (Application Specific Integrated Circuit), processori progettati ad-hoc. Si tratta di circuiti la cui logica e le cui interconnessioni possono essere riprogrammate elettricamente in tempi brevi, consentendo la realizzazione di uno svariato numero di circuiti elettronici senza la necessità di sostituire fisicamente un componente con un altro. Le FPGA possono essere utilizzate sia come elementi all interno di un sistema più complesso, agendo da coprocessori per svolgere computazioni intensive specifiche, sia con un ruolo primario nell architettura hardware, implementando al loro inerno le componenti principali del sistema (ad esempio CPU, bus e interfacce verso periferiche esterne). Una caratteristica rilevante di alcune famiglie delle FPGA di ultima generazione è la possibilità di effettuare una riconfigurazione parziale dinamica, ossia che coinvolge solo una parte dell area del componente, senza interrompere il normale funzionamento del resto del sistema. Questa capacità introduce grandi potenzialità nella creazione di sistemi embedded in quanto permette di ridurre le risorse hardware fisicamente presenti, configurando di volta in volta sull FPGA quelle che risultino necessarie in un determinato momento. Un altro vantaggio della riconfigurabilità parziale è dato dal fatto che sistemi di questo tipo mantengono una grande flessibilità anche dal lato hardware anche dopo essere stati messi in produzione, per cui possono essere facilmente adattati a cambiamenti nelle specifiche, nell ambiente di utilizzo e nei protocolli applicativi. iii

4 I sistemi riconfigurabili, o RSoC (Reconfigurable System-on-Chip), includono spesso anche uno o più processori general purpose. In questo caso è presente anche una componente software, che può essere costituita da un programma sviluppato ad-hoc che implementa sia parte delle funzionalità proprie dello scopo per cui viene realizzato il sistema, sia funzioni di gestione delle componenti hardware. In sistemi di una certa complessità, che presentano un numero elevato di moduli hardware, l utilizzo di un semplice eseguibile stand-alone può risultare alquanto limitante a causa della scarsa flessibilità e della necessità di gestire insieme diversi elementi a diversi livelli, quali le risorse hardware e la logica applicativa. Per questi motivi in casi del genere si ricorre all uso di un sistema operativo, in grado di gestire indipendentemente la piattaforma hardware e di fornire accesso alle risorse alla componente software, che in questo caso viene realizzata come uno o più processi in esecuzione all interno del sistema operativo. Un sistema operativo completo offre funzionalità avanzate, che includono tra l altro scheduling e gestione dei processi e della loro comunicazione. Questo tipo di approccio permette una maggiore riusabilità, in quanto non è necessario sviluppare da zero la componente software per ogni nuovo sistema in sviluppo, ma si dispone di una base comune, costituita dal sistema operativo, su cui appoggiare il codice specifico dell applicazione. Gli attuali sistemi operativi, tuttavia, non offrono nativamente supporto alla riconfigurabilità dinamica all interno dei sistemi embedded, che consenta di gestire in modo del tutto indipendente la riconfigurazione di parti dell FPGA e l utilizzo delle componenti hardware di volta in volta configurate su di essa. Scopo di questo lavoro di tesi è la definizione e l implementazione di una piattaforma hardware/software basata su sistema operativo che supporti la riconfigurabilità parziale dinamica e permetta di sfruttare le componenti hardware mappate su FPGA. La base di partenza dal lato software è il sistema operativo GNU/LINUX [Tor91], ed in particolare µclinux [Arc04], un port del kernel per processori che non dispongono di Memory Management Unit (MMU), molto utilizzati in ambienti embedded. Questa scelta è stata dettata da diversi fattori, tra cui il gran numero di piattaforme e di periferiche supportate, la disponibilità del codice sorgente e la struttura modulare del kernel, che lo rende flessibile e particolarmente adatto a sistemi con componenti hardware variabili. iv

5 Il primo passo del lavoro è consistito nella definizione di un architettura hardware atta a supportare la riconfigurazione parziale module-based, ossia riconfigurazione di aree dell FPGA su cui sono mappati interi componenti, quali interfacce verso periferiche o coprocessori. L architettura di riferimento è quella di Caronte, descritta in [San04a; San04b; FSS05], e può essere suddivisa in due parti: una fissa ed una riconfigurabile. La prima consiste degli elementi sempre presenti nel sistema, ad esempio processore, bus, memoria RAM, controllore per la riconfigurazione ed, eventualmente, altre periferiche come memoria Flash o Ethernet controller. La seconda parte è data dai moduli riconfigurabili (detti anche BlackBox) presenti all interno del sistema, ossia da un certo numero di componenti hardware dinamicamente inseribili e rimovibili dal sistema. Il controllore della riconfigurazione è un componente hardware integrato nell FPGA in grado di svolgere fisicamente la riconfigurazione parziale, modificando lo stato delle celle che la compongono. Questo elemento non è strettamente necessario, in quanto sarebbe anche possibile effettuare configurazioni dall esterno del sistema, demandando il controllo del processo a terze parti (ad esempio un PC). La presenza di questo componente, tuttavia, permette di realizzare un sistema completamente indipendente, in cui la gestione dei moduli hardware viene effettuata direttamente da uno o più processi software all interno del sistema. Nella metodologia proposta (Capitolo 3), diversamente da quella di Caronte, i moduli configurabili non sono visti come implementazione in hardware della logica applicativa che viene partizionata, ma sono componenti (chiamate anche IP-Core) che possono essere utilizzati da qualunque processo presente nel sistema. Caronte si basa su una componente software formata da un unico eseguibile che realizza funzioni di controllore della riconfigurazione e di scheduling dell allocazione delle BlackBox; non viene utilizzato un sistema operativo, ma tutta la gestione dell hardware è svolta direttamente dal programma stesso. L approccio seguito in questo lavoro di tesi, invece, è quello di gestire i componenti hardware mappati su FPGA mediante un sistema operativo, consentendo di aggiungere e rimuovere dinamicamente gli IP-Core, secondo uno schema simile al Plug-and-Play utilizzato da molti standard di connessione delle periferiche, quali il PCI o l USB. L architettura software proposta offre alcuni contributi innovativi alla creazione di sistemi embedded riconfigurabili basati su sistema operativo: supporto al controllore interno per la riconfigurazione. L estensione del kernel del sistema operativo in tal senso permette di gestire la riconfigurazione di v

6 singole porzioni dell FPGA, sia al fine di effettuare piccoli cambiamenti di configurazione delle componenti mappate (small-bits manipulation), sia per aggiungere o rimuovere interi componenti (module-based reconfiguration). I dati per la riconfigurazione vengono estratti dai normali bitstream parziali generati dai tool di sviluppo hardware. supporto ai componenti riconfigurabili. Per poter sfruttare i componenti mappati dinamicamente su FPGA all interno di applicazioni che girano a livello utente è necessario che il sistema operativo faccia da tramite nella comunicazione tra le due parti. Per questo motivo occorre che quest ultimo includa dei driver per gestire gli IP-Core, in modo simile a quanto accade con le normali periferiche del sistema. L architettura software proposta permette di caricare dinamicamente i driver necessari all utilizzo dei componenti riconfigurabili quando essi vengono aggiunti al sistema, fornendo allo stesso tempo un interfaccia di programmazione per lo sviluppo dei driver stessi semplificata rispetto a quella del kernel. Per quanto riguarda la gestione del controllore per la riconfigurazione, si rende necessaria la creazione di un driver per il sistema operativo che permetta di effettuare operazioni di lettura e scrittura per poter inviare all hardware i dati relativi alla riconfigurazione e per poter leggere eventuali informazioni circa lo stato dell FPGA. Finalità primaria del driver è anche quella di fornire un interfaccia di accesso semplice ai processi per poter accedere al componente, mascherando i dettagli fisici dell hardware sottostante. Questo compito è facilitato dal meccanismo standard utilizzato dal kernel di LINUX per l accesso alle periferiche, che consiste nel fornire ai processi (e quindi anche agli utenti del sistema) una visione delle periferiche mediante speciali file, chiamati device node. Lettura, scrittura ed esecuzione di apposite funzioni su questo tipo di file in reatà provocano l accesso alla periferica. Poiché un driver può gestire diverse periferiche dello stesso tipo, è possibile prevedere diversi scenari di utilizzo di riconfigurazione con controllori multipli, ad esempio qualora ne fossero disponibili più di uno sullo stesso chip, oppure si abbiano diverse FPGA, ciascuna con un proprio controllore. La gestione dei componenti riconfigurabili presenta aspetti più complessi rispetto a quella del controllore. In questo caso, infatti si vuole creare un infrastruttura che supporti aggiunta e rimozione dinamica dell hardware, per cui è necessario riconoscere il tipo di periferica che viene configurato e caricare il driver adatto alla sua gestione. Per ottenere questo scopo si propone l utilizzo di un gestore dei driver, chiamato IP-Core Manager, IPCM, in grado di ricevere vi

7 informazioni dall hardware riguardo al tipo di componenete mappato e ai suoi parametri di funzionamento (in particolare lo spazio di indirizzamento per la comunicazione) e di caricare i driver specifici per il tipo di periferica. L IPCM può essere visto come una specie di middleware tra l hardware ed il sistema operativo, che gestisce l interfacciamento tra di essi, riservando e liberando in modo opportuno le risorse necessarie e reindirizzando le chiamate di sistema (syscall) verso il corretto driver che le implementa. In questo modo l IPCM semplifica anche lo sviluppo dei driver per gli IP-Core, in quanto maschera l interfaccia di programmazione del kernel, svolgendo tutte le operazioni comuni alla gestione delle periferiche ed alleggerendo così quelle da svolgere all interno del driver. L implementazione della metodologia descritta, presentata nel Capitolo 4, è stata effettuata su due schede prototipali, la Virtex-II Pro Evaluation Board e Virtex-II Pro Development Board prodotte da Avnet, con FPGA della famiglia Virtex-II Pro di Xilinx. Queste FPGA dispongono di processori PPC405 integrati; la versione montata sulla Development Board, in particolare, ne integra due ed ha una maggiore area riconfigurabile rispetto a quella della Evaluation Board. Le schede sono fornite di un kit di sviluppo (Board Support Package, BSP), sia per la parte hardware che software, per la creazione di sistemi embedded, consistente di un progetto, da utilizzare con i tool EDK (Embedded Development Kit) e ISE Integrated Software Environment di Xilinx, per la creazione della componente hardware e di una versione appositamente modificata di ELDK (Embedded Linux Development Kit), una piattaforma per lo sviluppo di sistemi software basati su LINUX. Per quanto rigarda il progetto hardware fornito nel BSP, esso consiste di un sistema dotato di processore, bus OPB e PLB e include tutti i componenti fondamentali presenti sulla scheda, quali memorie RAM (SRAM e SDRAM) e Flash, porta seriale (utilizzata principalmente come console) e controller Ethernet. Tale sistema è stato esteso integrando l IP-Core che realizza il controllore per la riconfigurazione parziale, presente all interno della libreria di componenti offerta da EDK, chiamato Internal Configuration Access Port, ICAP. Dal punto di vista software il cuore dell implementazione è dato dallo sviluppo di due moduli per il kernel di LINUX. Il primo realizza un driver per l ICAP che permette di utilizzare il controllore come una periferica del sistema, per effettuare riconfigurazioni parziali. L accesso al componente è fornito mediante un device node a caratteri, chiamato ad esempio /dev/icap, che è possibile scrivere e leggere. Prima di procedere alla riconfigurazione effettiva vii

8 è necessario inviare al device, tramite una semplice scrittura, un bitstream differenza, che viene salvato all interno di un buffer di memoria nel driver. La riconfigurazione vera e propria viene affidata ad un apposito segnale inviato al componente, sempre utilizzando le primitive standard di comunicazione con l hardware fornite dal kernel. Il driver permette anche di effettuare letture dal buffer di memoria del driver, in modo da poter recuperare il bitstream caricato in quel momento in memoria. Il secondo modulo del kernel implementa l architettura dell IP-Core Manager (IPCM) descritta precedentemente. Esso non costituisce direttamente un driver per gli IP-Core riconfigurabili, ma funge da gestore per la loro registrazione e deregistrazione ed il caricamento dei loro driver, specifici per ogni tipo di periferica. L IPCM, infatti, mantiene un elenco dei Core configurati in un certo momento, assieme ai dati necessari alla gestione e comunicazione (quali tipo di perifica e spazio di indirizzamento) e dei driver di basso livello caricati per poterle sfruttare. Il modulo implementa le principali chiamate di sistema per lo scambio di dati con le periferiche (tra cui apertura, chiusura, lettura e scrittura), per cui ogni richiesta di questo tipo su di un device node relativo ai moduli riconfigurabili viene indirizzato all IPCM. Esso riconosce a quale periferica è destinata la richiesta la reindirizza alla funzione che realmente implementa la chiamata di sistema all interno dell opportuno driver. Oltre a ciò, l IP-Core Manager gestisce, sempre all interno delle chiamate di sistema, alcune operazioni comuni a tutti i driver, ad esempio di locking delle risorse per evitare problemi di deadlock nell accesso in lettura o scrittura ai dati, di inizializzazione delle strutture dati che identificano delle periferiche e di acquisizione e rilascio delle risorse di memoria e spazio di indirizzamento. Grazie a questa politica, l implementazione dei driver specifici per ciascun tipo di periferica risulta semplificato, in quanto risulta solamente necessario implementare le funzioni corrispondenti alle chiamate di sistema e registrarle presso l IPCM. Inoltre, poichè anche i driver devono essere realizzati come moduli del kernel per essere caricabili dinamicamente, è stato creato uno stub, ossia una parte di codice che viene unita in fase di linking a quello del driver vero e proprio. Questo stub esporta verso il kernel di LINUX l interfaccia tipica di un modulo ed implementa le funzioni di inizializzazione e terminazione del modulo stesso. L architettura hardware/software presentata ed implementata è stata provata in alcuni scenari applicativi per verificarne la validità, effettuando misurazioni dei tempi di riconfigurazione interna effettuata mediante l uso del driver viii

9 dell ICAP. Il Capitolo 5, oltra a presentare tali risultati, presenta alcune possibili applicazioni dell architettura hardware con diverse modifiche, consistenti nell includere o escludere alcune dei componenti fissi (ad esempio la memoria Flash o il controller Ethernet) a seconda dei requisiti in termini di disponibilità di area riconfigurabile. Vengono infine presentate alcune possibili estensioni dell architettura software. ix

10

11 Contents Riassunto della tesi iii 1 Introduction 1 2 State of the art Reconfigurable hardware platforms MorphoSys The Splash 2 processor The MOLEN polymorphic processor The TRIPS polymorphous architecture The D-morph The T-morph The S-morph From stand-alone to operating system API for self-reconfigurable platforms Embedded and Real-time Operating Systems ThreadX RTOS Nucleus PLUS RTOS µc/os-ii RTOS µclinux Embedded OS RTAI, a LINUX real-time extension OS for Reconfigurable Computing Kernel customization for embedded systems Dynamic reconfigurability support in Operating Systems Interface virtualization Runtime infrastructure The Hardware Abstraction Layer (HAL) The Inter-task communication system The task scheduler xi

12 Contents Dynamic reconfigurability in LINUX OS Conclusions Methodology Hardware architecture Support for partial reconfiguration The reconfigurable side Software architecture Dynamic reconfiguration management Reconfiguration controller driver Reconfiguration data Reconfiguration with multiple devices IP-Core devices management IP-Cores Plug-and-Play Dynamic drivers loading Conclusions Implementation Target system BSP hardware architecture Software support Boot process Hardware architecture Partial reconfiguration support Reconfigurable modules Software architecture LINUX kernel modules infrastructure The ICAP kernel module Module structure System calls and bitstream management Devices status information The IP-Core Manager Module structure and hierarchy IP-Cores management Low-level driver structure A sample IP-Core module: the LED driver Conclusions xii

13 Contents 5 Tests and results Hardware architectures Architecture without Ethernet controller Architecture without Flash memory Partial reconfiguration performances Conclusions and future work 85 A Abbreviations 88 References 95 xiii

14 List of Tables 4.1 Memory layout of the Avnet Virtex-II Pro Evaluation board Different memory amounts of the Avnet Virtex-II Pro boards Original Caronte vs. Caronte with LINUX on the XC2VP7 FPGA Resources usage on the Xilinx XC2VP7 FPGA Resources usage on the Xilinx XC2VP20 FPGA Internal reconfiguration performances xiv

15 List of Figures 2.1 Architecture of the MorphoSys M2 reconfigurable processor Architecture of the MOLEN polymorphic processor Internal structure of a TRIPS processor core SRP Hardware Architecture Software layers of the SRP architecture Scheduling process in an OS for Reconfigurable Computing Generating a call graph from application source code Virtualization layers in OS and reconfigurable coprocessors The OS4RS operating system and hardware architecture Structure of the OS4RS InterConnection Network The Egretarchitecture Hardware side of the reconfigurable architecture Software side of the reconfigurable architecture The Reconfiguration Controller working model Multiple reconfiguration controllers configurations Registration of a new IP-Core by the IPCM Avnet Virtex-II Pro Evaluation board Flash memory map Hardware architecture of the reconfigurable system Physical layout of the reconfigurable system on the FPGA Internal structure of a BlackBox ICAP device data structure Module registration process function Partial reconfiguration process with ICAP Sample output from /proc/icap files LINUX kernel and IPCM module hierarchy IP-Core driver data structure System calls list implemented by an IP-Core driver IP-Core device data structure xv

16 LIST OF FIGURES 4.13 Registration process for a new IP-Core System calls sequence from userspace to low-level driver Implementation structure of a low-level driver xvi

17 Chapter 1 Introduction Everything should be made as simple as possible, but not simpler. Albert Einstein Reconfigurable embedded systems are nowadays widely used, since they have the capability to modify their functionalities, adding or removing components and modify interconnections among them. This ability can result very useful to create hardware/software architectures able to deal with changing in specifications, operating environment, protocols and new technologies. Nowadays the most commonly used reconfigurable devices are Field Programmable Gate Arrays, FPGAs, employed both as a component of a more complex system (playing the role of a co-processor) in place of ASICs (Application Specific Integrated Circuit), and as central unit, integrating most of the components of the system. Other than permitting an unlimited number of reconfigurations, FPGAs can also be partially and dynamically reconfigured: this means that single portions of the architecture can be modified without blocking the entire system. The reconfiguration can affect small portions of the FPGA, such as the content of a look-up table (LUT) or a single interconnection (this is the so called small-bits manipulation), or involve a bigger area, as an entire hardware component mapped on the chip (the module-based reconfiguration). Furthermore, RAM-based FPGAs allow internal reconfiguration, which means that the system does not need any external device to manage the reconfiguration process, but it can be performed directly inside the FPGA itself; internal reconfiguration is also much faster than the external one. 1

18 Chapter 1. Introduction Due to the previously described capabilities, FPGAs can be used to create hardware/software platforms that keep their flexibility after deployment, allowing the development of complex System-on-Chip (SoC) and reducing the overall number of physical components, since many resources can be configured on request, replacing unused ones. Modern FPGAs can also contain a general-purpose processor, which can be both a physical CPU embedded in the FPGA fabric, or a soft core, mapped in a portion of the FPGA; in both scenarios there is a software application running on the processor (or processors) which realizes some of the system functionalities, usually acting also as a controller for the hardware components and interfacing with them. The software part of a reconfigurable system can be either a standalone code, dealing directly with hardware at a low level, or a complete operating system, including multiprocessing and resource scheduling. A standalone code is usually an application which makes use software libraries exporting functions to interface with hardware components; this approach can be acceptable for small systems, involving few components and configurations, but as soon as the complexity increases, both from the hardware and software side, it becomes more difficult to develop a complete application dealing all those aspects. The use of an operating system, instead, allows more flexibility on both sides, since it s possible to implement the software part as one or more userspace processes, introducing complex inter-process communication systems and scheduling techniques; hardware management, instead, is remitted to the OS, which is in charge to offer the processes an interface to access system peripherals at an higher level of abstraction. The counterpart for this added flexibility is the necessity to add a support to a standard operating system for both reconfiguration-specific hardware and for reconfigurable components. This means that the hardware and software parts of the system must be designed in order to allow the creation of a reconfigurable architecture. A very important aspect in the design of embedded system is reusability, both in hardware and software, since often a new system is built from scratch with custom components targeted to a specific application field which in turn require proper software management, accomplished by an appropriate code running on the processor. This is not a very flexible solution, since it requires a strong effort to project and develop a new system; nevertheless in many cases it s possible to find a common structure in the architecture of different systems: often they have some hardware components in common, such as processor, RAM and Flash memories or particular peripherals; in this cases it s also possi- 2

19 Chapter 1. Introduction ble to introduce a common software system for hardware management, changing only the application code and the specific drivers dealing with peripherals. This issue is made easier by the use of a modular OS, which can easily adapt to changes in undelying hardware by means of addition of software support (drivers) for the required peripherals. Aim of this thesis is to define an hardware/software platform on FPGA based on the GNU/LINUX [Tor91] kernel supporting internal dynamic partial reconfiguration. From the hardware point of view, the architecture of the system is composed of a general purpose processor, RAM memory, a few fixed peripherals (such as network interface, serial port, Flash memory and so on), some reconfigurable modules (IP-Cores) and a component which manages the internal reconfiguration process and the registration and deregistration of the Cores with the operating system. The innovations introduced by this work basically consist of two elements: support for the reconfiguration controller. The controller is an hardware component which requires a software support to be exploitable from within the operating system. This consists of a driver which extends the LINUX kernel implementing the necessary functions to support the partial dynamic reconfiguration of single portions of the FPGA. support for the reconfigurable components. The IP-Cores can be considered as system peripherals that need a software support (basically a driver) from the operating system (OS) to be exploited from processes running under it. This is done by the IP-Core Manager (IPCM), which consists of a layer interfacing from one side with the operating system and from the other with the reconfigurable components. The main goal of this software component is to allow a kind of Plug-and- Play of the Cores, supporting runtime loading of specific drivers for the configured hardware. The IPCM acts as a middleware, offering to the higher levels of the software hierarchy (the userspace processes) transparency for what concerns the underlying hardware. It exploits the LINUX kernel modular structure to allow loading of the drivers specific to IP-Cores as standard kernel modules, managing devices registration and deregistration process, thus linking the system calls for a peripheral to the appropriate driver. 3

20 Chapter 1. Introduction The low-level drivers for the IP-Cores are LINUX kernel modules as the IPCM, but they don t have to deal with the kernel directly; instead they only have to implement system calls, such as read or write functions. In this structure, only the lowest level of the hierarchy, the drivers implementing peripheral-specific functions, are hardware dependent, while the IP- Core Manager is not bound to a specific hardware architecture, thus allowing the portability of the structure on different platform. The thesis is divided in six chapters: Chapter 2 presents a review of the state of the art for what concerns reconfigurable embedded systems: from the hardware point of view, some architectures for reconfigurable processing are described, focusing on how they answer typical problems such as reconfiguration, dynamic allocation and scheduling of resources. For what concerns the software side, the chapter presents both stand-alone and operating system based platforms introducing software support for reconfiguration and exploting of reconfigurable resources. Chapter 3 presents the methodology adopted to define a new hardware/software reconfigurable platform based on the GNU/LINUX operating system, which allows dynamic addition and removal of hardware components and corresponding software drivers. Topic of Chapter 4 is the description of an implementation of the proposed methodology, targeted to specific hardware platforms, the Avnet Virtex-II Pro Evaluation Board and Virtex-II Pro Development Board. Finally, Chapters 5 and 6 describe the results obtained from the previously described implmentation, and some possible extensions and future works based on the described platform. 4

21 Chapter 2 State of the art Every new beginning comes from some other beginning s end. Seneca This chapter presents the state of the art related to this thesis work. The first sections provides a description of different hardware reconfigurable platforms, along with an evaluation of advantages and disadvantages of each solution. The rest of the chapter describes various software solutions, starting from reconfigurable systems based on stand-alone code to those which make use of a complete operating system. 2.1 Reconfigurable hardware platforms FPGAs configuration capabilities allow a great flexibility in hardware design and, as a consequence, the possibility to create a vast number of different reconfigurable systems. These vary from system composed of custom boards with FPGAs, often connected to a standard PC or workstation, to stand-alone systems including reconfigurable logic (RL) and general purpose processors (GPP), to System-on-Chip, completely implemented within a single FPGA mounted on a board, with only few physical components for I/O interfacing. There are different models of reconfiguration, which can be classified according to the following scheme [WB04]: who controls reconfiguration 5

22 2.1. Reconfigurable hardware platforms when configuration is generated which is the granularity of the reconfiguration The first subdivision is between external and internal reconfiguration. In the first scenario, the reconfiguration is managed by an external entity, usually a PC or a dedicated processor. Internal reconfiguration, instead is performed completely within FPGA boundaries; for this to be possible, the device must have a physical component dedicated to reconfiguration, such as the ICAP component in Xilinx FPGAs. For what concerns the configuration generation, it can be done in a complete static way, at design time, determining all possible configurations of the system. Each module must be synthesized and all possible connections between modules and the rest of the system must be considered. Other possibilities are runtime placement of pre-synthesized modules, which requires dynamic routing of interconnection signal or complete dynamic modules generation. This last option is currently impracticable, since it would require runtime synthesis of modules from VHDL (or other hardware description language) code, which requires a very long processor time. Reconfiguration can involve different granularity levels, depending on the size of the area reconfigured. The two typical approaches are small-bits and module-based: the first consists in modifying single portion of the design, such as single Configurable Logic Blocks (CLB) or I/O blocks parameters (as described in [EKLM03]), while the second involves the modification of a bigger FPGA area. The module-based approach consists in creating hardware components (modules) that can be added and removed from the system; each time a reconfiguration is applied, one or more modules are linked or unlinked from the system. This require the reconfiguration of entire FPGA areas, which are generally sets of columns, since actual FPGAs configuration can only be done on per-column basis MorphoSys MorphoSys, described in [LFL + 99], is a reconfigurable processor targeted to computational intensive applications. The MorphoSys architecture, shown in Figure 2.1, is composed of the following elements: TinyRISC, which is a general purpose RISC processor, 6

23 2.1. Reconfigurable hardware platforms RC-Array (Reconfigurable Cells array) which contains the reconfigurable logic, a framebuffer, the embedded data memory of the reconfigurable processor, DMA (Direct Memory Access), used to transfer data from external memory, context memory, holding configuration informations TinyRISC External Memory Frame Buffer RC Array DMA Context Memory Figure 2.1: Architecture of the MorphoSys M2 reconfigurable processor The execution model of the MorphoSys processor is based on the partitioning of applications in sequential and data-parallel tasks; the first are executed by the TinyRISC processor, which is a general purpose 32 bit RISC processor, the latter are mapped onto the RC array. This is composed of an 8 8 array of reconfigurable cells (RC), whose configurations are stored in the context memory. In the first version of the MorphoSys processor, the cells are connected in a pyramidal way, while in the second version, M2, all RC in a single row or column are connected together. During execution, configuration data are fetched from the context memory, while computational data for the RC-Array is loaded in the frame buffer from external memory. Transfers between the MorphoSys components and the external memory, such as data loading for the framebuffer and context loading for the context memory, are managed by the DMA and requested from TinyRISC processor. After all necessary data has been loaded, 7

24 2.1. Reconfigurable hardware platforms TinyRISC enables the RC-Array issuing a specific command. During the computation, it s possible to change context to specific RC, reconfiguring only the selected portion of the array. The MorphoSys architecture offers a flexible platform for computational intensive applications, such as video or image processing [PNBK02]. Every RC implements a complete ALU which can operate on 16-bit wide data words, offering complete data parallelism with up 32 contexts possibly loaded at the same time. Furthermore, using the RC-Array interconnection network, which can be configured in different ways, Reconfigurable cells can pass data each other, allowing a flexible computation flow. The MorphoSys system is based on coarse-grained reconfiguration model, since the RC is the minimal configuration unit. Yet, the architecture discussed in [LFL + 99; PNBK02] has some flows, since it mainly describes the low-level details of the hardware components, such as the composition of the RC-Array and interconnection network, but it does not present a methodology to implement the whole reconfigurable system. Also, it is not detailed how an application can be partitioned in sequential and parallel tasks to subdivide the computation on the MorphoSys components, and how they are managed by the scheduler The Splash 2 processor The Splash 2 processor [ABH + 93], developed at the IDA Supercomputing Research Center, is an parallel processor in which the computing elements are programmable FPGA devices. It s designed to accelerate computation in algorithms that have temporal or data parallelism. The system is composed of a normal workstation (a Sun SparcStation host), an interface board and an array of Splash boards (from 1 up to 16). The reconfigurable elements in the Splash system all consist of Xilinx XC4010 FPGAs. The interface board between the workstation and the array consists of an input and an output DMA channel, each controlled by an FPGA (called XL and XR), connected to the SparcStation host via Sun SBus channel. The XL element is connected to the first board, while the XR is attached to the last one. Splash boards consist of 16 FPGAs (X1... X16), a crossbar switch and a seventeenth FPGA (X0), which acts as a control element for the board. Within a board, an FPGA is connected to its left and right neighbour and to the crossbar switch. The boards are connected each other in a chain, and the X0 element of each board is also connected to the interface board. 8

25 2.1. Reconfigurable hardware platforms The workstation performs a wide range of operations, since it acts as a general controller for the reconfiguration of FPGA elements and crossbar switches, sends computational data and control signals to the array and collects the results. The Splash architecture has been designed to supply a Single-Instruction Multiple-Data (SIMD) computational model, where each board has all processing elements configured to perform the same operations on different data in parallel, but the flexibility of the architecture allows a lot of different computational models. As an example, pipelining can be used to perform a flow of computation on the same data connecting different programmable elements so that the output of an FPGA is the input of the next one. Dynamic reconfiguration in the Splash model consists in modifying two kind of elements within a board: the crossbar switch and/or the processing elements. In the first scenario, reconfiguration of the crossbar switch interconnections allow an easy way to modify the data flow in the system without the need to modify the single computing elements; in the second one, single FPGAs are reconfigured to change the kind of computation performed on the data. Programming for the Splash system is done writing behavioral description of the algorithm using the VHSIC Hardware Description Language (VHDL), which goes through a process of refinement and debugging using the Splash 2 simulator. The algorithm is then manually partitioned on the different processing elements. Splash architecture main advantages can be summed as follows: acceleration for applications with temporal (pipelining) or data (SIMD) parallelism high I/O bandwidth for stream-oriented computation highly configurable computational model and data flow (reconfiguring FPGAs and crossbar switch) scalable architecture, since new boards can be added to the chain The paper described in this section does not present a methodology to partition an implementation of an algorithm on the array modules, but the process must be done manually. This makes programming the Splash system quite difficult, as it requires direct and low-level knowledge of the physical implementation of the system. Also, there is no direct way to derive a configuration for the crossbar switch even when the mapping of functional units on the FPGAs is known. 9

26 2.1. Reconfigurable hardware platforms The MOLEN polymorphic processor MOLEN [KGV04] is a reconfigurable processor designed to resolve opcode space explosion, modularity and compatibility problems. It s composed of three main elements (Figure 2.2): the Core Processor, which is a general purpose processor (GPP). This is a standard fixed-hardware processor which implements normal instructions. the Reconfigurable Processor (RP), implementing custom instructions. the Arbiter, an element in charge of performing partial decode of instruction to select which element (the GPP or the RP) should be called to issue the current instruction. Main Memory Instruction Fetch Data Load/Store Arbiter Data Memory MUX/DEMUX Register File Exchange Registers Core Processor Reconfigurable Microcode Unit Reconfigurable Processor CCU Figure 2.2: Architecture of the MOLEN polymorphic processor There are also a Data Load/Store Unit which performs data transfer with the main memory and the Data Memory Mux/Demux to distribute data between the Reconfigurable or Core Processor. Code running on the MOLEN processor can use both standard instruction set, which is implemented in the general purpose processor, or custom instructions, 10

27 2.1. Reconfigurable hardware platforms provided by the Reconfigurable Processor. This is used to speed up the execution of some common computations which are application dependent, allowing a single instruction to perform more complex tasks and reducing execution times thanks to hardware increased speed. The Reconfigurable Processor has two components: the Reconfigurable Microcode (ρµ-code) and the Custom Computing Unit (CCU). The first holds the configurations for the CCU which implement the custom instructions and manages CCU setup and reconfiguration, the latter is the computing elements which performs the execution of custom instructions. It contains both reconfigurable hardware and memory. Execution of custom instructions on the Reconfigurable Processor is divided in two phases: set and execute. In the first, the ρµ-code sets up the CCU with the appropriate configuration for the requested instruction, applying a partial reconfiguration of the component; then the execution of the instruction takes place on the CCU. The execution of RP instructions is transparent from the software point of view, since there is no specific instruction used to perform microcode load and setup on the CCU; instead this is done automatically by the ρµ-code when such an instruction is requested. The MOLEN architecture has been implemented on a Xilinx Virtex-II Pro (XC2VP20-5) FPGA, using the internal PPC405 processor as general purpose processor, and tested with an MPEG-2 decoder as a sample application. SAD, DCT and IDCT operations have been implemented as reconfigurable hardware instructions on the RP. The tests have shown a speedup between 2.80 and 2.96 compared with the software implementation. MOLEN is a very flexible solution for the implementation of embedded processors; the main advantage of its architecture is the possibility to define a specific instruction set for each application, moving the most common computational units to hardware with a great execution speed advantage. Further, the application benefits a complete transparency of the location of instruction execution (GPP or RP), since the Arbiter takes care of selecting the proper processor to call for each instruction The TRIPS polymorphous architecture Aim of the TRIPS processor, as explained in [SNL + 04], is to create a trade-off between the flexibility of application of general purpose processors and the speedup gained by application-specific processing units. While GPPs are able to handle different workloads well, they can t manage parallelism models typ- 11

28 2.1. Reconfigurable hardware platforms ical of specific application domains. Specialized processors, instead, such as network, server, scientific, graphics and digital signal processors are designed to exploit the particular forms of parallelism introduced by a certain application field. The actual design trend move towards the latter kind of processing units, since there is an increasing demand for high performances, in terms of low computational time, high I/O bandwidth, low response latency. Specialized processor, however, are subjected to performance fragility, since applications can incur large swings in performance based on how well they map to the given design. One solution to this fragility would be to create heterogeneous processors composed of different specialized processing cores targeted to specific applications. This approach, however, would highly increase hardware complexity and there would be little design reuse among the different cores. TRIPS authors propose a different approach, with the creation of an homogeneous processor built on top of a variety of heterogeneous processors. The idea is to create a processor that is able to achieve an efficient execution of a broad class of applications. The main issues to define such a polymorphic architecture are the granularity of processors and, for a given granularity, which processor architecture should be used. Very fine-grained architectures, in fact, may achieve good results in applications with an high level of parallelism, but are not well-suited for general-purpose and serial ones. The TRIPS processor solution is to create an hardware coarse-grained architecture, composed of large cores, which provide partitionable and configurable infrastructure and Instruction Set Architecture (ISA) to achieve internal parallelism (Figure 2.3). The polymorphous features of TRIPS allow the compiler or the runtime system to subdivide a core at different granularity for concurrent applications to be run in parallel. TRIPS cores are essentially pools of distributed ALUs and memory banks, which contain the architectural support necessary for software to create different mappings and flows of computation and data. Programs Compiled For TRIPS are partitioned into large blocks of instructions with a single entry point, without internal loops, and multiple possible exit points. Each block commits atomically in regard of higher levels and interrupts are managed completely within the block. Here, the instruction set is a data-flow instruction set, which means that instructions are organized as producer-consumer pairs with links between them, instead of using registers to exchange data like in common CISC or RISC architectures. This is called explicit data-flow graph execution, EDGE. For this reason, instructions contain the name of the consumer instructions, called targets, thus allowing the compiler to create a 12

29 2.1. Reconfigurable hardware platforms Reg. Bank Reg. Bank Reg. Bank Reg. Bank Block Control Predictor I Cache D Cache LSQ I Cache D Cache LSQ I Cache To L2 Cache D Cache LSQ I Cache D Cache LSQ I Cache Figure 2.3: Internal structure of a TRIPS processor core graph structure and schedule a data-flow-like execution in the block. Communication among different blocks, instead, occurs through registers. Cores also contain reservation stations, where instructions are stored and from where they can be extracted for execution from a node located in the core. After execution, results can be forwarded to any other local node or remote reservation station; each nodes is directly connected do its nearest neighbour on top, bottom, left and right sides. TRIPS resources can be divided in three categories: common resources are fixed, hard-coded and always used in all working modes. Examples of this kind of resources are execution units in nodes and interconnection network among nodes. polymorphous resources are also always used, but can be configured according to the necessities of a specific computation. specialized resources are fixed and not always used. When a resource of this kind is not used, it can be disables. Examples are specific registers such as extra program counters. The TRIPS processor can operate in three modes, depending on the kind of parallelism desired: the first is called D-morph and allows instruction-level parallelism (ILP), the second is T-morph, for thread-level parallelism (TLP) and finally the S-morph for data-level parallelism (DLP). 13

30 2.1. Reconfigurable hardware platforms The D-morph The D-morph mode achieves ILP uses the TRIPS EDGE instruction set architecture to provide out-of-order instruction execution. The execution graph is partitioned in blocks which are allocated on one or more frames. A frame is a set of nodes in one of the TRIPS cores, which physically consists of a certain 2D area. Considering also the number of frames as a spacial coordinate, a graph blocks is scheduled on a 3D area, which is called A-frame, architectural frame. Since ILP requires an high-bandwidth instruction fetching, A-frames are encoded with a fixed size, as instruction in VLIW processors, but the only order imposed for execution of instruction is data dependency. When there are enough free frames for the allocation of a new A-frame, a prediction for the new A- frame is performed The T-morph Thread-level parallelism is implemented by the T-morph. Aim of this mode is to optimize processor utilization by mapping multiple threads at the same time on a single TRIPS core. Differently from the D-morph, for TLP frames are partitioned a priori and allocated to threads. Threads can be either row threads or frame threads according to the kind of allocation used. To control execution, T- morph uses a number of instruction pointers equal to the number of threads allocated; each time a threads ends an execution block, a new block is loaded The S-morph The last working mode of the TRIPS processor is the S-morph. This mode exploits the Core internal ALU architecture and fast communication to provide highperformance data parallelism. This type of parallelism is typical of applications such as digital signal processing, encryption and real-time graphics, which require performing a fixed set of operations on wide loads of data. Nevertheless, different kinds of application require different memory and instruction management: signal processing, for example, usually have predictable memory accesses and instruction loops, since it consists in performing a fixed set of computation on the data, such as applying filters (whose coefficients are stored in fixed memory locations). Real-time graphics, in turn, have a more unpredictable behavior, depending on the kind of data provided at a certain time, and may present a broader variety of execution patterns. 14