NEXT GENERATION SPACED ANTENNA WIND PROFILER TECHNOLGY

Size: px

Start display at page:

Download "NEXT GENERATION SPACED ANTENNA WIND PROFILER TECHNOLGY"

Madlyn Newton
6 years ago
Views:

1 P1.6 NEXT GENERATION SPACED ANTENNA WIND PROFILER TECHNOLGY Charles L. Martin *, William O.J. Brown, Steve A. Cohn and Michael Susedik National Center for Atmospheric Research, Boulder, Colorado 1 INTRODUCTION This article documents recent developments in wind profiling technology, implemented as improvements to the NCAR Multiple Antenna Profiling Radar (MAPR)(Cohn et al. 1997). MAPR is a 915 MHz boundary layer wind profiler that utilizes spaced antenna (SA) methods to observe the threedimensional wind vector over a vertical profile. The technique is fundamentally different from the typical beam swinging methods of the more common Doppler based wind profilers. The Doppler approach points the radar antenna off from the vertical in a sequenced pattern in order to obtain the orthogonal Doppler wind components. In the SA approach, a single volume is illuminated by the transmitted pulse. Physically separated receiving systems provide multiple time series of the radar returns, and the cross correlation between the receivers is used to determine the drift of refractive index patterns across the antenna pairs. The point of the preceding description is to illustrate two fundamental additional demands placed on a spaced antenna profiler, beyond that of a Doppler system. The first is that the data bandwidth is potentially increased by the number of independent receiving systems, which will number at least three, if three-dimensional winds are desired. MAPR utilizes four receivers. The second constraint is that it is desirable to maintain close antenna spacing in an SA system. For the same given antenna aperture used for either the Doppler and SA methods, the antenna would be divided between the two receivers on a given baseline in the SA system. The individual return signal strengths will be half as strong for each SA channel as compared to the Doppler system. Thus, the data bandwidth and channel signal strengths are driving considerations for improvement of SA profilers. 1.1 Recent Developments The MAPR development has proceeded in three stages. The initial system was created by grouping four sets of Doppler profiler hardware together. Four * Corresponding author address: Charles L. Martin, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80307; martinc@ucar.edu Operating System Linux Computer Dell Workstation; dual 833MHz Pentium Data archive format Graphics Library Numeric Libraries Languages Radar Processors DSP DSP Support DSP interrupt rate Transmitter Typical throughput data NCAR netcdf Trolltech Qt FFTW C++, IDL, C, TI assembler 4 x NCAR 8 x Texas TMS320 (50 MHz, 50 MIPS each) TI Code Composer, running under VMWare Up to 40 KHz 4KW peak 20 GB per day Table 1: MAPR System Summary chains of antennas, receivers, digitization and signal processing hardware were integrated and driven by a modified version of MS-DOS based Doppler profiler software. Naturally, only one copy of the radar transmit side of the system was required. The DOS based system demonstrated the feasibility of SA boundary layer profiling, but suffered greatly from a very limited data bandwidth, because measurements had to be suspended while data were written to the archive media. The absence of pulse coding had a devastating impact on the signal to noise ratio (SNR). Finally, the software was unstable and difficult to maintain. The next stage of MAPR development involved improvements on the software side of the system. The DOS based architecture was scrapped, and a new one was implemented under Unix. This effort required the creation of device drivers for three specialized computer boards: a timing generator, a hardware based coherent integrator card, and a commercial digital signal processor (DSP) card. Once the device drivers were in place, then a new suite of software applications were written to manage the data collection and processing.

2 The initial Unix based system produced an order of magnitude improvement in the MAPR capabilities (Brown et al., 1999) The Unix infrastructure permitted data archival to take place during the measurement cycle. The addition of pulse coding provided tremendous improvements in the SNR, and for the first time MAPR was able to make routine wind measurements in clear air conditions. Modern software allowed for greatly enhanced real-time display capability, as well as the ability to now calculate wind vectors in real-time, rather than in post processing, as had been done with the previous system. This version of MAPR conclusively demonstrated that the SA approach is a viable method for boundary layer wind profiling. In the third generation MAPR, recent advances in radar electronics and computer hardware have been utilized to make further enhancements in system capability. Improvements in the current system have focused on the following: The transmitter was replaced with a unit producing eight times the former transmitted power. The IF section, A/D, coherent integrators and digital signal processor were separate items in the receive chain for each channel of the previous system. These four components are consolidated into a single PCI based computer card: the NCAR PC-based Integrated Radar Acquisition Card () (Randall, 1999). promises improved SNR and more flexible front end signal processing, as well as the advantage of decreased hardware complexity. The PCI interface allows for increased data throughput. A modern dual processor Pentium workstation has increased the compute load capability, so that the higher data throughput can be processed. The use of a standard consumer computer allows the system to be easily migrated to new machines in step with the rapid evolution of computers. A major achievement in version 3 of MAPR is that 100% data throughput has been realized. This allows arbitrarily long time series to be collected without breaks. master, generating a timing signal to synchronize the other cards. The master also generates a number of waveforms that create the transmit pulse. The internal functions of the are described later, but it is useful at this point to note that all cards are programmed to behave identically, with the single exception that the master card is free running, whereas the other cards are triggered by the master signal. Likewise, all cards generate the transmitter timing signals, but only the master has these signals connected to the transmitter. Having identical code and configuration greatly simplifies the MAPR software. The box labeled TX represents the transmit side of the system. The transmitter timing is controlled by the master, in order that it be coordinated with the receiver sampling. The flexible has capabilities that allow for sophisticated and varied radar implementations, such as dual pulse repetition rates, variable gate spacing, etc. Disk storage is used for short-term storage of data, and Exabyte tape is used for archiving. Full network access is provided to the system. Note that the four cards are contained in an external PCI chassis that is bridged to the workstation through a PCI extender. This mitigates the need to use a workstation with a large number of extra PCI slots. 2.1 The processor has been a key component in the advancement of MAPR. It integrates several functions on a single PCI card, as shown in simplified form in Figure 2. The wave table signal generator is a central to the operation of. It is a 24 bit wide memory that is stepped through at 48 MHz. Each bit in a row of TX C C C C 2 HARDWARE ARCHITECTURE A simplified view of the MAPR hardware design is shown in Figure 1. There are four identical receiver chains. The component labeled represents the initial down conversion and filtering, after which a 60 MHz intermediate frequency signal is delivered to the. The succeeding quadrature conversion, detection, sampling and initial signal processing are performed by the cards. Since the transmitter and all receivers must be synchronized, one of the cards acts as a Workstation Figure 1. MAPR Hardware Block Diagram. : down conversion and initial filtering, TX: transmitter, C: circulator.

3 memory is placed on output lines that control the timing of other parts of. The sampling and accumulation in the digital IF section as well as the filling of the DSP First-In-First-Out buffers (FIFOs) are controlled in this way. Eight of the lines are brought to the card edge to use for off-card timing needs. One bit of the wave table is used to stop its current run; another bit is used to reset its address counter to zero. The generic output lines from the wave table are used to create the pulse to drive the radar transmitter. Since the wave table is controlling the sampling as well, the complete radar timing and ranging is established here. The wave table begins running on command from the programmable trigger timer. This timer thus establishes the basic pulse repetition frequency for the radar. The wave table typically is programmed to run once per transmit pulse, and then stop until the next pulse. The timer has features that support staggered pulse repetition frequencies. Each contains two identical and independent Digital Signal Processors (DSP). Data samples for a given gate are transferred into the FIFOs. When the number of samples in the FIFO reaches a specified quantity, an interrupt is triggered on the DSP. The DSPs read the samples out of the FIFOs. In most applications, this will take place during a DSP interrupt. The DSP can be programmed to perform any desired initial signal processing. The DSP then transfers its data to the host via the PCI bus. 3 SOFTWARE ARCHITECTURE The MAPR system software runs under the Linux operating system. The use of Linux has shown a number of important benefits: Robust multi-tasking allows a single MAPR workstation to perform a full suite of tasks, including data acquisition, real-time data analysis, real-time display, data archiving, computation of derived products, and automated network transmission of data products. These activities are spread over a number of independent processes, which isolates the development from the difficulties typical of brittle monolithic applications. Integration of required the development of a Linux driver for the card. The open nature of Linux and the wide availability of information and outside assistance made this undertaking feasible and successful. Development tools for Linux are of high quality, widely available, and either inexpensive or free. The real-time software is written in C++, and is well organized into a number of supporting class libraries. It should be noted that this code was ported Timing/ Trigger Transmit Timing Wave Table Digital IF, A/D, Accumulators FIFO DSP 0 PCI Interface Receiver IF Input FIFO DSP 1 Figure 2. Functional Block Diagram from the previous (non-) version of MAPR, which ran under Solaris. All of the non-device specific code ported quickly to the Linux system. The real-time code calculates and stores correlation functions. The correlation functions are then analyzed periodically by a suite of IDL programs, which compute winds and performance statistics. Shell scripts transfer final data products across network links, and manage the tape archive. XML is used to manage and store the radar s operating configuration. A common misconception in instrumentation design is that a so-called real-time operating system is required. The standard versions of Linux are not even faintly considered to be real-time systems. A real-time designation means that the software must respond in a deterministic and specified manner to external events. In MAPR, data are buffered at many points along the processing chain. After the radar returns have been digitized in the IF section, there is no further need for deterministic behavior. Since the buffers are large enough to store data during the intermittent loading of the system, a realtime operating system is not required. 3.1 Integration A major design goal was to integrate as a standard Linux device. The device driver provides the standard Unix character device interface, supporting read(), ioctl() and mmap() calls. Mmap() exposes the

4 wave table as a memory segment. Ioctl() calls are used to configure and control operations. The read() interface allows application programs to suspend while waiting for data to be delivered by the. Class libraries make use of these calls to provide an integrated high-level functional interface to. The Linux driver is interrupt driven. It acts in concert with code running in the DSPs of the card. When new data is available in a DSP, it sends an interrupt to the host system. The driver acknowledges the interrupt, and reads the data from the DSP via dual ported memory on the PCI bus. The data are placed in a buffer in the driver, and the waiting application is unblocked to satisfy its read() request. In the MAPR application, with its eight DSPs, there can be a like number of pending read() requests. Taking advantage of the Linux interrupt architecture avoids penalties that would be incurred in a polling type of design. Each contains two Texas Instruments TMS320 DSP chips. These processors read the digitized baseband values from independent FIFOs attached to each DSP. The wave table is programmed to route the values to a particular DSP FIFO, and so there is great flexibility in designing the data flow and processing for the system. The MAPR design splits the vertical profile in half, with one DSP processing the lower gates and the other DSP processing the upper ones. The DSP chips are running code that is written mostly in C, with a small portion written in TMS320 assembler. In order to simplify the software management, both DSPs are running an identical program. Each DSP is aware of whether it is processing the lower or upper gates. Actually there are only a few places in the DSP code where this is important, and the program has conditional tests to account for this. The DSP code is designed in the typical foreground/background architecture. An interrupt driven assembly language routine (the foreground task) reads data from the FIFO as it becomes available, performs the coherent summation, and stores the I and Q data values. The background task performs pulse decoding on the collected time series and then transmits a block of data to the host, consisting of all gates for a given time span. The DSPs must coordinate with the Linux driver code when transferring data to the host. This means that the Linux driver is expecting a defined interrupt and handshaking protocol from the DSP chips. This protocol is thus implemented in the DSP programs. The development tools used for the DSP coding are the standard Texas Instruments Windows based utilities (trade named Code Composer ). The VMWare for Linux emulator provided an excellent Windows environment, on the MAPR workstation, to host the DSP code development. This obviates the creates MaprDisplay creates Mapr creates MaprLink MaprMaster DataBuffer MaprCruncher GraphicsBuffer ProfilerFile acquisition thread compute thread display thread Figure 3. Main application thread and object usage.

5 need to maintain a separate DSP support system, and facilitates rapid turn around on DSP code development and debugging. 3.2 Data Acquisition Task The most important process running on the MAPR workstation is the data acquisition task. MAPR has heavy data throughput and computational requirements. In order to minimize the amount of shuffling and reformatting of data, this task was designed as a multi-threaded application, where the address space is visible to all threads. The transfer of data among the threads for the most part is simply the passing of buffer pointers. The data acquisition task has three functions, split among three threads (see Figure 3): A data acquisition thread that configures and starts the cards, and then reads time series data from them. A thread that computes intermediate data products, which include correlation functions, power spectra and SNR values. This thread also writes the time series and intermediate results to data files. A thread that provides the real-time display of the time series and data products. Dividing these activities among individual threads allowed for a clean division of the tasks, and facilitates robustness in the system. Each thread works with a pool of buffers. The acquisition thread can keep filling buffers even if the calculation thread falls shortly behind. The display thread simply stops plotting updates if new data are not delivered to it. 3.3 Class Libraries A short summary of the C++ class libraries is given next. These libraries have provided a flexible set of building blocks for the MAPR software. COFF implements a representation of the Common Object File Format, which is the output product of the DSP code linker. The class is used to extract the information necessary for the Linux side to load and run the DSP code on the. Interface is used to access and control the card through the Linux driver. MAPRMaster implements high-level control of the card. For example, given radar operating parameters, it will translate these into the correct wave table definition. This class is also used to access the DSP data stream. MAPRCruncher performs the real-time radar calculations and data archiving. It sits between MAPR Control and Display, receiving data from the former and passing products onto the latter. ProfilerFile defines a netcdf file structure for MAPR data. MaprDisplay provides the real-time display of raw data and computed products. It is built on the Trolltech Qt graphical toolbox. Qt was found to be well designed, and extremely easy to use in this application. 4 Performance Some understanding of various performance issues and bottlenecks has been gained during the development and initial deployment of the based MAPR. Although a quantitative analysis is still to be performed, some general observations can be made. 4.1 DSP interrupt processing On the, the range resolved data samples are placed into the FIFO, at the transmitter pulse repetition frequency. When the FIFO is full, it interrupts the DSP. The DSP empties the FIFO and coherently integrates (i.e. sums) the values in order to reduce the data rate and increase the SNR. The opportunity to experiment with alternative methods of coherent integration is a promising feature of. The pulse repetition frequency can be as high as 40kHz and so the DSP is responding to interrupts at this rate. It turns out that this is a heavy load for the DSP. For example, assume that the DSP can perform 50 million arithmetic operations per second. At a 40 khz interrupt rate, there is time to perform 1,250 operations per interrupt. If the DSP is processing 40 gates per pulse, then there are operations available per gate. While this is a sufficient number of operations for simple computations, it is not overly generous. Moreover, in this simple scenario, the interrupt service overhead and the significant computational load of the background processing has been ignored. The MAPR DSP interrupt service and pulse decoding routines are written in assembly language to maximize their efficiency. Nonetheless, it was found that the number of gates has to be limited when operating at the highest pulse repetition rates. 4.2 PCI bus transfers The PCI bus is advertised as having high bus transfer rates. On closer inspection, one finds that these are theoretical maxima, and that observed transfer rates are significantly lower. The actual PCI bandwidth for transferring data between a PCI card and the host memory seems to be a function of the host architecture. Ad hoc PCI bus bandwidth tests made on a number of workstations found maximum transfer rates which varied between about 4 MB/s and

6 10 MB/s. Although not conclusively demonstrated, it appeared that dual processor machines showed the lowest of the transfer rates. A PCI device can transfer data across the bus, acting in either a bus master mode or a slave mode. In the master mode, the PCI device is commanded by the host to start the transfer, and the device then independently oversees and completes the transfer. In the slave mode, the host must directly issue read or write instructions to the address space of the PCI device. provides the PCI slave mode interface. This has serious ramifications for the system. While the Linux interrupt driver is reading data from the, other CPU activity is suspended. If the data throughput rate is a significant fraction of the actual bus transfer rate, the processing on the host can be impacted by this transfer. In a shared memory dual processor system, it seems that both CPUs could be affected. For the normal clear air winds measurement configuration of MAPR, this did not appear significant. However, for Radio Acoustic Sounding System (RASS) type measurements, which have much higher data rates, it appears that the PCI bandwidth is a critical factor. Investigation into this issue is continuing. 4.3 Real-time calculations The current MAPR implementation uses the DSP for time series coherent integration and pulse decoding. These time series are then passed to the host for the remaining processing. The next computations create correlation functions, power spectra and SNR estimates. Almost all of the calculations involve the forward and inverse Fast Fourier Transform (FFT). With the four receivers of MAPR, this requires (per gate) 4 forward FFTs, 10 cross products to generate the cospectra, followed by 10 inverse FFTs. A typical wind measurement configuration would have a 512 point time series of 3 second length, over 50 gates. This amounts to 233 complex FFTs (of length 512) per second. Only a small percentage of the dual Pentiums is required to support this. It was found that the graphics update was actually a more expensive operation on the system. Note that the Fastest Fourier Transform in the West (FFTW) numerical library was used for the spectral processing. This free software package was found to be simple to use, robust and highly optimized. For the radar configuration listed above, and when saving all data and computed values to disk files, the overall workstation CPU load is about 25%. 5 SUMMARY The current generation of the MAPR has resulted in a system that has improved performance characteristics. The software and hardware architectures have facilitated the creation of an instrument that is robust, easily maintained and well adapted for ongoing enhancements. The hardware component part count is significantly reduced by using. MAPR provides an excellent test bed for advancing the techniques of spaced antenna wind profiling. 6 REFERENCES Brown, W. O. J., S. A. Cohn, C. L. Martin, G. Maclean, and M. E. Susedik, 1999: NCAR's multiple antenna profiler radar, Proc. 29th International Conf. on Radar Meteor., July, 1999, Montreal, Canada, Cohn, S. A., C. L. Holloway, S. P. Oncley, R. J. Doviak, and R. J. Lataitis, 1997: Validation of a UHF spaced antenna wind profiler for highresolution boundary layer observations, Radio Sci., 32, Randall, M., 1999: -II: PC based radar digital receiver/processor offers more than MIPS, Proc. 29 th International Conf. on Radar Meteor., July, 1999, Montreal, Canada,

10.3 A General Purpose FPGA Based Radar Controller and Signal Processor

1 10.3 A General Purpose FPGA Based Radar Controller and Signal Processor Charles Martin*, Eric Loew, Chris Burghart National Center for Atmospheric Research, Boulder, Colorado 1 Introduction 1 Radar implementations