Architectural Support for Large-Scale Visual Search. Carlo C. del Mundo Vincent Lee Armin Alaghi Luis Ceze Mark Oskin

Transcription

1 Architectural Support for Large-Scale Visual Search Carlo C. del Mundo Vincent Lee Armin Alaghi Luis Ceze Mark Oskin

2 Motivation: Visual Data & Their Applications Rebooting the IT Revolution, SIA, September Visual Applications 3D Reconstruction Augmented Reality Activity Recognition Content-based Search Algorithm Across These Apps: k-nearest neighbors (knn) Overall Challenges: High movement High compute requirements Idea: Co-design the memory subsytem with search algorithms.

3 State of the art: Today s Software Pipeline Feature Extraction 1 Extract shape, appearance, motion Feature Feature Feature Feature >1000 Index Features Scale of Index Trillions x Images Billions x Videos Billions x Audio Multimedia Database Query Extract features Feature Query Generation K-nearest neighbors 2?? Distance Calculation? Recover Result Result Result Result VEctor VEctor? Calculate pairwise metric distance Response Reverse Lookup Query Candidate Nearest Neighbor s 1 Wang et al., Action Recognition by Dense Trajectories, in CVPR Muja and Lowe, Scalable Nearest Neighbor Algorithms for High Dimensional Data, in PAMI 14.

4 State of the Art: Nearest Neighbor Search on Commodity Systems DRAM DRAM DRAM DRAM Intra-node Challenge: k-nearest neighbor search is heavily memory bound. Inter-node Challenge Each query vector generates k local responses. Query Video To get the global k nearest neighbors we need an all-to-one reduction; this makes the network the bottleneck.

5 Our Vision: Systems and Architecture Support for Nearest Neighbor Search Reduce Reduce Reduce Reduce Search Our Vision: A nearest neighbor content addressable memory () to alleviate memory boundedness by moving compute closer to. Query Video Reduce Reduce Reduction Our Vision: Systems support for innetwork joins and reductions to reduce global network bandwidth and scalability.

6 Intra-Node: Rebalancing Bandwidth and Compute Rebalance Bandwidth Use binary features 2. > 1000 Servicing One Query Algorithmic Growing Pains GB of communication 150 GFLOP of computation FLOP/byte FP Operators FMUL, FSUB FADD Bit Operators XOR POPCOUNT Rebalance Compute Move compute closer to memory. Storage SSDs or DRAM Our Vision. Nearest Neighbor CAM DRAM 1 Assume 512 KB feature vectors. N = 1 billion. Assume a single query needs to evaluate 10% of N with kd-trees. Distance metric is Euclidean. Note: we assume these numbers are from FLANN. 2 Gong and Lazebnik, Iterative Quantization: A Procustean Approach to Learning Binary Codes, in CVPR 11.

7 High-Level Overview: Nearest Neighbor Content Addressable Memory Idea: Achieve high internal bandwidth by interposing logic in memory. Our Vision: A class of memories akin to CAMs for high-throughput distance calculations. Input+ System+ Binary++Data k+=+2 Parameters distance+=+h id Output {id,+distance}

8 Architectural Overview: Inside the ODT ZQ RESET# CKE A12 CK, CK# CS# RAS# CAS# WE# Row Address A[15:0] Bank Address BA[2:0] Command Decode Control Mode Registers Address Register ZQCL, ZQCS Refresh Counter (RC = 8K) BC4 OTF ZQ CAL 3 3 Row Address MUX Bank Control To ODT/output drivers Bank 8 rowaddress latch and decoder Bank 2 rowaddress latch and decoder Bank 1 rowaddress latch and decoder Bank 0 rowaddress latch and decoder write Bank 0 Memory Array (65536 x 128 x 128) Sense Amplifiers knn Engine Accelerator Bypass Bank 1 Memory Array (65536 x 128 x 128) Sense Amplifiers write knn Engine read Accelerator Bypass write Bank 2 Memory Array (65536 x 128 x 128) Sense Amplifiers knn Engine read Accelerator Bypass write Bank 8 Memory Array (65536 x 128 x 128) Sense Amplifiers knn Engine read Accelerator Bypass r [31:0] Output Buffer ODT Control rvalid rready Engine Address EA[ADDRESS_BITS-1 :0] Engine Data w [31:0] Engine Operation cmd [2:0] Engine Control read Engine Read MUX I/O Gating DM Mask I/O Gating DM Mask I/O Gating DM Mask I/O Gating DM Mask

9 Programming Model: Integrating the in DRAMs DRAM Command Interface Programming API/Library Driver Layer Command Interface 1. additions are orthogonal to existing DRAM infrastructure. 2. We introduce a memory mapped configuration and control. 3. We introduce a command set which allows for configuration. DRAM Control Control Command Explanation DRAM Array and Controllers READ WRITE Reads the result out from the priority queues Writes initial query vector to accelerator memory DRAM Read/Write Interface Acceleration ENABLE DISABLE Signals to process read from DRAM Signals to ignore any from the DRAM DRAM Output Interface Systems Stack Output Interface RESET Reinitializes priority queues in preparation for next execution Command Set

10 Related Works: What came before the Software-based solutions Libraries [Arya, Mount, JACM 1998] ANN: Approximate Nearest Neighbors Library [Muja, Lowe, PAMI 2014] FLANN: Fast Library for Approximate Nearest Neighbors Algorithms [Datar et al., SCG 04] Locality-sensitive Hashing [Y. Weiss et al., NIPS 08] Spectral Hashing [Torralba et al., CVPR 08] Small codes for Recognition [Gong, Lazebnik, CVPR 11] Iterative Quantization Hardware-based solutions Nearest-Neighbors [Roberts, MS Thesis 90] PCAM: Proximity Content-Addressable Memory [Garcia et al., CVPRW 08] Nearest Neighbors on GPU In-memory computing [Patterson et al., IEEE MICRO 97] A Case for Intelligent RAM [D.G. Elliott, D&T 99] Computational RAM [Guo et al., MICRO 11] TCAMs for Data-Intensive Computing

11 Current Work: What We ve Done & Plan To Do Application Design & Characterization. Design & Implementation of VIRAL: VIdeo and Image Retrieval At Large-Scale.

12 Current Work: What We ve Done & Plan To Do Hardware Design. ASIC, Simulation, Design & Test. knn Engine Priority Shift Register Write Interface Sample Data Query Buffer Euclidean Distance Unit Index Buffer Accumulator knn Query Lane Priority Queue Shift Register Read Interface raddr > Memory Controller Write Interface Address Demux Data Arbiter knn Query Lane knn Query Lane knn Query Lane... knn Query Lane Read Interface Input Data > > > r knn Engine

13 Summary: Enabling visual applications via co-design Our Vision: Architectural support to enable high dimensional & high cardinality nearest neighbor search. Input+ System+ Binary++Data k+=+2 Parameters distance+=+l id Output {id,+distance} Our work enables visual applications such as: 3D Reconstruction Content-based Search Activity Recognition