Architecture of parallel processing in computer organization

Transcription

1 American Journal of Computer Science and Engineering 2014; 1(2): Published online August 20, 2014 ( Architecture of parallel processing in computer organization Prabhudev S Irabashetti 1, Gawali Anjali B. 2, Betkar Akshay S. 2 1 Department of Computer, University of Pune, Maharashtra, India 2 Vishwabharati Academy s college of Engg, University of Pune, Maharashtra, India address irabashetti@gmail.com (P. S. Irabashetti), Anjaligawali8@gmail.com (Gawali Anjali B.), Akshaybetkar74@gmail.com (Betkar Akshay S.) To cite this article Prabhudev S Irabashetti, Gawali Anjali B., Betkar Akshay S.. Architecture of Parallel Processing in Computer Organization, American Journal of Computer Science and Engineering. Vol. 1, No. 2, 2014, pp Abstract This paper review the reporting of parallel processing in processor organization. A parallel processing becomes more trendy, the oblige for improvement in parallel processing in processor organization becomes even more significant. Here we evaluate each multiple processor organization, 1) SISD: single instruction single data, 2) SIMD: Single instruction multiple data 3) MISD: Multiple instruction single data and 4) MIMD: Multiple instruction multiple data along with Vector Processor/Array Processor, Symmetric Multiprocessing, NUMA and Cluster. Keywords Data, Instruction, IS: Instruction Set, CU: Control Unit, MU: Memory Unit PU: Processing Unit, LM: Local Memory, PE: Processing Element, SMP: Symmetric Multiprocessor, NUMA: Non-Uniform Memory Access 1. Introduction The nomenclature of computer systems proposed by M. J. Flynn in 1966 has remained the focal point in the field. This is based on the notion of instruction and data streams that can be simultaneously manipulated by the machine. Parallelism is to reduce the turnaround time but even increase the CPU time due to overhead and increase the required memory due to duplicate data and instructions. Writing and debugging a parallel program is much more complicated than a sequential program, at least an order-ofmagnitude. Different architectures, different programming models are suitable for different applications and so the characteristics of the applications should make the decision for the selection of parallel hardware architecture and also the parallelization of the applications. A stream is just a sequence of items (instruction or data) SISD: A type of computer architecture in which there is a single instruction cycle, and operands are fetched in serial fashion into a single processing unit before execution. SIMD: A type of multiprocessor architecture in which there is a single instruction cycle, but multiple sets of operands may be fetched to multiple processing units and may be operated upon simultaneously within a single instruction cycle. MISD: is a type of parallel computing architecture where many functional units perform different operations on the same data. MIMD: A type of multiprocessor architecture in which several instruction cycles may be active at any given time, each independently fetching instructions and operands into multiple processing units and operating on them in a concurrent fashion. 2. Classification of Parallel Processor Architectures 2.1. SISD Model In computing, SISD (single instruction, single data) is a term referring to a computer architecture in which a Single processor, a uniprocessor, executes a single instruction stream, to operate on data stored in a single memory. SISD is one of the four main classifications. In this system classifications are based upon the number of concurrent

2 13 Prabhudev S Irabashetti et al.: Architecture of Parallel Processing in Computer Organization instructions and data streams present in the computer architecture. SISD can have concurrent processing characteristics. Instruction fetching and pipelined execution of instructions are common examples found in most modern SISD computer A serial (non-parallel) computer Single Instruction: Only one instruction stream is being acted on by the CPU during any one clock cycle Single Data: Only one data stream is being used as input during any one clock cycle Deterministic execution This is the oldest and even today, the most common type of computer Examples: older generation mainframes, minicomputers and workstations; most modern day PCs. Figure 1. Taxonomy of Parallel Processor Architectures. Figure 2. SISD model. Figure 3. SIMD model SIMD Model Single Instruction stream Multiple Data stream. is a class of Parallel computer. It describes computers with multiple processing elements that perform the same operation on multiple data points simultaneously. Thus, such machines exploit data level parallelism. SIMD is particularly applicable to common tasks like adjusting the contrast in a digital image or adjusting the volume of digital audio. Most modern CPU designs include SIMD instructions in order to improve the performance of multimedia use. A type of parallel computer Single Instruction: All processing units execute the same instruction at any given clock cycle Multiple Data: Each processing unit can operate on a different data element Best suited for specialized problems characterized by a high degree of regularity, such as graphics/image processing. Synchronous (lockstep) and deterministic execution Two varieties: Processor Arrays and Vector Pipelines Examples: Processor Arrays: Connection Machine CM-2, MasPar MP-1 & MP-2, ILLIAC IV Vector Pipelines: IBM 9000, Cray X-MP, Y-MP &

3 American Journal of Computer Science and Engineering 2014, 1(2): C90, Fujitsu VP, NEC SX-2, Hitachi S820, ETA10 Most modern computers, particularly those with graphics processor units (GPUs) employ SIMD instructions and execution units techniques. Specifically, they allow better scaling and use of computational resources than MISD does. However, one prominent example of MISD in computing are the Space Shuttle flight control computers Advantages An application that may take advantage of SIMD is one where the same value is being added to (or subtracted from) a large number of data points, a common operation in many multimedia applications. One example would be changing the brightness of an image. Each pixel of an image consists of three values for the brightness of the red (R), green (G) and blue (B) portions of the color. To change the brightness, the R, G and B values are read from memory, a value is added to (or subtracted from) them, and the resulting values are written back out to memory. With a SIMD processor there are two improvements to this process. For one the data is understood to be in blocks, and a number of values can be loaded all at once. Instead of a series of instructions saying "get this pixel, now get the next pixel", a SIMD processor will have a single instruction that effectively says "get n pixels" (where n is a number that varies from design to design). For a variety of reasons, this can take much less time than "getting" each pixel individually, as with traditional CPU design. Another advantage is that SIMD systems typically include only those instructions that can be applied to all of the data in one operation. In other words, if the SIMD system works by loading up eight data points at once, the add operation being applied to the data will happen to all eight values at the same time. Although the same is true for any super-scalar processor design, the level of parallelism in a SIMD system is typically much higher MISD Model In computing, MISD (multiple instruction, single data) is a type of parallel computing architecture where many functional units perform different operations on the same data. Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel Figure 4. MISD model. A type of parallel computer Multiple Instructions: Each processing unit operates on the data independently via separate instruction streams. Single Data: A single data stream is fed into multiple processing units. Few actual examples of this class of parallel computer have ever existed. One is the experimental Carnegie-Mellon C.mmp computer (1971). Some imaginable uses might be: o multiple frequency filters operating on a single signal stream o Multiple cryptography algorithms attempting to crack a single coded message MIMD Model In computing, MIMD (multiple instructions, multiple data) is a technique employed to achieve parallelism. Machines using MIMD have a number of processors that function asynchronously and independently. At any time, different processors may be executing different instructions on different pieces of data. MIMD architectures may be used in a number of application areas such as computeraided design/computer-aided manufacturing, simulation, modeling, and as communication switches. MIMD machines can be of either shared memory or distributed memory categories. These classifications are based on how MIMD processors access memory Shared memory machines may be of the bus-based, extended, or hierarchical type. Distributed memory machines may have

4 15 Prabhudev S Irabashetti et al.: Architecture of Parallel Processing in Computer Organization hypercube or mesh interconnection schemes. A multi-core CPU is an MIMD machine. SIMD computers require wide design effort resulting in larger product development times. Since the underlying serial processors change so rapidly, SIMD computers suffer from fast obsolescence. Their regular nature of many applications also makes SIMD architectures less suitable 2.6. Vector Processor/Array Processor Figure 5. MIMD Model. A type of parallel computer Multiple Instruction: Every processor may be executing a different instruction stream Multiple Data: Every processor may be working with a different data stream Execution can be synchronous or asynchronous, deterministic or non-deterministic Currently, the most common type of parallel computer - most modern supercomputers fall into this category. Examples: most current supercomputers, networked parallel computer clusters and "grids", multiprocessor SMP computers, multi-core PCs. Note: many MIMD architectures also include SIMD execution sub-components The Central Processing Unit (CPU), implements instruction set containing instructions that operate on an array of data having at least two indices. Single vector instruction specifies a great deal of work like as executing the whole loop. In addition, vector processors provide vector instructions. With the use of single processor these multiple instructions can execute with the help of Pipelining. The computation of each result is independent of the computation of previous results, allowing a very deep pipeline without any data hazards. A single vector instruction specifies a tremendous amount of work it is the same as executing an entire loop. Thus, the instruction bandwidth requirement is reduced. Control hazards are non-existent because an entire loop is replaced by a vector instruction whose behavior is predetermined Comparison of SIMD and MIMD SIMD computers require less hardware than MIMD computers because they have only one control unit. SIMD computers require less memory because only one copy of the program needs to be stored. In contrast, MIMD computers store the program and operating system at each processor. However, the relative unpopularity of SIMD processors as general purpose compute engines can be attributed to their specialized hardware architectures, economic factors, design constraints, product life-cycle, and application characteristics. In contrast, platforms supporting the SPMD paradigm can be built from inexpensive off-theshelf components with relatively little effort in a short amount of time. Figure 6. Vector Processor. There are two primary types of architectures for vector processors: vector-register processors and memory- memory vector processors. In a vector-register processor, all vector operations except load and store are among the vector registers. In a memory-memory vector processor, all vector operations are memory to memory Advantages of Vector Processor: Each result is independent of previous results by allowing deep pipelines and high clock rates. A single vector instruction performs a great deal of work - meaningless fetches and fewer branches. Vector instructions access memory a block at a time which allows memory latency to be amortized over many elements.

5 American Journal of Computer Science and Engineering 2014, 1(2): Vector instructions access memory with known patterns, which allows multiple memory banks to simultaneously supply operands. Less memory access = faster processing time Symmetric Multiprocessing (SMP) In computing, symmetric multiprocessing or SMP involves a multiprocessor computer hardware architecture where two or more identical processors are connected to a single shared main memory and are controlled by a single OS instance. Most common multiprocessor systems today use an SMP architecture. In the case of multi-core processors, the SMP architecture applies to the cores, treating them as separate processors. Processors may be interconnected using buses, crossbar switches or on-chip Mesh network. The bottleneck in the scalability of SMP using buses or crossbar switches is the bandwidth and power consumption of the interconnect among the various processors, the memory, and the disk arrays. Mesh architectures avoid these bottlenecks, and provide nearly linear scalability to much higher processor counts at the sacrifice of programmability SMP Advantages Performance: If some work can be done in parallel Availability: Since all processors can perform the same functions, failure of a single processor does not halt the system Incremental growth: User can enhance performance by adding additional processors Scaling: Vendors can offer range of products based on number of processors Figure 7. Symmetric Multiprocessing System Non Uniform Memory Access (NUMA) NUMA refers to the designing of computer memory in multiprocessor system while the memory access time depends on the memory location. Under NUMA processor can access the local memory faster than the non-local memory. NUMA architectures logically follow in scaling from symmetric multiprocessing (SMP) architectures. NUMA can remove the problem, arrives when number of processors accesses the shared memory. For that NUMA provides the separate memory for each processor to eliminate demerits of tightly coupled systems. NUMA systems include additional hardware or software to move data between memory banks. This operation slows the processors attached to those banks, so the overall speed increase due to NUMA depends heavily on the nature of the running tasks Cluster Figure 8. NUMA. A cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone computers cooperatively working together as a single, integrated computing resource. A typical cluster: Network: Faster, closer connection than a typical network (LAN) Low latency communication protocols Looser connection than SMP Clusters are responsible to improve the performance of the system and provide the availability of the processor, while typically being much more cost-effective than single computers of comparable speed or availability. Clustering relies on the centralized management approach to make nodes available Advantages of Clusters Low Cost: Customers can eliminate the cost and complexity of procuring Elasticity: We can add and remove computer resources to meet the size and time requirements for your workloads. Run Jobs Anytime, Anywhere: You can launch compute jobs using simple APIs or management tools and automate workflows for maximum efficiency and scalability. You can increase your speed of innovation by accessing computer resources in minutes instead of spending time in queues.

6 17 Prabhudev S Irabashetti et al.: Architecture of Parallel Processing in Computer Organization 3. Conclusion In this paper we have talked about the Architectures of all parallel processing type in computer organization. We also explained the types of parallel processing with example and advantages of Single Instruction Multiple Data and evaluation of SIMD and MIMD. We presented detailed explanation about Vector processing, SMP (symmetric multiprocessor), NUMA (Non-uniform memory access), Cluster and its advantages [4] pdf [5] nter04_ganesan.pdf Biography Prabhudev S Irabashetti received his M.Tech. degree from Davangere University, Davangere in June He is presently working as Assistant Professor in Computer Department, Vishwabharati Academy s College of Engg Ahmednagar, Maharashtra, India. He is currently guiding B.E and M.E. students of VACOE Ahmedanagar. He has presented 2 papers in National Conference, and published 2 papers in International Journals Gawali Anjali B.(BE Computer Engg.) Student- Vishwabharti Academy s College of Engg., Ahmednagar. University of Pune- Pune. Maharastra-India Figure 9. Clustering of Processors. References [1] Introduction to Parallel Processing, Cornell Theory Center Workshop, [2] O. Serlin, The Serlin Report On Parallel Processing, No.54, pp. 8-13, November Betkar Akshay S. (BE Computer Engg.) Student- Vishwabharti Academy s College of Engg., Ahmednagar. University of Pune- Pune. Maharastra-India [3] Advances in Parallel Computing from the Past to the Future, Dr.P Sammulal, Volume 1, Issue 4, September 2013