Computer Architecture

Size: px

Start display at page:

Download "Computer Architecture"

Arlene Simon
5 years ago
Views:

1 Computer Architecture PCI and PCI Express február 22. Budapest Gábor Horváth associate professor BUTE Dept. of Networked Systems and Services

2 2 The PCI standard PCI = Peripheral Component Interface, 1992, Intel Main features: CPU independent (PC, Macintosh, DEC, HP, SUN servers) Supports device auto-configuration Supports single and burst transmission modes 32 bit or 64 bit data units Synchronous bus clocked at 33 MHz (66 MHz from 1995) Maximal number of devices on a PCI bus: 32 Maximal number of PCI busses in a PCI system: 256 Implements energy management

3 3 PCI based systems PCI in Intel Pentium Pro:

4 4 PCI based systems PCI from Pentium 4 on: North bridge: Memory Controller Hub, MCH South bridge: I/O Controller Hub, ICH

5 5 Components of the PCI Main components Host/PCI bridge Translates I/O and memory requests of the CPU PCI transactions PCI devices Max. 32 devices/bus (max. 10 is more realistic) 1-8 functions/device (logical I/O devices on the same physical devide) PCI/PCI bridge Connects another PCI bus to the PCI bus Number of I/O devices in theory: 8 functions/device * 32 devices/bus * 256 buses/system = logical devices/system

6 6 Data transmission Each PCI device has max. 6 windows A window can be An address range in the memory address space An address range in the I/O address space When the CPU performs a read/write to a window of a PCI device it reads from/writes to the PCI device Devices can tell the system How many windows they need in the memory and in the I/O address space How large windows they need The windows are allocated to the devices by the BIOS and the op. system They tell the devices where their windows are PCI devices are listening on the bus, and respond to operations affecting their windows

7 7 Data transmission Example: CPU reads from the memory The address falls into a window of a PCI device The Host/PCI bridge Detects that the address is a PCI address On 32-bit PCs: addresses above 3 GB are considered to be PCI addresses Initiates a Memory read transaction on the PCI bus Obtains the right to use the bus (GNT) Puts the address and a memory read command to the PCI bus A device recognises that it falls into one of its windows Indicates it using the DEVSEL signal Data transmission starts Byte enable tells the device what byte order to use (and more) Data can be transmitted only when both the initiator is ready (IRDY) and the target device is ready (TRDY) The transmission ends when the initiator clears the FRAME signal The Host/PCI bridge returns the data obtained from the device to the CPU

8 Data transmission Computer Architecture Gábor Horváth, BME-HIT 8

9 9 Data transmission Flow control: The initiator is able to indicate that it is ready for the next data unit through line IRDY The target device is able to indicate that it is ready for the next data unit through line TRDY Transaction models: Programmed I/O. Initiated by: CPU, target: PCI device DMA. Initiated by: PCI device, target: system memory Peer-to-peer transfer. Initiated by: PCI device, target: PCI device

10 10 PCI commands C/BE Commands 0000 Interrupt Acknowledge 0010 I/O Read 0011 I/O Write 0110 Memory Read 0111 Memory Write 1010 Configuration Read 1011 Configuration Write

11 11 PCI commands C/BE Commands 0000 Interrupt Acknowledge 0010 I/O Read 0011 I/O Write 0110 Memory Read 0111 Memory Write 1010 Configuration Read 1011 Configuration Write

Can take the delay sensitivity of some devices into account Example: A, B: delay sensitive,

12 12 Arbitration Parallel arbitration: Hidden arbitration: The contention for the bus and the selection of the next winner occurs during the current transaction Algorithm: Must be fair Can take the delay sensitivity of some devices into account Example: A, B: delay sensitive, X, Y, Z not delay sensitive Order: A, B, X, A, B, Y, A, B, Z, A, B, X, A, B, Y, A, B, Z, etc.

13 13 Interrupts Two ways to generate interrupts By using one of the interrupt lines of the PCI bus By message signalled interrupts (MSI) By using the interrupt lines: Each PCI slot has four interrupt lines: INTA, INTB, INTC, INTD Can be shared or dedicated The mapping (which CPU interrupt it generates) can be read from the device by configuration transactions The interrupt signals of several PCI devices can be mapped onto the same CPU interrupt software polling is needed to find out which device generated the interrupt Message signalled interrupts: The PCI device writes a specific data to a specific address The Host/PCI bridge listens to that address, and generates a CPU interrupt when the specific data is written

14 14 Configuration Each device has 64 configuration registers, 32-bit each Configuration registers Can be read...or changed These operations are platform dependent Identifying a configuration register: On PC, special I/O addresses are used: 0CF8h: the above defined address of the configuration register 0CFCh: the content of the configuration register

15 Configuration Reading/writing a configuration register Host/PCI bridge detects an I/O operation at addresses 0CF8h and 0CFCh The register ID contains the device ID as well There is a dedicated signal IDSEL among each device and the Host/PCI bridge it sets the IDSEL of the selected device to 0 (1 for the others) Initiates a configuration read/write transaction only the selected device is listening! C/BE lines: code of the configuration read/write A/D lines: Type of configuration The number identifying the target function (logical device) Which register is of interest Transmitting the content of the configuration register: as done with ordinary data Computer Architecture Gábor Horváth, BME-HIT 15

16 16 Configuration Content of the registers: The first 16 has a fixed purpose: VendorID, DeviceID, Revision Class Code Interrupt Pin Interrupt Line Base Address Register During the boot process of the system: The devices are enumerated systematically The windows are allocated to devices Device drivers are loaded The device drivers configure the devices further using configuration transactions...

17 17 PCI interface Mandatory: Clock, Reset A/D[0...31] C/BE[0...3] FRAME IRDY, TRDY, STOP DEVSEL, IDSEL Parity, PERR, SERR REQ, GNT Optional: A/D[ ] C/BE[4...7] INTA, INTB, INTC, INTD REQ64, ACK64 CLKRUN 64 bit PCI slot 32 bit PCI slot

18 18 PCI Express Goals: To achieve higher transmission speed than PCI To keep full software compatibility with PCI Most signifficant changes: Serial data transmission To eliminate the problem of singal shift among parallel lines Point-to-point connections No shared medium no contention, no waiting, no starvation, no arbitration

19 Simple PCI Express based systems Computer Architecture Gábor Horváth, BME-HIT 19

20 20 Simple PCI Express based systems Serial differential data transmission: No clock signal (detected automatically from the 0/1 transitions) A pair of wires: 500 MB/s Full duplex communication: 2 serial pair of wires for both directions Two devices can be connected by more full duplex lines The name of 1 full duplex serial line: lane Part of the standard: 1x, 4x, 8x, 16x, 32x lanes More lanes allow parallel transmission of data But not in a synchronous way!!! With 32x lanes we have 32x500 MB/s = 16 GB/s

21 PCI Express based systems with switches Computer Architecture Gábor Horváth, BME-HIT 21

22 22 PCI Express based systems with switches Works like a packet switched network Devices are connected to the root complex through switches in a tree topology Passing through several switches can be necessary to reach the target device Transactions are travelling from switch to switch as packets

23 23 Transmission of transactions Transactions are processed by 3 layers before getting transmitted on the serial line:

24 Transmission of transactions Transaction layer: produces packets from transactions Header: Command, byte order (PCI: C/BE) Address (PCI: Address) Payload: Data to transmit CRC (for error detection) Computer Architecture Gábor Horváth, BME-HIT 24

25 Transmission of transactions Data Link Layer: error-free transmission of the transaction packet to the next hop An other header: sequence number (auto-incremented) An other CRC (protects the sequence number as well) Computer Architecture Gábor Horváth, BME-HIT 25

26 Transmission of transactions Physical Layer: transmission through the serial line Marks the boundaries of the packets by special bit sequences Ensures the presence of the necessary amount of 0/1 transitions: Scrambles the bits 10 bits are used to encode 8 bits Computer Architecture Gábor Horváth, BME-HIT 26

27 27 Transmission of transactions Packet format: Overhead when transmitting 32 bits: 1+2+3*4+1* = 28 byte

28 28 Transmission of transactions Receiver side, physical layer: Detects packet boundaries Decodes 8/10 bit encoding Decodes scrambling

29 29 Transmission of transactions Receiver side, Data Link Layer Checks CRC and the sequence number Sends back negative or positive acknowledgement depending on the checks In case of a negative acknowledgement the sender retransmits the packet

30 30 Transmission of transactions Receiver side, Transaction layer Checks CRC Executes transaction

31 31 Transmission of transactions Posted vs. Non-posted transactions: Posted: no reply is needed, eg. Write transactions Non-posted: reply needed, eg. Read transactions The transaction is only a read request The target device sends back the data requested in a separate transaction QoS (Quality of Service): the header has a traffic class field To indicate urgency of the transaction Urgent transactions can go before less urgent ones

32 32 Interrupts and configuration Interrupts Message signalled interrutps (MSI) just like in PCI Interrupt line based interrupts: PCI Express does not have interrupt lines It emulates them for compatibility: The device sends two packets to the root complex:» Packet 1: I have pulled my interrupt line down to 0» Packet 2: (a bit later): I have pulled it back to 1 The interrupt is then handled as in PCI Configuration: The number of registers a device can have is increased from 64 to 1024 First 64 can be read and written as in PCI Further registers: different addressing is needed, not detailed here

33 33 PCI Express slots PCIe x1 PCIe x16 PCI

Introduction Electrical Considerations Data Transfer Synchronization Bus Arbitration VME Bus Local Buses PCI Bus PCI Bus Variants Serial Buses

Introduction Electrical Considerations Data Transfer Synchronization Bus Arbitration VME Bus Local Buses PCI Bus PCI Bus Variants Serial Buses 1 Most of the integrated I/O subsystems are connected to the