Jim Harris. Principal Software Engineer. Data Center Group

Transcription

1 Jim Harris Principal Software Engineer Data Center Group

2 Notices and Disclaimers Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. Some results have been estimated or simulated using internal Intel analysis or architecture simulation or modeling, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance.. Intel processors of the same SKU may vary in frequency or power as a result of natural variability in the production process. Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate. Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision # The benchmark results may need to be revised as additional testing is conducted. The results depend on the specific platform configurations and workloads utilized in the testing, and may not be applicable to any particular user's components, computer system or workloads. The results are not necessarily representative of other benchmarks and other benchmark results may show greater or lesser impact from mitigations. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit Results have been estimated or simulated using internal Intel analysis or architecture simulation or modeling, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at The cost reduction scenarios described are intended to enable you to get a better understanding of how the purchase of a given In tel based product, combined with a number of situation-specific variables, might affect future costs and savings. Circumstances will vary and there may be unaccounted-for costs related to the use and deployment of a given product. Nothing in this document should be interpreted as either a promise of or contract for a given level of costs or cost reduction. No computer system can be absolutely secure Intel Corporation. Intel, the Intel logo, Xeon and Xeon logos are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. 2

3 Agenda Current State of the Project Looking Forward Call to Action 3

4 Current State of the Project Looking Forward Call to Action 4

5 One year ago... SPDK delivering exciting features and performance, but... Was difficult for people outside Intel to contribute No transparency on work going on inside Intel Internal automated test framework SPDK needed to truly become an open source community not just an Intel open source code repository! 5

6 Since then... Discussions Main Web Presence Real Time Chat w/ Development Community Weekly Calls Multiple Annual Meetups Backlog and Ideas for Things to Do Code Reviews & Repo Continuous Integration 6

7 Community Growth GitHub Traffic +60% Mailing List Traffic +200% Intel internal data based on public available GitHub and mailing list data. 7

8 SPDK Patch Commit Statistics SPDK Version Total Commits Non-Intel Committers Non-Intel Commits % % % % % Non-Intel Commit % 8

9 SPDK Patch Commit Statistics SPDK Version Total Commits Non-Intel Committers Non-Intel Commits % % % % % Non-Intel Commit % 9

10 SPDK SPDK Architecture Storage Protocols NVMe-oF* RDMA NVMe iscsi SCSI Integration Storage Services NVMe 3 rd Party Linux AIO Block Device Abstraction (bdev) Ceph RBD malloc BlobFS Blobstore RocksDB Ceph Tools fio Drivers NVMe-oF * Initiator NVMe Devices RDMA NVMe * PCIe Driver Intel QuickData Technology Driver 10

11 SPDK Architecture SPDK Added since Storage Protocols NVMe-oF* RDMA NVMe vhost-nvme iscsi SCSI vhost-scsi vhost-blk Linux nbd Integration VPP TCP/IP Storage Services Drivers NVMe NVMe-oF * Initiator 3 rd Party Linux AIO NVMe Devices RDMA Block Device Abstraction (bdev) Logical Volumes Ceph RBD NVMe * PCIe Driver PMDK blk GPT virtio (scsi/blk) virtio- PCIe virtio iscsi vhostuser QoS malloc BlobFS Blobstore Intel QuickData Technology Driver RocksDB Ceph QEMU Tools fio nvme-cli 11

12 vhost Provide SPDK-based storage to QEMU-based VMs virtio-scsi virtio-blk nvme (experimental) and non-qemu host processes containers 12

13 Logical Volumes Dynamic partitioning Thin provisioning Clones and snapshots vhost-scsi LUN vhost-blk device NVMe-oF namespace iscsi LUN lvol lvol lvol lvol Logical Volume Store NVMe* SSD

14 Userspace TCP/IP FD.io The Fast Data Project VPP Vector Packet Processing Utilize VCL (VPP Communications Library) for accelerated TCP processing VCL Shared Memory SPDK iscsi TCP Socket Abstraction POSIX NIC VPP Linux Kernel NIC 14

15 Current State of the Project Looking Forward Call to Action 15

16 SPDK Architecture SPDK Added since Storage Protocols NVMe-oF* RDMA NVMe vhost-nvme iscsi SCSI vhost-scsi vhost-blk Linux nbd Integration VPP TCP/IP Storage Services Drivers NVMe NVMe-oF * Initiator 3 rd Party Linux AIO NVMe Devices RDMA Block Device Abstraction (bdev) Logical Volumes Ceph RBD NVMe * PCIe Driver PMDK blk GPT virtio (scsi/blk) virtio- PCIe virtio iscsi vhostuser QoS malloc BlobFS Blobstore Intel QuickData Technology Driver RocksDB Ceph QEMU Tools fio nvme-cli 16

17 SPDK Architecture SPDK Added since In Progress Storage Protocols NVMe-oF* RDMA TCP NVMe vhost-nvme iscsi SCSI vhost-scsi vhost-blk Linux nbd Integration Cinder VPP TCP/IP Storage Services Drivers NVMe NVMe-oF * Initiator 3 rd Party Linux AIO NVMe Devices RDMA TCP Block Device Abstraction (bdev) Logical Volumes Ceph RBD NVMe * PCIe Driver PMDK blk GPT virtio (scsi/blk) virtio- PCIe virtio QoS DPDK Encryption iscsi vhostuser malloc BlobFS Blobstore Intel QuickData Technology Driver RocksDB Ceph QEMU Tools fio nvme-cli spdk-cli 17

18 Orchestration Cinder plugin for SPDK Work in progress Provision ephemeral storage using SPDK: NVMe-oF target Logical volumes Dynamic configuration via JSON-RPC Enables SPDK ephemeral storage provisioning for OpenStack! 18

19 Accelerators DPDK Framework Hardware Accelerators Optimized Software crypto bdev KEY lvol bdev crypto bdev KEY lvol bdev crypto bdev KEY lvol bdev DPDK Framework cryptodev Logical Volume Store NVMe bdev Namespace NVMe Controller 19

20 open channel Extend nvme driver API Ideal environment for Open Channel based applications nvme bdev Module ocssd bdev Module non-block use cases (log, object) nvme Driver Block API Open Channel API Standard NVMe SSD Open Channel NVMe SSD Open Channel NVMe SSD 20

21 Configuration Full JSON-RPC Configuration spdkcli - Terminal-based user interface 21

22 FUTURE WORK Compression Database Integration Block Device Aggregation Mirroring/Replication Fibre Channel Deduplication 22

23 Current State of the Project Looking Forward Call to Action 23

24 Call To Action Learn today (and tomorrow) from SPDK experts Join the community! 24

25