NBDE: How I could have slept better at night

Size: px

Start display at page:

Download "NBDE: How I could have slept better at night"

Christopher Thompson
5 years ago
Views:

1 NBDE: How I could have slept better at night Chuck Mattern Principal Solution Architect Red Hat

2 My old intro Red Hat Customer 18 years Linux User and Admin (TAMU, Slackware, Red Hat (& Enterprise), SuSE, Yggdrasil, Mandrake, Debian, CentOS, Scientific, Fedora) 26 years Unix User and Admin (Coherent, UNIXWare, DG-UX, HP-UX, AT&T B3, Solaris, AIX, OpenBSD, Dynix/PTX, DEC Unix, Ultix, SCO, PrimeOS) 27 years VMWare ESX (Engineer & Architect) 5 years Indus International (Unix Admin, Certified Solaris Admin (OS, Networking and Storage) 1998) 1.5 years Home Depot (Loss Prevention Supervisor, Programmer, Sys Admin, Architect, Principal Engineer, Red Hat Certified Engineer (RHEL 4 (2005), 6 (2011)) 27 years The Paradies Shops (Sr. Manager: Server, Network, Telephony, Desktop ) 1.5 years Red Hat (Solution Architect, Red Hat Certified Engineer (RHEL 7 (2016)) ~5 years

3 I m Irish, Italian and Sysadmin-ish We tend to talk with our hands We get excited We are passionate We like to share stories

4 Preface: Some core concepts Nobody is so horrible that he can t be the perfect bad example. -John Kelly Only in self discipline will you ever find freedom -Hon. James A. Walsh et al There but for the grace of God go I. -Hon. James A. Walsh et al

5 My Cautionary Tale 2,000+ sites across the US (including Puerto Rico, Hawaii and Guam) 4,000+ ESX hosts 2,000+ iscsi storage units 2,000+ Windows 2003 VMs 10,000+ RHEL VMs Global deduplicating compressing backup/recovery solution living on the same storage unit as the other VMs and replicating to a central site Fractional T1 to each location sharing credit auth and VOIP No local technical staff What could possibly go wrong?...oh yeah, my support team was 5 Engineers...

6 image via Peakpx

7 Enter the PRS Portable Recovery Server Run! Don t walk Grab the best castoff desktop you can find in the basement Snag two 1TB SAS disks a spare NIC and a gig of RAM from Microcenter Base install of RHEL4, mirroring the disks Encrypt the root volume with luks and use something tough like K&tx#vQ2*HW@9ucB! Remember, it s a $50-$100M a year business, in a box! Expose all spare disk via NFS Mount that up to your ESX host via primary NIC Build out a temporary recovery VM via ESX on the NFS share Replicate backup data Munge through and rename, re-ip everything under the covers Slap the remote location IP on the secondary NIC cause DHCP lived on one of the dead VMs (can you say down hard?) Shutdown and pack it in a box you found in the basement with styro-peanuts you stole^h^h^h borrowed from the shipping folks Drive like a maniac to Delta Dash then...wait...

8 What is LUKS? Linux Unified Key Setup from Clemens Fruhwirth in 2004 Originally for Linux, now there are Android (yeah I know it s Linux under there) Windows maybe elsewhere?

9 Translating: It ll be OK, I promise By Servershop24 [CC BY-SA 3.0 ( Wikimedia Commons By Dallastechline, Inc. [CC BY-SA 3.0 ( via Wikimedia Commons With a DR solution based on a scavenged desktop I had difficulty establishing credibility with my end customer even though I had a well thought out technical solution to the issue at hand. Talking a non-technical user through decrypting the root volume with a password such as K&tx#vQ2*HW@9ucB! Did not make things any easier

10 image via Peakpx

11 What is NBDE? Network Bound Disk Encryption Linux systems can decrypt volumes, even root volumes, over the network Based on clevis and tang clevis framework for the client side inserts into dracut has several pins tang for the server side one of the clevis pins License: CC0 Public Domain Robust Clevis On Vehicle

12 Where can I use NBDE? Laptops (duh ) Workstations Servers yep, even portable ones.

13 Logical View of Clevis and Tang

14 Architectural View

15 Server Installation

16 Server Installation and Configuration ~]# yum install -y tang [omitted] Installed: tang.x86_64 0:6-1.el7 Dependency Installed: http-parser.x86_64 0: el7_4 libjose.x86_64 0:10-1.el7 jose.x86_64 0:10-1.el7 Complete! ~]# systemctl enable tangd.socket --now Created symlink from /etc/systemd/system/multi-user.target.wants/tangd.socket to /usr/lib/systemd/system/tangd.socket. ~]# systemctl status tangd.socket tangd.socket - Tang Server socket Loaded: loaded (/usr/lib/systemd/system/tangd.socket; enabled; vendor preset: disabled) Active: active (listening) since Tue :01:23 UTC; 11s ago Listen: [::]:80 (Stream) Accepted: 0; Connected: 0 Oct 16 06:01:23 tang3.mobile.roninprinciples.com systemd[1]: Listening on Tan... Oct 16 06:01:23 tang3.mobile.roninprinciples.com systemd[1]: Starting Tang Se... Hint: Some lines were ellipsized, use -l to show in full. [root@tang3 ~]# firewall-cmd --add-service=http success [root@tang3 ~]# firewall-cmd --add-service=http --permanent success [root@tang3 ~]#

17 Client Installation

18 Client Installation: Software ~]# yum install -y clevis-dracut [omitted] Installed: clevis-dracut.x86_64 0:7-8.el7 Dependency Installed: clevis.x86_64 0:7-8.el7 clevis-luks.x86_64 0:7-8.el7 clevis-systemd.x86_64 0:7-8.el7 jose.x86_64 0:10-1.el7 libjose.x86_64 0:10-1.el7 libluksmeta.x86_64 0:8-1.el7 libpcap.x86_64 14: el7 luksmeta.x86_64 0:8-1.el7 nmap-ncat.x86_64 2: el7 tpm2-abrmd.x86_64 0: el7 tpm2-tools.x86_64 0: el7 tpm2-tss.x86_64 0: el7 tpm2-tss-devel.x86_64 0: el7 Complete! ~]#

19 Client Installation: luks Status ~]# cryptsetup luksdump /dev/vda2 LUKS header information for /dev/vda2 Version: 1 Cipher name: aes Cipher mode: xts-plain64 Hash spec: sha256 Payload offset: 4096 MK bits: 512 MK digest: 58 e6 af 4c 89 a8 05 f1 f9 fc 8d d c0 1c d7 43 MK salt: d8 c2 51 ae cd e7 3b d5 f7 9b dd 20 b9 3f e f0 c1 35 6a e b3 96 MK iterations: UUID: 80e b-45fd-88cd-7e8ec6b195c2 Key Slot 0: ENABLED Iterations: Salt: a6 6a 9f 45 a0 fb 11 f2 a4 e0 a a7 b6 0a c8 5a ce 5f 5a 7f c4 0e 87 e4 fc 68 Key material offset: 8 AF stripes: 4000 Key Slot 1: DISABLED Key Slot 2: DISABLED Key Slot 3: DISABLED Key Slot 4: DISABLED Key Slot 5: DISABLED Key Slot 6: DISABLED Key Slot 7: DISABLED [root@clevis ~]#

20 Client Installation: Configure clevis ~]# clevis luks bind -d /dev/vda2 sss ' { "t": 2, "pins": {"tang": [ {"url": " {"url": " {"url": " ] } } ' The advertisement contains the following signing keys: TepHUGV79tG8Cs0L9XPQh2s0f8A Do you wish to trust these keys? [ynyn] y The advertisement contains the following signing keys: _te0s8q9omn7gf4hqhehl9irsac Do you wish to trust these keys? [ynyn] y The advertisement contains the following signing keys: LdsB17ihj8MhRCaM8OiHEKkw2q8 Do you wish to trust these keys? [ynyn] y Enter existing LUKS password: [root@clevis ~]# Note: This example assumes a single block devise supporting an LVM volume group. Configurations with multiple block devices will require additional configuration.

21 Client Installation: luks Status ~]# cryptsetup luksdump /dev/vda2 LUKS header information for /dev/vda2 Version: 1 Cipher name: aes Cipher mode: xts-plain64 Hash spec: sha256 Payload offset: 4096 MK bits: 512 MK digest: 58 e6 af 4c 89 a8 05 f1 f9 fc 8d d c0 1c d7 43 MK salt: d8 c2 51 ae cd e7 3b d5 f7 9b dd 20 b9 3f e f0 c1 35 6a e b3 96 MK iterations: UUID: 80e b-45fd-88cd-7e8ec6b195c2 Key Slot 0: ENABLED Iterations: Salt: a6 6a 9f 45 a0 fb 11 f2 a4 e0 a a7 b6 0a c8 5a ce 5f 5a 7f c4 0e 87 e4 fc 68 Key material offset: 8 AF stripes: 4000 Key Slot 1: ENABLED Iterations: Salt: 12 8b 7e cd d8 79 b fd 4c bd d 1f ec aa a 14 8b 65 b1 e1 95 a2 de 3c cc eb Key material offset: 1016 AF stripes: 4000 Key Slot 2: DISABLED Key Slot 3: DISABLED Key Slot 4: DISABLED Key Slot 5: DISABLED Key Slot 6: DISABLED Key Slot 7: DISABLED [root@clevis ~]#

22 Client Installation: luksmeta status ~]# luksmeta show -d /dev/vda2 0 active empty 1 active cb6e ff-40da-a84a-07ab9ab5715e 2 inactive empty 3 inactive empty 4 inactive empty 5 inactive empty 6 inactive empty 7 inactive empty [root@clevis ~]#

23 Delivering: It ll be OK, I promise By Servershop24 [CC BY-SA 3.0 ( from Wikimedia Commons By Dallastechline, Inc. [CC BY-SA 3.0 ( via Wikimedia Commons

24 License: CC0 Public Domain Jeff Rowley

25 A few of the finer points No encryption needed in flight Luks key is never transmitted Only the encrypting key is transferred over the wire Encrypted paraphrase is stored in luks header

26 Encryption Walk Through

27 Encrypting a sample passphrase [root@clevis ~]# echo 'Good Morning Columbus, Ohio!' clevis encrypt sss ' { "t": 2, "pins": {"tang": [ {"url": " {"url": " {"url": " ] } } ' >gmco.jwe The advertisement contains the following signing keys: TepHUGV79tG8Cs0L9XPQh2s0f8A Do you wish to trust these keys? [ynyn] y The advertisement contains the following signing keys: _te0s8q9omn7gf4hqhehl9irsac Do you wish to trust these keys? [ynyn] y The advertisement contains the following signing keys: LdsB17ihj8MhRCaM8OiHEKkw2q8 Do you wish to trust these keys? [ynyn] y [root@clevis ~]#

28 Decrypting a sample passphrase With two servers down the threshold of 2 out of 3 tang servers cannot be met: [root@clevis ~]# clevis decrypt <gmco.jwe Error communicating with the server! Error communicating with the server! [root@clevis ~]# Once at least 2 of the 3 servers are online we can decrypt the passphrase: [root@clevis ~]# clevis decrypt <gmco.jwe Good Morning Columbus, Ohio! [root@clevis ~]#

29 It s not just tang for breakfast anymore Shamir s Secret Sharing from Adi Shamir Allows for combinations of multiple kinds of pins tang tpm2 http math too painful for mere mortals think of it as the intersection of RAID and cryptography for now see the Wikipedia link below if you re a cryptographer, mathematician or just like pain

30 Magical things you can do with SSS

31 Magical things you can do with SSS

32 Magical things you can do with SSS

33 Magical things you can do with SSS

34 Magical things you can do with SSS

35 Magical things you can do with SSS

36 Magical things you can do with SSS

37 Magical things you can do with SSS

38 Magical things you can do with SSS

39 Quick sample incantation (human readable) clevis luks bind -d /dev/vda2 sss ' {"t": 2, "pins": {"tang": [ {"url": " {"url": " {"url": " ] } }'

40 Thank you for attending Ohio Linux Fest!

41 Resources & Credits Portions of the content were based on presentation from: Nathaniel McCallum Brian Atkisson Jim Wildman Technical references: luks: cryptsetup Samir s Secret Sharing: clevis: tang:

New RHEL 7.5 features: VDO, USBGuard, NBDE and AIDE. RHUG Q Marc Skinner Principal Solutions Architect 3/21/2018

New RHEL 7.5 features: VDO, USBGuard, NBDE and AIDE RHUG Q1.2018 Marc Skinner Principal Solutions Architect 3/21/2018 RHEL7.5beta :: New Features Storage - Virtual Data Optimizer (VDO) Security - NBDE