Ahead in the Cloud. Matt Wood TECHNOLOGY EVANGELIST

Transcription

1 Ahead in the Cloud Matt Wood TECHNOLOGY EVANGELIST

2 Hello.

3

4 Thank you.

5

6 Don t be afraid to be a bit technical!

7

8 There Will Be Code

9 3

10 1 Building blocks

11

12 Infrastructure services

13

14 Compute

15

16 Storage

17

18 Databases

19

20 Services + Support

21

22 5 years young

23 ?

24

25 On demand

26

27 Pay as you go

28

29 Pay for what you use

30

31 Elastic

32 Capacity Estimated demand Time

33 Capacity Infrastructure Investment Estimated demand Time

34 Capacity Infrastructure Real demand Time

35 Capacity Elastic capacity Real demand Time

36

37 Flexible

38

39 API

40

41 Agility

42

43 Development agility

44

45 Business agility

46 Focus on your stuff

47 Idea Product

48 Idea Product Heavy lifting

49 Scale Redundancy Admins 70% Idea Product Heavy lifting Capacity Management Procurement

50 30% Idea Product Infrastructure

51 Maintenance Monitoring 30% Idea Product Infrastructure Logging New releases & environment

52 Idea Product AWS

53 Idea Product AWS

54

55 Why?

56

57 How?

58

59 On board

60 Strategy Migration On board Dev/test Production

61

62 2 Cloud by Example

63

64 CloudFormation aws.amazon.com/cloudformation

65

66 Define a full infrastructure stack

67

68 Template

69

70 EC2 SNS Auto-scaling SimpleDB RDS SQS Resources Elastic Beanstalk CloudWatch Security groups Tags

71 Template CloudFormation Provisioned resources

72

73 Complete definition

74

75 Atomic

76

77 Idempotent

78 Anatomy of a template

79 JSON

80 Plain text Perfect for version control JSON Validate-able

81

82 Declarative language

83 { "AWSTemplateFormatVersion" : " ", } "Description" : "Create an EC2 instances", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" }, "ap-northeast-1" : { "AMI" : "ami-8e08a38f" } } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "AZ" : { "Description" : "Availability Zone of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] } }, "PublicIP" : { "Description" : "Public IP address of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] } } }

84 { "AWSTemplateFormatVersion" : " ", "Description" : "Create an EC2 instances", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" }, "ap-northeast-1" : { "AMI" : "ami-8e08a38f" } } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, Headers Parameters Mappings Resources } "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "AZ" : { "Description" : "Availability Zone of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] } }, "PublicIP" : { "Description" : "Public IP address of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] } } } Outputs

85

86 Parameters Provision-time specification Command line options

87 "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } },

88

89 Mappings Conditionals Case statements

90 "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" }, "ap-northeast-1" : { "AMI" : "ami-8e08a38f" } } },

91 "Mappings": { "AWSInstanceType2Arch" : { "t1.micro" : { "Arch" : "64" }, "m1.large" : { "Arch" : "64" }, "m1.xlarge" : { "Arch" : "64" }, "m2.xlarge" : { "Arch" : "64" }, "m2.2xlarge" : { "Arch" : "64" }, "m2.4xlarge" : { "Arch" : "64" }, "c1.xlarge" : { "Arch" : "64" }, "cc1.4xlarge" : { "Arch" : "64" } },

92

93 Resources

94 "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }

95 "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }

96 "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }

97 "KeyName" : { "Ref" : "KeyName" }, Parameter reference

98 "ImageId" : { }, "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]

99 Map conditional "ImageId" : { }, "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]

100 "ImageId" : { }, "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ] Name of map

101 "ImageId" : { }, "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ] Intrinsic property reference

102

103 Outputs Returned values

104 } "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "AZ" : { "Description" : "Availability Zone of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] } }, "PublicIP" : { "Description" : "Public IP address of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] } } }

105

106 Deliver via API

107

108 Validate via API

109

110 Deliver via S3

111

112 Growing library

113

114 Elastic Beanstalk aws.amazon.com/elasticbeanstalk

115

116

117

118 Your Application Application Service Java Web Platform HTTP Service Tomcat Language Interpreter Java Operating System Linux Server Amazon EC2

119

120 Upload WAR

121

122

123

124

125

126

127

128

129 Auto-scaling

130

131 6 am

132 10 am

133 10 am

134 10 am

135 7 pm

136 7 pm

137 7 pm

138

139 Elastic MapReduce aws.amazon.com/emr

140

141 Hosted Hadoop

142

143 Without the muck

144 Input data S3

145 S3 Input data Code Elastic MapReduce

146 S3 Input data Code Elastic MapReduce Name node

147 S3 Input data Code Elastic MapReduce Name node Elastic cluster

148 S3 Input data Code Elastic MapReduce Name node HDFS Elastic cluster

149 S3 Input data Code Elastic MapReduce Name node Queries + BI HDFS Via JDBC, Pig, Hive Elastic cluster

150 S3 Input data Code Elastic MapReduce Name node Output S3 + SimpleDB Queries + BI HDFS Via JDBC, Pig, Hive Elastic cluster

151 S3 Input data Elastic MapReduce Output S3 + SimpleDB

152

153

154

155

156

157

158

159

160

161

162 It s all just Hadoop

163

164 HDFS + S3

165

166 Hive, Pig, Cascading, Streaming

167

168 API driven

169

170 FREE

171 3 Security at every level

172

173 Shared responsibility

174

175 Requirement based access

176

177 Certification

178

179 ISO SAS 70 Type II

180

181 PCI DSS Level 1

182 Control objectives Security organisation Employee lifecycle Logical security Secure data handling Physical security Environmental safeguards Change management Data integrity Incident handling Availability and redundancy

183

184 Default deny firewall

185

186 Security groups

187 Web tier Ports 80 and 443 Open to the Internet Application tier ssh access Engineering team All other ports blocked Database tier

188 DDOS Man in the Middle IP spoofing

189 Data access control

190 Identity and access

191 Resource isolation Virtual Private Cloud

192 Customer s isolated AWS resources Subnet 1 Subnet 2 VPN Gateway Router Secure VPN connection over the internet Subnet 3 Subnet 4 Customer s network Amazon Web Services infrastructure

193 aws.amazon.com/security

194 In summary...

195 Undifferentiated heavy lifting

196 Focus on your stuff

197 From app hosting to analytics

198 Available on-demand

199 Pay as you go

200 Security at every level

201 aws.amazon.com

202 One more thing...

203 FREE TIER

204 One year 5Gb storage 750 hours of EC2 FREE TIER Messaging 3Gb of S3 & SimpleDB For new customers

205

206

207

208 Thank you!

209 QUESTIONS + COMMENTS ON TWITTER