Ahead in the Cloud. with Amazon Web Services. Matt Wood TECHNOLOGY EVANGELIST

Transcription

1 Ahead in the Cloud with Amazon Web Services Matt Wood TECHNOLOGY EVANGELIST

2 Hello.

3 Thank you.

4 Consumer business Seller business

5 Decades of experience Operations, management and scale

6 Programmatic access

7 Unexpected innovation

8 Blinding flash of the obvious

9

10 5 years young

11 Compute Storage Databases Services & Support

12 4

13 Cloud Foundations concepts 4 Maximising value Security in the Cloud

14 1 Cloud concepts Or: five things I wish I d known before getting started

15 1. Signing up

16 On the web

17

18

19 Free tier For new customers

20 750 hours of compute 10Gb network storage Free tier 5Gb Simple Storage Service SimpleDB, queues, notications

21 2. Interacting

22 API driven

23 REST API driven SOAP

24

25 CLI

26 ec2-run-instances

27 ec2-terminate-instances

28 SDK

29 Ruby Java.Net SDK ios PHP Android

30 Management console

31

32 Linux

33 Certificate based root access

34 mza$ ssh -i web/us-east/aws-web.pem Last login: Wed Jun 22 11:15: from _ ) CentOS _ ( / v5.4 \ HVMx64 Welcome to an EC2 Public Image :-) [root@ip ~]#

35 Windows

36 Administrator access

37

38 3. Storage options

39 Ephemeral storage

40 Included with compute Ephemeral storage Lost at termination Not backed up

41 When it s gone, it s gone

42 Elastic Block Store

43 Network attached Mount as volume Elastic Block Store Snapshot Persistent

44 Persistent Root partitions. Mount as volumes.

45 S3

46 Highly durable Highly available S3 Tolerant to two simultaneous failures

47 % durability

48 Objects in S3 Billions of objects 556B Q Q Q Q Q Q3 2011

49 370,000 peak transactions per second

50 4. Payment options

51 Pay as you go

52 Gb/month

53 ECU/hour

54 No minimum

55 No subscriptions

56 Pricing tiers

57 Consolidated billing

58 Options

59 On-demand

60 Reserved capacity

61 Spot instances

62 On-demand: $0.57 per hour

63 Bandwidth

64 Free inbound

65 Reduced outbound

66 Pricing calculator

67 calculator.s3.amazonaws.com/calc5.html

68 5. Availability Zones

69

70

71 us-east us-west eu-west ap-southeast ap-northeast

72 eu-west-1a eu-west-1b eu-west-1c

73 2 Foundations

74 Compute

75 Elastic Compute Cloud

76 Windows + Linux instances

77 Instance sizes

78 High memory Micro High CPU Instance sizes Standard Cluster Compute

79 Dual Nehalem 1.7Tb disk HVM

80 Amazon Machine Image

81 State Config Code Services Amazon Machine Image ami e

82 Pre-configured

83 Custom

84 Private Custom Public

85 Elastic Block Store

86 m1.large

87 m1.large 100Gb

88 m1.large Persistent 100Gb Scalable Elastic block store Snapshot

89 m1.large AMI SNAPSHOT 100Gb

90 Template m1.large AMI AMI SNAPSHOT 100Gb SNAPSHOT

91 Template m1.large AMI AMI SNAPSHOT 100Gb SNAPSHOT Private, stored securely

92 m1.large AMI AMI SNAPSHOT 100Gb SNAPSHOT

93 m1.large AMI m1.large SNAPSHOT 100Gb 100Gb

94 m1.large m1.large m1.large m1.large 100Gb 100Gb 100Gb 100Gb m1.large m1.large m1.large m1.large 100Gb 100Gb 100Gb 100Gb

95

96

97

98

99

100

101

102

103

104

105

106

107 cc1.4xlarge

108 cc1.4xlarge 100Gb

109

110

111

112

113 Mount, and away you go...

114 Oracle

115 Oracle Database 11G Oracle E-business suite Oracle Fusion middleware Oracle Oracle Enterprise manager Oracle Enterprise Linux Oracle on RDS

116 Licence on EC2 Oracle Use existing licences

117 Days to minutes

118 Certified + supported

119 SAP

120 Rapid Deployment Solutions SAP BusinessObjects SAP Licence for EC2 Use existing licences

121 Storage

122 Simple Storage Service

123 Files in directories

124 Objects in buckets

125

126

127 Large objects 5Tb

128 Import/Export

129 S3 websites

130 Databases

131 Flexibility

132 AMI SNAPSHOT Packaged + ready to roll

133 Oracle 11g 32 and 64 bit

134 IBM DB2 32 and 64 bit

135 SQL Server 32 and 64 bit

136 PostgreSQL and EnterpriseDB

137

138 oracle-corporation

139

140

141

142 running

143 Configure, optimise

144 Snapshots + backup

145 Relational Database Service

146 MySQL Oracle Relational Database Service

147 Managed

148 Rapid

149 Scalable storage

150 Scalable instance

151 Synchronous replication eu-west-1a eu-west-1b High availability. Multi-AZ.

152 Asynchronous replication Read replicas

153 Remove the muck

154 Extra services

155 Simple Queue Service

156 Simple Notification Service

157 Elastic MapReduce

158 Input data S3

159 S3 Input data Code Elastic MapReduce

160 S3 Input data Code Elastic MapReduce Name node

161 S3 Input data Code Elastic MapReduce Name node Elastic cluster

162 S3 Input data Code Elastic MapReduce Name node HDFS Elastic cluster

163 S3 Input data Code Elastic MapReduce Name node Queries + BI HDFS Via JDBC, Pig, Hive Elastic cluster

164 S3 Input data Code Elastic MapReduce Name node Output S3 + SimpleDB Queries + BI HDFS Via JDBC, Pig, Hive Elastic cluster

165 S3 Input data Elastic MapReduce Output S3 + SimpleDB

166 Undifferentiated heavy lifting

167 Platform grows with you

168 Premium support Bronze, Silver, Gold, Platinum

169 3 Maximising Value

170 Elasticity

171 Design for elasticity

172 Horizontal scale

173 Everything fails, all the time Werner Vogels

174 Build for failure

175 Build for availability

176 Treat your data as your royal garden

177 Compute as an interchangeable resource

178 Auto-scaling

179 Elastic Load Balancer

180 Choice of instance sizes

181 Elastic by default SimpleDB, SQS, SNS, S3

182 Automatic

183 Applications in the cloud

184 3 tiers

185 Application tier Code Configuration

186 Application tier Code Configuration Operating system Integration settings Service tier Launch configuration Services + configuration

187 Application tier Code Configuration Operating system Integration settings Service tier Launch configuration Services + configuration AMIs Architecture Multi-AZ Infrastructure tier Scaling rules Security groups Middleware

188 Value baked into each tier

189 Value in application

190 Value in service tier

191 Optimisation Configuration Value in service tier Technology choices

192 Value in infrastructure

193 Engine room Optimised Value in infrastructure Scalable Fault tolerant

194 Maximising Automation maximises this value value

195 Automate everything

196 CloudFormation

197 Template

198 Define a full infrastructure stack

199 EC2 SNS Auto-scaling SimpleDB RDS SQS Resources Elastic Beanstalk CloudWatch Security groups Tags

200 Template CloudFormation Provisioned resources

201 Complete definition

202 Atomic

203 Idempotent

204 Free Only pay for the provisioned resources

205 JSON

206 Plain text Perfect for version control JSON Validate-able

207 Declarative language

208 { "AWSTemplateFormatVersion" : " ", } "Description" : "Create an EC2 instances", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" }, "ap-northeast-1" : { "AMI" : "ami-8e08a38f" } } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "AZ" : { "Description" : "Availability Zone of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] } }, "PublicIP" : { "Description" : "Public IP address of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] } } }

209 { "AWSTemplateFormatVersion" : " ", "Description" : "Create an EC2 instances", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" }, "ap-northeast-1" : { "AMI" : "ami-8e08a38f" } } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, Headers Parameters Mappings Resources } "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "AZ" : { "Description" : "Availability Zone of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] } }, "PublicIP" : { "Description" : "Public IP address of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] } } } Outputs

210 Elastic Beanstalk

211 Java web applications

212 Upload WAR

213

214 Your Application Application Service Java Web Platform HTTP Service Tomcat Language Interpreter Java Operating System Linux Server Amazon EC2

215 Best practice

216 Highly available

217 Customisable

218 Flexible

219 Free Only pay for the provisioned resources

220 Monitored

221 Insight

222 CloudWatch

223 Free 5 minute resolution

224 Detailed monitoring 1 minute resolution

225 CPUUtlization DiskReadBytes DiskReadOps DiskWriteBytes DiskWriteOps NetworkIn NetworkOut DatabaseConnections FreeStorageSpace ReadLatency ReadThroughput SwapUsage WriteLatency WriteThroughput

226 Custom metrics

227 Price aware

228 Mix On-demand, Reserved Capacity and Spot

229 Relate to business metrics

230 Cost per user

231 Cost per operation

232 White papers: aws.amazon.com/ whitepapers

233 Premature optimisation

234 Maximise value

235 4 Security in the Cloud

236 Shared responsibility

237 Requirement based access

238 Certification

239 ISO SAS 70 Type II

240 PCI DSS Level 1

241 Control objectives Security organisation Employee lifecycle Logical security Secure data handling Physical security Environmental safeguards Change management Data integrity Incident handling Availability and redundancy

242 DDOS Man in the Middle IP spoofing

243 Data access control

244 Detailed logging

245 Data stays local

246 Identity and access control

247 API level rights management

248 Account

249 Billing Account credentials Account MFA

250 Account DBA Developer Sys admin Finance Roles

251 Account DBA Developer Sys admin Finance Roles Sally Robert Users Chris

252 Security credentials Multifactor authentication Management console access Data read/write access API level access

253 Account DBA Developer Sys admin Finance Roles Sally Robert Users Chris

254 Network isolation

255 Virtual Private Cloud

256 Virtual network topology

257 IP address range Public, private subnets Virtual network topology Route tables Network gateways

258 Network access control

259 Inbound Outbound Network access control S3 access VPN

260 Dedicated instances

261 Public facing website Public subnet

262 Public subnet Network ACLs + security groups Private subnet Multi-tier applications

263 Public subnet Private subnet IPsec VPN On-premise Extend your data centre

264 Private subnet IPsec VPN On-premise Extend your data centre

265 DR

266 Backup to EC2 and EBS DR VM import

267 aws.amazon.com/security

268 aws.amazon.com

269 Thank you!

270 QUESTIONS + FEEDBACK: ON TWITTER