Analytics"in"the"Cloud"from"AWS

Transcription

1 Analytics"in"the"Cloud"from"AWS January'10,'2017 "2016,"Amazon"Web"Services,"Inc."or"its"Affiliates."All"rights"reserved. AWS GovCloud (US)

2 Global&Infrastructure

3 Global&Infrastructure&of&AWS 17* Regions 44 Availability'Zones 68 Edge'Locations AWS GovCloud (US) Region &.Number.of.Availability.Zones AWS.GovCloud (2) EU *C2S.(2) Ireland'(3) US West Frankfurt (2) Oregon (3) London'(2) Northern California'(3) US.East Asia.Pacific N. Virginia'(5),'Ohio'(3) Singapore'(2) South America Sydney (2),'Tokyo'(3), São'Paulo'(3) Seoul (2),'Mumbai'(2) Canada Central'(2) China Beijing'(2) Announced.Regions Paris,'Ningxia

4 AWS&GovCloud&(US)&is&an&isolated&AWS&region Intended&for&customers&with&strict&regulatory&and&compliance& requirements&and&sensitive&data&or&workloads August"2011 Available'to'qualified'customers Compliance Safeguard'sensitive'data/systems Addresses'multiple'US'Government'regulations'and'security'requirements

5 AWS&GovCloud&(US)&distinguishing&features Data,'network,'and' machine'isolation' from'other'regions Separate'IAM'(unique' credentials) Dedicated'GovCloud' Management'Console Managed'by'US' Persons'on'US'soil Community'Cloud 'with' vetted'account'holders

6 Requirements&for&access&to&AWS&GovCloud&(US) Account&holder&must&be&a&US&Person& (defined&as&a&us&citizen&or&a&green&card&holder) US&entity&incorporated&to&do&business&in&the&United& States&and&is&based&on&US&soil Can&handle&export&control&data Learn.more:'

7 AWS&GovCloud&(US)&Summary AWS&GovCloud&(US)&is&physically&and&logically&isolated Separate&AZs,&Management&Console,&IAM&stack,&and&service&endpoints AWS&GovCloud&(US)&is&not&just&for&the&US&Government Available&to&qualified&contractors&and&organizations&in&regulated&industries Remember&the&AWS&Shared&Responsibility&Model Leverage&services&like&VPC,&IAM,&KMS&and&CloudHSM&to&secure&sensitive& workloads&and&manage&access&to&sensitive&data AWS&GovCloud&(US)&webpage:& AWS GovCloud (US)

8 IC'ITE'Commercial'Cloud:'AWS'C2S C2S'brings'Amazon'Web'Services'(AWS)' over'the' fence 'and'into'the'ic.'' C2S'is'an' air`gapped 'region'that'utilizes'the'same' innovation'as'aws'commercial'regions.'' AWS'will'bring'in'cutting'edge'products'and'services' for'you'to'utilize'as'technology'grows'and'changes.' And'you'only'pay'for'what'you'use.

9 Compute,&Analytic&and&Big& Data&Services

10 Compute:&EC2&GPU&and&FPGA&Instance&Types G2'` optimized'for'graphics`intensive'applications. P2'` designed'for'general`purpose'gpu'compute'applications'using' CUDA'and'OpenCL Newly.announced Amazon'Ec2'F1'Instances` Custom'FPGAs Amazon'EC2'Elastic'GPUs'for'attachable'graphics'Acceleration

11 AWS'Lambda:'Serverless event'processing Serverless'compute'service'that'runs'your' code'in'response'to'events. AWS'Lambda Extend'AWS'services'with'user:defined' custom'logic. Pay'only'for'the'requests'served'and' compute'time'required': billing'in' increments'of'100'milliseconds'

12 AWS&Lambda Take&advantage&of&new&computing&models Serverless architectures&free&you&from&managing&infrastructure Develop&without& worrying&about&servers& to&manage Setup&your&code&to&run& in&response&to&a& variety&of&triggers Code&is&executed&on& demand&with& continuous&scaling Pay&only&for&the&time& used&with&subqsecond& metering lets%you%run%code%at%cloudfront edge%locations%without%provisioning%or%managing% servers.%this%allows%developers%to%deliver%a%low%latency%user%experience%for%customized%web%applications.

13 Amazon&Kinesis:&streaming&data&made&easy Services&make&it&easy&to&capture,&deliver,&and&process&streams&on&AWS Kinesis"Streams For$Technical$Developers Kinesis"Analytics" For$all$developers,$data$scientists$ Kinesis"Firehose For$all$developers,$data$scientists$ Collect&and&stream&data& for&ordered,&replayable,& realqtime&processing Easily&analyze&data&streams& using&standard&sql&queries Easily&load&massive& volumes&of&streaming&data& into&s3,&amazon&redshift,& or&amazon&es

14 Amazon'Elasticsearch Service Semi:structured'/'unstructured'data'processing' Amazon' Elasticsearch' Service Distributed'search'and'analytics'engine Managed'service'using'Elasticsearch'and'Kibana Fully'managed': zero'admin Highly'available'and'reliable Tightly'integrated'with'other'AWS'services

15 ETL&with&AWS&Data&Pipeline

16 AWS&Glue Glue'automates'the'undifferentiated'heavy`lifting'of'ETL! Cataloging&data&sources&! Identifying&data&formats&and&data&types! Handling&errors! Managing&and&scaling&resources! Generating&Extract,&Transform,&Load&code! Executing&ETL&jobs;&managing&dependencies

17 AWS.Glue.Components Data.Catalog " Hive'metastore'compatible'metadata'repository'of'data'sources.' " Crawls'data'source'to'infer'table,'data'type,'partition'format.' " Generates'Python'code'to'move'data'from'source'to'destination.' " Edit'with'your'favorite'IDE;'share'code'snippets'using'Git. Job.Authoring Job.Execution " Runs'jobs'in'Spark'containers automatic'scaling'based'on'sla. " Glue'is'serverless'` only'pay'for'the'resources'you'consume.'

18 Elastic&MapReduce&(EMR) Reduces'complexity'of'big'data'processing Managed'Hadoop'2.7'and'2.4'infrastructure Handles'node'provisioning,'customization,'and'shutdown Provides'tools'to'debug'and'monitor'your'Hadoop'clusters Provides'tight'integration'with'AWS'services'(S3,'DynamoDB) 18

19 Amazon&Machine&Learning Easy`to`use'service'built'for'developers Robust,'powerful,'and'technology`based Ability'to'create'models'using'your'data Deployable'to'production'in'seconds

20 Amazon'QuickSight Visualize)and)explore Amazon' QuickSight Fast,'powered'by'SPICE,'automatically'scales.' Explore,'analyze,'share'insights'with'anyone. 1/10th'the'cost'of'traditional'BI'solutions. Broad'connectivity'with'AWS'data'services,'on: premises'data,'files'and'business'applications. Amazon"RDS Amazon"S3 Amazon"Redshift

21 Amazon&Athena:&Serverless Query&Processing&with&S3 Serverless'query'service'for'querying'data'in'S3'using'standard'SQL'with'no' infrastructure'to'manage No'data'loading'required;'query'directly'from'Amazon'S3 Amazon Athena Use'standard'ANSI'SQL'queries'with'support'for'joins,'JSON,'and'window' functions Support'for'multiple'data'formats'include'text,'CSV,'TSV,'JSON,'Avro,'ORC,' Parquet Pay'per'query'only'when'you re'running'queries'based'on'data'scanned.'if'you' compress'your'data,'you'pay'less'and'your'queries'run'faster

22 Other&re:Invent 2016&Announcements Amazon'Lex` build'conversational'interfaces'using'voice'and'text Amazon'Polly` turn'text'into'lifelike'speech Amazon'Rekognition` add'image'analysis'to'your'applications AWS'Batch` Enables'users'to'easily'and'efficiently'run'hundreds'of' thousands'of'batch'computing'jobs'on'aws Amazon'Lightsail` deploy'a'virtual'private'server'using'preconfigured' templates AWS'Greengrass` run'compute,'messaging'&'data'caching'for'connected' devices'in'a'secure'way,'even'when'not'connected'to'the'internet. AWS'Snowball.Edge` 100TB'data'transfer'device'with'on`board'storage'and' compute AWS'Snowmobile` Migrate'data'securely'via'a'truck'storing'up'to'100'PB'of' data

23 Implementing'a'Data'Lake'on'AWS Elasticsearch

24 AWS&and&IC&Marketplace 3,500+'software'listings Over'1,100'participating' ISVs Open'source'and' commercial'software'' Try'before'you'buy,' experiment'cheaply Procure'new'or'Bring`your` own`license Deployed'in'Commercial' and'c2s'regions 24

25 AWS'Commercial'Platform'Services Compute Networking Analytics EC2 EC2"Container" Elastic" Lambda Service Beanstalk VPC ELB Direct" Connect Route"53 EMR Kinesis Data"Pipeline Machine" Learning QuickSight Elasticsearch Service Developer&Tools Management&Tools Security&&&Identity CodeCommit CodeDeploy CodePipeline CloudWatch Cloud Formation CloudTrail Config OpsWorks Service" Catalog Identity"&" Access"Mgmt Directory" Service Trusted" Advisor Cloud"HSM Key" Mgmt"Service Web"App" Firewall Storage&&&Content&Delivery Application&Services Hubs S3 EBS Storage" Gateway Glacier EFS Snowball CloudFront API"Gateway AppStream CloudSearch Elastic" Transcoder SES SQS SWF SNS Mobile"Hub Mobile&Services Cognito Device"Farm Mobile" Analytics Database IOT Enterprise&Applications DB"Migration" RDS DynamoDB ElastiCache RedShift Simple"DB IoT WorkSpaces WorkDocs WorkMail Service Interaction AWS"Mgmt" Console Command" Line"Interface Marketplace Libraries&and&SDKs AWS"Marketplace Java Javascript Python (boto) PHP.NET Ruby Node.js ios Android Toolkit" for"vs Toolkit" Tools"for" AWS"CLI for"eclipse Powershell

26 AWS&GovCloud&(US)&Services Kinesis'Streams

27 Requesting&New&C2S&Services C2S&page&on&IC&ITE&home&page: C2S&Roadmap& Service&launch&announcements New&Service&Requests: On&the&right&side&of&the&C2S&page,&click& Request&New&Service Fill&out&the&form&with&POC&info,&list&of&new&services&or&features& requested,&estimated&users&and&usage,&and&potential&useqcase 27

28 Cloud"computing"has"become"the"new"normal Deploying&new& applications&to&the& cloud&by&default Migrating&existing& applications&as& quickly&as&possible Public&Sector&and&especially&the&US&IC&is&moving&to&AWS. AWS&services&more&than&a&million&active&customers&every&month&in&190&countries. Through&customer&obsession&and&an&increasing&pace&of&innovation,& AWS&is&the&fastest&technology&vendor&in&history&to&reach&a&$10B&run&rate.