Index. Big Data analytics, , 355 BigQuery ad-hoc system, 236 Apache access log files AccessLog declaration, 240 commas as delimiters, 242

Size: px

Start display at page:

Download "Index. Big Data analytics, , 355 BigQuery ad-hoc system, 236 Apache access log files AccessLog declaration, 240 commas as delimiters, 242"

Earl Tyler
5 years ago
Views:

1 Index A Access control lists (ACLs) default/predefined, permissions/scopes, Android key, 29 Apache access log files AccessLog declaration, 240 commas as delimiters, 242 CSV, 241 CustomLog directives, 239 environment variable declaration, 240 Github code-hosting platform, 242 JSON-encoded text file, 241 loading transformed data, location, 241 log entry, querying data, replacing tabs with string TAB, 242 tab characters as delimiters, 242 apply method, 271 Auth API keys, OAuth 2.0 (see OAuth 2.0) Autoscaler autoscaled managed instance groups, 81 autoscaling integrations, 77 gcloud CLI tool, instance template creation, 78 managed instance group, 76, 79 target utilization level, 77 utilization metric, 76 AvroIO class, 265 B Big Data analytics, , 355 BigQuery ad-hoc system, 236 Apache access log files AccessLog declaration, 240 commas as delimiters, 242 CSV, 241 CustomLog directives, 239 environment variable declaration, 240 Github code-hosting platform, 242 JSON-encoded text file, 241 loading transformed data, location, 241 log entry, querying data, replacing tabs with string TAB, 242 tab characters as delimiters, 242 columnar layout and distributed storage, 236 containers, 237 creating views, 252 data types, 238 date/time string, 238 Dremel research technology, 236 exporting data, importing data, interactive system, 236 petabytes size, 236 scalability, 236 shared multi-tenant architecture, 237 tree architecture, 236 Unix timestamp, 238 BigQueryIO class, 265 Bigtable, 211 Browser key, 29 C Capital expenditures (CapEx), 6 Cloud Dataflow construction access.log file, 267 AllowedOptions, input and output command, 268 PipelineOptions, 267 data sources and sinks AvroIO class, 265 BigQueryIO class, 265 Cloud Pub/Sub messages,

2 index Cloud Dataflow (cont.) PCollection objects, 264 TextIO class, 264 execution Cloud Storage, 268 Count transform, 270 ExtractLogExperience, 269 filtering operation, 269 FormatResultsFn, 272 options and application, 275 runner parameter, TopCodes, 271 Extract Logs UX, 268 PCollection, 259 pipelines, 255, 258 setup, 256 streaming data channels, 255 transformation (see Transformation) Cloud Datastore indexes autogenerate property, 222 Bigtable, 220 configuration file, 221 cost optimization, 229 datasets cursors, data repositories, 227 LIMIT, 227 OFFSET, 227 exploding, filtering, 229 index.yaml, 222 properties, 220 queries, querying ancestor, 220 and SDK, 221 sorting, 230 Cloud deployment manager configuration, 333 beta deployment-manager types, 337 boot images, 339 compute zones, 338 machine types, 338 resources property, 337 deployments management, 334, environment variables, 341 manifest, 334 parameters, 340 references, resources, 333 single compute engine VM, templates, 334, 339 Cloud domain name system (DNS) benefits, 295 change requests, 296 entities, 296 gcloud tool, 306 publishing internet identity adding IPv4 and IPv6 addresses, 302 adding multiple record sets, 303 creating managed zone, listing Cloud SQL instances, 302 listing DNS records, , 302, 306 open transaction, 303 PTR record, 305 rdns, 305 SPF records, 303 transaction object, TXT record, 304 TTL, 296 Cloud endpoints API, api_server, APIs explorer, 324 app.yaml, 311 client libraries generation authentication process, 328 endpointscfg.py, 326 fetchmeetingslist, 329 GoogleAccountCredential, 328 gradle module, 327 onactivityresult, 329 EndpointsAliasProperty, 317 NDB Datastore, 314 protorpc, 309 SDK, 310 services/request handlers EndpointModels.query_method, endpoints.method, 319 EndpointsModel.method, Cloud management system (CMS) autoscaler, 352 cloud launcher advantages, 351 click-to-deploy, 350 cloud DNS, 351 disadvantages, 351 WordPress, 350 cloud SQL, 351 definition, 349 health checks, 352 HTTP load balancer, 352 Cloud platform resources cloud client libraries, APIs, Developers Console permissions and auth, web-based interface, gcloud Tool, SDK, 20 projects, regions, zones, resources, and quotas,

3 Index Cloud publisher-subscriber approach, 265 lifecycle, 278 messages, processing messages pull subscriptions, push subscriptions, project and topic identifiers, 279 subscriptions acknowledgement deadline, pull, push, Python client library, 283 system, set up, 278 Cloud storage access control, ACLs (see Access control lists (ACLs)) advantages, 188 API access, 186 applications, 189 automation, 209 billing, 186 gcloud tool, 186 gsutil bucket creation, 187 cleaning, 188 execution, 187 installation, 185 uploading files, 187 handling errors, 200 lifecycle management, resumable upload, signed URL gsutil command-line tool, 197 signature, structure, strong consistency, 189 Comma-separated values (CSV), 241 Compute Engine autoscaler autoscaled managed instance groups, 81 autoscaling integrations, 77 gcloud CLI tool, 80 instance template creation, managed instance group, 76, target utilization level, 77 utilization metric, 76 global multi-datacenter web tier Asian nodes, 75 backend services, 70 content-based load balancing, 75 gcloud commands, global forwarding rules, 70 health check, 73 HTTP load-balancing service, 72 instance groups, 71 Linux commands, 71 load-distribution algorithm, 71 target HTTP proxy, 70 URL maps, 70, 76 load balancer definition, 65 forwarding rules, 65 HTTP load balancing, 65 load-distribution algorithm, 66 network load balancing, 65 target pools, PD, 55 VM (see Virtual machine (VM)) Container engine, Container_exec tool, 152 Core transformations combine, 263 Flatten, 264 GroupByKey, 263 ParDo, 262 D DevOps initiatives Container Engine cluster, 134 firewall rule, 137 gcloud command, 137 Kubernetes pod, 136 node, 134 Single Node Cluster, Wordpress installation, containers computational resources, 125 hypervisor models, LXC, 126 namespaces, 126 operating system level virtualization, 126 Docker client, 127 containers, 128 daemon, 127 hub, 127 images, 127 Kubernetes DNS, 134 Docker model, 132 Kubernetes cluster manager, 129 Kubernetes model, 132 labels and annotations, 131 namespace, 133 orchestration system, 129 pods, 129 replication controller, 130 service, 133 VolumeMounts property,

4 index DevOps toolbox cloud deployment manager (see Cloud deployment manager) hosted security services cloud debugger, 347 cloud security scanner, 347 cloud trace, 347 source and binary code management cloud repositories, 345 Google Container Registry, 347 Push-to-Deploy, 346 source code tools, 346 Docker container technology, 354 Domain name system (DNS), 295 E, F EndpointsAliasProperty, 317, 323 G gcloud bq command, 238 gcloud command-line interface (CLI), 160 GetResponseCodeFn class, 269 get_result() method, 97 Google APIs access control, 27 Auth (see Auth) OAuth 2.0, 27 OpenID Connect, 27 Translate API (see Translate API) Google App Engine (GAE), 361 api_version, 85 application outputs errors/uncaught exceptions, 86 deployment, 120 , handlers host, 85 integrated frameworks and libraries, key properties and ancestor paths batching and asynchronous operations, 96 Cloud Datastore, 95 meeting request, 95 ndb.keyproperty, 95 queries, request routing, transactions, 102 logs, 120 Memcache, Python 2.7, 85 scheduled tasks cron configuration, 119 parameters, 118 periodic format, 119 specific timing format, 119 SDK, 83 storing information additional arguments, 93 BlobKeyProperty, 91 BlobProperty, 90 BooleanProperty, 90 ComputedProperty, 92 data models, 90 DateProperty, 91 DateTimeProperty, 91 DynamicThing class, 93 Expando models, 93 file structure, 89 FloatProperty, 90 GenericProperty, 92 GeoPtProperty, 91 Google Cloud Datastore, 89 Google Cloud SQL, 89 Google Cloud Storage, 89 IntegerProperty, 90 JsonProperty, 92 KeyProperty, 91 LocalStructuredProperty, 91 PickleProperty, 92 StringProperty, 90 StructuredProperty, 91 TextProperty, 90 TimeProperty, 91 task queues administration console, 115 deferred tasks, deleting tasks, 116 handling tasks, push queues, threadsafe, 85 user management, 103 version number, 85 web applications, 84 YAML configuration files, 84 Google BigQuery. See BigQuery Google Cloud Datastore ancestor paths, availability, 212 cross-group transactions, 219 eventual consistency, 217 identifier, 216 indexes (see Cloud Datastore indexes) keys App Engine Python NDB, 215 entity retrieval, locality, 218 Node.js (JSON) API, 215 path, 216 protocol buffers, 215 transactionality,

5 Index limitations, 211 for LunchMate app, 213 model.py, 213 operation speed, 212 pricing, 231 properties, 213 scalability, 211 strong consistency, 217 test case, 219 Google Cloud DNS, 362. See also Cloud domain name system (DNS) Google Cloud Platform Amazon Web Services, economic innovations, 10 hardware innovations, 8 software innovations, 8 10 Google Cloud SQL, 362 features, 159 master Cloud SQL instance configuration steps, 174 my.cnf, 178 MySQL server software, 175 MySQL slave status, 179 MySQL Users, 178 mysqldump, 177, 179 server-id, 179 MySQL database (see MySQL database) performance-measuring, 181 Google Cloud Storage, 362 Google Compute Engine, 361 Google Container Engine, 362 Google Translate, 41 Gradle, 257, grep, 166 H Health-check requests, 154 I, J ios key, 29 IP-per-pod model, 132 K Keys-only query, 229 Kube-proxy, 133 Kubernetes container technology, 354 L Linux containers (LXC), 126 LunchMates application advantages, 360 architectures, 358 cloud datastore, 359 features, 358 Memcache, 359 networking libraries, 360 requirements, 358 scalability, 358 M main method, 272 Minimum viable product (MVP) approach, 355 Mobbo application, 356 MySQL database Cloud SQL instance binary logs, 170 database backup, 169, 171 gcloud CLI tool, 160 installation, 164 IPv4 Address, 162 read replica, 169, 171 SSL connection, 165 Wordpress CMS, 172 mysqldump, 177 MySQLslap, N named() method, 259 Namespaces, 126 Network address translation (NAT) router, 247 O OAuth 2.0. access request procedure, 30 application authentication, 31 user authentication client_secrets_json_path, 35 discovery classes, 36 manual access token, 36 parameters, 35 redirect, 36 refresh_token,

6 index OAuth 2.0. (cont.) 3-legged OAuth 2.0 user authentication flow, legged, urn\:ietf\:wg\:oauth\:2.0\:oob, 36 urn\:ietf\:wg\:oauth\:2.0\:oob\:auto, 36 OpenID Connect, 27 Operational expenditures (OpEx), 6 P, Q PARSE_IP function, 247 Persistent disk (PD), 55 Pointer record (PTR), Private IP addresses, 247 processelement method, 270 Projection queries, 230 Public cloud business benefits growth patterns, 7 pay-per-use business model, 7 self-service, 7 time to market, 6 economic benefits CapEx, 6 economies of scale, 5 OpEx, 6 ROI, 6 TCO, 5 technical benefits expertise, 5 resource utilization, 5 uptime, 4 put() method, 97 R RESTful services, 25 Return on investment (ROI), 6 Reverse DNS lookup/reverse DNS resolution (rdns), 305 Round-robin method, 66 S Sandboxed runtime, 354 Scalability, 7 Sender Policy Framework (SPF), 303 Server key, 29 ssl-certs describe command, 165 T, U Tablets, 212 TextIO class, 264 Three-legged authentication (3LO), 19 Time to live (TTL) parameter, 296 Total cost of ownership (TCO), 5 Transformation core transformations combine, 263 Flatten, 264 GroupByKey, 263 ParDo, 262 Count transform, 270 Developers Console, 260 ExtractLogExperience, 262 single transformation, 261 Top transform, 271 Translate API access batch query and response, 43 HTTP GET method, 41, 43 HTTP POST method, 43 HTTP query, 42 HTTP response, 42 phrases, 41 query words, 42 translate method, 41 translation request without source language, 42 using client programs, 47 discovering languages English sentence, 46 language query response, 44 retrieving language codes, 44 source language detection, 45 Spanish input, 46 specified target language, 45 URI template, 44 web-based Developers Console, 41 2-legged OAuth 2.0 user authentication, V, W, X, Y, Z Virtual machine (VM), 139 automatic scaling, 153 configuration elements, 154 CPU cores, 53 definition, 54 Docker containerization processes,

7 Index features, 139 firewall configuration, 153 gcloud command-line tool Apache web server, 64 command sequence lists, 59 default specifications, 60 external IP address, 64 firewall rules, 62 Google authentication screen, 58 Google login screen, 57 ingress network access, 60 instance creation, 59 instance deletion, 65 long URL and code, 59 project selection, 59 zone flag, 61 health-check requests, 154 instance types, 54 load balancer forwarding rule, 69 health checks, 66, 68 index.html file, 67 new instance creation, 66 target pool, web browser, 67 networks and firewalls, 55 resource settings control, 153 sandboxed App Engine, 143 sandbox environment, 139 SSH debugging, 149 standard and custom runtime, 140 <standard-runtime> options,

Developing Solutions for Google Cloud Platform (CPD200) Course Agenda

Developing Solutions for Google Cloud Platform (CPD200) Course Agenda Module 1: Developing Solutions for Google Cloud Platform Identify the advantages of Google Cloud Platform for solution development