PayPal Delivers World Class Customer Service, Worldwide Greg Gates, VP of Enterprise Ops Engineering Ramki Rosanuru, Sr. Engineering Manager-COE PayPal
PEGA in PayPal Why we choose PEGA? Bridge the gap between business and PD (Discovery - Automate Business Processes) Guided workflows, rich UI to our customer service agents Rapid application development environment Easy to deploy platform Where we are today? Front Office Application Used by10k customer service agents serving customers all over the world Dispute Management System Connecting the customers, processors, acquirers, issuers & merchants to solve disputes PayPal Escalations Team handling all escalation tickets Looking at migrating complex macros to robotics platform so they are organized and managed properly 2
and Consuming PEGA Out of the Box Features All-New Interaction Portal Next Best Action Interaction Portal UX Intelligent Guidance Accessibility Personalized Portal Concurrent Chat Streamline Data Presentment Simplification one tool Robotics Customer journeys Multichannel 3
Enterprise Challenges Proprietary technologies Legacy service integrations (no SOAP or REST) Custom authentication schemes Inhouse messaging system Existing enterprise systems (leverage enterprise time tested systems) Scalable messaging infrastructure Continuous integration & delivery o GIT, Nexus, Service Now, Rally, JIRA o Custom build & deployment Processes Encryption services, key management services 4
Enterprise Challenges PayPal Solution Approach Customization options in PEGA Customize the standard integration points Build & leverage custom Java Libraries using Rule Utility Functions (RUF) RUF-based approach RUF is fundamental to all enterprise integration in PayPal RUFs part of Common Framework Common Framework is built by COE and delivered as a component (Nexus Artifact) The Common Framework is bundled into Application Delivery package and deployed to target systems 5
Integration With Messaging Infrastructure 6
Why We Need Messaging Infrastructure A-sync processing Ability to process outside of PEGA process to improve the performance of the app Ability to process with Guaranteed Delivery (persistence, retry mechanisms) Message broadcast within and outside of PEGA environment Ability to publish/consume messages (both inbound/outbound) Need for caching Ability to cache the data before it is required (triggered from within and outside of PEGA) to be retrieved faster Horizontally scalable distributed system 7
Value-Add Features Built on PEGA Synchronous and asynchronous invocation of services with a change of a flag Just in time cache leveraging lead times Within PEGA application (between Screens) Outside of PEGA (between customer calls, before customer calls) Realtime Data Extraction Leverage declare network of PEGA Compress and publish to be processed outside of PEGA process Reliable & Scalable Batch Processing Distribute load (leveraging messaging) Use PEGA s SLA 8
Data Extraction Framework A Value-add Service Leveraging Messaging Framework 9
Data Extraction Needs Managing CSV file versioning is cumbersome PayPal relies on data replication to offline database for reporting & extraction purposes (Golden Gate) Ever-growing business needs Snapshot of data (via Batch) is not sufficient Challenges with the opening and processing the BLOB Challenges with the data formats (CSV files) Jobs need to be put on hold during deployments 10
What is Data Extraction Framework (DEx) Provides near Realtime Data Extracts Publishes Compressed Workobject Data to a message queue Extraction is processed outside of PEGA application space Extraction can be customized based on Extraction template Leverages Declare Network of PEGA Uses Checksum to ensure the data updates are preserved 11
DEx-Architecture Template Wizard Pega Platform Messaging Framework Extraction Platform Data Warehouse REST Service Declare Trigger De-Compress Trigger Activity Consumers Parse Compress Extract Publish Compass CS App Clustered Q Write to DB PEGA DATA BASE GOLDEN GATE REPLICATION DW DATA BASE 12
Benefits of DEx No Code: Ability to define data needs to be extracted from a work object. Real Time: Export data from work objects in real time Performance: Minimal Load on Runtime PEGA JVMs, extraction processing offloaded to external system. Generic: Will work with any PRPC application. Direct to DB: No CSV files, or other intermediate steps. DDL support: Extraction table SQLs autogenerated. 13
Reliable & Scalable Batch Processing A Value-Add Service Leveraging Messaging Framework 14
Why Reliable & Scalable Batch Processing (RSBP)? Common Batch Framework: Need for having one common framework for both time-based scheduled and on demand batch processing Scalability: Volume of work is highly variable, requiring a scalable solution; PRPC standard agents are not horizontally scalable Performance: PRPC Agents are single threaded so they process events synchronously Reliability: No way to auto restart the agent as soon as the agent goes down Resiliency: Solution must provide high availability and must be resilient if the processing for a task fails 15
RSBP Architecture SLA Agent REST Batch Publish SLA Agent Adv-Agent Batch Publish Messaging Framework (GEMS) Clustered Q Consumer Load balancer REST PEGA DATABASE Batch REST DATA BASE 16
Benefits Better scalability since both Publisher and Subscriber are horizontally scalable More than 50% performance improvement Workload is distributed - tasks can run asynchronously Tasks are independent and can run in parallel No contention during processing Leverages OOTB resources for processing SLA items 17
Hydra Just in Time Caching A Value-add Service Leveraging Messaging Framework 18
Consume Invoke Hydra Services Just in Time Caching Event-based Cache Creation Publish GEMS-HYDRA Cache Unified Service Invocation Internal, external REST, Payapal Custom Configuration-based Event Definitions Service Orchestration Sources Cache Store Standard Event Taxonomy Applications (PEGA) 19
Service Invocation Framework Unified Service Invocation Model 20
Why SIF? Need for integrating with proprietary technologies Support/adhere to PayPal security & invocation standards Need for Asynchronous and Parallel Invocation Need for pre-fetching capabilities and caching Support Mockup for Unit Testing Automation Integration with external systems (BING, Salesforce etc ) Need for integrating with monitoring framework to get complete aggregation of all the calls made with different metrics 21
SIF Architecture Pega Platform Application Layer GET POST DELETE PATCH PUT Security External Integration Monitoring Asynchronous Processing Parallel Processing Pre-Fetch Couchbase AdminAuthServ RiskAdminServ PayPal Domain Services Messaging Framework (GEMS) Bing Salesforce External Services PaymentServ CiproAttack Clustered Q Consumer Other Services 22
Benefits Support for GET, POST, PUT, DELETE and PATCH methods Provides abstraction to PayPal s security level nuances Improved performance Leveraged across Java platforms Provides asynchronous processing capabilities Provides parallel processing, pre-fetch processing Supports External Service Invocation Integrates seamlessly with monitoring framework 23
Embracing PayPal Security A Common Framework 24
Common Security Framework SSO OAuth Security Utilities Encryption Decryption Hashing Masking Integration with Key Management System 25
Application Stack 26
Integration With Productivity Tools A Value-add Service Leveraging Messaging Framework 27
Why Productivity Tools integration Continuous Delivery for faster and stable releases Streamlined and consistent processes Automation, eliminate manual-repetitive tasks to improve efficiency Easy to use external systems and processes Developer productivity Better insights for management into release progress 28
Integrations for Continuous Delivery Initiate DEV / FQA RQA GBC Go Live Create Development Branches Create CI jobs / pipeline GIT (SVN in past) Jenkins Enterprise Integrations List SVN GIT Jenkins PEGA 29
Integrations for Continuous Delivery Initiate DEV / FQA RQA GBC Go Live Check-out/Check-In : JAVA code GIT (SVN in past) Execute CI pipeline Capture DEV / QE effort (User Stories) Publish finished code for builds / CI Jenkins Rally Nexus Enterprise Integrations List SVN GIT Jenkins Nexus JMS Queues PEGA Async Tasks (Code Coverage / Reviews) JMS Queues / WL 30
Integrations for Continuous Delivery Initiate DEV / FQA RQA GBC Go Live Check-out/Check-In : JAVA code GIT (SVN in past) Execute CI pipeline Capture DEV / QE effort (User Stories) Publish finished code for builds / CI Async Tasks (Code Coverage / Reviews) Jenkins Rally Nexus JMS Queues / WL Enterprise Integrations List SVN GIT Jenkins Nexus JMS Queues **JIRA** **Service Now** PEGA Release Readiness (Tickets & Sign offs) JIRA/ServiceNow* 31