Intro Mount Vernon Data Systems LLC - incorporated Golden, CO - professional services, remote & on-site - database administration - database

Transcription

1 Intro Mount Vernon Data Systems LLC - incorporated Golden, CO - professional services, remote & on-site - database administration - database architecture - data modeling - database training 1

2 What Are We Doing Today? CRM/ERP PLATFORM MIGRATION swapping one biz-level applicationpackage for another Define the situation what is involved with this application/data swap? Determine the framework what kind of methodology are you going to use? Walk through the steps what can you expect duringthe lifetime of this project? Evaluate the results lessons learned 2

3 WHY DOES IT MATTER? You must know the boundaries of the CRM/ERP package in order to be successful. From Wikipedia: Customer relationship management (CRM) -manages a company s interaction with current and future customers. Enterprise resource planning (ERP) is a category of business-managementsoftware a suite of integrated applications that an organization can use to collect, store, manage and interpret data from many businessactivities, including: product planning, cost; manufacturingor service delivery; marketingand sales; inventory management; shipping and payment. ERP provides an integrated view of core business processes, often in realtime, using common databasesmaintained by a database management system. ERP systems track business resources cash, raw materials, production capacity and the status of business commitments: orders, purchase orders, and payroll. The applications that make up the system share data across various departments (manufacturing, purchasing, sales, accounting, etc.) that provide the data. [1] ERP facilitates information flow between all business functions, and manages connections to outside stakeholders. [2] A data architectis a practitioner of data architecture, an information technology discipline concerned with designing, creating, deploying and managing an organization's data architecture. Data architects define how the data will be stored, consumed, integrated and 3

4 managed by different data entities and IT systems, as well as any applications using or processing that data in some way. [1] It is closely allied with business architectureand is considered to be one of the four domains of enterprise architecture(business, data, applications and technology). Enterprise architectsare practitioners of enterprise architecture; an enterprise strategic managementdiscipline that operates within organizations. Enterprise architects work with stakeholders, both leadership and subject matter experts, to build a holistic view of the organization's strategy, processes, information, and information technologyassets. 3

5 Forplatform migrations there MUST BE a solid framework underlying the project, or the project will probably fail. Framework v. Methodology An architecture frameworkprovides principles and practices for creating and using the architecture description of a system. It structures architects' thinking by dividing the architecture description into domains, layers or views, and offers models - typically matrices and diagrams - for documenting each view. a software development methodology is a splitting of software development work into distinct phases (or stages) containing activities with the intent of better planning and management. It is often considered a subset of the systems development life cycle. Highlevels of organization will be needed for a project like this. The more moving parts, the more organization will be required. Survey of Architecture Frameworks counts no less than 69 distinct frameworks! 4

6 How s the project coming along? Still waiting for you to give me the specs so I can start. I told you it s a cloud app that does data! What more do you need? 5

7 The Zachman Framework WHAT, HOW, WHERE, WHO WHEN, WHY, from 6 different perspectives, from top-level strategists to front-line tactical workers. What is it? How does it work? How do you use it for this kind of project? The Zachman Framework, an enterprise ontology, a fundamental structure for Enterprise Architecturewhich provides a formal and structured way of viewingand defining an enterprise. Go to for more information. Ontology: a two-dimensional classification schema that reflects the intersection between two historical classifications, the primitive questions of What, How, When, Who, Where, and Why, and the reification, or transforming the abstract idea into reality. The Zachman Framework reification transformations are: Identification, Definition, Representation, Specification, Configuration and Instantiation. [1] The Zachman Framework is not a methodologyin that it does not imply any specific process for collecting, managing, or using the information that it describes.; [2] rather, it is an ontology whereby a schema for organizing architectural artifacts (in other words, design documents, specifications, and models) is used to take into account both who the artifact targets (for example, business owner and builder) and what particular issue (for example, data and functionality) is being addressed. [3] The framework is named after its creator John Zachman, who first developed the concept in the 1980s at IBM. It has been updated several times since. [4] 6

8 The 7D Method - Discover, Design, Develop, Deploy, DayToDay, Defend, Decommission What is it? A METHODOLOGY, a way to converting a framework into action. How does it work? - step-by-step, or -iterate at each step, or - a combination of step-by-step and iteration. How do you use it for this kind of project? Let s walk through the steps and see. TO GET A COPY GO TO it s a free download. 7

9 IF the project involves an ERP, which reaches into multiple organizational functions (customer-relationship mgmt., sales & marketing, financials, manufacturing & production, distribution, HR/payroll), you should employ both a framework and a methodology. IF your org functions are a set of silos, and/or you re replacing one silo, then you may di well using just a methodology as long as the target platform vendor has a solution for integrating with other org functions, such as Financials. Shooting from the hip does not qualify as either a framework or a methodology. Embed the relevant questions and strata from the Zachman Framework into the steps of the 7D Method Always ask WHAT, HOW, WHERE, WHO, WHEN, WHY at each stage of the 7D Method Zachman: WHO? will be on the stakeholder team, key players in the project? - Executive Sponsor - Biz Decision Makers - Platform (hardware, software) - Project Manager - Data Governance Rep - User Community Reps - System admin, Platform admin, Network admin, Security admin, Database Admin WHAT? will each team member s assignment be, comfortable, competent, capable & reliable? 8

10 - At what point in the project timeline will they be involved? Will they stay involved for the duration of the project? WHEN? does the project have to be completed? Work backwards - Determine requirements - Determine who will do what tasks - Determine offline estimates and best windows of opportunity - Determine when the upgrade/migration/go-live will happen - Determine the rollback plan, just in case WHERE? Are the new platforms for the upgraded/migrated SQL Server databases - Are the testing facilities for the upgrade and migration process - Are the new software licenses HOW? Will the project be managed? Will the risk be managed? Will the in-house staff be managed? Will the client community be managed? WHY? All these questions? - Because failure of the rollout will cost time/money/reputation and jobs - Lack of knowing the answers to these questions is an early-warning flag that the project is at risk! 8

11 Here s an example of a migration project which had an insufficient plan I joined the project an ERP migration when it was 4 months into a 5-month planned timeline. The data architect, who was responsible for identifying, extracting and transitioning the data from source to target systems, had just left the company.. on top of having to replace the data architect, the requirements had not been fully flushed out, causing much chaos and decision reconsiderations throughout the duration of the project. 1. Two extra months added to the project timeline (net result = bad, very costly) 2. Project manager on the vendor side changed mid-way through the project (net result = good) 3. Data requirements had not been locked down early in the project (net result = very bad new requirements continuously added up until 72 hours before the go-live data extract) 4. We got it done thru great teamwork & lots of late nights/weekends (net result = good) 9

12 1D: DISCOVER The Team who will be on the team, what parts will they play? The Data - which data is going to be migrated from the old to the new, which will be left behind? The Requirements what are they? Get them in writing, secure buy-in from all stakeholders. The Process how will it work? The Infrastructure you ll need access to both the old and the new servers, databases, and other data sources New infrastructure, upgraded software often provided by target vendor 10

13 1D: DISCOVER It s not unusual to discover that the data is stored in a variety of places in a variety of databases, SQL Server instances, file systems. Locate the data sources Identify the data at each source Determine if the data will be migrated or left behind Ensure you have full access to the data 11

14 2D: DESIGN Transfer Templates usually provided by the Target vendor helps define which data can be migrated and which cannot <WAR STORY> non-profit decided to migrate their cloud-based CRM to Microsoft CRM4M. - Cloud vendor (source) would not allow access to the raw data; would only deliver data in a set of spreadsheets, without documenting metadata, constraints, etc. 12

15 2D: DESIGN One strategy (this worked.) Import the data into SQL Server, creating a local source environment Use data modeling software to reverse engineer the data model, analyze for missing components Concurrently, attach a test copy of the UI to the newly-created source database(s) Test thoroughly Identify missing pieces Go back to the source vendor with request for missing data, negotiate additional data extracts, to compensate for their negligence. 13

16 3D: DEVELOP - Start writing code and building transforms - Data extracts, negotiate as many as possible - Make sure there s a test system for the users to review the data every time - Constant review and revise 14

17 3D: DEVELOP - User review & acceptance testing must be engineered into the process from the very onset of the project - Test UI - Minimally need screen shots, videos, visuals with call-outs - Optimally a working prototype which you can populate with real data WAR STORY: users not having a chance to see real data in a test environment until 1 month before go-live - rework of data requirements - additional data requirements - UI re-work - Functional changes in the UI - Additional delay on go-live - Greater cost & increased user frustration 15

18 3D: DEVELOP This is an iterative process 16

19 4D: DEPLOY -Go-live is a full TEAM effort - Final data extract (that s you, data architect!) starts the process - May be able to keep the source database as read-only during this deployment 17

20 5D: DAY TO DAY Post-Go-Live User training, user acceptance on-going Setting up database maintenance Determine RTO/RPO (recovery time/recovery point objectives) Use your own scripts, techniques Ola Hallengren s SQL Server Maintenance Solution ( Begin performance metrics collection & analysis Monitoring software Write your own Store & report NOTE: recent experience with commercial ERP pkgs, SQL Server maxdop must be set to 1. Further investigation showed that vendor was generating identity value from within the software, not using SQL Server identity property. Vendor did not share this information willingly. 18

21 6D: DEFEND Define the access groups what permission set is needed by each group? DML (read/write, readonly, exec stored procs) DDL (create objects, create permissions) Pair with permissions (NOT EVERYONE NEEDS TO BE A SYSADMIN, NOT EVERYONE NEEDS TO BE DBO ) Create/maintain SQL logins (NO ONE NEEDS TO USE THE SA ACCOUNT EXCEPT A DBA!) Set up backups VM snapshots, SQL backups, etc Integrate into the organizational DR plan - if no org plan for DR, create one for IT and push into the org Some Security Checks to make: Password strength check (passwords set to never expire change to regular expiration) Unsupported operating systems, or opsys on extended support upgrade SQL Server versions unsupported or on extended support upgrade Opsys or SQL Server not up-to-date on patch or CU releases upgrade NOTE: security patches require server restarts, may be missing Check for anti-spyware, anti-virus, weak firewall policies High number of domain admins/sql admins? Not needed! User accounts for non-existent users (no longer/never employed) Inactive AD computers, haven t checked in for the last 30 days Inactive AD users, haven t logged in for the last 30 days Unpopulated AD org units, remove or populate 19

22 7D: DECOMMISSION Turning off the old system, two approaches: - Keep old system as read-only, or archival - best used when minimal amounts of old data have been migrated to the new system - Example, migratelast 24 months of data to target system, maintain archive copy of the source system and access on an as-needed basis - Old system is needed for reports and to extract information that spans both systems - Get rid of the old system - If all the data has been migrated, no need to keep it around - mandatoryif - previous vendor will not accommodate archival use - Too costly to maintain 20

23 Know the DATA, know the REQUIREMENTS! Know what s going to be migrated and what s going to left behind. Open lines of communication with the Vendor teams, source & target. Negotiate as many data extracts as you can. Ensure that there s a test system available for user acceptance testing. 21

24 Q&A 22