EXPERIENCE THE POWER OF METADATA-DRIVEN AUTOMATION. Explore Features You Wish You Had DISCOVERY HUB.

Transcription

1 EXPERIENCE THE POWER OF METADATA-DRIVEN AUTOMATION Explore Features You Wish You Had DISCOVERY HUB

2 GENERAL INTRODUCTION MARKET CHALLENGES: TIME TO DATA MATTERS The need for joint cooperation between business and IT has never been more crucial. Growing customer demands, evolving market dynamics, increasing competition, and changing regulatory conditions are compounding the pressure on business decision makers who can t seem to get the data they need when they need it. As a result, there are increasing expectations that IT will deliver efficient data discovery and reporting options for business users. 64% of business managers have seen their decision window shrink in the past 12 months 42% of business professionals need to make data driven decisions within one day 18 MONTHS is the industry average for developing a traditional BI solution 6.3 WEEKS is the time to build one traditional BI report 1 The classic enterprise data warehouse, based on OLAP and a star schema, has limitations when it comes to this. The traditional approach, while emphasizing data governance and quality, has proven to be costly and time-consuming. These limitations led, at least in part, to the rise of data discovery as a market - as business users voted for speed of access and ease of analysis, over slower and less glamorous IT-led deployments. According to Dr. Barry Devlin, a leading expert in Data Warehousing and Business Intelligence, the market has again, come full circle. The concept of governed data discovery has now emerged, emphasizing the need to ensure the completeness, quality and consistency of the data available to business users. This means, IT departments have been re-engaged and are attempting, with support from technical vendors, to re-introduce consistency and quality in populating data discovery systems. 2

3 THE MARKETPLACE According to research 2, between 2015 and 2020, the number of data sources analyzed will jump 83%, bringing the 10-year growth total to 120% overall. Enterprise companies will rely on a projected 50 different data sources by Not only will the number of data sources grow, the data volume within each source will also increase rapidly. Add to this constant churn and upgrades as new technologies and new data sources are introduced. GROWTH IN DATA SOURCES High Performers Moderate Performers Underperformers Companies are hastily moving to a self-service analytics environment. However, Forbes Research 3 has pointed out the key challenges when adopting self-service BI: SALES 19% - Combining data from multiple sources 8% - Developing timely/actionable analysis 14% - Ensuring data security/integrity 11% - Getting the right information into the right hands 10% - Working with IT to enable access to needed data 9% - Obtaining buy-in from senior management 6% - Obtaining needed data from external partners 13% - Understanding what the data means (analytics) 10% - Other 3

4 When an organization deploys self-service tools, often a small group of business users latch on to the tools and create a profusion of conflicting reports and dashboards. As a result, other business users can t find the correct report or instead find confusion when similar reports show different outcomes. Implementing self-service tools without governance results in a myriad of conflicting reports that creates a Tower of Babel where everyone talks, but no one communicates 4. With a growing number of data sources and concerns about data quality, governance, self-service, agility and time-to-market, it s imperative for businesses to upgrade their data discovery models to remain competitive. Add to that the growing need to meet compliance regulations such as the new General Data Protection Regulation (GDPR). A governed approach is needed. The desired end-goal is self-service BI but in a governed fashion that marries the need for now with the need for process, providing the best of both worlds that includes: Governance and self-service Security and agility Trust and transparency DISCOVERY HUB Discovery Hub is a revolutionary new approach to data discovery. Discovery Hub brings together the best of both worlds: data governance and self-service BI. It enables business leaders to rapidly access and interpret data they know they can trust. And it frees IT from tedious and redundant tasks that you can readily automate. With Discovery Hub you can: Provide business users with access to the data they need when they need it Maintain governance, security and control Dramatically reduce the backlog of IT requests Rapidly respond to business change Facilitate the journey to compliance Discovery Hub is fully integrated, delivering faster time to value with fewer resources. It is the first truly combined platform covering all the disciplines needed for an organization to present all relevant data to business users in a usable and agile way. With Discovery Hub, there is no need for standalone data preparation, data blending, catalog or data quality tools. 4

5 DISCOVERY HUB : METADATA DRIVEN AND AUTOMATED Discovery Hub is a single metadata-driven, agile and automated software platform. It is designed to fully automate the creation, maintenance and operation of Discovery Hub. Automation makes it faster, easier and more accurate to access, model, govern and understand data. By combining the power of automation with a metadata-driven approach, you get results that are five times faster than traditional methods; including a 70% reduction in build costs and a 60% - 80% reduction in maintenance costs. Short iterations ensure that your analytics platform is constantly aligned with the needs of your business. With the power of automation, you can respond to changes quickly and efficiently, make updates easily and solve problems in minutes that once took days or weeks. But this is not a one size fits all approach. It is readily customizable. The software sits on top of the SQL Server stack with full visibility it is not a black box tool. You can add customized SSIS packages, views, user defined functions, scripts and stored procedures to meet your unique requirements, and still see the code output. You can even take full control of the code output and modify it if needed. By anchoring your code inside Discovery Hub you get version control without having to move and deploy your code from one environment to the next. The software will automatically do that for you. You can also free IT resources from reporting and analytics tasks by enabling self-service tools. You can minimize time spent turning complex data into valuable information without coding, and with better performance and more possibilities than ever before. Tedious and time consuming tasks like tracking historical data and incremental loading now take only a few minutes with this approach. Working with Discovery Hub also saves money. Because a robust metadata repository is the foundation of our approach, there is no need to purchase an expensive, stand-alone metadata repository that makes association with technical and operational metadata difficult. With Discovery Hub associations and inconsistencies are easily identified to provide a great foundation for metadata management, data governance and data quality. But that s not all. Discovery Hub is poised to support your journey toward cloud-based data warehousing. This includes dealing with challenges like hybrid environments, bandwidth optimization and security. By combining automation with the metadata-driven approach we eliminate many of the programming and management problems inherent in traditional project work flows. The result is that all parts of the organization can collaborate and make critical decisions faster, because accurate, consistent and timely data is now available. INTERACTIVE AND USER ORIENTED INTERFACE Discovery Hub is the only automated tool on the market that is designed from the business user s perspective. The intuitive graphical user-interface (GUI) makes it very easy for someone with domain knowledge to learn. With Discovery Hub you can view the natural data flow, and see visual tracing, diagrams, documentation etc. without the burden of open windows, sub menu s, hidden features, scripts, etc. The unique user interface provides simplicity within a complex environment, striking just the right balance in being both feature-rich and user-friendly. 5

6 DISCOVERY HUB SELECTED FEATURES DATA LINEAGE Finding errors can be a cumbersome process, but with Discovery Hub it is as easy as the click of a button. If you find yourself questioning a figure in a report, you can use the data lineage feature in Discovery Hub to trace that number back to its origin by simply clicking on the data point. This allows you to answer questions like: What are the sources for the report I m looking at? and Which transformations have been applied to the data? Answers to these questions ensure a higher level of trust and assist with achieving regulatory compliance in many industries, including healthcare, finance, banking, and manufacturing. The data lineage feature in Discovery Hub tracks all objects and their dependencies, and delivers an end-to-end visual representation of the data lifecycle. 6

7 IMPACT ANALYSIS One key challenge of any integration effort is determining the impact that a change may have. With Discovery Hub it s easy. You can perform impact analysis on tables and fields and automatically identify dependencies and/or potential impacts. For example, if a proposed change in your operational system will alter the Sales Amount from including tax to omitting tax, Discovery Hub will automatically display where the Sales Amount field is used. This helps determine the amount of additional work that needs to done before proposed changes are made. 7

8 EXECUTION ENGINE - MACHINE LEARNING With the intelligent execution engine of Discovery Hub, you get speed, agility, and optimized performance. Parallel execution allows you to execute multiple tasks in parallel instead of sequentially. Execution times for the individual objects are logged and Discovery Hub uses this information to optimize execution order. The biggest tasks are executed first, taking into account dependencies between objects. Based on automated analysis of the processing logs, an algorithm improves the parallelism of every load based on the actual data being processed, achieving better performance with each load. The execution order is optimized once in the beginning, as you might do when coding by hand, but also each time the project is executed, resulting in an accelerated tempo. 8

9 PERFORMANCE TRACING All execution times, down to the smallest tasks, are saved. This helps diagnose the cause when changes occur. For example: if the execution time has increased heavily within the last week, Discovery Hub will automatically compare the time with the previous week. This way you can identify when execution times change. By further exploring the execution function, you might find a specific table takes twice as long to load. It s easy to identify bottleneck issues or determine a sharp increase in the number of rows in an execution. 9

10 DOCUMENTATION The robust documentation automatically generated by Discovery Hub can help you maintain regulatory compliance in response to legislation like GDPR. With the click of a button, it will read the entire metadata model and describe it in a PDF. The documentation allows you to see the names, settings, description and code for every object. Discovery Hub documentation includes full-version control, to ensure that the generated documentation corresponds to the deployed version of the project. Comments may be added to all elements in the project. The documentation comes as a hyperlinked PDF document based on a template you can modify. 10

11 INCREMENTAL LOAD You can enable incremental loading with just a few clicks. This saves time and helps optimize system performance. And with a single click, you can determine whether rows deleted in the source should be deleted from the solution or tagged as deleted for later analysis. By only loading new and changed records, you can: reduce load times dramatically, improve system performance, save time and resources, conserve bandwidth and disc space and facilitate faster decision making. 11

12 SECURITY With the security features built into Discovery Hub you can identify users and access rights, and enable object and row-level security. Object level security ensures authenticated users only access the data, tables or columns they are authorized to see. Row-based security goes further by restricting access to specific data within tables. For example, sales managers can only see data in their particular region. 12

13 SEMANTIC LAYER Semantic layer modeling capabilities provide easy access to data that is relevant to a specific department or purpose. In a self-service environment, one of the challenges business users face is a lack of understanding for system terminology. As a result, users may misinterpret the data or reports. The semantic layer modeling capabilities built into Discovery Hub solves this problem by transforming the data in relevant ways and using descriptions that are easy for business users to understand. You only need to define the semantic model once, the software automatically delivers the model in the front-end tool of your choice. 13

14 VERSION CONTROL The automated version control in Discovery Hub has two major benefits. First, it provides robust documentation at the click of a button. And second, it enables you to readily restore a project to an earlier point in time. Each version of a project is uniquely identified and the state of the objects within that project are recorded. This can include a timestamp, a description of changes made and who made the changes. A new version is automatically generated every time an object is created, modified or deleted, which makes it very easy to restore a project in the event an error is made. 14

15 SLOWLY CHANGING DIMENSIONS Names and addresses change, sales reps come and go, companies introduce new products and phase out old ones. Handling dimensional data across time is one of the trickiest aspects of database design. But with Discovery Hub, it is easy. In fact, it can be set up in a matter of minutes. Discovery Hub fully supports slowly changing dimensions (SCD) so you can accurately reflect changes over time or at a particular point in time. The unique optimized model for detecting changes makes it extremely agile, and by far the best-in-class SCD implementation. 15

16 MULTIPLE ENVIRONMENTS Discovery Hub supports multiple dedicated environments - for development, testing, production, etc. This ensures the production environment is always online and available for end-users. It also facilitates automatic version control in the production environment. The functionality multiple environments provides a simple and easy-to-use interface, and with one click you can transfer your project between environments (for example, a project between Development and QA and from QA to Production). The features ensure that the entire project is transferred and that nothing is forgotten. By using differential deployment techniques, Discovery Hub only re-deploys the changes needed in the new environment. 16

17 MULTIPLE USERS Multiple users simply means that more people can work and collaborate on the same project at the same time. Using check-in and check-out functionality, Discovery Hub helps to prevent collisions. With Discovery Hub, you can track who is working on which projects (check in and check out) while allowing developers to work simultaneously. 17

18 ANY DATA SOURCE SOME INTELLIGENT VIA ADAPTERS Discovery Hub can connect and extract data from virtually any data source and comes equipped with Intelligent Adapters for some of the most popular source systems. These adapters retrieve data using the business logic of the source systems. This allows you to extract and synchronize data from any source system faster and more efficiently than ever before. The AnySource connector allows you to connect to any OLE DB or ADO data source. The Custom Data Source connector allows you to plug into any option. It also works in conjunction with a separate provider or driver to enable access to data sources not supported by the core Discovery Hub product API. Intelligent adapters exist for: SAP, Microsoft Dynamics CRM/NAV/AX/GP, Salesforce, Infor M3 Movex), Infor10, Unit4 Business World (Agresso) and more. 18

19 FULL INTEGRATION WITH EXISTING DWH Discovery Hub allows a traditionally built Data Warehouse (DWH) to be fully integrated into the automated platform. This translates to a lower starting cost for any organization with an existing DWH. The actual transition to automation can be started later, if needed, in a controlled and safe manner. 19

20 CLOUD At TimeXtender, we re keen to help customers and partners as they move towards a cloud-based Discovery Hub. However, we also realize that everyone wants to accept change at their own pace and on their own terms. That s why although we are working towards a pure cloud solution, TimeXtender will continue to offer an on-premise solution and will offer hybrid solutions that will suit the needs of every customer as they transition over. With TimeXtender s model, companies can select which data they want to deploy on-premises or in the cloud to address their specific operational requirements, while laying the foundation to gradually evolve to a full-cloud deployment over time. REFERENCES 1 IDC study The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things April State of Analytics, Insights on analytics trends from more than 2,000 business leaders worldwide, Salesforce Forbes Insights: Analytical Enablement, How Leaders Harness Distributed Business Intelligence To Drive Breakthrough Results 4 Eckerson Group: A Reference Architecture for Self-Service Analytics Balancing Agility and Governance, By Wayne Eckerson with Barry Devlin, September 2016, p.4 20