WebSphere Information Integrator Content Edition and OmniFIND

Transcription

1 IBM Software Group WebSphere Information Integrator Content Edition and OmniFIND Technical Overview 2004 IBM Corporation

2 WebSphere Information Integrator Content Edition The Problem and the Solution Integration and Connectors Federation Services Developer and End-user Services Portfolio Integrations Questions 2

3 The Problem: Multiple Silos of Content Don't know 1 repository 5% The Future of Content in the Enterprise, Connie Moore and Robert Markham 17% More than 15 repositories 25% 36% 2-5 repositories 14% repositories 4% 6-10 repositories Base: 81 North American decision-makers (multiple responses accepted) 3

4 Multiple Content Sources Complicate Enterprise Initiatives CUSTOMER SERVICE MARKETING FINANCE SALES & SUPPORT CUSTOMERS & PARTNERS LEGAL HR R&D CALL CENTER CRM / ERP SELF-SERVICE COMPLIANCE WEBSITES Imaging/ Document Mgmt Report Mgmt Web Content/ Media Asset Mgmt Database/ Custom Systems Network File Systems Workflow/ Business Process Mgmt 4

5 The Solution: II Content Edition CUSTOMER SERVICE MARKETING FINANCE SALES & SUPPORT CUSTOMERS & PARTNERS LEGAL HR R&D CALL CENTER CRM / ERP SELF-SERVICE COMPLIANCE WEBSITES Information Integrator Content Edition Imaging/ Document Mgmt Report Mgmt Web Content/ Media Asset Mgmt Database/ Custom Systems Network File Systems Workflow/ Business Process Mgmt 5

6 The Solution: II Content Edition (formerly known as VeniceBridge) 6

7 II Content Edition Functionality Real-time, actionable search Federated search IICE-enabled indexed search Portal application Single access point for all enterprise assets Customer or partner access to relevant content for self-service Virtual Records management Manage retention of all enterprise content Enterprise Information Integration (EII) Complete view of all information about a specific topic object, project or process Production workflow Access all the required content to enable people to perform actions in a business process 7

8 The Tao of II Content Edition Every user is known to the repository Repositories handle their own authentication and authorization This is sometimes supplemented, but NEVER bypassed Every II CE user has named sessions with the repositories being accessed Repositories can only do what they can do II CE doesn t generally compensate for missing features in repositories II CE offers a superset of functionality Repository profiles describe the capabilities of individual repositories Users know of only abstract repositories There is never an API class or method specific to a repository Users do know that multiple repositories are being dealt with Repositories are abstracted, not virtualized (only exception: Virtual Repositories) Content and metadata is never stored by II CE II CE is not a content management system, it doesn t have a repository Using II CE doesn t require replicating your content or meta-data 8

9 Developer and End User Services Federation Services Web Components Federated Search Session Pools Web Client Web Svcs API (SOAP) Virtual Repositories View Services Enterprise Applications WebSphere Application Server Metadata Mapping Java API Authentication/ Security Custom Applications URL Addressability Subscriptions Subscription Event Services Access Services Admin Tools Integration Services Connecter Service Provider Interface (SPI) Connector RMI Proxy Web Services Proxy Connector Connector Data Source Data Source Data Source 9

10 Integration Services Unified Content Model Content Functions CRUD Meta data Native content Content Classes (meta meta data) Checkin / checkout Versioning, Version History Security CRUD Renditions Annotations Compound Documents Folder Hierarchy Functions CRUD Content Classes Folder Contents Folders Filed In Security 10

11 Integration Services Unified Workflow Model Workflow Functions Queue Enumeration Groups for a Queue Users for a Queue Queue Contents Security CRUD Work Item Functions CRUD Work items Attachments Work Item Classes (meta meta data) Get History Complete Lock/Unlock Suspend Resume/Reassign Ad-hoc Route 11

12 Integration Services - Connectors Connectors do the work to support the Integration Services content and workflow models Out-of-the-box connectors to many sources (~20) Numerous options for distributing connectors Remote EJBs RMI Connector Web Services Connector Session Pooling Configurable stand-by pool of repository sessions improves performance and scalability Supports named and generic users 12

13 Integration Services - Connectors Chargeable part numbers Connector for Documentum Connector for Microsoft Connector for IBM Documentum 4i and workflow Documentum 5 and workflow Connector for FileNet FileNet Image Services and workflow FileNet Content Services FileNet Image Services Resource Adapter (ISRA) FileNet P8 Content Manager FileNet P8 BPM Connector for OpenText OpenText Livelink Microsoft Index Server/NTFS Microsoft SharePoint Services Connector for Stellent Stellent Content Server Connector for Interwoven Interwoven TeamSite Connector for Hummingbird Hummingbird Enterprise 2004 DM 5.1 DB2 Content Manager 8 DB2 CM OnDemand WebSphere MQ Workflow WebSphere Portal Doc. Mgr. Lotus Notes Lotus Domino.Doc RDBMS DB2 UDB Oracle Others through II Federation Others Microsoft NTFS File system (sample) 13

14 Integration Services - Connectors Connector SDK Same toolkit used to build all of our connectors Design is complete (mapping is what s required!) ~25 content Methods, ~15 workflow methods Repository profile to simplify development Simple Java classes Robust J2EE benefits Immediate leverage by platform No platform bleed into connector 100% forward compatibility Excellent docs / examples 14

15 Developer and End User Services Federation Services Web Components Federated Search Session Pools Web Client Web Svcs API (SOAP) Virtual Repositories View Services Enterprise Applications WebSphere Application Server Metadata Mapping Java API Authentication/ Security Custom Applications URL Addressability Subscriptions Subscription Event Services Access Services Admin Tools Integration Services Connecter Service Provider Interface (SPI) Connector RMI Proxy Web Services Proxy Connector Connector Data Source Data Source Data Source 15

16 Federation Services Federated Search Cross-repository, data mapped searches Meta data and full text searches Parallel search, single unified result set Actionable results Virtual Repositories Create virtual repositories of all the content and work items related to a specific project, process, business object or topic Works at many different levels of granularity Virtual Repositories can contain links to: Content, work items, folders, queues, virtual folders, smart folders, URLs, custom objects Supplemental meta data and security 16

17 Federation Services Metadata mapping (data maps) Discovery of content and workflow classes (schemas) Map disparate indexing schemes across repositories Effective for search and CRUD Named data maps for specific applications or uses Authentication and Single sign-on Authenticate once, access all repositories Still a specific named user Multiple directory services supported LDAP Active Directory Embedded 17

18 Federation Services View Services Server-side conversion Electronic document to HTML Image conversion and processing Client-side Java viewer component (JavaBean) Image viewing and processing Annotations Printing Signed Java applet 18

19 Federation Services Subscription Event Services Automated change notifications on content, searches and workflow Items are subscribed to, then monitored for change Events can be handled by custom handlers (fax, , workflow, synchronization) API, Framework Possible use cases: Content Integration: Services to facilitate synchronization between repositories Portal: Portlet interfaces to provide subscription notification of changes to specific documents or work items Collaboration: When a final contract is received, notifications should be sent to stake holders Publishing: Notifications to agents, brokers, and end-users on policy addendums via multi-channel strategy ( , workflow, fax, etc) 19

20 Developer and End User Services Federation Services Web Components Federated Search Session Pools Web Client Web Svcs API (SOAP) Virtual Repositories View Services Enterprise Applications WebSphere Application Server Metadata Mapping Java API Authentication/ Security Custom Applications URL Addressability Subscriptions Subscription Event Services Access Services Admin Tools Integration Services Connecter Service Provider Interface (SPI) Connector RMI Proxy Web Services Proxy Connector Connector Data Source Data Source Data Source 20

21 Developer & End-user Services Applications can work with IICE at any of these levels: Programmatic (API) Loosely coupled (Web Services) UI only (Web Components) By links (URL Addressability) IICE web client requires no development effort It can be set up quickly, to work with any number of repositories It can be customized in different ways for different users, if desired 21

22 Developer Services Integration Options Java API Rich object-oriented content management API Abstracts all J2EE and system architecture complexity Many good examples included Finally, developers can write generic, platform-independent ECM applications! Web Services All the key integration capabilities also available through SOAP Supports both Java and.net clients 22

23 Developer Services Options (continued) Web components Set of 20+ rich web UI components Building blocks for customizing applications or custom applications J2EE-based - MVC, Struts, JSP, XSLT, JSR 168 Various components cooperate in coordinated component groups using a shared event model Create new custom components URL addressability Create very loosely coupled applications using II Content Edition Use in Web applications, send in 23

24 Developer & End-user Services - Web client 24

25 Security Authentication and authorization are controlled by the underlying data sources; their security model is respected at all times Sessions are created for each user with each data source by the data source s authentication mechanism Authorization of access to the data source is then controlled for that session by the data source Having a single user for all users of a data source is discouraged IICE provides supplemental security services: Single sign-on system Supplemental authorization system Identity-aware session pooling 25

26 Administration and Deployment Administration Centralized configuration and logging Remote graphical and web-based administration Dynamic configuration (never take WebSphere down!) JMX-based administration of subscription event services System Architecture and deployment options Scale from one to dozens of servers Leverage J2EE application servers Load balancing Fault tolerance Network protocol options EJB-to-EJB RMI SOAP 26

27 IICE Architecture IBM Software Group WebSphere software Administrator Tool Viewer Applet WSDL Application Application Java API J2EE Application Server Servlet Container Template Processor (JSP/XSLT) Web Application II CE Services DB2 II CE Viewer Servlet Java API Apache SOAP Web Application Java API Logging Config EJB Container View Services Access Services Server Result Set SOAP Connector Proxy RMI Connector Proxy Connector 3 Repository 3 API Apache SOAP Connector 1 Repository 1 API RMI Connector Proxy Server Connector 2 Repository 2 API Repository 3 API - Customer Code - II Content Edition Platform - Source Licensed to Customer -3 rd Party / J2EE Application Server Repository 1 Repository 2 Repository 3-3 rd Party Repository 27

28 Scaling Model Single server or multi-server Application Application Application Application DB2 II Content Ed. API II Content Ed. API II Content Ed. API II Content Ed. API Access Services Connector Connector J2EE EJB Clustering II Content Ed. II Content Ed. Connector Connector Connector Repository Repository Single Server Repository Repository Multi Server J2EE EJB clustering J2EE Servlet clustering RMI Connector pooling Web Services load balancing 28

29 II Content Edition and the II Portfolio Information Integrator OmniFind Edition crawler Included with OmniFind Index and search enterprise content in the following repositories: FileNet Documentum Hummingbird Information Integrator Content Edition wrapper Wrapper included with IICE Access unstructured content from WebSphere II federated server Wrapper based on DB2 II 8.2 s Java wrapper SDK RDBMS connector for IICE Read-only access to database tables (columns appear as attributes) DB2 UDB V8.2 or higher, Oracle 10g, and others through II federation 29

30 II Content Edition and WebSphere WebSphere Application Server II Content Edition is a native J2EE application hosted entirely in WebSphere Leverages WebSphere for fault tolerance and load balancing (clustering) WebSphere Portal JSR 168-based integration between II Content Edition web components and WebSphere Portal II Content Edition web client can be hosted in WebSphere or WebSphere Portal Portal Environments supported by IICE BEA WebLogic, IBM WebSphere and Microsoft Sharepoint 30

31 What about JSR-170? What is it? A standard for Java access to content repositories Not yet widely supported, but that may change JSR-170 is an incomplete standard Important ECM functionality not covered (example- no support for workflow) IBM is part of the group working to address this in a future version Only covers access to a single repository, no federation capabilities Originally intended only as a standard for accessing content within web sites JSR-170 and IICE Currently no support for JSR-170 in IICE (or any other IBM CM product) However, since JSR-170 is just another type of repository, an IICE connector could be easily written for it, if it ever becomes popular 31

32 IBM Software Group WebSphere Information Integrator OmniFind Edition 2004 IBM Corporation

33 IBM Software Group How to Differentiate ECI and Search Opportunities One-way access Enterprise Search Retrieval and display Casual access -- generally part of a knowledge management strategy Indexes underlying content Focused on speed/quality of results Always about leaving content in place Enterprise Content Integration Bi-directional access Retrieval, display, full CRUD, conversion, browse, foldering, workflow, etc. Deeper access to content; generally part of a production app/workflow Accesses native content in real time Focused on full unification of systems Sometimes also about migrating content 33

34 IBM Software Group Why Enterprise Search Matters Business users need to find information quickly and easily High quality, end user search Great diversity in end user search requirements IT managers need a framework to integrate unstructured information Easy to install, configure, and manage Enterprise scale Total Cost of Ownership purchase cost, administration cost Enterprise application developer Integrate into existing portals and applications Develop with existing tools using existing skills 34

35 Information Management UIMA: A new standard for content processing and text analysis Defines a common interface for integrating text analysis modules Enables interoperability of different analytics solutions and enterprise applications Provides an SDK for building and composing text analytics Enables development of new and re-use of existing components for analysis Text Identify Relevant Entities Build Structure People, Places, Organizations, Relationships Parts, Problems, Conditions Topics, Products, Interests, Sentiment Times, Events, Threats, Plots, Associations Identify Language Find Words & Roots Categorization Named-entity extraction Identify Relationships Extracted Metadata and Facts Search Index Database Text Analysis Modules aka Annotators Applications 35

36 Information Management UIMA Component Architecture Key Concepts Common Analysis Structure ( CAS ) enables pluggable Annotators Annotators can leverage CAS to build on each other Different Annotators are relevant for different collections Analysis results can be sent to multiple Consumers Collection Processing Engine (CPE) Aggregate Analysis Engine Text, Chat, , Audio, Video Collection Reader CAS CAS Initializer Analysis Engine Annotator CAS Analysis Engine Annotator CAS CAS Consumer CAS Consumer CAS Consumer Ontologies Search Engine Index DBs Knowledge Bases 36

37 IBM Software Group WebSphere Information Integrator OmniFind Edition Delivers enhanced results with sub-second response Sophisticated relevancy algorithms for corporate content Scales for large collections or enterprises Up to 20 Million documents 1000s of concurrent users Fits easily into enterprise applications Java APIs Document level security Eases administration and maintenance Analysis features all under-the-covers 37

38 IBM Software Group OmniFind Key Technologies Annotation Text Analytics Plug-in Categorization Taxonomy Rule-based Indexing Global Analysis Static Ranking Store Security Parsing/ Tokenizing HTML/XML 200+ Doc Filters Advance Linguistic Searching Dynamic Ranking Fielded Search Dynamic Summary Parametric Search Spell Checking Crawling Scalable Web crawler Data Source crawlers Content Push Search Collections Enterprise Content 38

39 IBM Software Group Search Quality: State of the Art Ranking Dynamic, term-based factors (term freq) x (1/doc freq) Lexical affinities Where term is found - title, body, anchor text Weight of text - bold, italic, relative font size Static or document-based factors Metadata Links URLs Duplicate detection Factor weighting dynamically adjusted based on the type of query Navigational -- HR e.g. anchor text weighted higher, link analysis, Informational Changing intranet password e.g. term frequency, lexical affinity, Search quality tuned to collection type Intranet (linked documents) Based on date (newsgroup, document currency) Document Collection 39

40 IBM Software Group Differentiated Value for IBM Clients Enhances WebSphere Portal investments Accesses more sources Scales to larger implementations Leverages the taxonomy defined in the portal for navigation and classification Migrates rules for rule-based classification Surfaces similar portlet with additional features Extends DB2 Content Manager investment Integrate DB2 Content Manager repository into enterprise search applications Provides native DB2 Content Manager crawler Leverages Notes and Domino investment Search Lotus Notes file folders in enterprise search applications Provides native Lotus Notes crawler Supports native Domino security meaning it will allow authorized searches down to a an application level 40

41 IBM Software Group Comparing OmniFind to Other IBM Search Offerings WebSphere Portal 5.1 Search Engine Lotus Extended Search Workplace 2.5 Search WebSphere II OmniFind Edition V8.2 Embedded portal search technology to index and search Portal portlets and pages via Portal Site search technology, Web content, includes ILWWCM Web published content, Portal Document Manager content, and attachments Search broker technology included w/ WebSphere Portal Extend Brokers search across supported data sources and indexes. Complements WebSphere Portal search capabilities with reach to additional data, content and index search sources Embedded search included in Workplace products. Index/search capabilities to: Team Collaboration Web Conferencing Collaborative Learning People Finder Workplace Messaging (client) Enterprise search engine for intranets, extranets and public corporate websites. and industry applications Enterprise scale and broad reach to additional content sources for upgrading WebSphere Portal, Lotus Domino and Workplace customers. 41

42 IBM Software Group IBM Search Competition Competitor Competitor Weakness IBM Strength Verity Autonomy FAST Convera Google Microsoft Verity is largest with 450 employees and $113M revenue (2003) Small companies with limited resources Narrow focus on indexing and retrieval. Most products are very complex with high TCO Significantly higher price points Google Search Appliance Focused on Internet search Entering market. Initial focus on Internet search, unlikely to reach non-ms content repositories. IBM views search as an extension to a comprehensive information integration infrastructure that includes information management, content management and information retrieval technologies required for building enterprise applications in the on demand era. Customers understand they can rely on IBM as a long-term partner Search technology is a strategic element in IBM s software stack and, as such, is receiving tremendous focus and investment, $50M annually 42

43 IBM Software Group What about the Google Search Appliance? Google - focused on the retail internet market Advertising revenue focused - 95% of revenue Page-ranking system is not optimized for enterprises Corporate Intranets - fundamentally different: Less content, lower chance of finding the perfect match Poorly linked; linking process more centrally controlled Content stored in many different systems besides web Enterprise security needs differ Black box offering runs counter to enterprise HW/OS standards OmniFind is designed to produce the best results from enterprise content 43

44 IBM Software Group Proven Quality, Scale and Robustness on IBM Intranet GO Quality preferred 2:1 over prior technology 4 Searching 1 Crawling Scale -- 80K queries/day with sub-second response over 7M pages Indexes 9 M unique pages 10,000 websites 20 K per document Search Servers Crawler Processes 80 K queries/day 3 Indexing Build & Push Indexer 2 Parsing & Tokenizing 7 K queries/hour peak Stressed to 10x higher Robustness % availability since Sept. 24x7 operation 44

45 IBM Software Group The w3.ibm.com web site runs IBM search today! 45

46 IBM Software Group Tabs allow you to direct where you want to look. Advanced Search for more options Enter the search terms that you want, such as: How to change CMVC password 46

47 IBM Software Group Quicklinks are predefined searches Excellent results! Note highlighted terms and word stemming. Lexical affinity the proximity of the words help in search quality Each summary is built dynamically it is based on the search terms that you entered. 47

48 IBM Software Group Advanced Search gives a range of options for focusing your search 48

49 IBM Software Group Business Partner Extensions Endeca Multi Faceted, or Guided Navigation iphrase Athoc Self Service Applications, Natural Language Processing Subscription and Notification Sun & Son Expertise Location, Taxonomy Services, Knowledge Management Muse Global Access to specialized datasources Upshot Document Intelligence 49

50 IBM Software Group When to choose OmniFIND versus WebSphere Content Discovery Server Choose OmniFIND when: The need is for Bi-Directional Languages The need is text analytics 50

51 Summary IBM Software Group Most companies have multiple content management systems Information in these systems is often isolated from key applications and security/compliance policies IBM offers powerful, well differentiated products for content integration and enterprise search that enable organizations to better leverage and control distributed content assets These products are key to business solutions related to customer service, records management/compliance, research & intelligence, various production workflows, and more 51

52 Questions? 52