Concept - first iteration DAM 2.0 & CMIS CAUTION: this page simply describes the ideas and discussion of the first iteration of the DAM 2.0 & CMIS implementation effort. Some things are still up to date - others have been changed completely. Introduction Goals Scope Infrastructure setup Implementation plan Topics DAM API improvements Provider registry Renditions JCR provider improvements CMIS provider CMIS server module DAM module reorganization DAM API refactoring proposal UI framework improvements Link API improvements STK improvements Additional tasks Notes Appendix - Proposed API Appendix - Current API and implementation Introduction The objective is to have a CMIS integration for our DAM module, both for exposing the DAM over CMIS and for connecting to a CMIS server. Goals As a user I can browse assets on a CMIS server in the Assets app. As an editor I can use assets from a CMIS server in my pages by selecting them in a dialog. As an administrator I can configure one or more CMIS providers in the DAM. UPDATE As of January 22nd were shifting focus from the assets app onto the choose dialog. This is driven by the realization that browsing a read only
external server is of questionable use. It's more relevant to have different sources to pick from in the choose dialog. It will still be necessary to build this UI, at least for providers with a folders and files layout, using our standard components that we use for content apps. Also, in order for renditions and linking to use properly we will need to have proper providers. They will just not be accessible from the Assets app. Scope Write operations on a remote CMIS server is out of scope. Note that this contradicts using the Assets app on top of the DAM API only and tying the CMIS server to the DAM API rather than the DAM workspace. (UPDATE: both of these are out scope now). Infrastructure setup We have a Sharepoint installation to test with. https://www.evernote.com/shard/s249/sh/4401996f-4fc4-4748-ba1f-ff52b590b980/f5e00b6dff28719 7c2fe0a2a67721bcd Alfresco has a test server that we can use for testing. http://www.alfresco.com/cmis Implementation plan Imaging module that can work on images from the dam. Updated DAM API with new support for: 1. 2. folders provider registry provider capabilities (implementation option TBD) CMIS provider that supports listing folders and reading assets. Out of scope is: a. b. Versioning Activation models User mapping A mechanism for serving assets from any provider, similar to what the URI2Repository mapping for DAM does now. Extension to the Link api that supports linking to assets. Updated RichTextField that can use and save references to assets from any asset provider. Updated STKAssetRenderer that doesn't rely on JCR and uses new imaging module (see above). CMIS server module that uses the DAM workspace. For the App we see a plan in two steps: We develop a prototype, a separate app that uses only the DAM API. It will use a custom provider and have custom actions all based on the DAM API. We evaluate the prototype and decide to either merge the prototype into the Assets app and use the DAM API also for everything that is currently done directly on the JCR. or we merge in the prototype as a provider specific view, enhancing the assets app with this functionality. Topics
DAM API improvements We need a number of improvements to the DAM API to make it capable of supporting the needs of the Assets app and the CMIS server. Folder support The DAM API needs to have a concept for folders to make the assets navigable. Folders do not have metadata and cannot be linked to (href links, referencing them is supported) nor do they have renditions. We will introduce a new interface in the API called AssetFolder and a number of additional methods related to folders, primarily on the AssetProvider interface. AssetElement AssetFolder AssetProvider AssetElement.getName() AssetElement.getIdentifier() AssetElement.getPath() AssetElement.getAssetProviderIdentifier() AssetElement.getParent() AssetElement.getDepth() AssetElement.isFolder() AssetElement.isAsset() AssetElement.getLastModified() (might not make sense on folders) AssetFolder.getChildren() AssetProvider.getRootFolder() AssetProvider.getFolderForPath(String abspath) AssetProvider.getFolderForId(String abspath) AssetProvider.getAssetElementForPath(String abspath) AssetProvider.getAssetElementForId(String id) Modification operations Currently we only have support for reading assets. We need to expand the API to allow for: creating a folder deleting a folder renaming folder moving a folder creating an asset deleting an asset duplicating an asset renaming an asset setting metadata on an asset Should we have operations for copying and moving between providers? Capabilities discovery Not all providers will have the same feature set. We need a mechanism in which users of the API can test for certain capabilities. Option 1 An enum of capabilities were a provider either supports or doesn't support a certain one. The provider has two methods, one for getting all the supported capabilities and one for testing if it has a certain one. Example: provider.hascapability(capabilities.versioning) Option 2 A dedicated type returned by the provider with query methods for each capability. The advantage being that it can have methods return boolean, integers or enums where a set of responses make sense. Such as NONE, SOME, ALL. Example: provider.getcapabilities().isversioningsupported() Option 3
Use a key-value based approach that is extendable to allow for querying for properties not explicitly defined in the api. i.e new providers exposing capabilities that have not been defined before-hand. This should not use enums. Example: Boolean.valueOf(provider.hasCapability(Capabilities.VERSIONING)) Querying We currently have an AssetFilter class with basic properties for searching for specific assets. It contains a string that's used as an addition to the where clause. This will obviously not work for other providers, Maybe we could use a criteria like api. We could allow for passing the extra where clause in multiple languages, where JCR-SQL2 would be one and CMIS would be another. Other topics Paths must be strictly validated to not allow bypassing security by adding in. or.. references. Deleted nodes need special attention, we might need to make this part of the api. Provider registry There is currently no way to configure providers. The registry is hard coded to have only the JCR provider. Option 1 - Observed Manager An observed manager that looks for asset providers under each module node. Option 2 - DAM module config node (DECIDED) We use the config node in the dam module. This has the benefit of making ordering easier. Additionally We should make it possible to configure which media types are contained on a certain provider. Renditions Renditions are provided by implementations of the AssetRenderer registry. Every media type is tied to a particular renderer. There is currently two renderers, JcrAssetRenderer that simply generates a link starting with /dam, no resizing or other image operations are supported. The other is STKAssetRenderer which is described in the section on STK improvements. We need to have renderers that are provider independent. JCR provider improvements JcrAsset opens an input stream always, whether its used or not. It is likely not closed properly. JcrAsset should not act like a DTO, it should delegate back operations to the provider for operations such as opening the input stream. Assets app The assets app works directly on the JCR never going through the DAM API. As we add more providers this has to change to make it operate on remote data. Option 1 - Using DAM API only Expand the DAM API enough to make it support the entire feature set of the Assets app.
This includes activation and dealing with deleted nodes. We would basically need to write new actions that call into the jcr asset provider where it does the same thing as the current actions. Option 2 - Provider specific views (DECIDED) Allow the app to have custom views per provider allowing for an optimized user interface. The JCR view would be identical to what we have now, accessing the JCR directly. For CMIS we would have a new view that would use the DAM API only. It would be usable against all providers with similar functionality as CMIS. Providers such as Flickr and youtube which don't have traditional files-and-folder would have very customized views. We must also update the choose dialog and give it custom views. Optional Introduce uses/subtabs in Asset detail view/dialog ( UI Concept) Introduce metadate in Asset detail view/dialog ( UI Concept) Add metadata extractor logic based on the uploaded MediaType Add MediaType previews Video Sound Documents Add MediaType Editor (like image editor) Video --> select and create cover/thumbnail image DamUploadFile Use Asset Use transformer Create a vaadinimage component that perform the bridge between Asset's and the Imaging module. This mockup shows what we're aiming for. Note the tabs for each media type and the source selectors beneath within each tab.
CMIS provider The CMIS provider should be able to communicate with a CMIS server using both the web service and the atom binding. It should be possible to specify a path within the remote repository to use as root path. There's a number of things we need to decide on here: Versioning If we decide to support versioning we need to define this in the DAM API and add the appropriate methods of Activation list versions get version create version restore version Should we support a model where a customer has a public and an author cmis repository and wants to use activation to transfer from one to the other? We would want to prevent activation of a page if its using an asset in CMIS that hasn't been activated. The activation status must be a available on the asset. We've identified four models where activation comes in: same server for authoring and public same server, flag that makes public, magnolia should flag (flag is just an example, it could be a complicated mechanism) different servers, magnolia should copy different servers, servers copy, magnolia invokes the server to copy (basically same as model 2) Authentication and authorization There's two options here, we could decide to support both or support just option 1 to begin with. Option 1 - Use a single user on the CMIS server for all Magnolia users (DECIDED) Option 2 - Mapping users in Magnolia to CMIS server users Renditions mapping users requires some kind of sso otherwise we'd need to store passwords for the other end in Magnolia configured per cmis provider anonymous must have access commonly used for thumbnails we could take advantage of this for thumbnails however it should not be used as our renditions that need to work independently of provider we could though use our renditions for as cmis renditions returned with the cmis server module Mapping metadata CMIS supports properties on documents and custom properties. TODO can we support the full dublin core this way? Querying CMIS has its own query syntax. How we can map onto this in the DAM API? Session management session reuse, can it be reused between threads? use same session for request, i.e. thread local, is there an advantage to this? do we need to limit the number of connections executing at the same time? globally for all users must investigate the parallelism and thread safety of the api
Caching key folder listings per path and user (see above) Requirements M5 CMIS client module may access a CMIS server in order to read documents. The server has to be easily configurable CMIS Client module has to implement the AssetProvider interface. CMIS server module Two options here, either we expose the DAM JCR as we did for the DMS in 4.5, or we expose the DAM API and every provider. Option 1 - Expose DAM workspace (DECIDED) Essentially the same as the 4.5 compatible module but fitted for the DAM workspace instead. Option 2 - Expose arbitrary DAM provider Every provider in DAM is exposed as a CMIS repository. All functionality that we want support needs to be available on the DAM API. Out of scope Versioning Mapping our renditions to CMIS renditions Current state The current CMIS module is now able to access DAM repository. Create/Read Folders Create/Read Assets A direct access to the DAM repository is currently performed. New Implementation step will involved the usage of Assets Provider to expose Assets. Implementation is pushed under: https://git.magnolia-cms.com/gitweb/?p=modules/cmis.git;a=shortlog;h=refs/heads/magnolia-module-cmis-2.0 Workbench CMIS client url: http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws/cmis?wsdl DAM module reorganization Proposal: dam-api dam-jcr dam-app dam-templating DAM API refactoring proposal TODO greg
UI framework improvements The rich text field stores references to assets it links to. It must be able to support linking to asset from any provider. It is currently hard wired to work only for JCR assets. TODO research this Link API improvements Update / Review the core Link API. This API has to be able to handle Asset's composite keys. See the section on UI improvements. STK improvements STKAssetRenderer uses imaging to resize, crop and transform images. Imaging is tied to the JCR and can't be used for assets from other providers. We need to redesign this. Additional tasks Upgrade to latest Apache Chemistry Test with Adobe Drive Notes Links M5 DAM API 4.5 CMIS Server Module CMIS Worbench Logging configuration
org.apache.chemistry.opencmis.binding.spi.type=webservices org.apache.chemistry.opencmis.binding.webservices.repositoryservice=http:/ /localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/repositoryservi ce?wsdl org.apache.chemistry.opencmis.binding.webservices.navigationservice=http:/ /localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/navigationservi ce?wsdl org.apache.chemistry.opencmis.binding.webservices.objectservice=http://loc alhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/objectservice?wsdl org.apache.chemistry.opencmis.binding.webservices.versioningservice=http:/ /localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/versioningservi ce?wsdl org.apache.chemistry.opencmis.binding.webservices.discoveryservice=http:// localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/discoveryservice?wsdl org.apache.chemistry.opencmis.binding.webservices.multifilingservice=http: //localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/multifilingser vice?wsdl org.apache.chemistry.opencmis.binding.webservices.relationshipservice=http ://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/relationships ervice?wsdl org.apache.chemistry.opencmis.binding.webservices.aclservice=http://localh ost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/aclservice?wsdl org.apache.chemistry.opencmis.binding.webservices.policyservice=http://loc alhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/policyservice?wsdl org.apache.chemistry.opencmis.user=superuser org.apache.chemistry.opencmis.password=superuser org.apache.chemistry.opencmis.binding.compression=true Appendix - Proposed API
Appendix - Current API and implementation