Documentation External Synchronization FirstSpirit

Transcription

1 Documentation External Synchronization FirstSpirit Status RELEASED Department Copyright FS-Core 2018 e-spirit AG File name SYNC_EN_FirstSpirit_External_Synchronization e-spirit AG Stockholmer Allee Dortmund Germany T F info@e-spirit.com

2 Table of contents 1 Introduction Development process: simple scenario Development process: advanced scenario Restrictions Compatibility with Other FS Versions Getting started Application scenarios... 6 Components The "FSDevTools" command line tool Version control system Configuration Configuration Structuring and configuring a project Using External Synchronization (best practice) Step 1: Updating Step 2: Importing Step 3: Modifying Step 4: Exporting Step 5 Commit / Push FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 2

3 3.6 Conflict handling Simultaneous Modification of a Template Simultaneous changes to a template, different areas (disjunction) Simultaneous changes to a template, overlapping areas (conjunction) Simultaneous changes to a template, identical changes Name Space Conflicts Simultaneous creation of a page template with different content, same UID Different elements with the same UID in the same namespace (e.g., Page Store) FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 3

4 1 Introduction Support for the distributed development of FirstSpirit projects FirstSpirit projects are generally developed in a team. For distributed project development involving several team members, each team member works in their own local FirstSpirit development environment. The development processes operate at several workplaces (decentrally), at different times (asynchronously), and often at different development strands (nonlinearly). To support these projects successfully, adequate tools (such as version control systems, integrated development environments, etc.) need to be used. FirstSpirit External Synchronization supports these concepts by providing FirstSpirit objects in a format which allows developers: to export templates (and other project content) from a FirstSpirit project into a file system structure with a folder hierarchy and to import modified content from the file system structure back into a FirstSpirit project. This export/import functionality makes it possible to: FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 4

5 edit FirstSpirit objects externally (e.g., via an IDE see Simple Scenario External Editing (Chapter 1.1.1, page 6)). version FirstSpirit objects via an external version control system (therefore forming the basis for successful distributed development see Advanced Scenario Distributed Editing of FirstSpirit Objects) (Chapter 1.1.2, page 8). License In order to use the function from external components (such as the FSDevTools command line tool (Chapter 2.1.1, page 14)), it is necessary to establish a connection to the FirstSpirit Server. The API-Access license feature is required for this. Information about this documentation This documentation refers to distributed development and how FirstSpirit can support this with the FirstSpirit External Synchronization functionality. A knowledge of developing templates with FirstSpirit and of any external components that may be used is assumed. As the FirstSpirit development process is not the same in all companies and there are different ways of working with External Synchronization, this document describes best practice. This means that recommendations are provided on how External Synchronization can be best used. Other or new recommendations may arise as a result of experience with External Synchronization gathered in real projects by customers and partners. These will be constantly integrated into this document. As a result, user feedback is expressly welcomed. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 5

6 Application scenarios Development process: simple scenario In a simple scenario, the External Synchronization functionality can be used to support the collaboration between FirstSpirit and an integrated development environment ( IDE ) in the development process. The development is carried out in a local FirstSpirit project ( FirstSpirit Development Project ). From there, the relevant development state is exported to a local file system via External Synchronization. The FirstSpirit API is used for access. The corresponding commands ( fs-cli export, fs-cli import ) are called up via the FSDevTools command line tool and executed internally via the FirstSpirit API. The FirstSpirit objects exported in this way (templates, media, content, etc.) are transferred to the local file system in a readable format (xml, jsp, etc.). From there, the files can be edited via an IDE and written back into the file system again. These locally modified files can then be imported back into the FirstSpirit project via External Synchronization. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 6

7 Example: Exporting media and editing externally Step 1: Exporting templates and media folders (FirstSpirit file system) Step 2: Edit externally (file system) After the export has been completed successfully ( Export operation successful ) in the local sync directory, the files can be opened and edited in an IDE or another editor. In this example, the hex color value for the h3 heading is being set to another value. The external modification of the file (\DevProject\MediaStore\layout\css\contentFileContent.css) is saved locally (in the file system via the IDE). Step 3: Importing sync folders (file system FirstSpirit) The entire sync directory is then imported back into the FirstSpirit project via FSDevTools. FSDevTools command: fs-cli -p "DevProject" -sd D:/DevProject import FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 7

8 Result: The FirstSpirit object (MediaStore/layout/css/content) is detected as modified in the project. The modification to the CSS media object appears in the preview (after the object has been updated F5 ): Development process: advanced scenario In this scenario, several developers work on a project in a distributed manner: FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 8

9 Development within the local development environment The development is carried out in each developer s local development environment. Each developer therefore uses their own local FirstSpirit Server and their own local FirstSpirit project ( FirstSpirit Development Project ). From there, the relevant development state is exported to the local file system via External Synchronization. The FirstSpirit API is used for access. The corresponding commands ( fs-cli export, fs-cli import ) are called up via the FSDevTools command line tool and executed internally via the FirstSpirit API. The FirstSpirit objects exported in this way (templates, media, content, etc.) are transferred to the local file system in a readable format (xml, jsp, etc.). From there, the files can be edited via an IDE. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 9

10 Versioning via a version control system For a successful collaboration, the (further) development must always be carried out in the most current state possible; in other words, the modifications made by a developer must be made available in the other developers projects promptly. Each developer must also regularly export (Chapter 3.4, page 47) the development state of the FirstSpirit objects, publish their local modifications (Chapter 3.5, page 62), and update (Chapter 3.1, page 30) the changes made by other developers in their local project. The exported project content is also versioned using a version control system (here: Git). This means that each local development environment can fall back on its own local Git repository. In addition to the local repositories, there is also a further central Git repository ( remote Git repository ). Each developer pushes their modifications to the central repository. All developers involved can then adopt these changes in their own local repository and import them into the local project via External Synchronization. As a result, the local development states can be synchronized with the current (central) development state at any time. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 10

11 1.2 Restrictions FirstSpirit External Synchronization has been designed to support distributed development processes and must therefore only be used to distribute and synchronize templates (with all dependent FirstSpirit elements also required for the development process from other stores). We recommend using the FirstSpirit ContentTransport functionality for distributing editorial content. External Synchronization is not designed to transport mass data. Trans! FirstSpirit porting datasets / database content in particular can lead to a high volume of data. Information on Using External Components External Synchronization works together with external components (Chapter 2.1.2, page 17) (VCS, IDE, etc.) in the distributed development environment. These external components do FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 11

12 not form part of the FirstSpirit product. The relevant manufacturer (not e-spirit) is responsible for the functionality provided by these external components. When using the functionality, it is important to ensure that FirstSpirit provides the required interfaces for exporting and importing FirstSpirit objects but does not have an impact on the use of the exported content outside the FirstSpirit environment. Problems that occur when using the exported content in external components can be reported via FirstSpirit product support and will be rectified (if possible) if they refer to the FirstSpirit API interface. However, e-spirit is not obligated to fix bugs. 1.3 Compatibility with Other FS Versions Information on switching from a previous version of External Synchronization FirstSpirit External Synchronization functionality was introduced in FirstSpirit version 5.1. Using this, it was possible to export templates and other content from a FirstSpirit project into a file system structure with folder hierarchy and thus edit these externally. Since then, this functionality has been extensively enhanced and revised. For example, the export format has been changed and API methods added. External Synchronization now covers various requirements, particularly those which arise as a result of distributed development in concrete customer projects. When External Synchronization is mentioned in the text below, this always refers to the revised version. Modifications to export format Compatibility External Synchronization includes a new export format for FirstSpirit objects. If FirstSpirit project content had already been exported previously with an older FirstSpirit version, numerous sub-folders with the name.firstspirit will have been created (structure with the old export format). During the first export via External Synchronization, all existing sub-folders with the name.firstspirit are removed from the sync directory. A folder with the name.firstspirit is then only available on the uppermost directory level (structure with the new export format). The.FirstSpirit contains internal meta information for the successful synchronization of the external content with the FirstSpirit project. This folder must not be versioned! FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 12

13 a FirstSpirit version which still uses the old export format is upgraded or updated,! After the project in question must be fully exported so that the format changes take effect. The method setdeleteobsoletefiles must be set to true in the process. (This is the default setting when using FSDevTools (Chapter 2.1.1, page 14).) FirstSpirit versions which use the new export format (to versions which use! Downgrading the old export format) is not supported. In such cases, please contact e-spirit directly. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 13

14 2 Getting started 2.1 Components The "FSDevTools" command line tool ImportOperation and ExportOperation The External Synchronization functionality is based on interfaces which are released by e-spirit to be used by partners and customers. The following interfaces are entry points for External Synchronization: ImportOperation ExportOperation Package: de.espirit.firstspirit.store.access.nexport.operations Using the functionality via FSDevTools (recommended) The FSDevTools command line tool simplifies export and import operations between a FirstSpirit project and the file system (a target directory on the local workstation, the content of which is synchronized with a VCS if necessary) using predefined commands. FSDevTools provides secure and simple access to the External Synchronization functionality. We therefore recommend using FSDevTools when using the functionality. use with FirstSpirit External Synchronization at least FSDevTools version ! For or higher is required. As of FirstSpirit version : For use with FirstSpirit External Synchronization FSDevTools version or higher is required. Deployment From GitHub: e-spirit publishes additional functionalities at regular intervals in order to make FirstSpirit project development simpler and more convenient ( These FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 14

15 projects are developed, provided, tested in concrete internal and external scenarios, and regularly maintained by e-spirit. The FSDevTools command line tool is subject to the usual e-spirit stability and support guidelines. As part of the FSDevTools project, we are always looking for feedback, error messages, and feature requests at The source code of the project is also available on GitHub and can be forked and made available again via a pull request. (Note: With a fork, it is also possible to develop a customerspecific version of FSDevTools if required.) License In order to use the function from external components (such as the FSDevTools command line tool (Chapter 2.1.1, page 14)), it is necessary to establish a connection to the FirstSpirit Server. The API-Access license feature is required for this. Java A Java 8 installation is required for operating the FSDevTools command line tool. More requirements for operating FSDevTools can be found at: Installation Source code and documentation for the compilation, use, and functional expansion of the FSDevTools command line tool are published and available to download at: For the current version, see: The following installation packages are available for installation: FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 15

16 Tar archive (.tgz) for all Unix systems Zip archive (.zip) for Windows systems FSDevTools can be installed on the local workstation in any directory under any available, normal user account. In addition to fsdevtools-cli.jar, the relevant fs-access.jar is required to install and operate FSDevTools. Both jars must then be filed in the..\lib\ directory: Establishing a connection with the FirstSpirit Server After installation, the command line can be opened on the bin directory. All instructions relating to the command line start with the fs-cli call and the corresponding instruction. Using the fs-cli test call, a test connection is established with the FirstSpirit Server. The configuration parameters required for this (e.g., the host name) are entered directly via the command line, for example: fs-cli -h example.com.de -port c HTTP -p "DevProject" -u usera -pwd mypwd test Log output if the test is successful: ######### Test was successful ######### User: usera Host: example.com Port: 4242 Connection Mode: HTTP Project: DevProject Connection to FirstSpirit closed! Execution time: s FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 16

17 Calling up the integrated help function Information on using FSDevTools is available from the Command Line Interface Usage documentation or directly from the integrated FSDevTools help function. The integrated help function is called up via the command line: fs-cli help More information about the individual commands is available via the fs-cli help <command> call Version control system External version control systems (VSC) log and document changes to files and documents over time. A VCS can provide valuable support, even for one developer working on a project alone: For example, it allows each change relating to one branch to be isolated from the stable development state. As a result, changes can be developed in isolation from one another. At the same time, it ensures that unstable changes are not directly transferred to the stable development state. The developer does not transfer his modified artifacts back to the central master branch until a defined point in time ( Merge ). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 17

18 VCS are essential when developing in a team. In this case, far more changes by various developers have to be managed and merged. Each developer develops on their own branch and pushes modified artifacts back to the central master branch. As a result, conflicts occur where changes overlap one another. A version control system successfully merges all developers modified artifacts and automatically responds to the resulting conflicts wherever possible. Supported version control systems Git (recommended) Git is recommended for use with External Synchronization (see also Git version or higher is needed for use with FirstSpirit External Synchronization. For general notes on installing and configuring Git, see For notes on configuring Git in conjunction with External Synchronization, see link (Chapter 2.2.1, page 19). Further version control systems The use of External Synchronization with other version control systems has not yet been tested and therefore is not recommended. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 18

19 However, use with other version control systems is theoretically possible. Individual differences between the VCS must be taken into account. Example Move actions: Some VCS interpret moving from A to B as deleting in A and adding to B. Circular problems such as this have to be rectified individually on the basis of a project solution. Information on Using External Components External Synchronization works together with external components (Chapter 2.1.2, page 17) (VCS, IDE, etc.) in the distributed development environment. These external components do not form part of the FirstSpirit product. The relevant manufacturer (not e-spirit) is responsible for the functionality provided by these external components. When using the functionality, it is important to ensure that FirstSpirit provides the required interfaces for exporting and importing FirstSpirit objects but does not have an impact on the use of the exported content outside the FirstSpirit environment. Problems that occur when using the exported content in external components can be reported via FirstSpirit product support and will be rectified (if possible) if they refer to the FirstSpirit API interface. However, e-spirit is not obligated to fix bugs Configuration Configuration Information on configuring Git When using External Synchronization in conjunction with Git on Windows and non-windows systems, problems can arise with regard to line breaks, as Windows line breaks are saved in a different format to other operating systems. In line with the recommendation from the Git documentation, the core.autocrlf parameter must be set to true when a Windows system is used: $ git config --global core.autocrlf true This means that the line ends are converted into line ends that can be read by Windows during the check-out process. However, performance problems can arise as a result of this configuration. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 19

20 To ensure optimum performance, we therefore recommend using Unix line breaks, including in Windows. This can be done without any problems in most editors and IDEs. In addition, the core.eol parameter must be set to the value lf, either globally or just for the relevant project repository: $ git config --global core.eol lf Information on version management (FirstSpirit content) meta-information is not versioned! (.FirstSpirit folder)! Internal The.FirstSpirit folder on the top level of the Sync directory contains internal meta-information for the successful synchronization of external content with the FirstSpirit project. This internal data must not be versioned. If Git is used as a version control system, content can be left out of versioning using the.gitignore file, for example. In this case, the following line is added to the file:.firstspirit Configuring FSDevTools Creating a target folder for the export (in the file system) To enable data to be exported into the file system (or imported out of the file system), a target folder ( Sync Dir ) must be created in the user s local file system (for example, as a folder within a local repository) to start with. This means that each developer uses his own target folder and his own (local) repository, which may also include other versioned data in addition to the target folder: FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 20

21 The path to this target folder is then specified later when exporting and importing (using FSDevTools) (see How to Exporting project content (Chapter 3.4, page 47)).! The user must have write permissions for this target folder in the file system Structuring and configuring a project 1) Adding a central FirstSpirit project To begin with, a FirstSpirit project is created containing all content and project properties needed to develop the project, e.g.: Users Presentation channels Languages Resolutions Modules etc. 2) Structuring a FirstSpirit project (recommended) Exporting and importing FirstSpirit project content using External Synchronization should be complete and use clearly defined processes wherever possible. This helps to prevent problems resulting from incomplete project contents (in local development environments), and, as a result, conflicts. We recommend always exporting the entire Template Store. To make sure the referenced elements can also be synchronized with ease, they should be collected in a separate folder in the corresponding Store and become part of the synchronization process. This means that this folder must also be exported and imported during each synchronization process. When using database content, the corresponding CS2 objects also have to be exported and imported. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 21

22 can use Ctrl + R (or the context menu item Display extras / dependencies ) on a! You template to show which objects are dependent on the template and therefore have to be synchronized as well. We therefore recommend using a project structure in line with the following schema: Page content: Folder that contains all pages linked to the page references used as preview pages for the templates ( Properties tab). The folder can also contain sub-folders. Data sources: All of the projects datasets needed for template development. A matching data source must also be exported for each dataset. Media: A folder that contains all media needed for development. A reference to these media is normally established on the presentation channel tabs (e.g., via $CMS_REF(media:"...")$): Image files for icons Image files for buttons FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 22

23 Image files for navigation elements CSS files JavaScript files See, for example, the Layout (Chapter 1.1.1, page 7) folder on the Simple scenario page. The folder can also contain sub-folders. Site structure: A folder that contains all page references used as preview pages for the templates ( Properties tab). Global content: A folder that contains all global content referenced on the templates presentation channel tabs (e.g., via $CMS_VALUE(#global.gca("..."))$). The folder can also contain sub-folders. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 23

24 3) Creating local FirstSpirit projects In a distributed development environment, each developer works in his own local development environment. Each developer uses his own local FirstSpirit Server and his own local FirstSpirit project. The local project can be empty to begin with. The contents and settings can then be transferred out of the central FirstSpirit project (or remote repository) using External Synchronization. Alternatively, the central project can be added to the developers local servers using the Export project / Import project (ServerManager - Menu Project ) functions. of the developers and systems involved in the project must used standardized settings.! All This means that the local projects must have a standardized structure and identical project settings! Why is this important? An example: In a distributed development environment, multiple developers work on one FirstSpirit project. Using a version control system, the modified project contents and settings are distributed across multiple systems, e.g. in additional local development environments ( Development ) on a test system ( Test ) on a productive system ( Productive ) If, for example, the individual systems have a different number of languages in this scenario, this will always lead to conflicts if the project property projectproperty:languages is also transported via External Synchronization (in other words, as part of the export or import action). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 24

25 In this case, a Check-in / Export from the development system (number of languages: 2) into the VCS and then a Check-out / Import into the test system (number of languages: 10) leads to content being removed for the 8 languages not included. 4) Configuring permissions Users must hold the relevant permissions to export and import project content. Example: If objects are created in the project during an import, the user must have permission to Create objects and / or permission to Create folders. If objects are deleted during an import, permissions for Delete objects and / or Delete folders are required. actions may require administrator permissions. Server administrator permissions! Some are required to export and import server properties (users, fonts, etc.). At the very least, project administrator permissions are required to export and import project properties. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 25

26 3 Using External Synchronization (best practice) This chapter describes best practice cases for using External Synchronization in a distributed development environment. The steps described in the following apply to the developers and must be performed each time content is edited. It is important to make sure that the Import and Export steps in particular are performed after modification in order to ensure data consistency. The individual steps describe how project content is imported, modified, and exported between the file system and a FirstSpirit project. The process for managing and merging all changes is supported by a version control system. Conflicts that occur at different points during distributed development and that are not automatically dealt with by the version control system are described in the Resolving conflicts (Chapter 3.6, page 64) chapter along with specific scenarios and suggested solutions. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 26

27 Step-by-step workflow 1) Update From the repository to the file system (via VCS) Distributed development is based on a foundation of one FirstSpirit instance (or a central development state). The central development state is made available to all developers involved via a FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 27

28 remote repository. Within his own local development environment, each developer also works with his own local repository. At the start of each template editing process, the developer must collect the current state from the remote repository. In this process, all changes are first downloaded from the central repository (e.g., in Git via fetch ) and then merged with the state in the local repository. The versioned content from the local repository is transferred into the target folder ( Sync Dir ) in the local file system (e.g., in Git via checkout or pull ). During this process, external changes (made by other developers) are merged with the developer s own local changes ( merge ) For further information, see Update (Chapter 3.1, page 30). on conflict handling: Conflicts may occur when updating, particularly when merging! Notes changes. Possible conflicts and solution strategies are described under Conflict handling (Chapter 3.6, page 64). 2) Import From the file system to the FirstSpirit project (via External Synchronization) In the second step, the project s current development state is imported out of the file system into the local FirstSpirit project instance using FSDevTools (Chapter 2.1.1, page 14) (fs-cli import). During this process, objects are created, updated, moved, and deleted within the project. For further information, see Import (Chapter 3.2, page 33). 3) Modification Template development in FirstSpirit (using SiteArchitect) In step 3, the templates themselves are developed. The developer edits the templates or referenced elements, creates new templates, or deletes existing elements. In the project: Template development takes place directly within the local FirstSpirit project instance. In the file system: Theoretically, project content can also be edited directly in the file system. However, external editing does result in limitations (Chapter 3.3, page 44). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 28

29 Furthermore, all data edited externally must be imported into the local project instance to make sure the external changes are consistent. For further information, see Modification (Chapter 3.3, page 41). on conflict handling: If multiple developers are editing elements at the same time,! Notes conflicts may occur when importing data into the project and these conflicts may not be detected at VCS level (e.g., namespace conflicts in FirstSpirit). Potential conflicts and solution strategies are described under Conflict handling (Chapter 3.6, page 64). 4) Export From the FirstSpirit project to the file system (via External Synchronization) Once a stable state has been achieved following completion of work on a work package, the developer exports his current development state using FSDevTools (Chapter 2.1.1, page 14) (fs-cli export). During this process, the development state from the local FirstSpirit project instance is exported into the target folder in the file system. Objects are created, updated, moved, and deleted within the file system. This step is needed to transfer any modified content into the central development state. For further information, see Export (Chapter 3.4, page 47). 5) Commit / Push From the file system to the repository (via VCS) To start with, the development state from the target folder in the local system is confirmed in the local repository (e.g., in Git via commit ). Changes are then sent out of the local repository to the central repository (e.g., in Git using push ). The changes made by all developers involved are merged at this level ( merge ). During this process, conflicts are automatically rectified and changes are versioned. For further information, see Commit / Push (Chapter 3.5, page 62). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 29

30 Comments Information about the scenarios described The scenarios described are geared explicitly towards a distributed development model, i.e.: Multiple developers are involved. Each developer is using his own workstation with a local FirstSpirit Server. The FirstSpirit project data being edited by the developers are managed on each local FirstSpirit Server in a project DevProject The data in the FirstSpirit project DevProject are versioned using Git repository and synchronized between the developers workstations The Git repository already exists and can be accessed under the URL ssh://firstspirit.example/externalsync The Git repository is cloned in the D:\Git\DevProject directory on every workstation The scenarios use the FSDevTools command line tool (Chapter 2.1.1, page 14) (fs-cli) and the command-line-based Git client (git). workflow and its individual steps may differ if you use other tools to use External! The Synchronization and/or to version the project data. Information about documentation As the FirstSpirit development process is not the same in all companies and there are different ways of working with External Synchronization, this document describes best practice. This means that recommendations are provided on how External Synchronization can be best used. Other or new recommendations may arise as a result of experience with External Synchronization gathered in real projects by customers and partners. 3.1 Step 1: Updating From the repository to the file system In this step, the version control system is used to: FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 30

31 refresh the development state in the local repository via the remote repository and then transfer the development state from the local repository (in the target folder) into the local file system. See figure Step 1) Update (Chapter 3, page 27). Initial situation Central state (remote repository) Distributed development is based on a foundation of one central development state (in some cases, in combination with a central FirstSpirit instance), which collects the modified artifacts from all developers. In order to synchronize the local development states of all developers with the central development state, External Synchronization is used in conjunction with an external version control system (Chapter 2.1.2, page 17). All changes to the versioned content are managed in the version control system. The central development state is provided to all developers involved via a remote repository. Within his own local development environment, each developer also works with his own local repository (see figure REPO_DEV 1, REPO_DEV_2,...). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 31

32 Updating Updating the local repository For successful collaboration, the (further) development must always be carried out in the most current state possible; in other words, the modifications made other developers in the team must be transferred to your own local project as soon as possible. Before any template is edited, the latest state therefore has to be collected from the remote repository. developer must update his local repository so that it is in line with the latest version! Each directly before starting work on a work package. For this purpose, all changes are downloaded from the central repository to begin with (e.g., in Git via fetch ) and then merged with the version from the local repository. Transferring the latest state to the file system Versioned content in the target folder ( Sync Dir ) is transferred from the local repository to the local file system (the target folder is a folder within the local repository see Configuration (Chapter 2.2.1, page 20)). The required commands are executed by the developer and depend on the version control system used (e.g., in Git via checkout or pull ). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 32

33 During this process, the changes (made by other developers) are merged with the developer s own local changes ( merge ). on conflict handling: Conflicts may occur when updating, particularly when merging! Notes changes. Possible conflicts and solution strategies are described under Conflict handling (Chapter 3.6, page 64). Next steps Importing project data After a successful update, the project content can be imported into the local FirstSpirit project instance. Continue with Step 2) Import (Chapter 3.2, page 33). 3.2 Step 2: Importing From the file system into the FirstSpirit project In this step, External Synchronization is used to: import the development state from the local External Sync directory (local repository) in the file system into the local FirstSpirit project instance. The command line tool FSDevTools (fs-cli) is used for this process. See figure Step 2) Import (Chapter 3, page 27). recommend performing all export and import actions using FSDevTools.! We This tool uses pre-defined commands to automatically import and export data between the FirstSpirit project and the file system. The interfaces therefore do not have to be implemented individually (see also The "FSDevTools" command line tool (Chapter 2.1.1, page 14)). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 33

34 Initial situation What can be imported? principle, only FirstSpirit content that stems from an export with External Synchro! Innization can be imported. The following can be imported: Project content (objects): Content that is created and managed using SiteArchitect or ContentCreator. This includes templates, project structures, subtrees, and complete stores. When exporting project content via External Synchronization, it is filed according to a defined structure in the file system (see Sync directory structure (Chapter 3.4, page 58)). The system attempts to duplicate the structures from the project using a file hierarchy in the file system. Further internal information is also exported to the FirstSpirit objects, e.g., via the storeelement.xml. This file contains information such as the FirstSpirit object s ID, reference name, etc. Using this information as a basis (folder hierarchy and internal meta-information), objects can be imported from the file system back into a FirstSpirit project. Project content without this internal information cannot be imported! Conventional image files (apart from a folder hierarchy and additional information) also cannot be imported out of the target directory and into a FirstSpirit project. Project properties: Settings that are defined for a project using the ServerManager. These properties may be project-specific ( Project properties ) or global ( Server properties ) (e.g., languages). For some projects (especially in the context of FirstSpirit CaaS), user-defined project properties (custom properties) can be defined. These properties can also be exported. The following applies for all properties: If these properties (e.g., a certain language) are not available on a server, they are created on the server during the import process (as a server and project property). In this case, the technical user must possess the right permissions to create properties (see Importing project properties (Chapter 3.2, page 39)). Preparation The developer s local repository has already been updated using the version control system (see 1) Update (Chapter 3.1, page 30)). During this process, the central development state was transferred out of the remote repository. The project content and, if applicable, project properties are in the Sync directory (file system) and ready for import. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 34

35 Starting the command line tool (fs-cli) The command line is opened in the bin directory (in the FSDevTools installation directory). All instructions issued via the command line start with the call fs-cli and a corresponding instruction. D:\fs-cli\bin>fs-cli [command] Establishing a connection to FirstSpirit (fs-cli) To establish a connection to the (local) FirstSpirit server and the (local) project, the corresponding parameters (for host -h, Port-port, project name -p, user name -u, and password -pwd) are transferred, e.g.: -h example.com.de -port c HTTP -p "DevProject" -u usera -pwd mypwd To establish a connection to the file system, the path to the target folder -sd in the (local) file system also has to be specified, e.g.: -sd D:/Git/DevProject You also have the option of testing the connection using the call test, whereby the placeholder [connection_parameters] has to be replaced with the project-specific connection settings: D:\fs-cli\bin>fs-cli [connection_parameters] test Log output if the test is successful: ######### Test was successful ######### User: usera Host: example.com Port: 4242 Connection Mode: HTTP Project: DevProject Connection to FirstSpirit closed! Execution time: s FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 35

36 Importing Importing project content (fs-cli) In the next step, the current state is imported out of the local file system and into the local FirstSpirit project instance (see figure Importing ). The import process is launched with the command import, whereby the placeholder [connection_parameters] has to be replaced with the project-specific values: D:\fs-cli\bin>fs-cli [connection_parameters] import The content of the entire Sync folder is imported during the import process! Within the project new objects are created existing objects are modified existing objects are moved existing objects are removed import does not always mean that FirstSpirit objects or project properties are modified! An or added. An import process may also delete content from the project. The following applies for imported project content: If an object is exported to an external directory and then re-imported (into a different FirstSpirit project), the version history only receives the corresponding add or import actions, but not the entire version history of the object from the source project. on conflict handling: If multiple developers are editing elements at the same time,! Notes conflicts may occur when importing data into the project and these conflicts may not be detected at VCS level (e.g., namespace conflicts in FirstSpirit). Potential conflicts and solution strategies are described under Conflict handling (Chapter 3.6, page 64). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 36

37 Help function (fs-cli) Further project-specific settings for the import process can be viewed using the FSDevTools help function: D:\fs-cli\bin>fs-cli help import Recommendations for imports Distributed development is based on a foundation of one central FirstSpirit instance (or a central repository), which collects the modified artifacts of all developers (see figure below Dev_Project, Remote-Repository ). The export provided from this project must be complete and all dependencies between the individual exported objects must be taken into account (see Recommendations for exports (Chapter 3.4, page 54)). The import process for FirstSpirit project contents using External Synchronization must also be complete and must follow clearly defined processes where possible. Specific recommendations: FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 37

38 All FirstSpirit project instances involved (local target projects, central source project) should use identical settings (number of languages, etc.) (see Recommendations for imports (settings) (Chapter 3.2, page 40)). All FirstSpirit project instances involved should have a standardized structure (see Project structure (Chapter 2.2.2, page 21)). The import should always be complete (do not import stores or objects individually). This helps to prevent problems resulting from incomplete project contents (in local development environments), and, as a result, conflicts. FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 38

39 Importing database content (fs-cli) When using FSDevTools to import a database schema into a project (which does not yet contain the schema to be imported), you will be required to enter a database layer. The parameter {lm --layermapping} is used to configure the required layer during the import, e.g., via: D:\fs-cli\bin>fs-cli import -lm *:CREATE_NEW During this process (example above), the standard database (Derby) provided with FirstSpirit is activated for the project. If the schema is already in the project (into which data is being imported), information on layer mapping is not required. Further configuration options can be viewed using the help function for FSDevTools (see Examples ). Log output if mapping configuration is unavailable: Importing... Layer mapping is empty! getlayer() with context -> schema: Products Closing connection to FirstSpirit... (...) Missing mapping for source layer 'null'! Please specify a layer mapping. For more information type 'fs-cli help import'. Log output if mapping configuration is successful: Importing... Layer mapping: *:CREATE_NEW Create a default derby layer 'derby_project48910_0' for project Import operation successful Import done. The assignment information is kept for subsequent import actions. Importing project properties (fs-cli) In addition to the project content that is entered in SiteArchitect and/or ContentCreator, project properties can also be exported/imported using External Synchronization, even across servers. This method can be used, for example, to transfer the properties of a project (such as language FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 39

40 settings and resolutions) to an empty project as well as to synchronize the project configuration of multiple projects. Some project properties are dependent on server properties, e.g., users, groups, schedules, etc. These dependencies are automatically taken into account. permissions of a project administrator are a minimum requirement for importing! The and exporting project properties; server administrator permissions are required to import server properties. Recommendations for imports (settings) of the developers and systems involved in the project must used standardized settings.! All This means that the local projects must have a standardized structure and identical project settings! Why is this important? An example: In a distributed development environment, multiple developers work on one FirstSpirit project. Using a version control system, the modified project contents and settings are distributed across multiple systems, e.g. in additional local development environments ( Development ) on a test system ( Test ) on a productive system ( Productive ) If, for example, the individual systems have a different number of languages in this scenario, this will always lead to conflicts if the project property projectproperty:languages is also transported via External Synchronization (in other words, as part of the export or import action). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 40

41 In this case, a Check-in / Export from the development system (number of languages: 2) into the VCS and then a Check-out / Import into the test system (number of languages: 10) leads to content being removed for the 8 languages not included. Next steps Editing project content After a successful import, the project content can be edited in the local development environment. Continue with Step 3) Modification (Chapter 3.3, page 41). 3.3 Step 3: Modifying Creating, modifying, and deleting project content In this step, we will see how to edit project content. For this, the local development state can be changed internally using the local FirstSpirit project instance (recommended) or externally, directly in the file system (limitations apply) See figure Step 3) Modification (Chapter 3, page 27). FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 41

42 Initial situation Preparation The developer s local repository has already been updated using the version control system (see 1) Update (Chapter 3.1, page 30)). During this process, the central development state was transferred out of the remote repository. The project content and, if applicable, project properties are in the Sync directory (file system) and ready for import. Before further development, all content is imported into the local FirstSpirit project instance (see 2) Import (Chapter 3.2, page 33)). Editing project content In the FirstSpirit project (internally) Templates are developed directly in the local FirstSpirit project instance. Here you can do the following: creating objects modifying objects moving objects deleting objects. Each developer works in his own local environment. All changes between the (local) project instance and the development state in the (local) file system are performed using the export and import functions in External Synchronization : FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 42

43 In the file system (externally) At file system level, project contents and properties that are created via External Synchronization are available in a format that is legible to humans. Project contents, such as a CSS file from the project s Media Store, are also stored as a CSS file in the file system. The folder structures from the project are used in the file system. File names and further information from the project are also taken into account. This makes it easy to find and edit project content, even in the file system outside of the FirstSpirit project instance. However, this content then has to be imported back into the project (see Next steps (Chapter 3.3, page 47)). At file system level, you can modify objects move objects FirstSpirit SYNC_EN_FirstSpirit_External_Synchronization RELEASED 43