PRICIE + G Evaluation, Aggregation and Review Tool Canadian Army Institutional Version Handbook

Transcription

1 CAN UNCLASSIFIED PRICIE + G Evaluation, Aggregation and Review Tool Canadian Army Institutional Version Handbook Building, Installation, and Walkthrough Emile Pelletier DRDC Centre for Operational Research and Analysis Defence Research and Development Canada Reference Document DRDC-RDDC-2017-D133 December 2017 CAN UNCLASSIFIED

2 CAN UNCLASSIFIED IMPORTANT INFORMATIVE STATEMENTS Disclaimer: Her Majesty the Queen in right of Canada, as represented by the Minister of National Defence ("Canada"), makes no representations or warranties, express or implied, of any kind whatsoever, and assumes no liability for the accuracy, reliability, completeness, currency or usefulness of any information, product, process or material included in this document. Nothing in this document should be interpreted as an endorsement for the specific use of any tool, technique or process examined in it. Any reliance on, or use of, any information, product, process or material included in this document is at the sole risk of the person so using it or relying on it. Canada does not assume any liability in respect of any damages or losses arising out of or in. This document was reviewed for Controlled Goods by Defence Research and Development Canada (DRDC) using the Schedule to the Defence Production Act. Endorsement statement: This publication has been published by the Editorial Office of Defence Research and Development Canada, an agency of the Department of National Defence of Canada. Inquiries can be sent to: Template in use: (2011) SR Advanced Template_EN_4_ _V05_WW.dotm Her Majesty the Queen in Right of Canada (Department of National Defence), 2017 Sa Majesté la Reine en droit du Canada (Ministère de la Défense nationale), 2017 CAN UNCLASSIFIED

3 Abstract Defence Research & Development Canada Centre for Operational Research and Analysis has developed a SharePoint and InfoPath based collaborative tool for the Canadian Army to aid in the planning of and for the enhancement of Force Development Working Groups called the PRICIE + G Evaluation, Aggregation and Review (PEAR) tool. The working groups are known as PRICIE + G analysis working groups; PRICIE + G is a Canadian Army framework of capabilities. This Reference Document has the dual aim of detailing the steps involved in creating an official version of the PEAR tool starting from the existing version as well as being an interim user s guide. When an official PEAR tool is ready for use, this guide could still be used provided the official PEAR tool s various website addresses are inserted where applicable. DRDC-RDDC-2017-D133 i

4 Résumé Le Centre d analyse et de recherche opérationnelle de Recherche et développement pour la défense Canada a mis au point un outil de collaboration basé sur les logiciels SharePoint et InfoPath, et appelé l outil PRICIE + G d évaluation, d agrégation et de révision (PEAR) pour aider l Armée canadienne à planifier et à améliorer ses groupes de travail sur le développement des Forces. Les groupes de travail sont communément appelés groupes de travail analyse PRICIE + G; PRICIE + G étant un cadre des capacités de l Armée canadienne. Outre son utilité en tant que manuel provisoire de l utilisateur, ce document de référence vise à préciser les étapes à suivre pour créer une version officielle de l outil PEAR à partir de la version actuelle. Lorsque l outil PEAR définitif sera prêt à l emploi, le présent manuel pourra encore servir pourvu que les diverses adresses de sites Web dans l outil PEAR soient insérées aux endroits appropriés. ii DRDC-RDDC-2017-D133

5 Table of Contents Abstract i Résumé ii Table of Contents iii List of Figures v List of Tables vi 1 Summary Introduction Part I: PEAR Tool Build and Installation Overview Technical Installation Guide MS SharePoint Custom Lists Working Group (WG) List Discussion List MS SharePoint Workflows Discussion List New Item WG List Edit Item MS InfoPath Forms Evaluation Survey Form General Appearance and Flow Data Connections Review Survey Form General Appearance and Flow Data Connections Results Viewer Form General Appearance and Flow Data Connections MS Excel Macro-Enabled Workbook (Result Processing Workbook) Part II: Walkthrough OPI Steps Step 1: Adding CJIM to the WG List Step 2: Adding Discussions during Initializing by OPI into the Discussion List Step 2b: Editing Items in the Discussion List (e.g., edit Detail of a Discussion) Step 3: Distribute the Evaluation Survey to the SMEs Step 4: Aggregating Responses Collected in the Evaluation Survey Step 5: Distribute the Review Survey to the SMEs Step 6: Specify an Order of Discussions for the WG Step 7: Running the WG; Enter Results of the Discussions DRDC-RDDC-2017-D133 iii

6 4.2 SME Steps Step 1: Completing the Evaluation Survey Step 2: Completing the Review Survey iv DRDC-RDDC-2017-D133

7 List of Figures Figure 1: WG list Figure 2: WG list settings Figure 3: Discussion list Figure 4: Add new item dialogue Figure 5: Discussion list settings Figure 6: Discussion list new item workflow in MS SharePoint Designer Figure 7: Discussion list new item workflow in editor Figure 8: WG list edit item workflow in MS SharePoint Designer Figure 9: WG list edit item workflow in editor Figure 10: Evaluation Survey appearance (in design view) Figure 11: Review Survey appearance (in design view) Figure 12: Results viewer s Planning view (in design view) Figure 13: Results viewer s Running view (in design view) Figure 14: Result processing workbook Sheet Figure 15: Result processing workbook part of Sheet Figure 16: PEAR Tool process diagram Figure 17: Filtering discussion list by WG Figure 18: Discussion list with OPI-added discussions DRDC-RDDC-2017-D133 v

8 List of Tables Table 1: Evaluation Survey data connections Table 2: Review Survey data connections Table 3: Results viewer data connections Table 4: List of discussions added during Initializing by OPI for CJIM WG vi DRDC-RDDC-2017-D133

9 1 Summary PRICIE + G stands for: Personnel and Leadership; Research & Development, and Operational Research and Analysis (plus Experimentation); Infrastructure, Environment and Organization; Concepts and Doctrine; Information Management and Technology; Equipment and Support; and, Generate. The PRICIE + G Evaluation, Aggregation and Review (PEAR) tool provides Canadian Army subject matter experts, stakeholders, and Offices of Primary Interest a platform for collaboration/discussion and for managing discussions related to new and existing capabilities impacted or delivered by Canadian Army capital projects. The PEAR tool uses PRICIE + G for grouping, classifying and analyzing discussions. 2 Introduction The PEAR tool (herein referred to as the tool ) is designed primarily with Microsoft SharePoint and InfoPath. Excel is required for some important calculations and for processing quantitative results. This document is broken into two parts. Part I of this document deals with the design of the tool. Part II is a walkthrough, or demo for its users. The audience of Part I are Information Technology (IT) staff working in the Army Information Management group. The audience of Part II are the users and the Army Information Management group. DRDC-RDDC-2017-D133 1

10 3 Part I: PEAR Tool Build and Installation 3.1 Overview The PEAR tool is used for planning and running Working Groups (WGs) and collaborating between the subject matter experts (SMEs) and the Office of Primary Interest (OPI). Each of the SMEs and OPI have different roles to play in using the tool. SMEs have to complete two surveys in advance of the WG, by invitation and under instruction from the OPI, and the OPI, who does the planning for the WG, oversees the completion of PEAR process, which will aid the OPI in planning and running their WGs. What is involved in the PEAR process is covered in detail in Part II of this document. 3.2 Technical Installation Guide This section is a step-by-step process that features copying the current version of the PEAR tool that is found in the sandbox directory of the Army Collaborative Information Management Services (ACIMS), (called the Army Institutional Version), into a new SharePoint 2010 site. Step 1: Create PEAR template from the sandbox site: (including its content). 2 DRDC-RDDC-2017-D133

11 Step 2: Open SharePoint Designer 2010 and open the new site where you would like to install PEAR. Create a new subsite and select the PEAR template created in Step 1. Step 3A: While still in SharePoint Designer 2010, click on Workflows. Click on WG List Edit Item, and click Edit Workflow. DRDC-RDDC-2017-D133 3

12 Click on the first action (a new window will open). 4 DRDC-RDDC-2017-D133

13 Highlight the entire hyperlink and click on the icon with the image of a globe and chain link (a new window will open). Replace the portion of the hyperlink that points to the old sandbox site ( sandbox/pricieg/peartool/pear2 ) with the new site address, in both textboxes. Click OK twice to close the windows. DRDC-RDDC-2017-D133 5

14 Step 3B: Perform the same actions as in step 3A, but on the second action. Step 3C: Do not click the 3 rd action. If you do accidentally click on it and the window such as the ones in Step 3B opens, close the workflow editor and without saving your changes. Redo Steps 3A and B. For the 3 rd action, choose Properties from the list of choices (a new window will open). 6 DRDC-RDDC-2017-D133

15 Choose Body and click on the icon with three dots. The site address needs to be changed similarly as in the previous two steps in seven separate instances in total. Click OK. Save the workflow and then click Publish. DRDC-RDDC-2017-D133 7

16 Step 4: Go to the new site created in Step 2 using an internet browser, preferably Internet Explorer. Click on Lists. Choose Discussion List. From the List options, choose List Settings. Click Advanced Settings. Copy all the text from the URL Address from the Internet Explorer Window that appears after List= that stats with %7B. Using a text editor such as NotePad, change %7B into an open curly brace {, four instances of %2D into hyphens - and %7D into a close curly brace }. Keep this Global Unique Identifier (GUID) handy as it is required in the next step. Step 5: Click on Evaluation Survey Library. From the Library options, choose Library Settings. Click on Advanced Settings. Click (Edit Template). (InfoPath Designer opens.) In Fields, expand the Group DiscussionsList and choose properties under the ListID Field. 8 DRDC-RDDC-2017-D133

17 Paste the Discussion List GUID from Step 4 into the Value box and click OK. Click on the WG field. Click on Manage Rules and click on Rule 2. Click on the Change REST URL (REST stands for Representational State Transfer) action. Click on the f x button. DRDC-RDDC-2017-D133 9

18 Update the site URL to the new PEAR Site URL. In this case, the new formula reads: concat(concat(" c/discussionlist()?($expand=wg/id)&$filter=((wg/id%20eq%20", WGID, ""), ")%20and%20((Detail%20gt%20'')%20or%20(Comment%20gt%20''))%20and%20(Include%20e q%20true))", ""). Save the form then choose Quick Publish. Close InfoPath. Step 6 (Optional): Test the survey. Return to the PEAR tool home at its new location. Edit the WG 01 test item to be in the Evaluation Phase and then click on Evaluation Survey Library. Click add document. Try to add some comments and new discussions and then submit the survey. Check if the comments were added to the Discussion List. If so, then this completes this test. Step 7: Click on Review Survey Library. From the Library options, choose Library Settings. Click on Advanced Settings. Click (Edit Template). In Fields, expand the Group Status and click on the WG field. Click on Manage Rules and click on Rule DRDC-RDDC-2017-D133

19 Click on the Change REST URL action. Click on the f x button. Update the site URL to the new PEAR Site URL. Click OK. Save the form then choose Quick Publish. Click OK and quit InfoPath. Step 8 (Optional): Test the survey. Return to the PEAR tool home at its new location. Edit the WG 01 test item to be in the Review Phase and then click on Review Survey Library. Click add document (or click the link that is provided in the that is automatically sent to you). Complete the survey and click Submit. If the survey appears in the Review Survey library then this completes this test. DRDC-RDDC-2017-D133 11

20 Step 9: Click on Review Survey Library. From the Library options, under Current View: choose All Documents. From the Library options, click on Modify View. Copy the text from the URL Address from the Internet Explorer Window that appears after List= and after View= stopping at the &Source. Change all %2D into hyphens and remove the first and last three characters from the View GUID ( %7B and %7D ). Keep these strings handy. Choose View All Site Content under the Site Actions. Click on Requisite Documents. Choose Edit in Microsoft Excel from the list of choices under the ResultCalcOp file. 12 DRDC-RDDC-2017-D133

21 Do not enable content. Click on Connections from the Data command ribbon. Click on Properties for the data connection owssvr. Under the Definition Tab, in the Command Text box, replace the DRDC-RDDC-2017-D133 13

22 occurrences of the old PEAR Tool URL with the new PEAR Tool URL, and the portions in between the curly braces with the appropriate List and View GUIDs from earlier in this step. This is the complete text under Command Text from the example provided: <LIST><VIEWGUID>{97D7FEE6-6F55-49D2-94D4-6864BF23527A}</VIEWGUID><LISTNAME>{a1cf301a-d81f d45- b a17}</listname><listweb> EARInstall/_vti_bin</LISTWEB><LISTSUBWEB></LISTSUBWEB><ROOTFOLDER>/sandbox/ PRICIEG/newlocation/PEARInstall/Review%20Survey%20Test</ROOTFOLDER></LIST> Click on OK. (Sheet4 becomes the active sheet). Click on Sheet1 to make this the active sheet. Save the Workbook and quit Microsoft Excel. Step 10: Click on Result Viewer Forms Library. From the Library options choose Library Settings. Click on Advanced Settings. Click (Edit Template). In InfoPath Designer, choose Publish under the File command ribbon, and then click Export Source Files. 14 DRDC-RDDC-2017-D133

23 You should save the source files in a new folder in a location on your computer (the desktop will work). Close InfoPath. Inside the folder, find the file called manifest.xsf. Right-click on this file and select open with, then use a plain text editor (WordPad will work). Perform a Replace action that replaces all the old sandbox location addresses with the new site URL. In this example the Find what string was peartool/pear2/ and the Replace with string was newlocation/pearinstall/. Save the file and close the editor. Right-click on the manifest.xsf file and choose Design. Expand the group ordering and then for the Field ListID, choose properties and paste the Discussion List GUID (from step 4) to the Value of this Field. Save the form and then choose Quick Publish. Click OK and quit InfoPath. DRDC-RDDC-2017-D133 15

24 Step 11 (Optional): Test the Result viewer. Return to the PEAR tool home at its new location. Edit the WG 01 test item to be in the Planning Phase. Click the second link that is provided in the that is automatically sent to you. Choose the WG in the dropdown menu provided and then, on the next view presented, click Save and Close. If the survey appears in the Result Viewer Forms library then this completes this test. The PEAR tool is now installed in the new location. For troubleshooting, this reference document covers the components of the tool in more detail. 3.3 MS SharePoint Custom Lists Working Group (WG) List Figure 1: WG list. Figure 1 shows the default view of the WG List (with one WG, WG 01 new, shown). Additional columns are in this custom list, shown in Figure DRDC-RDDC-2017-D133

25 Figure 2: WG list settings. Two of the columns are calculated, WG which simply equals the Title, and Phase which equals the PEAR Phase. The list of choices for PEAR Phase is the following: 1. Initializing by OPI 2. Evaluation phase by WG 3. Aggregation phase by OPI 4. Review phase by WG DRDC-RDDC-2017-D133 17

26 5. Planning 6. Running 7. Closed Discussion List Figure 3: Discussion list. Figure 3 shows the Discussion list default view. In this view, group by uses Element, and (for when the list becomes too large, because it contains the data collected for all WGs) Filter for WG:Phase not equal to Closed. Using MS SharePoint Designer, the dialogue window for the Add new item command can be customized to request and specify which fields are required. The fields required to be not blank for the tool to function are shown in Figure 4 (red asterisks added manually in SharePoint Designer). 18 DRDC-RDDC-2017-D133

27 Figure 4: Add new item dialogue. The Discussion list contains many columns not shown in Figure 4. The full list of columns are in the Discussion list settings, shown in Figure 5. DRDC-RDDC-2017-D133 19

28 Figure 5: Discussion list settings. None of the columns in Figure 5 are set up as required in SharePoint, but the four previously mentioned columns do require data for PEAR to function properly. In fact, several steps had to be taken to modify Title column so that it too is not required. The choice values for Element are: 1. P: Personnel and Leadership 2. R: Research & Development, and Operational Research and Analysis (plus Experimentation) 3. I: Infrastructure, Environment and Organization 4. C: Concepts and Doctrine 5. I: Information Management and Technology 6. E: Equipment and Support 7. G: Generate 8. Other 20 DRDC-RDDC-2017-D133

29 The choice values for Topic are (in addition to allowing fill-in values): 1. P 2. Career Management Policy 3. Manning Priority 4. Tour Length 5. Regimental Affiliation 6. Military Occupational Structure Identification (MOS ID) Implications 7. Strategic Intake Plan (SIP) 8. Annual Military Occupation Review (AMOR) Implications 9. Army Succession Planning 10. Health, Welfare and Retention Initiatives 11. Proposed Person-Year (PY) Structure 12. Establishment Change (EC) Actions 13. R 14. Trials and Wargames 15. Research & Development (R & D) Planning 16. Army Experiment Series 17. Field Trials 18. I 19. Infrastructure Development/Re-Capitalization Plan 20. Second and Third Order Effects Infrastructure Funding 21. Programmatics 22. Occupancy Timelines 23. Environmental Impact 24. C DRDC-RDDC-2017-D133 21

30 25. Concept Development 26. Capability Development Record (CDR) requirements 27. Design Development Army of Tomorrow Linkages 28. Design Development Army of Today / Current Operations 29. Collective Training (CT) Concept 30. Continuation Training Concept 31. Battle Task Standard (BTS) & Training Packages 32. Doctrine Gap Analysis 33. I 34. Information Management / Information Technology (IM / IT) 35. Interoperability Canadian Armed Forces (CAF) Systems 36. Public Affairs / Internal Communications Plan 37. E 38. Linkages to Equipment Strategy 39. Procurement and Fielding 40. Entitlements / Distribution 41. Disposal / Divestment 42. Supply Concept 43. Maintenance Concept 44. G 45. Force Generation (FG) Output 46. FG Base and Managed Readiness Plan (MRP) 47. FG Training Requirements 48. FG Resource Allocation 49. Lead Mounting Division 22 DRDC-RDDC-2017-D133

31 50. CAF Support to FG 51. Professional Development (PD) Initiatives 52. Individual Training Concepts 53. Individual Training Programming 54. Other 55. Divisions 56. Corps and Branches 57. Lessons Learned 58. Allies WGDiscussion and WGDiscussionID are two columns needed for background processes of the tool. When a new item is added to the list, a workflow performs several background operations. DRDC-RDDC-2017-D133 23

32 3.4 MS SharePoint Workflows Workflows are created using MS SharePoint Designer Discussion List New Item Figure 6: Discussion list new item workflow in MS SharePoint Designer. 24 DRDC-RDDC-2017-D133

33 Figure 7: Discussion list new item workflow in editor. DRDC-RDDC-2017-D133 25

34 The Discussion List New Item workflow in editor is shown in Figure 7. Step 1 of this workflow simply adds a title value equal to the name of the person that created the item. In step 2, if the item is a new discussion (Discussion equal zero) the workflow will assign a sequential discussion number, using WGDiscussion as a lookup. To ensure the workflow will work for subsequent items, it clears WGDiscussion from the looked up item and sets the current item s WGDiscussion. When the item is a comment or a result item (non-zero Discussion) the workflow looks up the Element and Topic values using WGDiscussionID as the lookup key. WGDiscussionID is not unique in this list so the lookup performed returns those two fields from the first row in which WGDiscussionID matches the lookup key WG List Edit Item Figure 8: WG list edit item workflow in MS SharePoint Designer. 26 DRDC-RDDC-2017-D133

35 Figure 9: WG list edit item workflow in editor. The WG List Edit Item workflow sends an to the creator of the corresponding WG item as well as the person who edited the item, if different from the creator. The sent is dependent on the phase that the WG is entering, meaning the value of PEAR Phase. For the Evaluation and Review phases, this contains a link to the appropriate InfoPath survey for those phases which the OPI can then copy and send to the SMEs they have invited to participate. For the Analyze phase, the contains a link to an Excel workbook ResultCalcOp.xlsm for the preliminary step to this phase and the link to the results viewer InfoPath form. The workbook, hosted in the SharePoint document library, Requisite Documents, contains a data connection that needs to be manually refreshed. The workbook contains Visual Basic for Applications (VBA) macros that will perform this refresh, check-in the document and close it for the OPI. By clicking the link and following the instructions, the preliminary step is completed. 3.5 MS InfoPath Forms InfoPath forms are designed using InfoPath Designer. A form template (form) is the basis for the Evaluation and Review surveys that are distributed to the SMEs as well as the Results viewer in the tool. The terms form and survey will be used interchangeably. Forms consist of: form behaviours (rules), form data fields, data connections, including relational information on the SharePoint form library that houses the form template and completed forms, secondary data connections that can be dynamic or static and views (form data fields layout and formatting). Forms, although shown in the directory as a single file, are in fact bundles of multiple files called source files, DRDC-RDDC-2017-D133 27

36 which are responsible for describing different parts of the forms. The main file containing these parts is a.xsn file. It can be opened from within InfoPath Designer 2013 or by right-clicking on the file and selecting Design from the options. Sometimes it is required to hack the source files of a form. 1 First, choose Export Source Files from within InfoPath Designer, Publish options. This saves the many supporting files into their own folder. Locate and open the Manifest.xsf file with a text editor such as Notepad. The hack is essentially to modify a rule by adding a filter (i.e., [@name = Title ]) which sets the value of a particular instance of a repeating field. Save the Manifest.xsf file. When completed, the Manifest file must be opened by right-clicking and selecting Design. Then perform Save as to resave the new, hacked form in the original directory. Always create backups of the working form before performing the hack in case something goes wrong and the Manifest file will not open Evaluation Survey Form General Appearance and Flow Participants completing the Evaluation Survey are asked to evaluate a list of discussions for an upcoming WG, consider adding comments as well as new discussions. The maximum number of discussions that can be included in the tool is 200. The Evaluation Survey s appearance (in InfoPath Designer) is provided in Figure 10. It consists of four main parts: instructions and profile; evaluation of discussions (add comments); new discussions; and, conclude survey. Inactive sections are hidden from view when the form is being filled. 1 The hack is explained at the following link: 28 DRDC-RDDC-2017-D133

37 Figure 10: Evaluation Survey appearance (in design view). DRDC-RDDC-2017-D133 29

38 Each part has a button to activate as well as one to hide the section. Only one button is visible at a time per part, thanks to a set of flag fields within the form s design. When the form opens, Part 1 is shown (active) by default, whereas other parts are hidden. There are some form design choices that are intended to guide the participants into completing the survey as intended. For example, Part 3: new discussions only becomes available to the participants after all discussions in Part 2 have been viewed. While it is not necessary for participants to add any comments, they may be more likely to add comments with this type of design. It also helps to reduce duplicate or similar discussions from being introduced as new discussions by the SMEs in Part 3, because they should have viewed all of the existing discussions at that point in the survey. There could still be duplicate discussions however if different participants were completing their surveys at the same time: they could have the same idea at the same time. 2 Part 4 is enabled at the same time as Part 3, since Part 3 is optional. In Part 2, clicking View comments will reveal the comments contributed by the other participants (if any). Next and back buttons are used to navigate the discussions in the tool. The SME can add up to one comment for each by typing it into the Comment text box. When there are no discussions to view next, the buttons to activate Parts 3 and 4 will become enabled. To add a new discussion, the participant must first select the applicable PRICIE + G Element, then select from the choices provided or type a custom value into the Topic combo-box (a dropdown menu with ability to fill-in any value), provided it is less than 255 characters long and then fill in the textbox beside Elaborate, giving sufficient description of their discussion. The Submit new discussion button will become enabled. Changes to either the Element or Topic selections can be made at this time, and provided neither are made blank, the Submit new discussion button remains enabled. When ready, the participant can click Submit new discussion and their discussion will be added to the Discussion list on the SharePoint and these fields cleared for the ability to add more. Any form opened afterwards and/or any form that re-queries the Discussion list will contain this new discussion as part of its existing discussions. Essentially, the content of the Evaluation Survey is dynamic Data Connections The data connections for the Evaluation Survey form are given in Table 1. Connection name WG List RanksBilingual Table 1: Evaluation Survey data connections. Type Identifying information Data source type Retrieve SharePoint site: Secondary data data source Retrieve data EARTool/PEAR2/ XML file: RanksBilingual Secondary data source Data retrieved Every time form is opened 2 Within the amount of time that both of their surveys are open, each could add a new discussion that the others survey would not contain as part of the existing discussions. Updating occurs once, when the form is opened, and any time a new discussion is added by the current participant. The form, once open, cannot query new discussions (added by other participants) unless the current participant adds a new discussion of their own, at which time, if Part 2 is reactivated, a re-query of the Discussion list is made. The same is true for comments made by others: they will not be included until they are first submitted into the Discussion list, and the current form is loaded or a re-query is made. The implications of this design are deemed to be of minimal impact to the WG. 30 DRDC-RDDC-2017-D133

39 Connection name GetUserProfileB yname SubmitToList 3 REST Web Service PRICIEGData 4 SharePoint Library Submit Web Service Submit 5 Type Identifying information Data source type Retrieve Web service: Secondary data data source Service.asmx Operation: GetUserProfileByName Retrieve XML file: SubmitToList Secondary data REST Web Service Retrieve data Submit data Submit data Web service: EARTool/PEAR2/_vti_bin/listdata.svc/ DiscussionList()?($expand=WG/Id)&$f ilter=((wg/id%20eq%201)%20and%2 0(Detail%20gt%20'')) XML file: PRICIEGData SharePoint library or list: EARTool/PEAR2/Evaluation%20Surve y/ File name (expression): concat(firstname, " - ", WG, now()) Allow overwrite if form exists: yes Web service: EARTool/PEAR2/_vti_bin/lists.asmx Operation: UpdateListItems data source Secondary data source Secondary data source Data retrieved Every time form is opened Every time form is opened Any REST Web Service connection consists of a query URI. In this case, Table 1 shows the URI used during development, which must be valid. However, this initial URI is changed (by an InfoPath rule) when the WG is selected in Part 1. The new URI is a string generated by a formula concat(concat(" $expand=wg/id)&$filter=((wg/id%20eq%20", WGID, ""), ")%20and%20((Detail%20gt%20'')%20or%20(Comment%20gt%20''))%20and%20(Include%20eq%20tru e))", ""). The new URI preserves the source of the data from the initial URI, but may contain parameters specifying the data to be returned. All Web service, SharePoint site, library or list connections will require updating in InfoPath Designer as part of an install of the tool onto a new SharePoint site except GetUserProfileByName and SharePoint 3 See sections on CAML and on adding the SharePoint data connection to receive data following them (that part is not necessary but helps with ensuring the field names in the XML file match the column names in the destination SharePoint list) at InfoPath.aspx, for an example of this type of data connection. 4 See acabacb76605, for an example of this type of connection. 5 See the three sections, create the default view, add the connection to the SharePoint web service and add the submit section in the same reference as Footnote 3 for an example of this type of data connection. DRDC-RDDC-2017-D133 31

40 Library Submit. This is because URL addresses and SharePoint lists will have changed. Also, the XML file in SubmitToList may require changes especially if the new Discussion list columns aren t the exact same as the documented version of the tool. 6 The GetUserProfileByName connection will require updating if the tool is being installed on any SharePoint that is not Army Collaborative Information Management Services (ACIMS). The SharePoint Library Submit will be replaced entirely during publishing of the Evaluation Survey (settings in the InfoPath publishing wizard dialogues). Testing that these components function properly after updating them is highly recommended because once the submit to list function doesn t work properly it is very difficult to determine the cause of the problem Review Survey Form General Appearance and Flow Participants completing the Review Survey form are asked to provide work experience levels in PRICIE + G analysis by PRICIE + G Element and to then assess each discussion on two Assessment measures, Relevance and Complexity, using the Assessment matrix provided. The Review Survey s appearance (in InfoPath Designer) is provided in Figure 11. Similar in design to the Evaluation Survey, it consists of four main parts: instructions and profile; PRICIE + G analysis experience; review and assess discussions; and, conclude survey. 6 It may be necessary to modify the manifest file: for additional information see and the discussion at for an example most relevant to this tool. 32 DRDC-RDDC-2017-D133

41 Figure 11: Review Survey appearance (in design view). DRDC-RDDC-2017-D133 33

42 In the same manner as the Evaluation Survey, not all parts in the survey are active at the same time: SMEs must complete each part in turn, submitting only after having completed all sections Data Connections The data connections for the Review Survey form are given in Table 2. Connection name WG List GetUserProfileB yname REST Web Service SharePoint Library Submit Table 2: Review Survey data connections. Type Identifying information Data source type Retrieve SharePoint site: Secondary data data source Retrieve data REST Web Service Submit data RTool/PEAR2/ Web service: vice.asmx Operation: GetUserProfileByName Web service: RTool/PEAR2/_vti_bin/listdata.svc/Discus sionlist()?($expand=wg/id)&$filter=((w G/Id%20eq%201)%20and%20(Include%2 0eq%20true)) SharePoint library or list: RTool/PEAR2/Review%20Survey%20Test / File name (expression): concat(firstname, " - ", WG, now()) Allow overwrite if form exists: yes Secondary data source Secondary data source Data retrieved None of the data connections are substantially different than those in the Evaluation Survey with the same names. The WG List and REST Web Service connections will require updating during an install of the tool. The SharePoint Library Submit needs to be replaced during installation as well, using the InfoPath publish settings. During the publishing of the Review Survey, it is important to promote the Experience data fields (Exp 1 7) and the Assessments field onto the Review Survey Library (in Form Options Property Promotion). This is necessary for the Result processing and Results viewer components to function properly. Other required promoted fields are WG and FormStatus Results Viewer Form General Appearance and Flow The Results viewer is used by the OPI during Planning and Running of the WG. 7 For the Evaluation Survey, the promoted fields are the FirstName, LastName, Rank, Org, and WG but are not critical to any functional aspects of the tool. 34 DRDC-RDDC-2017-D133

43 The Results viewer s Planning view (in InfoPath Designer) is provided in Figure 12. The Running view (in InfoPath Designer) is provided in Figure 13. The Medians of all SME assessments High, Med-High, Medium, Med-Low or Low for Relevance and Complexity; and, Perfect, Significant, Moderate, Insignificant, Chance or <Chance for Agreement as well as the ranks and numbers of distinct ranks for each discussion can be viewed. For discussions within one of the seven PRICIE + G Elements, the Experts Assessments (if any) can also be viewed. Chance (<Chance) indicates there is no agreement (less agreement) beyond which can be expected by chance. Na is reserved to indicate there is insufficient data for a result. Experts Assessments are broken into two categories: only SMEs with significant experience; and, SMEs with at least moderate experience conducting PRICIE + G analysis within the PRICIE + G Element concerned. During Planning the discussions can be sorted by Relevance, Complexity, Agreement, Chronologically, by PRICIE + G Element, and by the custom ordering, once it has some discussions within it. The intent is for the OPI to create a custom ordering of discussions for the WG. During Running, when the group is holding the discussions, only the custom ordering and the PRICIE + G Element dropdown can be used to sort the discussions. Also, during Planning, when the custom ordering creation is ongoing, the OPI can test the partial custom ordering (discussions are ordered as during the WG when no PRICIE + G Element is selected), or within the Chronological ordering option only, the OPI may view only discussions not yet added to the custom ordering, which can make completing the custom ordering easier. Before the Running phase, the OPI must save the form with the completed custom ordering, update the phase of the WG in the WG list in SharePoint to Running, then return to the saved form for the WG. DRDC-RDDC-2017-D133 35

44 Figure 12: Results viewer s Planning view (in design view). 36 DRDC-RDDC-2017-D133

45 Data Connections Figure 13: Results viewer s Running view (in design view). The data connections for the Results viewer form are given in Table 3. Due to the large number of data connections in the Results viewer, Table 3 covers several pages. DRDC-RDDC-2017-D133 37

46 Connection name SubmitToList Relevance Median Mod Relevance Median Exp Kripp alpha Mod Kripp alpha Exp Complexity Median Mod Complexity Median Exp WG List Chronological Order Table 3: Results viewer data connections. Type Identifying information Data source type Retrieve XML file: SubmitToList Secondary data data source REST Web Service REST Web Service REST Web Service REST Web Service REST Web Service REST Web Service Retrieve data REST Web Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Releva_Median_Mod )?$f ormat=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Releva_Median_Exp )?$fo rmat=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Kripp_alpha_Mod )?$form at=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Kripp_alpha_Exp )?$forma t=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Complex_Median_Mod )? $format=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Complex_Median_Exp )?$ format=atom SharePoint site: RTool/PEAR2/ Web service: Secondary data source Secondary data source Secondary data source Secondary data source Secondary data source Secondary data source Secondary data source Secondary data source Data retrieved Every time form is opened 38 DRDC-RDDC-2017-D133

47 Connection name Agreement Rank All Agreement Order All Relevance Rank All Relevance Order All Complexity Rank All Complexity Order All Relevance Median All Type Identifying information Data source type Service RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Chrono_Order )?$format=a tom REST Web service: Secondary Web data source Service RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Agree_Rank_All )?$format REST Web Service REST Web Service REST Web Service REST Web Service REST Web Service REST Web Service =atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Agree_Order_All )?$forma t=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Releva_Rank_All )?$form at=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Releva_Order_All )?$form at=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Complex_Rank_All )?$for mat=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Complex_Order_All )?$for mat=atom Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Releva_Median_All )?$for Secondary data source Secondary data source Secondary data source Secondary data source Secondary data source Secondary data source Data retrieved DRDC-RDDC-2017-D133 39

48 Connection name Kripp alpha All PRICIEGData Discussions Complexity Median All Web Service Submit SharePoint Library Submit Type Identifying information Data source type mat=atom REST Web service: Secondary Web data source Service RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Kripp_alpha_All )?$format Retrieve data REST Web Service REST Web Service Submit data Submit data =atom XML file: PRICIEGData Web service: RTool/PEAR2/_vti_bin/listdata.svc/Discus sionlist()?($expand=wg/id)&$filter=((w G/Id%20eq%201)%20and%20(Include%2 0eq%20true)) Web service: RTool/PEAR2/_vti_bin/ExcelRest.aspx/Re quisite%20documents/resultcalcop.xlsm/ model/ranges( Complex_Median_All )?$f ormat=atom Web service: RTool/PEAR2/_vti_bin/lists.asmx Operation: UpdateListItems SharePoint library or list: RTool/PEAR2/Result%20Viewer%20Form s/ File name (expression): concat(wg, " - ", now()) Allow overwrite if form exists: yes Secondary data source Secondary data source Secondary data source Data retrieved Every time form is opened Each of the data connections using the Excel REST service refers to a named range within the Result processing workbook discussed in the next section. Within the Result viewer form rules, these REST queries are modified using formulas such as concat(" ents/resultcalcop.xlsm/model/ranges('complex_median_exp')?$format=atom&ranges('wg')=", WG, "&Ranges('PRICIE_El')=", ElementValue[RowInTable = Discussion]). Due to this fact, installing the tool on a different site on ACIMS would require changes in each case: in all the data connections in Table 3 using the Excel REST service and every Change REST URL rule within this form. 8 8 The Author has found a large number of changes is more easily effected on the source files of the InfoPath xsn file. Before attempting modifying any of the source files, be sure to save a backup of the xsn file so that if changes made 40 DRDC-RDDC-2017-D133

49 3.6 MS Excel Macro-Enabled Workbook (Result Processing Workbook) The Result processing workbook is a MS Excel macro-enabled workbook that contains a data connection to the specific List View (requires a VIEWGUID, in addition to a LISTNAME) of Review survey form library that shows all completed forms. The connection 9 imports the assessment values from each completed Review Survey in the Review Survey library. It should be set to Refresh data when opening the file and Refresh this connection on Refresh All, but not to Enable background refresh. Whenever the data is refreshed and the workbook saved back onto the SharePoint site (filename and SharePoint location must match the REST query strings used for the data connections in the Result viewer), the result viewer s REST queries will reflect the workbook. To open the workbook, locate the file in the Requisite Documents library in the SharePoint site. With the mouse cursor, hover over the file name to allow the dropdown to the right of the name to appear. From the dropdown menu, select Edit in Microsoft Excel. Do not Enable Content once it opens in Excel, because this will cause the workbook to follow the script (macro) which causes the data connection to be refreshed and the workbook to be saved back onto the SharePoint site and then close without prompt. Three sheets are present in the workbook. Sheet 1 is shown in Figure 14. to the source files causes an error while loading the manifest file, it is still possible to revert back to a last known working copy. Editing the source files individually using a text editor can speed up the process by using find and replace, for example. 9 To create this from scratch, start from SharePoint library and select Export to Excel, then from within Excel, select Export Connection File and save the file to your computer. Then, from a new workbook (the Result processing workbook), in Connections, select Add, Browse for More, find the file saved in the last step, then choose that file. After the connection is made, the My Documents view of the Review Survey library should be made the default view. DRDC-RDDC-2017-D133 41

50 A portion of Sheet 3 is shown in Figure 15. Figure 14: Result processing workbook Sheet DRDC-RDDC-2017-D133

51 Figure 15: Result processing workbook part of Sheet 3. The chart on Sheet 3 is the chart that is queried by the Result viewer in the data field called Experience Chart. That chart presents the data shown in cells B2:D9 on Sheet 3, itself populated from data contained on the next sheet, dynamically depending on the WG value, in cell B1. From column O, on to column HF, are the values contained in 200 columns (aligning with up to 200 discussions) that are responsible for processing the Review Survey assessments. They are treated in categories, first all responses, second, only the Expert responses (Exp for short), and then Moderate responses (Mod for short). On the calculation of Expert or Moderate responses, first the PRICIE + G Element on which the work experience level is sought is required to be input in cell C20, then the calculations are based only on counts where the respondent met the desired work experience level for the given PRICIE + G Element. 10 This is all done through use of REST queries from InfoPath Results viewer form to the Excel result processing file hosted on SharePoint (through what is known as Excel services). Sheet 4 is reserved for the data connection to the Review Survey library All Documents view. A VBA macro performs a text to columns operation for each Assessments value in the table, Table_owssvr, putting the result into the 200 columns immediately to the right of the Assessments column. In the event the number of columns in the Review Survey library All Documents view changes, these columns may not align with the proper column headings: Assmt 1, Assmt 2,, Assmt 200. The column headings would need to be realigned so as to appear immediately to the right of the Assessments column, and the column dimension of Table_owssvr may need to be adjusted before the tool will properly align SME assessments with the corresponding discussions. It is important to activate Sheet 1 before saving and then upload the workbook to the SharePoint library. 10 For R: Research & Development, and Operational Research and Analysis (plus Experimentation), due to the ampersand symbol, only R: Research is received as part of the REST query. A small change was made to the named range, PRICIEG_Elements, in the workbook to accommodate this. DRDC-RDDC-2017-D133 43

52 4 Part II: Walkthrough This second part of the document explains what to do in each step of using the tool. It is based on the current version of the tool, in the sandbox site on ACIMS. The tool s process is shown in a diagram in Figure 16. The SMEs complete two surveys during the Evaluation and Review phases. The OPI is responsible for the rest of the process. 4.1 OPI Steps Figure 16: PEAR Tool process diagram. The example used in the walkthrough is the Combined Joint Intelligence Modernization (CJIM) project which utilized the first working version of the tool in the Summer and Fall The first step for the OPI for a PRICIE + G analysis WG on CJIM was to create an item in the WG list on the SharePoint site Step 1: Adding CJIM to the WG List Navigate to the SharePoint site home page. Click on Add new item in the WG list. 44 DRDC-RDDC-2017-D133

53 Type CJIM into the Title field and click Save. The PEAR Phase will be set to the default value, Initializing by OPI. Once the WG CJIM is created, the OPI can begin adding discussions to the Discussion list. Table 4 presents a list of discussions added during the Initializing by OPI phase. Table 4: List of discussions added during Initializing by OPI for CJIM WG. PRICIE + G Element Topic Detail P: Personnel and Leadership Career Management Policy How many administrators (clerks) at each location will be required P: Personnel and Leadership Annual Military Occupation Review (AMOR) Implications What new training needs to be conducted for analysts P: Personnel and Leadership Proposed Person-Year (PY) Structure How many intrinsic transport personnel will be required to provide coordination/movement of unit assets and train personnel for personnel movement P: Personnel and Leadership Proposed Person-Year (PY) Structure How will feeding of personnel be accomplished P: Personnel and Leadership General Security How many security personnel DRDC-RDDC-2017-D133 45

54 PRICIE + G Element Topic Detail (commissionaires) will be required P: Personnel and Leadership Security in field How many security personnel will be required to provide security of facility in the field and who will provide it R: Research & Development, and Operational Research and Analysis (plus Experimentation) Trials What types of user trials will be necessary to validate capability and when will they be completed I: Infrastructure, Environment and Organization Infrastructure Development/Re- Capitalization Plan How much hangar space will be required for vehicles I: Infrastructure, Environment and Organization Infrastructure Development/Re- Capitalization Plan How much dedicated office space will be required C: Concepts and Doctrine Concept Development What will be the ramp up/ramp down concept for All Source Intelligence Centre (ASIC) Step 2: Adding Discussions during Initializing by OPI into the Discussion List From the home page click the link to the Discussion list. From the Discussion list, click Add new item. 46 DRDC-RDDC-2017-D133

55 Select Element, Topic (or enter topic), WG and enter Detail for discussion. DRDC-RDDC-2017-D133 47

56 Click save to finish adding the discussion to the list. After finished adding a discussion, the user is returned to the Discussion list page. This step must be repeated for each discussion. The Discussion list page contains the discussions for all WGs that are in the process of utilizing the tool. To show only the discussions for a given WG, the WG column can be filtered to the desired WG. Figure 17 shows how to do this. Figure 17: Filtering discussion list by WG Step 2b: Editing Items in the Discussion List (e.g., edit Detail of a Discussion) From the Discussion List page, locate the item to edit, hover over it with the mouse cursor, select the check box in the left-most column and click Edit Item in the List Tools, Items ribbon menu. When locating the desired item, it helps to know the list is arranged first in groupings by PRICIE + G Element, then sorted by discussion number (starts with 1 for each WG), then by chronological order by which the item was entered into the list. 48 DRDC-RDDC-2017-D133

57 Make the desired changes to the item using the form and click save. The tool may cease to function properly or as desired if these instructions are not followed: ensure that WG, Topic, Element and one of Detail, Comment and Result are not blank. Only modifications to the Topic, Element, WG, Include, Detail, Comment and Result fields are allowed. Do not input text in any combination of the Detail, Comment and/or Result fields text should appear in one of those three fields. DRDC-RDDC-2017-D133 49

58 Figure 18 presents a view of the Discussion list, filtered to the WG CJIM, after the Initializing by OPI phase has been completed. 50 DRDC-RDDC-2017-D133

59 Figure 18: Discussion list with OPI-added discussions Step 3: Distribute the Evaluation Survey to the SMEs From the Discussion list, within the Browse ribbon menu, click home to return to the WG list. Locate the WG, hover over it with the mouse cursor, select the check box in the left-most column and click Edit Item in the List Tools, Items ribbon menu. DRDC-RDDC-2017-D133 51

60 Choose the next PEAR Phase option, Evaluation phase by SME and click save. The tool will send both the creator of the WG item in the WG list, and the User completing this step (if they are different people) an with a link to the survey. Click the link and select open. 52 DRDC-RDDC-2017-D133

61 InfoPath opens the form by default. Completing this form takes some time, as participants (SMEs) are requested to evaluate and comment on the existing discussions and, then, consider if any else should be discussed and, if so, add new discussion(s). The more discussions involved, the longer it could take for participants to complete the survey. About 30 minutes to one hour should be a reasonable estimate to give the participants. After verifying the form s content, the OPI must compose an to the SMEs they have identified for the WG. The steps involved in identifying the SMEs for this process is beyond the scope of this document. Participants must have access to the ACIMS to complete the survey. If the OPI requires input from SMEs outside the network, they will have to be given hard copies of the discussions and comments after the participants using ACIMS have completed their surveys. That process is not described in this walk-through. The should contain: 1. The name of the WG (as entered in the WG list); 2. The link to the survey on ACIMS (the same link as the one in the tool generated ); 3. A specific time period for the SMEs to complete the survey (cut-off date); and, 4. Any other specific instructions from the OPI Step 4: Aggregating Responses Collected in the Evaluation Survey Any SME comment or new discussions submitted using the Evaluation survey are by default included for subsequent review and assessment by SMEs and at the WG. Aggregating responses thus requires specifying which comments or discussions to exclude. Complete this step only after the cut-off date given in the previous step. Set the WG phase of the WG CJIM to Aggregation phase by OPI (not shown). DRDC-RDDC-2017-D133 53

62 Within the Discussion list, expand the WG CJIM by clicking the small + symbol next to the WG or filter the discussions by the WG. Proceed to expand each PRICIE + G Element to view newly added items in the list. They will either be a comment or a new discussion. In this example, the OPI deemed that one discussion added by a SME is covered elsewhere or does not warrant discussion, and edits the item by unchecking the Include checkbox. This is how to remove discussions from the tool, rather than deleting the row from the list. (Because of background processes, i.e., workflows, deleting a row can cause the tool to malfunction.) 54 DRDC-RDDC-2017-D133

63 Effect the desired change (e.g., uncheck the checkbox) and click save. If there are comments linked to an unwanted discussion (appear directly beneath the discussion in the list), they must also be edited (de-selected) in this way. Any item can be edited by the OPI. Follow the instructions in Step 2b Step 5: Distribute the Review Survey to the SMEs From the Discussion list, within the Browse ribbon menu, click home to return to the WG list. DRDC-RDDC-2017-D133 55

64 Locate the WG, hover over it with the mouse cursor, select the check box in the left-most column and click Edit Item in the List Tools, Items ribbon menu. Choose the next PEAR Phase option, Review phase by SME and click save. 56 DRDC-RDDC-2017-D133

65 The tool will send both the creator of the WG item in the WG list, and the User completing this step (if they are different people) an with a link to the survey. Click the link and select open. DRDC-RDDC-2017-D133 57

66 The Review Survey should not take as long to complete as the Evaluation Survey. This is because only quantitative data is requested, in the form of assessments of the discussions. After verifying the form s content, the OPI must compose an to the SMEs they have identified for the WG. The steps involved in identifying the SMEs for this process is beyond the scope of this document. Participants must have access to the Army Collaborative Information Management Services (ACIMS) to complete the survey. If the OPI requires input from SMEs outside the network, they will have to be given hard copies of the survey. That process is not described in this walkthrough. The should contain: 1. The name of the WG (as entered in the WG list); 2. The link to the survey on ACIMS (the same link as the one in the tool generated ); 3. A specific time period for the SMEs to complete the survey (cut-off date); and 4. Any other specific instructions from the OPI Step 6: Specify an Order of Discussions for the WG Navigate to the SharePoint site home page. Locate the WG, hover over it with the mouse cursor, select the check box in the left-most column and click Edit Item in the List Tools, Items ribbon menu. 58 DRDC-RDDC-2017-D133

67 Choose the next PEAR Phase option, Planning and click save. The tool will send both the creator of the WG item in the WG list, and the User completing this step (if they are different people) an with three links, one to the Result Processing Excel file, another to the Results Viewer template and the third to the SharePoint library where the InfoPath forms are stored (for the WG). DRDC-RDDC-2017-D133 59

68 Follow the instructions in the to update the Result processing workbook before opening the Result viewer. In the Result viewer, select the WG CJIM and click next. If the wrong WG is selected, the form must be reloaded. Only the WGs currently in the Planning phase will be shown in the dropdown. 60 DRDC-RDDC-2017-D133

69 Click next. The discussions can be viewed using the back and next buttons to navigate. Select a PRICIE + G Element to view the discussions in that element, or choose from one of the orderings available. The assessment results can be viewed by clicking Show assessment results. DRDC-RDDC-2017-D133 61

70 Specify a custom ordering by numbering the discussions in the desired order (1, 2, 3, ). Leave the custom ordering blank or zero to omit it from the custom ordering. Click save and close when your custom order is complete Step 7: Running the WG; Enter Results of the Discussions This step is for entering records of decision at the WG into the tool. Navigate to the SharePoint site home page. Locate the WG, hover over it with the mouse cursor, select the check box in the left-most column and click Edit Item in the List Tools, Items ribbon menu. 62 DRDC-RDDC-2017-D133

71 Choose the next PEAR Phase option, Running and click save. Return to the saved Results viewer form on the SharePoint site. aspx Click on the Name of the file to open and select Open. In the initial view, click Next to start holding the discussions. DRDC-RDDC-2017-D133 63

72 To enter results, type them into the Result text box for each discussion. The Results viewer form can be saved by clicking the disc-shaped icon in the top left of InfoPath at any time (e.g., at the end of the day) to resume at a later time. When all results have been collected (e.g., at the end of the WG) click Submit all results and close to add the results to the Discussion list in the SharePoint site. 4.2 SME Steps Step 1: Completing the Evaluation Survey As an SME requested to complete the Evaluation Survey, an will have been received. 64 DRDC-RDDC-2017-D133

73 It is important to complete the survey a within the timelines specified by the OPI in this . Click on the link to the survey and select open when prompted. Some versions may differ in appearance. A warning message may appear. Click Ok to proceed. DRDC-RDDC-2017-D133 65

74 When the survey opens, some personal information will be populated such as name and . Select the WG title from the dropdown and complete the remaining personal information fields. Click on Part Use the back and next buttons to navigate through the discussions adding any comments. If other comments were added previously (by other SMEs completing the survey), they may be viewed by clicking View comments. 11 If the wrong WG is selected, at this point it cannot be changed and the survey needs to be restarted. 66 DRDC-RDDC-2017-D133

75 Once all discussions have been evaluated, Part 3 and Part 4 are enabled. Click Part 3 to add new discussions. To add a discussion, select the PRICIE + G Element and Topic and type an elaboration for the discussion. Click on Submit new discussion. Once the survey has been completed to satisfaction, click Part 4 and click Submit all comments and close. This will close the survey. If applicable, it is possible to save a partially complete survey to the desktop (or some other location) and continue it later (not shown) Step 2: Completing the Review Survey As an SME requested to complete the Review Survey, an will have been received. DRDC-RDDC-2017-D133 67

76 Open the survey similarly to the previous step. Select CJIM from the WG title dropdown and then proceed with Step 2. Select the values that most represent your work experience in each PRICIE + G Element. 68 DRDC-RDDC-2017-D133

77 Click on Part 3, then click next to begin. For each discussion, make an assessment using the Assessment matrix. For example to assess the discussion a Medium Complexity and High Relevance choose the option in the fourth column and third row. An assessment is required for each discussion before Part 4 is enabled. However, it is always possible to select Don t know for either Assessment Measure, or both, if desired. This has no effect on the results of the Review Survey. When all assessments are complete, click Part 4, then Submit your answers and close. DRDC-RDDC-2017-D133 69

78 70 DRDC-RDDC-2017-D133