Working with Workbooks In Datameer, you can create a workbook to get to new insights with your data. Inside the workbook, you can add additional data sources, change the column and sheet names, collapse columns, apply formulas, view workbook details, and more. You can also copy workbooks, import worksheets from other workbooks, duplicate workbooks and worksheets, and exchange datasources. Creating a Workbook Sample data and full data Sample data messages Full data messages Opening Existing Workbooks Opening a workbook using a URL Opening a prior version of a workbook Adding Additional Data Sources Exchanging Data Sources Working with Sheets and Columns Viewing sheets within a workbook Distinguishing different sheet types Setting up sheet names Deleting sheets in a workbook Duplicating sheets in a workbook Reordering sheets in a workbook Reordering columns in a workbook Sheet dependency graph and navigation Importing sheets Suggested column names Setting column names Resizing columns Hiding and expanding columns Inserting columns Removing columns Cell context menu options Formatting columns Shading rows in a workbook Copy cells Workbook column inspector Revert Actions Using Formulas Creating formulas Editing formulas Completing Workbook Level Tasks Saving workbooks Creating multiple workbooks Calculating a workbook Viewing full results in a workbook Going to line in a workbook Viewing workbook details Changing workbook settings Integrating workbook results with other systems Setting Workbook Sharing Permissions Security Sharing Full results sharing Creating a Workbook To create a workbook: orkbook. From the File menu, select Add Data. 4. Choose Workbooks from the + (plus) button or right-click in the navigation bar on the left side of the screen and select Create New > W Navigate to the folder where the data sources are located and select the name of the data source you want to use. Click Add Data. As of Datameer 6.3 When using data from a sheet in another workbook or an import job in Datameer 6.3, the data from that source sheet is copied to the new sheet if the new sheet is marked as kept. In previous versions, the source sheet was referenced instead, which led to workbook data not being deleted by housekeeping.
The workbook is populated with a sample of data from that data source. To use data in Datameer, that data must be part of a data source. If you have a source of data you want to use for analysis, such as an Excel spreadsheet, you need to create a data source using that data, or have a system administrator create one for you. Sample data and full data Data displayed in Datameer workbooks is a sampling of the full data from the source. When creating or editing an import job, file upload, or data link, you set the number of sample records to populate a workbook from the specific data source being used. A data source added to a workbook displays the number of sample records configured for that source. When creating calculations in the workbook, those calculations are being applied only to the sample data until the workbook has run. After running a workbook, the calculations made in the workbook are applied to the full data and can be viewed or downloaded by right-clicking on the workbook and selecting Browse All Data. The next time the workbook is opened after running, while the calculations have been applied to the full data, you still view the sample size configured from the data source. As of Datameer v6.4 Status notifications have been added to the bottom of workbook worksheets to show if the worksheet is dispaying information based on sample or full data and if the worksheet is filtered.
Sample data messages If the worksheet's caculations are based only on sample data, the notification is orange. It tells you how many sample rows are being displayed in the worksheet and that you can run the workbook to display the sheet caculations based on the full data from the data source. If the worksheet is displaying no information, this could be that none of the origial sample data fits the criteria of the caculations. Running the workbook takes into account all current worksheet caculations and might bring in new sample data to populate the worksheet if data from the data source meet the caculation criteria. Full data messages If the worksheet's caculations are based on the full data, the notification is green. It tells you how many sample rows are being displayed in the worksheet out of how many total records from the data source. If all the records of the source are being used, the status bar displays the total record count. Opening Existing Workbooks To open an existing workbook: On the FileBrowser tab, find the workbook you want to open. Highlight the workbook by clicking on it. Double-click to open or right-click on the highlighted workbook and select Open.
Opening a workbook using a URL A workbook can be opened by a user with permissions to view the workbook by entering a URL into the browser. There are two valid URLs that can open a workbook, using the ID or the path. Workbook ID: http(s)://<server>:<port>/workbook/<wbkid> Example: https://localhost:8080/workbook/25 Workbook path: http(s)://<server>:<port>/workbook?path=<path/to/workbook.wbk> Example: https://localhost:8080/workbook?path=users/matthew/seasonal_earnings Open a workbook to a specific worksheet: http(s)://<server>:<port>/workbook?path=<path/to/workbook.wbk>&sheet=<sheetn ame> Example: https://localhost:8080/workbook?path=/users/matthew/seasonal_earnings.wbk&sheet=products Opening a prior version of a workbook Datameer gives you the option of saving previous versions of a workbook when it is run more than once. Learn more about optimizing your workbooks and data retention. To open a prior version of a workbook: On the File Browser tab, find the workbook you want to open. Highlight the workbook by clicking on it. Right-click on the highlighted workbook and select Show Details.
4. Find the previously run workbook iteration in the Current Data section. Click the Show Data icon. 5. From the Full Results page, you can open the workbook or download the data. For convenience, other previous ran workbook iterations have a link.
Adding Additional Data Sources To add additional data sources to an existing workbook: 4. From the File menu, choose Add Data or click the Add Data icon on the toolbar. Navigate to the folder where the data sources are located and select the name of the data source you want to use. Click Add Data. Repeat the previous steps for each data source you want to use. For each data source you select, a tab is added to the workbook, containing a sample of data from that data source. You can add multiple data sources at the same time by holding the SHIFT button. Exchanging Data Sources You can exchange data sources to run a workbook on a different set of similar data, which is faster than recreating the workbook. For example, you can run a workbook on data for each month by exchanging the data source to point to the data for the next month. Each datasource must have the same structure. Further analysis applies only to the new data after the exchange. This option is only available when the current data sheet uses a data source imported by adding data. The option isn't available for sheets created using filters, joins, sorts, external datasheets, or formulas. You must also have write permissions for this workbook to use this option. To exchange data sources in an existing workbook: From the File menu, choose Exchange Data or click the Exchange Data icon on the toolbar. Navigate to the folder where the datasources are located and select the name of the data source you want to use. Click Exchange Data. Errors might occur when exchanging data if column names have changed. On a joined sheet, original references to columns remain even if that column name doesn't exist in the new exchanged data. If this happens, replace the joined sheet reference with the updated column name from the exchanged data. Working with Sheets and Columns You can set up sheet names, resize columns, collapse and show columns, rename worksheets, import worksheets from other workbooks, and more. Viewing sheets within a workbook There are multiple ways to navigate between sheets within a workbook: Click the sheet tab you want to view a the bottom of the page. Select the Sheets menu and click the sheet you want to view. Click on the Sheet Dependencies button in the toolbar and switch between sheets by clicking on sheets that are dependent with each other. Distinguishing different sheet types There are eight different worksheet types in Datameer. Only the formula ( Fx) worksheet type analyzes data using the formula bar or with the function wizard. Data source Join Filter Clustering Decision tree Recommendations Column dependencies Formula Setting up sheet names You can rename workbook sheets. To change the name of a sheet:
Right-click the sheet name and click Rename. Enter the new name and press Enter. To undo the change, right-click the name again and click Undo. Deleting sheets in a workbook You can delete sheets in a workbook that you don't want to use or keep. To delete sheets: Click the tab in the workbook that you want to delete. Right-click the tab and choose Delete. Click Delete. Duplicating sheets in a workbook You can duplicate workbook sheets - so you can create variations on a theme. The new sheet is named with the next available number. If there are five sheets and you duplicated Sheet3, the new sheet is named Sheet6. To duplicate sheets in a workbook: Click the tab in the workbook. From the Workbook menu, select DuplicateSheet, or click the Duplicate Worksheet icon on the toolbar. You can also right-click the tab and select Duplicate. Select the columns you want to copy to the new sheet and click CreateSheetCopy. Available as of Datameer v6.3 In Datameer v6.3, you cannot only copy the raw data of a sheet with formulas, joins, sorts, or clustering, but you can also copy the underlying logic and actions to the new sheet. If a sheet in a workbook has actions applied to it, you can copy both the logic and the data or select the Copy data only checkbox to copy over only the data in the sheet without the logic. Reordering sheets in a workbook You can reorder sheet locations inside of a workbook. To reorder sheets in a workbook: Right-click the sheet tab. Select Move. Click the space between the current sheet tabs where you want the selected sheet to be placed. Reordering columns in a workbook You can reorder columns on a sheet inside of a workbook. To reorder columns on a sheet: Left-click and drag the column name to the new position. Sheet dependency graph and navigation The graph overlay shows the current selected sheet with in and outgoing sheets in a visual manner. View the sheet dependency overlay by clicking the Sheet Dependencies button in the toolbar. The current sheet in which you have selected with be highlighted in the middle of the graph. To navigate between dependent sheets, click the sheet name located on the graphic overlay. To close the sheet dependencies overlay, click the toolbar button or click X on the current sheet. Importing sheets Importing sheets allows you to chain workbooks together. To import sheets from one workbook into another workbook:
4. 5. Open the workbook you want to import the worksheet into. From the Workbook menu, select Add Data, or click the Add Data icon on the toolbar. Select the workbook you want to use and click Add Data. Select the sheets you want to import and click Add Data. The new worksheet tabs are added to the right of the existing worksheets. Suggested column names Datameer creates suggested column names when the user creates new columns using the Formula Builder. These names are based off the column name used in the formula and the type of formula used. If you want to change these names refer to setting up column names. Setting column names Column names can only contain letters (only standard, capital or lower-case characters), numbers, and/or underscores. Column names must begin with a letter or underscore. Column names are case sensitive. Currently there isn't a parameter available that can disable the case-sensitive nature of these names. You can rename the column names on workbook sheets that are editable. The status line just above the list of tabs tells you if a particular sheet is writable. To edit the column name: Right-click the column name and click Rename. Enter the new name and press Enter. To undo the change, right-click the name again and click Undo. Resizing columns To resize columns: Place the mouse two column headings. The icon changes to a double-ended arrow. Drag the column marker to the desired width. Click to lock the width. You can also double-click a column name to fully expand the column. Hiding and expanding columns To hide (collapse) a column, right-click the column name and select Hide Column. Double-click the hidden column to bring it back into view.
A toggle columns toolbar button displays all columns of the current worksheet. You can use this menu to show or hide columns as a batch process.
Inserting columns To insert a new column: Right-click a column name. Select Insert Column. A column is added to the left of the column chosen. Each workbook can contain a maximum of 702 columns. Removing columns To remove a column: Right-click a column name. Select RemoveColumn. Cell context menu options Available as of Datameer v6.4
To access the cell context menu, right-click on a cell within a workbook. The cell context menu includes three options: Copy - Copy the cell value to the clipboard. Filter by - A shortcut to open the filter feature and auto completes the fields using the equals expression to filter for matchs the cell's value. Exclude - A shortcut to open the filter feature and auto completes the fileds using the does not equal expression to filter for matches that don't match the cell's value. Formatting columns To format the contents of a column: Right-click a column name. Select Format Cells. General Column text color Column background color Column text alignment (right, left, center) Numbers Date Thousands separator (comma to separate numbers by thousand) Select the amount of decimal places to display in the workbook Input a parse pattern to display the date. (The default doesn't display milliseconds)
Shading rows in a workbook The worksheet highlights rows by each group series when a GROUPBY function is used.
Copy cells Available as of Datameer version 6. Double-click a cell to select it, and then right-click it and select Copy Cell Content to copy its contents to the browser's clipboard. This functionality doesn't work with the Safari browser.
Workbook column inspector Available as of Datameer v6.4 The column inspector gives you important information about the data of each column in your workbook. Use this feature for quick insights into the data that each of your columns contain.
Information displayed in the column inspector: The name of the column being inspected. The number and percentage of valid and empty rows. A vertical bar chart displays all the different values and their frequency. Mousing over a bar in the chart displays the value(s) and count. If there are over 32 unique number type values, the bars group the values in bins. If there are over 32 unique non-number type values, the vertical bar chart is not displayed. The middle row of statisitcs displays the column's count of unique, minimum, mean, and maximum value. Mousing over a value displays the full value. Number and date field types are display all values. Othe field types display only the unique count. The bottom horizontal bar chart displays the top unique values for the column and their respective counts. Mousing over a bar in the chart displays the value and count. Click the See more link below the bar chart to view additional column values in decending order. To open or close the workbook column inspector: Click View in the menu bar and select/unselect Inspector. Click the Inspector icon in the tool bar.
Revert Actions Available as of Datameer version 6. If you would like to revert an action that you performed in a workbook, you can use the Undo and Redo buttons available in the toolbar. By default, you can undo or redo the last two actions performed on a workbook that modify the workbook state in the database. These actions don't include changes such as font style or column size, as the workbook's state remains the same. You can also access these options through the Edi t menu or using keyboard shortcuts (Cmd+Shift+Z for undo, Cmd+Shift+Z for redo). If the cursor is in a text field, using the keyboard shortcut reverts only the most recent changes in the text field, not the whole workbook. Administrators can adjust the settings for these buttons in default.properties and system.properties. There, admins can edit the amount of time the history of a workbook is kept, the maximum number of actions a a user can revert, and how many workbooks can retain a history at the same time. To make sure this functionality isn't causing performance issues, you can check various statistics about the memory usage and workbook states in <Datameer>/logs/workbook_undo.log. The log is set to record the following values separated by comma: absolutememoryconsumption,absolutememoryusageinbytes,generalmemoryusage,historiesinuse,historystatesinmemor y,maxmemory,relativeusageinpercent,usedmemory,usersworking,workbooksinuse All undo related actions are collected, but for memory performance, the system prints the values to the log once per minute. Using Formulas Formulas provide the ability to analyze your data in powerful ways. Creating formulas You can create formulas on workbook sheets that are editable. When you click the data area of a column that has a formula associated with it, the formula displays above the workbook. You can also create a new sheet and create formulas that reference fields from other sheets. Double-click a column to display the Formula Builder. To create a formula using the Formula Builder: Double-click the data area in a column and the Formula Builder opens. Or you can click the Fx button on the formula line. (As of Datame er v6.4, click the Fx button to access the formula builder.) Select the categories in the left column to choose the type of function. Choose a function from the list on the right. 4. Enter the argument or arguments shown for that function. You can click a sheet and select a column. 5. Click the Plus icon to enter additional arguments (if supported). 6. Click OK. The resulting function displays next to the fx icon. To learn more, see using the Formula Builder. To use operators in a formula: Click the data area in a column and the current formula if one exists displays above the workbook. Create or edit the formula using operators such as < or > or <= and press Enter. To use regular expressions in a formula: Click the data area in a column and the current formula if one exists displays above the workbook. Create or edit the formula using regular expressions press Enter. See Using Regular Expressions to see some examples. Datameer offers an API to extend the built-in formulas. See the Developer's Guide to learn more. Editing formulas To edit formulas: Click the data area in a column and the current formula displays above the workbook.
Edit the formula and click Enter. Completing Workbook Level Tasks You can save workbooks, view or change workbook settings, link a workbook to multiple data sources, calculate a workbook, and more. Saving workbooks When you save a workbook, you also specify the workbook settings. If you don't know which settings to use, you can keep the default settings and you can change them later or the system administrator can set them. See Configuring Workbook Settings to learn more. To save workbooks: 4. 5. From the File menu, select Save, or click the Save Workbook icon on the toolbar. Navigate to the folder where you want to save this workbook or choose to save the workbook in a new folder. Enter a name in the Save as field. Click Save. Fill in workbook settings: Select the appropriate trigger for when the workbook runs. Select data retention method for how long workbook runs is stored. Write a workbook description. 4. Select which sheets need to have results saved. Storage space is saved by only selecting useable data. 5. Specify how to deal with errors in workbook formulas. 6. Add email addresses that receive a notification if the workbook experiences an error or each time the workbook preforms a successful run. 7. Add custom Hadoop or logging properties. 8. Select if the workbook should first run immediately after saving. 6. Click Save when you are finished. As of Datameer v6.4 Steps 5 and 6 are no longer needed to save a workbook. If workbook settings need to be updated after saving, right-click on the workbook in the file browser and select Configure. To save a copy of a workbook: From the File menu, select Save As, or click the Save Workbook As icon on the toolbar. Navigate to the folder where you want to save the workbook, enter a name, and click Save As. Creating multiple workbooks You can create multiple workbooks that reference the same data set: 4. From the File menu, select New. Save your changes if needed. In the new workbook, from the File menu, select Add Data. Navigate to the data source you want to use and select the data source. Click Add Data. Calculating a workbook From the workbook you can calculate the workbook using the entire data set. Depending on the volume of data involved, this process might take awhile. From the Workbook menu, select Calculate or click the Calculate Full Workbook button on the toolbar. While the data is being calculated, you can click Abort if you want to calculate at a later time. Once the calculation is complete, the Full Data page opens. You can click Next button to view additional pages of data or click Go To Line and enter a line number to view a specific record. 4. Click Edit to return to the workbook. After making changes to an existing workbook, the next time the workbook is run, the changes are applied in the workbook calculation. Historical data before the change to the workbook isn't updated. Viewing full results in a workbook To view full results:
From the Workbook menu, select View Full Results, or click the View Full Results icon on the toolbar. Click Open to view the data set in the worksheet view or click Next at the bottom of the table to view more records. Going to line in a workbook To go to a specific record while in the workbook: From the Workbook menu, select Go to Line or click the Go to Line button on the toolbar. Enter a line number to view a specific record and click Go. Viewing workbook details You can view information about the workbooks that you have open. Workbook information gives general information about the current workbook, including: Name of the workbook Number of sheets the workbook contain The reprocessing schedule The last time the workbook was processed Notifications set for the workbook. Sheet summary information gives advanced information on individual pages within the current workbook including the following: Sheet type Sheet type Description Job path Connection Last executed Kept Partitioned Formulas us Source sheet x x x x x x Formula sheet x x x x Filtered sheet x x x Joined sheet x x x Sorted sheet x x Union sheet x x x To view workbook details: From the Workbook menu, select Workbook Info or click the Workbook Info button on the toolbar. The workbook information displays in a new window. Changing workbook settings You can set up the schedule of when a job runs when you create a workbook or you can change the schedule settings later. See Configuring Workbook Settings to learn more about scheduling. Integrating workbook results with other systems If you use other systems or BI tools, you can connect and consume results generated by Datameer workbooks so they can be leveraged for other processes or reporting mechanisms using the integration link. The default limit for number of rows to download is 100,000, as this functionality is intended for small aggregated data sets. To adjust the record download limit, change the rest.download-data.records-max=100000 property in the <datameer_install>/conf /default.properties file. Setting the value to 0 unlocks the limit, but increasing the number could result in slower processing for the Datameer conductor. Your local system might also have download restrictions. If you want to download more than 100,000 rows, we suggest adjusting the number based on environmental variables from Datameer, your infrastructure, and external consuming tools. Here is an example how it can be used in Power BI: Click Copy Integration Link in the Download dialog for full results page of a workbook. If you are using a browser that doesn't support copying to the clipboard, copy the provided URL instead.
4. 5. 6. Go to Power BI Desktop and in the Get Data workflow, find Other > Web. Paste the URL in dialog and click OK. Select Basic authentication and enter your Datameer username and password for the environment linked previously and click Connect. On the next screen review and confirm the data to be loaded. Data is loaded and you can take full advantage of Power BI capabilities. You can also publish your Powe rbi dashboards and reports to Power BI Cloud maintaining reference to this Datameer connection. Setting Workbook Sharing Permissions Security Only users with admin rights can set or change permissions. To view sharing permissions and security settings for a workbook: 4. 5. Click the File Browser tab at the top of the page. Select the Workbooks folder. Click to highlight the workbook for which you want to edit permissions. Right-click the workbook and select Information or click the Information button on the tool bar. The sharing and full results sharing permissions can be found by clicking the Lock icon in the info box. Sharing Owner: Groups: The owner box shows the current owner of the workbook. The owner has read, write, and execute privileges. Changing the owner can only be completed by someone with administrative privileges. User groups can be added to share the workbook. The administrator or owner of the workbook can set groups to read, write, and execute the workbook. By adding the read, write, or execute settings to Others, the admin or user can set permissions for all users. Full results sharing
Full results sharing give the admin or owner the ability to give selected groups the ability to read the full data from the workbook. Only users that have access to the workbook in from the sharing section are eligible to view the full data of a workbook. In order to create infographics from workbook data that you are not the owner from, users must have full results sharing permissions.