(Results Oriented Management) Knowledge Base Article
TableofContents ROM Overview... 3 Background... 3 Browsers... 4 Login... 4 My ROM Reports Home Screen Overview... 5 Header Overview... 5 Main Menu Links Overview... 6 My Settings... 7 My Reports (Link)... 8 All Reports (Link)... 8 Overview of a Report... 10 Report Definitions... 11 Toolboxes... 12 Data Tables... 14 Drilldown Numbers... 16 ROM Report Views... 17 Primary Views... 17 Additional View Types... 18 Report Filters... 18 Understanding Measures... 20 Tips for Understanding a Report Measure... 20 ROM Core Measures Federal... 21 Composite Score Estimates... 22 ROM Core Measures Other Management Measures... 23 ROM Help... 24 Assigning Cases for Reporting... 26 Ohio Management Hierarchy... 26 Rules for Assigning Case Outcomes... 26 Page 2 of 26 Last Updated: 01/06/12
ROM Overview Navigating ROM This Knowledge Base Article provides an overview of the Ohio ROM (Results Oriented Managements) Reports system and discusses the following topics: Navigating to the ROM reports Accessing report information Understanding reporting measures, such as Federal Outcomes Using the report labels, legends, and help screens Background ROM is a web application being administered by ODJFS. The ROM data is pulled from the Ohio SACWIS system and updated nightly to the reporting system. ROM Reports resides on a secure server managed by the State of Ohio ODJFS. The ROM history data dates back to children who were in care anytime after April 1, 2008. Page 3 of 26 Last Updated: 01/06/12
Browsers Navigating ROM ROM works in the following internet browsers: Internet Explorer (IE-6 has display problems see suggestions below) Firefox Google Chrome The ROM reports appear correctly in Microsoft Internet Explorer (IE) version 7 or higher. However, if your computer has IE-6, it is strongly suggested that you click one of the following links to obtain a free Browser download. Then, follow the directions for installing it and set a bookmark for easy access to view the ROM reports correctly. Firefox: http://www.mozilla.com/en-us/firefox/fx/ Google Chrome: http://www.google.com/chrome Login ROM URL: https://rom/jfsreports/login.aspx To log in (as shown in red below): 1. In the User Name field, enter your user name (same as your Novell ID). 2. On the first use, enter your temporary password. 3. Upon system prompt, enter a permanent password that meets the following strong password criteria: Must be at least 8 characters long Must contain one lower case letter Cannot be user-id, last name, or first name Must contain one number Must contain one upper case letter Must contain one of the following symbols: ~! @ # $ % ^ & * () + = 4. If you forgot your password, enter your user name in the User Name field. 5. Then, click the Forgot Your Password link. ROM asks you to enter an email address so a temporary password can be sent. When you receive the temporary password, use it to re-set your password. Once logged in, the My ROM Reports Home for <Your Name> screen appears. Page 4 of 26 Last Updated: 01/06/12
My ROM Reports Home Screen Overview After logging in, the following information displays on the screen: Welcome with your name (shown in red below). News and Notices (shown in gold). This section presents system information from the state administrator regarding: Changes in Reports New Reports or Features Agency Announcements, such as: Messages from the Director Reaching Benchmarks or Goals Training Information Your Name Your Name Your Name Header Overview In the report header (blue banner at the top), the following information appears: State logo, report system name, and software version number User name Logout button (to end your session) Data current through: (date the data were pulled) Main menu links that are discussed in more detail below Your Name Page 5 of 26 Last Updated: 01/06/12
Main Menu Links Overview My Reports list (mouse over to expand). See more information about this below. Resources list (mouse over to expand) Contact Us list (mouse over to expand) Help list (mouse over to expand). Refer to the ROM Help sub-section later in this Knowledge Base Article for additional information. Page 6 of 26 Last Updated: 01/06/12
My Settings 1. To change your settings, click My Setting in the header. The following screen appears. 2. Adjust the following ROM settings, as needed, using the fields on this screen: Change your password To customize your reports by setting defaults for the: Unit Level View and time period Drilldown option (amount of data you will receive) Page 7 of 26 Last Updated: 01/06/12
My Reports (Link) Navigating ROM The My Reports screen displays the ROM report menu. 1. To view the ROM report menu, click a My Reports link. Important: My Reports can be accessed by clicking any of the three links (shown below). The first time the My Reports screen appears, ROM displays all of the reports available. However, My Reports can be customized to show only the reports that you want to view. All Reports (Link) 1. To customize the report menu on your My Reports screen or to view all of the reports, click the All Reports link. Important: All Reports can be accessed by clicking either of the links shown here. The following screen appears displaying all of the report names as shown below. Page 8 of 26 Last Updated: 01/06/12
2. Select or deselect the checkbox in front of each report as needed to customize the menu (shown in gold below). 3. When complete, click the Save Selected Reports link (top of the screen). The My Reports screen appears displaying only the selected reports. Additional Screen Information: Both the My Reports and All Reports menus are organized the same way. As shown in gold, the reports are grouped into three columns: Management Reports, Federal Outcome Indicators, and Summary Reports As shown in red, the report measures are grouped into various categories. As shown in green, each report measure is a link that can be clicked to view an explanation of the measure (discussed more below). Page 9 of 26 Last Updated: 01/06/12
Overview of a Report 1. To receive a specific measure explanation, click the appropriate link on the All Reports screen. As shown in this example, the measure explanation appears. Each section is discussed in more detail below. The left toolbar is the ROM Control Panel comprised of Toolboxes Views Report Header Graph Expand Toolbox Data Table Legend Page 10 of 26 Last Updated: 01/06/12
Report Definitions Views: Provides different views of the data, such as trends over time, unit comparisons, etc. Views are also discussed in the next steps. Report Header: Describes the report by showing the name of the report, a brief definition of the report measure, and the time period used for the report. Graph: Displays the met outcome or other descriptive data from the table. The columns within the chart show both Goals and Outcomes, as applicable. Data Table: Presents the data (numbers and percents) and enables drilldown to the case level. See the Data Tables sub-section below for more details. Legend: Defines the labels used in the table. Control Panel: Displays toolboxes to adjust the report parameters (e.g., units, time periods, etc.) Toolboxes: Sets parameters for the reports. Toolboxes can be collapsed or expanded. See Toolboxes sub-section below for more details. 2. Click the appropriate tab (view) to change the way data displays on the screen. The data display appears according to the tab selected. Important: For additional information on views, refer to the ROM Report Views section later in this Knowledge Base Article. 3. If needed, click the Show Full Screen link to view the data on the full screen. Page 11 of 26 Last Updated: 01/06/12
4. To return, click the Show Toolbar link to return to the original view. Toolboxes Toolboxes appear along the left side of the Measure Explanation screen. The selections made inside a toolbox remains selected until they are changed. Each toolbox can be collapsed or expanded by clicking the (^) on the right side of the toolbox header. Reports Toolbox Displays the name of the report being viewed at the top. Generate Direct Link for this Page: Provides a link to the specific report with all the parameters you selected. Print Report(s): Generates a PDF file that allows you to print the report you defined. Related Reports: Provides a drop-down list of similar reports to the one you have selected. Last 3 Reports: Provides a drop-down list of the last three reports you viewed. Page 12 of 26 Last Updated: 01/06/12
Unit Level Toolbox Displays the management hierarchy in the drop-down lists. Select the County or sub-unit you want from the drop-down lists. When used, the units must be set in order from the top-level down. Time Period Toolbox Displays the report s start date and end date. To change the report s time period, complete the following steps: Select the year and month in one (or both) Start Date and/or End Date fields Click the Apply Time Period Changes link. Data Display Toolbox Controls how data displays in the report. The options are: Report Period By: Monthly, Quarterly, etc. Display Type in Graph: Percentages or Count Display Type in Table: Grouped or Ungrouped Select a Drilldown Display Option: Specifies the amount of data retrieved in a drill down from Case (the least) to Full (the most). Filters Toolbox Displays a sub-set of children defined by their characteristics or other identifiable factors (e.g. judicial district, geographic area) Filter settings include: Filters Applied: Displays the filters you applied Manage Filters: Displays a new screen where to select and define your filters Clear Filters: Clears all filter settings Quick Filters (with drop-down boxes) Select parameters for up to five quick filters (set by the ROM Site Administrator) Note: Additional information is provided in the Filter section below. Page 13 of 26 Last Updated: 01/06/12
Other Toolbox Tips On each toolbox (right side of the header): Click the? to get an explanation about the toolbar. Click the up arrow ^ to collapse the toolbox. To get additional help, on the main menu: Click the Help link. Click the About Page link. Data Tables This example shows an outcome report table displaying the following elements discussed below: Page 14 of 26 Last Updated: 01/06/12
Chart Rows Below are some common categories for outcome reports. Depending on the type, other management reports have their own unique categories. Met: Outcome was met (Example: Reunified in less than 12 months). Not Met: Outcome was not met (Example: Reunified in 12 months or more). Total: Number of children in the denominator of measure ( Met plus Not Met ). Cohort Label: Dates used to select the children for the report. Grouped: If the row has a + sign, the data can be grouped into smaller (finer) categories. Click the + sign to expand the details and the sign to reduce the details. Chart Columns Trends: Report periods Unit: Management units Crosstab: Variable for analysis Contents Numbers and Percents Blue Numbers (discussed more below) Row Labels: Sorts data ascending or descending (grouped data will not sort) Click for Drilldown: Allows for drilldown to view data for that group of children Page 15 of 26 Last Updated: 01/06/12
Drilldown Numbers Navigating ROM When you drilldown (or click) on a blue number, ROM generates a spreadsheet for the selected case and provides detailed data for each child. 1. In any data table, click the appropriate blue number. As shown below, ROM generates a spreadsheet. If the number of records is above 2,000, the system automatically imports the drilldown information into an Excel spreadsheet for download. 2. To sort a column, click the column header once for ascending. 3. To sort in descending order, click the column header again. 4. To view the data dictionary, click the Data Dictionary link in the Drilldown toolbox. 5. To save the spreadsheet, click the Save Spreadsheet link in the Drilldown toolbox. 6. To return to the previous screen, click the Show Full Screen link or the Back button on your Browser. Important: From privacy perspective, only save the spreadsheet information on password protected or encrypted drives. Page 16 of 26 Last Updated: 01/06/12
As shown above, drilldown spreadsheet elements include: Report Header Report Title (All denominators are in parenthesis) Report Sub-Title Report Time Period Data Table (Row and column labels in table) Not shown: Cohort (Event and time parameter in last row of the table) Legend (All rows in the table defined) Note: If a report is saved to an Excel spreadsheet, the report parameters that you set will also be included in the other workbook tabs of the spreadsheet. ROM Report Views Primary Views There are three primary types of ROM data views: Trend: Displays the changes in outcomes over a period of time Unit: Compares management units for a particular time or reporting period Crosstab: Analyzes outcomes by available variables, such as client characteristics, geographic areas, case information (e.g. time in care, case plan goal, TPR) Page 17 of 26 Last Updated: 01/06/12
Additional View Types Navigating ROM The primary views are offered as By Report Period, Annualized or Cumulative. Other available views vary by Report and Measure type. By Report Period Trend: Per reporting period (each a separate reporting period) Unit: Over a time period (for the whole time period that is specified) Crosstab: Over a time period (same as Unit) Annualized: Each data point represents a rolling year based on the end of a reporting period Trend: Annualized (each reporting period is a rolling year) Unit: Annualized (the year of the last full month in the time period) Crosstab: Annualized (same as Unit) Cumulative: Each data point is cumulative for a fiscal year Trend: Cumulative (shows cumulative results by reporting period) Unit: Cumulative (shows the most recent month in the time period) Crosstab: Cumulative (same as Unit) Report Filters Filters allow you to select groups of cases where children share certain characteristics, such as a specific age, judicial district, etc. For example, you could set a filter to select children who are less than 15 years of age. Once set, the ROM report will only include that age group. There are two ways to set a filter. You can select parameters for up to five quick filters in the Filter toolbox drop-down lists (as set by the ROM Site Administrator). Or, select the Manage Filter link which provides a more extensive filter list. Page 18 of 26 Last Updated: 01/06/12
1. To apply a filter to your reports, click the Manage Filter link in the Filters toolbox. Important: You can also apply and clear filters using the links in this toolbox. The Select Filter screen appears. 2. In the Available Filters field, select a filter name. 3. Click the Add Selected Filters link. The filter name moves to the Selected Filters field. Add (or remove) selected filters as needed by using the other functions (links) listed between the Available Filters field and the Selected Filters field. As shown in gold, when you move a value into the Selected Filters field, additional filter fields may appear in the Select Filter Values section. 4. In the Select Filter Values section, set the value for each filter, as needed. Example: Click the Add All Filters link to look at the different ways that values can be set. 5. When complete, click the Save Filters and Return link. Page 19 of 26 Last Updated: 01/06/12
Understanding Measures Navigating ROM Understanding measures and how they are used is often the most difficult part of using ROM. Some Federal measures are very complex and can be constructed differently than other measures. To understand what the ROM data is saying, you need to understand how the measure is constructed. Additionally, management must consider how factors (such as policies, practices, and people) are impacting outcomes in order to better focus improvement efforts. ROM provides information on the screen to help you understand how each measure is constructed. This section will review the major aspects of measure construction. Tips for Understanding a Report Measure 1. Locate the population (denominator). This is labeled Total in the data table. 2. Locate the dates that were used to define the population (denominator). 3. Locate the report legend to obtain more detailed information about the measure. Page 20 of 26 Last Updated: 01/06/12
ROM Core Measures Federal There are 19 Federal measures, including: 2 safety measures 15 outcome indicators 2 visitation indicators Except for Median measures, the Federal measures have been positively oriented. In other words, the higher the percentage, the better the performance. Compared with other federal reports, the following two measures were also changed to a positive orientation. Re-entry The federal measure takes into account the percent of children reunified who then re-entered foster care. The ROM measure takes into account the same children, but looks at the percent that do not re-enter foster care (maintaining reunification). Aging Out (Emancipated) The federal measures takes into account the percent of children who were discharged to emancipation in three years or longer (negatively oriented) ROM looks at the percentage of children who were discharged to emancipation and in foster care fewer than three years (positively oriented) The Federal measures reported in ROM are computed the same way as in the Child and Family Services Review (CFSR) (Data Profile). However, the difference is in the completeness of the children included in the base dataset. These differences are as follows: ROM uses a longitudinal data structure that captures all removal episodes (every time a child enters and exits foster care) CFSR uses AFCARS which is a point in time dataset that looks primarily at the last removal episodes in each file (missed episodes of care). AFCARS is Federallydefined foster care data that is sent to the Federal government. Page 21 of 26 Last Updated: 01/06/12
Composite Score Estimates Navigating ROM The Federal government has developed a statistical method of scoring performance on groups of outcome measures. They use these scores to determine whether states are achieving the Federal standards set for each composite. ROM approximates the federal method for computing composite scores. These are the four composite score estimates and the measures that comprise the grouping in ROM: Important: The following are estimates that may be invalid for smaller sub-state units (counties) and certainly are not valid at the supervisor or worker level. Composite 1: Timeliness and Permanency of Reunification Reunification in 12 months (of those reunified) Median months to reunification (of those reunified) Reunification in 12 months of Entry (of 1st time removals 12 months ago) Maintain reunifications for 12 months (of those reunified 12 months ago) Composite 2: Timeliness of Adoptions Adopted in less than 24 months (of those adopted) Median months to adoption (of those adopted) Adopted within last 12 months (of those in care 17+ mos. as of 12 mos. ago) Legally freed for adoption (of those in care 17+ months not TPR 6 months ago) Adopted in less than 12 months of TPR (of those TPR 12 months ago) Composite 3: Achieving Permanency for Children in Foster Care for Long Periods of Time Permanency achieved Prior to Turning 18 (of those in care 24 months as of 12 months ago) Permanency achieved (of those free for adoption and discharged) In care less than 3 years (of those emancipated or turning 18) Composite 4: Placement Stability Placement Stability: 2 or fewer placements (of those in care under 12 months) Placement Stability: 2 or fewer placement (of those in care 12-23 months) Placement Stability: 2 or fewer placements (of those in care 24+ months) Page 22 of 26 Last Updated: 01/06/12
ROM Core Measures Other Management Measures There are currently 18 additional management reports available in ROM. In the Reports menu (on the left), click on each report to review the captured data. The reports are listed below. The denominator (children included) is in parentheses for each report measure. Caseload Caseload Counts (foster care) on first day of month Out-of-home or trial home visit Entries and exits (on chart y-axis to the right) Caseload Counts (in care 17+ months) Without TPR (termination of parental rights), with TPR or in adoptive placement Entering and exiting status Entered 17 th month (during period) Discharged to permanency (during period) Discharged no permanency (during period) Descriptive point in time (last day of the month) Level of Care (of those in care) Length of Stay (for those in care) CPS Indicators (Child Protective Services) Report Conclusions (of conclusions made) Investigations Completed within 30 days (of reports received 30 days ago) Initial Face-to-Face Contact within 24 hours (of accepted reports) Pending Investigations (of accepted not completed reports) Caseworker Child Contact Caseworker visits every full month (in care 1+ month In Federal FY [Fiscal Year]) Visit Months with in-home (for visit months of those visited every month) Monthly Worker-Child Visitation (of those in care the entire month) in ROM but revisions pending Worker-Child Visitation Pending/Completed (those in care start of current month) Page 23 of 26 Last Updated: 01/06/12
Alternative outcome measures Permanency in 12 months (of those entered care 12 months ago) Permanency in 24 months (of those entered care 24 months ago) No re-entry (of those discharged 12 months ago) Countdown Reports Countdown to Permanency (of those entered care in last 24 months) Countdown to TPR (of those starting 17th month in last 24 months) Countdown to Adoption/other Permanency (those given TPR in last 24 months) Summary Reports There are four pre-defined Summary reports that have the set of federal outcome measures included in each Composite. In the Reports menu (on the right column under the CFSR Composite Summary), click each Summary report and use the toolbar to change the parameters (e.g. Unit Level, Time Period). ROM Help 1. To obtain help, click the About Page link on the main menu. A help screen for that screen appears. Page 24 of 26 Last Updated: 01/06/12
2. Click the appropriate Contents menu link to expand the link and search for the topic. 3. To obtain report definition help, click the Report Definition link. A report definition help screen appears. 4. Click a Definitions link to expand the content and search for the desired topic. Page 25 of 26 Last Updated: 01/06/12
Assigning Cases for Reporting Ohio Management Hierarchy The Unit Level toolbox enables you to generate reports based on cases that are assigned to the unit or caseload that you manage. To better understand that data, the following are rules for how cases are associated with each staff person. The Unit mutually-exclusive hierarchy levels are as follows in ROM: Case is assigned a Worker who reports to a Supervisor who is assigned to a Manager that is within an Agency which is part of a County which is part of the State Rules for Assigning Case Outcomes The following rules apply when assigning case outcomes in ROM: Open Cases Assignments are made as of the last day in the report period (month), examples are: Placement Stability 2 or fewer placement settings Level of care Discharged Cases Assignment is made at the time of federal discharge, examples are: Reunification in 12 months Assigned to hierarchy in place at the time of reunification End of observation period Assignment made at the end of observation period for entry cohort measures Adopted in less than 12 months of TPR (of those TRP 12 months ago) Reunified in 12 months (of 1st time removals) Other Recurrence of maltreatment assigned as of date investigation completed Page 26 of 26 Last Updated: 01/06/12