Benchmarking Software Tools Assessment

Transcription

1 Benchmarking Software Tools Assessment ET Project Number: Project Manager: Jaclyn Hood Pacific Gas and Electric Company Prepared By: Abhijeet Pande, Ryan Schmidt Heschong Mahone Group Gold Country Blvd, Ste 103 Gold River, CA Issued: December 31, 2012 Copyright, 2013, Pacific Gas and Electric Company. All rights reserved.

2 ACKNOWLEDGEMENTS Pacific Gas and Electric Company s Emerging Technologies Program is responsible for this project. It was developed as part of Pacific Gas and Electric Company s Emerging Technology program under internal project number. Heschong Mahone Group conducted this technology evaluation for Pacific Gas and Electric Company with overall guidance and management from Jaclyn Hood, PG&E. For more information on this project, contact JKHn@pge.com. LEGAL NOTICE This report was prepared for Pacific Gas and Electric Company for use by its employees and agents. Neither Pacific Gas and Electric Company nor any of its employees and agents: (1) makes any written or oral warranty, expressed or implied, including, but not limited to those concerning merchantability or fitness for a particular purpose; (2) assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, process, method, or policy contained herein; or (3) represents that its use would not infringe any privately owned rights, including, but not limited to, patents, trademarks, or copyrights. i

3 ABBREVIATIONS AND ACRONYMS API 1 Benchmarking 2 BEARS BTU Building Energy Use Benchmarking System CBEURT DOE Energy Baseline 2 EPA EUI Portfolio Manager HDD, CDD Application Program Interface. A set of routines, protocols, and tools for building software applications. API s make it easier for program developers by providing the program building blocks The process of accounting for and comparing a metered building s current energy performance with its energy baseline, or comparing a metered building s energy performance with the energy performance of similar types of buildings (based on use, such as comparing the energy performance of a hospital to that of other hospitals). Benchmarking can be used to compare performance over time, within and between peer groups, or to document top performers Building Energy Asset Rating System British Thermal Unit, a unit of energy into which energy use data from various fuel types (electricity, natural gas, etc.) is converted to enable calculation of wholebuilding energy in a common unit A tool or system of tools that enables the energy performance of a metered building to be benchmarked. See definition of benchmarking above California Building Energy Use Rating Tool U.S. Department of Energy An initial period of metered energy consumption used as a point of reference for comparison purposes. For example, the Portfolio Manager tool uses a 12-month period of metered building energy consumption as the energy baseline. U.S. Environmental Protection Agency Energy Use Intensity, a unit of measurement that describes a building s energy use per unit of floor area, which enables comparison of energy performance across buildings of various sizes. EUI values are typically presented in kbtu/sf or kwh/sf. EPA s ENERGY STAR Portfolio Manager Heating and Cooling Degree Days, respectively. Metric frequently used to account for relative heating and cooling conditioning demand differences between various locations. Calculated in a variety of ways, but based on a reference temperature (typically 65 degrees Fahrenheit) and hourly or daily average temperatures, summed up over the course of a year Adopted from the Building Energy Use Benchmarking Guidance provided as part of Energy Independence and Security Act of 2007 Section 432- Benchmarking of Federal Facilities i

4 IOU LBNL Metered Building 2 ORNL Rating 2 Site Energy Use Intensity Source Energy Use Intensity Investor Owned Utility Lawrence Berkley National Laboratory A building with one or more meters (advanced or standard) installed to measure energy consumed within that building. Metered energy includes electricity, natural gas, and steam. Other utilities may be metered as an energy or water management best practice. Oakridge National Laboratory The relative indicator of performance obtained from a benchmarking tool. The rating allows the energy performance of the metered building or facility to be compared over time with itself and with the energy performances of similar types of buildings and facilities (based on use, such as comparing the energy performance of a hospital to that of other hospitals). A building s actual (metered) energy use intensity during a given period (see EUI above). Site energy is the most common convention used for discussing building energy consumption. Portfolio Manager calculates Site EUI as the total of all energy used on site for the chosen year divided by the square footage of the facility. Data used to calculate this measure include energy-use data and square footage. Represents the total amount of raw fuel required to generate the energy to operate a building. Source EUI incorporates all transmission, delivery, and production losses from all energy types consumed on-site, thereby enabling a complete assessment of a building s energy impacts. Portfolio Manager's national energy performance ratings evaluate the performance of buildings that use all types of energy. To compare this diverse set of commercial buildings equitably, the ratings must express the consumption of each type of energy in a single common unit. EPA has determined that source energy is the most equitable unit of evaluation. ii

5 FIGURES Figure 1: CBEURT Rating Graphic Figure 2: Building Sample Summary Characteristics Figure 3: Screenshot of Portfolio Manager main interface after logging in Figure 4: Screenshot of Portfolio Manager steps for entering space and meter data Figure 5: WegoWise screenshot of Study Building 11 s Energy Use Profile Figure 6: WegoWise screenshot of a sample report for all buildings in a portfolio Figure 7: CBEURT spreadsheet screenshot showing the Energy Performance Rating Figure 8: CBEURT screenshot showing input requirements for space use Figure 9: EnergyIQ - Peer Group Filters (screenshot) Figure 10: EnergyIQ Project Peer Group Filters Summary Figure 11: Energy IQ Peer Group Information and EUI Distribution (screenshot) Figure 12: Energy IQ Dashboard Tracking Figure 13: Benchmarking Results Summary Comparison Figure 14: Software Tools Rating Metrics Summary Figure 15: Benchmarking Results Portfolio Manager and EnergyIQ Comparison iii

6 CONTENTS EXECUTIVE SUMMARY 1 INTRODUCTION 3 What is Benchmarking?... 3 Assessment Objectives and Scope... 3 Assessment Limitations... 4 BACKGROUND 6 Regulatory Context for Commercial Building Benchmarking in California 6 California Benchmarking regulations: AB Benchmarking Through IOU Programs... 6 EMERGING TOOLS/ TECHNOLOGY EVALUATION 8 ENERGY STAR Portfolio Manager... 8 Description... 8 Characteristics... 9 Inputs and Results Limitations Melon / WegoWise Description Characteristics Inputs and Results Limitations California Building Energy Use Rating Tool (CBEURT) Description Characteristics Inputs and Results Limitations EnergyIQ Description Characteristics Inputs and Results Limitations BENCHMARKING TOOL STATUS REVIEW THROUGH A FORUM 18 Participants Tool Presentations Facilitated Discussion Future Direction for Tool Improvements COMPARITIVE ANALYSIS OF ASSESSED TOOLS: TECHNICAL APPROACH/ TEST METHODOLOGY21 iv

7 Analysis/Test Plan Rating Process ENERGY STAR Portfolio Manager Melon/WegoWise CBEURT EnergyIQ Results ASSESSMENT RESULTS 34 Databases, Metrics, and Methodology Complements and Synergies Benchmarking Software Tools in the Context of Benchmarking Commercial Buildings in California APPENDIX A: TOOL DESCRIPTION TABLES 37 APPENDIX B: BENCHMARKING SOFTWARE TOOLS FORUM 38 Forum Presentations Forum Discussion Notes v

8 EXECUTIVE SUMMARY Pacific Gas and Electric Company (PG&E) contracted with the Heschong Mahone Group (HMG) to provide an evaluation of selected benchmarking software tools that could integrate with existing California regulations and utility programs and support goals identified in the California Long Term Energy Efficiency Strategic Plan. The four tools evaluated include ENERGY STAR Portfolio Manager (Portfolio Manager), Melon/WegoWise, the California Building Energy Use Rating Tool (CBEURT), and EnergyIQ. Portfolio Manager is a free, web-based tool maintained by the U.S. Environmental Protection Agency (EPA). The EPA claims that Portfolio Manager has benchmarked nearly 40% 3 of the commercial building market nationwide. Portfolio Manager is the oldest and most established benchmarking software tool in the current market. It leverages a national building energy use database that is updated every four years, and offers a nationally recognized, technically robust rating system anchored by its ENERGY STAR score. Portfolio Manager facilitates automated data uploads directly from the customer s utility account to Portfolio Manager. A potential impediment to greater adoption is its user-interface, deemed by some to lack user-friendliness, and a limited number of building types and sizes that can be benchmarked. The EPA is preparing for a major Portfolio Manager overhaul to launch in 2013, which could enhance usability and reach of this established tool. Melon is a private company attempting to develop a user-friendly interface to Portfolio Manager while relying on Portfolio Manager to provide analytics and the ENERGY STAR score. Within the past two weeks Melon has merged with WegoWise, a company with an existing benchmarking tool supporting multi-family buildings. Melon s developers plan that their web-based tool will connect to Portfolio Manager at the back-end to generate ENERGY STAR scores and reports while the front end interface will provide additional functionalities such as suggestions for operational improvement and peak demand management. While still conceptual, there is potential for synergistic ties with utility energy efficiency programs. The tool is in development and not yet functional, so full evaluation of its features, functionality, and methodology, as well as assessment of its promised added-value to Portfolio Manager was not feasible for this study. Unlike other tools assessed in this project, it will be a fee-based tool with costs paid by users and/or utilities or other organizations. The California Building Energy Use Rating Tool (CBEURT) is being developed by the CEC in cooperation with the Oakridge National Laboratory (ONRL), and is projected to be a webbased operational rating tool. CBEURT complements the Portfolio Manager generated reports and can be used for voluntary disclosure in compliance with Assembly Bill (AB) It requires similar inputs to Portfolio Manager, but its ratings are based on a California-specific building peer group developed using the California Commercial End Use Survey (CEUS) database. 4 In distinct contrast to other software tools, CBEURT rates buildings on the Zero Energy Performance Index (ZEPI), based on an absolute energy intensity scale. A CBEURT score of 0 indicates zero net energy (ZNE), while a score of 100 represents the typical California building of similar type More information about the California CEUS is available at: 1

9 EnergyIQ was developed by the U.S. Department of Energy's (DOE) Lawrence Berkeley National Laboratory and sponsored by the CEC s Public Interest Energy Research (PIER) program. The tool focuses on flexibility and added-value to existing users of Portfolio Manager. It offers the capability of benchmarking against both national as well as California-only building databases. It is leveraging its integrated databases to develop a user recommendation component to provide a bridge between increased building energy use knowledge to galvanizing energy efficiency actions. The EnergyIQ platform also encourages third party applications through its API, (application program interface), which can be tailored to suit specific building industry and third party applications. HMG produced benchmark rating results using thorough testing of each tool with actual but anonymous building characteristic and energy use data provided by PG&E. As the tool rating metrics vary, direct comparisons between their results are difficult. In addition, functional challenges and limitations were encountered across the tools, but ultimately the results for each were generally consistent. Though on different rating scales, each tool ranked the benchmarked buildings relative to each other consistently as having good, bad, or average performance. Differences noted lay primarily in the degree of rating stringency, potentially attributable, at least in part, to the differing databases and filters used to define the peer group benchmark. Corresponding emissions outputs also varied, significantly so with EnergyIQ. It is apparent that a single tool is unlikely to serve all of California s benchmarking needs, but rather that there will be market space for tools spanning a range of complexity and purpose. A common foundation could be instrumental to tying various efforts together and enabling comparisons across diverse tool methodologies and the buildings and users they serve. Organically developed over more than a decade, Portfolio Manager appears to be just such a foundation. In fact, all the tools assessed here build on and extend from Portfolio Manager in some manner. They act as complements to Portfolio Manager s groundbreaking efforts that have established it as benchmarking s benchmark. The practice of benchmarking building energy remains relatively new to California and will require coordinated support from regulators, utilities, and private as well as public sector developers to continue to gain public acceptance. We anticipate that a strong public/private partnership in the benchmarking industry and its software tools will open numerous possibilities in the near future, and address many of the challenges being faced by users, utilities and regulators. 2

10 INTRODUCTION WHAT IS BENCHMARKING? The U.S. Department of Energy (DOE) defines benchmarking as the process of accounting for and comparing a metered building s current energy performance with its energy baseline, or comparing a metered building s energy performance with the energy performance of similar types of buildings (such as a large office to other large offices). Benchmarking can be used to compare performance over time, within and between peer groups, or to document top performers and opportunities. Existing building owners may use rating methods to determine if investment in energy efficiency retrofits is justified. Developers use benchmarking ratings to showcase high performance buildings and attract potential owners/ tenants who are interested in minimizing facility operating costs. 5 The term benchmarking differs subtly from rating systems, although the two go hand-in-hand and are often used synonymously. Rating systems are primarily classified as a. Operational rating systems, and/or b. Asset rating systems The former is based on a building s actual energy consumption. The tools developed for operational rating systems are also referred to as consumption-based tools. Asset-rating systems, also called design-oriented tools, measure a building s technical energy efficiency potential based on building and system characteristics (such as equipment type and age) input into specific software programs to simulate predicted energy use. Based on the level of software sophistication, knowledge of occupant behavior, equipment maintenance, and other operational characteristics they estimate, asset-rating systems may predict building energy use that varies significantly from actual energy use. For the purposes of this study, only operational rating systems were studied. Thus, for this report, unless otherwise noted the term benchmarking refers only to operational rating systems. Formal definitions of terms are included under Abbreviations and Acronyms section of this report. ASSESSMENT OBJECTIVES AND SCOPE As part of the Energy Efficiency Portfolio Decision, the California investorowned utilities (IOUs) were directed by the California Public Utilities Commission (CPUC) to assess benchmarking tools for use in IOU programmatic efforts. Pacific Gas and Electric (PG&E), on behalf of the California IOUs, contracted HMG to provide an assessment of four current commercial benchmarking software tools that have potential to be integrated with ongoing state and utility programs, as well as adapt to current California regulations

11 In the IOU program filings, building energy audits come under the energy advisement banner and benchmarking is foundational to leading customer education as well as automating efficiency awareness as part of these efforts. The CPUC Decision [(D.) ] required the IOUs to include assessment of EPA s ENERGY STAR Portfolio Manager (Portfolio Manager) tool and the CEC s Building Energy Asset Rating System (BEARS) along with other tools as necessary. However, the BEARS tool, which will provide an asset rating (unlike Portfolio Manager) was not available for assessment at the time of this project. The IOUs instead will propose to pilot the BEARS tool in , as directed in D (p.219). In place of BEARS, this project assessed the California Building Energy Use Rating Tool (CBEURT). CBEURT is more akin to Portfolio Manager than BEARS in that it is also an operational rating tool, based on metered energy use rather than building characteristics data. Along with Portfolio Manager and CBEURT, HMG was also asked to identify and evaluate additional tools that could be viable options for commercial building benchmarking in California. This study evaluates a total of four benchmarking software tools: Portfolio Manager, Melon/WegoWise, CBEURT, and EnergyIQ. HMG quantified the accuracy, effectiveness, and viability of these tools so as to provide a basis for future use of benchmarking tools in IOU programs. A specific focus of this study was to evaluate the tools for benchmarking small and medium businesses, as well as building types beyond those currently covered by Portfolio Manager. HMG conducted a comprehensive review of existing literature, data, and standards for each tool to understand their viability and usability. We also collected information about each tool s development, current capabilities, and plans for future updates. As part of this review, we identified the advantages and limitations of using each tool, and reviewed the scope of each tool based on the number of building types that it can benchmark. HMG convened a forum to identify the strengths and weaknesses of each tool from the perspective of the tool s developers as well as IOU staff with experience and/or interest in supporting benchmarking efforts. This forum is summarized in section titled Benchmarking Tool Status Review Through a Forum. Finally, HMG performed an evaluation and comparison of each tool s functionality and final outputs and rating metrics. Anonymous building energy use data for seven buildings with two different space types - Office and Retail- was obtained from PG&E. The resulting outputs were compared across the four tools and evaluated for ease of generation, content quality, and overall effectiveness to motivate and benefit energy efficiency practices for its user. ASSESSMENT LIMITATIONS The study did not look into many other comparable tools such as ASHRAE s Building IQ, BEARS, Retroficiency, Pulse Energy, The Weidt Group s B3 Benchmarking and many other upcoming private sector ventures. Some of these tools were excluded from the study due to their lack of readiness for evaluation, others due to data privacy and confidentiality issues, while some were excluded because of time and budget limitations of this project. It was also decided to exclude asset-based rating tools and limit the focus to operational systems, given the context for which these tools are being evaluated. With more time and resources a comparison across the 4

12 two rating approaches could provide beneficial insights into how they complement each other, as well as which tools could offer a comprehensive platform bringing together the best of both worlds. 5

13 BACKGROUND REGULATORY CONTEXT FOR COMMERCIAL BUILDING BENCHMARKING IN CALIFORNIA The California Public Utilities Commission (CPUC), along with the Investor Owned Utilities (IOUs) and in collaboration with the California Energy Commission (CEC) have been advocating for benchmarking of commercial buildings in California for many years, keeping in line with the oft-quoted principle you can only manage what you measure. CALIFORNIA BENCHMARKING REGULATIONS: AB1103 In November 2008, the Governor signed into law Assembly Bill 1103 (AB1103). This law requires the owner of a nonresidential building to disclose his or her building s most recent energy use data during real estate transactions with buyers, lenders, or lessees. Subsequently, the California Energy Commission developed regulations for carrying out the mandates of AB Building owners are required to create an ENERGY STAR Portfolio Manager (Portfolio Manager) account and provide the necessary reports enabled therein, requiring at least the most recent 12 months of energy use data for the entire building. The regulation requires utilities (both IOUs as well as publicly owned utilities) to provide this upon owner s request, and to upload it directly to Portfolio Manager. AB1103 implementation will be phased in based on building size. The first phase, for buildings with total gross floor area greater than 50,000 square feet, goes into effect July 1, BENCHMARKING THROUGH IOU PROGRAMS Other than for compliance with AB1103, benchmarking can also play a substantial role as a feed-in for utility programs, as well as support targeting low performing buildings for program participation. It can be used to track improvements and reinforce behaviors that reduce a building s energy use. As part of the portfolios of programs, IOUs have conducted studies to collect building energy use data specific to their service territories and benchmark these buildings through existing benchmarking tools. During the Benchmarking Software Tools Assessment Forum conducted as part of this project, IOU s discussed the issue of not having building level indicators in their billing system. This is a fundamental challenge that utilities face to benchmarking large number of buildings in an automated and user friendly manner mapping utility meter numbers to corresponding building addresses. Utilities often do not having consistent building level indicators such as addresses in their billing system, which only identifies a customer with account / service account ID s or meter numbers. 6

14 One IOU statewide goal aimed at overcoming this barrier was to pilot proxy benchmarking commercial buildings during the program cycle. A recent pilot toward this end was conducted by Southern California Edison (SCE) in its territory. Before this effort, SCE used the California End Use Survey (CEUS) as the benchmark. The project included an exhaustive collateral campaign conducted to reach out to customers through repeated mail-in reminders for sending in the survey. The campaign had impressive success in customer information retention. However, due to privacy laws the utilities are still not able to post customer data obtained from the campaign into Portfolio Manager without explicit customer consent. 7

15 EMERGING TOOLS/ TECHNOLOGY EVALUATION ENERGY STAR PORTFOLIO MANAGER DESCRIPTION The EPA's ENERGY STAR Portfolio Manager (Portfolio Manager) is a free, web-based benchmarking tool widely used nationwide for commercial and industrial sector benchmarking. In the commercial sector, it provides an operational benchmark and a recognized numerical rating known as the ENERGY STAR score. Approximately 40% of all commercial office space in the United States has been benchmarked through Portfolio Manager thus far. 3 Outside of California, Portfolio Manager has been adopted for voluntary or mandatory benchmarking activities in New York City, Washington DC, Seattle, and by Canada as their national tool for existing commercial and institutional buildings. Based on a review of EPA s documentation, Portfolio Manager compares a building s source energy use (weather-normalized) to that of a national database of buildings developed from the Department of Energy s most recent Commercial Building Energy Consumption Survey (CBECS). 6 The tool also calculates the associated greenhouse gas emissions for a building, and offers water use tracking. It filters the CBECS database for buildings with use/operating characteristics similar to the benchmarked building, including occupancy type and density, operating hours, and size. Portfolio Manager analytics produce a score between 1 and 100, with the highest potential score (100) representing the energy consumption of the most efficient building operation and the lowest score (1) representing the least efficient building operation. Because the score accounts for differences in business activities, two buildings can have the same source EUI (energy use intensity) but receive different ENERGY STAR scores. An engineer-verified score of 75 or higher (i.e., the building performs better than 75% of all comparable buildings nationwide) qualifies the building for the ENERGY STAR certification. While Portfolio Manager allows users to use default values as a way of encouraging tool use, buildings receiving ENERGY STAR certification cannot rely on default input values. Portfolio Manager currently provides an ENERGY STAR score for just 15 distinct building occupancy types (for details see Appendix A). Mixed-use buildings must have a dominant occupancy representing 50% or more of the floor area to receive a score. For buildings not receiving a score, Portfolio Manager benchmarking support includes comparisons to the national median source EUI according to property type. For obtaining either the ENERGY STAR score or the EUI, at least 12 consecutive months of data is required for each fuel type used in the building. Portfolio Manager also offers tracking capabilities. Customers of participating utilities can authorize their utility to upload energy use data directly to Portfolio Manager on an ongoing basis. The EPA works with utilities to provide these automated data 6 8

16 integration capabilities. Users can share data across accounts, which can enable portfolio tracking. While the online interface is functional, some assert that it could be more user-friendly. Third party tools, such as Melon / WegoWise (discussed in the next section) claim to bring a simpler front-end interface to Portfolio Manager. A major upgrade to the Portfolio Manager tool is scheduled for Spring 2013, with updated data exchange services and potentially an easier-to-use interface. The upgrade will include 18 broad property types with more than 80 primary space functions. CHARACTERISTICS In terms of data analytics, Portfolio Manager uses empirical building survey data obtained from CBECS to define peer groups. CBECS collects representative samples from 15 building space types across the nation for buildings larger than 1,000 square feet. The last CBECS was conducted in 2003, and consisted of a sample size of 5,215 buildings (about twice the number of CEUS, across significantly fewer building types). Due to sample size, Portfolio Manager cannot provide a score for buildings smaller than 5,000 square feet, with some exceptions. 7 The results of the next CBECS are expected to be incorporated into the upgraded version of Portfolio Manager to be launched in June Portfolio Manager also weather-normalizes user energy data inputs, using zip code level weather data obtained from the National Climatic Data Center (NCDC). Weather normalized energy consumption allows for a comparison of a building s energy use over time, accounting for year-to-year fluctuations in weather. Through this procedure, the energy in a given year is normalized to represent the energy that would have been consumed under 30-year average weather conditions. Whereas the ENERGY STAR score compares one building to its peers, the process of weather normalization compares a building to itself. This process in Portfolio Manager requires monthly energy data, which is then matched with monthly temperature data. Buildings with electric data entries spanning more than 65 days are not able have their electric energy use weather normalized. For the same reason, normalization is not attempted on fuels other than electricity or gas because actual monthly consumption is typically not known (e.g. there may be one annual delivery of fuel oil). Although these fuels cannot be normalized, they are still included in the total energy consumption. 8 Since 1999, when it was first made publically available, Portfolio Manager has become an established tool for commercial benchmarking. It is used in conjunction with many local/ state initiatives and utility programs nationwide. It has welldocumented resources and support structure for both individual users and energy providers. For example, EPA works closely with utilities to facilitate their set up of data exchange services (formerly known as Automated Benchmarking Services, or 7 According to the EPA ( in order to be eligible to receive a national energy performance rating in Portfolio Manager, all buildings must be at least 5,000 sf, with four exceptions: 1. If the building is a bank, it may be as small as 1,000 square foot 2. If the building is a hospital, it must be at least 20,000 square foot 3. If the building is a house of worship, it may be as small as 1,000 square foot 4. Data Centers do not have a square foot minimum

17 ABS). ABS lets utilities automatically upload historical and ongoing energy use data directly into the user s account, thus reducing time and effort for the user. Portfolio Manager also encourages third-party developers to create front end interfaces based on unique sector preferences. INPUTS AND RESULTS In addition to monthly energy use data, other mandatory inputs include building location, gross floor area, and space type. A complete list of mandatory and optional inputs is provided in Appendix A. In situations where the user does not have complete energy data available, Portfolio Manager allows users to identify any meter as temporary. This feature can be useful when actual meter readings for a specific period are unknown, but the user would still like to estimate their energy performance for that period. Portfolio Manager also allows the use of default values for certain inputs such as weekly operating hours, workers on main shift, and percentage of heated/ cooled space. The idea is to encourage users to input accurate information as often as possible without posing too great a challenge for them to complete the process. LIMITATIONS Portfolio Manager provides insights about the overall building performance by benchmarking the energy use but does not go beyond that into evaluating individual building system performance or providing specific recommendations for reducing energy use. Although the ENERGY STAR website does provide online resources to the users for creating action plans, training staff, and available energy efficient products, these are not state/ region specific, and do not link to local utility programs. The rating system (ENERGY STAR score) is currently available for only 15 building types and for buildings greater than 1,000 square feet. This is designed to match the building types and size collected by CBECS. 9 Therefore, unless the CBECS sampling methodologies are expanded to include more building types and smaller building sizes, Portfolio Manager will maintain the same limitations of building type and size. Overall, we felt that the Portfolio Manager interface could be improved if it guided users through data entry one screen at a time. However, it should be noted that Portfolio Manager is undergoing a major functionality upgrade, which will be launched in June Many issues with user friendliness may be addressed in the upgraded version. Please see section titled Comparitive Analysis of Assessed Tools: Technical Approach/ Test Methodology for more details regarding the Portfolio Manager interface. As part of past program process evaluations undertaken by HMG, 10 we have gained insights into the challenges of benchmarking using Portfolio Manager from the 9 According to the CBECS website, it is desirable to sample large buildings at a higher rate than small buildings because of the relatively large and variable amount of energy that is consumed in these buildings and since square footage is highly correlated with energy consumption Non-Residential Process Evaluation Study for SDG&E and SCG conducted by HMG. A complete report of the process evaluation was submitted to Sempra utilities (Program Manager: Rob Rubin) and is also available through HMG (Project Manager: Marian Goebes). 10

18 perspective of building owners, utility staff, vendors, and IOU program Account Executives. As part of these evaluations we had found that gathering accurate information that can be considered of value requires a significant amount of time and resources. While default values make the process easier, they also dilute the quality of information obtained, making it unreliable for informing future efficiency decisions. We include below, feedback obtained for benchmarking, specifically that related to the Portfolio Manager tool itself, from utility staff, vendors, Account Executives and building owners during the 2012 nonresidential process evaluation, which provides useful insight relevant to this study: Portfolio Manager can be difficult to use and does not comprehensively address all relevant building types. Users can enter data for all building types and view reports of energy intensity; however, users in excluded building types cannot receive an ENERGY STAR score The data entry is complicated. Gathering the data is time consuming and can be difficult to obtain. Customers turn to AEs [account executives] and vendors when they struggle to complete benchmarking. AEs and vendors input default values into the tool because the actual data is not easily accessible. Inputting default values into the tool takes at least 30 minutes; inputting actual data can take several hours per facility. The benchmarking score is meaningless when customers input default values into the tool. The results often make little sense and are rarely used again. Benchmarking has stopped at least a few customers from participating in IOU programs. There is no method of tracking which sites have been benchmarked, so customers may have to submit again for additional EE [energy efficiency] projects. MELON / WEGOWISE DESCRIPTION According to its developers, Melon was conceived as a user-friendly front end interface for ENERGY STAR Portfolio Manager, with some added functionality. During our data collection and review stage, Melon was still in the development stage and was acquired by WegoWise 11 a few days before the writing of this report. WegoWise currently works with a number of utilities across the United States to provide multifamily benchmarking. Given the recent merger, the combined tool plans to roll out a fully functional product next year (2013). The developers vision for the functional tool is that once users submit their data within WegoWise, the back end of the tool will connect to Portfolio Manager to 11 WegoWise is a private sector company that provides online tools for benchmarking multi-family buildings. As this report was being written, Melon was acquired by WegoWise. 11

19 generate the ENERGY STAR score. Therefore, the building energy use analysis database used to generate the score will be the same as that used by Portfolio Manager (i.e. CBECS). The current WegoWise platform allows for account sharing and setting up master accounts. Master account holders (utilities, portfolio owners) have the ability to run detailed, customized reports. Users will have the option to purchase other reports in the future which will analyze the user's smart meter interval data. WegoWise will also provide recommendations on how to improve ENERGY STAR scores, including recommending programs from utilities in the local territory that have partnered with WegoWise. There is also the ability to provide white-labeled solutions reflecting the IOU or the program name as the front face of the tool. When Melon/WegoWise is ready for use, the user will need to provide his/her utility username, password and account number(s) for the utility customer facing system, which WegoWise will use to link to the utility account and import utility data into the tool. WegoWise currently has the ability to import 'Green Button' data 12 as well as other bills available electronically. Once the utility data is procured, the tool will communicate with Portfolio Manager through its web services to generate the reports and ENERGY STAR score. CHARACTERISTICS Melon/WegoWise will share the same characteristics relating to data analytics as Portfolio Manager. Per its developers, the WegoWise platform also offers, or plans to offer, the following functionality beyond that of Portfolio Manager: User-friendly interface: The tool will aim to make it easier for non-technical building owners and operators to obtain an ENERGY STAR score. Ease of acquiring utility data: The platform will automatically import utility bill data from utility servers and send it to Portfolio Manager. Users will need to create a Portfolio Manager account and select WegoWise as an Energy Service Provider. Account sharing between building owner and tenant: WegoWise allows building owners to set up master accounts and then sub accounts for tenants. The tenants can upload their utility data, and control the type of access allowed to the building owner ( View or Edit access). The building owner can then submit the entire building data to Portfolio Manager. However, tenant cooperation/authorization is still required. Opportunity for customer engagement: a utility can elect to create a privately branded portal (for a fee) to engage directly with customers. This is not currently possible with Portfolio Manager. 12 Green Button is an industry-led effort launched in January 2012 that responds to a White House call-to-action to provide electricity customers with easy access to their energy usage data in a consumer-friendly and computerfriendly format via a "Green Button" on electric utilities' website. Green Button is based on a common technical standard developed in collaboration with a public-private partnership supported by the Commerce Department's National Institute of Standards and Technology. Voluntary adoption of a consensus standard by utilities across the Nation allows software developers and other entrepreneurs to leverage a sufficiently large market to support the creation of innovative applications that can help consumers make the most of their energy usage information. So far, two utilities - Pacific Gas & Electric and San Diego Gas & Electric - have implemented live functionality on their websites. (Source: 12

20 We were unable to test how a first-time user would set up a commercial facility in WegoWise since the commercial building tool is not fully developed yet. However, the commercial tool will share certain aspects of the currently operational multi-family platform (such as account sharing, visually-rich outputs, and customer support). INPUTS AND RESULTS At a minimum, users will have to enter the same amount of data that they would enter for Portfolio Manager. This includes location information, building space type, building vintage, and twelve months of consecutive utility data (which may be entered manually or uploaded directly from the utility system). WegoWise will also provide generic recommendations based on the building s ENERGY STAR score and type of building. Since the product is not fully developed at this time, we are unable to evaluate the added value of the tool s recommendations to users. Also, at the time of writing this report, the WegoWise tool is not set up to connect to Portfolio Manager for commercial buildings. We reviewed the currently available features of the WegoWise portal, but these did not yet have the relevant options/drop-downs for commercial buildings. The outputs can be seen in the form of a graph of combined energy use or individual fuel breakdown, and displayed as a function of building square footage (total) or conditioned square footage of the building (e.g. kbtu/sf, or kbtu/ conditioned sf). Screenshots of WegoWise s user interface are included as part of Comparitive Analysis of Assessed Tools: Technical Approach/ Test Methodology; Rating Process; Melon/WegoWise section. Please also see screenshots presented by the developers during the Benchmarking tools assessment forum in Appendix B of this report. Outputs also include cost breakdown by fuel type, and CO 2 emissions generated. As of now, the CO 2 emissions are calculated using a simple conversion factor based on national averages, but that may change as the tool develops. LIMITATIONS Given the current stage of the tool, we were not able to determine all its limitations. Since the tool will connect to Portfolio Manager on the back-end, it will inherit all the data limitations that apply to Portfolio Manager- the use of a national building database as opposed to a more focused (state or region-specific) database of buildings, and the inherent limitations of Portfolio Manager's computational model. We were also not able to determine how WegoWise will report source and site EUIs for buildings that are not eligible for an ENERGY STAR score. While Portfolio Manager is a free tool, Melon/WegoWise will be priced for individual users, groups (such as large enterprises) and utility companies. The tool developers are in the process of developing pricing models for end users and utility customers (for e.g. if the utility contracts with WegoWise, it can offer it for free to its customers for the first year, after which the customer can pay for the subscription). For WegoWise to be able to upload the customer s building and energy data into Portfolio Manager, the user will need to enable Portfolio Manager to accept the incoming data from a third-party. Therefore, though a user will not need to input any building or energy data into Portfolio Manager, he or she will still need to create a Portfolio Manager account and authorize WegoWise from within the Portfolio Manager account to update building and energy data. Currently, this section is under 13

21 Automated Benchmarking Services Console in Portfolio Manager, and potentially difficult to find for a first-time user. While not an inherent drawback of WegoWise itself, it adds a limitation for users not familiar with Portfolio Manager (who are the intended audience of a tool like WegoWise). However, as a private, paid service, WegoWise may offer enhanced customer support to guide users through this process. CALIFORNIA BUILDING ENERGY USE RATING TOOL (CBEURT) DESCRIPTION CBEURT is developed by the CEC in cooperation with the Oakridge National Laboratory (ONRL), and is projected to be a web-based operational rating tool. Although the scoring methodology and reporting graphics for CBEURT are similar to BEARS, it is distinct from and not directly tied in any way to BEARS, which is also in development. CBEURT complements the Portfolio Manager generated reports and can be used for voluntary disclosure as required by California Assembly Bill CBUERT requires similar inputs to Portfolio Manager, but its ratings are based on a California-specific building peer group developed using the California End Use Survey (CEUS) database. 4 CBEURT rates buildings on the Zero Energy Performance Index (ZEPI), where a score of 0 indicates zero net energy (ZNE) and a score of 100 represents the typical California building of similar type. Figure 1 portrays the planned graphical presentation of a CBEURT rating. Since buildings are only benchmarked against California peers, the median EUI for a building may be different than a similar building in Portfolio Manager. FIGURE 1: CBEURT RATING GRAPHIC CBEURT benchmarks against the approximately 2700 buildings in the CEUS database. The CEUS data was collected in 2003 and released in There has been no research thus far on how the scores might be skewed from using older data. HMG is not aware of any plans for a revised CEUS database in the future. As of December 2012, CBEURT exists as a stand-alone spreadsheet in Excel. The benchmarking score is simply presented as a number within the spreadsheet. There is currently no functionality to transfer data or interface with Portfolio Manager, nor are there any portfolio capabilities (i.e. if a user has 100 buildings, he/she would have to enter the data 100 times into the spreadsheet to obtain 100 distinct scores). Though as an Excel document, the user can archive building ratings simply by saving them under a new file name. The beta version of the spreadsheet can be obtained from the CEC upon request. 14

22 CHARACTERISTICS CBEURT can accommodate 39 building space types (for details see Appendix A), including multiple space types in a single building, as long as the square footage of all individual space type equals the total building floor area. There is no maximum allowed number of space types for a given building. The model allows users to note the percent of total heating and cooling that is being used by each space type in the building. CBEURT can also benchmark spaces as small as 1,000 sf, although it limits each space type has assigned to minimum and maximum sizes based on industry standards. CBEURT ratings are weather normalized using building zip code to identify annual heating degree days (HDD) and cooling degree days (CDD) from the National Climatic Data Center to adjust for climate zone on an annual basis, as opposed to Portfolio Manager which weather normalizes on a monthly basis. Research from ORNL showed that banks and office buildings (the largest segment of the commercial market being targeted for benchmarking) are consistently heated and cooled year round, and thus HDD and CDD showed no statistically significant correlation with annual building performance. INPUTS AND RESULTS The CBEURT model generates a rating using annual energy data including inputs for building location, annual energy consumption, space type, hours of operation, number of workers, number of computers and conditioned floor area. CBEURT allows for electricity, natural gas, propane, district steam, cogenerated district steam, and renewable energy production inputs, and allows users to input as many energy types as apply to the site. Total square footage, operating hours, workers, computers, heating and cooling are portioned out by space type as well. Default values for hours of operation are suggested based on building size and type. The current model is contained in a locked Excel workbook. All input cells are color coded and easy to locate, however whether an individual input is required versus optional is not clearly distinguished. Any invalid inputs or errors are noted in a side panel. The CBEURT score is generated at the top of the screen in large font, with no explanation for the rating scale (see Figure 1, on the previous page). Other outputs from the model include site and source energy and metric tons of CO2/year. Though the building score is easy to find, other results are not highlighted and could be easily overlooked. LIMITATIONS CBEURT s main limitation is that it is not yet a web-based tracking tool. Buildings in a portfolio cannot be effectively tracked or managed as an excel-based tool. This also means that all utility data must be manually inputted and updated. The tool cannot interface with utility websites. There is also currently no user assistance, guidelines or help functions defining input requirements. 15

23 ENERGYIQ DESCRIPTION This free, web-based tool under active development is available for use and is designed to be a next step for buildings already set up in Portfolio Manager, with which it interfaces, providing deeper, complementary comparisons to user-defined peer groups and a California-based building database, as well as guidance towards actionable improvements. Billed as Action-Oriented Energy Benchmarking, EnergyIQ was developed and is maintained by LBNL in collaboration with Itron, Usability.org, Big Head Technologies, utest, and Usable Buildings Trust (UBT), UK (Dr. William Bordass). Development has been funded by the CEC s Public Interest Energy Research (PIER) funds and the California Air and Resources Board. EnergyIQ has had a soft launch and sees approximately 6,000 visitors per year. It has been accessed in 105 countries, though 80% of the hundreds of buildings benchmarked through EnergyIQ are in the U.S., with a majority of these being in California. Unique to EnergyIQ s development is its API and related support, enabling thirdparties to integrate the power of EnergyIQ methodology and analysis with their own tools and interfaces. The reasoning is that an API allows industries to customize the user interface and other benchmarking functionalities to niche markets and for EnergyIQ to therefore reach a broader market base. So far about 20 organizations have signed up for the API. LBNL aims to make the API as hands-off as possible, providing sample code and support for people using the API, without mandating any design parameters. CHARACTERISTICS EnergyIQ is intended to provide a minimal effort bridge between the whole building benchmarking of Portfolio Manager and investment grade building audits. It focuses on flexibility, enabling users to define their own peer group for comparisons, starting with a choice of statewide (CEUS) or national (CBECS) databases against which to benchmark their building. A building s peer groups can be further filtered for benchmarking against total energy, energy from specific fuel types, and/or specific end use energy or end-use technologies (building features). EnergyIQ uses annual building energy use data that is not weather-normalized, since the tool aims to minimize the amount of data that a customer has to enter in order to simplify the process weather-normalization generally requires at least monthly energy use data. The tool can disaggregate end use energy by taking the known end use energy components provided by the customer and filling in the rest based on defaults established from its database and the user s defined peer group. Features like lighting ballasts, HVAC type, and controls, can also be shown by count in comparison with peer groups. A key aspect of EnergyIQ is its potential to provide actionable recommendations for building energy improvements, tailored to the specific site based on the information and key building features provided by the customer. Based on the type of 16

24 equipment included in buildings in the database peer group, as well as any end-use data provided by users, the tool s ACT module will provide recommendations, either specific measures or potentially programs that could support improving a building s performance. Though EnergyIQ s Benchmarking and Tracking components are functional, its action module (ACT) remains incomplete pending resolution of IT implementation issues (per the developer). Once its ACT component is in place, EnergyIQ will be officially launched and more aggressively promoted to the commercial building industry. INPUTS AND RESULTS The original database that forms the backend of EnergyIQ is the California Commercial End-Use Survey (CEUS), which provides details on energy use and characteristics for about 2800 buildings and 62 building types. EnergyIQ s building types follow those in CEUS (for details see Appendix A). National data from the Commercial Buildings Energy Consumption Survey (CBECS) was subsequently incorporated, allowing benchmarking across the country. Users can import their building data from EPA's Portfolio Manager. EnergyIQ interfaces with Portfolio Manager and compliments the rating with recommendations for improvement. With recently secured additional funding, the EnergyIQ team plans to make the interface more user friendly, and provide better graphical representations and ways to normalize data. It does not currently have the capability of benchmarking against other users in the system. LIMITATIONS EnergyIQ is not necessarily a stand-alone tool, by design, and is most functional when used in conjunction with an account in and a reasonable understanding of Portfolio Manager. It does not provide an ENERGY STAR score nor does it produce the documentation required by implementation of AB 1103 disclosure requirements. Per the tool site and developers, the link and data transfer to Portfolio Manager are functional but in process of improvement. While the interface is visually pleasant, fully leveraging and integrating the tool across its (currently two, eventually three) components Benchmark, Track, and Act was found to be not fully intuitive. Also, like Portfolio Manager, response to user input was occasionally slow, noticeably so. This may be an inevitable trade-off to providing powerful, on-the-fly database filtering and analytics accessed over internet connections of varying speed and quality, of which EnergyIQ has no control. The multiple input and peer filtering options that on one hand provide tool flexibility and enable tailoring it to user-specific criteria, also make for the following potential limitations: 1. The resulting magnitude of repeated manual inputs could lead to user error. This is especially a concern if multiple buildings with differing characteristics and peer group filters are being entered. Bulk imports, more centralized filter definitions, or other changes reducing manual input could benefit tool functionality. 2. Overly specific peer filtering can result in a very limited quantity of buildings from the database making up the peer group against which a user s building is benchmarked, which could be misleading if results are taken out of context and/or the peer group performance is inappropriately extrapolated. 17

25 A methodology limitation is Energy IQ s lack of weather normalization, which the tool developer indicated is intended to simplify the tool for users. This could result, however, in energy use fluctuations resulting from annual weather variations exaggerating or obscuring energy savings from a user s property improvement actions. BENCHMARKING TOOL STATUS REVIEW THROUGH A FORUM To support assessment of benchmarking software tools on behalf of the IOUs, HMG hosted a half-day benchmarking software tools forum at its Oakland offices on November 29, The forum brought together tool developers, IOU staff, and other informed and interested parties in order to review the software tools identified for study in this project. In the context of informing statewide benchmarking strategies and from the perspective of tool developers and IOU staff or contractors involved in benchmarking, forum goals included the following: Clarify each tool s background, intent, and anticipated future development Identify each tool s strengths and target applications as well as its weaknesses and challenges Discuss each tool s usability, applicability, and effectiveness for various customer segments and building types in California This report section summarizes the scope and content of the forum discussion, with supporting documents including presentations from developers of each tool and forum notes included in Appendix B. Information presented and discussed at the forum inevitably overlapped with and further informed content of this report s other sections. Where appropriate, this section does not duplicate tool details incorporated elsewhere. PARTICIPANTS Attendees consisted of private and public sector tool developers or implementers, utility staff involved with benchmarking, and HMG project staff having experience working with benchmarking tools and/or guidelines. Participants represented the following organizations: Public sector developers: U.S. Environmental Protection Agency (EPA), Lawrence Berkeley National Laboratory (LBNL), and California Energy Commission (CEC) Private sector developers: MelonPower and WegoWise Investor-Owned Utilities: Pacific Gas & Electric Company, Southern California Edison, and Sempra Heschong Mahone Group project staff 18

26 TOOL PRESENTATIONS The forum topics focused on the four tools selected for this assessment. The first portion included each developer leading a brief overview and discussion of their respective tools. Presentations generally included discussion of tool purpose and intent, targeted users, data input requirements, databases and methodology, and current development status. Content and slides from each developer s presentations are included in Appendix B. California Building Energy Use Rating Tool (CBEURT) Keith Roberts from the CEC attended the forum remotely and presented the California Building Energy Use Rating Tool (CBEURT). ENERGY STAR Portfolio Manager Tracy Narel from EPA attended the forum remotely and presented the ENERGY STAR Portfolio Manager tool. Alice Liddell (ICFI) supports EPA with the ENERGY STAR program and attended the forum in person. Melon / WegoWise Craig Isakow is the founder and CEO of Melon Power, and attended the forum remotely. Dan Teague from WegoWise attended the forum in person. Melon will be partnering with WegoWise, an established name in multi-family benchmarking in the development of this tool. EnergyIQ Paul Mathew is a co-team lead for development of the EnergyIQ tool at LBNL, and attended the forum remotely. Mr. Mathew gave an overview of the tool features and screenshots of the tool capabilities, including graphical representations and comparisons to relevant peer groups. FACILITATED DISCUSSION Forum participants discussed various challenges faced in implementing and complying with California s building performance disclosure regulations, AB 1103, and for promoting benchmarking the available software tools to IOU customers. Main discussion areas are highlighted and summarized below. Mapping building addresses to meter numbers: It was general group consensus that one of the biggest challenges to utility support of benchmarking and compliance with AB 1103 regulations are the lack of building level indicators in the existing IOU billing systems. IOUs do not track energy use against a property address, but rather by service address IDs, which are not reliably assigned to a correct or consistently entered address. Database addresses are those entered by customers. Identifying and aggregating meters (and their energy use) to buildings is an ongoing issue. Protecting customer privacy: For protecting customer privacy, the utilities need to aggregate data or create other filters (such as in the case of single tenant occupancy). Melon spoke about the current regulations in Washington D.C where tenants can be fined if they refuse to provide necessary data. This structure does not exist in California. Obtaining tenant authorization, 19

27 especially if the tenant is no longer reachable is an issue that needs further discussion in California. IT coordination with EPA: AB1103 mandates the use of Portfolio Manager for benchmarking and documentation, and the utilities are required to provide the customers with 12 months of energy use data when requested. In order to automate the process, utilities can set up data exchange services in coordination with the EPA so that data can be uploaded directly into the customer s Portfolio Manager account. Setting this service up can pose a considerable strain on utility resources. Tools like Melon could potentially reduce this burden by integrating their programming with the existing data portals within the utility systems. Continued public/private partnerships that reduce the IT burdens are needed if all buildings are to be benchmarked effectively. Data input vs. quality results: From a user perspective, the forum participants discussed the trade-off between having cumbersome data inputs within a benchmarking tool that can discourage its use as opposed to the use of default values, which can make the process easier but affects the quality and reliability of results. One potential solution is to continue to offer and refine the step-by-step guides and training programs offered by the utilities and EPA, which help customers understand the general concept of benchmarking while still letting defaults move the process along (when required). Portfolio Manager s new upgrade roll out in June 2013 may make it much easier for customers to benchmark using Portfolio Manager. Reaching smaller businesses and benchmarking smaller buildings: The lack of operating staff or knowledge of building performance and/or comfort parameters and issues can make benchmarking smaller buildings a difficult process. For these reasons, buildings currently benchmarked tend to be larger buildings with on-site staff and more intensive energy use. Portfolio Manager for instance does not target small businesses nor do most of the other tools currently available. CBEURT does provide an opportunity in this regard, as it has the capacity to benchmark buildings as small as 1,000 sf, as well as numerous minority (less than 50%) space types within a building. FUTURE DIRECTION FOR TOOL IMPROVEMENTS Attendees mentioned that third party tool developers along with the existing tools such as Portfolio Manager will play an important role in the future growth of benchmarking efforts. There is potential for collaboration to make benchmarking commonplace and there will need to be a number of different tools to support different owners, occupants and industries. Another potential direction of growth is to develop cohesive tools where asset and operational ratings can complement each other, providing additional value to users. 20

28 COMPARITIVE ANALYSIS OF ASSESSED TOOLS: TECHNICAL APPROACH/ TEST METHODOLOGY ANALYSIS/TEST PLAN HMG s assessed the software tool outputs relative to inputs and process, as well as compared results across the assessed tools. Anonymous building and billing data was provided to HMG by PG&E. Initial data included annual energy use and basic building data for twenty-five buildings representing grocery, hotel, library, healthcare, office, religion, school, and retail building types. For Portfolio Manager specifically (and WegoWise by reference), monthly energy use data is needed. This level of data was available for seven of the initial twenty-five buildings, including four offices and three small retail buildings. The office building type was selected due to its large footprint across California. The small retail building type was selected due to the project scope identifying a specific interest in a tool s capability to support benchmarking of smaller businesses and properties. For these reasons, the comparative analysis focused on the office and small retail building types. In the interest of time and resources, HMG selected five buildings (three offices and two retail) with the aim to achieve a reasonable diversity across building type, size, and California climate zone. Building # Type City Zip Code Climate Zone Gross Floor Area (sf) Site EUI (kbtu/sf-yr) 10 Large Office Cupertino , Medium Office Fresno , Small Office Selma , Small Retail San Jose , Small Retail Seaside , FIGURE 2: BUILDING SAMPLE SUMMARY CHARACTERISTICS RATING PROCESS ENERGY STAR PORTFOLIO MANAGER As has been noted in this report, EPA is preparing for a major Portfolio Manager overhaul to be rolled out in It is anticipated that in addition to functional improvements, this upgrade will also enhance the experience of using and benchmarking buildings through Portfolio Manager. 21

29 That said, Portfolio Manager s current interface is often critiqued as not being user friendly enough. While inputting the data to benchmark buildings for this project HMG found that while functional, at some points it could be confusing for new users. This section walks through the basic steps involved in benchmarking with Portfolio Manager. CREATING A NEW PROPERTY/ FACILITY After a user creates an account, the tool automatically creates a sample facility for that account. The user then has the choice to click on the facility and find the Edit button, located inconspicuously in the upper right corner, or he/she can delete the sample facility and click Add a Property, also located inconspicuously. Figure 3 shows a screen shot. FIGURE 3: SCREENSHOT OF PORTFOLIO MANAGER MAIN INTERFACE AFTER LOGGING IN Mandatory information at this stage includes facility type, address, and year built. We noted that there is no validation of the building address which means that user errors are not caught. The tool also does not warn the user if the facility has already been benchmarked in Portfolio Manager an issue brought up during the nonresidential process evaluation conducted by HMG (see 22

30 Emerging Tools/ Technology Evaluation section). ENTERING SPACE USE AND METER DATA An example of an unclear juncture in the Portfolio Manager data entry process is at this next step. Once basic property information has been entered, the user sees a content-heavy screen with multiple sections. While it is not readily apparent, the required next step is to input additional data by clicking the Add space button under the Space Use section (the second section in Figure 4). Once done, the next screen inside the Facility Space section has the data entry requirement clearly identified. FIGURE 4: SCREENSHOT OF PORTFOLIO MANAGER STEPS FOR ENTERING SPACE AND METER DATA Below the Space Use is the Energy Meters section with buttons to add, update or view energy meter data. Again, the buttons are small and not obvious that they are the next steps for data entry. Once the user finds the Add meter button, the input screens are relatively straightforward. HMG noted a potential limitation for benchmarking buildings with more than one meter. If one of the meters does not have 12 complete months of data (as may be the case with short-term tenants for a multi-user facility), then Portfolio Manager will fail to generate a score for the entire building. There is no functionality to automatically combine two or more meters within a building for the same usage date. A user has to do this manually. 23

31 A significant and unique Portfolio Manager data entry feature is its capability to bulk import facility characteristic and energy use data via spreadsheet templates, something that other tools have thus far not developed. (EnergyIQ does feature automated 24-hr data import from Portfolio Manager, but buildings entered directly into that tool must be done so manually) Portfolio Manager users download the appropriate Microsoft Excel template for their building type from Portfolio Manager, populate it using the instructions provided in the template, and then it to ENERGY STAR administrators. There is a 3-14 day wait period before the data is populated. REPORTING AND ANALYSIS Although not utilized in the one-time benchmarking involved with this project, Portfolio Manager also includes a reporting module. This feature enables users to generate standard and user-defined charts summarizing energy use performance of single buildings or across a portfolio. Reports can provide a fast and powerful means of summarizing energy use, costs, etc. for a building or defined set of buildings over time. Alternatively, Portfolio Manager can export data and calculated metrics to Excel for custom analysis and reporting. MELON/WEGOWISE This tool is still in development for commercial buildings, and thus for this study HMG was unable to evaluate how a facility is created or how energy use data is added. HMG provided the building data to the tool developers, who uploaded it into the WegoWise platform and provided us with access to a demo account. The following two figures include screenshots from the WegoWise portal portraying data provided by PG&E. Figure 5 shows the annual energy use profile for a single office building. Figure 6 shows the current functionality for comparing buildings across a portfolio, using the project data identified in section titled Error! Reference source not found.. 24

32 FIGURE 5: WEGOWISE SCREENSHOT OF STUDY BUILDING 11 S ENERGY USE PROFILE FIGURE 6: WEGOWISE SCREENSHOT OF A SAMPLE REPORT FOR ALL BUILDINGS IN A PORTFOLIO. CBEURT CREATING A NEW PROPERTY/ FACILITY Since there is currently no web-based user interface or login to access CBEURT, creating a new building can be done simply by opening the workbook and updating building information. All spaces for inputting data are labeled and marked as yellow. All other cells are locked and cannot be changed. Outputs update automatically as inputs change, so users can see in real time how reducing their energy consumption or adding renewables would affect their score. The rating is provided as a standalone number on top of the spreadsheet as shown in Figure 7 below. 25

33 FIGURE 7: CBEURT SPREADSHEET SCREENSHOT SHOWING THE ENERGY PERFORMANCE RATING. ENTERING SPACE USE AND METER DATA As shown in Figure 8, space use can be divided by operating hours, square footage, number of workers, computers, and heating and cooling for any of the 39 space types, as long as 100% of the building area is accounted for. Figure 8 also demonstrates that if the assigned area is outside of this defined range, an error will appear noting that the entered area is too small or too large for the space type. Energy use data is input as annual rather than monthly consumption. Currently all data must be input manually. There is no ability for bulk data import or to import data through a link to utility databases. FIGURE 8: CBEURT SCREENSHOT SHOWING INPUT REQUIREMENTS FOR SPACE USE. A building s number of computers, hours of operation, and number of workers are input manually, but if actual values are not known, suggested defaults are generated based on space types. 26

34 ENERGYIQ Energy IQ s web-based interface is in active and available for use. While the tool includes capability to auto-transfer data from Portfolio Manager, for this project HMG created an account and input data for five sample buildings directly into EnergyIQ. BUILDING CHARACTERISTICS Once an account is set up, via the My Buildings tab a user enters data for his or her buildings including at minimum the following mandatory information: Building Details Name, country, zip code, total floor area, year constructed Peer Group Floor area range(s), Vintage(s), Locations (regions), and Building type(s) Energy Use For each energy type : Annual Use, Annual Cost ($), Billing Period Start Date Source Energy, CO 2, and CO 2 e default factors are provided, but can be user-modified A number of optional inputs are available for an informed user to further define building-specific parameters, such as known energy end-use totals (from a submeter, for example). For required building inputs not provided in HMG s anonymous sample dataset, including year built and operating hours, estimates were developed by the project team. PEER GROUP CHARACTERISTICS Once building data is input, they can be benchmarked through a series of steps requiring the user to define the following benchmarking criteria: Reference database (CEUS or CBECS) Metric: total energy, end-use energy subset, or end-use category Peer group filters Floor area range(s), Vintage(s), Locations (regions), and Building type(s) 27

35 Figure 9 below demonstrates example peer-group selection options. CEUS database and Total Energy metric are already defined in this example, on the previous input page. FIGURE 9: ENERGYIQ - PEER GROUP FILTERS (SCREENSHOT) Figure 10 summarizes peer group filters selected for this study. While in theory one could always define a peer group to precisely match the desired building characteristics, peer group filtering is in practice limited by the selected database. Balance was sought in our analysis between being as specific to building characteristics as possible while maintaining a reasonable sample size (number of peer group buildings) against which to benchmark. Some trial and error was involved; in one case the initial peer filters were so specific that they resulted in less than 10 peer buildings. The peer group filters were subsequently broadened to generate a larger sample. 28

36 Peer Group Filters Building Type Database: California Buildings (CEUS) # of peer Metric: Total Energy buildings Large Office Medium Office Small Office Small Retail Small Retail size: 25, ,000 sf size: 0-25,000 sf Vintage: All years Central Coast 25 type: Central Valley 33 all offices Central Valley 68 Central Valley, Central Coast type: shop in mall, shop in strip mall, other retail store FIGURE 10: ENERGYIQ PROJECT PEER GROUP FILTERS SUMMARY Once a peer group is defined, an EnergyIQ page summarizes its characteristics, including the distribution (site or source EUI, Emissions, or Cost) of the peer group as well as how many buildings it includes. A user can then choose one of his defined buildings to graphically integrate into the peer group data to see that building s percentile rating. Figure 11 includes an example of such a summary page. FIGURE 11: ENERGY IQ PEER GROUP INFORMATION AND EUI DISTRIBUTION (SCREENSHOT) It should be noted that at the time HMG tested EnergyIQ it was found to have a handful of software bugs preventing full tool functionality. Most were minor, and while they could affect confidence of potential users, HMG expects that by the official launch of EnergyIQ these bugs will be addressed. An example of one such bug was EUI output errors resulting from issues with energy use data conversion calculations. 29

37 Tool developers are aware of and addressing this issue. A work-around requiring data to be input in specific energy units for both electricity and natural gas enabled successful data input and tool testing in the interim. TRACKING AND ACTING Ongoing use of the EnergyIQ tool could enable tracking of the user s building portfolio over time, including incorporating user-defined Targets for various parameters such as total or end-use energy, emissions, etc. Figure 12 shows a sample graphic that can be included on a user s dashboard to facilitate tracking and monitoring. As previously discussed, EnergyIQ s ACT component may be a powerful tool feature, but as of December 2012 is not yet functional and therefore was not evaluated by HMG. FIGURE 12: ENERGY IQ DASHBOARD TRACKING RESULTS Figure 13 summarizes benchmarking rating results across the software tools, buildings, and climate zones evaluated. Since Portfolio Manager generates the rating score for Melon, it is not explicitly included in the Figure 13 summary. Building Data Rating Emissions (metric tons CO2 / year) Building Type City Climate Zone ESPM CBEURT EnergyIQ ESPM CBEURT EnergyIQ 10 Large Office San Jose Medium Office Fresno Small Office Selma Small Retail San Jose Small Retail Seaside FIGURE 13: BENCHMARKING RESULTS SUMMARY COMPARISON 30