H.4 Mini-Project: Building Calibration ME 418 // Spring 2024 Background and Problem Statement: In lieu of a final exam, we will have a final project for this course. This homework assignment will serve as an introduction to the software tool for the project and give you a chance to establish a baseline energy model for your subsequent (final project) analysis. The objective of this homework is to create and calibrate a building energy model that reflects either your current house/apartment or your house in your hometown. To do so, you will use the BEopt software produced by NREL, information about the building structure, and historic energy usage data. Instructions for Calibrating the Model: 1. Download and install BEopt 3.0.1 from https://www.nrel.gov/buildings/beopt.html 2. Watch a tutorial video on using BEopt (optional) a. https://www.youtube.com/watch?v=92fUQEsTjk4 b. https://www.youtube.com/playlist?list=PLHC0xDtkdjgec8QhVt7exJY3tpSLEFk-d 3. Draw the building, including foundation, space types, and roofs (Geometry Screen) 4. Select appropriate building orientation, construction materials, equipment, and operating characteristics (Options Screen) 5. Select building location and utility rate (Site Screen) 6. Download the appropriate weather data (Site Screen) 7. Run the building simulation using a Design Analysis (from dropdown menu) a. Select DView option and use 60-minute timestep 8. Examine the energy consumption results in Data Viewer a. Select "Monthly” tab at the top of the viewer b. Select “Electricity: Total (kWh)" on the right side of the window and unselect other options C. Click the small wrench icon in the bottom-right corner of the viewer d. Select "Show sum over time step" e. Right-click the middle of the viewer and select “Send data to Excel" 9. Compare monthly energy data to your electricity bills from the previous 12-months a. Calculate the error between your actual historic energy usage and the building model b. If available, you may use more precise energy data (e.g. hourly, so long as you have that information from your utility company) 10. Iterate your building model by adjusting various building components, setpoints, or schedules to calibrate your model within the requirements of ASHRAE Guideline 14. Do not modify any known values (e.g., furnace type or window construction), only those which you had to assume in your initial design. Deliverable: Submit a model calibration report in which you describe the actual building and your calibrated building model. Show predicted monthly energy usage compared with historical data. Calculate the CvRMSE and NMBE for your building model. Clearly identify and justify any assumptions used in the model. Comment on the quality of your model. Submit a written report (~3 pages, including figures and tables) and your BEopt model file to Canvas by the start of class on the due date. Due: In-class on Tuesday, April 2, 2024 Page 1 of 4 H.4 Mini-Project: Building Calibration ME 418 // Spring 2024 (Abridged) Excerpt from ASHRAE Handbook – Fundamentals, Ch. 19.7 Model Calibration Calibration is the use of known data (e.g., utility bills) on the observed relationship between a dependent variable (e.g., simulation output) and an independent variable (e.g., simulation input) to make estimates of other values of the parameters from new observations of the dependent variable. For energy simulation models, calibration typically involves observation of changes in simulation output as simulation inputs are modified, with the goal of identifying a set of parameters leading to simulation outputs that match measured building performance. The perception of validity and usefulness of any energy simulation model is largely determined by how closely the simulation output matches actual building performance, usually in terms of energy consumption. The process of calibrating a model to match actual performance can be a complex and time- consuming endeavor. Identifying and isolating sources of discrepancy between the results of a model and actual data are not always possible, as described in the section on Uncertainty in Modeling. In addition, energy model calibration typically involves several input parameters that must be calibrated using a relatively limited amount of measured data; because of combinatorial complexity, calibration is an underdetermined system in which there can exist many unique (and substantially different) models that are within a tolerable error. The quality of a calibration is often evaluated in terms of statistical indicators that quantify discrepancies between the model output and measured output. These metrics are based on time steps, and it may be the case that a model calibrated to low-resolution (e.g., monthly) whole-building data is highly inaccurate when compared to higher-resolution temporal (e.g., hourly) or spatial (e.g., level of a thermal zone) data. Among statistical indicators, the normalized mean bias error (NMBE) and the coefficient of variance of the root mean square error (Cv(RMSE)), [defined below], are widely used. In these equations, the values are summed for each time step (e.g., monthly or hourly values) over the course of an evaluation period (e.g., year), and the parameter V is the building performance variable under consideration (usually monthly whole-building energy consumption): where, NMBE = Σ(Vactual Vmodeled) (N-1) · Vactual • 100% Cv (RMSE) = Σ(Vactual - Vmodeled)² N-1 actual · 100% Vactual = parameter's measured or metered value for each timestep (e.g., month) N = number of timesteps being analyzed during period of evaluation In many energy-estimating programs, secondary systems are represented by a mix of component models and simplified system models. For example, it is common for air and hydronic distribution systems to be represented by energy models that do not explicitly include components such as pipes/ducts, coils, and valves/dampers. Those methods are described in the following section. Page 2 of 4 H.4 Mini-Project: Building Calibration ME 418 // Spring 2024 ASHRAE Guideline 14-2014 states that a model can be considered calibrated if NMBE <5% and CVRMSE < 15% when monthly data are used, or NMBE < 10% and CVRMSE < 30% when hourly data are used. Because CVRMSE is the positive average sum-squared error divided by the actual mean, it can be considered the percent error between the simulation and measured data. Because NMBE is a signed error divided by the mean, it indicates bias percent for under- (NMBE > 0) or overshooting (NMBE < 0) the actual data during the period of evaluation. Additional resources and tools useful for energy model calibration include ASHRAE RP-1051 (Reddy 2006), ASHRAE RP-1093 (Abushakra et al. 2000), and NREL Technical Report 5500-60127 (Robertson 2013). The main challenges of manual model calibration are that it is labor intensive and time consuming, it requires a high level of user skill and knowledge in both simulation and practical building operation, and the results often vary with the individual performing the calibration. Several practical difficulties prevent achieving a calibrated simulation or a simulation that closely reflects actual building performance, including (1) measurement and adaptation of weather data for use by simulation programs (e.g., converting global horizontal solar into beam and diffuse solar radiation), (2) collecting reliable actual meteorological year data for a specific building during the type period in which energy-use data was collected (Bhandari et al. 2012), (3) choice of methods used to calibrate the model, and (4) choice of methods used to measure required input parameters for the simulation (i.e., building mass, infiltration coefficients, and shading coefficients). Calibrated models typically involve a large number of input parameters to be calibrated, a high degree of expertise, multiple iterations, and substantial computing time. Every model, calibrated or not, carries assumptions and simplifications that are often deemed reasonable, but should be reevaluated when the model is used for different purposes. Calibration techniques can be roughly classified as either manual or automated methods. Manual calibration methods include graphical analysis and sensitivity analysis. Examples of methods used for automated calibration include Bayesian analysis, pattern matching, and multiobjective optimization. Manual calibration is an iterative approach that can be labor intensive and involves separate manipulation of individual parameters. This approach involves using an existing building simulation model and “tuning” or calibrating the various input parameters so that simulation program output matches with observed energy use. Calibration can be performed using data from any time period (e.g., monthly, or only a few weeks over the year), but the final calibrated model is likely to be less accurate when fewer data are used during the calibration process. Hourly monitored energy data (most compatible with the time step adopted by most building energy simulation programs) can allow development of more accurate calibrated models, but calibrators often work without hourly data. During the manual calibration process, graphical representations and/or statistics comparing modeled data to measured data are displayed in an attempt to elucidate the value that input parameters could be set to in order to improve the match between simulation output and measured data. Calibrators often use sensitivity analyses to focus calibration efforts on the parameters that make the biggest difference in terms of energy use. In contrast to manual calibration, automated techniques use mathematical, algorithmic techniques implemented as computer software. Bayesian analysis, pattern-based calibration, and multiobjective optimization are methods used for automated calibration. Page 3 of 4 * H.4 Mini-Project: Building Calibration ME 418 // Spring 2024 Example Calibration Data You can use this example data to verify your own NMBE and Cv(RMSE) calculations. Month Energy Bill Data [kWh] Model Output [kWh] January 431 437 February 447 375 March 440 367 April 307 394 May 483 534 June 771 672 July 720 807 August 794 836 September 725 606 October 469 438 November 320 365 December 472 420 Model Output Energy Bill Data 900 800 700 600 500 400 300 200 100 Mnothly Electricity Use (kWh) 1 2 3 4 5 6 7 8 9 10 Month 11 12 NMBE = = 2.2% 128 [kWh] 11 · 531.6 [kWh] 59704 [kWh²] 11 531.6 [kWh] Cv (RMSE) = 13.9% Page 4 of 4