Evaluating Modeled Data

Sampling data are standard inputs to the PHA process. However, these data often do not have spatial or temporal coverage to characterize all site-specific exposure scenarios. The health assessor needs to evaluate the available sampling data and decide if it adequately characterizes contamination for the time interval and geographical area of interest, and whether it can be used to support public health conclusions and recommendations.

Another important input to the PHA process is modeling data, which can supplement the available sampling data and better define the range of possible contaminant concentrations. Sampling will never tell you environmental contamination levels at every location and time of interest, but modeling can. Modeling results can be useful for many purposes (see text box on the right).

Health assessors may encounter many different types of models. Perhaps the most common is environmental fate and transport models, which simulate the movement of contaminants within and across environmental media. Some examples include:

• Hydrogeological models that estimate how contaminants move through groundwater.
• Air dispersion or photochemical models that simulate how contaminants move from sources through the air to downwind locations, and they may even evaluate the potential for atmospheric deposition to soils and surface waters.
• Bioaccumulation models that simulate the uptake of contaminants from surface waters and sediments into and through the aquatic food chain.

Additionally, site documents may include modeling analyses that simulate community members’ exposures to contaminants and the fate of contaminants within their bodies after exposures. For sites with lead exposure concerns, for instance, it is common to encounter modeling with EPA’s Integrated Exposure Uptake Biokinetic (IEUBK) model. This is just a partial list of the types of modeling studies that health assessors may encounter.

Despite the many different types of models available, a common theme is that they can be used to estimate (not measure) contamination levels at various times and locations, and these estimates can supplement sampling data; and they can fill data gaps in cases where sampling data are not available.

Health assessors are not expected to become experts on every model they encounter. ATSDR has experts with most modeling applications, and health assessors should consult with data scientists experienced with modeling whenever they must review or conduct a modeling study.

That said, health assessors should approach their review of modeling data in a similar fashion to their review of sampling data. For instance, it is important to understand why modeling was conducted and what questions the modeling was meant to answer. While some sampling reports have DQOs that clearly state why sampling was performed, modeling reports generally do not; but you may find modeling protocols that clearly state modeling objectives and anticipated limitations and uncertainties. These protocols are important to review. Similarly, health assessors need to review modeling studies to ensure their estimated concentrations are of a known and high quality. Important considerations for such reviews are described in the background on models, modeling approaches, and reviewing modeling results sections.

Background on Models

An extremely broad range of models are available to estimate levels of environmental contamination. These include statistical tools that predict the concentration of contamination for certain times and locations by interpolating among observed values. They also include mathematical models that hindcast or forecast environmental contaminants in various media (e.g., air, groundwater, surface water, soil) based on a mechanistic understanding of fate and transport mechanisms (e.g., diffusion, convection) and user inputs.

Health assessors may encounter models that vary greatly in complexity. These range from simple web calculators based on a few user inputs to highly sophisticated software programs based on thousands of lines of code and detailed user input files. Modelers sometimes classify models into screening applications and refined models. Screening models are typically simple applications that run from relatively few inputs and provide quick results; they also typically embody conservative assumptions such that outputs are upper-bound estimates of actual contamination levels. Refined models, on the other hand, tend to be more rigorous and computationally intensive; and they provide more detailed representations of physical, chemical, and biological processes. Ideally, the model you are reviewing or using has been published in the scientific literature or vetted by EPA or other environmental agencies.

Modeling Approaches

Environmental scientists often apply tiered approaches when conducting modeling. A first step typically involves application of screening models, because these can quickly provide a sense for whether estimated contamination levels require further investigation. Given that screening models invoke worst-case assumptions, their predictions are often viewed as upper-bound estimates of actual environmental contamination levels. If these outputs are below levels of health concern, no further modeling is conducted. Screening models are protective assessments of environmental contamination levels.

On the other hand, when initial investigations or screening model results are greater than levels of health concern, environmental scientists may conduct more refined modeling to generate more realistic estimates of environmental contamination. In comparison to screening models, refined models generally are more sophisticated, require considerably more user inputs, and are more computationally demanding. While screening modeling results tend to be upper-bound estimates of actual environmental contamination, refined modeling results tend to present best estimates of actual conditions, and the estimated concentrations may be lower or higher than observed levels. Refined models are predictive assessments of environmental contamination levels.

Despite these general guidelines, both screening and refined models include uncertainties and limitations. The quality of any given modeling application depends on many factors, such as the selected modeling application, the appropriateness of model assumptions, and the accuracy of model inputs. Thus, health assessors need to carefully consider these factors when determining whether the model results are appropriate for inclusion in the PHA process.

Reviewing Modeling Results

When reviewing modeling studies, you ultimately need to evaluate how accurately model predictions represent actual conditions. This determination is essentially an evaluation of model uncertainty, which is extremely difficult to quantify. It is important to note that such uncertainties exist with every model, even those touted as being the most realistic representations of environmental media.

EPA, consulting companies, researchers, and many other scientists have developed models for environmental applications. Many journals and books have also addressed modeling fate and transport of environmental contaminants in specific media. ATSDR does not expect health assessors to be capable of critically reviewing every possible modeling study. However, if you want to learn more about specific modeling applications, you can conduct online searches and literature reviews and refer to ATSDR’s Modeling Guidance Memo (under development). Remember that modeling studies can be very difficult to review. If you do not have the expertise to critically review a modeling application, seek input from colleagues who are experienced with the model.

Though health assessors often defer to experts within or affiliated with ATSDR to conduct detailed model evaluations, you should consider certain basic issues to identify model limitations and uncertainties:

• How thorough is the documentation of the modeling? Can another modeler generate the same outputs from the documentation provided?
• Is the model designed to generate health-protective, upper-bound predictions (like a screening model)? Or predictions of the actual real-world values (like a refined model)? What is the likelihood that the model underestimates or overestimates actual contamination levels?

• What are the limitations and assumptions of the model? Is it being applied to a scenario for which it was designed?
• Has model performance been documented? Have model predictions been compared to observed values for your site, or for similar sites?
• What input parameters were used? How were they determined? Are they realistic? Are they upper- or lower-bound input parameters?
• Are model outputs extremely sensitive to the values of particular inputs?
• How consistent are modeled data and sampling results?
• How broad are the uncertainty bounds on critical model outputs?

If you decide that modeling data should be considered for the PHA process, you must prominently distinguish the data based on models (estimates) from those based on environmental sampling (measurements). Your documentation should describe the model used, especially its uncertainties, limitations, and assumptions.

Consider recommending additional environmental sampling in cases where important public health decisions are based strictly on modeling data. This decision also may be affected by the nature and extent of community concerns — some community members may not be satisfied knowing that decisions about their health hinge on the results of a modeling analysis.