The uncertainty variables focus on the substantive sources of expressed scientific uncertainty. We intended to capture the kind of informational problem that created the uncertainty. We were concerned only with final uncertainty, uncertainty that remained in the mind of the panel and we were not interested in flaws and problems in studies that the panel reviewed but dismissed in light of better evidence. Therefore we coded only the summaries and conclusions of the documents and, if they contained a special section called uncertainty analysis, we coded that too.

We also did not second-guess the panels or approach their work in a critical fashion. The uncertainty variables are simply descriptions of what the scientists thought remained uncertain, after all the evidence had been considered. While our ultimate unit is the sentence (or a set of adjacent sentences), the coders had to consider the wider context to code the sentence or sentences properly.

We developed and Uncertainty Ontology and a Judgment Ontology.

Uncertainty Ontology

The 28 variables follow a hierarchical ontology. They are defined by a decision tree where the coder has to code at the most specific (lowest) level possible 

Judgment Ontology

Judgment variables describe how the panel judges the weight of the evidence and it follows more closely the language they use to do so. There are five judgment variables.

Hedging: a way of indicating that experts have doubt about or a lack of total commitment to a proposition they present.

Confidence: emphatic commitment to a proposition.

Expert assumption: the expert is aware that studies or models make certain assumptions about the world. These assumptions are not directly supported by evidence, but according to the expert, this does not pose any problem.

Precaution: a way of dealing with uncertainty. Making conservative assumptions or building conclusions around “worst case scenarios” is a way of creating certainty where data and models fail to provide it.

Disagreement: Disagreement is a staple of science, but here we are interested in only disagreements that the report treats as unresolved.