Know more

How is performed the reliability assessment?

The reliability assessment is performed based on the document’s criteria values. Criteria take into account the reliability of the data source itself, the means used to collect data (often the Material and methods section of the paper) and the accuracy and precision of the statistical procedures that have been implemented.

When the source, the means to collect data and the implemented statistical procedures are known an expert gives his/her opinion about how reliable is a data with this information. This opinion is expressed linguistically, chosen from a set of limited modalities (or combinations of them): very unreliable, slightly unreliable, neutral, slightly reliable, very reliable and unknown. A set of reliability values is associated to each modality. Modalities are available online by criterion: Ontology -> IC2ACV -> View judgements on criteria.

Each value of a set stands for a different combination of source/means to collect data/statistical procedures. Typically, a reliability value for a paper ranges between 1 (lowest reliability value) to 5 or 7 (highest reliability value).

To a document are then associated several sets of reliability values (one for each criterion). These sets are merged together in way to deal with conflicting evidences (i.e. assessment of high reliability for an aspect but of low reliability for another) and a formula is then used to extract the lowest and highest reliability expectations for a given paper.