The Review Algebra • review

library(review)

Motivation

Many data workflows require human review before publication. Official statistics, digital humanities, archives, machine learning, and data engineering all rely on iterative assessment of candidate values before they become part of a released dataset.

The goal of review is to make semantic review explicit and reproducible. Rather than overwriting existing values, each review round creates a new version of one or more reviewable claims while preserving the previous versions and their provenance. This review history can be inspected, reproduced, and extended with additional review rounds.

Creating reviewable claims

The review package provides four verbs for reproducible semantic review:

claims_df() creates reviewable claims.
review() allocates a review round and labels the intended review task.
document() records how the review was carried out.
approve() approves the reviewed values as the current candidate claims while preserving the complete review history.

claims <- revisions(
  Orange,
  scope_var = "age",
  subject_var = "Tree"
)

head(claims, n = 6)
#>   claim_id  age Tree circumference_candidate
#> 1        1  118    1                      30
#> 2        2  484    1                      58
#> 3        3  664    1                      87
#> 4        4 1004    1                     115
#> 5        5 1231    1                     120
#> 6        6 1372    1                     142

revisions() constructs a semantically enriched claims_df tibble that separates variables into three structural roles and one or more reviewable variables.

Identifier uniquely identifies each claim.
Scope defines the context in which the claim is made.
Subject identifies the entity being described.
Reviewable variables contain the values that may change during semantic review.

The structural variables remain stable throughout the review process, while reviewable variables may acquire successive reviewed versions.

Readers familiar with the tidy data principles will recognise this separation. Structural variables play a role analogous to identifiers, dimensions, and attributes in statistical data production, while the reviewable variables correspond to measured values that are iteratively reviewed and improved. The structural variables provide the context for interpretation and allow claims to be grouped, filtered, or compared, whereas the reviewable variables capture the semantic content that evolves during the review process.

names(claims)
#> [1] "claim_id"                "age"                    
#> [3] "Tree"                    "circumference_candidate"

Notice that circumference has become circumference_candidate. Candidate values are the current working version of each reviewable variable and form the starting point for the first review round.

First review round

A review round allocates a new review column for a reviewable variable and records a label describing the intended review task. The label serves as an instruction before the review is carried out and later identifies the completed review activity.

reviewed <- claims |>
  review(
    "circumference",
    review_id = "remeasure",
    label = "Remeasure the circumference of each tree."
  )

attr(reviewed, "review_label")
#>                                   remeasure 
#> "Remeasure the circumference of each tree."

The review algebra is independent of the review interface. Reviewers may use whatever environment best suits the task, provided that the reviewed values are written back into the allocated review columns.

The package deliberately does not prescribe how the review is carried out. For example, the review may be

a dplyr::mutate() pipeline,
values imported from a reviewed CSV file,
an interactive R console session,
a Shiny application,
an Excel or LibreOffice spreadsheet,
an OpenRefine workflow.

For illustration we edit one value directly.

reviewed$circumference_review_1[1] <- 31

Once the review values have been entered, the review round can be documented.

reviewed <- reviewed |>
  document(
    revision = "circumference_remeasure",
    agent = person("Jane", "Doe", role = "rev"),
    used = "doi:10.5281/zenodo.1234567",
    comment = "Verified against the laboratory notebook."
  )

attr(reviewed, "prov_comment")
#>                     circumference_remeasure 
#> "Verified against the laboratory notebook."

document() records how the review was performed, who performed it, which evidence or resources informed the review, and optional reviewer comments documenting how the review task was interpreted or why particular decisions were made.

Provenance is stored separately from the reviewed values, allowing the same review data to be interpreted or exported in different provenance models.

Readers familiar with FAIR data principles, reproducible research, or statistical and digital heritage workflows may recognise a common problem: as data are cleaned, harmonised, and reviewed, an increasing share of the knowledge about why particular decisions were made remains only in the analyst’s or curator’s head. The resulting datasets may be reusable, but the review process itself is often difficult to inspect, reproduce, or audit.

The review algebra aims to record as much of this review history as possible while requiring as little additional documentation effort as possible. By separating reviewed values from the provenance of the review, it records which agents (people, software, or AI models) performed which activities, and what entities (datasets, publications, files, or other resources) informed those decisions.

review() and document() deliberately separate the description of a review task from the documentation of its execution.

A review label describes what reviewers are asked to do. It therefore serves as a task description before the review and as the label of the completed review activity afterwards.
A reviewer comment explains how the task was interpreted or why particular review decisions were made.

This lightweight provenance layer follows the concepts of the PROV data model without requiring users to work directly with RDF or ontologies. It can later be serialised to PROV-O or aligned with standards such as SDMX, DataCite, or archival provenance models. The objective is to improve the reviewability, auditability, reproducibility, and ultimately the trustworthiness and reusability of reviewed data while remaining compatible with ordinary R data frames.

attr(reviewed, "prov_activity")
#>               candidate circumference_remeasure 
#>                "create"                      NA

attr(reviewed, "prov_agent")
#>               candidate circumference_remeasure 
#>                      NA        "Jane Doe [rev]"

attr(reviewed, "prov_used")
#>                    candidate      circumference_remeasure 
#>                           NA "doi:10.5281/zenodo.1234567"

Complete Review Workflow

claims <- claims_df(
  Orange,
  scope_var = "age",
  subject_var = "Tree"
) |>
  review("circumference",
    review_id = "remeasure"
  )

claims$circumference_remeasure[1] <- 31

claims <- claims |>
  document(
    revision = "circumference_remeasure",
    agent = person("Jane", "Doe", role = "rev"),
    used = "doi:10.5281/zenodo.1234567"
  ) |>
  review("circumference",
    review_id = "OpenRefine"
  )

claims$circumference_OpenRefine[1] <- 32
claims <- claims |>
  document(
    revision = "circumference_OpenRefine",
    agent = utils::person("Joe", "Doe", role = "ctb"),
    used = "doi:10.2908/NAMA_10_GDP"
  ) |>
  approve()

Adding subsequent rounds of review

Each review round is allocated from the current candidate values. After approve(), those candidate values become the approved result of the previous review, allowing further review rounds to continue from the updated claims.

Consider this second-round review:

reviewed <- reviewed |>
  review("circumference")

Again, the review itself may take place using any suitable workflow.

reviewed$circumference_review_2[1] <- 32

Each review round has its own provenance. Different reviewers, software, or evidence can therefore contribute to successive stages of the review.

reviewed <- reviewed |>
  document(
    revision = "circumference_review_2",
    agent = "OpenRefine 3.10",
    used = "doi:10.2908/NAMA_10_GDP"
  )

The review history is therefore

circumference_candidate
            ↓
circumference_review_1
            ↓
circumference_review_2

Finalising a review

Once the review is complete, the current reviewed values become the new candidate values. The review history is retained, allowing future review rounds to continue from the released version while preserving the complete review trail.

approved <- reviewed |>
  approve(agent = utils::person("Joe", "Doe", role = "rev"))

The review history is preserved, while future review rounds begin from the finalised candidate values.

names(approved)
#> [1] "claim_id"                "age"                    
#> [3] "Tree"                    "circumference_candidate"
#> [5] "circumference_2"         "circumference_remeasure"
#> [7] "circumference_review_1"  "circumference_review_2" 
#> [9] "circumference_approved"

head(approved, 6)
#>   claim_id  age Tree circumference_candidate circumference_2
#> 1        1  118    1                      30              30
#> 2        2  484    1                      58              58
#> 3        3  664    1                      87              87
#> 4        4 1004    1                     115             115
#> 5        5 1231    1                     120             120
#> 6        6 1372    1                     142             142
#>   circumference_remeasure circumference_review_1 circumference_review_2
#> 1                      30                     31                     32
#> 2                      58                     31                     32
#> 3                      87                     31                     32
#> 4                     115                     31                     32
#> 5                     120                     31                     32
#> 6                     142                     31                     32
#>   circumference_approved
#> 1                     30
#> 2                     58
#> 3                     87
#> 4                    115
#> 5                    120
#> 6                    142

The complete review algebra therefore consists of four operations: create reviewable claims, allocate review rounds, document (explain) each review, and promote the current review into the next candidate version.

Dual structure and complete workflow

The review algebra can be viewed as a dual extension of tidy data.

An ordinary tidy data frame represents observations in rows and variables in columns. The tidyverse provides an expressive algebra for transforming these observations through operations such as mutate(), filter(), select(), join(), and pivot_*().

The review algebra introduces a second algebra that is structurally aligned with the tidy data algebra. The upper layer contains successive semantic states of the same claim vectors (reviewable variables), while the lower layer records provenance describing with statements about the transitions between those states how the transition took place.

                 REVIEW MATRIX

┌──────────────────────────────────────────────────────┐
│ Upper layer: review states                           │
│                                                      │
│ candidate → review 1 → review 2 → ... → reviewed     │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ Lower layer: provenance                              │
│                                                      │
│ activity • agent • resources • comments              │
└──────────────────────────────────────────────────────┘

The two layers share the same structure. Every review column in the upper layer has a corresponding provenance description in the lower layer. This shared structure makes it possible to derive provenance automatically at different levels of detail.

A single provenance statement may describe an entire review column:

“The bibliographic information was verified by Librarian 1.”

or, if required, the provenance can be refined to individual claims:

“Rows 001, 004 and 005 were corrected after consulting manuscript B.”

Because both layers share the same identifiers and ordering, column-level provenance and cell-level provenance are different resolutions of the same underlying model.

Review sequencing

A review column represents a semantic mutation of a reviewable variable.

Unlike ordinary mutate() operations, review mutations are ordered because their sequence may affect the final result. For example, a two-stage imputation or a sequence of authority-file reconciliations is generally not commutative.

For this reason, every review column belongs to a strictly ordered review chain.

A complete workflow may consist of several reviews or refinements.

               review algebra

bibliography_candidate
          │
          ▼
bibliography_author_year
          │
          ▼
bibliography_publisher
          │
          ▼
bibliography_reviewed


              provenance algebra

author_year
    activity
    agent
    used
    ...

publisher
    activity
    agent
    used

The column names provide meaningful identifiers for the review stages, whereas the package records their ordering internally. This allows review stages to be named according to their intended purpose while preserving the temporal sequence of the review.

Each review stage therefore has two complementary descriptions.

A label records the intended review activity before it is carried out. An explanation records what actually happened during the review.

The label expresses the planned activity:

“Verify authors and publication years.”

whereas the explanation documents its execution in an optional comment:

“Verified by Librarian 1 using the National Bibliography. Three publication years were corrected.”

This distinction mirrors the distinction in provenance models between an activity specification and the execution of that activity.

Relation to Supply and Use and Input-Output Tables

Readers familiar with Supply and Use Tables (SUTs) or Symmetric Input-Output Tables (SIOTs) may recognise a similar design pattern.

Supply and Use Tables (SUTs) and Symmetric Input-Output Tables (SIOTs) consist of several matrices that describe different aspects of the same economy while sharing a common ordering of industries and products. Because these matrices are structurally aligned, aggregates such as gross domestic product (GDP) become deterministic functions of the detailed accounts (f.e., total consumption, total import.) Changes in one part of the system therefore propagate consistently through the others.

The review algebra applies the same principle to semantic review. The upper matrix records successive semantic states of a tidy data frame, while the lower matrix records the provenance associated with those states. Because the two matrices share the same structure, provenance can be generated at the level of entire review stages or, when necessary, for individual reviewed claims.

Unlike many review systems, review does not store separate audit logs or provenance graphs as primary data structures. Instead, it stores successive semantic states of reviewable variables together with lightweight provenance describing each review stage. Audit reports, provenance graphs, release notes, and cell-level revision histories are deterministic projections of this representation rather than independently maintained artefacts.

Garamantas example

The review algebra is independent of the application domain. The same workflow can be applied to statistical observations, archival metadata, cultural heritage collections, or machine-generated annotations.

garamantas <- data.frame(
  resources = c(
    "https://garamantas.lv/en/file/475833",
    "https://garamantas.lv/en/file/471397",
    "https://garamantas.lv/en/file/475825"
  ),
  instance_of = rep("photograph", 3),
  depicts = rep("building", 3)
)

garamantas_claims <- revisions(
  garamantas,
  scope_var = "instance_of",
  subject_var = "resources"
) |>
  review(c("depicts"))

garamantas_claims$depicts_1[2] <- "group of people"

Although the domain differs, the review algebra remains unchanged:

revisions()
   ↓
review()
   ↓
document()
   ↓
approve()
   ↓
next_revisions()

garamantas_reviewed <- garamantas_claims |>
  document(
    revision = "depicts_1",
    agent = person("Alice", "Curator", role = "rev"),
    used = "doi:example001"
  ) |>
  approve()

print(garamantas_reviewed)
#>   claim_id instance_of                            resources depicts_candidate
#> 1        1  photograph https://garamantas.lv/en/file/475833          building
#> 2        2  photograph https://garamantas.lv/en/file/471397          building
#> 3        3  photograph https://garamantas.lv/en/file/475825          building
#>         depicts_1 depicts_approved
#> 1        building         building
#> 2 group of people  group of people
#> 3        building         building

attributes(garamantas_reviewed)
#> $names
#> [1] "claim_id"          "instance_of"       "resources"        
#> [4] "depicts_candidate" "depicts_1"         "depicts_approved" 
#> 
#> $row.names
#> [1] 1 2 3
#> 
#> $id
#> [1] "claim_id"
#> 
#> $scope
#> [1] "instance_of"
#> 
#> $subject
#> [1] "resources"
#> 
#> $reviewable
#> [1] "depicts"
#> 
#> $prov_id
#> [1] "candidate" "depicts_1"
#> 
#> $prov_activity
#> candidate depicts_1 
#>  "create"        NA 
#> 
#> $prov_agent
#>             candidate             depicts_1 
#>                    NA "Alice Curator [rev]" 
#> 
#> $prov_used
#>        candidate        depicts_1 
#>               NA "doi:example001" 
#> 
#> $prov_comment
#> depicts_1 
#>        NA 
#> 
#> $review_label
#>  1 
#> NA 
#> 
#> $review_sequence
#> candidate         1 
#>         0         1 
#> 
#> $class
#> [1] "reviewed_df" "claims_df"   "data.frame" 
#> 
#> $approval_activity
#> [1] "approval"
#> 
#> $approval_agent
#> [1] NA

Conclusion

The review package provides a human-centric semantic review system for data that can be normalised into tidy tabular form. Whether observations originate from spreadsheets, relational databases, file systems, RDF graphs, or other structured sources, and represent statistical, digital humanities, or natural science observations, the review algebra remains unchanged once they have been represented as reviewable claims.

Observation layer
    │
    ├── data.frame
    ├── CSV
    ├── RDF
    ├── File system
    └── Database
            │
            ▼
      revisions()
            │
            ▼
      Semantic stabilisation
      review()
      document()
      approve()
            │
            ▼
     Stable semantic object
            │
    ├── RDF
    ├── Frictionless
    ├── Data frame
    └── SQL

The package does not attempt to replace existing data engineering,
transformation, or knowledge representation tools. Instead, it adds a lightweight
semantic review layer that can be inserted into existing tidyverse workflows.
Its aim is to make semantic review reproducible by recording review states,
review activities, and their provenance while remaining compatible with ordinary
data-frame operations.

The central design principle is that review should be independent of the
original data source. Any resource that can be normalised into reviewable tidy
claims can be reviewed using the same workflow. This makes the review algebra
equally applicable to conventional tabular data, flattened RDF graphs,
filesystem observations, and other structured resources, while leaving more
complex graph structures or unstructured documents to specialised review
approaches.

The package provides a vectorised review workspace for creating, reviewing,
documenting, and finalising semantic claims without prescribing how reviews are
performed. Review activities may take place entirely within R or be delegated to
external tools such as spreadsheets, databases, or interactive review
interfaces before the reviewed claims are returned to the workflow.

Although review is not an ontology or knowledge graph toolkit, its design is
informed by provenance, policy, and statistical quality-control models,
allowing review histories to be audited, serialised, and exchanged in a
reproducible form. The accompanying vignette From Review Algebra to Provenance Modelling describes how review activities can be represented using lightweight
provenance metadata.

Once a review has been completed, the resulting semantic claims can be
transformed into the representation most appropriate for publication,
preservation, or exchange. The companion dataset package package provides an
R-native representation for describing and serialising released datasets,
supporting reproducible and interoperable publication using existing metadata
standards.