Measurement
How to measure judgment, trust, and context — the foundations of evaluation.
The Core Argument
What gets measured gets managed. But judgment, trust, and context are difficult to measure. Developing measurement frameworks is essential.
We cannot improve what we cannot measure. Without measurement, we do not know whether trust infrastructure is working — or how to improve it.
But measurement is difficult. Judgment, trust, and context are complex, qualitative, and context-dependent. They resist simple metrics.
Developing measurement frameworks is a research priority.
Measuring Judgment
How to evaluate the quality of judgment.
Outcome-based measurement
Measure the outcomes of decisions. Did the decision achieve its intended goals? This is the most direct measure of judgment.
Process-based measurement
Measure the quality of the decision-making process. Was it systematic? Was it based on evidence? Was it well-documented?
Peer evaluation
Have peers evaluate decisions and judgment. This captures qualitative dimensions that metrics may miss.
Calibration
Calibrate judgment against known benchmarks. How does judgment compare to expert opinion, historical data, or AI predictions?
The Acacia perspective:
No single measure is sufficient. Measuring judgment requires a multi-method approach — combining outcome, process, peer, and calibration measures.
Measuring Trust
How to evaluate trustworthiness.
Survey-based measurement
Ask people about their trust. Surveys can capture perceptions of trustworthiness, confidence, and reliability.
Behavioural measurement
Observe behaviour. Do people rely on the system? Do they follow through on commitments? Behaviour reveals trust.
Institutional measurement
Evaluate institutions. Are they transparent? Are they accountable? Are they legitimate? Institutional quality is a proxy for trust.
Network measurement
Analyse networks. How connected are trust relationships? Network structure can reveal patterns of trust.
Key insight:
Trust is multi-dimensional. Measuring it requires combining different methods — surveys, behaviour, institutions, and networks.
Measuring Context
How to evaluate the quality of context.
Completeness
Is the context complete? Does it capture the situation, history, constraints, and relationships? Completeness is a measure of context quality.
Transferability
Can the context be transferred to others? Is it documented, structured, and accessible? Transferability is a measure of context usability.
Relevance
Is the context relevant? Does it address the key factors that matter for the decision? Relevance is a measure of context value.
Accountability
Is the context accountable? Is it clear who documented it and why? Accountability is a measure of context trustworthiness.
The Acacia perspective:
ContextOS is designed to make context measurable. It provides structure, documentation, and accountability — enabling the measurement of context quality.
Reading path: