OBSERVER/ Format

An Open Knowledge Standard for AI Agents and LLMs

What is an OBSERVER Folder?

The OBSERVER Format is a variant of the Open Knowledge Format (OKF), designed to give quantitative analysis from any field a more efficient machine-readable structure.

The OBSERVER Format provides white-box transparency of quantitative analysis to increase the efficiency and accuracy of black-box AI systems.

An OBSERVER Folder is a hierarchical reporting structure that immediately answers the question “What do I need to know?” and provides transparent tools of verification, so that every reader, human and machine, can place the maximum warranted trust in the information the analysis provides.

What is the OBSERVER Format Structure?

One analysis, one folder, read from the top down.

OBSERVER/                            # The folder. One analysis.
  1.0_INDEX.md                       # The statement of purpose. The folder map. The metric definitions.
  1.1_LOG.md                         # The update log. Relevant to recurring and one-off reports alike.
  2.0_VALENCE.md                     # The highest-value information.
  3.0_PERCEPTION.md                  # All computed metrics.
  4.0_RAW_DATA.<ext>                 # The raw data the computations ran on (.csv, .json, .parquet, ...).
  README.md / AGENTS.md / CLAUDE.md  # Identical entry files that point to 1.0_INDEX.md.
  references/                        # Optional. Supporting code and documents.

How Does OBSERVER Enable Efficient Information Return & Increased Transparency of AI Agent And LLM Outputs?

OBSERVER/
│
├── README.md / AGENTS.md / CLAUDE.md  Entry files: a short map that points to 1.0_INDEX.md.
│
├── 1.0_INDEX.md                       What this Observer measures, and the rules it uses.
│
├── 1.1_LOG.md                         When it ran and what changed.
│
│   [Efficient perception of high-value information, with citations
│    that can be confirmed within the hierarchy]
├── 2.0_VALENCE.md                     'trust' and 'verify' fields on every object
│
│   [Optional white-box depth 1: 'trust' formulas checked for
│    consistency with 1.0_INDEX.md; optionally confirmed against the
│    cited source or outside references]
├── 3.0_PERCEPTION.md                  'trust' and 'verify' fields on every object
│
│   [Optional white-box depth 2: 'verify' inputs confirmed to exist;
│    optionally, the perception computation replicated]
└── 4.0_RAW_DATA.<ext>                 Rows and columns that support or refute any 'verify' map

Each layer compresses the one below it. A reader descends only as far as the task requires.

The Structural White Box-ing of Quantitative Analysis for AI.

Any time information is assigned high valence, or importance, it must be accompanied by methods of verification through which that trust can be confirmed. The trust fields that accompany every metric value in an OBSERVER Folder contain the formulas used in each calculation, written in Python for easy replication. The verify fields, which also accompany every metric value, list the rows and columns of the raw data that were used to reach the reported value. verify fields may also note data gaps or other limitations in the raw data, along with any methods used to accommodate them. Where proprietary methods of analysis keep computational formulas from public view, that opacity must be stated explicitly, to give the appropriate context to the information the OBSERVER assigns high valence.

Every value names the formula that produced it and the rows it was computed from. Where a formula is proprietary, the opacity is declared, not hidden.

OBSERVER is an Open Standard.

The OBSERVER Format was originally developed by Ian Carleton and is released as an open standard. The standard welcomes further development from stakeholders at every level.

Start Building

Specification: The complete OBSERVER Format 0.1: folder structure, every field, conformance, and a worked example.
Quickstart: The Agent Skill, a ready-to-paste prompt, and a validator, to take an analysis from raw data to a conformant folder.

What Is The Intention of OBSERVER?

The OBSERVER Format is modeled on human perception, or observation, in certain scientific fields. We exist in a reality, we computationally reduce that reality into inputs that we more easily understand, and we compare those inputs to determine what’s probably important and what we can probably ignore. True observation requires a method of prioritizing what we are perceiving relative to the other things we are perceiving, and relative to what we have perceived in the past. While we engage in perception, the amount of information, all the numerous pieces of reality being perceived, fluctuates and influences the range of observations available to us. Sometimes we’re conscious of some of those changes, sometimes we’re not, and sometimes those changes are in things we have no way of being consciously aware of, because we don’t have a way to measure them.

A (Really Simple) Model of Computational Observation

We exist in reality (I know, this part is pretty obvious), and that raw reality contains more information than we can process, or compute.

Because we can’t fundamentally experience every dimension of raw reality, we can only perform computation on the parts of reality that we can experience. This is known as being a computationally constrained observer. We reduce the dimensions of raw reality into inputs for further computational processing.

The further computational processing leads to outputs which are further abstracted away from raw reality but, if we’ve done our jobs effectively, contain the maximum amount of information that can be derived from our computationally constrained experience of reality. This is generally known as perception.

Even our computationally reduced perception of reality is still a lot to sort through. We further reduce the dimensions of our experience by paying attention to specific dimensions: the specific information which stands out when compared to everything else we are perceiving in the current moment, everything we have perceived in the past, or both. The best model for this computational method of state asymmetry constructing perceived reality comes from Weber’s law in psychology, which states “the minimum increase of stimulus which will produce a perceptible increase in sensation is proportional to the pre-existent stimulus.” This maxim indicates that perception is based on change, on relative comparison. It’s fairly common to refer to this relative comparison, resulting in a high-stimulus to low-stimulus range, as representing the valence of the information which produced the stimulus.

Weber–Fechner: each just-noticeable step in sensation requires a stimulus increase proportional to the stimulus already present.

How This Model Applies to AI Agents and LLMs Researching On Behalf Of Human Users.

OBSERVER Folders model the quantitative analysis of data sets on the same hierarchy of computation that we experience. The complete data set is raw reality. The total of calculations performed on that raw reality is a computational constraint. The perception of that raw reality is further abstracted by formulaic computations which, again, if we’ve done our jobs effectively, contain the maximum amount of information that can be derived from that computationally constrained experience of the raw reality of the data set. The perception of the data, the outputs of the computational perception calculated from the data set, are then selected for what stands out when relatively compared to the rest of the computed perception, resulting in the high-valence subset of perception.

Perception and an OBSERVER Folder reduce information through the same stages. Only the folder can be read back down to its source.

Psychology uses Weber–Fechner to compute this valence, but we can use any number of computational methods for relative observation, and because of this broader list of options it’s important to be transparent about the methods we choose. In the specific case of quantitative analysis, we can also explicitly state the computational methods we used to computationally constrain the raw reality of the data and generate the perception from which valent information was selected. Furthermore, we can expose the raw reality of the underlying data set to allow others to ensure that our computational methods of perception have accurately compressed that reality in the manner we intended.

These processes, captured in the OBSERVER Folder’s hierarchical structure, allow others [AI agents / LLMs / human operators] to perceive the most important things to perceive from any quantitative analysis in a way that focuses attention [and token spend] in the most efficient manner. After that high-valence information has been perceived, readers can choose to verify those perceptions to whatever depth is most relevant to the goal that caused them to seek out the information in the first place: to whatever depth answers the question “What do I need to know?” in a manner that verifies the valent perception as accurate, establishes trust in the communicated perception, and lets the reader confidently integrate it into their own perceived reality.