Quickstart

You are producing an OBSERVER Folder: one quantitative analysis arranged so that
the highest-value results come first and every published value carries the
formula that produced it and the raw rows it was computed from.

THE ANALYSIS
<<ANALYSIS>>

THE RAW DATA
<<DATA>>

THE METRICS
<<METRICS>>

WHAT MAKES A VALUE IMPORTANT
<<IMPORTANCE>>

THE ONE RULE
No value is published unless a reader can recompute it from the folder alone.
Everything below serves that rule. If you cannot make a value reproducible, say
so and leave it out rather than publishing it unverifiable.

WHAT YOU PRODUCE

  OBSERVER/
    1.0_INDEX.md         the contract: purpose, metrics, rules of elevation, data dictionary
    1.1_LOG.md           report history; every contract change, always
    2.0_VALENCE.md       the results the rules elevated, ranked
    3.0_PERCEPTION.md    every computed value
    4.0_RAW_DATA.csv     the inputs (.json/.parquet, or a 4.0_RAW_DATA/ directory);
                         absent when the data is referenced or withheld
    README.md, AGENTS.md byte-identical entry files
    references/          optional: shared formula code, source documents

Above roughly a dozen objects, write a build script that generates every file
from the computed objects rather than typing them. Make it deterministic: every
sort has an explicit key and every value is derived from the inputs, so a
rebuild from unchanged inputs produces identical bytes in everything except
generated.at. That field follows OKF: it records when the build actually ran.
Capture it once per build and write the same value into every file.

PROCEDURE

1. Fix the raw table. Decide its shape, then its KEY COLUMNS: the smallest set
   that uniquely identifies a row. Every row reference in the folder uses them.
   Missing observations stay missing — never forward-fill, back-fill or
   interpolate — and every window calculation must be gap-aware.
2. Give each metric a clear name, a description that states its units, and a
   one-sentence utility naming the decision it informs. Every object reports its
   unit in the unit field, spelled out (percent, percentage points, USD). Putting
   the unit in the name as well (_pct, _pp, _idx100) is a helpful convention,
   not a requirement. If you cannot write the utility, drop
   the metric.
3. Write each formula as a self-contained Python function named after the metric,
   with arguments named after the raw columns it reads. No I/O, no network, no
   undeclared randomness. Shared helpers go in one file inside references/,
   referenced by trust.resource and copied verbatim from the code that ran.
4. Choose one to three relative observations: comparisons that say how unusual a
   value is — against its peers now (cross_sectional), against its own past
   (temporal), or against an external reference (benchmark). Write each as Python
   and state the population precisely enough to rebuild. For a comparison against
   history, the population states the window length, how gaps within it are
   handled, and the minimum number of values required, and 4.0_RAW_DATA keeps
   every row the window reaches, not only the rows verify names.
5. Reduce them to one relative_value used for ranking and elevation. Usually take
   magnitudes, so a large fall counts as much as a large rise; say which you did.
   When it cannot be computed, it is null — never zero.
6. Write elevation rules that best fit the individual analysis from which the OBSERVER is constructed.
7. How many objects it elevates is your choice: the format sets no floor and no ceiling, and an
   empty 2.0_VALENCE.md is valid and meaningful. A cap (max_objects) is
   optional; if you declare one, state it in the 2.0_VALENCE.md frontmatter
   with max_objects and elevated_count (see below).
8. Emit one object per computed value into 3.0_PERCEPTION.md, as a single list
   ordered by relative_value descending across all metrics, nulls last.
9. Copy the objects the rule(s) selects into 2.0_VALENCE.md, adding rank and
   perception_ref. You may open that file with one plain sentence per object,
   each linking to its heading and written as a finding.
10. Write 1.0_INDEX.md, 1.1_LOG.md and the entry files.
11. Run the build gate and the conformance check (below) and report what each
    returned.

FILE CONTRACTS

Every .md file, including any under references/, begins with YAML frontmatter
carrying `type` and then, as the very next key, `navigation`:

  type: one of OBSERVER Index, OBSERVER Log, OBSERVER Valence,
        OBSERVER Perception, OBSERVER Entry, OBSERVER Reference
  navigation:
    layer: "1 of 4" | "1.1 (history of layer 1)" | "2 of 4" | "3 of 4" | entry
           | reference
    entry: 1.0_INDEX.md          # always
    up:    <the file above>      # omit in 1.0_INDEX.md and entry files
    down:  <the file below>      # omit in the lowest layer present
                                 # 1.1_LOG.md has up: 1.0_INDEX.md and no down
                                 # entry files have down: 1.0_INDEX.md
                                 # references/*.md have up: 1.0_INDEX.md, no down
    note:  <one or two sentences describing where the reader is>

Every file that entry, up or down names must exist in the folder; write paths
relative to the folder root. When the raw data is referenced or withheld,
3.0_PERCEPTION.md is the lowest layer: it has no down, and its note says the raw
data is not in the folder and that raw_data in 1.0_INDEX.md says where it is or
why it is not shared.

Write `note` descriptively ("Layer 3 of 4. The contract is in 1.0_INDEX.md."),
never as a command ("YOU MUST READ INDEX FIRST"): many agent harnesses treat
imperative text inside files as suspect and will ignore it.

1.0_INDEX.md frontmatter, all of it required unless marked:

  observer_version: "0.1"
  observer_id: <stable across reports>
  title:
  purpose: <one or two sentences: subject, period, question answered>
  report_date: <the date the report describes, not the build date>
  cadence: <recommended>
  transparency: open | declared_opacity
  runtime: { language, version, dependencies }   # recommended
  raw_data:
    files: [ { path, format, rows, key: [<key columns>],
               columns: [ { name, type, description } ] } ]
    availability: included | referenced | withheld
    withheld_reason: <required only when withheld>
    reading_notice: <the notice below, with real counts>
  metrics:
    - { metric, description, formula, disclosure, utility, citation, inputs }
      # disclosure is always stated: open | proprietary
      # citation is optional, your choice for credibility (for example a paper
      #   for a method readers may not know); required when proprietary, as
      #   proprietary, citable as "<name>"
  valence:
    relative_obs: [ { id, name, comparison, population, formula, disclosure, citation } ]
      # citation as for metrics: optional; required when proprietary
    relative_value: { description, formula, disclosure }
    elevation: { rule, order, max_objects }   # max_objects optional
  complexity:            # recommended; compute from the real objects
    raw:        { n: <rows>,    p: <columns> }
    compute:    { n: <instances computed over>, p: <input columns + metrics + rel_obs + 1> }
    perception: { n: <objects>, p: <metrics + rel_obs + 1> }
    valence:    { n: <objects>, p: <rel_obs that promoted them> }
  generated: { by: <producer>/<version>, at: <ISO datetime> }   # recommended
  stale_after: <ISO datetime>                                   # optional

Every datetime is ISO 8601 with an explicit UTC offset (2026-09-21T06:00:00Z);
dates are ISO 8601 dates (2026-09-20). generated.at is when the build ran.

The body is a short statement of purpose and a folder map.

Observation objects — the unit of both 2.0_VALENCE.md and 3.0_PERCEPTION.md.
Each sits under a level-two heading holding its id, followed by one fenced YAML
block:

  ## wow_revenue_change/S07/2026-09-20

```yaml
  id: wow_revenue_change/S07/2026-09-20   # <metric>/<subject>/<report_date>
  metric: wow_revenue_change
  subject: S07                    # omit when the metric has one subject
  report_date: 2026-09-20
  value: -0.2648
  unit: fraction
  observations:
    - { rel_ob: rel_ob_1, value: -3.12, note: <what it compares against> }
    - { rel_ob: rel_ob_2, value: -2.87, note: <what it compares against> }
  relative_value: 3.12            # null when it cannot be computed
  elevated: true                  # in 3.0_PERCEPTION.md
  rank: 2                         # in 2.0_VALENCE.md
  perception_ref: wow_revenue_change/S07/2026-09-20   # in 2.0_VALENCE.md
  trust:
    disclosure: open
    formula: |
      def wow_revenue_change(revenue_t, revenue_t_minus_1):
          return (revenue_t - revenue_t_minus_1) / revenue_t_minus_1
    resource: references/metrics.py     # optional
  verify:
    source: 4.0_RAW_DATA.csv
    rows: { store_id: S07, week_ending: [2026-09-13, 2026-09-20] }
    columns: [revenue]
    args:
      revenue_t:         { column: revenue, rows: { week_ending: 2026-09-20 } }
      revenue_t_minus_1: { column: revenue, rows: { week_ending: 2026-09-13 } }
    notes: <every missing, imputed, excluded or corrected input, and how it was handled>
```

trust.formula must match that metric's formula in 1.0_INDEX.md exactly, as text
(compared once the YAML is read, so indentation within the file may differ).
verify.rows selects on the key columns only — a value, a list, or {from, to} —
never by position. verify.columns lists every column the formula read.
verify.args maps each formula argument to a column and, optionally, a source
file and a narrower rows selector. source defaults to verify.source; name
another declared file when the argument comes from a joined table. Shared key
columns the entry omits are inherited from verify.rows; keys its own file does
not have do not apply. A selector that matches one row passes a value; several
rows pass a list ordered by the key. verify.columns lists every column read,
from every file. verify.args is required, naming exactly the formula's
arguments, whenever an argument is not named after a column, verify.rows
selects more than one row, or the formula reads more than one file; otherwise
omit it.

2.0_VALENCE.md frontmatter adds: observer_id, report_date, elevation_rule
(echoing the contract), object_count (0 is valid and meaningful), generated, and,
whenever the contract declares elevation.max_objects, two more keys:
max_objects (echoing the contract) and elevated_count (the number of objects
in 3.0_PERCEPTION.md with elevated: true). object_count then equals the smaller
of the two, and valence keeps the highest-ranked elevated objects; the list was
truncated exactly when elevated_count exceeds object_count. Its
objects must match their perception objects exactly in value, observations,
relative_value, trust and verify.

3.0_PERCEPTION.md frontmatter adds: observer_id, report_date, object_count,
generated. Its navigation.note must also contain the reading notice.

1.1_LOG.md is required even for a one-off report: date headings YYYY-MM-DD,
newest first, entries led by **Report**, **Data**, **Contract**, **Correction**
or **Initialization**. Every change to a formula, a relative observation, the
relative value or the elevation rule requires a **Contract** entry — changing how
importance is judged changes what readers are told is important, and can never be
silent.

README.md and AGENTS.md are byte-identical, carry type: OBSERVER Entry and a
navigation block (layer: entry, entry: 1.0_INDEX.md, down: 1.0_INDEX.md, no up),
state that they are identical copies, link to the OBSERVER Format specification,
and give the reading order and the reading notice. Write them from one string.

THE READING NOTICE, verbatim, with the real counts filled in. When the raw data
is included it must appear in three places: raw_data.reading_notice, the
navigation.note of 3.0_PERCEPTION.md, and the entry files. When it is referenced,
it appears only in raw_data.reading_notice, naming the URI; when it is withheld,
raw_data.withheld_reason takes its place, and the entry files' reading order
says the raw data is not in the folder and points to raw_data in 1.0_INDEX.md.

  4.0_RAW_DATA.csv: <rows> rows × <columns> columns. Trust verification with
  select columns and rows only. Full data set only intended for full reading when
  verification of values reported in 2.0_VALENCE.md and/or 3.0_PERCEPTION.md is
  explicitly requested by the user or programmatically mandated by the assigned
  workflow.

For a 4.0_RAW_DATA/ directory of several files, the first sentence describes the
directory, and each object's verify.source (e.g. 4.0_RAW_DATA/transactions.csv)
identifies the table it was computed from:

  4.0_RAW_DATA/: <files> files; each object's verify.source names the file it
  was computed from, and raw_data.files gives each file's rows and columns.
  [then the same three sentences as above]

HARD RULES

1. Never fill a gap. Not forward, not backward, not by interpolation. A frozen
   value divided by a moving base produces a plausible trend that is pure
   artifact.
2. Publish the formula you actually ran, not a description of it.
   Gate the build: if a published formula and the engine disagree, fail rather
   than publish. (The engine is whatever computed the values: your own code, or
   the code I attached.) Before writing the folder, evaluate every published
   formula against the raw table and require it to match the value you
   computed, to a stated tolerance, across the whole published range.
3. Declare rebasing anchors as constants inside the formula. "The first period
   available" silently rewrites history whenever the window grows.
4. Reference rows by key columns and objects by <metric>/<subject>/<report_date>.
   Never by position: files get re-sorted.
5. Explain every gap where it bites, in the `verify.notes` of the affected
   objects: those whose inputs actually lack the data, including for as long as
   a window reaches back over it, never in a single global remark.
6. Freeze any model you publish from: fit on a window ending before the first
   published period, keep the training boundary clear of the forecast horizon,
   and write the fitted coefficients into the published formula as constants. A
   refit is a new, announced version with a **Contract** log entry, never an
   edit.
7. State sign and direction conventions explicitly in each metric's description
   ("positive = stronger dollar", "positive = wages winning").
8. Undeclared opacity is the only thing forbidden outright. A proprietary method
   is published as disclosure: proprietary with a citable statement, and makes
   the Observer's transparency declared_opacity. So does withheld raw data.
9. relative_value is null, never zero, when a comparison cannot be computed.
10. Determinism: same inputs, same bytes, apart from generated.at, which records
    when the build ran.

BEFORE YOU FINISH

Run both checks and report what each returned:

  Build gate — are the published formulas the ones that ran? Evaluate every
  published trust.formula on the inputs its verify field names and require it to
  return the value you computed, to a stated tolerance, across the whole
  published range. If any disagrees, fix it rather than publish (hard rule 2).

  Conformance — is the folder well formed? Required files present with parseable
  frontmatter carrying type and navigation; every REQUIRED field present; every
  object with an id unique within its file, trust and verify; every metric and rel_ob referenced
  by an object declared in 1.0_INDEX.md; 2.0_VALENCE.md a subset of
  3.0_PERCEPTION.md with matching fields and ranks 1..n; when max_objects is
  declared, max_objects and elevated_count in the 2.0_VALENCE.md frontmatter
  and object_count equal to the smaller of the two; the reading notice in
  all three places; entry files byte-identical; transparency matching what is
  actually disclosed; contract changes logged.

These answer different questions: a folder can be perfectly conformant and
completely wrong, which is what the build gate exists to prevent.

Replicating the folder end to end is the consumer's to do, to whatever depth
their task needs; the Replicate variant below is how. Your obligation is that it
can be done from the folder alone: every open formula in trust exactly as it
ran, every input named in verify, and every history a relative observation
reaches back over retained in 4.0_RAW_DATA and described by its population. You
do not need to write or ship a replication script.

Then state plainly: how many objects were computed, how many were elevated, what
both checks returned, and anything you could not verify. An Observer that
overstates its own verifiability is worse than one that admits a limit.

ASK ME BEFORE PROCEEDING IF

- the key columns do not uniquely identify a row in the data I gave you;
- a metric's formula cannot be reconstructed from what I gave you, so publishing
  it would mean guessing at how it was computed;
- <<IMPORTANCE>> is unspecified — propose a rule, show me the objects it would
  elevate, and get agreement before writing the folder. A reasonable starting
  point for a recurring report is a z-score against each subject's own history,
  elevating |z| >= 2;
- the data cannot be shipped and you need to know whether to declare it
  referenced (with a checksum) or withheld (with a reason).

Otherwise proceed, state the assumptions you made, and produce the folder.

Here is an existing OBSERVER Folder and a new period of data.

<<FOLDER>>
<<NEW DATA>>

Produce the next report under the existing contract. Never change the contract
in 1.0_INDEX.md — a formula, a relative observation, the relative value, the
elevation rule — without my confirmation. If the new data seems to require a
change, stop before producing the report, tell me what you would change and why,
and wait. Once I confirm, make exactly the change I confirmed and record it in
1.1_LOG.md as a **Contract** entry saying what changed, why, and that I
confirmed it. That entry is not optional: changing how importance is judged
changes what readers are told is important.

Apply the new data to 4.0_RAW_DATA, then compare the raw data before and after
along every dimension and record the result in a **Data** entry: rows added and
rows removed (with counts, and the key values or ranges involved), columns added
and columns removed, and, for a 4.0_RAW_DATA/ directory, files added and files
removed. Say so explicitly when a dimension did not change, so the entry always
accounts for every direction. If a column the contract reads was removed, treat
it as a contract question and ask me, as above.

Update report_date everywhere it appears, recompute every object, re-elevate
under the existing rule, rewrite 2.0_VALENCE.md and 3.0_PERCEPTION.md, refresh
object_count, complexity, raw_data.files (rows and columns), the reading notice
counts and generated/stale_after, and add a **Report** entry to the log.

Then run the build gate and the conformance check and report what each returned,
plus anything that moved enough to be worth my attention.

Review this OBSERVER Folder against OBSERVER Format 0.1.

<<FOLDER>>

Report, in this order:

1. Conformance failures: missing files or REQUIRED fields, malformed frontmatter
   or navigation, objects without an id unique within their file or without trust
   and verify,
   metrics or rel_obs used but not declared, 2.0_VALENCE.md not a subset
   of 3.0_PERCEPTION.md, a declared max_objects without max_objects and
   elevated_count in the 2.0_VALENCE.md frontmatter, ranks out of order, the reading notice missing from any
   of its three places, entry files not byte-identical, transparency not matching
   what is disclosed, contract changes not logged.
2. Replication failures: values that do not reproduce when their own trust
   formula is run against the rows and columns their verify field names
   (through verify.args where present), verify.args missing where an argument
   is not a column or several rows are selected, and
   relative observations that do not reproduce from their declared population.
3. Substantive weaknesses that pass both checks: gaps handled without a note,
   filled or interpolated data, a rebasing anchor that moves, a model refitted on
   the data it is scored against, a relative_value of zero standing in for a
   value that could not be computed, an elevation rule a reader could not apply
   by hand, sign conventions left implicit, or 2.0_VALENCE.md opening sentences, where
   present, that restate YAML instead of stating findings.

For each finding give the file, the object id, and what to change. Where you
cannot verify something from the folder alone, say so — that inability is itself
a finding about the folder.

Replicate the values in this OBSERVER Folder.

<<FOLDER>>

SCOPE: <<SCOPE>>   (for example: the 2.0_VALENCE.md objects only; every object;
                    the objects for one subject)

Work from the folder alone: no code, data or assumptions from outside it.

1. Read 1.0_INDEX.md: runtime.dependencies, raw_data (files, key columns, column
   dictionary), metrics, and valence (relative observations, relative value,
   elevation rule).
2. For each object in scope whose trust.disclosure is open: define
   trust.formula exactly as written, loading any trust.resource from the folder.
   Select the rows verify.rows names from verify.source, using the key columns.
   Where verify.args is present, give each argument the column, file and rows
   its entry names (the file defaults to verify.source; shared keys an entry
   omits come from verify.rows; several rows arrive as a list ordered by the
   key). Where it is absent, give each argument the
   column of the same name from the single selected row. Compare the result
   with value at the published precision.
3. Recompute each relative observation from its declared population: peer
   values from 3.0_PERCEPTION.md; history by running the metric's formula over
   the earlier raw rows the population's window reaches, with its stated gap
   handling and minimum count.
4. Recompute relative_value, apply the elevation rule, and confirm each object's
   elevated flag, the contents and rank order of 2.0_VALENCE.md, and any
   max_objects cap.
5. For objects whose trust.disclosure is proprietary, replication is not
   possible by design: report the statement and any track_record instead.

Follow the raw data reading notice: read only the rows and columns that verify
fields and populations name, unless I have asked for the full data set.

Report, for each object in scope, one of: replicated; did not replicate (give
the published and the recomputed value); or could not be replicated (say what
the folder was missing). Anything you had to guess, such as which row fed which
argument or how a population's window should be read, is a finding about the
folder, not a success.