mareforma.open(path=None, *, ...)
Open the epistemic graph and return an EpistemicGraph.
Returns
EpistemicGraph
Raises
DatabaseError: if the database cannot be opened or the schema cannot migrateKeyNotFoundError: ifrequire_signed=Trueand no key is foundSigningError: ifrequire_rekor=Trueand the Rekor URL is unset or unreachable; or if the supplied Rekor log pubkey conflicts with the TOFU pin on.mareforma/rekor_log_pubkey.pemValueError: if bothrekor_log_pubkey_pemandrekor_log_pubkey_pathare supplied (mutually exclusive)
mareforma bootstrap once to generate an Ed25519 keypair at
~/.config/mareforma/key. After that, every assert_claim auto-signs.
TOFU pin behavior. When rekor_log_pubkey_pem (or rekor_log_pubkey_path) is supplied, the canonical DER bytes of the public key are persisted to .mareforma/rekor_log_pubkey.pem. Every subsequent mareforma.open() on the same project compares the supplied key against the pinned PEM and refuses silent rotation; you must delete the pin file to intentionally rotate. The first-pin write uses O_CREAT|O_EXCL, so two concurrent open() calls with different keys cannot race past existence checks and silently overwrite each other; the loser raises SigningError("...pinned to a different key by a concurrent ... call").
mareforma.schema()
Return the epistemic schema: valid values, defaults, and state transitions.
dict: stable across patch releases within a major version.
mareforma.restore(project_root, *, claims_toml=None, rekor_log_pubkey_pem=None, enforce_rekor_policy=False)
Rebuild a fresh graph.db from claims.toml for catastrophic-loss
recovery. Fresh-only: refuses to run if graph.db already has any
claims. Every signature is verified before any row is inserted;
fail-all-or-nothing.
Returns
dict with validators_restored, claims_restored, and
unsigned_in_signed_mode.
unsigned_in_signed_mode counts claims that came back carrying no signature in
a project that enrols a validator. Restore accepts them and warns; the read path
does not count them and discloses each one in lines_skipped. A non-zero value
means someone wrote to this project without its key. If those claims predate the
enrolment, the count is the size of what enrolling cost you. A claim that carries
a statement_cid but no signature_bundle is refused instead of counted here:
it was signed once and the signature is gone, which is tampering rather than a
keyless write.
Raises mareforma.db.RestoreError with .kind field:
EpistemicGraph
Returned bymareforma.open(). Do not instantiate directly.
assert_claim(text, *, classification, generated_by, supports, contradicts, source_name, idempotency_key, status, artifact_hash, evidence, seed, signer, predicate_payload, original_signature_bundle, grounding_sensor, observed_grounding, finding_record)
Assert a claim into the graph.
Returns
str: claim_id UUID
Raises
ValueError: iftextis empty,classificationis invalid,statusis invalid, or text exceeds 100k charsCycleDetectedError: ifsupports[]would create a cycle (A → ... → A)IdempotencyConflictError: if the sameidempotency_keyreplays with any divergent semantic field (text,classification,generated_by,supports,contradicts,source_name,artifact_hash,evidence,observed_grounding,original_signature_bundle)IllegalStateTransitionError: if a transition violates the state-machine triggersChainIntegrityError: if theprev_hashappend-only chain check failsDatabaseError: on SQLite write failure
supports[] signed by distinct keys, all are promoted to REPLICATED. The promotion fires only when at least one upstream is itself ESTABLISHED (the ESTABLISHED-upstream rule, strict by default, so replication-of-noise is not replication). When both peers supply artifact_hash, equal data collapses the pair to one line and does not promote on data alone; distinct data counts as independent; absent data never blocks.
Idempotency
idempotency_key is retry safety only.
Retry safety: same key plus matching semantic fields returns the existing claim_id with no INSERT. Use whenever a run may be interrupted and retried:
text, classification, generated_by, supports, contradicts, source_name, artifact_hash, evidence, observed_grounding, original_signature_bundle) raises IdempotencyConflictError listing every mismatch, rather than silently dropping the new content. Use a different key or reconcile the conflict.
Not a convergence mechanism: two agents reaching the same conclusion converge through the epistemic ladder, not by sharing a key. Both cite the same ESTABLISHED upstream in supports[] and sign with distinct keys, and REPLICATED fires. Collapsing two authors into one row would erase the second contribution, so mareforma refuses it.
query(text=None, *, min_support=None, classification=None, limit=20, include_unverified=False, include_invalidated=False, refutation_filter=None)
Query claims ordered by support level (descending) then recency (descending).
Returns
list[dict]: each dict contains:
claim_id, text, classification, support_level, idempotency_key,
validated_by, validated_at, status, source_name, generated_by,
supports_json, contradicts_json, comparison_summary, branch_id,
unresolved, signature_bundle, transparency_logged,
validation_signature, validator_keyid, asserter_keyid, artifact_hash,
prev_hash, ev_risk_of_bias, ev_inconsistency, ev_indirectness,
ev_imprecision, ev_pub_bias, evidence_json, statement_cid,
t_invalid, convergence_retry_needed, predicate_payload,
original_signature_bundle, observed_grounding, created_at,
updated_at. asserter_keyid is the signing key the independence axis
counts. observed_grounding is the stored grounding verdict (the queryable
denormalisation of the signed predicate’s grounding record).
convergence_retry_needed is set on a claim whose promotion check failed and
is waiting for refresh_convergence(). Plus two reputation projections
computed at query time:
validator_reputation: int: for ESTABLISHED rows, the count of ESTABLISHED claims signed by the same validator.0otherwise.generator_enrolled: bool:Trueiff the claim’s signing keyid is in the validators table.
single_trust_domain: bool and
trust_domain_root: str | None, disclosing whether all validators trace to
one root of trust. It is a disclosure, not a Sybil guard. Rows below
ESTABLISHED omit both.
Raises ValueError if min_support or classification is invalid.
REPLICATED / ESTABLISHED rows are re-verified on read: a row whose signature fails to verify is excluded from the result. Never raises.
min_support and the support_level result key are the legacy stored promotion ladder. They stay functional this release, but their REPLICATED / ESTABLISHED public labels are deprecated for v0.4.0. To read trust, use the derived axes graph.proposition_status(prop) returns: status (the answer, per content_id) and question_status (the question, per frame_id).
search(query, *, min_support=None, classification=None, limit=20, include_unverified=False, include_invalidated=False)
Full-text search over claim text using SQLite FTS5 (unicode61 tokenizer,
diacritics folded). Returns claim dicts ordered by FTS5 rank (best
match first). Same projection as query().
"*", "**") are refused; they would scan
the whole table.
Parameters: query is required and must be a non-empty FTS5 MATCH
expression; it is the positional query() takes as text. min_support,
classification, limit, include_unverified and include_invalidated
behave as in query(). refutation_filter is query-only and search()
raises TypeError on it.
Raises ValueError on empty / pure-wildcard / malformed FTS5 syntax.
record_replication_verdict(*, verdict_id, cluster_id, member_claim_id, other_claim_id, method, confidence)
Insert a signed replication verdict. The graph’s loaded signer signs the
verdict; its keyid must be enrolled in the project’s validators table
(chain walk back to a self-signed root, same gate as validate()).
The OSS core accepts verdicts; the predicates that generate them
(semantic-cluster, cross-method, hash-match, shared-resolved-upstream)
live outside the OSS and call this method to write their output. Any
third-party verdict-issuer can integrate against this protocol.
Side effect: promotes the referenced claims from
PRELIMINARY to
REPLICATED (only when still PRELIMINARY AND status='open' AND t_invalid IS NULL). INSERT + promotion run in one BEGIN IMMEDIATE
transaction so a concurrent contradiction cannot land between the writes.
Raises VerdictIssuerError: no signer loaded, signer’s keyid not
enrolled (or chain broken), method not in the allowed enum, referenced
claim_id missing.
record_contradiction_verdict(*, verdict_id, member_claim_id, other_claim_id, confidence)
Insert a signed contradiction verdict. Sets t_invalid on the older of
the two referenced claims via the contradiction_invalidates_older AFTER
INSERT trigger; default query() / search() then excludes the
invalidated claim.
Raises
VerdictIssuerError: same gates as record_replication_verdict,
plus self-contradiction.
replication_verdicts(*, member_claim_id=None, cluster_id=None, include_invalidated=False)
List signed replication verdicts, optionally filtered.
query(). Pass include_invalidated=True for
audit-mode listings.
Returns list[dict]: each dict carries verdict_id, cluster_id,
member_claim_id, other_claim_id, method, confidence_json,
issuer_keyid, signature (raw bytes), created_at.
contradiction_verdicts(*, claim_id=None, include_invalidated=False)
List signed contradiction verdicts, optionally filtered by either side
of the pair.
include_invalidated=True since the
contradiction verdict IS the evidence for invalidation; auditing “why
was this invalidated” requires audit mode.
Returns list[dict]: each dict carries verdict_id,
member_claim_id, other_claim_id, confidence_json, issuer_keyid,
signature, created_at.
get_validator_reputation()
Returns {validator_keyid: count} for every enrolled validator. Count
is the number of ESTABLISHED claims whose validation envelope was
signed by that keyid. Validators with zero promotions appear with
count=0. Derived state, recomputed on every call.
query_for_llm(text=None, *, min_support=None, classification=None, limit=20)
Same shape as query() with two changes: the text and comparison_summary fields are sanitized (zero-width / bidi / control characters stripped, length capped) AND wrapped in <untrusted_data>...</untrusted_data> delimiters; metadata labels (source_name, generated_by, validated_by) are sanitized but not wrapped.
Use this when retrieved claims will be spliced into an LLM prompt: claim text is written by earlier agents and may contain stored prompt-injection payloads.
<untrusted_data> is data) is the caller’s responsibility.
For one-off content that doesn’t come from the graph, mareforma.sanitize_for_llm(...) and mareforma.wrap_untrusted(...) are public primitives.
get_claim(claim_id)
Return a single claim dict by ID.
claim_id: str
Returns dict | None: None if not found. Same field shape as query(). A REPLICATED / ESTABLISHED row is re-verified on read; if its signature fails to verify the dict is returned with verified=False rather than excluded. Never raises.
update_claim(claim_id, *, status=None, text=None, supports=None, contradicts=None, comparison_summary=None)
Update the mutable fields of an existing claim.
status and comparison_summary are always editable. text, supports, and contradicts are bound into the signed payload and refuse to move once the claim carries a signature bundle.
A status change is editorial: it produces no signed envelope, names no validator keyid, and nothing in mareforma records who pulled the lever. Any process with write access to the database can flip an ESTABLISHED claim to retracted. For a retraction another party can verify, assert a new claim with contradicts=[claim_id] under a validator key; that leaves a signed envelope and a contradiction verdict restore re-verifies.
Two processes updating the same claim are serialised by SQLite at the row level, last writer wins, with no conflict detection.
Returns None
Raises
ClaimNotFoundError: if the claim does not existSignedClaimImmutableError: ontext/supports/contradictsagainst a signed claimIllegalStateTransitionError: if thestatusmove violates the state machineValueError: on an invalidstatusor emptytext
refutation_status(claim_id)
Return whether anything in the graph pushes back on a claim.
dict with keys state, reason, signal. state is clean | contradicted | contested | retracted. signal names how strong the evidence behind that state is: signed-verdict for a contradiction verdict an enrolled validator signed, editorial for an unsigned status flip, none when there is no signal at all. A contested state reached editorially and one reached by signed verdict read the same in state and differ only here.
Raises ClaimNotFoundError if the claim does not exist.
trust_map(claim_id, *, reexec_record=None)
Return the per-claim TrustMap: every trust property placed at the tier its answer actually comes from, with the residual named.
Returns
TrustMap | None: None if the claim does not exist. Its properties carry one entry per axis, in order: attributability, provenance, grounding, faithfulness, methodological_validity, leakage, independence, contestation, standing, trust_root, witnessing. get(name) returns that entry or None.
Each entry is a TrustProperty with name, tier, value, and residual. tier is COMPUTED (derived from stored evidence), PROXIED (computed through a proxy signal whose bound is named), or DEFERRED (not evaluated, with the reason named so the gap is explicit rather than silent). residual says what the answer does not cover. Read it: a COMPUTED axis is still bounded by its residual.
to_dict() is canonicalizable and canonical_digest() is sha256:<hex> over its RFC 8785 bytes, so a map can be pinned in CI and compared across runs. The mareforma map CLI renders the same object as text, JSON, or one self-contained HTML file.
query_provenance(claim_id, *, depth=4)
Return the lineage around a claim in one JSON-serialisable object, for an agent prompt, a PROV-O export, or audit evidence.
claim carries the focal row plus the role attestations in its DSSE envelope. upstream and downstream are lists of {"claim_id", "depth", "position", "row"} walked depth hops through supports[]; depth=0 returns the focal claim and metadata only. contradictions holds both directions plus any signed contradiction verdicts, replication the clusters this claim sits in.
Every signed envelope is returned verbatim from the row, so a consumer re-verifies against the enrolled validators rather than trusting this call.
Returns dict
Raises ClaimNotFoundError if the claim does not exist.
validate(claim_id, *, validated_by=None, evidence_seen=None)
Promote a REPLICATED claim to ESTABLISHED. Identity-gated.
mareforma bootstrap or mareforma.open(key_path=...)) AND that key must be enrolled in the project’s validators table. The first key opened against a fresh graph auto-enrolls as the root validator. The validation event itself is signed: a DSSE-style envelope binding (claim_id, validator_keyid, validated_at, evidence_seen) is persisted to the row’s validation_signature column, so the promotion is independently verifiable.
validated_by is a cosmetic display label. The authenticated identity is the keyid embedded in the signed envelope.
evidence_seen is an optional list of claim_ids the validator declares to have reviewed before signing. None is normalized to [] and bound into the signed envelope as a positive “I reviewed nothing” admission. Each cited entry must be a strict-v4 UUID matching an existing claim with created_at <= validated_at. The validator’s enumeration is self-declared (mareforma cannot prove what was actually read), but the envelope shifts “a human pressed a button” to “a human pressed a button AND named the evidence they consulted.”
Parameters
When
validation_signature is supplied directly to db.validate_claim (advanced/test path), mareforma also decodes the envelope’s signed payload and refuses if its evidence_seen field disagrees with the evidence_seen kwarg. The signed envelope and the validated list must bind the same citations exactly (same items, same order); a direct caller cannot launder fraudulent citations through the on-disk envelope.
Raises
ClaimNotFoundError: if the claim does not existValueError: ifsupport_levelis notREPLICATED, no signer is loaded, or the loaded signer is not an enrolled validatorEvidenceCitationError: if anyevidence_seenentry is not a strict-v4 UUID, does not point to an existing claim, post-datesvalidated_at, or disagrees with the validation envelope’s signedevidence_seenfieldLLMValidatorPromotionError: if the loaded signer is enrolled withvalidator_type='llm'. LLM-typed validators sign validation envelopes but cannot promote past REPLICATED; a human-typed validator has to make the callSelfValidationError: if the loaded signer’s keyid is the one on the claim’ssignature_bundle. Promotion needs an external witness, so on a single-key project this is the expected outcomeInvalidValidationEnvelopeError: if the signed envelope fails a structural or cryptographic gate (malformed payload, wrongpayloadType, signer not enrolled, signature does not verify, or a payload field disagreeing with the row being promoted)
MareformaError directly, not ValueError, so a handler that catches ValueError alone lets all three through.
health()
Single-call audit summary aggregating core counters. Pure observability over existing surfaces; no side effects.
unsigned_claims, unresolved_claims, dangling_supports, convergence_errors, convergence_retry_pending). Non-zero values don’t by themselves indicate a defect; they indicate something the operator should look at.
Returns dict[str, int] with the seven keys above.
convergence_errors
Read-only count of SQLite errors swallowed during convergence detection since the graph was opened.
refresh_convergence() retries them.
Returns int, reset to zero on every open.
read_verify_exclusions
Read-only count of rows query() and search() dropped because their signature did not re-verify.
.mareforma/health.jsonl as read_verify_excluded, rate-limited to the 1st, 2nd, 4th, 8th occurrence and so on, and again whenever the running total has at least doubled since the last line, so a sudden spike is not lost between powers of two. The exclusion is a state every later read finds again, and a long-lived reader would otherwise write one line per poll. The counter itself is exact.
Zero means nothing was excluded from the reads this session made, not that the graph is untampered. Non-zero means a claim on disk failed re-verification: name it with get_claim() or mareforma verify.
Returns int, reset to zero on every open.
read_unverified_exclusions
Read-only count of rows query() and search() held back behind the default verified filter.
include_unverified=True. Unlike the verify exclusions above, a flag does bring these back: they are not tampered rows, they are rows the default read does not vouch for. Without the count, a project whose claims were written under an unenrolled key answers a query with an empty list, and a caller reads “there is nothing here” for a record that is not empty.
The count is taken only when a read comes back short, since a full page already says has_more. It is bounded by its own fixed 5,000-row scan ceiling rather than the read’s (which scales with limit), so the disclosed number does not move with the page size you asked for, and a count that stopped at the ceiling is reported as a floor. The MCP query_claims and search_claims tools report the per-call number as unverified_excluded, and add unverified_excluded_is_at_least: true when the count stopped at its ceiling. The same event is appended to .mareforma/health.jsonl as read_unverified_excluded with outcome degraded, under the same rate limit as read_verify_excluded.
Returns int, reset to zero on every open.
refresh_convergence()
Retry convergence detection (PRELIMINARY → REPLICATED) for every claim flagged convergence_retry_needed=1. Without this method, a swallowed SQLite error during detection would leave the claim stuck at PRELIMINARY forever.
dict with keys checked, retried_ok, promoted, still_pending. retried_ok counts the claims whose detection ran cleanly this pass; promoted is the subset whose support level actually moved. A claim with no converging peer recovers cleanly and promotes nothing, so retried_ok without promoted is the normal outcome.
refresh_unsigned()
Retry transparency-log submission for every signed-but-unlogged claim when the graph was opened with rekor_url=....
rekor_url is unset.
Two recovery paths:
- Sidecar replay: when the original Rekor submission succeeded but the claims-row UPDATE failed (recorded in
rekor_inclusions), the stored coords are re-attached to the row in a single local UPDATE. No network call, no duplicate Rekor entry. - Re-submit: when no sidecar row exists, the envelope is submitted to Rekor again. Used only when the original submission has no persisted record.
assert_claim) is skipped with a warning.
Returns dict with keys checked, logged, still_unlogged.
find_dangling_supports()
Return UUID-shaped supports[] entries pointing to claims that do not exist in this graph. DOIs and other free-form strings are external references and are NOT flagged.
list[dict] of {"claim_id", "dangling_ref"} dicts sorted deterministically. Empty list when the graph is clean.
classify_supports(values)
Classify each entry in a supports[] / contradicts[] list as claim | doi | external. Pure-function (no network, no DB read).
list[dict] of {"value", "type"} dicts in input order.
enroll_validator(pubkey_pem, *, identity, validator_type="human")
Enroll an additional validator on this project. The currently loaded signer (which must itself be enrolled) signs the enrollment envelope.
Raises
ValueError: no signer loaded, or the loaded signer is not enrolledInvalidIdentityError: identity contains rejected charactersInvalidValidatorTypeError:validator_typeis not'human'or'llm'ValidatorAlreadyEnrolledError: key already enrolled (the message distinguishes a normal duplicate from a chain-broken row)
require_rekor_witnessing()
Declare that this project’s findings must be witnessed by the transparency log before they can converge. The root validator signs a project-policy envelope; the declaration is persisted and emitted to claims.toml. On recovery, restore(..., enforce_rekor_policy=True) verifies this envelope against the enrolled root and refuses to mark any signed claim convergence-eligible without a verified, claim-bound inclusion proof, closing the strip-route where an edited claims.toml makes an unwitnessed claim look ready.
ProjectPolicyError(aValueError): no signer loaded, or the loaded signer is not the project’s single root validator
list_validators()
Return the project’s validator rows, ordered by enrolled_at.
list[dict]: each dict carries keyid, pubkey_pem, identity, validator_type, enrolled_at, enrolled_by_keyid, enrollment_envelope, and verified. verified is False when the row’s enrollment chain does not walk back to the project’s single self-signed root, so a row written straight into the database is never reported as an enrollment.
get_tools(*, generated_by="agent", include_deprecated_aliases=False)
Return [query_graph, record_claim] as plain Python callables with behavioral contracts in their docstrings. Wrap with any framework’s tool adapter in one line.
Returns
list: [query_graph, record_claim]
query_graph(topic, min_support="PRELIMINARY") -> str: routes throughquery_for_llm. Returns a JSON string of matching claims with free-text fields sanitized and wrapped in<untrusted_data>...</untrusted_data>.record_claim(text, classification="INFERRED", supports=None, contradicts=None, source="") -> str: returnsclaim_id.
backup() / defer_backup()
claims.toml is refreshed after every mutation. For a bulk import or a loop of writes, group the mutations under one rewrite:
defer_backup() block each mutation marks the backup due rather than rewriting the whole file; the write happens once when the outermost block exits (windows nest), and claims.toml reflects committed state again after it. graph.backup() forces a write, for example at the end of a batch. The write is atomic, so a crash cannot truncate the recovery artifact.
close()
Close the graph database connection.
__exit__ calls close() automatically. A still-open defer_backup() window is flushed first, so a graph closed mid-batch still leaves claims.toml current.
Exceptions
Each exception lives in the submodule that raises it. Import from the submodule shown in the table.from mareforma import RekorInclusionError.
The submodule paths in the table tell you where the source lives. They
also tell you what a broad catch covers: the
mareforma.db and
mareforma.trust exceptions subclass MareformaError. Every other
module’s exceptions sit outside that tree, so except MareformaError
around a signing, enrollment, observer, verifier or bundle-verification
call catches nothing.
Catch those by name, or by the base their module gives them:
SigningError, mareforma.validators.ValidatorError,
BundleVerificationError.
Schema-mismatch message (raised when open_db() finds a graph.db whose user_version differs from mareforma’s _SCHEMA_VERSION): "graph.db has user_version=N but this mareforma expects user_version=M. The dev branch does not migrate schemas. Delete .mareforma/graph.db to start fresh; claims.toml is a human-readable record of the prior state."
v0.3.3 surface
mareforma.db.open_db_from_db_path(db_path)
Open the graph DB from a direct path to graph.db, not a project root. Honours the supplied filename when it sits outside the conventional <root>/.mareforma/graph.db layout (where mareforma.open() would re-derive that path).
graph.db at a literal path outside the conventional <root>/.mareforma/graph.db layout.
Capability-shaped predicate URI constants
mareforma.predicate_types exposes URN-form constants for every reserved predicate. Re-exported at the top level so adapter authors can from mareforma import TOOL_CALL_V1.
Core-owned (writer in the core):
CLAIM_V1:urn:mareforma:predicate:claim:v1EPISTEMIC_GRAPH_V1:urn:mareforma:predicate:epistemic-graph:v1CLAIM_WITH_ROLES_V1:urn:mareforma:predicate:claim-with-roles:v1
mareforma.adapters.* or a third-party adapter):
TOOL_CALL_V1:urn:mareforma:predicate:tool-call:v1CONTAINER_EXEC_V1:urn:mareforma:predicate:container-exec:v1CODE_VARIATION_V1:urn:mareforma:predicate:code-variation:v1HYPOTHESIS_V1:urn:mareforma:predicate:hypothesis:v1LITERATURE_INSIGHT_V1:urn:mareforma:predicate:literature-insight:v1SCIENCE_SKILL_V1:urn:mareforma:predicate:science-skill:v1META_CLAIM_V1:urn:mareforma:predicate:meta-claim:v1WORKSHOP_EVENT_V1:urn:mareforma:predicate:workshop-event:v1AGENT_TRACE_V1,INGESTED_TRACE_V1,LLM_OUTPUT_V1,REVIEW_V1,PEER_REVIEW_V1,ELO_MATCH_V1,TOURNAMENT_BRACKET_V1: additional reserved namespaces.- Wet-lab assay family:
WET_LAB_ASSAY_V1plus_FLOW_CYTOMETRY,_SEQUENCING,_IMAGING,_PROTEOMICS,_ELECTROPHYSIOLOGYsiblings. REPLICATION_ATTESTATION_V1,COMPOUNDING_ATTESTATION_V1,SEMANTIC_GROUNDING_V1,DOI_RESOLUTION_V1.
mareforma.events
Typed Protocol contract for adapter event sources.
EventSource and EventHandler are @runtime_checkable Protocols: isinstance(obj, EventSource) at adapter construction time fails loudly on a missing subscribe / unsubscribe / handle_event attribute. EventPayload and ClaimResult are TypedDicts. Source-name constants prevent string-typo dispatch bugs.
mareforma.tools
Structural contract for any wrappable callable.
mareforma.canonicalize
Registry-based canonicalizer surface for adapter authors. Distinct from the internal envelope canonicaliser (mareforma._canonical); adapters use the public registry so claim result_canonical_form fields can name forms by registered string.
mareforma.canonicalize also registers the specialty forms rdkit-canonical-smiles-v1, smiles-nfc-fallback-v1, fasta-nfc-v1, fasta-nfc-v2, pdb-atom-sorted-v1, pdb-atom-sorted-v2 via the auto-imported specialty submodule. New FASTA callers want fasta-nfc-v2, which absorbs column wrap and CRLF; fasta-nfc-v1 keeps internal line breaks and stays registered only because digests are recorded against its bytes. New PDB callers want pdb-atom-sorted-v2, which reads the hybrid-36 serials of a structure past 99999 atoms that pdb-atom-sorted-v1 sorts as 0. rdkit-canonical-smiles-v1 needs rdkit (pip install mareforma[chem]) and refuses without it; a host that wants the NFC-stripped string form asks for smiles-nfc-fallback-v1 by name, so the persisted form name says which algorithm produced the bytes.
mareforma.adapters.*
Three opt-in adapter packages: see Mareforma adapters (under the Adapter framework section) for full integration examples. Quick reference:
All three ship in the wheel and run on core dependencies, so none needs an install extra. Importing one registers its predicate URIs.
v0.3.4 surface
mareforma.trust
The trust layer turns a free-text claim into a structured finding: a
content-addressed proposition, a pre-registered prediction, a computed bearing,
and a derived status. See Findings for the narrative. It
is additive: every finding still rides a signed claim, so it appears in
query() with a support_level like any other claim.
Proposition(subject, relation, object, direction=Direction.UNSPECIFIED, scope={}, magnitude=None)
Frozen, value-typed, content-addressed claim about the world.
Direction enum: INCREASES, DECREASES, NO_EFFECT (one contrary family),
PRESENT, ABSENT (a second), UNSPECIFIED (the rejection sentinel, never
stored). REGISTRABLE_DIRECTIONS is the closed set of the five storable values.
normalize_token(s) exposes the NFC + casefold + whitespace-collapse used at
identity time.
Prediction(test_type, alpha=0.05, *, direction_of_interest=None, equivalence_lower=None, equivalence_upper=None, inference_regime=InferenceRegime.FREQUENTIST)
The pre-registered decision rule. TestType.SUPERIORITY requires
direction_of_interest (DirectionOfInterest.INCREASE / DECREASE);
TestType.EQUIVALENCE requires equivalence_lower < equivalence_upper. alpha
must be in (0, 0.5): the gate is one-sided at alpha over a 1 - 2*alpha CI, so
a larger alpha marks every p-value significant and leaves no valid CI level.
Raises ValueError on an inconsistent combination.
EffectEstimate(estimate_value, effect_type, scale=Scale.RAW, *, p_value=None, ci_lower=None, ci_upper=None, ci_level=None, n_total=None)
A point estimate plus the uncertainty the gate needs. Supply a p_value, a full
(ci_lower, ci_upper, ci_level) triple, or both. Validates on construction and
raises InconsistentEstimateError for a non-finite value, a partial CI triple,
p_value outside [0, 1], ci_level outside (0, 1), a CI that does not
bracket the estimate, or a non-positive n_total.
EffectType:SMD,Hedges_g,OR,logOR,RR,HR,COR,ZCOR,MD,ROM,beta,log2FC,GEN(metaformeasurevalues).Scale:raw/log.null_value(effect_type, scale): the “no effect” value:1for raw-scale ratio types (OR/RR/HR/ROM),0otherwise.Contrast(control_type=ControlType.NEGATIVE)andEvidenceLine(estimate, data_id, contrast=…, modality=…, provenance_id=…, design_type=…)build the evidence tree; a finding may carry several lines.data_idis the dataset guard the run-distinct independence count uses so the same dataset is not counted twice.
compute_bearing(estimate, prediction) -> Bearing
The gate. Returns a frozen Bearing(direction, significant) where direction
is BearingDirection.SUPPORTS / REFUTES / NEUTRAL. Raises
InconsistentEstimateError when the estimate cannot drive the requested gate
(e.g. an equivalence test with no CI, or a CI at the wrong level for the
prediction’s alpha).
gates_for(prediction) returns the same decision rule as an ordered
short-circuit gates[] chain, and evaluate_gates(estimate, gates) runs it.
A one-element chain is bearing-identical to compute_bearing; a multi-gate
chain raises NotImplementedError until its precedence is designed.
compute_status(independent_support, independent_refute) -> Status
The state machine: UNTESTED (0, 0); CONTESTED (≥1 support and ≥1 refute);
REFUTED (≥1 refute, 0 support); CONVERGENT (≥2 support, 0 refute);
PRELIMINARY (exactly 1 support, 0 refute). CONVERGENT is a convergence
marker, not a corroboration or independence verdict: cross-model error
correlation is unmodeled and is the named residual. compute_frame_status(contrary_independent_support)
returns FrameStatus.CONTESTED when a contrary proposition in the same frame
has ≥1 independent supporting line, else CONSISTENT. Independence is counted by
distinct signer (the claim’s asserter_keyid) with data_id and model lineage
as guards, so one signer yields at most one support and one refute; distinct
generated_by run is only the fallback for legacy or unsigned lines.
STATUS_POLICY is the policy stamp
("status_policy@v4"), independent of the package version.
Errors: TrustError (base), NonFalsifiablePropositionError,
InconsistentEstimateError, NoRegisteredPlanError, FindingPlanForkError,
PostHocPlanError, PlanNotRetirableError.
EpistemicGraph trust methods
register_proposition(proposition) -> str
Register a falsifiable Proposition; returns its content_id. Idempotent on
content_id. Raises NonFalsifiablePropositionError if the proposition has no
direction or an empty scope.
register_plan(proposition, prediction, *, generated_by=None) -> str
Pre-register a decision rule against a proposition, before the numbers are
seen. Registers the proposition, writes the predictions row with
preregistered=1, and writes its own signed plan attestation claim (under
idempotency key plan:{plan_id}, Rekor-anchorable like any other claim).
Returns the content-addressed plan_id. Idempotent: re-registering the same
prediction is a no-op on both the row and the claim. Raises
NonFalsifiablePropositionError on a non-falsifiable proposition.
retire_plan(plan_id, *, alpha, reason) -> dict
Retire a plan the gates cannot run and re-register its evidence under an alpha
they can. A plan written by a release with a wider alpha bound can state a rule
that decides nothing (alpha at or above 0.5 marks every p-value significant and
asks for a confidence level of zero or less). The graph still restores, but
every evidence line under that plan drops out of the counts and the proposition
reads UNTESTED with lines_skipped non-zero. The predictions row is
append-only and cannot be deleted, so there is nothing to correct in place.
The drop names the plan to pass here: it is recorded on
.mareforma/health.jsonl as ungateable_plan_skipped with the plan_id.
One call, one transaction, three effects:
- Registers the replacement: the retired plan’s own rule at
alpha. Only the alpha moves, so a repair cannot re-choose the side of the null once the numbers are known. The row carriespreregistered=0and its attestation names what it supersedes and why: it was registered after the evidence, and the record says so rather than reading as an original pre-registration. - Records the retirement in
plan_retirementsand writes a signed retirement attestation whose text renders the plan, the replacement and the reason, sorestorere-derives the record from signed material. - Leaves the retired row exactly as registered. The read path gates the evidence that stood under it against the replacement from here on, so the lines count again.
dict with plan_id, superseded_by, reason, retired_at,
claim_id (the retirement attestation), plan_claim_id (the replacement’s
attestation) and lines_recovered. Idempotent: retiring the same plan at the
same alpha returns the recorded receipt.
Retirement is the operator’s call and never automatic. It is not a way to
withdraw evidence: only a plan whose rule cannot be run is retirable, and those
lines already count zero, so a retirement can recover a dropped line and never
drop a counted one. Raises NoRegisteredPlanError when no such plan exists, and
PlanNotRetirableError when the plan’s rule still runs, when it is already
retired (a second retirement would let the operator shop for the alpha that
reads best), or when nothing would be recovered. An alpha outside (0, 0.5)
raises ValueError, the same bound every registration is held to.
submit_finding(proposition, prediction, estimate=None, *, data_id=None, data_bytes=None, data_source=None, lines=None, generated_by=None, control_type=None, modality=None, provenance_id=None, design_type=None, code_ref=None, idempotency_key=None, grounding=None, grounding_strict=False) -> dict
Submit a finding against an already-registered plan. Same shape and return as
assert_finding, with one difference: the plan must already exist (else
NoRegisteredPlanError), and the finding’s signed supports[] cites the plan
attestation’s claim_id, so the plan → finding edge is cryptographic, not
denormalised metadata. The finding’s identity is its full data_id set: a
re-submission carrying a partial overlap or a different plan raises
FindingPlanForkError rather than silently returning the prior bearing. The
existence check and the writes run in one transaction.
A plan registered with preregistered=1 must pre-date the run’s first
execution, and the run’s first execution is the earliest finding already
written under the same generated_by token anywhere in the graph. A plan
registered after that raises PostHocPlanError, refused before any write.
The scope is the run token, not the proposition: reuse one stable
generated_by across studies and the first finding under it fixes the run’s
start for every later plan. Pre-register before the run executes, or submit
under a fresh run token. Omitting generated_by is not an exemption, since
the write resolves it to the default run token and the gate asks about that
same token.
assert_finding(proposition, prediction, estimate=None, *, data_id=None, data_bytes=None, data_source=None, lines=None, generated_by=None, control_type=None, modality=None, provenance_id=None, design_type=None, code_ref=None, idempotency_key=None, grounding=None, grounding_strict=False) -> dict
Record a finding: validate the inputs, compute a bearing per line, write a signed
claim as the attestation, persist the evidence tree, and derive the proposition’s
status. Pass either estimate + data_id (one line) or lines (a sequence of
EvidenceLine, the multi-line evidence tree), never both. A finding with no
lines raises ValueError; a generated_by that is blank or whitespace raises
ValueError, since independence is counted by run.
dict with finding_id, content_id, plan_id, claim_id,
bearing ({"direction", "significant"}), bearings (the per-line list, one
entry for a single-line finding), status, idempotent (bool), grounding
(the stored observed-grounding record, None when no verdict was supplied),
model_lineage (the model or method lineage recorded on the evidence lines,
None when no model call was observed), and proposition_status (the full
view below). An idempotent replay reports the grounding and model_lineage
stored on the reused finding, not the ones passed to the replay.
Idempotent on the finding’s data_id set: re-asserting the same dataset(s)
returns the prior finding rather than double-counting it. All validation runs
before the signed claim is written, so a rejected finding never leaves an
orphan claim. Raises NonFalsifiablePropositionError /
InconsistentEstimateError on bad input.
The one-shot composes register_plan + submit_finding internally; its
synthesised plan is flagged preregistered=0, so a genuine up-front
pre-registration via register_plan stays distinguishable from it.
proposition_status(proposition_or_content_id) -> dict | None
The retrieval view for one proposition. Accepts a content_id or a
Proposition. None if not registered.
Returns dict with content_id, frame_id, direction, status,
independent_support, independent_refute, question_status (the state of the
question, either consistent or divided), frame_status (its retired
predecessor, removed in v0.4.0), status_policy, lines_skipped, the count of
this proposition’s evidence lines that were left out of the counts, and
post_hoc. A dropped refutation reads as consensus, so read lines_skipped next
to the two counts.
Each skipped line records its reason once in .mareforma/health.jsonl, so a
non-zero lines_skipped is always explainable. It is written once per handle,
not once per read, so polling proposition_status does not grow the file:
The first two are ordinary lifecycle. The rest mean a row was rewritten under
the read path, and
restore refuses a backup carrying any of them except
unregistered_signer_skipped, which an honest backup can legitimately hold when
it carries a participant’s finding whose key this project never enrolled.
post_hoc is true when the count rests on a plan that was not
pre-registered, either a one-shot plan or the replacement retire_plan
resolved a stranded line to, whose alpha was chosen with the estimates already
in view. It lets a reader tell a post-hoc gate from a pre-registered one.
A line dropped because its plan states a rule no gate can run is recoverable;
see retire_plan above.
get_proposition(content_id) -> dict | None
The stored proposition row as a dict, or None.
query_frame(frame_id_or_proposition, *, min_status=None) -> list[dict]
Everything known about a question (frame), each entry a proposition_status
view. Accepts a frame_id or a Proposition. min_status filters to a floor on
the UNTESTED < PRELIMINARY < CONVERGENT ladder (the only valid floors;
REFUTED / CONTESTED are off the ladder and excluded by any floor). Raises
ValueError on an invalid floor.
v0.3.8 surface
Themareforma.observe package computes whether cited data actually flowed into a
finding. See Grounding for the model.
mareforma.observe.observe(cites=None, *, content_address=False)
Context manager over the code that authors a finding. cites is the source(s)
the finding cites: a path, a URL, a sha256: data id, or an iterable of these.
Set content_address=True to match a cited sha256: id against the hash of a
read’s returned bytes instead of by identifier. Yields an ObserveHandle whose
.verdict holds the computed GroundingVerdict after the block closes. Reading
.verdict inside the open block raises ScopeNotClosedError.
GroundingVerdict
The computed verdict plus its receipt. Fields: grounding (an ObservedGrounding
member), reason (str), cited_sources, grounded_sources, reads, seams,
matched_identifier, version, reads_seen, opens_detected, model_lineage.
-
grounded_sourcesis the cited sources a matching non-empty read was actually observed for: the subset ofcited_sourcesthat the read-side binding gate checks against. Empty forUNGROUNDED/OPAQUE. -
model_lineageis aModelLineage | Nonecaptured at the model-call boundary, tieredCOMPUTED/PROXY/UNVERIFIABLElikedata_id, andNonewhen no model call was observed. It rides the evidence line, not the signed receipt or its digest. See themodel_lineagerow in Data model for the stored shape. -
groundingisObservedGrounding.GROUNDED/UNGROUNDED/OPAQUE. -
receipt()returns the full canonicalizable evidence;receipt_digest()is itssha256:digest. -
from_receipt(receipt)rebuilds a verdict from areceipt()dict, the inverse ofreceipt(). It does not restoremodel_lineage: the receipt does not carry the lineage, so a verdict read back from a persisted receipt hasNonethere. -
to_signed_dict()is the compact{version, grounding, reason, cited_sources, grounded_sources, receipt_digest}record bound into the signed statement. mareforma binds the digest, not the receipt, so a caller that keeps the receipt out of band can detect mutation. -
read_coverage_fraction()isreads_seen / opens_detected, both counted over the cited paths the observer saw opened, orNonewhen no cited path was opened. Below 1.0 means a cited source was opened through a reader the observer could not read through.
perturbation_oracle(run_fn, base_input, perturb=None, *, repeats=1, metric=None, effect_threshold=0.0, noise_multiplier=3.0, multiplicity=1, thin_sigma_guard=False, determinism_rtol=1e-06, determinism_atol=0.0, on_progress=None)
The independent causal check. Runs run_fn on the base input and on perturbed
inputs and returns an OracleResult whose influence is INFLUENCED,
NOT_INFLUENCED, UNDECIDABLE, or NOT_TESTED. With repeats > 1 the spread of
the base runs sets the noise floor a real effect must clear, so a stochastic
pipeline does not read as influenced by its own jitter. At repeats=1 no noise is
measured, so the floor is 0; the result carries noise_measured=False and
noise_is_thin=True and says so in its reason.
perturb=None, the default, derives the whole null family from the input’s data
shape through scramble_family (a scalar, a mapping of scalars, or a sequence of
scalars, including a namedtuple or an array): content-destroying nulls (zeroed,
constant) and marginal-preserving ones (permuted, reversed). The verdict
routes on the PROFILE across that family, not on one move: INFLUENCED only when
every null moves the finding past the threshold, NOT_INFLUENCED only when none
does, and UNDECIDABLE when some move and others do not, which is the honest
reading of a genuine mean (invariant under a reordering, moved by zeroing) and
must never be called hollow. Each perturbation is scored against the base on its
own, so opposing perturbations do not cancel; perturbation_effects carries one
effect per null, scramble_names names them, and null_outcomes carries the
classification the verdict was routed on (MOVED / AMBIGUOUS /
BELOW_DOMAIN_FLOOR / FLAT), which flat_nulls, ambiguous_nulls,
below_domain_floor_nulls and blind_spot_line() all read rather than
re-deriving. BELOW_DOMAIN_FLOOR means the finding moved measurably but by less
than the effect_threshold the caller declared; it routes like FLAT for the
verdict and is named apart from it, because a null the finding moved a million
units under did not hold invariant.
NOT_TESTED means the oracle produced no verdict, with a typed
not_tested_reason: unsupported-shape (no family fits the input),
null-construction-failed (building a perturbed input raised),
target-failed (the unperturbed run raised), crashed-under-null,
unreducible-value, and non-finite-value (a run came out NaN or infinite, so
there is no comparable number). It is never a verdict in disguise: the three
measurement numbers are None on such a row, so no reader takes a zero off it.
A null identical to the base cannot perturb anything, so it is dropped rather than
run and its name goes in dropped_nulls; a constant sequence loses both
marginal-preserving nulls this way, and the reason says so, because a verdict over
a family the data narrowed is a narrower claim. Supplying perturb yourself (a
callable for one null, a sequence for several) sets caller_chose_nulls=True: a
chosen null is a place to fish, and a single one cannot reach the mixed profile at
all, so its NOT_INFLUENCED is weaker than the derived family’s. An empty
sequence raises NoPerturbationsError rather than reading as NOT_TESTED.
Two keywords widen the decision threshold before the influence call is made.
multiplicity=n declares the finding is one of n, and the widening is
sqrt(2 ln(n * number_of_nulls)) sigmas, because taking the max across the nulls
is itself a multiple comparison; that is the expected size of the largest spurious
deviation across a family that size, not a quantile, so it controls no stated
error rate. thin_sigma_guard=True widens the same margin when the noise floor
rests on fewer than five repeats, and defaults off. Neither is a no-op on the
zero-config path: the derived family has several nulls, so the widening fires from
the null count even at multiplicity=1. Neither reaches a pipeline with no
measurable noise, where there is no sigma to widen; multiplicity_applied records
whether the widening actually reached the threshold.
A deterministic pipeline has no run-to-run noise, so the noise floor is measured
at 0 and there is no sigma to build a threshold from. determinism_rtol=1e-6
sets the float-equality band for that case: a move within determinism_rtol * |base mean| of the finding’s magnitude is indistinguishable from summation-order
artifacts (BLAS thread counts, reduction order) and is classified AMBIGUOUS,
which routes the verdict to UNDECIDABLE, never INFLUENCED; a move above the
band is MOVED; a move of exactly 0 is FLAT, a provable invariant. The verdict
is then the profile over all of them, so exactly-0 under every null is
NOT_INFLUENCED while exactly-0 under some and a real move under others is
UNDECIDABLE. This is what keeps the zero-config oracle
from degenerating to exact float equality on the modal deterministic target. A
result also carries deterministic=True when the floor was measured at 0 over
more than one repeat, distinct from an unmeasured single-run floor.
determinism_atol adds an absolute lower bound to that band, for a finding whose
magnitude is near 0 where the relative band alone would vanish; it defaults to 0.
reconcile(grounding, influence) relates the observer’s flow verdict to the
oracle’s influence verdict and returns a ReconcileResult carrying the
relation and a reason.
summarize(verdicts) -> GroundingReport
Aggregates an iterable of GroundingVerdict into the split a report states:
fractions(), incidental_read_rate, mean_read_coverage, and
opaque_dominates(threshold=0.5). A sweep that reports an influence rate over a
corpus needs multiplicity set to the corpus size on every perturbation_oracle
call, or the rate counts findings that cleared the bar on noise; use
influence_sweep(findings, ...), which runs the corpus and computes the
multiplicity from the count so it is never left at 1.
mareforma.exporters
JSONLDExporter is the class behind mareforma export. The CLI reference
documents the formats and their conformance bounds; import it directly when you
want the graph in a process rather than on disk.
Oracle types
mareforma.observe exports the types the oracle’s fields are drawn from, so a
caller branches on a value instead of matching prose.
The derived family differs by shape, and the names appear in
scramble_names,
dropped_nulls and blind_spot_line():
A null identical to the base cannot perturb anything, so it is dropped and named
in
dropped_nulls rather than run.
mareforma.selfcheck
Seeded failures a correct instrument must catch, shipped in the wheel so an
installed user can run them without the source tree.
run_selfcheck(tmp_path) returns a KillSwitchOutcome per fixture, each
carrying name, expectation, observed and caught. The four are a silent
zero-row fallback, an excluded partition, a number produced with no execution,
and a decoy incidental read. SELF_CHECKS is the tuple of the underlying
functions if you want to run one on its own.
This is a self-check, not a sensitivity demonstration. Every fixture runs its
own synthetic pipeline rather than your target, and all four are accidents rather
than the evasive case. Passing them does not show the instrument is sensitive on
your own pipeline’s path.
summarize_influence(records) -> InfluenceReport and influence_records(receipts)
The influence arm of the measurement. influence_records(receipts) flattens the
influence list each receipt carries (one record per cited source) into a flat
list of per-edge records; summarize_influence(records) aggregates them.
InfluenceReport reports the count of each verdict (INFLUENCED,
NOT_INFLUENCED, UNDECIDABLE, NOT_TESTED), the not_tested_by_reason bucket,
not_tested_dominates(threshold=0.5), and influenced_fraction over decided
edges: INFLUENCED plus NOT_INFLUENCED, and nothing else. UNDECIDABLE is the
oracle declining to call an edge, so it is excluded from the denominator and named
separately in the summary line, and tested counts everything the oracle ran on
at all. Without that split a corpus of honest invariants and a corpus of hollow
findings print the same rate. The unit is the edge, so a finding citing several
sources contributes several edges; the inference unit is the run, so
distinct_runs rides alongside the rate, and a report whose writer stamped no run
count says so rather than printing zero. A receipts file with no influence
record yields an empty arm rather than a fabricated one.
summarize_influence_receipts(receipts) is the two composed, and
summarize_pilot returns all three arms on a PilotReport.
grounding= on assert_finding and submit_finding
Both gain an optional grounding parameter that takes a GroundingVerdict. The
verdict is bound into the signed statement, the chain hash, and a queryable
column, and re-checked on restore. The return dict gains a grounding key (the
stored record, or None). A finding whose verdict is not GROUNDED never counts
toward a support-level promotion; a claim asserted without a verdict is
unaffected and its signed bytes are identical to a pre-observer claim. Asserting a
claim while an observe() scope is still open raises.