Modules and trade-offs

Every other concept page treats one specific behavior as swappable: a source in The pipeline, a matcher in Predicates. This page steps back and treats “swappable” itself as the organizing idea: what a module actually is, how one gets into a running binary, and what it means for an operator that every axis of behavior in blockwatcher is one.

flowchart LR
    rpc["RPC endpoints<br/>(external chains)"]
    %% Source modules are family-owned; the running list is GET /catalog, not this map.
    sources["Sources<br/>one per network"]
    sinks["Sinks<br/>webhook · script · log"]
    storage["Storage<br/>checkpoints · dead letters · resources"]
    api["REST API"]
    metrics["Metrics"]
    engine["engine<br/>bounded channels · checkpoints"]

    subgraph pipeline["Engine pipeline"]
        direction LR
        decoder["Decoder"]
        matcher["Matcher<br/>predicates"]
        gate["Gate<br/>threshold · max_once"]
        decoder --> matcher
        matcher --> gate
    end

    rpc --> sources
    sources -->|"decode and match"| decoder
    gate --> sinks
    api -->|"manages resources"| storage
    storage <--> engine
    engine -->|"drives"| pipeline
    engine -.->|"reports"| metrics

    classDef module fill:none,stroke:#a9a3e3
    classDef core fill:none,stroke:#8a8d86,stroke-dasharray: 5 5

    class rpc,sources,sinks,decoder,matcher,gate module
    class engine,api,storage,metrics core
classDef dim fill:none,stroke:#999999,color:#999999,opacity:0.35
classDef focus fill:#ffd43b,stroke:#d9480f,stroke-width:3px,color:#1a1a1a
class rpc,api,metrics,engine dim
class sources,decoder,matcher,gate,sinks,storage focus
click sources "selectors.html"
click decoder "chain-agnosticism.html"
click matcher "predicates.html"
click gate "gates.html"
click sinks "delivery.html"
click storage "resources.html"
click api "../reference/http-api.html"
click metrics "../reference/observability.html"
click engine "pipeline.html"

Key takeaways

• blockwatcher has exactly six ports (Source, Decoder, Matcher, Gate, Sink, Storage), and a module is one concrete implementation of exactly one of them.
• A module never straddles two ports, even when several modules live in the same crate, and configuration resolves a name against the running binary’s own module catalog.
• Two modules behind the same port can trade off completeness against something else. That trade-off is a property of the module, documented on the module’s catalog row — chain-family rows live on the family page.
• Switching modules is the operator’s actual lever for a trade-off: changing one module leaves every other port’s behavior untouched, since none of them are written against one module’s assumptions.

Everything an operator composes is a module

An blockwatcher deployment has exactly six axes along which behavior can differ: how it reads a chain, how it turns raw bytes into blockwatcher’s canonical shape, how it decides whether a decoded occurrence counts, when a predicate-true hit becomes a delivery, where a match goes, and where resources and operational state persist. Those six axes are the six ports blockwatcher defines (Source, Decoder, Matcher, Gate, Sink, Storage), and a module is, by definition, one concrete implementation of exactly one of them. Nothing in blockwatcher is configured by writing code against a bespoke integration point; every one of these six choices is made the same way, by naming a module in configuration and giving it whatever config object that module expects.

The six ports look like this. Chain-family sources and decoders are registered the same way; they are named on each family page, not here.

flowchart TD
    source["Source"]
    decoder["Decoder"]
    matcher["Matcher"] --> expr["expr"]
    gate["Gate"] --> threshold["threshold"]
    gate --> maxonce["max_once"]
    sink["Sink"] --> webhook["webhook"]
    sink --> script["script"]
    sink --> log["log"]
    storage["Storage"] --> memory["memory"]
    storage --> sqlite["sqlite"]
    storage --> postgres["postgres"]

    %% Layout only, no meaning: these invisible edges wrap the six ports
    %% into bands. Without them every port sits on one row, which renders
    %% too wide for the content column and shrinks the labels. Keep them.
    source ~~~ sink
    decoder ~~~ storage
    expr ~~~ maxonce

ModuleCatalog (crates/blockwatcher-core/src/catalog.rs) is where that naming resolves: one name-keyed map per port family, each holding factory functions folded in from every compiled-in module’s own get_all() enumeration. A config that names a module absent from the catalog (a typo, or a module simply not compiled into this build) fails at boot or at write time with EngineError::UnknownModule, listing every alternative that actually is registered, via unknown and the family! macro’s lookup arm (catalog.rs), rather than panicking or silently no-op’ing that pipeline stage. Because the list comes from the catalog itself, the message always names what this particular binary carries, never a superset the workspace merely contains somewhere.

One module, one port, one name

A module never straddles two ports. A chain-family crate ships one Decoder and one or more Source implementations as separate modules, each registered under its own name and each satisfying exactly one port’s trait, never blending source and decode logic into a single type. Both kinds of module fold through build_catalog (crates/blockwatcher-embed/src/catalog.rs), each get_all() enumeration separately. Which module handles which resource is itself fixed by the port: a network names its Source module, a spec’s chain determines its Decoder, a sink resource names its Sink module, blockwatcher.toml names the one Matcher and one Storage backend for the whole process. An operator never picks a module without also picking, structurally, which port it fills.

Trade-offs are a property of the module, not a setting

Because two modules behind the same port are interchangeable at the trait level, they are free to differ arbitrarily in the trade-offs they make. blockwatcher leans on that rather than trying to expose every axis as a tunable knob on one do-everything implementation. The clearest illustration is two sources behind Source: one that only reports activity once it has a fixed chain position (and can therefore resume after a crash), against one that reports earlier at the cost of a cursor that is not a chain position. That pair, and every other chain-family trade-off, is documented on the family page, not as a setting on a shared source. evm-subgraph is a third acquisition module on the EVM family page, not a second registration path and not a knob on that rpc/mempool pair: it polls a documented GraphQL logs schema and emits the same confirmed-block envelopes evm-rpc does, covering only what that subgraph indexed.

The same pattern of “trade-off lives in which module you picked, not in a shared config surface” carries across the other five ports as well: the webhook sink trades a network dependency for delivering anywhere HTTP reaches; the log sink trades reach for having none; sqlite storage trades a single-writer constraint for surviving a restart, memory storage trades restart survival for zero setup; threshold trades per-hit delivery for a session digest that does not stall the checkpoint; max_once trades later in-window hits for a single alert.

The module catalog

Every row in the matcher, gate, sink, and storage table below is a module registered into ModuleCatalog by one of the get_all() calls build_catalog (crates/blockwatcher-embed/src/catalog.rs) folds together. Chain-family sources and decoders live on the family page. The running binary’s full list is GET /catalog.

Each source module’s family-page row states two facts: whether it reports a confirmed tip, and which cold-start field it moves (if any). GET /catalog advertises the tip half as caps.confirmed_tip on each source object.

The table below is exhaustive for matcher, gate, sink, and storage modules: every such module compiled into this binary appears here, and every row names something a real registration call actually produces.

Omit-gate is engine passthrough, not a catalog row an operator must name. Matcher is still one module for the whole process. Gate is per monitor, like sink.

Module selection is by name in configuration; a name the running binary never registered fails loudly, with the list of names it actually did register, rather than silently doing nothing.

Why this is the operator’s actual lever

Because a trade-off lives inside the module rather than in a shared knob, “choose your trade-offs” and “choose your modules” are the same action for an operator. Wanting a different point on a family’s latency-vs-completeness trade-off means naming a different source module on the network, not flipping a setting on one source that tries to serve both goals at once. The six-port boundary is what makes that swap safe to make at all: switching sources changes nothing about how the decoder, the matcher, a gate, or any configured sink behaves, because none of them were ever written against one source’s assumptions to begin with.