Message Bus

The Bus is an event-driven message bus for inter-workflow communication, external webhook delivery, and real-time integrations. Workflows publish messages to topics; subscriptions route those messages to pull queues, workflow triggers, or outbound webhooks. External systems can also publish inbound messages via HMAC-validated HTTP endpoints, without needing a JWT or workspace API key.

---

When to use

Use the Bus when you need to:

• Decouple two workflows so one does not directly call the other
• React to events from external systems (e.g. a payment service publishing payments/completed)
• Fan out a single event to multiple downstream workflows or webhooks
• Conditionally route messages based on their payload content
• Push events from ProvenanceOne workflows to external systems in real time

Use a direct workflow trigger (POST /workflows/{id}/trigger) instead of the Bus when you need synchronous, single-recipient delivery with a return value.

---

Key concepts

Topic — a hierarchical string with / as a separator. Examples: orders/created, payments/failed, alerts/security/high. Topics are not pre-declared; publishing to a topic creates it implicitly.

Message — the unit of data on the Bus. Key fields:

Field	Description
msgId	Unique message identifier
topic	The topic the message was published to
topicKey	Optional per-entity partition key
payload	Message body (JSON); large payloads stored in object storage via payloadRef if >400 KB
payloadRef	storage reference for payloads exceeding 400 KB
producerId	ID of the agent, workflow, or external system that published
producerKind	agent | workflow | system | webhook
correlationId	Optional ID to correlate related messages across runs
hopCount	Incremented on each delivery; used for loop detection
originRunId	Run ID of the originating workflow run
publishedAt	ISO 8601 timestamp
TTL	Message expiry

Subscription — a rule that matches messages on a topic and routes them to a delivery target.

Bus API keys — separate from workspace API keys. Bus API keys have publish and/or subscribe scopes scoped to specific topic patterns. Use bus API keys for external systems, not workspace API keys.

---

Delivery types

Pull (message queue)

Messages are placed in a message queue. The subscriber polls the queue using GET /bus/messages or the message queue endpoint directly. Use pull delivery when you need to batch-process messages or control consumption rate.

Workflow trigger

A matching message triggers a new workflow run. The message payload is passed as the run's input. This is the primary pattern for event-driven automation: publish orders/created → subscription triggers the order-processing workflow.

Webhook (outbound HTTP)

A matching message triggers an outbound HTTP POST from the busdeliver serverless function to a configured URL. The serverless function consumes from a message queue, so delivery is at-least-once. Use this to push ProvenanceOne events to external systems.

---

Topic filters and payload filters

Topic filter — subscriptions match topics using exact strings or wildcards. The * wildcard matches a single path segment:

• orders/created — exact match
• orders/* — matches orders/created, orders/updated, orders/shipped
• alerts/*/* — matches alerts/security/high but not alerts/high

Payload filter — an optional JSONPath expression evaluated against the message payload. Only messages where the expression evaluates to truthy are delivered. Example: $.amount > 100 delivers only messages where the amount field exceeds 100.

---

Inbound: external systems publishing to the Bus

External systems can publish to a subscription's inbound endpoint without a workspace JWT:

POST /bus/inbound/{subscriptionId}

Authentication is HMAC-SHA256. The external system signs the request body with the shared secret configured on the bus API key and includes the signature in the request. The platform validates the signature before accepting the message.

This endpoint is the primary integration point for event-driven ingestion from external services (e.g. a payment processor publishing payments/completed events).

---

Bus Schemas

Attach a JSON Schema to a topic to validate messages at publish time. If a message fails schema validation, it is rejected and a bus.message_delivery_failed event is emitted. Schema management endpoints: POST/GET/PATCH/DELETE /bus/schemas.

---

Circular loop protection

The hopCount field is incremented each time a message is delivered to a subscription. If hopCount exceeds the platform maximum, delivery is blocked and a bus.cycle_blocked audit event is emitted. This prevents infinite routing loops where workflow A publishes a message that triggers workflow B, which publishes a message that triggers workflow A.

Warning: Circular routing is a common mistake in event-driven architectures. Design topic naming and subscription filters to ensure messages produced by a triggered workflow do not match the subscription that triggered it. Use payload filters or distinct topic names to break cycles.

---

Bus as MCP (Bus MCP)

Agents can interact with the Bus directly via MCP tool calls through the Bus MCP interface:

• GET /bus/mcp/resources — lists available Bus topics and subscriptions as MCP resources
• POST /bus/mcp/resources/{name}/call — publishes a message or reads from a subscription

This allows agents to publish events or consume queued messages as part of their tool-use loop within a workflow, without requiring a separate HTTP integration.

---

How it works: Step-by-step example

Scenario: A payment service publishes orders/created events. An order-processing workflow should start for each new order, but only for orders over $500.

1. Configure a bus API key with publish scope on the orders/* topic pattern. Provide the key to the payment service.
2. Create a subscription with:
   • topicFilter: orders/created
   • payloadFilter: $.amount > 500
   • deliveryType: workflow
   • workflowId: the ID of your order-processing workflow
3. The payment service calls POST /bus/inbound/{subscriptionId} with the order payload, signed with the bus API key's HMAC secret.
4. ProvenanceOne validates the signature, evaluates the payload filter, and — if the amount exceeds 500 — triggers a new run of the order-processing workflow with the order payload as input.
5. Audit events emitted: bus.message_published, bus.message_delivered, run.started.

---

Configuration options

Subscription fields

Field	Type	Required	Default	Description
topicFilter	string	Yes	—	Exact topic or wildcard pattern (e.g. orders/*)
payloadFilter	string	No	—	JSONPath expression for conditional delivery
deliveryType	enum	Yes	—	pull | workflow | webhook
workflowId	string	Workflow only	—	Workflow to trigger on delivery
webhookUrl	string	Webhook only	—	Outbound HTTP endpoint
ttl	integer	No	—	Message TTL in seconds

Bus API key fields

Field	Type	Required	Default	Description
name	string	Yes	—	Display name
scopes	array	Yes	—	publish and/or subscribe
topicPattern	string	Yes	—	Topic pattern this key is scoped to

---

Common mistakes

• Using workspace API keys for external bus publishers. Bus API keys are separate and scoped to topic patterns. Workspace API keys should not be distributed to external systems. Create a dedicated bus API key with the minimum topic scope.
• Creating subscriptions without payload filters. Without a filter, every message on the topic triggers delivery. Add payload filters to avoid triggering workflows on irrelevant messages.
• Publishing from a triggered workflow back to the same topic. This creates a loop. The hopCount mechanism will eventually block it, but it is better to avoid the loop by design: use a different topic for the output messages, or add a payload filter that excludes messages produced by the workflow itself.
• Ignoring bus.message_delivery_failed events. These events indicate messages that were not delivered. Monitor this event type in your audit digest or alerting configuration.

---

Troubleshooting

Messages are not triggering the workflow — check the subscription topicFilter matches the exact topic published to, and that the payloadFilter (if set) evaluates to true for the message payload. Use GET /bus/messages/{msgId} to inspect the raw message.

bus.cycle_blocked events in the audit log — a routing loop has been detected. Review the workflow that triggered the message and adjust its output topic or add payload conditions to prevent re-triggering.

Inbound POST /bus/inbound/{subscriptionId} returns 401 — the HMAC signature is invalid. Verify the external system is signing the raw request body with the correct shared secret, using HMAC-SHA256.

Large payloads rejected — messages with payloads over 400 KB must use the payloadRef field. Store the payload in a datastore first and pass the storage reference in payloadRef rather than embedding the payload inline.

---

Security and permissions

Bus API keys are scoped per topic pattern and are separate from workspace API keys. Inbound endpoints use HMAC-SHA256 authentication, not JWT. Outbound webhook delivery does not include a workspace JWT in the outbound request — the external system receives the message payload only.

Audit events are emitted for all bus operations. High-volume topics can generate large audit volumes; configure your audit digest to filter to medium risk or above if needed.

Operation	Required scope / role
Publish via API	Bus API key with publish scope, or editor/admin
Manage subscriptions	editor or admin
Manage bus API keys	admin
View metrics	editor or admin
Manage schemas	editor or admin

---

Related pages

• Workflows
• Audit Log
• API Reference
• Security Model

---

FAQ