Data — Datastores and Secrets

ProvenanceOne provides two distinct storage primitives: datastores for binary and structured objects (object-backed) and secrets for credentials and sensitive strings (backed by the secrets vault). They serve different purposes and have different access controls. Datastores are read and written by workflow steps; secrets are injected into agents, skills, and connections at runtime without being exposed in run outputs.

---

When to use

Use a datastore when you need to:

• Pass files, CSVs, JSON blobs, or other objects between workflow steps
• Store the output of one run so a later run can consume it
• Archive results for later retrieval or audit

Use a secret when you need to:

• Store an API key, password, token, or certificate that a skill or agent needs at runtime
• Rotate a credential on a schedule or on demand
• Provide a credential to a connection without embedding it in workflow configuration

Do not use a datastore to store secrets — datastores are not encrypted with the same access controls as the secrets vault. Do not use secrets for large objects — the secrets vault is optimised for short credential strings, not file storage.

---

Key concepts

Workspace scope — all datastores and secrets belong to exactly one workspace. Resources in workspace A are never accessible to workspace B.

Datastore — an object-backed object store. A workspace can have multiple datastores. Each datastore holds a flat namespace of objects addressed by key. Keys are path-like strings (e.g. reports/2026-05-01.csv) but the backing store is not a filesystem.

Secret — a name-value pair (or structured JSON) stored in the secrets vault. Secrets are referenced by ID in agent and skill configuration. The value is never returned in list or get API responses; it is only injected at runtime or via a deliberate reveal call (which is audited as a high-risk event).

Storage step — agents access datastores via the storage step kind inside a workflow. Agents do not have direct access to object storage or any database. The platform mediates all data access through step kinds defined in the workflow DAG.

Data quality — ProvenanceOne stores what you put in. Agents read what is stored. If source data is incomplete, stale, or malformed, agent output will reflect that. Validate and clean data before writing it to a datastore that agents will consume.

---

How it works

Datastores

1. Create a datastore via Data → New Datastore in the UI or POST /datastores. Provide a name and optional description.
2. Upload objects using POST /datastores/{id}/upload-url to get a pre-signed upload URL, then PUT the file directly to that URL.
3. In a workflow, add a storage step. Configure it to read from or write to the datastore by specifying the datastore ID and object key.
4. The agent step that follows the storage step receives the object content in its input context.
5. To retrieve an object outside a workflow, call GET /datastores/{id}/objects/{key+}.
6. To delete an object, call DELETE /datastores/{id}/objects/{key+}. This emits a datastore.object_deleted audit event.

Secrets

1. Create a secret via Data → Secrets → New Secret or POST /secrets. Provide a name and value.
2. Reference the secret by ID in an agent, skill, or connection configuration. The platform injects the value at runtime.
3. To rotate the secret value, call POST /secrets/{id}/rotate and supply the new value. This emits secret.rotated.
4. To reveal the raw value (e.g. for debugging), call POST /secrets/{id}/reveal. This is a high-risk operation and emits secret.accessed. It requires the secrets:read API key scope or admin role.
5. To delete a secret, call DELETE /secrets/{id}. Ensure no workflow or connection references it before deleting.

Warning: POST /secrets/{id}/reveal emits a secret.accessed audit event with risk level high. Every reveal is logged with the actor identity, timestamp, and IP. This cannot be suppressed.

---

Configuration options

Datastore fields

Field	Type	Required	Default	Description
name	string	Yes	—	Human-readable name for the datastore
description	string	No	—	Optional description

Secret fields

Field	Type	Required	Default	Description
name	string	Yes	—	Identifier used in agent/skill config
value	string	Yes	—	The credential or sensitive string; never returned in GET responses
description	string	No	—	Optional notes

---

Examples

Read a CSV from a datastore in a workflow

Add a storage step with:

• action: get
• datastoreId: the ID of your datastore
• key: reports/2026-05-01.csv

Map the step output to the next agent step's context.fileContent input field.

Upload a file via API

# 1. Get a pre-signed upload URL
curl -X POST https://api.provenanceone.ai/datastores/{id}/upload-url \
  -H "x-api-key: YOUR_KEY" \
  -d '{"key": "reports/2026-05-01.csv", "contentType": "text/csv"}'

# 2. PUT the file to the returned URL (no auth header — pre-signed URL handles auth)
curl -X PUT "PRESIGNED_URL" \
  -H "Content-Type: text/csv" \
  --data-binary @report.csv

Rotate a secret

curl -X POST https://api.provenanceone.ai/secrets/{id}/rotate \
  -H "Authorization: Bearer YOUR_JWT" \
  -d '{"value": "new-credential-value"}'

---

Common mistakes

• Storing secrets in datastore objects. If you write an API key to a datastore object, it is not protected by the secrets vault access controls. Use the secrets primitive for all credentials.
• Forgetting to update references after rotating a secret. After rotation, any hardcoded copies of the old value (e.g. in environment variables outside ProvenanceOne) will break. the secrets vault is the system of record.
• Uploading large files without using the pre-signed URL flow. The platform API has a request size limit. Always use POST /datastores/{id}/upload-url for objects larger than a few kilobytes.
• Not cleaning up unused datastores. Datastores count toward workspace storage. Delete datastores and objects that are no longer needed.

---

Troubleshooting

POST /datastores/{id}/upload-url returns 403 — confirm your API key includes the relevant write scope and that the datastore belongs to your workspace.

Secret value is undefined in a workflow run — check that the secret ID referenced in the agent or skill config matches an existing secret. Secret IDs are case-sensitive UUIDs.

POST /secrets/{id}/reveal is not available — this endpoint requires the secrets:read scope on your API key, or a JWT from an admin account. editor and viewer roles cannot reveal secrets.

Datastore object not found after upload — confirm the key used in the storage step exactly matches the key used in the upload URL request, including any path separators.

---

Security and permissions

Operation	admin	editor	viewer
Create / delete datastore	Yes	Yes	No
Upload / delete objects	Yes	Yes	No
Read objects	Yes	Yes	Yes
Create / update / delete secret	Yes	Yes	No
Reveal secret (secret.accessed)	Yes	No	No
Rotate secret	Yes	Yes	No

Secrets are stored in the secrets vault with at-rest encryption using platform-managed keys. Datastore objects are stored in object storage with at-rest encryption. All data is encrypted in transit using TLS.

See /docs/security for the full security model and /security/data-handling for data handling details.

---

Related pages

• Security
• Connections
• Workflows
• Audit Log

---

FAQ