Content ingestion

Sellable product

This capability is granted by an API key scoped to the `` product (product face). See the product reference below.

Generic multi-protocol inbound fetch engine. Register a source (rss | api | social | bulk); the engine fetches it conditionally (etag/last-modified unchanged-skip), rate-limited per domain, with retry/backoff, a per-domain circuit breaker, backpressure, and a replayable DLQ. Fetched items are pulled as opaque RawItems (GET /v1/items) or POSTed to a per-source callback_url. Per-source credentials are resolved server-side (Infisical), never accepted in body or persisted plaintext.

• Group: Content & media
• Contract: contracts/content-ingestion/v1/openapi.yaml
• Public base: https://api.vagarylabs.com (the consolidated API gateway — one host, per-brand sibling api.<zone>)
• Auth: a product API key (vgk_…) issued from the console — Authorization: Bearer vgk_…
• Product face (customer-keyed):
  • https://api.vagarylabs.com/product/v1/ingestion/sources

Endpoints

Method	Path	Summary
GET	/health	Liveness + readiness (store connected, tenancy enforced, in-flight, backpressure)
GET	/metrics	Prometheus exposition
POST	/v1/sources	Register a source (rss
GET	/v1/sources	List sources (org-scoped)
GET	/v1/sources/{source_id}	Get one source (org-scoped; credentials_ref is never echoed)
PATCH	/v1/sources/{source_id}	Update mutable source fields (cadence/callback/credentials_ref/enabled/config) — owner-strict
DELETE	/v1/sources/{source_id}	Retire a source (owner-strict)
GET	/v1/sources/{source_id}/health	Adapter health for a source (reachability + latency)
POST	/v1/fetch	Submit an on-demand fetch job (by source_id OR an inline source) — async
GET	/v1/fetch/{job_id}	Poll a fetch job's status
GET	/v1/items	Pull fetched RawItems (cursor-paginated; content_base64 is opaque)
POST	/v1/dlq/{source_id}/replay	Replay dead-lettered fetches for a source
POST	/v1/kb/sources	Register + ingest a KB source (doc upload
GET	/v1/kb/sources	List KB sources (doc/web) for the caller's org
GET	/v1/kb/sources/{source_id}	Get a KB source
DELETE	/v1/kb/sources/{source_id}	Retire a KB source (remove its docs from the KB, then delete the source)
POST	/v1/kb/sources/{source_id}/sync	(Re-)ingest a source into the KB — the external scheduler calls this on cadence (auto-sync)
POST	/v1/kb/query	Query the tenant KB → citation-bearing results (retrieval include_citations=true)

Schemas

RegisterSourceRequest

Field	Type	Description
organization_id	string	product-tier tenant (I6); resolved from the verified identity/service token when omitted
type	string
url	string
cadence_seconds	integer	scheduled-tick interval per source
credentials_ref	string	Infisical ref for per-source credentials — never the secret itself
callback_url	string	opt-in outbound POST target for RawItem batches
config	object	adapter field-mapping (api/social) — items_path/title_key/url_key/published_key

RegisterSourceResult

Field	Type	Description
source_id	string
organization_id	string
type	string

UpdateSourceRequest

Field	Type	Description
cadence_seconds	integer
callback_url	string
credentials_ref	string
enabled	boolean
config	object

Source

Field	Type	Description
source_id	string
organization_id	string
type	string
url	string
cadence_seconds	integer
callback_url	string
enabled	boolean
last_fetch_at	string
last_status	string

AdapterHealth

Field	Type	Description
healthy	boolean
latency_ms	number
message	string

FetchRequest

Provide EITHER source_id (a registered source) OR an inline source {type, url, ...}.

Field	Type	Description
organization_id	string
source_id	string	a registered source to fetch
since	string	only items published at/after this
source	object	inline source (registered on-the-fly; disabled unless cadence_seconds is given)

FetchAccepted

Field	Type	Description
job_id	string
source_id	string
status	string

FetchJob

Field	Type	Description
job_id	string
source_id	string
status	string
items_count	integer
dlq_count	integer
error	string
created_at	string
updated_at	string

---

Generated by scripts/gen-capability-docs.py from contracts/content-ingestion/v1/openapi.yaml — the contract IS the source of truth; edit the contract, not this page.