Architecture
Separation of Concerns
apiexec splits the problem of reliable API data retrieval into two independent layers:
Stream<T>- the execution layer. Handles retry, backoff, adaptive chunking, prefetch pipelining, cancellation, and stuck-cursor detection. Knows nothing about the specific API being called.VendorAdapter<T>- the API-specific layer. Knows how to build requests, parse responses, advance cursors, and decide whether an error is retryable. Knows nothing about execution strategy.
You configure the adapter; the stream engine orchestrates it. Swapping adapters - from Datadog v1 to a generic REST pager - requires no changes to the execution core.
System Diagram
┌─────────────────────────────────────────────────────────┐
│ User code - any language │
│ stream_create / stream_next / stream_destroy │
└────────────────────────┬────────────────────────────────┘
│ C linkage (extern "C")
┌────────────────────────▼────────────────────────────────┐
│ c_api/c_api.h (ABI freeze point) │
│ stream_create(adapter, config_json, policy_json) │
│ stream_has_next / stream_next_batch_v1 │
│ stream_next_v2_ts / stream_next_v2_sc │
│ stream_cancel / stream_destroy │
└────────┬──────────────────────────────┬─────────────────┘
│ │
▼ ▼
StreamRegistry ExecutionPolicy
resolves adapter controls retry, backoff,
name → concrete type chunk sizing, prefetch depth
│
▼
┌────────────────────────────────────────────────────────┐
│ ExecutionEngine<T> (generic C++ core) │
│ • double-buffered prefetch (std::async) │
│ • cursor advancement loop │
│ • retry + exponential backoff via ExecutionPolicy │
│ • stuck-cursor detection │
│ • graceful cancellation │
└──────────────┬─────────────────────────────────────────┘
│
▼
VendorAdapter<T>
build_request() → constructs HTTP request
parse_response() → decodes API response into T
is_retryable() → 429/5xx policy
retry_after() → honour Retry-After header
next_cursor() → advance pagination state
│
├── generic_rest (cursor-paginated REST)
├── datadog_metrics (Datadog Metrics API, time-window)
├── AIAdapter (base)
│ ├── openai (OpenAI Chat Completions)
│ └── anthropic (Anthropic Messages API)
└── (register your own via StreamRegistry)Prefetch Pipeline
The execution engine uses a double-buffered prefetch loop. While the
caller processes the current batch, the next HTTP request runs concurrently
in a background std::async task.
time → Caller: [process batch 0] [process batch 1] Engine: [ fetch batch 1 ] [ fetch batch 2 ]
This hides network latency almost entirely when batch processing time is shorter than fetch time - which is the common case for analytics workloads. Measured pipeline utilization is typically 45–55% on well-sized queries.
Prefetch depth
The default prefetch depth is 1 (one batch ahead). Increase it via
ExecutionPolicy.prefetch_depth in policy_json for workloads where
processing is fast relative to fetch latency.
The C ABI Boundary
The C++ core is entirely hidden behind a plain C header. This is the ABI freeze point: function signatures, struct layouts, and error code values are stable across library versions.
Language bindings call into this layer via their native FFI mechanism:
| Language | Binding mechanism | Notes |
|---|---|---|
| C | Direct call | No overhead |
| C++ | Direct call | Can also include the C++ headers directly |
| Python | ctypes or cffi | Ships as a wheel with bundled .so |
| Node.js | node-ffi-napi or N-API addon | |
| Go | cgo | |
| Java | JNI | |
| Rust | extern "C" block |
All bindings see the same semantics: one stream_create, one
stream_destroy, caller-owned buffers, and int32_t error codes.
Source Layout
The repository follows a strict layering rule: each directory may only depend on the directories above it.
codebase/
source/
core/ Zero-dep interfaces (Strategy + Template Method patterns)
transport/ CurlTransport behind ITransport
policy/ DefaultPolicy + CostAwarePolicy
adapters/ 4 vendor adapters + AIAdapter base + registry
c_api/ ABI-stable C surface (APIEXEC_ABI_VERSION=1)
examples/ Summarise example with cost budget
bindings/
go/ cgo wrapper + tests
rust/ Safe FFI wrapper + tests
python/ ctypes binding + tests
java/ JNA binding
js/ ffi-napi binding
tests/
unit/ 12 unit test suites
integration/ 5 integration tests + benchmark
fuzz/ libFuzzer targets for adapter parse paths
soak/ Configurable-duration stability testAdapter Interface
To write a custom adapter, implement a single C++ interface:
template <typename T>
class VendorAdapter {
public:
virtual Request build_request(const Cursor& cursor) = 0;
virtual ParseResult parse_response(const Response& resp, std::vector<T>& out) = 0;
virtual bool is_retryable(int http_status) = 0;
virtual Duration retry_after(const Response& resp) = 0;
virtual Cursor next_cursor(const Response& resp, const Cursor& current) = 0;
virtual bool is_exhausted(const Response& resp) = 0;
};Register the adapter with StreamRegistry::register_adapter("my-adapter", factory),
then pass "my-adapter" as the first argument to stream_create. The
execution engine handles everything else.