ApexKit Advanced Architecture, Internals, and Performance Optimization — ApexKit Docs

# ApexKit Advanced Architecture, Internals, and Performance Optimization This guide is a deep dive into the engineering, architecture, and internals of ApexKit. It is designed for platform engineers and advanced developers seeking to optimize, extend, and deploy ApexKit at enterprise scale. --- ## 1. System Architecture & Request Flow ApexKit is compiled as a unified, highly optimized Rust binary containing an embedded Axum HTTP/gRPC server, a Rust-native JS execution context (Boa), a full-text indexing engine (Tantivy), and concurrent SQLite connection pools. ### Flowchart: Internal Request Flow ```mermaid graph TD Client[HTTP / WebSocket Client] -->|1. Incoming Request| Axum[Axum Web Server] Axum -->|2. Scoping & Auth| AuthMiddleware[Auth & Scope Resolver Middleware] AuthMiddleware -->|3. Route Handlers| Router[API Router / GraphQL Controller] Router -->|4. If Hook / Script Triggers| Boa[Boa JS Engine Context] Boa -->|5. $db Namespaced API| DbBridge[Database Bridge] Router -->|4b. Standard CRUD / Query| DbBridge DbBridge -->|6. WAL Writes Queue| Batcher[WriteManager WAL Batching Queue] Batcher -->|7. Multi-DB Connection Pool| SQLite[(SQLite Files: core.db, data.db, logs.db, ...)] DbBridge -->|8b. Index Search Keys| Tantivy[Tantivy Full-Text Search Engine] DbBridge -->|8c. Match Distances| HNSW[HNSW Vector Search Index] ``` ### Request Processing Lifecycle: 1. **Axum Ingress**: The client connects via HTTP/WebSocket/SSE to the Axum engine. 2. **Scope Resolution**: The Scoping Middleware decodes the Authorization Bearer JWT. It extracts the `scope` claim (e.g. `tenant:client_alpha` or `sandbox:session_123`) and maps the request's connection paths and environment keys strictly to `/storage/tenants/client_alpha/` or `/storage/sandboxes/session_123/`. 3. **Interception (Boa Runtime)**: If database triggers or endpoint hooks are registered (e.g. `before_user_create`), Axum boots a lightweight, sandboxed `Boa` execution context. 4. **Storage Orchestration**: Operations are marshaled to the unified `Db` traits, multiplexing writes into WAL batching queues (`WriteManager`) and routing read queries to open connection pools. --- ## 2. Dynamic Storage Abstraction: Local vs. S3 ApexKit decouples file metadata from physical file storage, providing a unified storage manager capable of switching or migrating backends transparently. ```mermaid graph LR API[Storage API Controller] -->|Unified Interface| StorageBackend[StorageBackend Trait] StorageBackend -->|Local FS Driver| Local[Local File System storage/system/uploads/] StorageBackend -->|AWS SDK S3 Driver| S3[S3-Compatible Cloud Storage AWS, MinIO, Cloudflare R2] ``` ### Storage Backends: - **Local Filesystem Backend**: Saves uploaded files directly to scoped folders on disk (e.g., `storage/system/uploads/{filename}`). It handles instant thumbnail generation via the `image` and `webp` crates, caching output files in an in-memory `thumb_cache` (M