diff --git a/.codewhale/state/subagents.v1.lock b/.codewhale/state/subagents.v1.lock new file mode 100644 index 0000000..e69de29 diff --git a/AGENTS.md.bak b/AGENTS.md.bak new file mode 100644 index 0000000..e63d6a3 --- /dev/null +++ b/AGENTS.md.bak @@ -0,0 +1,45 @@ +# WSG - WGPU Simple Graphics Library + +## Project Type +Rust workspace (2024 edition) wrapping [wgpu](https://github.com/gfx-rs/wgpu) for simple 3D drawing operations. + +## Workspace Structure +``` +Cargo.toml # workspace root — no dependencies here +lib/Cargo.toml # wsg-lib crate: wgpu 30.0.0, winit 0.30 +examples/Cargo.toml # depends on wsg-lib via path reference +lib/lib.rs # lib entry point +lib/context.rs # Context type (aggregates wgpu objects: Instance, Surface, Adapter, Device, Queue) +lib/renderer.rs # renderer implementation +examples/src/main.rs # example binary +``` + +**Key convention**: `wsg-lib` is referenced from `examples/` via relative path (`path = "../lib"`). Do not publish this to crates.io as-is — it uses a local path dependency. + +## Essential Commands +| Action | Command | +|--------|---------| +| Build everything | `cargo build --workspace` | +| Run examples | `cargo run -p examples` | +| Test | `cargo test --workspace` | +| Check | `cargo check --workspace` | +| Format | `cargo fmt --all` | + +No custom scripts or linting tooling beyond standard Cargo conventions. + +## Architecture Overview +The library's purpose is to abstract the five core wgpu objects into a single **Context**: + +- **Instance** — GPU backend selection (Vulkan/Metal/DX12) +- **Surface** — window rendering surface (via winit) +- **Adapter** — physical/logical GPU device +- **Device** — buffer/texture/pipeline creation +- **Queue** — command submission + +WGPU doesn't have a native "Context" object — this type groups them together for a simpler user API. See README.md for the French documentation of each component. + +## Gotchas +- Rust 2024 edition is used. Ensure your Rust toolchain supports it (`rustup update`). +- wgpu 30.0.0 is pinned in `lib/Cargo.toml`. The comment says "check the latest version" — verify compatibility before upgrading. +- No feature flags, no dev-dependencies, no tests yet. Adding any requires updating both `Cargo.toml` files if the dependency spans crates. +- The workspace has no `[workspace.dependencies]` section. Dependencies are declared per-crate rather than centrally. diff --git a/LEAN-CTX.md b/LEAN-CTX.md new file mode 100644 index 0000000..4ed5729 --- /dev/null +++ b/LEAN-CTX.md @@ -0,0 +1,50 @@ + +# lean-ctx — Context Engineering Layer + + +## Tool Mapping (MANDATORY — use instead of native equivalents) +| Instead of | Use | Example | +|------------|-----|---------| +| Read/cat/head/tail | `ctx_read(path, mode)` | `ctx_read("src/main.rs", "full")` | +| Grep/rg/find | `ctx_search(pattern, path)` | `ctx_search("fn handle", "src/")` | +| Shell/bash | `ctx_shell(command)` | `ctx_shell("cargo test")` | +| Edit (when Read unavailable) | `ctx_edit(path, old, new)` | `ctx_edit("f.rs", "old", "new")` | + +## ctx_read Mode Selection +| Goal | Mode | When | +|------|------|------| +| Edit this file | `full` | Before any edit | +| Understand API | `signatures` | Context-only, won't edit | +| Re-read after edit | `diff` | Post-edit verification | +| Large file overview | `map` | >500 lines, won't edit | +| Specific region | `lines:N-M` | Know exact location | +| Unsure | `auto` | System selects optimal mode | + +## Workflow (follow this order) +1. **Orient:** `ctx_overview(task)` or `ctx_compose(task, path)` for unfamiliar tasks +2. **Locate:** `ctx_search(pattern, path)` for exact text; `ctx_semantic_search(query)` for concepts +3. **Read:** `ctx_read(path, mode)` with appropriate mode from table above +4. **Edit:** `ctx_edit(path, old_string, new_string)` or native Edit if available +5. **Verify:** `ctx_read(path, "diff")` + `ctx_shell("test command")` +6. **Record:** `ctx_knowledge(action="remember", content="...")` for non-obvious findings + +## Proactive (use without being asked) +- `ctx_overview(task)` — at session start for orientation +- `ctx_compress` — when context grows large (at phase boundaries) +- `ctx_knowledge(action="wakeup")` — at session start to surface prior findings + +## Compression Bypass (only when compressed output hides needed detail) +`ctx_read(path, "lines:N-M")` → `ctx_read(path, "full")` → `ctx_shell(cmd, raw=true)` +Return to compressed defaults after one expanded retrieval. + +## Risk Gate (before high-impact edits) +Before editing exported symbols, auth, DB schemas, or 3+ files: run `ctx_impact(action="analyze")` +and `ctx_callgraph(action="callers")` to confirm blast radius. + +## Session +- **Start:** `ctx_session(action="status")` + `ctx_knowledge(action="wakeup")` +- **End:** `ctx_session(action="decision", content="what was done + next steps")` +- **On [CHECKPOINT]:** `ctx_session(action="task", value="current status")` + +NEVER use native Read/Grep/Shell when ctx_* equivalents are available. + diff --git a/docs/rules/DOCUMENTATION.md b/docs/rules/DOCUMENTATION.md new file mode 100644 index 0000000..09b0277 --- /dev/null +++ b/docs/rules/DOCUMENTATION.md @@ -0,0 +1,44 @@ +--- +type: Reference +title: IAgent Documentation Rules +description: Rules and guidelines for documentation in the IAgent project, following OKF v0.2 specification +tags: [documentation, guidelines, standards] +status: stable +generated: { by: human:jerome, at: 2026-07-31T00:00:00Z } +--- + +# IAgent Documentation Rules + +## Language + +All documentation is written in English; as a convention, the code itself uses English for variable names, function names, etc. + +## Code Documentation + +Every source file and configuration file must be systematically documented following the rules defined in this DOCUMENTATION.md file. Additionally, every directory must contain its own README.md file summarizing and explaining the module's organization at that level: what is the overall responsibility of the files grouped in this directory, which ones they are, and what each one does. + +We assume the reader has professional algorithmic knowledge but may not necessarily be a Rust specialist. The reader does know the project's domain — LLM logic, clients, and agents. Documentation should therefore be tailored for a professional developer who knows some programming languages (not necessarily Rust). + +### General Rule + +Documentation must describe what is coded and what purpose it serves. A LLM reading the code and documentation should be able to verify whether: +- the documentation correctly describes what the code does, +- the code doesn't do something other than what the documentation says. + +### File Headers + +Each module or file must include documentation explaining the module's responsibility and how it interacts with other modules in the program, at least those within its own directory. This documentation must detail the main objects (Struct, Enum, Trait) manipulated in the module and the primary functions that carry the module's core logic. + +## Within a File's Code + +### Object and Function Headers + +At the header of each object, describe what the object represents and its purpose. At the header of each function, describe what the function does, what inputs it expects, and what it returns. Also describe when or by whom it is typically called. This part of the description should fit within three lines maximum. If the function body exceeds fifteen lines of code, also add its internal steps and how it accomplishes them, in three lines maximum. If the function has points of attention or complex technical resolutions (such as a specific Rust idiom or library trick for solving an ownership or lifetime problem), these are documented after the function header in up to three lines of explanation. + +### In the Code Body + +If an object or function presents a particularity or specific technical point, then a descriptive comment is inserted directly into the code body or function body. If a point of attention or technical point was described in the function header, then a comment in the code body reminds where this point is located. + +## Documentation Maintenance + +The rules defined in this file are regularly applied across all code documentation to ensure consistency between code evolution and its documentation. diff --git a/docs/rules/SPEC.md b/docs/rules/SPEC.md new file mode 100644 index 0000000..a516d50 --- /dev/null +++ b/docs/rules/SPEC.md @@ -0,0 +1,1003 @@ +# Open Knowledge Format (OKF) + +**Version 0.2** + +OKF is an open, human- and agent-friendly format for representing +*knowledge*: the metadata, context, and curated insight that surrounds +data and systems. It is designed to be authored by people, generated by +agents, exchanged across organizations, and consumed by both. + +The format is intentionally minimal: a directory of markdown files with +YAML frontmatter. There is no schema registry, no central authority, and +no required tooling. If you can `cat` a file, you can read OKF; if you +can `git clone` a repo, you can ship it. + +This document is self-contained: it specifies everything needed to +produce and consume OKF v0.2. A summary of what changed from v0.1 is in +§13. + +--- + +## 1. Motivation + +The space of knowledge representation for AI agents is evolving quickly, +and many incompatible conventions are emerging. OKF takes the position +that knowledge is best represented in commonly accessible, established +formats that are: + +- **Readable** by humans without tooling. +- **Parseable** by agents without bespoke SDKs. +- **Diffable** in version control. +- **Portable** across tools, organizations, and time. + +Increasingly, a knowledge corpus is not authored once and then read: it +is **continuously written and maintained by agents**. When most concepts +are machine-generated, a consumer needs answers that a plain +markdown-plus-frontmatter convention does not make first-class: + +1. What was this created from, and how was it verified? (**provenance**) +2. How much should I trust it? (**trust**) +3. Is it still true? (**freshness**) +4. Is it the current version? (**lifecycle**) +5. Was this number produced the way we said it must be? (**attestation**) + +OKF v0.2 makes provenance, trust, lifecycle, and attestation first-class +while keeping the format minimally opinionated. The format is minimally +opinionated. It standardizes only the small set of structural conventions +needed to make a knowledge corpus self-describing — anything beyond that +is left to the producer. + +### Goals + +1. Define a universal format that **producers** (people, agents, export + pipelines) can write into. +2. Inform how **consumers** (agents, UIs, search indexes, deterministic + code) should read and traverse it. +3. Facilitate **exchange** of knowledge across systems and organizations. +4. Standardize the small set of frontmatter fields that make an + agent-maintained corpus **trustable**, without prescribing any runtime. + +### Non-goals + +- Defining a fixed taxonomy of concept types. +- Prescribing storage, serving, or query infrastructure. +- Replacing domain-specific schemas (Avro, Protobuf, OpenAPI, and so on). + OKF *references* them; it does not subsume them. +- Specifying a packaging or invocation standard for the code an executor + or attester points at. OKF fixes the interface, not the packaging. + +--- + +## 2. Terminology + +- **Knowledge Bundle** (or **bundle**): A self-contained, hierarchical + collection of knowledge documents. The unit of distribution. +- **Concept**: A single unit of knowledge within a bundle, represented as + one markdown document. It may describe a tangible asset (a table, an + API), an abstract idea (a metric, a business process), or anything in + between. +- **Concept ID**: The path of the concept's file within the bundle, with + the `.md` suffix removed. +- **Frontmatter**: A YAML metadata block delimited by `---` at the top of + a markdown file. +- **Body**: Everything in the file after the frontmatter. +- **Link**: A standard markdown link from one concept to another, used to + express relationships beyond the implicit parent/child hierarchy. +- **Source**: A material a concept derives from, external or internal to + the bundle, recorded in the `sources` frontmatter field. +- **Provenance**: The set of sources a concept derives from. +- **Credibility signal**: An objective, per-source fact (`author`, + `usage_count`, `last_modified`) used to infer trust; OKF records the + signals, not a verdict (see §5.1). +- **Actor**: A string identifying who or what performed an action, using + the convention `/` for agents, `human:` for + people, and `process:` for automated processes (see §7). +- **Trust tier**: A level derived from a concept's `verified` field: + unverified, machine-confirmed, or human-reviewed (see §5.3). +- **Attested Computation**: A concept (`type: Attested Computation`) + carrying a sanctioned way to compute a value, so a consumer can confirm + the value was produced by running it (see §10). +- **Executor**: Run instructions or code that executes a computation and + returns a receipt (see §10.2). +- **Receipt**: The evidence a run returns, shaped by `executor.receipt`; a + runtime artifact, not stored in the bundle (see §10). +- **Attester**: Deterministic (no-LLM) code that inspects a receipt and + returns a verdict (see §10.2). + +--- + +## 3. Bundle structure + +A bundle is a directory tree of markdown files. The directory structure +is independent of the domain: producers organize concepts however makes +sense for the knowledge being captured. + +``` +path/to/bundle/ + index.md # Optional. Directory listing for progressive disclosure. + log.md # Optional. Chronological history of updates. + .md # A concept at the bundle root. + / # Subdirectories organize concepts into groups. + index.md + .md + / + ... +``` + +A bundle MAY be distributed as: + +- A git repository (recommended, since it provides history, attribution, + and diffs). +- A tarball or zip archive of the directory. +- A subdirectory within a larger repository. + +### 3.1 Reserved filenames + +The following filenames have defined meaning at any level of the +hierarchy and MUST NOT be used for concept documents: + +| Filename | Purpose | +|------------|----------------------------------| +| `index.md` | Directory listing. See §8. | +| `log.md` | Update history. See §9. | + +All other `.md` files are concept documents. + +Tags remain a first-class concept through the `tags` frontmatter field +(§4.1). OKF does not specify a separate file format for aggregating +documents by tag; a consumer that wants a tag-browsing view can +synthesize one at consumption time by scanning frontmatter. + +--- + +## 4. Concept documents + +Every concept is a UTF-8 markdown file with two parts: + +1. A **YAML frontmatter block**, delimited by `---` on its own line at the + start of the file and a closing `---` on its own line. +2. A **markdown body**, containing free-form content. + +### 4.1 Frontmatter + +```yaml +--- +type: # REQUIRED +title: +description: +resource: +tags: [, , ...] # Optional +# ... trust, lifecycle, provenance, and computation families (see §5, §10) +# ... other producer-defined key/value pairs +--- +``` + +**Required:** + +- `type`: A short string identifying the kind of concept. Consumers use it + for routing, filtering, and presentation. Example values: + `BigQuery Table`, `BigQuery Dataset`, `API Endpoint`, `Metric`, + `Playbook`, `Reference`, `Attested Computation`. + + Type values are **not** registered centrally. Producers SHOULD pick + values that are descriptive and self-explanatory; consumers MUST + tolerate unknown types gracefully, typically by treating them as generic + concepts. + +`type` is the only always-required key; a concept carrying just `type` is +fully conformant (§11). + +**Recommended:** + +- `title`: Human-readable display name. If omitted, consumers MAY derive a + title from the filename. +- `description`: A single sentence summarizing the concept. Used by + `index.md` generators, search snippets, and previews. +- `resource`: A URI that uniquely identifies the underlying asset the + concept describes. Absent for concepts that describe abstract ideas + rather than physical resources. +- `tags`: A YAML list of short strings for cross-cutting categorization. + +The optional **provenance**, **trust**, and **lifecycle** families (§5) and +the **computation** fields for Attested Computation concepts (§10) may also +appear. + +**Extensions:** Producers MAY include any additional keys. Consumers +SHOULD preserve unknown keys when round-tripping and MUST NOT reject +documents with unrecognized fields. + +### 4.2 Body + +The body is standard markdown. Producers SHOULD favor structural markdown +(headings, lists, tables, fenced code blocks) over freeform prose, since +structure aids both human reading and agent retrieval. + +There are no required body sections. The following headings have +**conventional** meaning and SHOULD be used when applicable: + +| Heading | Purpose | +|-----------------|--------------------------------------------------------| +| `# Schema` | Structured description of an asset's columns/fields. | +| `# Examples` | Concrete usage examples, often as fenced code blocks. | +| `# Computation` | The sanctioned computation of an Attested Computation. See §10. | + +Per-claim attribution to external sources uses markdown footnotes keyed to +`sources` entries rather than a body citations list (§5.1). + +### 4.3 Example: a concept bound to a resource + +```markdown +--- +type: BigQuery Table +title: Customer Orders +description: One row per completed customer order across all channels. +resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders +tags: [sales, orders, revenue] +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-05-28T14:30:00Z } +--- + +# Schema + +| Column | Type | Description | +|---------------|-----------|------------------------------------------| +| `order_id` | STRING | Globally unique order identifier. | +| `customer_id` | STRING | Foreign key into [customers](/tables/customers.md). | +| `total_usd` | NUMERIC | Order total in US dollars. | +| `placed_at` | TIMESTAMP | When the customer submitted the order. | + +# Joins + +Joined with [customers](/tables/customers.md) on `customer_id`. +``` + +### 4.4 Example: a concept not bound to a resource + +```markdown +--- +type: Playbook +title: "Incident response: data freshness alert" +description: Steps to triage a freshness alert on the orders pipeline. +tags: [oncall, incident] +generated: { by: human:ahormati, at: 2026-04-12T09:00:00Z } +--- + +# Trigger + +A freshness alert fires when `orders` lags more than 30 minutes behind its +expected SLA. See the [orders table](/tables/orders.md). + +# Steps + +1. Check the [ingestion job dashboard](https://example.com/dash). +2. ... +``` + +--- + +## 5. Provenance, trust, and lifecycle + +These frontmatter families make "where did this come from," "how much +should I trust it," and "is it still current" answerable from frontmatter. +All are optional. Their absence carries meaning: an unverified concept is +distinguishable from a verified one, but is never rejected (§11). + +### 5.1 Provenance: `sources` + +`sources` records the materials a concept derives from, external or +internal to the bundle. + +```yaml +sources: + - id: ga4-schema + resource: https://developers.google.com/analytics/bigquery/export-schema + title: GA4 BigQuery Export schema + author: team:ga4-docs + usage_count: 5000 + last_modified: 2026-05-30 +usage_window: { from: 2026-06-01, to: 2026-06-30 } +``` + +Each `sources` entry: + +- `resource`: REQUIRED within an entry. Names either a concrete artifact a + consumer can follow (an absolute URL, a bundle-relative path, or a path + into a `references/` subdirectory, §6) or a population or scope descriptor + it cannot (for example `all queries in BigQuery project X`). +- `id`: Optional. A stable key used to attribute individual claims (see + below). SHOULD be present when the body cites the source. +- `title`: Optional. Human-readable label for the source. +- The optional credibility signals `author`, `usage_count`, and + `last_modified`, described next. + +**Source credibility signals.** OKF records objective, per-source signals +so a consumer can judge how much to trust a concept by judging the sources +it was extracted from. It does not store a credibility score: a score is +subjective, unportable across consumers, and goes stale. Credibility is +*inferred* from the signals, the same way trust tiers are (§5.3), not +stored. Each signal is optional and lives on a `sources` entry: + +- `author`: Who or what produced the source, in the actor convention (§7). + An authority signal. +- `usage_count`: How often `resource` was exercised (dashboard views, query + executions, page reads) over `usage_window`. An adoption and liveness + signal. For a single artifact it is that artifact's own exercise count; + for a scope descriptor it is the number of exercises within the scope that + touch the concept. +- `last_modified`: When the source itself last changed (`YYYY-MM-DD`). A + recency signal, distinct from `generated.at` (§5.2), which records when + the concept was written. +- `usage_window`: Written once as a sibling of `sources`, it frames every + `usage_count` with a `{ from, to }` date range. A single entry MAY carry + its own `usage_window` to override the shared one. + +`usage_count` is a coarse signal. It is comparable at the +alive-versus-dead and order-of-magnitude level, and against a source's own +history over time, but not as a precise cross-kind ranking: a scheduled +query's executions and a human's deliberate dashboard views do not carry +equal weight. Consumers SHOULD read it as liveness and trend, not as a +score. + +Lineage is expressed through links, not a dedicated field. When a +`resource` points at another OKF concept, the derivation edge already +exists in the bundle graph (§6), so a consumer MAY recurse into that +source's own `sources` and let credibility propagate. External leaf sources +carry only their intrinsic signals. Deeper lineage (an explicit external +`derived_from`, or data lineage) is out of scope for v0.2. + +**Per-claim attribution.** To attribute a specific claim, use a markdown +footnote whose label is a `sources[].id`: + +```markdown +The `events_` table is sharded daily as `events_YYYYMMDD`.[^ga4-schema] + +[^ga4-schema]: GA4 BigQuery Export schema +``` + +The footnote label is the join key into `sources`; consumers resolve +attribution through the matching entry, not by parsing the footnote prose. +Labels are keyed rather than positional (`sources[0]`) because agents +constantly rewrite these documents: a positional index misattributes +silently the moment the list is reordered, whereas a stable `id` survives +reordering. + +### 5.2 Trust: `generated` and `verified` + +`generated` records how the current content was produced. `verified` +records who or what has confirmed the content against its sources or +`resource`. They are kept distinct because who *wrote* a concept need not +be who *confirmed* it. + +```yaml +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z } +``` + +- `generated.by`: REQUIRED within `generated`. An actor (§7). +- `generated.at`: An ISO 8601 datetime marking the content's last + meaningful change. Consumers use it to tell a recent edit from a stale + fact. + +```yaml +verified: + - { by: human:ahormati, at: 2026-06-25T09:00:00Z } + - { by: process:finance-nightly, at: 2026-06-26T02:00:00Z } +``` + +- `verified`: A list of verification events, each with `by` (an actor) and + `at` (an ISO 8601 datetime). Multiple entries capture independent + checks, for example a human sign-off plus a nightly process. "How + recently" is the latest `at`. +- `verified` is independent of `generated.at`: content can change without + re-confirmation, and facts can be re-confirmed without regeneration. +- A single verifier MAY be written as one `{ by, at }` mapping without the + list dash. Consumers MUST treat a bare mapping as a one-element list: + +```yaml +verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z } +``` + +### 5.3 Trust tiers + +Consumers derive a trust tier from `verified`, lowest to highest: + +- No `verified` key ⇒ **unverified**. +- `verified` by non-`human:` actors only ⇒ **machine-confirmed**. +- `verified` by a `human:` actor ⇒ **human-reviewed**. + +A concept with no trust frontmatter is still consumable; consumers MUST +NOT reject it (§11). Trust tiers are advisory signals, not access control. + +### 5.4 Lifecycle: `status` + +```yaml +status: stable # draft | stable | deprecated +``` + +- `draft`: not yet reviewed; possibly incomplete. +- `stable`: default; ready for consumption. +- `deprecated`: kept for links and history; no longer current. + +Absent `status` ⇒ `stable`. + +### 5.5 Lifecycle: `stale_after` + +```yaml +stale_after: 2026-09-23 # absolute date; content is stale on/after this day +``` + +Optional. An absolute date (`YYYY-MM-DD`). A concept is stale when +`today >= stale_after`. An absolute date, not a relative TTL, keeps the +staleness decision a plain date comparison with no reference to when the +concept was read. + +--- + +## 6. Cross-linking and paths + +### 6.1 Links between concepts + +Concepts MAY link to other concepts using standard markdown links. Two +forms are supported: + +- **Absolute (bundle-relative):** begins with `/`, interpreted relative to + the bundle root. This is the **recommended** form because it is stable + when documents are moved within their subdirectory. + + ```markdown + See the [customers table](/tables/customers.md) for the join key. + ``` + +- **Relative:** a standard markdown relative path. + + ```markdown + See the [neighboring concept](./other.md). + ``` + +A link from concept A to concept B asserts a *relationship*. The specific +kind (parent/child, references, joins-with, depends-on) is conveyed by the +surrounding prose, not by the link itself. Consumers that build a graph +view typically treat all links as directed edges of an untyped +relationship. + +Consumers MUST tolerate broken links: a link whose target does not exist +in the bundle is not malformed; it may simply represent not-yet-written +knowledge. + +### 6.2 Path-valued fields + +Several fields name a path or URI: `resource`, `sources[].resource`, +`computation`, `executor.resource`, and `attester.resource` (§10). A +`sources[].resource` may instead be a scope descriptor (§5.1), in which +case it is not a path. Each path-valued field accepts: + +- an absolute URL (for example `https://...`), +- a bundle-relative path beginning with `/`, or +- a relative path (for example `../computations/revenue.md`). + +### 6.3 The `references/` convention + +A `references/` subdirectory conventionally mirrors external material, run +instructions, or code as first-class concepts within the bundle. Sources, +executors, and attesters commonly point into it (for example +`references/attesters/revenue.py`). It is a naming convention, not a +requirement. + +--- + +## 7. Actor convention + +Fields that record an identity (`generated.by`, `verified[].by`) use a +single actor convention: + +- `/` for agents and tools, for example + `reference_agent/gemini-2.5-pro`. +- `human:` for a person, for example `human:ahormati`. +- `process:` for an automated process, for example + `process:finance-nightly`. + +Consumers that classify trust (§5.3) key off the `human:` prefix, so +producers MUST use it for hand-authored or human-confirmed content. + +--- + +## 8. Index files + +An `index.md` file MAY appear in any directory, including the bundle root. +It enumerates the directory's contents to support **progressive +disclosure**: letting a human or agent see what is available before +opening individual documents. + +Index files contain no frontmatter, with one exception: a bundle-root +`index.md` MAY carry an `okf_version` key (§12). The body uses one or more +sections, each grouping concepts under a heading: + +```markdown +# Section / Group Heading + +* [Title 1](relative-url-1) - short description of item 1 +* [Title 2](relative-url-2) - short description of item 2 + +# Another Section + +* [Subdirectory](subdir/) - short description of the subdirectory +``` + +Entries SHOULD include the description from the linked concept's +frontmatter. Producers MAY generate `index.md` automatically; consumers +MAY synthesize one on the fly when none is present. + +--- + +## 9. Log files + +A `log.md` file MAY appear at any level of the hierarchy to record the +history of changes to that scope. The format is a flat list of +date-grouped entries, newest first: + +```markdown +# Directory Update Log + +## 2026-05-22 +* **Update**: Added a BigQuery table reference for [Customer Metrics](/tables/customer-metrics.md). +* **Creation**: Established the [Dataplex Playbook](/playbooks/dataplex.md). + +## 2026-05-15 +* **Initialization**: Created foundational directory structure. +``` + +Date headings MUST use ISO 8601 `YYYY-MM-DD` form. Log entries are prose; +the leading bold word (`**Update**`, `**Creation**`, `**Deprecation**`) is +a convention, not a requirement. + +--- + +## 10. Attested computations concept + +An Attested Computation concept carries not just what a value *means* but a +sanctioned way to *compute* it, so a consumer can confirm the agent ran the +blessed computation instead of improvising its own. Provenance (§5.1) +answers "where did this claim come from"; attestation answers "was this +number produced the way we said it must be." OKF records the computation +and the means to check it; it does not execute anything itself. + +### 10.1 A computation is its own concept + +A sanctioned computation is a standalone concept of +`type: Attested Computation`. A concept that needs the value (a `Metric`, a +`BigQuery Table`) links to it with a normal markdown link (§6). Three +properties motivate the standalone concept: + +- **`runtime` defines what `parameters` mean.** A parameter is a SQL bind + variable, a dbt var, or a Python argument depending on the runtime. + Keeping `runtime` and `parameters` in one frontmatter makes the binding + semantics self-evident. +- **One computation, many consumers.** The same computation can back a + metric, a dashboard concept, and a report; as a concept it is referenced + once and reused. +- **Trust state is per computation.** `verified`, `stale_after`, and a + single `attester` describe one thing. Revenue, profit, and margin each + verify and attest independently, which is three concepts, not three + entries in one frontmatter. + +### 10.2 Contract fields + +The contract is the concept's top-level frontmatter. In addition to the +provenance, trust, and lifecycle families (§5), an Attested Computation +concept carries: + +- `runtime`: REQUIRED for this type. The single field that says how to run + the computation, and so how the executor and attester interpret it and + what `parameters` mean. Example values: `bigquery`, `postgres`, `dbt`, + `python`, `Looker`. +- `parameters`: A list of the typed, named holes the agent may fill. Each + entry: `{ name, type, required }`. Binding semantics follow `runtime`. +- `computation`: Optional. A path (§6.2) to a file holding the + computation, used instead of an inline body fence (see §10.3). Absent ⇒ + the body `# Computation` fence is the computation. +- `executor`: How the computation is run. `resource` names run + instructions or code; a runner (an agent, or deterministic consumer + code) follows it. `receipt` declares the fields a run must return, the + evidence the attester inspects (for example a BigQuery `job_id` and the + SQL the job actually executed). +- `attester`: The deterministic check. `resource` names code (no LLM) that + takes a receipt and returns a verdict. It is meant to run consumer-side. + +What sits behind a `resource` (a Skill, a script, a container) is a +packaging choice; OKF fixes the interface, not the packaging (§1). + +```markdown +--- +type: Attested Computation +title: Revenue for fiscal year +description: Recognized revenue for a fiscal year, per Finance's definition. +status: stable +runtime: bigquery +parameters: + - { name: year, type: integer, required: true } +executor: + resource: references/skills/run-on-bq.md + receipt: [job_id, executed_sql, result] +attester: + resource: references/attesters/revenue.py +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z } +verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z } +stale_after: 2026-09-23 +sources: + - id: rev-policy + resource: https://wiki.acme/finance/revenue-recognition + title: Revenue recognition policy +--- + +# Computation + + SELECT SUM(amount) AS revenue + FROM finance.recognized_revenue + WHERE fiscal_year = @year + +The computation binds only the declared `parameters`, per the recognition +policy.[^rev-policy] + +[^rev-policy]: Revenue recognition policy +``` + +### 10.3 The computation + +Provide the computation in one of two ways: + +- **Inline:** a single fenced code block in the body under `# Computation`. + Best for a short computation reviewed alongside the contract. +- **File:** set `computation` to a path (§6.2) and omit the body fence. + Best for a long or generated computation, or one already kept as a real + file shared with non-OKF tooling. + +```yaml +runtime: bigquery +computation: references/computations/lib/revenue.sql +parameters: + - { name: year, type: integer, required: true } +``` + +The agent MAY only supply *values* for the declared `parameters`; it MUST +NOT author or edit the computation. Binding `computation` with the +parameter values into the executable artifact is the consumer's job, and +the attester independently re-derives that same binding to compare against +what actually ran. Because the comparison is on the expanded, compiled +artifact the receipt carries (`executed_sql`, `compiled_sql`), a rewritten +query, a swapped computation file, or a mutated dependency fails the check. +A typed, parameter-only surface is what makes "did the sanctioned thing +run" a mechanical comparison rather than a judgement call. + +### 10.4 Concepts that use a computation + +A document is rarely a single computation. An income-statement overview +that discusses revenue, profit, and margin stays one readable concept and +links to one Attested Computation per figure: + +```markdown +--- +type: Metric +title: Revenue +description: Recognized revenue for a fiscal year. +tags: [finance, revenue] +status: stable +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z } +--- + +# Definition + +Recognized revenue sums `amount` over rows booked to the fiscal year, +computed by [the revenue computation](../computations/revenue.md). +``` + +Because each computation is its own concept, revenue can be fresh while +profit is past its `stale_after`, and each attests on its own run. +Co-locating them is a directory choice (a `computations/` folder with an +`index.md`), not a frontmatter one. + +### 10.5 How a consumer uses it (informative) + +This subsection is informative, not normative. The runtime artifacts below +are **not** stored in the bundle. + +1. **Discover** via `type: Attested Computation`, a frontmatter signal + liftable into `index.md`; a consumer reaches one directly or by + following a link from a concept that uses it. +2. **Load** the contract from frontmatter and the computation from the + body (or the file named by `computation`). +3. **Parameterize**: the agent supplies values for the declared parameters. +4. **Execute**: the executor runs the bound computation and returns a + receipt shaped by `executor.receipt`. +5. **Attest**: the consumer runs the attester over the receipt. It + confirms provenance (the computation that ran equals `computation` bound + with the claimed parameters, not agent-authored SQL) and fidelity (the + displayed value matches the receipt's authoritative source, re-read by + job id rather than taken from the agent's text). +6. **Gate**: refuse to display a failing attestation; warn or refuse when + `today >= stale_after`. On success, surface the verdict (for example a + link to the job log) so trust is visible. + +### 10.6 Verification versus attestation + +`verified` (§5.2) and attestation are distinct, and both exist: + +- `verified` confirms the *definition* still matches policy. It is + doc-level, slow, and recorded in the bundle. +- Attestation confirms a single *run* produced the value the sanctioned + way. It is per-call, runtime, and not stored in the bundle. + +A concept with a stale definition can still attest cleanly, and a +freshly-verified definition still requires attestation on each run, which +is why both are needed. + +--- + +## 11. Conformance + +A bundle is **conformant** with OKF v0.2 if: + +1. Every non-reserved `.md` file in the tree contains a parseable YAML + frontmatter block. +2. Every frontmatter block contains a non-empty `type` field. +3. Every reserved filename (`index.md`, `log.md`) follows the structure in + §8 and §9 respectively when present. + +When the trust, lifecycle, provenance, or computation families are +present, producers SHOULD follow §5 through §10, and consumers: + +- MUST treat a bare `verified` mapping as a one-element list (§5.2). +- MUST NOT reject a concept for missing any optional family (§5.3). +- SHOULD derive trust tiers and staleness only from the fields specified + here, and SHOULD surface, not silently drop, a failing attestation + (§10.5). + +Consumers SHOULD treat all other constraints as soft guidance. In +particular, consumers MUST NOT reject a bundle because of: + +- Missing optional frontmatter fields. +- Unknown `type` values. +- Unknown additional frontmatter keys. +- Broken cross-links. +- Missing `index.md` files. + +--- + +## 12. Versioning + +This document specifies OKF version **0.2**. Revisions are versioned as +`.`: + +- A **minor** version bump introduces backward-compatible additions (new + optional fields, new conventional section headings). +- A **major** version bump may make breaking changes (renaming required + fields, changing reserved filenames). + +Bundles MAY declare the version they target with `okf_version: "0.2"` in a +bundle-root `index.md` frontmatter block (the only place frontmatter is +permitted in an `index.md`). Consumers that do not understand the declared +version SHOULD attempt best-effort consumption rather than refusing the +bundle. + +### Considered and deferred + +The following are intentionally left to a future revision: + +- The full runtime protocol: receipt and verdict wire formats, and the + attestation lifecycle around a run. +- The attester ABI, portability, and sandboxing, likely bundled with + future work on serving and Skills. +- Attestation caching. +- Semantic-layer templates (Looker, dbt) where the attester comparison + shifts from SQL equality to model-and-binding equality. + +--- + +## 13. Changes from v0.1 + +v0.2 supersedes OKF v0.1 and is a minor version bump under §12, except for +two deliberate breaking changes called out below because they rename or +retire v0.1 fields. A v0.1 bundle is consumable by a v0.2 consumer under +the fallbacks noted here. + +### 13.1 Breaking changes + +- **`timestamp` is superseded by `generated.at`.** A concept's last + content change is now recorded as `generated: { by, at }` (§5.2). + Consumers MAY fall back to a legacy `timestamp` when `generated` is + absent. +- **The body `# Citations` list is superseded by `sources`.** Provenance + moves to frontmatter (§5.1). Consumers SHOULD read `sources` and MAY + still parse a legacy `# Citations` body list for v0.1 documents. + +### 13.2 Additive changes + +All of the following are additive: new optional keys, one new concept +type, and one new conventional heading. Their absence yields a plain v0.1 +concept. + +- New frontmatter families: `sources` with its per-source credibility + signals (`author`, `usage_count`, `last_modified`) and the `usage_window` + sibling; `generated`, `verified`; `status`, `stale_after` (§5). +- New concept type `Attested Computation` and its computation keys + `runtime`, `parameters`, `computation`, `executor`, `attester` (§10). +- New conventional body heading `# Computation` (§4.2). +- The actor convention for `generated.by` and `verified[].by` (§7). + +Everything else (bundle structure, reserved filenames, the required +`type`, recommended `title`/`description`/`resource`/`tags`, cross-linking, +index files, log files, permissive conformance) is carried forward +unchanged. + +--- + +## Appendix A: Worked example, an income statement + +One bundle exercising every family, shown as a v0.1 to v0.2 migration of an +income statement with two figures, revenue and gross profit. + +### v0.1 form + +A single doc: both figures in one concept, the SQL in prose an agent can +read, ignore, or rewrite, citations a flat list, and the only timestamp is +`timestamp`. + +```markdown +--- +type: Metric +title: Income statement (fiscal year) +description: Headline income-statement figures for a fiscal year. +tags: [finance, income-statement] +timestamp: '2026-05-28T22:53:05+00:00' +--- + +# Definition +The income statement reports revenue and gross profit for a fiscal year. + +# Revenue +Recognized revenue sums `amount` over rows booked to the fiscal year: + + SELECT SUM(amount) AS revenue + FROM finance.recognized_revenue + WHERE fiscal_year = + +# Gross profit +Gross profit by segment, per the cost-allocation standard: + + SELECT gross_profit FROM fct_income_statement + WHERE fiscal_year = AND segment = + +# Citations +- https://wiki.acme/finance/fpa-handbook +- https://wiki.acme/finance/revenue-recognition +- https://wiki.acme/finance/cost-allocation +``` + +### v0.2 form + +The two figures split into attested computations linked from a narrative +concept. Every family is populated, and the two computations sit in +deliberately different states so one consumer reaches two verdicts. + +``` +bundles/finance/ + metrics/income-statement.md type: Metric (narrates, links both) + computations/revenue.md type: Attested Computation (runtime: bigquery) + computations/profit.md type: Attested Computation (runtime: dbt) + references/skills/run-on-bq.md, run-dbt.md + references/attesters/sql-equality.py, dbt-binding.py +``` + +`metrics/income-statement.md`, the readable doc; trust lives on what it +links, not here: + +```markdown +--- +type: Metric +title: Income statement (fiscal year) +description: Headline income-statement figures for a fiscal year. +tags: [finance, income-statement] +status: stable +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z } +verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z } +stale_after: 2026-12-31 +sources: + - id: fpa-handbook + resource: https://wiki.acme/finance/fpa-handbook + title: FP&A reporting handbook +--- + +# Definition +The income statement reports [revenue](../computations/revenue.md) and +[gross profit](../computations/profit.md) for a fiscal year, per the FP&A +reporting handbook.[^fpa-handbook] Each figure is produced by a sanctioned, +attestable computation; this concept only narrates them. + +[^fpa-handbook]: FP&A reporting handbook +``` + +`computations/revenue.md`, BigQuery SQL, human-verified, fresh, and +corroborated by a live dashboard source carrying credibility signals: + +```markdown +--- +type: Attested Computation +title: Revenue for fiscal year +description: Recognized revenue for a fiscal year, per Finance's definition. +tags: [finance, revenue] +status: stable +runtime: bigquery +parameters: + - { name: year, type: integer, required: true } +executor: + resource: references/skills/run-on-bq.md + receipt: [job_id, executed_sql, result] +attester: + resource: references/attesters/sql-equality.py +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-28T14:00:00Z } +verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z } +stale_after: 2026-12-31 +sources: + - id: rev-policy + resource: https://wiki.acme/finance/revenue-recognition + title: Revenue recognition policy + author: team:finance-fpa + last_modified: 2026-04-02 + - id: exec-rev-dash + resource: dashboards/exec-revenue + title: Executive revenue dashboard + author: team:finance-fpa + usage_count: 5000 + last_modified: 2026-06-18 +usage_window: { from: 2026-06-01, to: 2026-06-30 } +--- + +# Computation + + SELECT SUM(amount) AS revenue + FROM finance.recognized_revenue + WHERE fiscal_year = @year + +Recognized revenue per the recognition policy,[^rev-policy] corroborated by +the executive revenue dashboard.[^exec-rev-dash] + +[^rev-policy]: Revenue recognition policy +[^exec-rev-dash]: Executive revenue dashboard +``` + +`computations/profit.md`, a dbt model, process-verified, and past its +`stale_after`: + +```markdown +--- +type: Attested Computation +title: Gross profit for fiscal year +description: Gross profit by segment for a fiscal year, per the cost-allocation standard. +tags: [finance, profit] +status: stable +runtime: dbt +parameters: + - { name: year, type: integer, required: true } + - { name: segment, type: string, required: true } +executor: + resource: references/skills/run-dbt.md + receipt: [run_id, compiled_sql, result] +attester: + resource: references/attesters/dbt-binding.py +generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-14T14:00:00Z } +verified: { by: process:finance-nightly, at: 2026-06-12T08:00:00Z } +stale_after: 2026-06-15 +sources: + - id: cost-alloc + resource: https://wiki.acme/finance/cost-allocation + title: Cost allocation standard +--- + +# Computation + + SELECT gross_profit + FROM {{ ref('fct_income_statement') }} + WHERE fiscal_year = {{ var('year') }} + AND segment = {{ var('segment') }} + +Gross profit by segment per the cost-allocation standard.[^cost-alloc] + +[^cost-alloc]: Cost allocation standard +``` diff --git a/docs/tech/ARCHI_APP.md b/docs/tech/ARCHI_APP.md new file mode 100644 index 0000000..4ac89db --- /dev/null +++ b/docs/tech/ARCHI_APP.md @@ -0,0 +1,91 @@ +--- +type: Architecture +title: wsg_lib Engine Architecture +description: Technical architecture and design principles of the wsg_lib rendering engine +tags: [architecture, rendering, graphics, wgpu, engine] +status: stable +generated: { by: human:jerome, at: 2026-07-31T00:00:00Z } +--- + +# Architecture du Moteur wsg_lib + +wsg_lib est un moteur de rendu modulaire basé sur wgpu. Il adopte une architecture à deux niveaux : une façade de haut niveau pour la productivité et un accès bas niveau pour un contrôle total. + +## 1. Philosophie et Principes + +- **Abstraction vs Transparence** : Le moteur masque la complexité (wgpu, winit, gestion des Frame) via `App`, tout en exposant les briques élémentaires pour les utilisateurs avancés. +- **Approche orientée Scène** : Le rendu repose sur la composition d'une `Scene` contenant les entités, matériaux et géométries. +- **Pipeline Data-Driven** : Les ressources (Shaders, Meshes, Materials) sont découplées. Le `PipelineCache` gère automatiquement la compilation et la réutilisation des pipelines GPU. + +## 2. Organisation des Modules (`lib/src/`) + +- **`core/`** : Plomberie système (`Context`, `Renderer`, `Frame`). Accès bas niveau. +- **`pipeline/`** : `PipelineCache` pour la gestion des états GPU et shaders. +- **`resources/`** : Données (`Mesh`, `Material`, `Vertex`). +- **`scene/`** : Hiérarchie et stockage des objets à visualiser (Entités, Transformations). +- **`shaders/`** : Assets WGSL. +- **`utils/`** : Utilitaires transverses. + +## 3. Interfaces de Haut Niveau (`App` & `AppHandler`) + +### L'objet `App` + +La façade `App` orchestre la boucle de jeu. Elle encapsule : + +- Le cycle de vie de la fenêtre. +- La boucle d'événements. +- La gestion automatique des Frame (acquisition et présentation). + +### Le trait `AppHandler` + +L'utilisateur implémente ce trait pour définir la logique métier : + +```rust +pub trait AppHandler { + // Appelé avant la préparation de la frame + fn update(&mut self, _app: &mut App) {} + + // Appelé au moment de la présentation + fn render(&mut self, app: &mut App); +} +``` + +## 4. Workflow et Cycle de Vie + +### A. Initialisation (Configuration) + +- **Shaders** : Chargés avant la renderloop. +- **PipelineCache** : Enregistre les shaders. +- **Matériaux** : Créés avec un shader associé. Un mesh sans matériau explicite utilise `basic_shader` par défaut. +- **Scene** : Assemblage des objets. L'utilisateur peuple la scène via `app.scene`. + +### B. Boucle de Rendu (Automatisée) + +Le moteur gère la renderloop interne : + +1. **Update** : Appel à `AppHandler::update`. +2. **Acquisition** : Gestion interne de `wgpu::SurfaceTexture`. +3. **Render** : Appel à `AppHandler::render` où l'utilisateur exécute `app.render(scene)`. +4. **Présentation** : Gestion interne de `present()`. + +## 5. Accès Avancé + +Les utilisateurs souhaitant ignorer l'abstraction `App` peuvent accéder directement à : + +- `wsg_lib::core::Context` et `Renderer` pour gérer manuellement les RenderPass. +- `wsg_lib::pipeline::PipelineCache` pour des besoins de shaders personnalisés. +- `winit` pour la gestion précise des événements système. + +## 6. Structure des données (pour LLM) + +``` +App (Facade) -> Scene (Conteneur) -> Entities -> Mesh + Material (Shader) + | + +-> Renderer (WGPU) <-> PipelineCache (Shaders) +``` + +## Notes pour l'implémentation future + +- `app.render(scene)` : Cette méthode doit devenir l'API principale pour le rendu de la scène complète. +- Trait `AppHandler` : Il est recommandé de faire passer la `Scene` ou une référence à celle-ci comme argument ou de permettre à `AppHandler` d'être le lieu où la `Scene` est manipulée (ex : `MyGame { scene: Scene, ... }`). +- `PipelineCache` : Son utilisation doit être invisible pour l'utilisateur standard lors de la création d'un `Material`. diff --git a/docs/tech/ARCHI_ARENES.md b/docs/tech/ARCHI_ARENES.md new file mode 100644 index 0000000..b4409eb --- /dev/null +++ b/docs/tech/ARCHI_ARENES.md @@ -0,0 +1,218 @@ +--- +type: Technical Specification +title: Generational Arena Resource Management with slotmap +description: Technical specification for efficient and safe resource management using generational arenas implemented via the slotmap crate +tags: [architecture, resources, performance, safety, slotmap, arena] +status: stable +generated: { by: human:jerome, at: 2026-07-31T00:00:00Z } +--- + +# Fiche Technique : Gestion des Ressources avec des Arènes Générationalles (`slotmap`) + +Cette fiche technique détaille l'implémentation recommandée pour gérer efficacement et en toute sécurité les ressources (maillages, textures, matériaux, lumières, etc.) au sein du moteur graphique WSG. Nous utilisons le concept d'**arène générationalle**, implémenté via la crate `slotmap`, pour bénéficier d'IDs stables, de performances optimales, de sécurité accrue et de fonctionnalités avancées comme les `SecondaryMap`. + +## Objectifs + +* **Stabilité des IDs :** Garantir que les identifiants (Keys) des ressources restent valides même si d'autres ressources sont supprimées. +* **Performance :** Accéder aux ressources via un ID de manière aussi rapide que possible (accès quasi direct via index). +* **Sécurité :** Empêcher l'utilisation accidentelle d'IDs obsolètes ("Dangling IDs") qui pointeraient vers des objets supprimés ou réaffectés. +* **Flexibilité :** Permettre l'ajout et la suppression de ressources dynamiquement. +* **Extensibilité Future :** Profiter des fonctionnalités avancées de `slotmap` comme les `SecondaryMap` pour attacher des données dynamiques ou transitoires aux ressources existantes sans modifier leur structure principale. + +## Concepts Clés + +### 1. Arène Typée + +Chaque type de ressource nécessite une arène séparée. Par exemple : + +* `SlotMap` pour stocker les `Mesh` +* `SlotMap` pour stocker les `Material` +* `SlotMap` pour stocker les `Texture` +* `SlotMap` pour stocker les `Light` + +Cela permet d'optimiser l'accès et de garantir la cohérence des types. + +### 2. Handles (Identifiants) Typés + +Un Handle est un objet spécial généré par l'arène lors de l'insertion d'une ressource. Il sert de référence stable à cette ressource. Nous utilisons des types personnalisés (struct wrappers) pour typer fortement ces Handles, empêchant les erreurs de mélange entre types de ressources. + +### 3. Génération (Generation) + +Pour renforcer la sécurité, chaque Handle encapsule non seulement un **index** (où l'objet est stocké dans le tableau interne de l'arène), mais aussi un numéro de **génération**. Lorsqu'un objet est supprimé, l'emplacement dans le tableau interne est marqué comme vide, mais le numéro de génération associé à cet emplacement est incrémenté. Lorsque ce même emplacement est réutilisé pour un nouvel objet, le nouvel objet reçoit le même index mais une génération plus récente. Si un ancien Handle (avec un index et une ancienne génération) est utilisé pour tenter d'accéder à l'arène, le système vérifie si la génération du Handle correspond à celle stockée à l'index. Si ce n'est pas le cas, l'accès est refusé, empêchant l'utilisation d'un Handle périmé. + +## Implémentation avec `slotmap` + +### Dépendance + +Ajoutez `slotmap` à votre `Cargo.toml` : + +```toml +[dependencies] +slotmap = { version = "1.0", features = ["serde"] } # Inclure 'serde' si nécessaire, sinon omettre la feature +``` + +(Note : slotmap a une dépendance sur serde par défaut. Si vous n'avez absolument pas besoin de sérialisation/désérialisation des arènes, vous pouvez potentiellement chercher une alternative légère comme thunderdome, mais slotmap est le standard et offre plus de fonctionnalités). + + +## Structure de Base et Typage Fort + +```rust +use slotmap::{SlotMap, new_key_type}; + +// --- Définition des types de ressources --- +// Ces structs doivent être définies ailleurs dans votre code +#[derive(Debug)] +pub struct Mesh { + // ... champs du mesh ... +} + +#[derive(Debug)] +pub struct Material { + // ... champs du material ... +} + +#[derive(Debug)] +pub struct Texture { + // ... champs de la texture ... +} + +#[derive(Debug)] +pub struct Light { + // ... champs de la lumière ... +} + +// --- Définition des Handles typés --- +// Ces lignes créent des types uniques pour chaque Handle +new_key_type! { pub struct MeshId; } +new_key_type! { pub struct MaterialId; } +new_key_type! { pub struct TextureId; } +new_key_type! { pub struct LightId; } + +// --- Définition des arènes --- +pub struct ResourceManager { + meshes: SlotMap, + materials: SlotMap, + textures: SlotMap, + lights: SlotMap, + // Ajoutez d'autres arènes pour d'autres types si nécessaire +} + +impl ResourceManager { + pub fn new() -> Self { + // Optionnel : spécifier une capacité initiale estimée pour chaque arène + // let estimated_mesh_count = 100; + // let meshes = SlotMap::with_capacity_and_key(estimated_mesh_count); + // ... + Self { + meshes: SlotMap::new(), + materials: SlotMap::new(), + textures: SlotMap::new(), + lights: SlotMap::new(), + } + } + + // --- Méthodes pour ajouter des ressources --- + pub fn add_mesh(&mut self, mesh: Mesh) -> MeshId { // Retourne un Handle typé + self.meshes.insert(mesh) + } + + pub fn add_material(&mut self, material: Material) -> MaterialId { + self.materials.insert(material) + } + + pub fn add_texture(&mut self, texture: Texture) -> TextureId { + self.textures.insert(texture) + } + + pub fn add_light(&mut self, light: Light) -> LightId { + self.lights.insert(light) + } + + // --- Méthodes pour accéder aux ressources --- + pub fn get_mesh(&self, handle: MeshId) -> Option<&Mesh> { + self.meshes.get(handle) + } + + pub fn get_material(&self, handle: MaterialId) -> Option<&Material> { + self.materials.get(handle) + } + + pub fn get_texture(&self, handle: TextureId) -> Option<&Texture> { + self.textures.get(handle) + } + + pub fn get_light(&self, handle: LightId) -> Option<&Light> { + self.lights.get(handle) + } + + // --- Méthodes pour accéder aux ressources mutables (utile dans update(), mais à éviter pendant le rendu) --- + pub fn get_mesh_mut(&mut self, handle: MeshId) -> Option<&mut Mesh> { + self.meshes.get_mut(handle) + } + + pub fn get_material_mut(&mut self, handle: MaterialId) -> Option<&mut Material> { + self.materials.get_mut(handle) + } + + pub fn get_texture_mut(&mut self, handle: TextureId) -> Option<&mut Texture> { + self.textures.get_mut(handle) + } + + pub fn get_light_mut(&mut self, handle: LightId) -> Option<&mut Light> { + self.lights.get_mut(handle) + } + + // --- Méthodes pour supprimer des ressources --- + pub fn remove_mesh(&mut self, handle: MeshId) -> Option { // Option retourné est la ressource supprimée + self.meshes.remove(handle) + } + + pub fn remove_material(&mut self, handle: MaterialId) -> Option { + self.materials.remove(handle) + } + + pub fn remove_texture(&mut self, handle: TextureId) -> Option { + self.textures.remove(handle) + } + + pub fn remove_light(&mut self, handle: LightId) -> Option { + self.lights.remove(handle) + } + + // --- Méthode pour vérifier si un Handle est toujours valide --- + pub fn contains_mesh(&self, handle: MeshId) -> bool { + self.meshes.contains_key(handle) + } + + pub fn contains_material(&self, handle: MaterialId) -> bool { + self.materials.contains_key(handle) + } + + pub fn contains_texture(&self, handle: TextureId) -> bool { + self.textures.contains_key(handle) + } + + pub fn contains_light(&self, handle: LightId) -> bool { + self.lights.contains_key(handle) + } +} +``` + +# Bonnes Pratiques d'Utilisation + +1. Initialisation Groupée : Encouragez les utilisateurs de WSG à créer la majorité de leurs ressources statiques (maillages de niveau, matériaux de base, textures fixes, lumières ambiantes, etc.) avant de lancer la boucle de rendu principale. Vous pouvez éventuellement fournir une fonction reserve_initial_capacities(&mut resource_manager, expected_counts...) qui appelle SlotMap::reserve() pour optimiser la mémoire initiale. +2. Stocker les Handles Typés : Les entités de la scène (ou les objets graphiques) doivent stocker les Handles typés (MeshId, MaterialId, etc.) retournés lors de l'ajout des ressources. Par exemple, un objet GameObject pourrait contenir un Option pour son Mesh, un Option pour son Material, etc. Le typage fort empêche les erreurs de mélange. +3. Accès pendant le rendu : Pendant la phase de rendu (render()), accédez aux ressources via les Handles typés stockés. Utilisez get() (lecture seule) pour éviter les conflits avec les systèmes de mise à jour concurrents. +4. Accès pendant la mise à jour : Pendant la phase de mise à jour (update()), vous pouvez utiliser get_mut() si des modifications sont nécessaires. Soyez vigilant à la gestion des lifetimes et de la mutabilité. +5. Validation : Avant d'utiliser un Handle potentiellement ancien ou incertain, vérifiez sa validité avec contains_* si l'opération n'est pas critique, ou laissez get() renvoyer None si le Handle est invalide. +6. Suppression Dynamique : Bien que possible, la suppression de ressources pendant la boucle de rendu doit être faite avec prudence. Assurez-vous que les entités ou objets qui référençaient cette ressource soient informés ou nettoyés pour éviter d'utiliser des Handles invalides. La suppression est souvent mieux gérée en fin de frame ou via un système de "marquage pour suppression" suivi d'un nettoyage différé. +7. Futur : SecondaryMaps : slotmap permet d'utiliser des SecondaryMap pour associer dynamiquement des données à des ressources existantes sans modifier leur structure principale. Par exemple, SecondaryMap pourrait stocker les transformations actuelles de chaque maillage. Cela peut être utile pour le rendu ou pour des systèmes de physique/transformation indépendants. + +# Avantages de cette Approche + +* Simplicité d'utilisation : Les développeurs utilisent des Handles typés stables, sans se soucier des références Rust ou des lifetimes complexes pour les ressources partagées. +* Performance : Les accès sont rapides, proches de l'accès direct via index, grâce à l'implémentation interne de slotmap. +* Sécurité : Le système de génération empêche efficacement l'utilisation de Handles invalides, ce qui peut causer des plantages ou des bugs subtils. +* Conformité avec Rust : Respecte les principes de propriété et de sécurité mémoire de Rust sans recourir à Rc> ou d'autres constructions potentiellement coûteuses ou moins sûres pour la gestion partagée des ressources. +* Typage Fort : Les types MeshId, MaterialId, etc., empêchent les erreurs de compilation liées au mélange de Handles de types différents. +* Extensibilité : L'écosystème slotmap (SecondaryMap) offre des perspectives pour des architectures plus complexes à l'avenir. diff --git a/docs/tech/ARCHI_CPU_GPU.md b/docs/tech/ARCHI_CPU_GPU.md new file mode 100644 index 0000000..2acbf2b --- /dev/null +++ b/docs/tech/ARCHI_CPU_GPU.md @@ -0,0 +1,62 @@ +--- +type: Technical Specification +title: GPU-Driven 3D Rendering Architecture with wGPU +description: Technical specification for GPU-driven 3D rendering architecture using wgpu, focusing on CPU-GPU workload distribution and performance optimization +tags: [architecture, rendering, gpu, cpu, performance, wgpu] +status: stable +generated: { by: human:jerome, at: 2026-07-31T00:00:00Z } +--- + +Architecture de Rendu 3D GPU-Driven avec wGPU : +Bonnes Pratiques & Guide d'Implémentation + +Ce document sert de spécification technique et de trame d'implémentation pour l'architecture de rendu 3D pilotée par le GPU (GPU-Driven Rendering) utilisant wgpu. L'objectif est de déléguer un maximum de charges de calcul au GPU pour soulager le CPU et maximiser les performances de parallélisme. + +1. Répartition des Rôles : CPU vs GPU (La Source de Vérité) + +Pour éviter les goulets d'étranglement dus aux allers-retours sur le bus PCIe, la règle d'or est la suivante : Le CPU est le cerveau logique, le GPU est l'exécutant visuel. + +Côté CPU (Source de Vérité) +- Ce qu'il conserve : Les données logiques et les transformations brutes des objets (ex: Vec contenant la position, la rotation, et l'échelle). +- Ce qu'il fait : Il gère la logique de jeu, l'IA, le réseau et les interactions globales. +- Ce qu'il ne fait plus : Il ne calcule plus les matrices de transformation mondiales (World Matrices) en masse, et ne fait plus de tests de visibilité unitaires. + +Côté GPU (Exécutant Autonome) +- Ce qu'il calcule : Les World Matrices, le Frustum Culling, et la génération des listes de dessin indirectes. +- Ce qu'il conserve : Les buffers de données persistants en VRAM (Storage Buffers) qui vivent d'une frame à l'autre sans jamais redescendre vers le CPU. + +2. Le Pipeline d'Exécution par Frame (Ordre des Passes) + +L'exécution des tâches s'appuie sur une structure séquentielle stricte au sein d'un même CommandEncoder. Le driver et wGPU s'occupent des barrières de mémoire implicites entre chaque étape. + +``` +[ CPU : Envoi des Transforms bruts ] + ↓ +[ Pass 1 : Compute (Calcul World Matrices + Frustum Culling + Indirect Draw Buffer) ] + ↓ (Barrière de mémoire automatique gérée par le driver) +[ Pass 2 : Render (Draw Indexed Indirect basé sur les objets visibles) ] +``` + +Étape par étape : +- Mise à jour CPU (Minimaliste) : Le CPU écrit les transformations brutes (Transform) modifiées dans un buffer GPU mappé (via un mécanisme de Double Buffering pour éviter les conflits de lecture/écriture). +- Pass de Calcul (Compute Pass) : + - Calcul des World Matrices : Un compute shader lit les transformations brutes et génère la matrice 4x4 finale pour chaque mesh. + - Frustum Culling GPU : Le même compute shader (ou un compute pass dédié) compare la Bounding Box (AABB) de chaque objet avec les plans de la caméra (matrice de projection/vue). + - Remplissage du Buffer Indirect : Si l'objet est visible, son identifiant est injecté dans un buffer de commandes de dessin indirect (Indirect Draw Buffer). +- Pass de Rendu (Render Pass) : + - Le CPU émet une unique commande globale : draw_indexed_indirect. + - Le GPU pioche directement dans le buffer préparé par le compute pass et dessine uniquement les objets visibles, sans intervention du CPU. + +3. Stratégie de Synchronisation + - Sécurité de l'ordre : L'ordre d'appel des méthodes sur le CommandEncoder (begin_compute_pass suivi de begin_render_pass) garantit l'ordre d'exécution séquentiel sur le GPU. + - Barrières de mémoire : Le pilote insère automatiquement les barrières nécessaires pour s'assurer que le buffer de la WorldMatrix et le buffer Indirect sont complètement écrits par le compute shader avant d'être lus par le render pipeline. + - Éviter le Readback (map_async) : Sauf cas exceptionnel (débug ou interaction scriptée critique), aucune donnée géométrique ou de position ne doit remonter du GPU vers le CPU. Le CPU fait confiance à sa propre structure de données initiale pour la logique métier. + +4. Synthèse des Structures de Données en VRAM + +Pour implémenter cette architecture, prévoyez l'utilisation des buffers wGPU suivants : +Nom du Buffer,Rôle,Type wGPU,Direction du flux +Transform Buffer,Stocke les positions/rotations/échelles brutes.,Storage Buffer,CPU → GPU +Matrix Buffer,Stocke les World Matrices finales calculées.,Storage Buffer,GPU (Calculé) → GPU (Lu par le Render) +Bounding Box Buffer,Stocke les AABB de chaque mesh pour le culling.,Storage Buffer,CPU → GPU (Statique) +Indirect Draw Buffer,Contient la liste dynamique des objets à dessiner.,Indirect Buffer + Storage,GPU (Rempli par Compute) → GPU (Lu par Render) diff --git a/docs/tech/ARCHI_RENDU.md b/docs/tech/ARCHI_RENDU.md new file mode 100644 index 0000000..30ff9a8 --- /dev/null +++ b/docs/tech/ARCHI_RENDU.md @@ -0,0 +1,56 @@ +--- +type: Technical Specification +title: Rendering Architecture: Update/Render Cycle and Data Management +description: Technical specification for the rendering architecture of wsg_lib, defining strategies for mutability and data management to maximize performance and memory safety in Rust +tags: [architecture, rendering, rust, performance, memory-safety] +status: stable +generated: { by: human:jerome, at: 2026-07-31T00:00:00Z } +--- + +# Architecture de Rendu : Cycle Update/Render et Gestion des Données + +Ce document définit la stratégie de gestion de la mutabilité et des données du moteur wsg_lib, conçue pour maximiser la performance et garantir la sécurité mémoire via Rust. + +## 1. La Dichotomie Update / Render + +Pour éviter les conflits de données et optimiser le pipeline GPU, le moteur sépare strictement le cycle de vie de la frame en deux phases : + +### Phase Update (Mutabilité Totale) + +- L'utilisateur peut modifier librement l'état de la Scene (transformations, propriétés des matériaux, ajout/suppression d'entités). +- C'est l'unique zone de mutation autorisée. Le système est en "lecture-écriture". + +### Phase Render (Lecture et Orchestration) + +- La Scene est considérée comme immuable vis-à-vis du rendu. +- Le moteur itère automatiquement sur les entités pour soumettre les commandes au GPU. +- L'utilisateur dispose d'une "trappe" via `AppHandler::render()` pour injecter du code de rendu personnalisé, mais sans modifier l'état métier des objets. + +## 2. Gestion des Données : Indirection par ID (Handle) + +Pour contourner les limitations du Borrow Checker de Rust lors de l'accès aux ressources, le moteur utilise une approche par **Indirection (Handles/IDs)**. + +- **HashMaps et Vecs indexés** : Les ressources (`Mesh`, `Material`) ne sont pas stockées sous forme de références directes (`&Mesh`) dans les entités. Elles sont stockées dans des conteneurs centralisés dans la Scene. +- **Identifiants (Handles)** : Chaque entité possède un `MeshId` ou `MaterialId`. +- **Avantage** : Cela élimine les problèmes de durées de vie (lifetimes) complexes. Vous pouvez passer des IDs partout sans bloquer la mutabilité des conteneurs parents. +- **Performance** : Cette approche permet au moteur de trier les entités par `MaterialId` avant le rendu, réduisant drastiquement les changements d'état GPU (**State Change Overhead**). + +## 3. Points d'Attention du Borrow Checker + +Bien que cette architecture facilite la gestion de la mémoire, des règles strictes s'appliquent à `App` : + +- **Conflit de Mutabilité** : App contient à la fois la Scene et le Renderer. Il est interdit d'emprunter `&mut scene` et `&mut renderer` simultanément. +- **Solution** : Dans la boucle de rendu interne (`App::run`), le moteur doit être structuré pour séquencer les accès : `let scene = &app.scene;` suivi de `let renderer = &mut app.renderer;` puis `renderer.render_scene(scene);`. +- **Séparation des Responsabilités** : Le RenderLoop doit posséder la main sur l'ordonnancement pour éviter que l'utilisateur ne tente de muter la scène pendant que le renderer est en train de lire les données. + +## 4. Synthèse des Avantages + +- **Performance (Batching)** : Le rendu automatique par le moteur permet d'implémenter des stratégies de rendu optimales invisibles pour l'utilisateur. +- **Ergonomie** : L'utilisateur n'écrit pas de boucles de rendu complexes. Il se concentre sur sa logique métier dans `update()`. +- **Sécurité** : L'utilisation d'IDs évite les cycles de références et les références pendantes, rendant le code plus sûr et plus facile à maintenir. + +## 5. Guide de Développement pour l'Utilisateur + +> "Si vous devez changer la position d'un objet ou son matériau, faites-le dans `update()`. Si vous avez besoin d'afficher un élément de debug ou un rendu spécial, faites-le dans `render()`, mais traitez les objets de la scène comme des données en lecture seule." + +Cette structure permet au projet d'être extrêmement scalable. L'ajout futur de fonctionnalités (Lumières, Textures, Caméras) ne nécessitera que d'ajouter de nouveaux conteneurs dans la Scene et de mettre à jour le système de tri dans `Renderer::render_scene()`. diff --git a/docs/tech/FRAME_LOOP.md b/docs/tech/FRAME_LOOP.md new file mode 100644 index 0000000..7ade79f --- /dev/null +++ b/docs/tech/FRAME_LOOP.md @@ -0,0 +1,39 @@ +--- +type: Technical Specification +title: Frame Loop Architecture +description: Technical specification for the frame loop architecture in wsg_lib, detailing the immutable frame lifetime cycle and resource management +tags: [architecture, rendering, frame-loop, gpu, wgpu] +status: stable +generated: { by: human:jerome, at: 2026-07-31T00:00:00Z } +--- + +# La Boucle de Rendu (Frame Loop) + +Pour afficher quelque chose, nous suivons un cycle immuable appelé la Frame Lifetime. Dans ton `main.rs` (l'orchestrateur), le flux est désormais le suivant : + +- **Context::begin_frame() :** Acquiert la surface texture et crée la TextureView. +- **Renderer::render(...)** : Utilise le CommandEncoder pour écrire les ordres de dessin. +- **Context::end_frame()** : Soumet les commandes à la file (`queue`) et présente l'image. + +--- + +## Pourquoi cette séparation est vitale + +Le bloc `{ let mut render_pass = ... }` est crucial. Dans Rust, `render_pass` emprunte mutablement `encoder`. Il doit être détruit (via la fin du bloc ou un `drop()`) avant que tu puisses appeler `encoder.finish()`. Si tu oublies cela, le compilateur Rust refusera de compiler, empêchant ainsi des bugs critiques de synchronisation GPU. + +--- + +## Ressources : Persistantes vs Par-Frame + +Avec notre nouvelle architecture "Atelier", la distinction est devenue encore plus nette : + +| Élément | Durée de vie | Pourquoi ? | +|---------|-------------|------------| +| SurfaceConfiguration | Persistante | Ne change qu'au redimensionnement. | +| RenderPipeline | Persistante | Stocké dans le PipelineCache (`Arc`), compilation unique. | +| Material | Persistante | Définit le look ; partage le pipeline via `Arc`. | +| Mesh | Persistante | Les données géométriques sont envoyées une fois au GPU. | +| CommandEncoder | Par-Frame | Ton "carnet de notes" temporaire pour les ordres du GPU. | +| TextureView | Par-Frame | Fenêtre temporaire sur la texture active du swapchain. | + +---