The Processing Model

Iteration, Configuration, and Order of Operations

The Core Iteration Pattern

The most important concept to grasp is rescile’s implicit iteration pattern. Think of each .toml configuration file as a module that contains a loop.

The Golden Rule: A configuration file with origin_resource = "application" is a for-each loop. It tells rescile to run the rules in that file once for every application in your graph.

This is the core mental model. You don’t write imperative loops; you declare a set of rules (or “functions”) that operate on a collection of resources. rescile handles the iteration automatically.

Consider a simple model file data/models/server.toml:

# data/models/server.toml
origin_resource = "application"

# Rule Block 1
[[create_resource]]
resource_type = "server"
relation_type = "HOSTED_ON"
name = "{{ origin_resource.name }}_server"
# ...

# Rule Block 2
[[create_resource]]
match_on = [{ property = "environment", value = "prod" }]
resource_type = "monitoring_agent"
name = "agent_for_{{ origin_resource.name }}"
# ...

The declarative code above is equivalent to the following imperative pseudo-code:

// The 'origin_resource' directive sets up the loop.
all_applications = graph.find_resources(type="application")

for current_application in all_applications:
  // --- Evaluate Rule Block 1 ---
  // Corresponds to the first [[create_resource]] block.
  create_server_for(current_application)

  // --- Evaluate Rule Block 2 ---
  // Corresponds to the second [[create_resource]] block.
  if current_application.environment == "prod":
    create_monitoring_agent_for(current_application)

This declarative, for-each processing model is the foundation for all of rescile’s engines.

Data-Driven Iteration with create_from

As an alternative to iterating over resources in the graph, rescile supports a powerful data-driven iteration model using the create_from directive within a [[create_resource]] block. This allows you to generate resources from an abstract list of data defined in the file’s header, such as from an imported JSON file.

This shifts the paradigm from decorating an existing graph to bootstrapping parts of the graph from configuration data.

# data/models/providers.toml
# No origin_resource is defined.

# 1. Import raw data from a JSON file.
providers = { "json!" = "shared/providers.json" }

# 2. Use a template to transform it into a simple list.
provider_list = { "template!" = '''{{ providers | map(attribute="name") | safe }}''' }

# 3. Iterate over the `provider_list` data source.
[[create_resource]]
# The resource type is defined by `resource_type`. If it were omitted,
# the resource type would default to the name of the list, "provider_list".
create_from = { list = "provider_list" }
resource_type = "provider"
name = "ext-{{ value }}"

In this case, resource_type = "provider" explicitly sets the type. If it were omitted, the type would default to the list name, provider_list. The create_from block also supports an as key, which has the highest priority for defining the resource type.

In this model, the loop is not over graph resources, but over the items in the provider_list. The special origin_resource variable is unavailable; instead, the value variable is automatically available in templates as {{ value }}.

Anatomy of a Configuration File

rescile configuration files (.toml) follow a consistent structure. Understanding this structure is key to writing effective rules. Each file consists of a header and one or more rule blocks.

Think of a .toml file as a single source file or module.

# --- HEADER ---
# Analogous to module-level scope or global variables.
origin_resource = "application" # The subject of the file's for-each loop.

# File-scoped data, available to all Rule Blocks.
server_specs = { standard_cpu = 4, premium_cpu = 8 }
ip_ranges = { "json!" = "shared/ip_ranges.json" } # Load external data.

# Global variables (standard strings) are evaluated once. `origin_resource` is a list of ALL applications.
total_apps = "{{ origin_resource | length }}"
# Iteration variables (`template!` or `function!`) are evaluated per iteration. `origin_resource` is the current application.
app_name_upper = { "function!" = "{{ origin_resource.name | upper }}" }

# --- RULE BLOCK 1 --- (Analogous to a function or method)
# A single instruction to be executed for each `origin_resource`.
[[create_resource]]
# Directives (keywords/parameters) that configure the Rule Block.
resource_type = "server"
relation_type = "HOSTS"
name = "server_{{ origin_resource.name }}"

# A sub-table for defining properties of the new resource.
[create_resource.properties]
cpu_cores = "{{ server_specs.standard_cpu }}"
status = "provisioning"

# --- RULE BLOCK 2 --- (Another function)
# Evaluated independently for each `origin_resource`.
[[create_resource]]
# Conditional logic: this block only runs if the origin_resource matches.
match_on = [{ property = "environment", value = "prod" }]

resource_type = "monitoring_agent"
name = "agent_{{ origin_resource.name }}"

Let’s break down these components:

a. Header (Module Scope)

The header is the section at the top of the file, before any [[...]] blocks. It sets the context for all rules in the file.

• origin_resource = "...": This is the most important directive, defining the subject of the for-each loop. If omitted, the file is processed only once in a global context.

When origin_resource is omitted, rules are processed in a global context. A rule will be processed exactly once unless it uses the create_from directive with a list to iterate over a data source. In this global mode, the origin_resource variable is not available in templates. This is useful for creating singleton resources or for bootstrapping a graph from abstract data.

• File-Scoped Data: Any other key-value pair or table defined in the header (like server_specs) becomes available as a variable in all templates within that file. This is perfect for defining constants or loading configuration data using the json! directive.
  • Standard template strings (e.g., "{{ ... }}") are evaluated globally before the loop, where origin_resource is a list of all matching resources.
  • Iteration templates (e.g., { "template!" = "{{ ... }}" } or { "function!" = "{{ ... }}" }) are evaluated locally inside the loop, where origin_resource is the current single resource being processed.

The context for all rules in the file is further enriched by any global module parameters passed via the command line when the configuration is run as part of a module.

b. Rule Blocks (Functions) and Directives (Keywords)

A Rule Block is a TOML “array of tables,” denoted by double square brackets (e.g., [[create_resource]], [[copy_property]]). Think of each block as a single, executable function or instruction that rescile will evaluate during its loop.

Inside a Rule Block, you define Directives (key-value pairs) that act as keywords or parameters for that function. Common directives include resource_type, name, relation_type, and the conditional match_on.

The Order of Operations

rescile processes your configuration files in a strict, multi-phase sequence. After each of the main data processing phases (Asset Loading, Model Application, and Compliance Application), an auto-linking phase runs to create relationships based on property conventions. Understanding this order is crucial for predicting how the graph will be built.

flowchart TD
    A[<i class='fa-solid fa-file-csv'></i> 1. Asset Loading] --> AL[<i class='fa-solid fa-link'></i> Auto-linking]
    AL --> B
    subgraph B [<i class='fa-solid fa-code'></i> 2. Model Application]
        direction TB
        B_Loop["<i class='fa-solid fa-repeat'></i> Model Stabilization Loop"] --> B_Ops["[[create_resource]]<br>[[link_resources]]<br>[[copy_property]]"]
    end
    B --> BL[<i class='fa-solid fa-link'></i> Auto-linking]
    BL --> C[<i class='fa-solid fa-shield'></i> 3. Compliance Application]
    C --> CL[<i class='fa-solid fa-link'></i> Auto-linking]
    CL --> D[<i class='fa-solid fa-file-invoice'></i> 4. Output Generation]
    D --> E[<i class='fa-solid fa-sitemap'></i> Final Graph]

1. Asset Loading (data/assets/*.csv): All CSV files are read first. This phase populates the graph with the initial set of resources. The subsequent auto-linking phase then creates relationships based on foreign key conventions. (See Asset Management)

2. Model Application (data/models/*.toml): Model files are processed to build out the architectural graph. rescile automatically detects dependencies between models (e.g., if model_B.toml reads resources created by model_A.toml) and executes them in the correct order. The core iteration pattern is applied for each model file. (See Architectural Models)

3. Compliance Application (data/compliance/*.toml): After the complete architectural graph is built, compliance files are processed. They also use the iteration pattern to find resources and relationships, mutating the graph to enforce governance. (See Compliance as Code)

4. Output Generation (data/output/*.toml): Finally, output files are processed. They query the final, governed graph to generate structured data artifacts. (See Output Generation)