Processors

Processors transform Records in-flight between extraction and sinking. Each processor receives a Record (Entity + Edges), modifies the Entity, and returns the updated Record. Edges pass through unchanged.

Processors are defined in the processors block of a recipe and execute sequentially -- the output of one processor becomes the input of the next:

processors:
  - name: enrich
    config:
      attributes:
        domain: payments
  - name: labels
    config:
      labels:
        source: meteor

Supported Processors

Processor	Description
`enrich`	Append custom key-value pairs to `entity.properties`
`labels`	Append labels into `entity.properties.labels`
`script`	Transform the entity using a user-defined Tengo script

enrich

Merges the configured key-value pairs into the entity's properties map. Useful for adding metadata that does not exist in the source system.

processors:
  - name: enrich
    config:
      attributes:
        team: data-platform
        environment: production

Key	Type	Description	Required
`attributes`	`map[string]any`	Key-value pairs merged into `entity.properties`	yes

More details

labels

Merges the configured labels into entity.properties.labels. If the labels key does not yet exist in properties, it is created.

processors:
  - name: labels
    config:
      labels:
        source: meteor
        classification: internal

Key	Type	Description	Required
`labels`	`map[string]string`	Key-value pairs merged into `entity.properties.labels`	yes

More details

script

Transforms the entity using a user-defined script. The entity is exposed to the script as an entity variable (a map). Currently Tengo is the only supported engine. The script has full control over entity fields, including the ability to make HTTP calls to external services.

processors:
  - name: script
    config:
      engine: tengo
      script: |
        entity.name = entity.name + " (processed)"

Key	Type	Description	Required
`engine`	`string`	Script engine (`tengo`)	yes
`script`	`string`	Inline Tengo script	yes

More details

Chaining Processors

Processors execute sequentially in recipe order. If a processor fails, the entire recipe execution fails -- there is no skip-on-error behavior.