Commands
Meteor currently supports the following commands:
- run: Execute recipes for metadata extraction.
- lint: Validate recipe files for errors.
- recipe init: Bootstrap a new recipe.
- recipe gen: Generate multiple recipes from a template.
- plugins list: List available extractors, sinks, and processors.
- plugins info: Display detailed information about a plugin.
- entities: List entity types across all extractors.
- edges: List edge types across all extractors.
- version: Print the Meteor version.
Running recipes
# run meteor for specified recipe
$ meteor run recipe.yml
# run all recipes in the specified directory
$ meteor run _recipes/
# run all recipes in the current directory
$ meteor run .
# dry-run to preview extracted records without sending to sinks
$ meteor run recipe.yml --dry-run
# extract only the first 10 records for testing
$ meteor run recipe.yml --dry-run --limit 10
# override log level for debugging
$ meteor run recipe.yml --log-level debug
# use a custom config file
$ meteor run recipe.yml --config /path/to/meteor.yaml
# pass environment variables from a file
$ meteor run recipe.yml --var secrets.yaml
Flags
--config | -c | ./meteor.yaml | File path for agent level config |
--var | | | Path to config file with env variables for recipe |
--log-level | | | Override log level (debug, info, warn, error) |
--dry-run | | false | Extract records without sending to sinks |
--limit | | 0 | Maximum number of records to extract (0 = unlimited) |
Linting recipes
# validate a single recipe
$ meteor lint recipe.yml
# lint all recipes in the specified directory
$ meteor lint _recipes/
# lint all recipes in the current directory
$ meteor lint .
# lint with debug logging
$ meteor lint recipe.yml --log-level debug
Flags
--log-level | | Override log level (debug, info, warn, error) |
Creating sample recipes
# bootstrap a recipe with a bigquery extractor and a console sink
$ meteor recipe init sample -n mycompany -e bigquery -s console
# bootstrap recipe with multiple sinks and a processor
$ meteor recipe init sample -n mycompany -e bigquery -s compass,kafka -p enrich
# save the generated recipe to a file
$ meteor recipe init sample -n mycompany -e postgres -s console > recipe.yaml
# interactive mode (prompts for extractor, sinks, processors)
$ meteor recipe init sample -n mycompany
Flags
--scope | -n | Yes | URN namespace for the recipe |
--extractor | -e | No | Type of extractor (interactive if omitted) |
--sinks | -s | No | Comma-separated list of sink types (interactive if omitted) |
--processors | -p | No | Comma-separated list of processor types (interactive if omitted) |
Generating multiple recipes
# generate multiple recipes from a template
$ meteor recipe gen template.yaml -d data.yaml -o ./output-dir
Flags
--output | -o | Output directory |
--data | -d | Template data file |
Listing plugins
# list all available plugins (extractors, sinks, processors)
$ meteor plugins list
# list only extractors
$ meteor plugins list --type extractor
# list only sinks
$ meteor plugins list --type sink
# list only processors
$ meteor plugins list --type processor
# filter extractors by entity type
$ meteor plugins list --entity-type table
# filter extractors by edge type
$ meteor plugins list --edge-type derived_from
# filter extractors by tag
$ meteor plugins list --tag gcp
# output as JSON for scripting
$ meteor plugins list --format json
Flags
--type | | | Filter by plugin type (extractor, sink, processor) |
--entity-type | | | Filter extractors by entity type |
--edge-type | | | Filter extractors by edge type |
--tag | | | Filter extractors by tag |
--format | -f | table | Output format (table, json) |
# display information about a plugin
$ meteor plugins info bigquery
$ meteor plugins info compass
$ meteor plugins info enrich
# show full markdown documentation
$ meteor plugins info bigquery --full
# output as JSON
$ meteor plugins info bigquery --format json
# interactive mode (prompts for plugin selection)
$ meteor plugins info
Flags
--full | | false | Show full markdown documentation |
--format | -f | table | Output format (table, json) |
Listing entity types
# list all entity types and which extractors produce them
$ meteor entities
# output as JSON
$ meteor entities --format json
Flags
--format | -f | table | Output format (table, json) |
Listing edge types
# list all edge types and which extractors produce them
$ meteor edges
# output as JSON
$ meteor edges --format json
Flags
--format | -f | table | Output format (table, json) |
Getting help
# check for meteor help
$ meteor --help
# get help for a specific command
$ meteor run --help
$ meteor plugins list --help