Pipeline Service

Central orchestrator for Haystack pipeline management and execution.

The PipelineService provides:

  • Pipeline CRUD - Create, read, update, delete pipeline definitions

  • Validation - Validates pipeline definitions against ComponentRegistry

  • Building - Constructs Haystack Pipeline objects from YAML/dict definitions

  • Execution - Asynchronous pipeline execution

  • Caching - Version-based pipeline caching for performance

Key Components

  • PipelineRepository - Database persistence for pipelines and executions

Execution Flow

  1. Pipeline definition validated

  2. Pipeline object constructed from definition

  3. Pipeline executed with input documents

  4. Results processed and returned