DataOps Platform - Data Collector Engine Guide

Home
Pipeline Configuration

Release Notes
Installation
Configuration
Upgrade
Pipeline Concepts and Design
Pipeline Configuration
- Retrying the Pipeline
- Rate Limit
- Advanced Options
  Pipelines and most pipeline stages include advanced options with default values that should work in most cases. By default, each pipeline and stage hides the advanced options. Advanced options can include individual properties or complete tabs.
- Simple and Bulk Edit Mode
- Runtime Values
  Runtime values are values that you define outside of the pipeline and use for stage and pipeline properties. You can change the values for each pipeline run without having to edit the pipeline.
- Event Generation
- Webhooks
- Notifications
- SSL/TLS Encryption
  Many stages can use SSL/TLS encryption to securely connect to the external system.
- Security in Amazon Stages
- Security in Google Cloud Stages
- Security in Kafka Stages
- Kafka Message Keys
- Authentication in Salesforce Stages
- Implicit and Explicit Validation
- Expression Configuration
- Configuring a Pipeline
Data Formats
Origins
Processors
Destinations
Executors
Dataflow Triggers
Solutions
Multithreaded Pipelines
Microservice Pipelines
Orchestration Pipelines
Data Preview
Rules and Alerts
Tutorial
Troubleshooting
Error Codes
Glossary
Data Formats by Stage
Expression Language
Regular Expressions
Grok Patterns

Pipeline Configuration

Retrying the Pipeline
Rate Limit
Advanced Options
Pipelines and most pipeline stages include advanced options with default values that should work in most cases. By default, each pipeline and stage hides the advanced options. Advanced options can include individual properties or complete tabs.
Simple and Bulk Edit Mode
Runtime Values
Runtime values are values that you define outside of the pipeline and use for stage and pipeline properties. You can change the values for each pipeline run without having to edit the pipeline.
Event Generation
Webhooks
Notifications
SSL/TLS Encryption
Many stages can use SSL/TLS encryption to securely connect to the external system.
Security in Amazon Stages
Security in Google Cloud Stages
Security in Kafka Stages
Kafka Message Keys
Authentication in Salesforce Stages
Implicit and Explicit Validation
Expression Configuration
Configuring a Pipeline

© 2022 StreamSets, Inc.