441 lines
8.6 KiB
Markdown
441 lines
8.6 KiB
Markdown
# SAP Data Intelligence Operators Reference
|
|
|
|
Complete reference for built-in operators in SAP Data Intelligence.
|
|
|
|
## Table of Contents
|
|
|
|
1. [Operator Generations](#operator-generations)
|
|
2. [ABAP Operators](#abap-operators)
|
|
3. [File and Storage Operators](#file-and-storage-operators)
|
|
4. [Database Operators](#database-operators)
|
|
5. [Messaging Operators](#messaging-operators)
|
|
6. [Script Operators](#script-operators)
|
|
7. [Data Processing Operators](#data-processing-operators)
|
|
8. [Machine Learning Operators](#machine-learning-operators)
|
|
9. [Integration Operators](#integration-operators)
|
|
10. [Workflow Operators](#workflow-operators)
|
|
|
|
---
|
|
|
|
## Operator Generations
|
|
|
|
### Generation 1 (Gen1)
|
|
|
|
Legacy operators with broad compatibility.
|
|
|
|
**Characteristics:**
|
|
- Process-based execution
|
|
- Manual error handling
|
|
- Suitable for simpler workflows
|
|
- Compatible with older graphs
|
|
|
|
### Generation 2 (Gen2)
|
|
|
|
Enhanced operators with advanced features.
|
|
|
|
**Characteristics:**
|
|
- Improved error recovery
|
|
- State management with snapshots
|
|
- Native multiplexing support
|
|
- Better performance characteristics
|
|
|
|
**Gen2 Exclusive Features:**
|
|
- Automatic graph recovery from failures
|
|
- Periodic state checkpoints
|
|
- Structured native message streaming
|
|
- Enhanced Python3 operator
|
|
|
|
**Critical Rule:** Gen1 and Gen2 operators cannot be mixed in the same graph.
|
|
|
|
---
|
|
|
|
## ABAP Operators
|
|
|
|
Operators for integrating ABAP-based SAP systems.
|
|
|
|
### ABAP CDS Reader
|
|
|
|
Reads data from ABAP CDS views.
|
|
|
|
**Configuration:**
|
|
- Connection: ABAP system connection
|
|
- CDS View: Target view name
|
|
- Selection: Filter criteria
|
|
- Package Size: Batch size for reading
|
|
|
|
**Ports:**
|
|
- Output: CDS view data
|
|
|
|
### ABAP Table Reader
|
|
|
|
Reads data from ABAP tables.
|
|
|
|
**Configuration:**
|
|
- Connection: ABAP system connection
|
|
- Table: Target table name
|
|
- Fields: Column selection
|
|
- Where Clause: Filter condition
|
|
|
|
### SLT Connector
|
|
|
|
Connects to SAP Landscape Transformation for real-time replication.
|
|
|
|
**Configuration:**
|
|
- Mass Transfer ID: SLT configuration
|
|
- Table: Source table
|
|
- Initial Load: Enable/disable full load
|
|
- Delta Load: Enable/disable CDC
|
|
|
|
### ODP Consumer
|
|
|
|
Consumes Operational Data Provisioning sources.
|
|
|
|
**Configuration:**
|
|
- Connection: ABAP connection
|
|
- ODP Context: Extraction context (SAPI, ABAP CDS, etc.)
|
|
- ODP Name: Data provider name
|
|
- Extraction Mode: Full or Delta
|
|
|
|
---
|
|
|
|
## File and Storage Operators
|
|
|
|
### Binary File Consumer
|
|
|
|
Reads binary files from storage.
|
|
|
|
**Configuration:**
|
|
- Connection: Storage connection
|
|
- Path: File path pattern
|
|
- Recursive: Include subdirectories
|
|
|
|
**Output:** Binary content
|
|
|
|
### Binary File Producer
|
|
|
|
Writes binary content to files.
|
|
|
|
**Configuration:**
|
|
- Connection: Storage connection
|
|
- Path: Output file path
|
|
- Mode: Overwrite, Append, or Fail if exists
|
|
|
|
### Structured File Consumer
|
|
|
|
Reads structured data from files (CSV, Parquet, ORC, JSON).
|
|
|
|
**Configuration:**
|
|
- Connection: Storage connection
|
|
- Source: File path or pattern
|
|
- Format: CSV, Parquet, ORC, JSON
|
|
- Schema: Column definitions
|
|
|
|
### Structured File Producer
|
|
|
|
Writes structured data to files.
|
|
|
|
**Configuration:**
|
|
- Connection: Storage connection
|
|
- Target: Output path
|
|
- Format: CSV, Parquet, ORC, JSON
|
|
- Partition: Partitioning strategy
|
|
|
|
### Cloud Storage Operators
|
|
|
|
**Amazon S3:**
|
|
- S3 Consumer: Read from S3 buckets
|
|
- S3 Producer: Write to S3 buckets
|
|
|
|
**Azure Blob/ADLS:**
|
|
- Azure Blob Consumer/Producer
|
|
- ADLS Gen2 Consumer/Producer
|
|
|
|
**Google Cloud Storage:**
|
|
- GCS Consumer: Read from GCS buckets
|
|
- GCS Producer: Write to GCS buckets
|
|
|
|
**HDFS:**
|
|
- HDFS Consumer: Read from Hadoop clusters
|
|
- HDFS Producer: Write to Hadoop clusters
|
|
|
|
---
|
|
|
|
## Database Operators
|
|
|
|
### SAP HANA Operators
|
|
|
|
**HANA Client:**
|
|
- Executes SQL statements
|
|
- Supports DDL, DML, queries
|
|
|
|
**Table Consumer:**
|
|
- Reads from HANA tables
|
|
- Supports filtering and projection
|
|
|
|
**Table Producer:**
|
|
- Writes to HANA tables
|
|
- Supports INSERT, UPSERT, DELETE
|
|
|
|
**Flowgraph Executor:**
|
|
- Runs HANA calculation views
|
|
- Executes stored procedures
|
|
|
|
### SQL Operators
|
|
|
|
**SQL Consumer:**
|
|
- Executes SELECT queries
|
|
- Supports parameterized queries
|
|
|
|
**SQL Executor:**
|
|
- Runs DDL/DML statements
|
|
- Returns affected row count
|
|
|
|
### Supported Databases
|
|
|
|
- SAP HANA (Cloud and on-premise)
|
|
- SAP BW/4HANA
|
|
- Microsoft SQL Server
|
|
- Oracle Database
|
|
- PostgreSQL
|
|
- MySQL
|
|
- SAP IQ/Sybase
|
|
|
|
---
|
|
|
|
## Messaging Operators
|
|
|
|
### Kafka Operators
|
|
|
|
**Kafka Consumer:**
|
|
- Subscribes to Kafka topics
|
|
- Supports consumer groups
|
|
- Offset management (earliest, latest, committed)
|
|
|
|
**Kafka Producer:**
|
|
- Publishes to Kafka topics
|
|
- Key/value serialization
|
|
- Partitioning strategies
|
|
|
|
### MQTT Operators
|
|
|
|
**MQTT Consumer:**
|
|
- Subscribes to MQTT topics
|
|
- QoS level configuration
|
|
|
|
**MQTT Producer:**
|
|
- Publishes MQTT messages
|
|
- Retain flag support
|
|
|
|
### Additional Messaging
|
|
|
|
- **NATS**: Lightweight messaging
|
|
- **WAMP**: Web Application Messaging Protocol
|
|
- **AWS SNS**: Amazon Simple Notification Service
|
|
- **SAP Event Mesh**: SAP cloud messaging
|
|
|
|
---
|
|
|
|
## Script Operators
|
|
|
|
### Python Operator (Gen2)
|
|
|
|
Execute Python code within graphs.
|
|
|
|
**Configuration:**
|
|
- Script: Python code
|
|
- Codelanguage: python36 or python39
|
|
|
|
**Example:**
|
|
```python
|
|
def on_input(msg_id, header, body):
|
|
# Process input
|
|
result = transform(body)
|
|
api.send("output", api.Message(result))
|
|
|
|
api.set_port_callback("input", on_input)
|
|
```
|
|
|
|
### JavaScript Operator
|
|
|
|
Execute JavaScript/Node.js code.
|
|
|
|
**Configuration:**
|
|
- Script: JavaScript code
|
|
|
|
**Example:**
|
|
```javascript
|
|
$.setPortCallback("input", function(ctx, s) {
|
|
var result = process(s);
|
|
$.output(result);
|
|
});
|
|
```
|
|
|
|
### R Operator
|
|
|
|
Execute R scripts for statistical analysis.
|
|
|
|
**Configuration:**
|
|
- Script: R code
|
|
- Libraries: Required R packages
|
|
|
|
### Go Operator
|
|
|
|
Execute Go code for high-performance processing.
|
|
|
|
---
|
|
|
|
## Data Processing Operators
|
|
|
|
### Data Transform
|
|
|
|
Visual SQL-like transformation editor.
|
|
|
|
**Nodes:**
|
|
- Projection: Column selection/transformation
|
|
- Aggregation: GROUP BY with functions
|
|
- Join: Combine datasets
|
|
- Union: Merge datasets
|
|
- Case: Conditional logic
|
|
- Filter: Row filtering
|
|
|
|
### Data Quality Operators
|
|
|
|
**Validation Rule:**
|
|
- Define data quality rules
|
|
- Generate validation reports
|
|
|
|
**Anonymization:**
|
|
- Mask sensitive data
|
|
- Hash, shuffle, or generalize
|
|
|
|
**Data Mask:**
|
|
- Apply masking patterns
|
|
- Preserve format while anonymizing
|
|
|
|
### Conversion Operators
|
|
|
|
**Type Converters:**
|
|
- Binary to Table
|
|
- Table to Binary
|
|
- Dynamic to Static
|
|
- Static to Dynamic
|
|
|
|
**Format Converters:**
|
|
- JSON Parser/Formatter
|
|
- CSV Parser/Formatter
|
|
- Avro Encoder/Decoder
|
|
|
|
---
|
|
|
|
## Machine Learning Operators
|
|
|
|
### TensorFlow Operators
|
|
|
|
**TensorFlow Training:**
|
|
- Train TensorFlow models
|
|
- Distributed training support
|
|
|
|
**TensorFlow Serving:**
|
|
- Deploy TensorFlow models
|
|
- REST API inference
|
|
|
|
### PyTorch Operators
|
|
|
|
**PyTorch Training:**
|
|
- Train PyTorch models
|
|
- GPU acceleration
|
|
|
|
### HANA ML Operators
|
|
|
|
**HANA ML Training:**
|
|
- Train models in HANA
|
|
- Automated ML (AutoML)
|
|
|
|
**HANA ML Inference:**
|
|
- Score data with HANA ML models
|
|
|
|
### Metrics Operators
|
|
|
|
**Submit Metrics:**
|
|
- Track training metrics
|
|
- Integration with Metrics Explorer
|
|
|
|
---
|
|
|
|
## Integration Operators
|
|
|
|
### OData Operators
|
|
|
|
**OData Consumer:**
|
|
- Query OData services
|
|
- Supports v2 and v4
|
|
|
|
**OData Producer:**
|
|
- Expose data as OData
|
|
- CRUD operations
|
|
|
|
### REST API Operators
|
|
|
|
**REST Client:**
|
|
- Call REST APIs
|
|
- Configurable HTTP methods
|
|
- Header/body templates
|
|
|
|
**OpenAPI Client:**
|
|
- Generate clients from OpenAPI specs
|
|
- Automatic request/response handling
|
|
|
|
### SAP Integration
|
|
|
|
**SAP CPI Operator:**
|
|
- Trigger SAP Cloud Platform Integration flows
|
|
|
|
**SAP Application Consumer/Producer:**
|
|
- Connect to SAP applications
|
|
- S/4HANA, ECC, SuccessFactors
|
|
|
|
---
|
|
|
|
## Workflow Operators
|
|
|
|
### Data Workflow Operators
|
|
|
|
**Workflow Trigger:**
|
|
- Start workflow execution
|
|
- Scheduled or event-based
|
|
|
|
**Workflow Terminator:**
|
|
- End workflow with status
|
|
|
|
**Pipeline Executor:**
|
|
- Run child pipelines
|
|
- Pass parameters
|
|
|
|
### Control Flow
|
|
|
|
**BW Process Chain:**
|
|
- Execute SAP BW process chains
|
|
|
|
**Data Services Job:**
|
|
- Run SAP Data Services jobs
|
|
|
|
**HANA Flowgraph:**
|
|
- Execute HANA calculation views
|
|
|
|
### Notification
|
|
|
|
**Email Notification:**
|
|
- Send status emails
|
|
- Configurable templates
|
|
|
|
---
|
|
|
|
## Documentation Links
|
|
|
|
- **Operators Reference**: [https://github.com/SAP-docs/sap-hana-cloud-data-intelligence/tree/main/docs/repositoryobjects/data-intelligence-operators](https://github.com/SAP-docs/sap-hana-cloud-data-intelligence/tree/main/docs/repositoryobjects/data-intelligence-operators)
|
|
- **Graphs Reference**: [https://github.com/SAP-docs/sap-hana-cloud-data-intelligence/tree/main/docs/repositoryobjects/data-intelligence-graphs](https://github.com/SAP-docs/sap-hana-cloud-data-intelligence/tree/main/docs/repositoryobjects/data-intelligence-graphs)
|
|
|
|
---
|
|
|
|
**Last Updated**: 2025-11-22
|