Overview
Avro Source Component is SSIS Data Flow Component for retrieving data from an Apache Avro file that supports multiple outputs via the composite records pattern.
- Supports reading the Apache Avro files.
- Supports the following Avro sources: File and Variable.
- Component metadata is automatically retrieved from the provided Avro file.
- Apache Avro file schema is automatically retrieved from the provided file and gets populated in the component editor.
- Supports composite outputs. Besides the root Avro Source Output that contains the top-level fields, for any nested arrays, corresponding composite outputs get populated.
- Supports an error output for redirecting problematic records (in case of a failure processing the field values).
The component is in `beta` status and is planned to be released in 2.3 version.
Quick Start
In this section we will show you how to set up an Avro Source component.
- In the SSIS Toolbox, locate the COZYROC's Avro Source component and drag it onto the Data Flow canvas.
- Double-click on the component on the canvas.
- Once the component editor opens, select the fIle connection from File Connection menu. If the file is in the correct format, its data schema will be visualized in the Schema text editor.
- When clicking on Columns tab the component would prepare the outputs and external columns by analyzing the existing data in the Schema text editor. Please note that the Avro Source can have multiple outputs (see the article about composite records), which columns you can see. The data in these outputs can be processed by downstream transformation and destination components(e.g. multiple OLE DB Destinations can store the data in SQL Server database).
- Click OK to close the component editor.
Congratulations! You have successfully configured the Avro Source component.
Parameters
Configuration
Use the parameters below to configure the component.
Indicates the source of Avro data. The following options are available:
Value Description File Select an existing File Connection Manager or create a new one. Variable The Avro data is available in a variable. Select a variable or create a new one. JSON string representing the schema of the Avro file.
Knowledge Base
What's New
2.3
- New: Introduced component.
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.