Overview
Configuration is provided for establishing connections with Amazon S3 service. The configuration is used in the REST Connection Manager.
Setup
Press 
icon to get more information about the connection parameters.
Obtain data
Use REST Source component to get data from service resource.
Insert data
Use REST Destination component to insert data into service resource.
Manage remote files and directories
Use File Transfer Task to manage remote files and directories.
Advanced settings for managing file uploads
When uploading a file optional request headers can be specified. The headers can be specified in the File Transfer Task property AdditionalOptions. The property is optional and can be shown by clicking the Expand Options button.
The format of the property is [header name]=[value]. Multiple headers are separated by newline.
Example how to set canned access control list and server-side encryption:
x-amz-acl=private x-amz-server-side-encryption=AES256
To get full list of available headers for small files smaller than 20MB go to PutObject. For large files go to CreateMultipartUpload.
Quick Start
In this QS, we will show you step-by-step how to create a connection to an Amazon S3 REST API using COZYROC's REST Connection Manager.
Step 1. Run "Visual Studio" and right click on Connection Managers tab and select New Connection.
Step 2. Then "Add SSIS Connection Manager" popup window opens, scroll down and select "Connection managers for REST Connection" then click Add.
Step 3. From the Configuration drop down menu select Amazon S3.
Step 5. Provide the relevant [bucket] and [region] in the "Server" address field, as well as the Access Key and Secret Key.
Step 7. If the test is successful, then click on OK button.
Congratulations! You have successfully created a connection to an Amazon S3 REST API using COZYROC's REST Connection Manager.
Configuration
Base URL address: https://[bucket].s3.[region].amazonaws.com.
- Amazon AWS Signature v4
-
The authentication uses a parameters-based authentication.
The authentication has the following user-defined parameters:
- Access Key: Required. Specify access key.
- Secret Key: Required. Specify secret key.
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
{{=aws4s.authorizationHeader (request, connection)}} -
Host:
{{=request.signedRequest.headers['Host']}} -
X-Amz-Content-Sha256:
{{=request.signedRequest.headers['X-Amz-Content-Sha256']}} -
X-Amz-Date:
{{=request.signedRequest.headers['X-Amz-Date']}}
Documentation: https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html.
- Amazon AWS Signature v4 Temporary Credentials
-
The authentication uses a parameters-based authentication.
The authentication has the following user-defined parameters:
- Access Key: Required. Specify access key.
- Secret Key: Required. Specify secret key.
- Session Token: Required. Specify session token.
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
{{=aws4s.authorizationHeader (request, connection)}} -
Host:
{{=request.signedRequest.headers['Host']}} -
X-Amz-Content-Sha256:
{{=request.signedRequest.headers['X-Amz-Content-Sha256']}} -
X-Amz-Date:
{{=request.signedRequest.headers['X-Amz-Date']}} -
X-Amz-Security-Token:
{{=request.signedRequest.headers['X-Amz-Security-Token']}}
Documentation: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/.
The result is extracted from:{{=utils.ensureArray(response.ListBucketResult.Contents)}}.The action has the following user-defined parameters:
- query-max-keys: Optional. Sets the maximum number of keys returned in the response. The response might contain fewer keys but will never contain more.
- prefix: Optional. Limits the response to keys that begin with the specified prefix.
The following request parameters will be automatically processed:
-
max-keys:
{{=parameters.get('query-max-keys')}} -
prefix:
{{=parameters.get('prefix')}}
- [Create] action
-
Endpoint URL address:
/{{=item.Key}}.The following request parameters will be automatically processed:
-
application/octet-stream:
{{=item.Content}}
-
application/octet-stream:
- [Delete] action
-
Endpoint URL address:
/{{=item.Key}}.
- Key
Template: ShortText.
- LastModified
Template: DateTime. A read-only field.
- ETag
Template: ShortText. A read-only field.
- Size
Template: ShortText. A read-only field.
- Content
Data type: DT_IMAGE The value is gathered from
{{=item.Size ? '/' + item.Key : '' }}address.
- [Read] action
-
The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}}
-
_includeUserParameters:
- [Create] action
-
The action uses PUT method.The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}} -
item:
{{=item}}
-
_includeUserParameters:
- [Update] action
-
The action uses PUT method.The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}} -
item:
{{=item}}
-
_includeUserParameters:
- [Delete] action
-
The action uses DELETE method.
- ShortText
Data type: DT_WSTR ( length 255 )
- LongText
Data type: DT_WSTR ( length 1000 )
- DateTime
Data type: DT_DBTIMESTAMP
Knowledge Base
- Where can I find the documentation for the Amazon S3 Connection?
- Does COZYROC support TLS 1.1 and 1.2 protocols?
- Error Message: The connection type "xxxx" specified for connection manager "xxxx" is not recognized as a valid connection manager type. This error is returned when an attempt is made to create a connection manager for an unknown connection type. Check the spelling in the connection type name.
Related scripts
What's New
- New: Support for VPC endpoints.
- New: Introduced connection.
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.