Overview
Configuration is provided for establishing connections with Amazon Athena service. The configuration is used in the REST Connection Manager.
Setup
Press icon to get more information about the connection parameters.
Obtain data
Use REST Source component to get data from the service resource.
Insert data
Use REST Destination component to insert data into service resource.
Quick Start
In this guide, we will show you step-by-step how to create a connection to Amazon Athena REST API using COZYROC's REST Connection Manager.
Congratulations! You have now established a connection to your Amazon Athena instance.
In this guide, we will show how to read data from the Amazon Athena service resource using the COZYROC REST Source component.
In this guide, we will show how to write data to the Amazon Athena service resource using the COZYROC REST Destination component.
Configuration
Base URL address: https://athena.[REGION].amazonaws.com
.
- Amazon AWS Signature v4
-
The authentication uses a parameters-based authentication.
The authentication has the following user-defined parameters:
- Access Key: Required. Specify access key.
- Secret Key: Required. Specify secret key.
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
{{=aws4s.authorizationHeader (request, connection)}}
-
Host:
{{=request.signedRequest.headers['Host']}}
-
X-Amz-Content-Sha256:
{{=request.signedRequest.headers['X-Amz-Content-Sha256']}}
-
X-Amz-Date:
{{=request.signedRequest.headers['X-Amz-Date']}}
For further information, please check here.
- ShortText
Data type: DT_WSTR ( length 255 )
- LongText
Data type: DT_WSTR ( length 1000 )
- DateTime
Data type: DT_DBTIMESTAMP
- UnixTime
Data type: DT_DBTIMESTAMP The gathered value is processed with
{{=value && new Date(value * 1000).toUTCString()}}
expression.- engineVersionT
Field components:
-
Uses template:
ShortText
. - Contains the following components: EffectiveEngineVersion, SelectedEngineVersion.
-
Uses template:
- queryExecutionContextT
Field components:
-
Uses template:
ShortText
. - Contains the following components: Catalog, Database.
-
Uses template:
- resultConfingurationT
Field components:
-
Uses template:
ShortText
. - Contains the following components: EncryptionConfiguration, OutputLocation.
-
Uses template:
- [Read] action
-
The action uses POST method.
The result is paginated.The following request parameters will be automatically processed:
-
application/x-amz-json-1.1:
{{=_.extend({ MaxResults: parameters.batchSize, NextToken: parameters.iterator }, parameters)}}
-
application/x-amz-json-1.1:
- [Create] action
-
The action uses POST method.The following request parameters will be automatically processed:
-
X-Amz-Target:
{{=parameters.get('X-Amz-Target')}}
-
application/x-amz-json-1.1:
{{=item}}
-
_includeUserParameters:
{{=parameters}}
-
X-Amz-Target:
- [Update] action
-
The action uses POST method.The following request parameters will be automatically processed:
-
X-Amz-Target:
{{=parameters.get('X-Amz-Target')}}
-
application/x-amz-json-1.1:
{{=item}}
-
_includeUserParameters:
{{=parameters}}
-
X-Amz-Target:
- [Delete] action
-
The action uses POST method.The following request parameters will be automatically processed:
-
X-Amz-Target:
{{=parameters.get('X-Amz-Target')}}
-
application/x-amz-json-1.1:
{{=item}}
-
_includeUserParameters:
{{=parameters}}
-
X-Amz-Target:
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=connection.execute({ url: '/', method: 'POST', headers: { 'X-Amz-Target': 'AmazonAthena.BatchGetNamedQuery' }, body: { content: { NamedQueryIds: response.NamedQueryIds }, contentType: 'application/x-amz-json-1.1' } }).NamedQueries }}
.The action has the following user-defined parameters:
- WorkGroup: Specify workgroup name. Optional.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListNamedQueries
For further information check here.
- [Read single] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=[response.NamedQuery]}}
.The action has the following user-defined parameters:
- NamedQueryId: Required. Specify named query identifier.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.GetNamedQuery
For further information check here.
- [Create] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.CreateNamedQuery
-
application/x-amz-json-1.1:
{{=_.extend({ ClientRequestToken: Math.random() }, item)}}
For further information check here.
-
X-Amz-Target:
- [Delete] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.DeleteNamedQuery
For further information check here.
-
X-Amz-Target:
- NamedQueryId
Template: ShortText.
- Database
Template: ShortText.
- Description
Template: ShortText.
- Name
Template: ShortText.
- QueryString
Template: ShortText.
- WorkGroup
Template: ShortText.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=connection.execute({ url: '/', method: 'POST', headers: { 'X-Amz-Target': 'AmazonAthena.BatchGetQueryExecution' }, body: { content: { QueryExecutionIds: response.QueryExecutionIds }, contentType: 'application/x-amz-json-1.1' } }).QueryExecutions }}
.The action has the following user-defined parameters:
- WorkGroup: Specify workgroup name. Optional.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListQueryExecutions
For further information check here.
- [Read single] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=[response.QueryExecution]}}
.The action has the following user-defined parameters:
- QueryExecutionId: Required. Specify query execution identifier.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.GetQueryExecution
For further information check here.
- [Create] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.StartQueryExecution
-
application/x-amz-json-1.1:
{{=_.extend({ ClientRequestToken: Math.random(), QueryString: item.Query }, item)}}
For further information check here.
-
X-Amz-Target:
- [Delete] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.StopQueryExecution
For further information check here.
-
X-Amz-Target:
- EngineVersion
Template: engineVersionT.
- Query
Template: ShortText.
- QueryExecutionContext
Template: queryExecutionContextT.
- QueryExecutionId
Template: ShortText.
- ResultConfiguration
Template: resultConfingurationT.
- StatementType
Template: ShortText.
- Statistics
Field components:
-
Uses template:
ShortText
. - Contains the following components: DataManifestLocation, DataScannedInBytes, EngineExecutionTimeInMillis, QueryPlanningTimeInMillis, QueryQueueTimeInMillis, ServiceProcessingTimeInMillis, TotalExecutionTimeInMillis.
-
Uses template:
- Status
Field components:
-
Uses template:
UnixTime
. - Contains the following components: CompletionDateTime, State, StateChangeReason, SubmissionDateTime.
-
Uses template:
- WorkGroup
Template: ShortText.
Based on resource template Base.
- [Read] action
-
The result is extracted from:{{=Athena.getQueryResult(parameters)}}
.
The result is paginated.The action has the following user-defined parameters:
- QueryExecutionId: Required. Specify the query execution identifier.
- Timeout: Specify timeout in seconds to wait for query result. Optional.
- [External]
-
The external fields URL address:
/
. The external fields list is extracted from:{{=response.ResultSet.ResultSetMetadata.ColumnInfo}}
.- {{=external.Name}}
Data type: {{=Athena.getDataType(external.Type)}} ( length 255 )
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=response.DataCatalogsSummary}}
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListDataCatalogs
For further information check here.
-
X-Amz-Target:
- [Create] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.CreateDataCatalog
For further information check here.
-
X-Amz-Target:
- [Update] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.UpdateDataCatalog
For further information check here.
-
X-Amz-Target:
- [Delete] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.DeleteDataCatalog
For further information check here.
-
X-Amz-Target:
- Name
Template: ShortText. The gathered value is processed with
{{=item.Name || item.CatalogName}}
expression.- Type
Template: ShortText.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=response.PreparedStatements}}
.The action has the following user-defined parameters:
- WorkGroup: Required. Specify the workgroup name.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListPreparedStatements
For further information check here.
- [Create] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.CreatePreparedStatement
For further information check here.
-
X-Amz-Target:
- [Update] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.UpdatePreparedStatement
For further information check here.
-
X-Amz-Target:
- [Delete] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.DeletePreparedStatement
For further information check here.
-
X-Amz-Target:
- StatementName
A key field. Template: ShortText.
- LastModifiedTime
Template: UnixTime. A read-only field.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=response.WorkGroups}}
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListWorkGroups
For further information check here.
-
X-Amz-Target:
- [Create] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.CreateWorkGroup
For further information check here.
-
X-Amz-Target:
- [Update] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.UpdateWorkGroup
For further information check here.
-
X-Amz-Target:
- [Delete] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.DeleteWorkGroup
-
application/x-amz-json-1.1:
{{={ WorkGroup: item.Name } }}
For further information check here.
-
X-Amz-Target:
- CreationTime
Template: UnixTime. A read-only field.
- Description
Template: ShortText.
- EngineVersion
Template: engineVersionT.
- Name
Template: ShortText.
- State
Template: ShortText.
- Tags
Field components:
- An array.
- Contains the following components: Key, Value.
- Configuration
The value is gathered from
/
address. The gathered value is processed with{{=response.WorkGroup.Configuration}}
expression. Field components:-
Uses template:
engineVersionT
. - Contains the following components: BytesScannedCutoffPerQuery, EnforceWorkGroupConfiguration, EngineVersion, PublishCloudWatchMetricsEnabled, RequesterPaysEnabled, ResultConfiguration, ResultConfigurationUpdates.
-
Uses template:
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=response.DatabaseList}}
.The action has the following user-defined parameters:
- CatalogName: Required. Specify the catalog name.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListDatabases
For further information check here.
- [Read single] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=[response.Database]}}
.The action has the following user-defined parameters:
- CatalogName: Required. Specify the catalog name.
- DatabaseName: Required. Specify the database name.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.GetDatabase
For further information check here.
- Description
Template: ShortText.
- Name
Template: ShortText.
- Parameters
Field components:
-
Uses template:
ShortText
. - Contains the following components: string.
-
Uses template:
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=response.TableMetadataList}}
.The action has the following user-defined parameters:
- CatalogName: Required. Specify the catalog name.
- DatabaseName: Required. Specify the database name.
- Expression: Provide a regex filter that pattern-matches table names. Optional.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListTableMetadata
For further information check here.
- [Read single] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=[response.TableMetadata]}}
.The action has the following user-defined parameters:
- CatalogName: Required. Specify the catalog name.
- DatabaseName: Required. Specify the database name.
- TableName: Required. Specify the table name.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.GetTableMetadata
- Columns
Field components:
- An array.
- Contains the following components: Comment, Name, Type.
- CreateTime
Template: UnixTime. A read-only field.
- LastAccessTime
Template: UnixTime. A read-only field.
- Name
Template: ShortText.
- Parameters
The gathered value is processed with
{{=_.map(value, function(v, k) { return { name: k, value: v }; })}}
expression. Field components:-
Uses template:
ShortText
. - Contains the following components: name, value.
-
Uses template:
- PartitionKeys
Field components:
- An array.
- Contains the following components: Comment, Name, Type.
- TableType
Template: ShortText.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=response.EngineVersions}}
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListEngineVersions
-
X-Amz-Target:
- EffectiveEngineVersion
Template: ShortText.
- SelectedEngineVersion
Template: ShortText.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/
.
The result is extracted from:{{=[response]}}
.The action has the following user-defined parameters:
- ResourceARN: Required. Specify the ResourceARN.
The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.ListTagsForResource
For further information check here.
- [Create] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.TagResource
For further information check here.
-
X-Amz-Target:
- [Delete] action
-
Endpoint URL address:
/
.The following request parameters will be automatically processed:
-
X-Amz-Target:
AmazonAthena.UntagResource
-
application/x-amz-json-1.1:
{{={ ResourceARN: item.ResourceARN, TagKeys: _.pluck(item.Tags, 'Key') } }}
For further information check here.
-
X-Amz-Target:
- ResourceARN
Template: ShortText.
- Tags
Field components:
- An array.
- Contains the following components: Key, Value.
What's New
- New: Introduced connection.
Related documentation
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.