AWS Glue Data Source Connector
Configures, tests, and troubleshoots AWS Glue connections to JDBC databases, Redshift, Snowflake, and BigQuery with secure credential and VPC handling.
Streamline data lake and ETL setup by establishing robust AWS Glue connections to external data stores, including Oracle, SQL Server, PostgreSQL, MySQL, RDS, Amazon Redshift, Snowflake, and Google BigQuery. This skill automates candidate source discovery, configures network prerequisites like VPC subnets and security groups, enforces security best practices using AWS Secrets Manager and IAM DB authentication, and executes two-phase connection testing (network sanity checks and engine-level queries) to ensure seamless data movement before running Glue jobs.
Key Features
01Validates VPC routing, S3 VPC gateway endpoints, security groups, and subnet AZ matching
0213 GitHub stars
03Integrates secure credential storage using AWS Secrets Manager or IAM DB authentication
04Configures Glue connections for JDBC databases, Redshift, Snowflake, and BigQuery
05Discovers existing RDS clusters, database instances, and Redshift clusters automatically
06Performs two-phase connection testing including API health checks and engine query smoke tests
Use Cases
01Establishing secure AWS Glue connectivity to on-premises or cloud databases before running ETL jobs
02Connecting modern cloud warehouses like Snowflake and BigQuery to AWS data lake pipelines
03Diagnosing and resolving Glue connection timeouts, SSL handshake failures, and VPC endpoint misconfigurations