AWS Made Easy with Boto3
If you’re a starting out as a Data Engineer and using AWS, then life gets a whole lot easier with the use of Boto3, the AWS SDK for Python. Boto3 simplifies integration of your Python applications, libraries, or scripts with AWS services like Amazon S3, EC2, DynamoDB and more. Well, that’s what the documentation says. […]
Query S3 using S3 Select and SQL
S3 Select is a highly valuable and in my option one of the most underappreciated features within AWS S3. As a Data Engineer, it is a must-have in your toolkit. What is S3 Select? A feature within S3 that allows you the Data Engineer to run simple SQL queries on objects in S3 buckets. For […]
Comparing Amazon S3 Storage Options: s3n, s3a, and s3
When I’m building pipelines, it is common to access S3 at some point in the process. In some articles and tutorials, S3N or S3A may be mentioned in the connection string for S3. What is the difference? I look into the differences here. Basically In a nutshell, S3N and S3A are storage options provided by […]