Greetings,
Today I wanted to take a look into AWS S3 service and all it offers. After reviewing the job descriptions for Python Developers, I noticed that many roles required the knowledge of AWS S3 and Boto3. In this blog I wanted to share some S3 notes that will help others along the way. Let's Go!
Boto3 - Python SDK for AWS which allows directly create, update and delete AWS resources from your Python scripts.
Amazon S3 - Simple Storage Service (2006) AWS first service
- S3 Focus on General Object Storage on the Cloud. ie: Like Dropbox.
- Big files, small files, media content, source code, spreadsheet, etc
- Scalable, Highly Avaliable, Durable, Support integrations with AWS
Useful in a variety of context:
- Website Hosting
- Database Backups
- Data Processing Pipelines
Core Concepts:
- Buckets: Is like a container or a general purpose file system.
- Objects: Content stored inside the buckets
- Media, JSON, ZIP files or Jar files, etc.
- Max Size : 5TB
- Accessed by:
ie: URL: http://s3.amazonaws.com/<bucketname>/<objetname>
ie: programmatically through code
ie: Interacting through the AWS console
S3 Storage Classes:
- allow you to reduce costs, but with sacrifices.... ie: availability, latency, etc...
- Examples: Standard, Intelligent, Infrequent Access, Glacier
- Each tier above has different pricing, latency, availability.
- Standard Tier (Hot Data) -> Infrequent Access (after 30 days) -> Glacier (Cold Data) (after 90 days). This can be configured and automated.
- Lifecycle Rules automate the data movement process.
S3 Security:
- Public access blocked by default.
- Data Protection - Highly durable and available guarantees, encryption in transit and at rest
- Access - Access and resourced based controls with AWS IAM.
- Auditing - Access logs, action based logs, alarms
- Infrasructure security - Built on top of AWS Cloud Infrastructure.
Pricing:
- Dependent on Storage Class
- Three main factors: Storage, Access (GET, POST, etc) Transfer
- 100GB of Storage, 10K PUT Requests, 10 Read Requests
- Standard Tier
- Infrequent
- Glacier
Free Tier: 5GB(Standard Tier), 20K GET, 2K Put