Use requests.get(url, stream=True) followed by s3_client.upload_fileobj(r.raw, bucket, key) .
Downloading files directly to an Amazon S3 bucket is a common architectural pattern for developers who need to transfer large datasets, automate backups, or migrate media without consuming local disk space. By streaming data directly from a source URL to S3, you avoid the bottlenecks of local storage and the latency of multi-step transfers. Top Methods to Download Files Directly to S3 download file directly to s3 bucket
If you are using Python, you can use the requests library to fetch the file as a stream and boto3 to upload it as a file-like object. Use requests
For quick transfers on an EC2 instance or local machine, you can pipe the output of a download command directly into the AWS CLI cp command using the dash ( - ) character to represent standard input. Top Methods to Download Files Directly to S3
Large files that would exceed your local disk capacity.