The s3fs Python library (part of the fsspec ecosystem) is the most common way to handle S3 files programmatically. It includes a built-in download method that supports recursion.
: Setting recursive=True tells the library to walk through all subdirectories and download every object found under that prefix. s3fs download recursive
import s3fs # Initialize the filesystem fs = s3fs.S3FileSystem(key='YOUR_ACCESS_KEY', secret='YOUR_SECRET_KEY') # Recursive download from S3 to local # rpath: The S3 path (bucket/folder) # lpath: Your local destination directory fs.download(rpath='my-bucket/remote-folder/', lpath='/path/to/local/folder/', recursive=True) Use code with caution. The s3fs Python library (part of the fsspec
: Since s3fs mimics a POSIX filesystem, cp -r triggers a series of GET requests to S3 for every file in the directory. s3fs download recursive