S3 download zip file

Notify me of new comments via email. Notify me of new posts via email. Skip to content. Like this: Like Loading Leave a Reply Cancel reply Enter your comment here Fill in your details below or click an icon to log in:. Email required Address never made public. Name required. After looking around for a while, it looked like most web developers faced this same problem and somehow fixed it but the only available information online was not satisfactory to me.

The old way of sharing files in Amazon S3 is to download files from S3 storage, zip them locally, and upload them back to S3. While this method works, it is definitely not good if you have less storage and need to create a large zip file. It is also very resource intensive and time consuming. It also requires cleanup at the end and the entire process consumes almost twice the storage needed for the zipped file.

Another possible option is to compress the files before putting them in S3 storage. This method works well for static files but does not address the case where clients need an assortment of files.

It also does not cover dynamically generated files. All those solutions are good, but they all fail to address the real world issue where files are generated dynamically.

There is no way to know upfront what clients require or which files need sharing. It also does not address the situation where a user or web developer has less RAM and storage memory to squander around. Find centralized, trusted content and collaborate around the technologies you use most.

Connect and share knowledge within a single location that is structured and easy to search. S3 is not a file server, nor does it offer operating system file services, such as data manipulation. Yes, you can use AWS lambda to do the same thing, but lambda is bounds to seconds 15 mins execution timeout Thus it is recommended to allocate more RAM to boost lambda execution performance.

Just access the file directly. If you intend to share the file publicly, you must look into security issue seriously and impose download restriction. AWS traffics out to internet is never cheap. If you need individual files objects in S3 compressed, then it is possible to do so in a round-about way.

Hi I recently have to do that for my application -- serve a bundle of files in zip format through a url link that the users can download. In a nutshell, first create an object using BytesIO method, then use the ZipFile method to write into this object by iterating all the s3 objects, then use put method on this zip object and create a presiged url for it.

First, call this function to get the zip object, ObjectKeys are the s3 objects that you need to put into the zip file. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Collectives on Stack Overflow. Learn more. Asked 4 years, 8 months ago. Active 1 year ago. Viewed 50k times.

Improve this question.