How To Determine the Size of an S3 Bucket
This one came up while working on my home network / photo management setup. I’ve set my Synology DS216+ NAS to use Cloud Sync to back up my files to an Amazon S3 bucket (see this post for some more information on using S3 for backups). The problem was it was taking a very long time, and I needed to figure out how much had transferred.
Unfortunately, Amazon has no simple mechanism for determining the size of an S3 bucket. I found a couple posts on StackOverflow showing how to do it, but they seemed overly complex.
While you can get a bucket size using several third party GUI tools, the command line approach is quick and easy. It does require the Amazon Command line Tools to be installed, and access keys generated, but once that’s done, you can quickly query Amazon for just about anything.
Here’s the command I used to determine the size of my bucket. This is on a mac:
$ aws s3 ls s3://BUCKETNAME --recursive | awk 'total+=$3 ENDprint "total =",total/1024/1024," MB"'
This will give back something like:
total = 245032 MB
Voila! Time for that command can vary depending on the size of the bucket. For me, with around 20,000 photos stored, it takes about 20 seconds.