Skip to content

Instantly share code, notes, and snippets.

@DaveOC90
Forked from emersonf/s3etag.sh
Last active January 10, 2020 09:57
Show Gist options
  • Select an option

  • Save DaveOC90/9215f0245fc76e0a0480c7a324938ada to your computer and use it in GitHub Desktop.

Select an option

Save DaveOC90/9215f0245fc76e0a0480c7a324938ada to your computer and use it in GitHub Desktop.
A Bash script to compute ETag values for S3 multipart uploads on OS X.
#!/bin/bash
# From https://gist.github.com/emersonf/7413337
# 12/01/17 Minor edits for ubuntu compatilibility by DaveOC90
if [ $# -ne 2 ]; then
echo "Usage: $0 file partSizeInMb";
exit 0;
fi
file=$1
if [ ! -f $file ]; then
echo "Error: $file not found."
exit 1;
fi
partSizeInMb=$2
fileSizeInMb=$(du -m $file | cut -f 1)
parts=$(($fileSizeInMb / $partSizeInMb))
if [[ $(($fileSizeInMb % $partSizeInMb)) -gt 0 ]]; then
parts=$(($parts + 1));
fi
checksumFile=$(mktemp -t s3md5.XXX)
for part in $(seq 0 $(($parts-1)));do
skip=$(($partSizeInMb * $part))
$(dd bs=1M count=$partSizeInMb skip=$skip if=$file 2>/dev/null | md5sum | awk '{print $1}' >>$checksumFile)
done
echo $(xxd -r -p $checksumFile | md5sum | awk '{print $1}')-$parts
rm $checksumFile
@kbaynes
Copy link

kbaynes commented Jan 22, 2019

Excellent. I can confirm that this works on Ubuntu 18, unlike the forked script.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment