| CARVIEW |
Navigation
Quick search
Feedback
Did you find this page useful? Do you have a suggestion to improve the documentation? Give us feedback.
If you would like to suggest an improvement or fix for the AWS CLI, check out our contributing guide on GitHub.
User Guide
First time using the AWS CLI? See the User Guide for help getting started.
AWS CLI S3 FAQ¶
S3 FAQ¶
Below are common questions regarding the use of Amazon S3 in the AWS CLI.
Q: Does the AWS CLI validate checksums?¶
The AWS CLI will attempt to perform checksum validation for uploading and downloading files, as described below.
Upload¶
The AWS CLI v2 will calculate and auto-populate a x-amz-checksum-<algorithm> HTTP header by
default for each upload, where <algorithm> is the algorithm used to calculate the checksum.
By default, the Cyclic Redundancy Check 64 (CRC64NVME) algorithm
is used to calculate checksums, but an alternative algorithm can be specified by using the
--checksum-algorithm argument on high-level aws s3 commands. The checksum algorithms
supported by the AWS CLI v2 are:
- CRC64NVME (Recommended)
- CRC32
- CRC32C
- SHA1
- SHA256
Amazon S3 will use the algorithm specified in the header to calculate the checksum of the object. If it does not match the checksum provided, the object will not be stored and an error message will be returned. Otherwise, the checksum is stored in object metadata that you can use later to verify data integrity of download operations (see Download section).
Note
Note that the AWS CLI will perform the above checksum calculations for commands that perform uploads. This
includes high-level commands like aws s3 cp, aws s3 sync, and aws s3 mv, and low-level commands
like aws s3api put-object and aws s3api upload-part.”
For high-level command invocations that result in uploading multiple files (e.g. aws s3 sync),
the same checksum algorithm will be used for all file uploads included in the command execution.
For more information about verifying data integrity in Amazon S3, see Checking object integrity in Amazon S3 in the Amazon S3 User Guide.
Download¶
The AWS CLI will attempt to verify the checksum of downloads when possible. If a non-MD5 checksum is returned with a downloaded object, the CLI will use the same algorithm to recalculate the checksum and verify it matches the one stored in Amazon S3. If checksum validation fails, an error is raised and the request will NOT be retried.
Note
Note that the AWS CLI will perform the above checksum calculations for commands that perform uploads. This includes high-level commands likeaws s3 cp, aws s3 sync, and aws s3 mv, and low-level commands
like aws s3api get-object”For more information about verifying data integrity in Amazon S3, see Checking object integrity in Amazon S3 in the Amazon S3 User Guide.