Monitoring data transfers with Amazon CloudWatch metrics (original) (raw)
Monitoring data transfers with Amazon CloudWatch metrics - AWS DataSync
CloudWatch metrics for DataSync
Amazon CloudWatch provides metrics to track DataSync transfer performance and troubleshoot issues with your transfer task.
You can monitor AWS DataSync transfer performance by using Amazon CloudWatch metrics. DataSync metrics are automatically sent to CloudWatch in 5-minute intervals (regardless of how you configure logging). The metrics are retained for a period of 15 months.
To see CloudWatch metrics for DataSync, you can use the following tools:
- The CloudWatch console
- The CloudWatch CLI
- The CloudWatch API
- The DataSync console (on the task execution's details page)
For more information, see the Amazon CloudWatch User Guide.
CloudWatch metrics for DataSync
DataSync metrics use the aws/datasync namespace and provide metrics for the following dimensions:
- AgentId – The unique ID of the agent (if your task uses an agent).
- TaskId – The unique ID of the task. It takes the form of
task-`01234567890abcdef`.
The aws/datasync namespace includes the following metrics. Some metrics aren't available with every task mode.
| CloudWatch metric | Task mode support | Description |
|---|---|---|
| BytesCompressed | Basic | The number of physical bytes that DataSync transfers over the network after compression (if compression is possible). This number is typically less than BytesTransferred unless the data isn't compressible. Unit: Bytes |
| BytesPreparedDestination | Basic | The number of logical bytes that DataSync prepares at the destination location. Unit: Bytes |
| BytesPreparedSource | Basic | The number of logical bytes that DataSync prepares at the source location. Unit: Bytes |
| BytesTransferred | Basic | The number of bytes that DataSync sends to the network before compression (if compression is possible). For the number of bytes transferred over the network, see the BytesCompressed metric. Unit: Bytes |
| BytesVerifiedDestination | Basic | The number of logical bytes that DataSync verifies at the destination location. Unit: Bytes |
| BytesVerifiedSource | Basic | The number of logical bytes that DataSync verifies at the source location. Units: Bytes |
| BytesWritten | Enhanced, Basic | The number of logical bytes that DataSync writes to the destination location. Unit: Bytes |
| FilesDeleted | Enhanced, Basic | The number of files, objects, and directories that DataSync deletes in your destination location. If you don't configure your task todelete data in the destination that isn't in the source, the value is always 0. Unit: Count |
| FilesListedSource | Enhanced | The number of objects that DataSync finds at your source location. Unit: Count |
| FilesPrepared | Enhanced | The number of objects that DataSync will attempt to transfer after comparing your source and destination locations. For more information, see How DataSync prepares your data transfer. This metric isn't applicable if you configure your task to transfer all data. In that scenario, DataSync copies everything from the source to the destination without comparing differences between the locations. Unit: Count |
| FilesPreparedDestination | Basic | The number of files, objects, and directories that DataSync prepares at the destination location. Unit: Count |
| FilesPreparedSource | Basic | The number of files, objects, and directories that DataSync prepares at the source location. Unit: Count |
| FilesSkipped | Basic | The number of files, objects, and directories that DataSync skips during your transfer. Unit: Count |
| FilesTransferred | Enhanced, Basic | The number of files, objects, and directories that DataSync transfers over the network. This value is updated periodically during thetask execution when something is read from the source and sent over the network. NoteThis value can be less thanEstimatedFilesToTransfer in a DescribeTaskExecution response if DataSync fails to transfer something. In some cases, this value can also be greater than EstimatedFilesToTransfer. This metric is implementation-specific for some location types, so don't use it as an exact indication of what transferred or to monitor your task execution. Unit: Count |
| FilesVerified | Enhanced | The number of objects that DataSync verifies during your transfer. Unit: Count |
| FilesVerifiedDestination | Basic | The number of files, objects, and directories that DataSync verifies at the destination location. Unit: Count |
| FilesVerifiedSource | Basic | The number of files, objects, and directories that DataSync verifies at the source location. Unit: Count |
Understanding data transfer performance counters
Monitoring data transfers with task reports
Did this page help you? - Yes
Thanks for letting us know we're doing a good job!
If you've got a moment, please tell us what we did right so we can do more of it.
Did this page help you? - No
Thanks for letting us know this page needs work. We're sorry we let you down.
If you've got a moment, please tell us how we can make the documentation better.