[Bps-public-commit] app-aws-cloudwatch-monitor branch 0.03/split-metric-payload-upload created. 7e4b3f1511c4a880c151b61900da1dd43fb72064
BPS Git Server
git at git.bestpractical.com
Tue May 17 21:18:52 UTC 2022
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "app-aws-cloudwatch-monitor".
The branch, 0.03/split-metric-payload-upload has been created
at 7e4b3f1511c4a880c151b61900da1dd43fb72064 (commit)
- Log -----------------------------------------------------------------
commit 7e4b3f1511c4a880c151b61900da1dd43fb72064
Author: Blaine Motsinger <blaine at bestpractical.com>
Date: Wed May 11 18:22:27 2022 -0500
Add POSIX to requirements
diff --git a/Makefile.PL b/Makefile.PL
index aa61497..2f156c1 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -48,6 +48,7 @@ WriteMakefile(
'Module::Loader' => 0,
'parent' => 0,
'Pod::Usage' => '1.67', # rewrite in 1.62, bugfixes in 1.67
+ 'POSIX' => 0,
'strict' => 0,
'Try::Tiny' => 0,
'URI::Escape' => 0,
commit fdec15299258685a8afcc7ddd41c78b4188a1faf
Author: Blaine Motsinger <blaine at bestpractical.com>
Date: Wed May 11 18:19:18 2022 -0500
Remove metric upload limitation from docs
diff --git a/README b/README
index cfbc7c6..e278fe6 100644
--- a/README
+++ b/README
@@ -60,30 +60,6 @@ CONFIGURATION
exists, "config.ini" will be loaded from there regardless of a config
file in "/etc/aws-cloudwatch-monitor/".
-KNOWN LIMITATIONS
- AWS CloudWatch limits each upload to no more than 20 different metrics
- AWS CloudWatch will return a 400 response if attempting to upload more
- than 20 different metrics at once.
-
- <https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_Pu
- tMetricData.html>
-
- A metrics collection can quickly exceed 20 metrics since each check
- module gathers multiple metrics.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs --check Memory --check DiskSpace --check Inode --disk-path /
- Failed to call CloudWatch: HTTP 400. Message: The collection MetricData must not have a size greater than 20.
-
- Until this limitation is worked around in a future release of
- "App::AWS::CloudWatch::Monitor", splitting the checks into separate
- aws-cloudwatch-monitor commands allows the uploads to succeed.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
- aws-cloudwatch-monitor --check Memory --check DiskSpace --check Inode --disk-path /
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
BUGS AND ENHANCEMENTS
Please report any bugs or feature requests at rt.cpan.org
<https://rt.cpan.org/Public/Dist/Display.html?Name=App-AWS-CloudWatch-Mo
diff --git a/README.md b/README.md
index 5cea11c..691dede 100644
--- a/README.md
+++ b/README.md
@@ -58,27 +58,6 @@ After creating the file, edit and update the values accordingly.
**NOTE:** If the `$ENV{HOME}/.config/aws-cloudwatch-monitor/` directory exists, `config.ini` will be loaded from there regardless of a config file in `/etc/aws-cloudwatch-monitor/`.
-# KNOWN LIMITATIONS
-
-## AWS CloudWatch limits each upload to no more than 20 different metrics
-
-AWS CloudWatch will return a 400 response if attempting to upload more than 20 different metrics at once.
-
-[https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API\_PutMetricData.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_PutMetricData.html)
-
-A metrics collection can quickly exceed 20 metrics since each check module gathers multiple metrics.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs --check Memory --check DiskSpace --check Inode --disk-path /
- Failed to call CloudWatch: HTTP 400. Message: The collection MetricData must not have a size greater than 20.
-
-Until this limitation is worked around in a future release of `App::AWS::CloudWatch::Monitor`, splitting the checks into separate [aws-cloudwatch-monitor](https://metacpan.org/pod/aws-cloudwatch-monitor) commands allows the uploads to succeed.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
- aws-cloudwatch-monitor --check Memory --check DiskSpace --check Inode --disk-path /
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
# BUGS AND ENHANCEMENTS
Please report any bugs or feature requests at [rt.cpan.org](https://rt.cpan.org/Public/Dist/Display.html?Name=App-AWS-CloudWatch-Monitor).
diff --git a/lib/App/AWS/CloudWatch/Monitor.pm b/lib/App/AWS/CloudWatch/Monitor.pm
index 2914609..1ecf5a5 100644
--- a/lib/App/AWS/CloudWatch/Monitor.pm
+++ b/lib/App/AWS/CloudWatch/Monitor.pm
@@ -246,27 +246,6 @@ After creating the file, edit and update the values accordingly.
B<NOTE:> If the C<$ENV{HOME}/.config/aws-cloudwatch-monitor/> directory exists, C<config.ini> will be loaded from there regardless of a config file in C</etc/aws-cloudwatch-monitor/>.
-=head1 KNOWN LIMITATIONS
-
-=head2 AWS CloudWatch limits each upload to no more than 20 different metrics
-
-AWS CloudWatch will return a 400 response if attempting to upload more than 20 different metrics at once.
-
-L<https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_PutMetricData.html>
-
-A metrics collection can quickly exceed 20 metrics since each check module gathers multiple metrics.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs --check Memory --check DiskSpace --check Inode --disk-path /
- Failed to call CloudWatch: HTTP 400. Message: The collection MetricData must not have a size greater than 20.
-
-Until this limitation is worked around in a future release of C<App::AWS::CloudWatch::Monitor>, splitting the checks into separate L<aws-cloudwatch-monitor> commands allows the uploads to succeed.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
- aws-cloudwatch-monitor --check Memory --check DiskSpace --check Inode --disk-path /
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
=head1 BUGS AND ENHANCEMENTS
Please report any bugs or feature requests at L<rt.cpan.org|https://rt.cpan.org/Public/Dist/Display.html?Name=App-AWS-CloudWatch-Monitor>.
commit cad07ee4aa465ae11e1d2f17aca8d25452b9734d
Author: Blaine Motsinger <blaine at bestpractical.com>
Date: Wed May 11 17:01:01 2022 -0500
Split metrics into sets of 20 to upload to AWS
AWS CloudWatch will return a 400 response if attempting to upload
more than 20 different metrics at once.
This commit works around that limitation by uploading metrics to
AWS in sets of 20.
diff --git a/lib/App/AWS/CloudWatch/Monitor.pm b/lib/App/AWS/CloudWatch/Monitor.pm
index 60eb77c..2914609 100644
--- a/lib/App/AWS/CloudWatch/Monitor.pm
+++ b/lib/App/AWS/CloudWatch/Monitor.pm
@@ -22,6 +22,7 @@ use App::AWS::CloudWatch::Monitor::CloudWatchClient;
use List::Util;
use Try::Tiny;
use Module::Loader;
+use POSIX ();
our $VERSION = '0.01';
@@ -56,9 +57,7 @@ sub run {
sleep( rand(20) );
}
- my $param = {};
- $param->{Input}{Namespace} = 'System/Linux';
- $param->{Input}{MetricData} = [];
+ my @metrics_all = ();
my $checks = delete $opt->{check};
@@ -95,11 +94,11 @@ sub run {
push( @{ $metric->{Dimensions} }, { 'Name' => 'InstanceId', 'Value' => $instance_id } );
$metric->{Timestamp} = App::AWS::CloudWatch::Monitor::CloudWatchClient::get_offset_time(NOW);
- push( @{ $param->{Input}{MetricData} }, $metric );
+ push( @metrics_all, $metric );
}
}
- unless ( scalar @{ $param->{Input}{MetricData} } ) {
+ unless ( scalar @metrics_all ) {
print "\nNo metrics to upload; exiting\n\n";
exit;
}
@@ -109,24 +108,37 @@ sub run {
$opt->{retries} = 2;
$opt->{'user-agent'} = CLIENT_NAME . "/$VERSION";
- my $response = App::AWS::CloudWatch::Monitor::CloudWatchClient::call_json( 'PutMetricData', $param, $opt );
- my $code = $response->code;
- my $message = $response->message;
+ my $num_of_sets = POSIX::ceil( scalar @metrics_all / 20 );
+ if ( !$opt->{'from-cron'} && $num_of_sets > 1 ) {
+ print "Splitting metrics into $num_of_sets uploads\n";
+ }
+
+ while ( my @metric_set = splice @metrics_all, 0, 20 ) {
+ my $param = {};
+ $param->{Input}{Namespace} = 'System/Linux';
+ $param->{Input}{MetricData} = [];
+
+ push( @{ $param->{Input}{MetricData} }, @metric_set );
- if ( $code == 200 && !$opt->{'from-cron'} ) {
- if ( $opt->{verify} ) {
- print "\nVerification completed successfully. No actual metrics sent to CloudWatch.\n\n";
+ my $response = App::AWS::CloudWatch::Monitor::CloudWatchClient::call_json( 'PutMetricData', $param, $opt );
+ my $code = $response->code;
+ my $message = $response->message;
+
+ if ( $code == 200 && !$opt->{'from-cron'} ) {
+ if ( $opt->{verify} ) {
+ print "Verification completed successfully. No actual metrics sent to CloudWatch.\n";
+ }
+ else {
+ my $request_id = $response->headers->{'x-amzn-requestid'};
+ print "Successfully reported metrics to CloudWatch. Reference Id: $request_id\n";
+ }
}
- else {
- my $request_id = $response->headers->{'x-amzn-requestid'};
- print "\nSuccessfully reported metrics to CloudWatch. Reference Id: $request_id\n\n";
+ elsif ( $code < 100 ) {
+ die "error: $message\n";
+ }
+ elsif ( $code != 200 ) {
+ die "Failed to call CloudWatch: HTTP $code. Message: $message\n";
}
- }
- elsif ( $code < 100 ) {
- die "error: $message\n";
- }
- elsif ( $code != 200 ) {
- die "Failed to call CloudWatch: HTTP $code. Message: $message\n";
}
return;
-----------------------------------------------------------------------
hooks/post-receive
--
app-aws-cloudwatch-monitor
More information about the Bps-public-commit
mailing list