[Bps-public-commit] app-aws-cloudwatch-monitor branch split-metric-payload-upload created. 7c924c2062453721d8697b5e7f64ad1e633928e4

BPS Git Server git at git.bestpractical.com
Thu Jul 28 16:44:47 UTC 2022


This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "app-aws-cloudwatch-monitor".

The branch, split-metric-payload-upload has been created
        at  7c924c2062453721d8697b5e7f64ad1e633928e4 (commit)

- Log -----------------------------------------------------------------
commit 7c924c2062453721d8697b5e7f64ad1e633928e4
Author: Blaine Motsinger <blaine at bestpractical.com>
Date:   Wed May 11 18:22:27 2022 -0500

    Add POSIX to requirements

diff --git a/Makefile.PL b/Makefile.PL
index 9370602..a7cf72d 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -49,6 +49,7 @@ WriteMakefile(
         'Module::Loader' => 0,
         'parent' => 0,
         'Pod::Usage' => '1.67',  # rewrite in 1.62, bugfixes in 1.67
+        'POSIX' => 0,
         'strict' => 0,
         'Time::Piece' => 0,
         'Try::Tiny' => 0,
commit 18ba4249b7b4440fef096083391f36c00a946728
Author: Blaine Motsinger <blaine at bestpractical.com>
Date:   Wed May 11 18:19:18 2022 -0500

    Remove metric upload limitation from docs

diff --git a/README b/README
index 718c282..1390184 100644
--- a/README
+++ b/README
@@ -69,30 +69,6 @@ CONFIGURATION
     exists, "config.ini" will be loaded from there regardless of a config
     file in "/etc/aws-cloudwatch-monitor/".
 
-KNOWN LIMITATIONS
-  AWS CloudWatch limits each upload to no more than 20 different metrics
-    AWS CloudWatch will return a 400 response if attempting to upload more
-    than 20 different metrics at once.
-
-    <https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_Pu
-    tMetricData.html>
-
-    A metrics collection can quickly exceed 20 metrics since each check
-    module gathers multiple metrics.
-
-     aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs --check Memory --check DiskSpace --check Inode --disk-path /
-     Failed to call CloudWatch: HTTP 400. Message: The collection MetricData must not have a size greater than 20.
-
-    Until this limitation is worked around in a future release of
-    "App::AWS::CloudWatch::Monitor", splitting the checks into separate
-    aws-cloudwatch-monitor commands allows the uploads to succeed.
-
-     aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs
-     Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
-     aws-cloudwatch-monitor --check Memory --check DiskSpace --check Inode --disk-path /
-     Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
 BUGS AND ENHANCEMENTS
     Please report any bugs or feature requests at rt.cpan.org
     <https://rt.cpan.org/Public/Dist/Display.html?Name=App-AWS-CloudWatch-Mo
diff --git a/README.md b/README.md
index 3eb725f..6bf27bc 100644
--- a/README.md
+++ b/README.md
@@ -68,27 +68,6 @@ After creating the file, edit and update the values accordingly.
 
 **NOTE:** If the `$ENV{HOME}/.config/aws-cloudwatch-monitor/` directory exists, `config.ini` will be loaded from there regardless of a config file in `/etc/aws-cloudwatch-monitor/`.
 
-# KNOWN LIMITATIONS
-
-## AWS CloudWatch limits each upload to no more than 20 different metrics
-
-AWS CloudWatch will return a 400 response if attempting to upload more than 20 different metrics at once.
-
-[https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API\_PutMetricData.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_PutMetricData.html)
-
-A metrics collection can quickly exceed 20 metrics since each check module gathers multiple metrics.
-
-    aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs --check Memory --check DiskSpace --check Inode --disk-path /
-    Failed to call CloudWatch: HTTP 400. Message: The collection MetricData must not have a size greater than 20.
-
-Until this limitation is worked around in a future release of `App::AWS::CloudWatch::Monitor`, splitting the checks into separate [aws-cloudwatch-monitor](https://metacpan.org/pod/aws-cloudwatch-monitor) commands allows the uploads to succeed.
-
-    aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs
-    Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
-    aws-cloudwatch-monitor --check Memory --check DiskSpace --check Inode --disk-path /
-    Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
 # BUGS AND ENHANCEMENTS
 
 Please report any bugs or feature requests at [rt.cpan.org](https://rt.cpan.org/Public/Dist/Display.html?Name=App-AWS-CloudWatch-Monitor).
diff --git a/lib/App/AWS/CloudWatch/Monitor.pm b/lib/App/AWS/CloudWatch/Monitor.pm
index 7bbaf85..94385ce 100644
--- a/lib/App/AWS/CloudWatch/Monitor.pm
+++ b/lib/App/AWS/CloudWatch/Monitor.pm
@@ -256,27 +256,6 @@ After creating the file, edit and update the values accordingly.
 
 B<NOTE:> If the C<$ENV{HOME}/.config/aws-cloudwatch-monitor/> directory exists, C<config.ini> will be loaded from there regardless of a config file in C</etc/aws-cloudwatch-monitor/>.
 
-=head1 KNOWN LIMITATIONS
-
-=head2 AWS CloudWatch limits each upload to no more than 20 different metrics
-
-AWS CloudWatch will return a 400 response if attempting to upload more than 20 different metrics at once.
-
-L<https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_PutMetricData.html>
-
-A metrics collection can quickly exceed 20 metrics since each check module gathers multiple metrics.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs --check Memory --check DiskSpace --check Inode --disk-path /
- Failed to call CloudWatch: HTTP 400. Message: The collection MetricData must not have a size greater than 20.
-
-Until this limitation is worked around in a future release of C<App::AWS::CloudWatch::Monitor>, splitting the checks into separate L<aws-cloudwatch-monitor> commands allows the uploads to succeed.
-
- aws-cloudwatch-monitor --check Process --process apache --process postgres --process master --process emacs
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
- aws-cloudwatch-monitor --check Memory --check DiskSpace --check Inode --disk-path /
- Successfully reported metrics to CloudWatch. Reference Id: <snip>
-
 =head1 BUGS AND ENHANCEMENTS
 
 Please report any bugs or feature requests at L<rt.cpan.org|https://rt.cpan.org/Public/Dist/Display.html?Name=App-AWS-CloudWatch-Monitor>.
commit 66c6cc7ed4677d386db815e572527dd509c8b89c
Author: Blaine Motsinger <blaine at bestpractical.com>
Date:   Wed May 11 17:01:01 2022 -0500

    Split metrics into sets of 20 to upload to AWS
    
    AWS CloudWatch will return a 400 response if attempting to upload
    more than 20 different metrics at once.
    
    This commit works around that limitation by uploading metrics to
    AWS in sets of 20.

diff --git a/lib/App/AWS/CloudWatch/Monitor.pm b/lib/App/AWS/CloudWatch/Monitor.pm
index 1eca77b..7bbaf85 100644
--- a/lib/App/AWS/CloudWatch/Monitor.pm
+++ b/lib/App/AWS/CloudWatch/Monitor.pm
@@ -22,6 +22,7 @@ use App::AWS::CloudWatch::Monitor::CloudWatchClient;
 use List::Util;
 use Try::Tiny;
 use Module::Loader;
+use POSIX ();
 
 our $VERSION = '0.03';
 
@@ -56,9 +57,7 @@ sub run {
         sleep( rand(20) );
     }
 
-    my $param = {};
-    $param->{Input}{Namespace}  = 'System/Linux';
-    $param->{Input}{MetricData} = [];
+    my @metrics_all = ();
 
     my $checks = delete $opt->{check};
 
@@ -95,11 +94,11 @@ sub run {
             push( @{ $metric->{Dimensions} }, { 'Name' => 'InstanceId', 'Value' => $instance_id } );
             $metric->{Timestamp} = App::AWS::CloudWatch::Monitor::CloudWatchClient::get_timestamp();
 
-            push( @{ $param->{Input}{MetricData} }, $metric );
+            push( @metrics_all, $metric );
         }
     }
 
-    unless ( scalar @{ $param->{Input}{MetricData} } ) {
+    unless ( scalar @metrics_all ) {
         print "\nNo metrics to upload; exiting\n\n";
         exit;
     }
@@ -109,24 +108,37 @@ sub run {
     $opt->{retries}             = 2;
     $opt->{'user-agent'}        = CLIENT_NAME . "/$VERSION";
 
-    my $response = App::AWS::CloudWatch::Monitor::CloudWatchClient::call_json( 'PutMetricData', $param, $opt );
-    my $code     = $response->code;
-    my $message  = $response->message;
+    my $num_of_sets = POSIX::ceil( scalar @metrics_all / 20 );
+    if ( !$opt->{'from-cron'} && $num_of_sets > 1 ) {
+        print "Splitting metrics into $num_of_sets uploads\n";
+    }
+
+    while ( my @metric_set = splice @metrics_all, 0, 20 ) {
+        my $param = {};
+        $param->{Input}{Namespace}  = 'System/Linux';
+        $param->{Input}{MetricData} = [];
+
+        push( @{ $param->{Input}{MetricData} }, @metric_set );
 
-    if ( $code == 200 && !$opt->{'from-cron'} ) {
-        if ( $opt->{verify} ) {
-            print "\nVerification completed successfully. No actual metrics sent to CloudWatch.\n\n";
+        my $response = App::AWS::CloudWatch::Monitor::CloudWatchClient::call_json( 'PutMetricData', $param, $opt );
+        my $code     = $response->code;
+        my $message  = $response->message;
+
+        if ( $code == 200 && !$opt->{'from-cron'} ) {
+            if ( $opt->{verify} ) {
+                print "Verification completed successfully. No actual metrics sent to CloudWatch.\n";
+            }
+            else {
+                my $request_id = $response->headers->{'x-amzn-requestid'};
+                print "Successfully reported metrics to CloudWatch. Reference Id: $request_id\n";
+            }
         }
-        else {
-            my $request_id = $response->headers->{'x-amzn-requestid'};
-            print "\nSuccessfully reported metrics to CloudWatch. Reference Id: $request_id\n\n";
+        elsif ( $code < 100 ) {
+            die "error: $message\n";
+        }
+        elsif ( $code != 200 ) {
+            die "Failed to call CloudWatch: HTTP $code. Message: $message\n";
         }
-    }
-    elsif ( $code < 100 ) {
-        die "error: $message\n";
-    }
-    elsif ( $code != 200 ) {
-        die "Failed to call CloudWatch: HTTP $code. Message: $message\n";
     }
 
     return;
-----------------------------------------------------------------------


hooks/post-receive
-- 
app-aws-cloudwatch-monitor


More information about the Bps-public-commit mailing list