Category: Tip of the day!
-
Retrieving puppet facts from AWS System Manager
AWS System Manager makes it easy to store and retrieve parameters for use across servers, services, and applications in AWS. One great benefit is storing secrets for use, as needed. I recently needed to retrieve some parameters to place in a configuration file via puppet and wrote a short script to retrieve these values as…
-
Powershell to ElasticSearch to find ElastAlert
I recently worked on an interesting project where I needed to use a powershell script to query ElasticSearch to find a document that was inserted via ElastAlert. The purpose of this exercise was to determine whether or not a service had been marked down recently, which would determine whether an operation ran that might take…
-
ruby aws-sdk strikes again…
When using ruby to upload files to S3 and trying to use multipart upload, beware the following ArgumentError: …param_validator.rb:32:in `validate!’: unexpected value at params[:server_side_encryption] (ArgumentError) … from /var/lib/jenkins/.gem/ruby/gems/aws-sdk-core-3.6.0/lib/seahorse/client/request.rb:70:in `send_request’ from /var/lib/jenkins/.gem/ruby/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/client.rb:3980:in `list_parts’ … The options passed to list_parts must not include “server_side_encryption”. I always forget to remove this parameter. A good way that I have…
-
ELK + EA — Silencing ElastAlert Alerts
Many shops are realizing the benefit of the ELK stack / Elastic Stack, and the great flexibility that it brings to an infrastructure in the form of centralized logging and reporting which has always been critical when troubleshooting difficult and/or distributed problems. Having many input options (via elastic beats) to choose from, and lots of…
-
Adding git branch and aws profile to your bash prompt…
As a consultant who works in AWS for numerous clients, one of the most important things to keep track of is which AWS CLI profile am I currently on. To help clarify this, I’ve recently added the AWS profile to my bash prompt to remove all doubt. In addition, I’ve added a prompt for the…
-
Throttling Requests with the Ruby aws-sdk
A common problem of late is throttling requests when using the ruby aws-sdk gem to access AWS services. Handling these exceptions is fairly trivial with a while loop like the following: retry_count = 0 retry_success = 0 while retry_success == 0 retry_success = 1 begin # # enter code to interact with AWS here #…
-
Ruby, Rackspace Cloudfiles, Performance, and Fog
I work quite a bit with Rackspace cloudfiles and use ruby to upload and download files using the fog gem. There have been some major performance issues with apps that use the fog gem where there is a lack of understanding around how the container get, new, and save methods work. The basic issue is…
-
Manually creating dokuwiki pages
I recently worked on a project where it would be really convenient to create some dokuwik pages using a script on some html content. I ended up using the CPAN/Perl module to convert HTML content to dokuwiki content, and then script the process necessary to create the pages inside dokuwiki. In general, for a page…
-
AWS Route 53 DNS Migration Adventure
I recently worked with a client to transfer authoritative control of one of their domains to AWS Route 53 and it was a minor adventure. It was trivial to setup the hosted zone in Route 53 and import the zone file, although not all records were created properly. In particular, the MX records that were…
-
Puppet node inheritance deprecation
Puppet 4.0 will deprecate node inheritance which is currently a common way to organize resources. I have been using node inheritance to group common configurations into a basic role and then inherit that with a node declaration like the following: # site.pp … node webserver { # add all generic web server configuration here }…