Book Review: Instant Chef Starter, by John Ewart

April 7th, 2013

Instant Chef Starter is an introductory book about Chef, an open-source configuration management and automation platform. John Ewart and Packt Publishing have published a book that will allow a system administrator with no prior Chef experience to get Chef up and running within a day, if not a few hours, by using this guide. If one has already decided that buying a book is the best way to learn Chef basics, I can easily recommend this one.

The book opens with a clear introduction of Chef, along with descriptions of basic components, concepts, and terminology. The biggest benefit to using Chef, or any configuration management software, is to automate and ease the burden of multi-server administration.

(Note that John elected to omit Chef solo and hosted Chef from this book.)

The following section covers installation with a promise to give clear guidance on installation to Debian-based and Redhat-based distributions, as well as a source based install. While the Debian-based and source-based installation instructions were very clear and easy to follow, the Redhat-based instructions were missing. I’m sure that finding the proper installation method would not be difficult for most, and given most Chef users are Debian/Ubuntu proponents, this omission is minor.

Over the next few sections, John takes us through bootstrapping a Chef client, managing cookbooks, recipies, and attributes, as well as data bags and templates. I like the introduction via knife and the web UI and then moving onto knife for more command line power. This allows the reader to see both sides of Chef management and choose what might work best for them.

While this book was designed to introduce Chef to a beginner, I would have liked to see mention made of idempotent operations, source control backed cookbooks and recipes, as well as running chef-client regularly to maintain system state over time rather than ad-hoc execution.

While managing a few servers is not a daunting task for any but the most beginner of system administrator, I would have liked to see an example made mention of which better quantifies the benefits of using a configuration management tool-set once the servers being managed reaches the double digits or further. Take, for instance, the management of 30 servers. Once an operation must be performed on 30 servers that takes 2 minutes for each server, the result is 1 hour of time if no issues are discovered through the process. This could be off-set with Chef in under 10 minutes and off to more important tasks.

While I did find a couple of issues with the book, overall I enjoyed reading it, and found that it provided clear instruction on how to deploy Chef into an enterprise environment. I regularly use Chef to manage client systems and appreciate the benefits that it brings to a system administrator or devops engineer. Configuration management advances and brings standardization to the profession of system administration.

John Ewart, thanks for the good read. I hope to see more from you in the future.

Upcoming book review: Instant Chef Starter

April 4th, 2013

Packt Publishing has sent me a copy of Instant Chef Starter, by John Ewart to review. I should have this review up within the week.

This is a great opportunity as one of the most important tools that any system administrator can learn is configuration management, and Chef is a leader in that space. Chef is unique in that it integrates with major cloud providers and allows management from provisioning to deletion of the virtual server behind the service.

Proxy Splunk via Apache

March 22nd, 2013

I have had to setup Apache to proxy splunk several times over the past 6 months and keep forgetting the splunk configuration to make this work.

Be sure to set the following in /opt/splunk/etc/system/local/web.conf:

enableSplunkWebSSL = 0
root_endpoint = /splunk
tools.proxy.on = True

This assumes a proxy configuration of the following for apache 2.2:

ProxyPass /splunk http://localhost:8000/splunk
ProxyPassReverse /splunk http://localhost:8000/splunk

Be sure to secure your proxy before enabling!

Restart splunk:

/opt/splunk/bin/splunk stop
/opt/splunk/bin/splunk start

How to completely remove a file from git!

February 11th, 2013

I recently made a mistake and committed an ISO file to git that was 2GB in size. I did not immediately notice this issue and made several local commits without a push to github working properly. I did some research and figured out how to fix this problem.

Note that this might not be a good idea if you’ve successfully pushed to your remote repository and share it with othes.

1. Remove the file from each commit with git filter-branch:

git filter-branch -f --tree-filter 'rm -f $FILENAME' HEAD

2. Remove the references:

git reflog expire --expire=now --all

3. Run garbage collection:

git gc --aggressive --prune=now

That worked for me. Let me know if you have a similar experience at: linux (at) itsecureadmin (dot) com.

ldapmodify fails with “Server is unwilling to perform (53)”

January 2nd, 2013

I recentlyr an into an issue when setting up a new LDAP directory using OpenLDAP 2.4.23 on Mac OSX. The issue was that I would get the following error when attempting to modify any entry in the directory:

modifying entry "olcDatabase={1}bdb,cn=config"
ldap_modify: Server is unwilling to perform (53)
        additional info: shadow context; no update referral

A few web searches for this error indicated that it might be bad credentials or that the server was setup as a replication consumer which would forced a read only state. I had dumped the directory from another server where it was a MMR member and thought I had removed the required parts to make it work here (olcSyncRepl, olcServerID, etc..).

I confirmed that the credentials were correct by issuing a search using the credentials used in my attempt to modify the directory.

The problem was that I had the oldMirrorMode directive set to FALSE. The fix was to remove this from the LDIF that I was importing with slapadd and re-import.

Note that you could alternately modify olcMirrorMode and set it to TRUE which should resolve this scenario if you are running MMR and require the olcSyncRepl directives.

AWS VPC DB Security Group

December 18th, 2012

The other day I was working with a client and creating a CloudFormation template that used RDS instances within a VPC. I found that while creating the DB security group object that I was getting an error like the following:

STACK_EVENT  CloudFormationName  DBSecurityGroupName
       AWS::RDS::DBSecurityGroup                2012-12-17T22:30:20Z  CREATE_FAILED
       Please see the documentation for authorizing DBSecurityGroup ingress. For VPC,
       EC2SecurityGroupName and EC2SecurityGroupOwner must be omitted.To
       authorize only the source address of this request (and no other address), pass
       xx.xx.xx.xx/32 as the CIDRIP parameter.

It turns out that beyond the requirement for a DB subnet group, I also needed to change the way that I create DB security groups within the VPC. I solved this problem by using the CIDRIP parameter and included the IP ranges of two private subnets:

    "DBSecurityGroupName": {
       "Type": "AWS::RDS::DBSecurityGroup",
       "Properties": {
          "EC2VpcId" : { "Ref" : "VpcName" },
          "DBSecurityGroupIngress" : [ { "CIDRIP": "10.1.201.0/24" }, { "CIDRIP": "10.1.301.0/24" } ],
          "GroupDescription": "Application Server Access"
        }
    },

The examples given on the official docs page did not help with this issue, I found that I was experimenting until I was able to get this working. I copied the examples and they failed for this particular scenario.

Configure MAC OSX Network Interface from the Command Line

December 10th, 2012

Command line network configuration for the MAC is quite a bit different when compared to Linux or Unix. The networksetup command is used instead of ifconfig to configure devices.

The first step is to get a list of all network services – these are really physical and virtual devices, VPNs, etc..

networksetup -listallnetworkservices

The next step is to get the current settings for the network service that you might want to configure – the following example is using the service “Ethernet” and is enclosed in quotes as some of the services have spaces in the names:

networksetup -getinfo "Ethernet"

To configure an interface manually, use something like the following:

networksetup -setmanual "Ethernet" 192.168.1.10 255.255.255.0 192.168.1.1

Confirm that the settings are correct by issuing the getinfo command once more:

networksetup -getinfo "Ethernet"

MAC OSX: Create a new user on the command line.

November 9th, 2012

One common task that any system administrator will perform is to add a new user to a system or application. I recently had a need to add a new user to a MAC server over an SSH connection and found it much more difficult than it should have been. I ended up writing a script to perform the task.

The script located here will create a new user, add them to the admin and staff groups, make their primary group id staff (gid 20) and be sure their userid is the next available one, as well as creating the default home directory based on the English.lproj template.

Let me know how it works for you.

Enable Remote Desktop Sharing on MAC OSX 10.7 From the CLI

November 9th, 2012

I recently had an issue with a customer where I needed to enable remote screen sharing (VNC) on a MAC OSX server from an SSH terminal.

Thanks to this post, I was able to get it started up with this command.

sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -access -on -restart -agent -privs -all

To connect from a MAC client, open Finder, Go, Connect to Server, and type something like the following:

vnc://x.x.x.x

Nginx and Daemontools – nginx can’t bind – (98: Address already in use.)

September 24th, 2012

A common question or issue that I’ve noted when helping customers when using nginx with daemontools is a flood of messages to the error log relating to a failure to bind on port 80 (or whichever port is configured) like the following:

==> logs/error.log <==
2012/09/24 17:41:48 [emerg] 27628#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:49 [emerg] 27637#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:50 [emerg] 27646#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:51 [emerg] 27657#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:52 [emerg] 27671#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:53 [emerg] 27680#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:54 [emerg] 27689#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:55 [emerg] 27699#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:56 [emerg] 27708#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2012/09/24 17:41:57 [emerg] 27725#0: bind() to 0.0.0.0:80 failed (98: Address already in use)

This is caused by daemontools starting nginx successfully, and then nginx daemonizes, and then daemontools tries to start nginx again, unsuccessfully, logging an error to the log.

The solution to this problem is to disable daemon mode in the main section of the nginx.conf:

daemon off;

Once this addition to the configuration file is made, kill all nginx processes and allow daemontools to restart. The error should no longer be logged.

(Assuming nginx version 1.2.3 for this option.)