February 5th, 2010
I often run into issues on the command line where I’d like to perform a non-greedy search and replace. This is not possible with sed, grep, or egrep, AFAIK, so I must resort to perl. Here is how it’s done:
perl -pe 's|"http://(.*?)/.*$|$1|g'
The above example will take a list of requested URIs or search domains from an apache log and print out only the domain. I’m using pipes instead of slashes in the regex to eliminate the need to escape the slashes.
Note the question mark used to make a non-greedy match.
Posted in Open Source Software | No Comments »
February 4th, 2010
On busy sites the (internal dummy connection) message in apache logs can be a major annoyance as it fills the logs. A simple way to filter this out is to use the following log declarations (assuming IPV6 capable host):
SetEnvIf Remote_Addr "::1" dontlog
CustomLog /var/log/httpd/access_log combined env=!dontlog
RE: http://wiki.apache.org/httpd/InternalDummyConnection
Posted in Open Source Software | No Comments »
February 3rd, 2010
I recently ran into an issue where I needed to decode some base64 text from the command line and used perl to manage the task:
perl -MMIME::Base64 -ne 'print decode_base64($_)' < /tmp/input.txt
RE: http://perldoc.perl.org/MIME/Base64.html
Posted in Open Source Software | No Comments »
February 2nd, 2010
The following syntax can be used to remove a user from an OpenLDAP group:
$ ldapmodify -x -D "cn=manager,dc=example,dc=com" -W <<EOF
> dn: cn=sshusers,ou=groups,dc=example,dc=com
> changetype: modify
> delete: memberUid
> memberUid: previousMember
> EOF
Posted in Open Source Software | No Comments »
January 25th, 2010
Control characters require an escape sequence prior to using in a search and replace operation in VIM. In Linux, the escape character is Ctl-v. Windows requires a Ctl-q.
Remove all ^M characters from a file on Linux:
%s/^V^M//g
Remove all ESCAPES (\27, ^[) from a file on Windows:
%s/^q^[//g
Posted in Open Source Software | No Comments »
January 6th, 2010
To rename all html files in a particular directory to shtml files, use the following loop:
for file in *.html
do
mv ${file} ${file%%.html}.shtml
done
This uses the ${variable%%match} format which strips the longest match from the end of the variable.
Posted in Linux, Open Source Software, Tip of the day! | No Comments »
November 9th, 2009
I recently upgraded some hosts to Fedora 11 which has Samba 3.4 included. I configure most of the hosts I control to be integrated with Active Directory for authentication and this upgrade broke that authentication.
The problem was that the winbind daemon was not able to query the LDAP server which was used as the idmap directory due to the way I had configured the idmap backend in smb.conf. This caused winbind and samba to restart successfully and I could enumerate groups and users perfectly well in bulk. I was not able, however, to enumerate a single user using ‘wbinfo -i ‘ or groups and I was not able to login.
This problem was caused by this config value:
idmap backend = "ldap:ldap://ldap1,ldap://ldap2"
After a bit of troubleshooting, I discovered that winbind was not able to query the LDAP server successfully. I fixed this issue by changing the above config value to:
idmap backend = ldap:ldap://ldap1 ldap://ldap2
I then restarted winbind and tested failover by enumerating a few users and then stopping the primary ldap server (ldap1) and enumerating a few more users.
Posted in Uncategorized | No Comments »
June 1st, 2009
A fairly typical scenario when installing software that does not come fro the distribution package manager is to install an application and find that it cannot find a library necessary to run, although the library is definitely installed. I recently ran into this issue when compiling spine on an old Debian Sarge system.
$ ./spine
./spine: error while loading shared libraries: libmysqlclient_r.so.15: cannot open shared object file: No such file or directory
I verified that this file existed with locate.
$ locate libmysqlclient_r.so.15
/usr/local/mysql-5.0.45-linux-i686/lib/libmysqlclient_r.so.15.0.0
/usr/local/mysql-5.0.45-linux-i686/lib/libmysqlclient_r.so.15
The problem was that I did not have the mysql library directory included in /etc/ld.so.conf or /etc/ld.so.conf.d. To fix the problem, I added the mysql library path to /etc/ld.so.conf and re-ran ldconfig using sudo.
#/etc/ld.so.conf/usr/local/lib/sasl2
/usr/local/lib
/lib
/usr/lib
/usr/lib/atlas
/usr/local/mysql-5.0.45-linux-i686/lib
include /etc/ld.so.conf.d/*.conf
Then run ldconfig:
$ sudo ldconfig
I was then able to run spine without any issues.
Posted in Uncategorized | No Comments »
May 21st, 2009
When reviewing the history file in bash, it’s terrible not knowing when a command was executed. Using the HISTTIMEFORMAT variable in a .bashrc file, the timestamp can be added to all commands.
# ~/.bashrc
HISTTIMEFORMAT=”%m/%d/%y %I:%M:%S %p ”
Sample output:
525 05/21/09 07:56:46 PM tail -f /var/log/messages /var/log/secure
As you can see, the command is preceeded by the line number and a timestamp.
Posted in Uncategorized | No Comments »
May 21st, 2009
I just spent a bit of time troubleshooting a path issue with sudo on a CentOS server. This is a note to self to preserve this bit of knowledge.
When dealing with sudo, the only time the PATH environment variable will be preserved is if it’s in .bashrc and NOT in .bash_profile.
Posted in Uncategorized | No Comments »