Setting up secure certificates with lighttpd

apache, https, java, lighttpd

Of late, I’ve started to prefer lighttpd to nginx and Apache for several reasons. The configuration of lighttpd is incredibly easy to get running and I’ve found that speed wise, lighttpd doesn’t run any slower than Apache.

I need to setup secure certificates on lighttpd and so I have written up some basic instructions to get this up and running.

Under the “SSL Support” section, you will find some nice exampls for getting this running. This kind of worked for me, but I wanted to forward all requests on the default port 80 to the HTTPS default port 443.

Once you have your pem file and your ca file, you will need to make sure that every request to port 443 will locate the correct private key as well as the CA.

$SERVER[“socket”] == “:443” {
ssl.engine = “enable”
ssl.pemfile = “/etc/lighttpd/certs/meltwater.pem”
ssl.ca-file = “/etc/lighttpd/certs/chain.crt”
}

From there, it will be important that for each individual host, you will redirect all traffic onto port 80 to 443, whilst also specifying where the actual tomcat port is (if necessary).

$HTTP[“host”] =~ “your.domain.com” {

  1. the below ensures that the hostname is extracted using a regexp, so that the user can be re-directed to https (port 443)

$HTTP[“scheme”] == “http” {
$HTTP[“host”] =~ “.*” {
url.redirect = (“.*” => “https://%0$0”)
}
}

  1. this is of course, optional for if you are running a java application on tomcat, but can be adjusted for any other port or application.

proxy.server = (
“” => (
“tomcat” => (
“host” => “127.0.0.1”,
“port” => 8080,
“fix-redirects” => 1
)
)
)

  1. and finally, where your document root is for the app/page

server.document-root = “/var/app”
accesslog.filename = “/var/log/app/application.log”
}

rsync: verbosity with progress bar

linux, rsync

I like to have a verbose, informative rsync when copying files from server to server.

I use the following options as part of my rsync, so that I have a progress bar in human readable format. This is also verbose so that it shows the current status of the file transfer.

rsync -AavlXpzh –progress some_folder/ me@200.100.50.10:~me/some_folder

Puppet error: read server certificate: certificate verify

puppet

If you see an error in puppet, the chances are that you have some kind of configuration setting
Warning: Unable to fetch my node definition, but the agent run will continue:

Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppet]
Error: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppet]
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppet]

The chances are that you have some conflicts when using an old puppet master, the SSL Certificates might be configured wrong.

find /var/lib/puppet -type f -print0 |xargs -0r rm

Puppet flags “Invalid group” when adding users

puppet

If you see a message that tells you “Could not evaluate: invalid group” in Puppet, or any kind of error that says “Invalid group” when running a puppet test it’s possible that you might have some old configuration from another puppet master for adding users.

If that’s the case, you would only need to delete a user then run puppet again and hopefully it’ll use your new puppet settings and re-add the user.

Add a new user to Jenkins with backend access and no administrative access

jenkins

On a Jenkins machine that a colleague setup, I needed to ensure that I was able to login and make changes to the Jenkins instance.

In this instance we were actually using hudson.security.FullControlOnceLoggedInAuthorizationStrategy, so the login was open to the public. The only thing that would restrict access to this Jenkins instance would be the firewall – no problem.

The first thing to do is to check for that information in the config.xml file, which is in the Jenkins base directory. In some Jenkins instances, you might find some user information within that file; but the vast majority of user data is actually stored in a user directory one level above the Jenkins base dir.

SSH into the machine, to the Jenkins base dir and then into the users directory, you will find a list of directories for each user. You’ve got a couple of options now, the first would be to copy a colleagues directory and rename it as your own login user. cd into the newly created directory and locate the config.xml.

In there, modify the fullName as well as the emailAddress nodes in there. The final and most important thing is to setup a passwordHash. You might have your password hashed on another instance of Jenkins elsewhere, you can use that to copy it across. Alternatively, as Jenkins uses jbCrypt, which is a Java implementation of a Blowfish hashing code, you can create one manually.

Afterwards, you shouldn’t even need to restart Jenkins, but you’re going to be able to login to the Jenkins instance manually. With my AuthorizationStrategy, I had administrative privileges.

Capistrano SSH authentication failure

capistrano

After moving a Capistrano build server from one machine to another, I noticed some failures during the first build. This might’ve been an environmental issue, but instead of going for the easy option I preferred to modify this directly in the Capfile.

My error message received was:
connection failed for: some-linux-server01 (Net::SSH::AuthenticationFailed: www-data)

I tried a host of options, adjusting the SSH config, /etc/hosts, and specifying ssh_options[:forward_agent] = true in the Capfile. Nothing worked.

First thing I wanted to do, was setup some more verbose Capistrano logging:
ssh_options[:verbose] = :debug

Then try and running a cap deploy again. That information may shed some light on any issue you have. In my case, it appeared to be a recurring issue with my SSH config – as we’d upgraded both Ruby, Capistrano and the OS – I preferred for the job to be done in the Capfile instead.

Subsequently, I planned to setup the Capfile to locate my SSH keys directly, whilst ignoring config.

The solution:
ssh_options[:keys] = [File.join(ENV[“HOME”], “.ssh”, “id_rsa”)]
ssh_options[:config] = false

…added to the Capfile, commit and then attempt another deploy.

Setup LAMP and phpMyAdmin from scratch on Ubuntu 10+

apache, lamp, linux, mysql, php, phpmyadmin

It’s really easy to get a LAMP webserver up and running, so I thought I’d post some nice easy instructions for installing on a Ubuntu box.

1) sudo apt-get install mysql-server mysql-client
-> you will need to choose a MySQL Root user password

2) sudo apt-get install apache2
-> confirm in your browser that you are able to view a page for your IP address. This could be localhost, or your local IP address.

3) sudo apt-get install php5 libapache2-mod-php5
-> you’re going to need to install PHP next as above.

4) /etc/init.d/apache2 restart
-> …and of course an Apache restart so that PHP is activated

5) sudo apt-get install phpmyadmin
-> you’re going to be asked which webserver to configure. go for apache2. it’ll then probably request your password you selected for MySQL.

6) vim /etc/apache2/apache2.conf
-> use your favorite editor to open up the apache config file

7) Include /etc/phpmyadmin/apache.conf
-> add the following somewhere in the file.

8) /etc/init.d/apache2 restart

9) http://yourhomepage.com/phpmyadmin
-> test 🙂

Linux locale setting failure

linux

When your locales are screwed, you’re going to notice a lot of problems even trying to curl from the command line, for e.g.:
Sorry, command-not-found has crashed! Please file a bug report at:

If you see an error message like this:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = “UTF-8”,
LANG = “en_US.UTF-8”

root@mwtokyo3:~# dpkg-reconfigure locales
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = “UTF-8”,
LANG = “en_US.UTF-8”
are supported and installed on your system.
perl: warning: Falling back to the standard locale (“C”).
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Generating locales…
en_AG.UTF-8… up-to-date
en_AU.UTF-8… up-to-date
en_BW.UTF-8… up-to-date
en_CA.UTF-8… up-to-date
en_DK.UTF-8… up-to-date
en_GB.UTF-8… up-to-date
en_HK.UTF-8… up-to-date
en_IE.UTF-8… up-to-date
en_IN.UTF-8… up-to-date
en_NG.UTF-8… up-to-date
en_NZ.UTF-8… up-to-date
en_PH.UTF-8… up-to-date
en_SG.UTF-8… up-to-date
en_US.UTF-8… up-to-date
en_ZA.UTF-8… up-to-date
en_ZM.UTF-8… up-to-date
en_ZW.UTF-8… up-to-date
Generation complete.

Just add this to your ~/.profile and ~/.bashrc
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

Then:
source ~/.profile
source ~/.bashrc

Then try again, all should be good.

Creating a second instance of MySQL on the same machine

mysql

It’s possible to run multiple instances of MySQL on the same machine, albeit running on a separate port. This guide attempts to explain quickly and easily how it’s done.

So what do we do? First things first, we need to make the data directory for the new mysql2 instance. That can be easily found from running ‘ps’ and identifying your current instance of mysql, let’s call it mysql1 in this case.

In my case, we had /var/lib/mysql, so I created mysql2:
mkdir /var/lib/mysql2
chown -R mysql.mysql /var/lib/mysql2

I also want some separate log file destinations:
mkdir /var/log/mysql2
chown -R mysql.mysql /var/log/mysql2

Then we get into the nitty gritty! Changing the my configuration file. That’s located on Debian systems in /etc/my.cnf and then I copied it to /etc/my2.cnf

In the new second config file, you need to specify a new port. Let’s say the default port for the first instance is 3306, I now should add “port=3307” into my2.cnf

The same rule will apply for changing the ‘datadir’ to the new directory in the config file, as well as the socket, you need to create a new connection socket for the secondary instance.

Also inside of the config file, you should specify where your new log-error file is as well as a new pid-file (if these are already in my.cnf, that is).

When you’re happy, just flick through the my2.cnf and double check that you’ve configured everything properly. Mine looked fine, so onto the next step.

run:
mysql_install_db –user=mysql –datadir=/var/lib/mysql2/

You’ll see some output on the command line, it should hopefully have no ERRORs in there. I also like to just double check my other instance of MySQL is still up and working OK using ps.

Now, let’s start our instance, shall we?
mysqld_safe –defaults-file=/etc/my2.cnf &

Verify it works:
mysql -S /var/lib/mysql2/mysql2.sock