But it got me thinking, there is another think I like to keep an eye on on our servers… log files. Oh, and I was looking for good excuse to play around with Sinatra. So, “with a one, and-a-two, and-a-three…” we have Sinatra-Tailer.

You can read all about it on the github page, but in short it simply performs a tail and displays the last X lines of the log file.

tail -n /path/to/my/log/file.log

Features

1. refreshes the list every X seconds, set by the user
2. only one config.yml file to edit
3. supports file globs, so you can grab a whole list of file with one line
4. specify the number of lines to show
5. completely unobtrusive js, because I’m cool?

Requirements

Sinatra framework (tested on 0.9.1.1, earlier may work)

I whipped this up pretty quickly so I’m sure there are a few bugs. There is some testing for a few unit tests, but nothing functional.

One word of warning, if you want to put this on a production server my recommendation is to put it on a separate port (like 9876) and for heaven’s sake, at a minimum use http basic authentication. From the sinatra readme:

 use Rack::Auth::Basic do |username, password|
    username == 'admin' && password == 'secret'
  end

Enjoy!

The Problem

Here in my development palace I have a couple of laptops that I use for development and an Ubuntu (hardy) server as a Rails/Rack sandbox that I run with Passenger. While I have always been able to serve up either a Rails or Sinatra project, I typically have 3 or 4 going at the same time. But I had to access them with urls like: http://192.168.0.2/project1 and http://192.168.0.2/project2.

So, what’s the problem, eh?

Well, for starters, you can’t do anything concerning subdomains. That’s a show stopper for many apps.

Second, it really messes with your environment variables like ["PATH_INFO"]. So if you do any routing and you expect the base url to be ‘/’, it won’t be. The root url will be ‘/projectx’. Bummer.

The Solution

To remedy the issues with non-tld virtual hosts (also called sub URIs?) we need to add the domains to the laptops and server, then add the virtualhosts on the server for apache. This means that I can now goto http://project1.ubuntu.remote url to get to my rails/rack app.

For reference, I am using the following:

• Laptops: Mac OS X, 10.5.7 and 10.4.11
• Server: Ubuntu Desktop 8.04LTS, Apache/2.2.8 (Ubuntu) mod_fastcgi/2.4.6 PHP/5.2.4-2ubuntu5.5 with Suhosin-Patch Phusion_Passenger/2.0.6

On Each Laptop

These instructions are for Mac OS X, but should be identical for your favorite flavor of *nix. We are going to add some friendly names to our ubuntu server using it’s ip address (192.168.0.2). In this example we will be able to access the root web documents using http://ubuntu.remote, and one of our Sinatra projects using http://project1.ubuntu.remote.

Important! Don’t use a name ending in ‘.local’. While it did work for me, I read about a few instances where some systems will not resolve domain names ending in .local outside the local machine. Also, don’t change anything else in your /etc/hosts file except to add the lines at the end.

mate /etc/hosts

# add the following line to the END of your /etc/hosts file and save
# note: there is a <space> character between the ip address and each host name
192.168.0.2 ubuntu.remote project1.ubuntu.remote

On The Server: Hosts File

Nothing will work until we tell the server machine that it can accept incoming traffic from our new friendly names. Similar to on the mac, we need to edit our /etc/hosts file but we will add a slightly different line. In this case we are telling the server that incoming traffic from ‘ubuntu.remote’ and ‘gems.ubuntu.remote’ are just names for the localhost (127.0.0.1).

sudo pico /etc/hosts

# add the following line to the end of our /etc/hosts file and save
# you can added after your existing 127.0.0.1 line if you wish
127.0.0.1 ubuntu.remote project1.ubuntu.remote

On The Server: VirtualHosts

As a final step we will need to add or change the virtual host definitions for your rails/rack project. Depending on your Apache setup or Linux distro, the default enabled site may be slightly different and you may have to adapt your virtual host file to your default setup.

First, in my /etc/apache2/apache.conf file, I have the following line near the end of the file. This was to eliminate a warning every time apache was started/restarted. It may affect your virtual host default, but shouldn’t, so add it if you get errors.

#added near end of /etc/apache2/apache.conf
ServerName localhost

Next we need to check our default virtual host setup to see how the NameVirtualHost is defined. We will duplicate that in each new virtual host that we add. The parts we are looking is the NameVirtualHost and VirtualHost near the top. They could be ‘NameVirtualHost *’ and ‘<VirtualHost *>’ or ‘NameVirtualHost *:80′ and ‘<VirtualHost *:80>’. See the pattern? We need to use the same definition in our new virtual files.

Let’s check our default virtual host setup:

cd /etc/apache2/sites-enabled
ls # your are looking for the name of your default site file

  total 0

  lrwxrwxrwx 1 root root 36 2009-02-27 18:49 000-default -> /etc/apache2/sites-available/default

cat /etc/apache2/sites-enabled/000-default

  NameVirtualHost *
  <VirtualHost *>
    ServerAdmin webmaster@localhost
  # snipped several lines #
  </VirtualHost>

On my system the NameVirtualHost is ‘*’, so that is what we need to use for the new virtual host. In this case, I am going to add a virtual host file for a rack based project. It’s a Sinatra app, but that doesn’t matter as this will work for any rack based project, even a rails/rack project (since rails 2.2.2?).

Also, I like using the Apache2 system to enable and disable sites. It’s clean, easy to use, and if any apache admin comes along they will know exactly what is going on.

It’s also work mentioning that you should disable or delete any existing virtual host files that you were using previously.

cd /etc/apache2/sites-available
sudo pico gems

# enter this as your virtual host setup
<VirtualHost *>
  ServerName project1.ubuntu.remote

  DocumentRoot "/path/to/your/application/project/public"

  RackEnv production
  <directory "/path/to/your/application/project/public">
    Order allow,deny
    Allow from all
  </directory>
</VirtualHost>

# save the file and reload apache

sudo /etc/init.d/apache2 reload

The Fun Part

Assuming you saw no errors when you reloaded apache, you should be able to go to your new urls see both the root files and your rack application.

http://ubuntu.remote

http://project1.ubuntu.remote

Dyn-O-Mite!

Bonus Points!

For extra credit, let’s say you have admin access to your router, and you are one of those gotta-have-control-of-everything dudes (or dudettes) so you installed a 3rd party firmware. You’re in luck! You can probably add some custom entries in your DNS and you don’t have to edit the /etc/hosts files on your laptop.

I installed Tomato firmware on my Linksys wireless router. Tomato uses Dnsmasq with allows you to set internal host names.

Maybe I’ll cover that in another post.

• A production server, not a development server, with only what is necessary
• A virtual machine that runs under VMware Player
• Ubuntu Server, 8.04 is the current, referred to as Hardy Heron
• Apache2 with Phusion Passenger to handle Rails applications
• Ruby, Rubygems, Ruby on Rails, a full Rails stack
• MySQL Server and SQLite, with phpMyAdmin to admin MySQL server
• All native drivers for Rails (MySQL and SQLite)
• OpenSSH server to access the server
• Postfix to send (only) emails

What you need to start:

1. VMware Player installed. Download it from VMware directly and follow their instructions (aptitude has older version 1.x)
2. Download the Ubuntu 8.04 Server ISO file for your machine. For torrent users, download the current i386 torrent file.
3. Download my VMX and VMDK files (vmplayer config and empty virtual disk).

Step 1: Setup Directory and Config File

Create a new directory somwhere (I called mine ‘Ubuntu840Server’) and copy the Ubuntu ISO file and the VMX and VMDK (don’t forget to unzip) files into it. You will need to do some quick editing to the Ubuntu840Server.vmx file, so fire up your favorite editor (in Ubuntu Desktop, right-click and Open with “Text Editor”). The only sections you may need to edit are:

• ide1:1.filename, make sure it matches the name of your ISO file, something like “ubuntu-8.04.1-serverr-i386.iso”
• fileSearchPath, change to the path of the directory you created

Step 2: Run VMWare Player

Run VMware Player and open the Ubuntu840Server.vmx. If all the config settings are correct and the ISO files is in the directory, you should boot into Ubuntu Installer.

Step 3: Install Ubuntu Server

I won’t go into details here because Ubuntu has step-by-step instructions on how to install. But the short story is: accept the defaults.

There is one selection that will make installation easier, select LAMP Server when asked for the Software selection. LAMP stands for Linux/Apache/MySQL/PHP. It will install all those packages and configure them automatically for you.

Note: even though I selected a LAMP server, for some reason it did not install Apache/Mysql/PHP. If this happens to you, just do the following after the server re-boots:

sudo apt-get install apache mysql-server php5 libapache2-mod-php5 php5-mysql


Step 4: Get the Latest Updates

If everything went well, you should reboot into Ubuntu Server and see the login: prompt. Go ahead and login with the username and password you supplied during the installation. Make sure you have the latest updates for Ubuntu by running (this may take a while):

sudo apt-get update
sudo apt-get upgrade


Step 5: Install OpenSSH

I like to administer my VMs from the terminal via SSH. Ubuntu Server does not come preconfigured with an SSH server, so lets install one:

sudo apt-get install openssh-client
sudo apt-get install openssh-server


You don’t really need the openssh-client, but I like to have it in case I ever need to ssh from the server to somewhere else. I can’t recommend enough that you secure your SSH server. If you server is exposed to the internet, it will be attacked, and one of the favorite vulnerabilities is SSH. Read this article on how to Secure an OpenSSH Server!

Step 6: Install phpMyAdmin

Since we are using MySQL, having a graphic administration program is nice and it’s hard to beat phpMyAdmin. I installed it from aptitude, but it did not set the proper aliases for apache2, so there are a few extra steps:

sudo apt-get install phpmyadmin

# close and save pico, then enter this to enable the phpmyadmin site
sudo a2ensite phpmyadmin
sudo /etc/init.d/apache2 restart


Step 7: Install Ruby

Next we need to install ruby. We’ll use the aptitude pre-built binaries to save time and make it easier to update in the future, but you may wish to install it from source. We also install the sqlite3 database because it is the default used by Rails 2.X and many test packages, but you can omit if you want.

sudo aptitude install ruby1.8-dev ruby1.8 ri1.8 rdoc1.8 irb1.8 libreadline-ruby1.8 libruby1.8 libopenssl-ruby sqlite3 libsqlite3-ruby1.8 build-essential mysql-client libmysqlclient15-dev postfix


Note that we installed the ‘build-essential’ package. This package is necessary for you to install any other software from source (like native msql drivers and phusion passenger). We also install the postfix package, you can just accept the defaults.

Next you will need to set some symlinks so that the libraries will be located when needed.

sudo ln -s /usr/bin/ruby1.8 /usr/bin/ruby
sudo ln -s /usr/bin/ri1.8 /usr/bin/ri
sudo ln -s /usr/bin/rdoc1.8 /usr/bin/rdoc
sudo ln -s /usr/bin/irb1.8 /usr/bin/irb


Now, when you should see your ruby package when you check the version from the terminal.

karl@ubuntuserver:~$ ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [i486-linux]


Step 8: Install RubyGems

RubyGems is a package manager like aptitude that makes is trivial to add/update/delete resources that ruby (and Rails) will need. It’s also how you will install and update Rails. We are going to install RubyGems from source as the aptitude package is quite out-of-date. RubyGems is a self-updating application, and when installed from aptitude, aptitude will complain and you may accidentally downgrade RubyGems.

# create a sources directory in your home
mkdir ~/sources
cd ~/sources

#update rubygems
sudo gem update --system
sudo gem update


Step 9: Install Ruby on Rails

All this work, and we are finally here. Too bad installing Rails is so easy. Just this one line.

sudo gem install rails
and a couple more gems you will probably need…
# install the native (faster) mysql driver
sudo gem install mysql

# if you want native (faster) sqlite3 drivers, for production
sudo apt-get install libsqlite3-dev
sudo gem install sqlite3-ruby

You can install any more gems you may need at this time with ’sudo gem install package-name‘, but if you are using rails 2.1 or greater, you can now add rubygem requirements to your environment.rb file and use rake to automatically installed gem dependencies.

Step 10: Install Phusion Passenger

There are many ways to host a Rails application, but at this time, Phusion Passenger gets the nod for the easiest to deploy and our experience has been rock-solid in reliability. These install steps are from the Passenger website installation instructions:

sudo gem install passenger
sudo apt-get install apache2-prefork-dev
sudo passenger-install-apache2-module
# and follow the instructions

# restart apache webserver
sudo /etc/init.d/apache2 restart


You will also be given a sample virtual host file to setup your Rails application. At this point you can copy an existing Rails app over to the new server, or follow along for a quick setup of a new Rails app.

Step 11: Create a Rails Application (optional)

We are going to create a very quick Rails application to test that our installation is working.

cd ~/
rails tester
script/generate scaffold Blog name:string content:text
rake db:migrate RAILS_ENV=production

# activate this site
sudo a2ensite tester
sudo /etc/init.d/apache2 restart


Now you can visit your site by going to http://address_of_server/ and you should see your rails application!

Problem: Our Cub Scout den leader sends email reminders to all the parent about upcoming events. I decided to setup a gCal (google calendar) will all the events so that other parents could just check the calendar as they wish, or subscribe to the ical file to integrate with their own calendar. But several of the parents don’t have calendars (shame, shame) and everyone still liked the idea of receiving email reminders. So I decided to write a ruby script to email reminders automatically two days before the event.

Solution: Ruby is a great language to handle this. Especially since there are gems to handle many of the functions we need. Some of the gems we will be using are as follows:

• open-uri: to get the ical file via http
• net/smtp: to send the email, you could use ActionMailer as well
• erb: email body is in a template.html.erb file for easy editing, erb processes the file
• yaml: a list of sent notifications are logged in a yaml file
• icalendar: the superb ical manipulation gem from Jeff Rose

All you need to do is setup a cron job to run this script. I chose to run the script at 8pm every evening.

I’m going to break this script apart by methods for easier commenting. You can download the complete script file and templates to follow along. In the single script file is logging and some formatting methods that don’t really need explanation here.

The script all assembled with some nice logging: notification_script.rb
A sample email template file: mail_template.html.erb
A sample recipient groupt list: group_list.txt

Setup and Requires

First, you are going to need the icalendar gem (documentation), which you can install with:

sudo gem install icalendar

For the beginning of our script, we’ll need a few other libraries to be required, and set a constant or two.

require 'rubygems'
require 'open-uri'
require 'icalendar'
require 'date'
require 'net/smtp'
require 'erb'
require 'yaml'

ICAL_URL = 'http://www.google.com/calendar/ical...path_to_your_ical_file'
FROM_EMAIL = 'your_email_address@domain.com'
MAIL_TEMPLATE_FILE = 'mail_template.html.erb'
SENT_NOTIFICATIONS_FILE = 'sent_notifications.yml'
GROUP_LIST_FILE = 'group_list.txt'
DAYS_BEFORE_TO_SEND = 2

All of these requires should already be installed in your standard ruby library. Following the requires are some constants you need to edit. The ICAL_URL constant is the url that you can look up from google calendars by going to your Calendar Settings, and selecting Calendar Details. You should see the ICAL (green) button that when you click, it shows you the url for the ical file of your calendar. Copy and paste this url into the ICAL_URL constant. You will also want to change the FROM_EMAIL to your email address, or the one you want notifications to be from.

The other constants are the names of the files that should be in the same directory as the script. You can rename them or change the paths as necessary.

The Main Event

I like to wrap my main events in a method called ‘init()’ and place it near the top of the script file so that it is quick and easy to find. You don’t have to do this, but supporting methods will have to come before your code.

def init
  begin
  myFile = open(ICAL_URL).read

  cals = Icalendar.parse(myFile)
  cal = cals.first

  cal.events.each{ |e|
    time_diff = e.dtstart - DateTime.now
    if time_diff < (DAYS_BEFORE_TO_SEND * 24 * 60 * 60) &amp;amp;&amp;amp; time_diff > 0

      if notified?(e.uid)
        #log '    notification previousy sent.'

      else
        send_to = group_list
        msg = ERB.new(File.read( MAIL_TEMPLATE_FILE )).result( binding )

        send_email('Den 4 Notifications', 'your_email@domain.com', send_to, e.summary, msg)

        add_notification_to_list(e.uid)
      end
    end
    }

  rescue
    #log the error
  end
end

Pretty easy, huh? Let start breaking this down in greater detail:

Step 1: Read the iCal File

Read the contents of the ical file at the ICAL_URL location. Using the open-uri library, the content of the ical file is read into a string object.

Step 2: Parse the iCal File with Icalendar

The Icalendar gem first requires that we parse the ical file, so that it can create the representative ’calendar’ and ‘event’ objects for reading. There is the potential of more than one calendar in the ics file. But in our case, we know there is only one, so we need to get that first ‘calendar’ object.

Step 3: Loop Through the Calendar Events

We need to look at every calendar ‘event’ object in the calendar so that we can look at the start date (e.dtstart) so see if it’s within the DAYS_BEFORE_TO_SEND.

Step 4: Previous Notification Sent? Ready Email Template

The ‘notified?’ method will tell us if this event has received previous notification. The e.uid attribute is a unique identifier associated with every event (an event id, if you will). We check that e.uid against a list of saved uid’s.

Then we can read the MAIL_TEMPLATE_FILE and use erb to process the file, substituting the various object attributes (variables). This makes it easy to change the email template without editing our script file.

Step 5: Send the Email Message

A Couple of things happen here that we’ll discuss in the ’send_email’ method, but suffice it to say that this sends the email to all the addresses in the GROUP_LIST_FILE.

Step 6: Add Notification to List

We save a list of the event uids (e.uid) that have been notified so that we don’t send a notification again.

Pretty spiffy, huh?

Now lets show some of the supporting methods and discuss.

Method: notified?

This method reads the SENT_NOTIFICATIONS_FILE yaml file to see if the passed ‘uid’ exists in the list of events with previous notification. The loop looks at each item in the ‘y_content’ array to see if the ‘uid’ is a match. If there is a match, return true (I like the explicitness of using return here). You can see how the yaml file is structured in the ‘add_notification_to_list’ method.

def notified?(uid)
  if File.exist?( SENT_NOTIFICATIONS_FILE )
    File.open( SENT_NOTIFICATIONS_FILE ) { |yf|
      y_content = YAML.load( yf )

      if y_content.is_a? Array
        y_content.each{ |e| return true if e[:uid] == uid }
      end
    }
  end

  return false  #not found
end

Method: group_list

The GROUP_LIST_FILE is a text file with the address of whom we want to send the notification emails to, separated by a linefeed. You can change this to suit your needs, like comma or pipe separation. It returns the list of email addresses in an array.

If the GROUP_LIST_FILE does not exist or there is an error reading the file, a default email address is returned so that at least someone gets notified. I would normally not do it his way and let the rescue raise an error and halt the script.

def group_list
  begin
    IO.read( GROUP_LIST_FILE ).split(/\n/)
  rescue
    return Array['default_email@domain.com']
  end
end

Method: send_email

Here we get to use the net/smtp gem to send the email. I chose this because it has fairly low overhead and it actually pretty easy to use. I’m not sure you can use it to send attachments, but in our case, we are just sending HTML emails.

A heredoc is used to create the string for the message header. I’m not such a fan of heredoc, so this is my one and only time I’ll use it.

Notice that the ‘Content-Type: text/html’ is set so that the receive recognizes this as an HTML email and not plain text. Use ‘Content-Type: text/plain’ if you’re sending a text only email.

You will have to set your SMTP server setting were you see ‘mail.my_server.com’. For more information on this, see the NET::SMTP documentation. Because there are many configurations for this, I didn’t place constants at the beginning of the script. The NET::SMTP docs pretty much cover it.

I would have liked to send all the messages using BCC, but alas I could find no documentation in NET::SMTP supporting BCC. In my case, everyone know everyone else, so it’s no big whoop.

def send_email(from_alias, from, to_array, subject, message)
	msg = <<END_OF_MESSAGE
From: #{from_alias} <#{from}>
To: #{to_array.collect{ |item| '<' + item + '>' }.join(',')}
Subject: [CUB SCOUTS]#{subject}
Content-Type: text/html

#{message}
END_OF_MESSAGE

	Net::SMTP.start( 'mail.my_server.com', 25 ) do |smtp|
		smtp.send_message msg, from, to_array
	end
end

Method: add_notification_to_list

Here we add an event uid to a yaml file and timestamp it. I’m sure there are a million ways to do this, but I just wanted to play around reading/writing yaml files.

There is one thing I don’t like about this though… the entire yaml file is read in, the new event is added to the end, and the resulting yaml replaces the entire existing file. I could not find a reliable way to append a yaml object to an existing yaml file. Well, sort of, but it creates an new array object hash for each append, instead of having all events in one array of hashes. Yeah, yeah, I’m being picky.

def add_notification_to_list(uid)
  if File.exists?( SENT_NOTIFICATIONS_FILE )
    y_content = YAML.load( File.open( SENT_NOTIFICATIONS_FILE ) )
  end

  y_content = [] unless y_content.is_a?( Array )

  File.open( SENT_NOTIFICATIONS_FILE, 'w+' ) { |yf|
    y_content.concat [ :date => Time.now, :uid => "#{uid}" ]
    YAML.dump( y_content, yf )
  }
end

Put it All Together…

The script all assembled with some nice logging: notification_script.rb
A sample email template file: mail_template.html.erb
A sample recipient groupt list: group_list.txt

All that’s left is to put all the files in a directory that has read/write access (for the log file and notification list file) and setup a cron job. Now you have automated notifications from your calendar.