Rss Feed Tweeter button Facebook button Technorati button Reddit button Linkedin button Webonews button Delicious button Digg button Flickr button Stumbleupon button Newsvine button

A Waage Blog

Ruby, Rails, Life

Archive for March, 2010

How to Set up DB server on Amazon EC2 with data stored on EBS drive formatted with XFS

with 17 comments

Working with Amazon Web Services (AWS) such as EC2, S3, and EBS takes some time to get familiar with all the tools, acronyms, and especially paradigm of thinking about servers. Once you get into it, you quickly find much old (outdated) documentation, and you can easily find yourself spending hours searching the web for blog posts to help in your quest!

After much research, I decided to create a Mysql database server on a ‘instance-store’ small ec2 server, with all the precious Mysql data stored on a EBS volume attached to my server. This way, I could easily create snapshot backups of my databases for FAST backups. Here’s the procedure I decided on. It involves symlinking Mysql config files and data directories onto the EBS volume.

Another trick I used because I needed to migrate about 20 GiB’s of data to get started, was that I initially set up an “X-tra large” instance, with 10 GiB’s RAM to handle the data import. After the data was migrated and imported to my database, I simply terminated my X-Large instance and spun up a small instance connected to the same EBS volume! All the databases were preserved nicely and I did not have to waste money paying for an X-Large instance anymore. This exemplifies the value of thinking in the “cloud” mindset – where you can spin up and down servers in a matter of seconds! Hope this article helps someone else out there!

Great Reference: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1663

# Create small instance (I used the AMI for i.386 CentOS 5.4)

# Create EBS volume and attach it to the EC2 (as /dev/sdh in my example)

# Install mysql-server

yum install mysql-server

# Install xfsprogs so you can format EBS drive with XFS

yum install xfsprogs
modprobe xfs
mkfs.xfs /dev/sdh

# Add to your /etc/fstab so it mounts on boot

echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir -m 000 /vol
sudo mount /vol

# Create directories that will store persistant Mysql data

mkdir /vol/etc /vol/lib /vol/log

# Move the mysql config and data files to your newly mounted drive

mv /etc/my.cnf  /vol/etc/
mv /var/lib/mysql /vol/lib/
mv /var/log/mysqld.log /vol/log/

# Symlink the mysql directories so that mysql uses the mounted EBS drive

ln -s /vol/lib/mysql /var/lib/mysql
ln -s /vol/etc/my.cnf /etc/my.cnf
ln -s /vol/log/mysqld.log /var/log/mysqld.log

Now restart your Mysql server and check your logs if you have any errors!

service mysqld start

# If you have errors, watch the logs
tail -f /var/log/mysqld.log

Good luck !

Written by Andrew Waage

March 19th, 2010 at 3:56 pm