amazon

EBS volumes are attached to EC2 instances that are running. So if you have a solr instance that starts on boot and needs to use that EBS volume you have a problem. How do you make solr wait for that EBS volume to be ready? How do you format it? How do you make all this idempotent.

I've seen celery drop tasks on the floor because the queuers clock is to far ahead of the workers. I've seen boto fail to update Route53 because a clock was wrong. Time is super important. Here is a little script to look at the output of ntpq -pn and try and explain whats going on.