Our New Infrastructure
As some of you may already know I’m relatively new to the Galaxy Zoo team having joined in January. One of the main reasons I was brought on board the team was to help plan and implement a more scalable web architecture for the future.
Galaxy Zoo 2 launched on Tuesday and was our first live test of our new approach. I’m pleased to say that apart from a small hiccup with our database server on day two we’ve managed to collect 3.7 million classifications in just over three days – well done everyone!
*Warning here begins gory technical details!* You can safely stop reading here unless you are a techy.
Galaxy Zoo is now hosted on Amazon Web Services. We are using a combination of S3 (their storage solution) for hosting the galaxy images, EC2 (their compute service) for the Galaxy Zoo website (and our super-secret API) and EBS (Elastic Block Store) for instant backups of our databases. The great thing about Amazon Web Services is that you pay for what you use (by the hour) and more significantly you can scale up to as many servers as you like to handle the load you are experiencing when busy.
Those of you who are particularly observant may have also noticed that the Zoo 2 site is now written in the web framework Ruby on Rails. Rails is great fit for us, it’s a modern web framework that follows a great design pattern (MVC) and encourages best practice development.
For those of you who have found the last two paragraphs interesting, I’m going to find a place where I can write about what we are up to on the technical side of things. Watch this space!
Cheers,
Arfon.
Galaxy Zoo 
Yes! very interested techy here!
Hi Arfon semi-tech here!
For some strange reason, I am still having problems logging into the site.
My password has been reset 5times approx – each time I have been able to do some classifing-great! I log out, try to login again, the same response cannot login, user name margo – reset password? Not sure why this is happening…Perhaps you can enlighten a semi-anorak?
PS. The team are doing a great job!
Not a techy but finding it interesting anyway!
Over a million classifications a day. We will have to slow down or we will run out of heavens at this rate!
Arfon, have a look at ZXTM from Zeus. Based in Cambridge but dont hold it against them:-) Loves Ruby, EC2 and peaks of traffic.
Cheers,
Nick
I’m having exactly the same login problem as Margo above, repeatedly having to renew password etc.
Hmm, maybe that’s why I’m getting some new error messages, Arfon. Right now I have one that starts out “getFrames() has failed:
The processing time seems a bit faster, but I’ve gotten lockups like this one several times in the past hour, & have to re-login, sometimes twice.
Any idea what’s happening?
Margo – I’ve just activated your account. Perhaps the email asking you to do it got lost? Anyway, that should solve the problem
Louis – I think you’ve misspelt your email address so I can’t get in touch with you.
Chris
Louis – found it! You should be fine now too.
Chris -many thanks. This am I was able to login successfully and start classifying. Hopefully, I won’t have to post anymore tales of woe?
Thanks a lot Arfon. We Zooites will surely feel more secured with capable technical support around.
JKHC.
Thanks for your information, I feel very happy to join in our jornny.
I would love to hear more about the technical details. Is this your first project using amazon web services? Are there other programmers on the team? How many other Rails projects have you done? What other technologies have you used in the past and how does this compare?
Hi Mac.
This is the team’s first use of AWS yes but as we move to bigger and bigger projects we’ll be using it a lot more. We are a pretty small team of developers at Galaxy Zoo (2,3 or 4 depending on how you count us!) but I’ve been working with Rails now for about 3 years. Previously I was working at the Wellcome Trust Sanger Institute in Cambridge (sanger.ac.uk) using Rails for tracking of DNA samples within the institute which was pretty good fun.
I’ve worked with Perl, PHP, C and even FORTRAN but I like Ruby the most and and I like Rails in particular for web projects as it encourages best practise software development techniques.
Cheers
Arfon
Don’t know if this is the right place for this but, while classifying yesterday I was struck by the highly pixilated images in the galaxy database. I noticed a tiny offset in red and blue images in a lot of the galaxies. That immediately brought to mind the 3D images in old comic books and (really) old movies. So, I grabbed my 5 year old grandson’s floppy, cheap, cardboard and plastic 3D glasses and took another look at some of the galaxies I was classifying. Very interesting! Now I don’t know if it was just my brain playing with what it saw but in several of the images I was able to pick out some interesting structure that I hadn’t seen without the glasses. Anyone else tried this? Is it even acceptable to use 3D glasses when classifying galaxies? If so, does anyone know where to find a good set of optical quality glass 3D glasses? Thanks?