Galaxy Zoo Upgrade: Better Galaxies, Better Science
Since I joined the team in 2018, citizen scientists like you have given us over 2 million classifications for 50,000 galaxies. We rely on these classifications for our research: from spiral arm winding, to merging galaxies, to star formation – and that’s just in the last month!
We want to get as much science as possible out of every single click. Your time is valuable and we have an almost unlimited pile of galaxies to classify. To do this, we’ve spent the past year designing a system to prioritise which galaxies you see on the site – which you can choose to access via the ‘Enhanced’ workflow.
This workflow depends on a new automated galaxy classifier using machine learning – an AI, if you like. Our AI is good at classifying boring, easy galaxies very fast. You are a much better classifier, able to make sense of the most difficult galaxies and even make new discoveries like Voorwerpen, but unfortunately need to eat and sleep and so on. Our idea is to have you and the AI work together.
The AI can guess which challenging galaxies, if classified by you, would best help it to learn. Each morning, we upload around 100 of these extra-helpful galaxies. The next day, we collect the classifications and use them to teach our AI. Thanks to your classifications, our AI should improve over time. We also upload thousands of random galaxies and show each to 3 humans, to check our AI is working and to keep an eye out for anything exciting.
With this approach, we combine human skill with AI speed to classify far more galaxies and do better science. For each new survey:
- 40 humans classify the most challenging and helpful galaxies
- Each galaxy is seen by 3 humans
- The AI learns to predict well on all the simple galaxies not yet classified
What does this mean in practice? Those choosing the ‘Enhanced’ workflow will see somewhat fewer simple galaxies (like the ones on the right), and somewhat more galaxies which are diverse, interesting and unusual (like the ones on the left). You will still see both interesting and simple galaxies, and still see every galaxy if you make enough classifications.
With our new system, you’ll see somewhat more galaxies like the ones on the left, and somewhat fewer like the ones on the right.
We would love for you to join in with our upgrade, because it helps us do more science. But if you like Galaxy Zoo just the way it is, no problem – we’ve made a copy (the ‘Classic’ workflow) that still shows random galaxies, just as we always have. If you’d like to know more, check out this post for more detail or read our paper. Separately, we’re also experimenting with sending short messages – check out this post to learn more.
Myself and the Galaxy Zoo team are really excited to see what you’ll discover. Let’s get started.
Scaling Galaxy Zoo with Bayesian Neural Networks
This is a technical overview of our recent paper (Walmsley 2019) aimed at astronomers. If you’d like an introduction to how machine learning improves Galaxy Zoo, check out this blog.
I’d love to be able to take every galaxy and say something about it’s morphology. The more galaxies we label, the more specific questions we can answer. When you want to know what fraction of low-mass barred spiral galaxies host AGN, suddenly it really matters that you have a lot of labelled galaxies to divide up.
But there’s a problem: humans don’t scale. Surveys keep getting bigger, but we will always have the same number of volunteers (applying order-of-magnitude astronomer math).
We’re struggling to keep pace now. When EUCLID (2022), LSST (2023) and WFIRST (2025ish) come online, we’ll start to look silly.
To keep up, Galaxy Zoo needs an automatic classifier. Other researchers have used responses that we’ve already collected from volunteers to train classifiers. The best performing of these are convolutional neural networks (CNNs) – a type of deep learning model tailored for image recognition. But CNNs have a drawback. They don’t easily handle uncertainty.
When learning, they implicitly assume that all labels are equally confident – which is definitely not the case for Galaxy Zoo (more in the section below). And when making (regression) predictions, they only give a ‘best guess’ answer with no error bars.
In our paper, we use Bayesian CNNs for morphology classification. Our Bayesian CNNs provide two key improvements:
- They account for varying uncertainty when learning from volunteer responses
- They predict full posteriors over the morphology of each galaxy
Using our Bayesian CNN, we can learn from noisy labels and make reliable predictions (with error bars) for hundreds of millions of galaxies.
How Bayesian Convolutional Neural Networks Work
There’s two key steps to creating Bayesian CNNs.
1. Predict the parameters of a probability distribution, not the label itself
Training neural networks is much like any other fitting problem: you tweak the model to match the observations. If all the labels are equally uncertain, you can just minimise the difference between your predictions and the observed values. But for Galaxy Zoo, many labels are more confident than others. If I observe that, for some galaxy, 30% of volunteers say “barred”, my confidence in that 30% massively depends on how many people replied – was it 4 or 40?
Instead, we predict the probability that a typical volunteer will say “Bar”, and minimise how surprised we should be given the total number of volunteers who replied. This way, our model understands that errors on galaxies where many volunteers replied are worse than errors on galaxies where few volunteers replied – letting it learn from every galaxy.
2. Use Dropout to Pretend to Train Many Networks
Our model now makes probabilistic predictions. But what if we had trained a different model? It would make slightly different probabilistic predictions. We need to marginalise over the possible models we might have trained. To do this, we use dropout. Dropout turns off many random neurons in our model, permuting our network into a new one each time we make predictions.
Below, you can see our Bayesian CNN in action. Each row is a galaxy (shown to the left). In the central column, our CNN makes a single probabilistic prediction (the probability that a typical volunteer would say “Bar”). We can interpret that as a posterior for the probability that k of N volunteers would say “Bar” – shown in black. On the right, we marginalise over many CNN using dropout. Each CNN posterior (grey) is different, but we can marginalise over them to get the posterior over many CNN (green) – our Bayesian prediction.
Read more about it in the paper.
Active Learning
Modern surveys will image hundreds of millions of galaxies – more than we can show to volunteers. Given that, which galaxies should we classify with volunteers, and which by our Bayesian CNN?
Ideally we would only show volunteers the images that the model would find most informative. The model should be able to ask – hey, these galaxies would be really helpful to learn from– can you label them for me please? Then the humans would label them and the model would retrain. This is active learning.
In our experiments, applying active learning reduces the number of galaxies needed to reach a given performance level by up to 35-60% (See the paper).
We can use our posteriors to work out which galaxies are most informative. Remember that we use dropout to approximate training many models (see above). We show in the paper that informative galaxies are galaxies where those models confidently disagree.
This is only possible because we think about labels probabilistically and approximate training many models.
What galaxies are informative? Exactly the galaxies you would intuitively expect.
- The model strongly prefers diverse featured galaxies over ellipticals
- For identifying bars, the model prefers galaxies which are better resolved (lower redshift)
This selection is completely automatic. Indeed, I didn’t realise the lower redshift preference until I looked at the images!
I’m excited to see what science can be done as we move from morphology catalogs of hundreds of thousands of galaxies to hundreds of millions. If you’d like to know more or you have any questions, get in touch in the comments or on Twitter (@mike_w_ai, @chrislintott, @yaringal).
Cheers,
Mike
One Million for Zooniverse – and One for Galaxy Zoo!
Galaxy Zoo started in 2007 because astronomers had 1,000,000 galaxies that needed to be sorted, classified, and examined. After the incredible response from the public, the zookeepers realized that this kind of problem wasn’t limited to galaxies, nor even just to astronomy, and the Zooniverse was born.
Now, seven actual years, close to 30 projects, more than 60 publications, and hundreds of years’ worth of human effort later, the Zooniverse has just registered its 1,000,000th volunteer. Given that Galaxy Zoo was the project that led to the creation of the Zooniverse, it seems fitting that its millionth citizen scientist joined to classify galaxies! That volunteer (whose identity we won’t divulge unless s/he gives us permission) joins over 400,000 others who have classified galaxies near and far. That number is 40% of the Zooniverse’s overall total — meaning that, while Galaxy Zoo has a large and vibrant community of volunteers and scientists, most people who join Zooniverse start off contributing to a different project. Many of them try other projects after their first: over on the Zooniverse blog Rob described the additions we’ve made to the Zooniverse Home area so that everyone who brought us to a million can see their own contribution “fingerprint” on the Zooniverse. Here’s what mine currently looks like:
The blue at the left is Galaxy Zoo; the dark orange is Snapshot Serengeti. #addict
Our millionth volunteer gets a cheesy prize (but hopefully useful: a Zooniverse tote bag and mug), and while we’d like to give that same prize to the 999,999 who came before him/her and to everyone who contributes to Galaxy Zoo and all Zooniverse projects, perhaps it’s more fitting that we say to everyone what’s really on our mind right now:
Galactic-Scale Gratitude. You all are awesome.
Evolutionary Paths In Galaxy Morphology: A Galaxy Zoo Conference
This week much of the team has been in Sydney, Australia, for the Evolutionary Paths In Galaxy Morphology conference. It’s a meeting centered largely around Galaxy Zoo, but it’s more generally about galaxy evolution, and how Galaxy Zoo fits into our overall (ever unfolding) picture of galaxy evolution.
There’s a lot to that legacy already, and it’s still being written.
The first talk of the conference was a public talk by Chris, fitting for a project that would not have been possible without public participation. Chris also gave a science talk later in the conference, summarizing many of the different results from Galaxy Zoo (and with a focus on presenting the results of team members who couldn’t be at the meeting). For me, Karen’s talk describing secular galaxy evolution and detailing the various recent results that have led us to believe “slow” evolution is very important was a highlight of Tuesday, and the audience questions seemed to express a wish that she could have gone on for longer to tie even more of it together. When the scientists at a conference want you to keep going after your 30 minutes are up, you know you’ve given a good talk.
In fact, all of the talks from team members were very well received, and over the course of the week so far we’ve seen how our results compare to and complement those of others, some using Galaxy Zoo data, some not. We’ve had a number of interesting talks describing the sometimes surprising ways the motions of stars and gas in galaxies compare with the visual morphologies. Where (and how bright) the stars and dust are in a galaxy doesn’t always give clues to the shape of the stars’ orbits, nor the extent and configuration of the gas that often makes up a large fraction of a galaxy’s mass.
Karen explains her simple and clear diagram showing different galaxy evolutionary processes.
This goes the other way, too: knowing the velocities of stars and gas in a galaxy doesn’t necessarily tell you what kinds of stars they are, how they got there, or what they’re doing right now. I suspect a combination of this kinematic information with the image information (at visual and other wavelengths) will in the future be a more often used and more powerful diagnostic tool for galaxies than either alone.
Overall, the meeting was definitely a success, and throughout the meeting we tried to keep a record of things so that others could keep up with the conference even if they weren’t able to attend. There was a lot of active tweeting about the conference, for example, and Karen and I took turns recording the tweets so that we’d have a record of each day of the Twitter discussion. Here those are, courtesy of Storify:
Also, remember at our last hangout when we said we’d have a hangout from Sydney? That proved a bit difficult, not just because of the packed meeting schedule but also because of bandwidth issues: overburdened conference and hotel wifi connections just aren’t really up to the task of streaming a hangout. We eventually found a place, but then it turned out there was construction going on next door, so instead of the sunny patio we had intended to run the hangout from we ended up in an upstairs bedroom to get as far away from the noise as possible. Ah, well. You can see our detailed discussion of how the meeting went below, including random contributions from the jackhammer next door (but only for the first few minutes):
(click here for the podcast version)
And now we’ll all return (eventually) to our respective institutions to reflect on the meeting, start work on whatever new ideas the conference discussions, talks and posters started brewing, and continue the work we had set aside for the past week. None of this is really as easy as it sounds; the best meetings are often the most exhausting, so it takes some time to recover. I asked our fearless leader Chris if he had a pithy statement to sum up his feeling of exhilarated post-meeting fatigue, and he took my keyboard and offered the following:
gt ;////cry;gvlbhul,kubmc ;dptfvglyknjuy,pt vgybhjnomk
I’m sure that, if any tears were shed, they were tears of joy. This is a great project and it’s only getting better.
Left to Right: Tom, Kevin, Bob, Amit, Ed, Chris S, Bill, Kyle, Chris L, Ivy, Brooke, Karen, Julie
Quench Boost: A How-To-Guide, Part 4
Now that we’ve been initiated into the cool waters of Tools (Part 1), we’ve compared our *own* galaxies to the rest of the post-quenched sample (Part 2), and we’ve put your classifications to use, looking for what makes post-quench galaxies special compared to the rest of the riff-raff (Part 3), we’re ready for Part 4 of the Quench ‘How-To-Guide’.
This segment is inspired by a post on Quench Talk in response to Part 3 of this guide. One of our esteemed zoo-ite mods noted:
There are more Quench Sample mergers (505) than Control mergers (245)… It seems to suggest mergers have a role to play in quenching star formation as well.
Whoa! That’s a statistically significant difference and will be a really cool result if it holds up under further investigation!
I’ve been thinking about this potential result in the context of the Kaviraj article, summarized by Michael Zevin at http://postquench.blogspot.com/. The articles finds evidence that massive post-quenched galaxies appear to require different quenching mechanisms than lower-mass post-quenched galaxies. I wondered — can our data speak to their result?
Let’s find out!
Step 1: Copy this Dashboard to your Quench Tools environment, as you did in Part 3 of this guide.
- This starter Dashboard provides a series of tables that have filtered the Control sample data into sources showing merger signatures and those that do not, as well as sources in low, mid, and high mass bins.
- Mass, in this case, refers to the total stellar mass of each galaxy. You can see what limits I set for each mass bin by looking at the filter statements under the ‘Prompt’ in each Table.
Step 2: Compare the mass histogram for the Control galaxies with merger signatures with the mass histogram for the total sample of Control galaxies.
- Click ‘Tools’ and choose ‘Histogram’ in the pop-up options.
- Choose ‘Control’ as the ‘Data Source’.
- Choose ‘log_mass’ as the x-axis, and limit the range from 6 to 12.
- Repeat the above, but choose ‘Control – Merging’ as the ‘Data Source’.
The result will look similar to the figure below. Can you tell by eye if there’s a trend with mass in terms of the fraction of Control galaxies with merger signatures?
It’s subtle to see it in this visualization. Instead, let’s look at the fractions themselves.
Step 3: Letting the numbers guide us… Is there a higher fraction of Control galaxies with merger signatures at the low-mass end? At the high-mass end? Neither?
To answer this question, we need to know, for each mass bin, the fraction of Control galaxies that show merger signatures. I.e.,
Luckily, Tools can give us this information.
- Click on the ‘Control – Low Mass’ Table and scroll to its lower right.
- You’ll see the words ‘1527 Total Items’.
- There are 1527 Control galaxies in the low mass bin.
- Similarly, if you look in the lower right of the ‘Control – Merging – Low Mass’ Table, you’ll see that there are 131 galaxies in this category.
- This means that the merger fraction for the low mass bin is 131/1527 or 8.6%.
- Find the fraction for the middle and high mass bins.
Does the fraction increase or decrease with mass?
Step 4: Repeat the above steps but for the post-quenched galaxy sample.
You may want to open a new Dashboard to keep your window from getting too cluttered.
Step 5: How do the results compare for our post-quenched galaxies versus our Control galaxies? How can we best visualize these results?
- In thinking about the answer to this question, you might want to make a plot of mass (on the x-axis) versus merger fraction (on the y-axis) for the Control galaxies.
- On that same graph, you’d also show the results for the post-quenched galaxies.
- To determine what mass value to use, consider taking the median mass value for each mass bin.
- Determine this by clicking on ‘Tools’, choosing ‘Statistics’ in the pop-up options, selecting ‘Control – Low Mass’ as your ‘Data Source’, and selecting ‘Log Mass’ as the ‘Field’.
- This ‘Statistics’ Tool gives you the mean, median, mode, and other values.
- You could plot the results with pen on paper, use Google spreadsheets, or whatever plotting software you prefer. Unfortunately Tools, at this point, doesn’t provide this functionality.
It’d be awesome if you posted an image of your results here or at Quench Talk. We can then compare results, identify the best way to visualize this for the article, and build on what we’ve found.
You might also consider repeating the above but testing for the effect of choosing different, wider, or narrower mass bins. Does that change the results? It’d be really useful to know if it does.
101: The Great Debate
This is the first in a new series of blog posts under the title of ‘Galaxies 101’. These posts aim to explore the history and basics of the science of galaxies. I’ll be covering some of people who helped us understand these ‘Island Universes’ as well as some of the basics that would be taught during a first year undergraduate galaxies course at university.
It is fortunate that these posts are beginning in the week of the 90th anniversary of The Great Debate which occurred on April 26th, 1920. The Great Debate – or the Shapely-Curtis Debate – took place at the Smithsonian Museum of Natural History between two eminent astronomers, Harlow Shapley and Heber Curtis. Shapely was arguing that the ‘spiral nebulae’, that were observed at the time, were within our own Galaxy – and that our Galaxy was the Universe. He also argued that the Sun was not at its centre. Conversely, Curtis argued that the Sun was at the centre of our Galaxy but that the ‘spiral nebulae’ were not inside our Galaxy at all. He suggested instead that the Universe was much larger than our Galaxy and that these nebulae were in fact other, ‘island’ universes.
Below is a drawing of the ‘spiral nebula’ M51. This is an observation by Lord Rosse, drawn in 1845 using the 72-inch Birr Telescope at Armagh Observatory in the UK.
With 90 years of hindsight we can now say that Shapely and Curtis were both right and wrong. The Sun is not at the centre of the Galaxy and the Galaxy is only one of hundreds of billions of galaxies in the Universe. But how was the argument resolved? The answer, in part, comes from a very famous name in astronomy: Hubble.
Less a decade after the Great Debate took place, Edwin Hubble used the largest telescope in the world – the 100-inch Hooker Telescope on Mount Wilson – to observe Cepheid variable stars in the Andromeda Nebula/Galaxy. Cepheid variables are a type of pulsating stars whose pulsation periods are precisely proportional to their luminosities. This makes Cepheid variable stars a ‘standard candle’ – an object where the brightness is a known quantity. If you can observe the apparent brightness of a standard candle, then you can determine its distance by a simple inverse square law. Since Cepheid variable stars have pulse rates proportional to their luminosity, if you can measure the pulse rate of a Cepheid variable anywhere in the Universe, then you can determine how far away it is. This is what Edwin Hubble did in 1925 and he calculated the distance to Andromeda as 1.5 million light years.
At the time, Shapely thought that our Galaxy was around 300,000 light years across and Curtis believed it was around 30,000 light years. Hubble’s measurement placed Andromeda well outside our galaxy and showed that Curtis was correct in thinking that the ‘spiral nebulae’ could indeed be other galaxies. Today we think the Milky Way is about 100,000 light years across and that Andromeda is 2.5 million light years away.
The discoveries of the 1920s started a whole new adventure for astronomy. The Universe had gotten a lot bigger and was about to expand much, much more. It is important to remember that Shapely, although wrong about the nature of the nebulae, did correctly assert that the Sun was not at the centre of the Galaxy. This is the kind of Copernican shift that makes people think about things differently and it is important to realise that the issues discussed during the Great Debate were complex. For our benefit though, the Great Debate is a starting point for exploring the relatively new study of galaxies. Humanity’s view of the Universe, and our place within it, has changed an awful lot since 1920. The study of galaxies has had a lot to do with that.
If you want to read more about the Great Debate, and what else helped to resolve the arguments I can recommend this excellent NASA site. You can also read the published text of the debate online.
[Andromeda image credit: Robert Gendler]
Galaxy Zoo 