Speak your weight
Following Anze’s post we’ve had a number of requests from people to see their weighting. Indeed we did think about this for a bit, but alas I reckon it’s a bad idea to publish users weights for a number of reasons – but mainly because it is very unclear what the weights really say about a user. For example, a user might be really upset to see that they have a low-weighting, indicating that they often disagree with the majority of people (if we use a democratic weighting scheme). They might then try to alter their classifying behaviour so to ‘improve’ their weighting. This would be disastrous, because perhaps they are actually excellent at classifying, and much more meticulous than the majority of other users. In this case a low weighting would be good! The majority ain’t always right, right?!
If providing this kind of feedback was to have any effect then this would be bad – because it would mean our data becomes correlated in complicated ways that we can’t trace i.e. the results from one week can affect the next week. Ultimately everyone will want to up their weighting, and I can imagine a horrible situation where everyone just clicks elliptical all the time! Because this would give everyone great weightings – but completely ruin the project!
Therefore, you see that knowing your weight can only have a bad effect (if it isn’t going to have an effect then there’s no need to know 😉 ). Ideally we don’t want anything to have an effect on you – we want everything to be as unbiased and open and transparent as possible when it comes to analysing the results.
Plus the weights change all the time, as more classifications are made and it is computationally very intensive to compute them. Further, there’s an infinite number of ways of working out the weightings! We could see how well you agree with each other for just the bright galaxies, or how well you agree with an ‘expert’. But the point of all this is that we do not know the true morphology of these galaxies – and therefore we cannot give you a true weight (ie. how well you are classifying).
I hope that helps to explain our situation a bit! I appreciate that it must be a bit frustrating not to get more feedback on your classifications. Perhaps when this phase of the project is wrapped up then we can feedback more… and ultimately all this data is probably going to be made public! Cheers, Kate
Galaxy Zoo 
I already know my weighting will be low. I should blame it all on the mouse responding to a click only when I’ve already moved to the wrong button, but sometimes I’m -a–m-o-r-o-n- accident prone …
Don’t worry, oino, it happens to me a lot, too! 😀
i for one would like to see another tutorial – one that helps us identify some of the less obvious galaxies.
for example, is there a good way to tell an E7 elliptical from an edge-on spiral? i think i’ve seen hundreds of borderline cases.
how about E0 vs S0?
a new tutorial would save me daily embarassment in the forum 🙂
The truth is that with limited resolution it is difficult to tell a high axial-ratio elliptical from an edge-on spiral, and especially an S0 from an elliptical. Experts will have nearly as much difficulty as you in these cases. With the quality typical of Galaxy Zoo images the best in the field would probably misclassify a face-on S0 roughly half of the time. Actually, the fact that different Galaxy Zoo classifiers will disagree over such galaxies is very useful in helping us to understand the uncertainties in the classifications.
If you like the idea of doing more detailed classifications, just wait for Galaxy Zoo 2, coming soon!
All these hints about GZ2 are very tantalising – when… where… wha…?
Wlindoe (Wendy) – embarrassment on the forum is not permitted. Come on, I’m a moderator and I make mistakes all the time. Making mistakes is part of learning. If we don’t learn, the world won’t have its supply of astronomers. And if the world doesn’t have its supply of astronomers, we won’t get things like Galaxy Zoo . . .
I’m thinking about writing a little tutorial myself on various things that would qualify as “star/don’t know” and why. Such as nebulae, comets, irregulars, etc. But it depends how long GZ1 is still operating. And it would be on the forum, not Galaxy Zoo. What do people think?
Hi Kate!
It is ok not to show us our “weightings” or if we are right or wrong. I just do my very best, even not to know how good or bad I am. It is a statistical project, so it is best to minimize all possible influences.
Bye
Weighting aside, there are some quality-control improvements that could be made.
1. Currently, the “Trial” requires 8/15 correct to proceed to classification. I think that’s kind of low.
2. The Trial tells your score, but not which you labelled incorrectly, which is probably important.
3. I see no reason why a trial (but call it something else) couldn’t have an easy/med/hard section for those trying to measure and improve their skills.
I will also mention the Stardust-at-home project throws in known “ringers” from time to time, just to see if you’re paying attention or are just running up your click-count. I personally like that, and I suspect this manifests itself as more accurate responses.
OK, so low-weight users may be either worse or better classifiers than the herd – the point is they are outliers. Since I assume it would be good to move the herd in a better direction (and the herd would like to go there) some study of the better outliers might be worthwhile for GZ2.
I certainly agree that the tutorial could do with a lot more on classifying “difficult” cases. Rings should be there (I’m sure they weren’t even in the FAQ when I started), and something about what to do with long things (elongate ellipse or edge-on spiral? if there’s a lump in the middle of the sausage does that make it more likely a spiral?).
And please, pretty please, give feedback on my induction trial score – if only because people who care enough to do this at all will want to know.
I expect GZ2 with incorporate mirrored, monochrome, and colour-shifted/scrambled images from the start, so you don’t have to retrofit a bias study as in GZ1.
Ever thought of getting some known experts to run with the herd of naive users by way of an extra statistical control?
OK, I think it’s my turn to respond.
>1. Currently, the “Trial” requires 8/15 correct to proceed to classification. I >think that’s kind of low.
It seems to be working fine; it seems a long time ago now but we came to that decision via some trials.
>2. The Trial tells your score, but not which you labelled incorrectly, which is >probably important.
Yes, you’re right we should have done this. From memory the problem was that the trial always incorporates the same galaxies and we didn’t want to give away the ‘right’ answers. We’d do this differently now.
>3. I see no reason why a trial (but call it something else) couldn’t have an >easy/med/hard section for those trying to measure and improve their skills.
That’s something we’ll look at for zoo 2. We were trying to keep things as simple as possible for zoo 1.
>Since I assume it would be good to move the herd in a better direction (and >the herd would like to go there) some study of the better outliers might be >worthwhile for GZ2.
We’re definitely looking at this – you’re right that being an outlier doesn’t necessarily make you wrong.
>Rings should be there
We thought they’d be very rare – one of the surprises is that they’re not.
>I expect GZ2 will incorporate mirrored, monochrome, and colour-shifted/scrambled images from the start, so you don’t have to retrofit a bias study as in GZ1.
In GZ2 we will be fully incorporating any required bias checks we think we might need. We didn’t include them to begin with in GZ1 as we thought we’d struggle to get enough users to classify the main sample, let alone do this.
>Ever thought of getting some known experts to run with the herd of naive users by way of an extra statistical control?
Yes, we do that – that’s how we know the classifications are good, for starters.
This is totally off topic but i love the smiley face!!!!!!!
My Friend & i completey luv the smiley face heheheheh
xx
Oh yeah luv the stars and galxies
Oops – super-smiley now back to normal size!!! I should be more careful when messing with the CSS 😉