IMHO this isn't a good task for the Turks. If it's reasonable that a Turk might click 'Looks good to me!' most of the time, you'll gather a hoard of them that just click/script that same response 100% of the time and not look at the question.
Of course you can include decoy questions to try and detect this behavior, but I just don't see the whole MTurk review process being terribly reliable overall. I just don't think the worker quality is that great, and they will do whatever possible to just churn through HITs (because they get paid zero for them).
Of course you can include decoy questions to try and detect this behavior, but I just don't see the whole MTurk review process being terribly reliable overall. I just don't think the worker quality is that great, and they will do whatever possible to just churn through HITs (because they get paid zero for them).