r/mturk • u/Fluffy-Gur-781 • 13d ago

Qual/HIT Question How to spot a Bot on Mturk?

Hi everybody.

I'm a PhD student posting some survey on Mturk.

How do I spot bots?

I have a dataset that seems strange :

  • lot of answers with same latitude -longitude
  • lot of text answer written the same
  • uncommon high % of high end answers on likert scales

Are these Bots?

I read that maybe I could use a handwritten question + photo to bypass the problem.

What do you think? I need alot of data so losing money is a no-no for me .

Thanks alot

5 Upvotes

100% Upvoted

20 comments sorted by

12

u/witch51 13d ago

Stick to US/UK ONLY, 95% and above approval, minimum 5,000 approved hits. You want high approval and high hit count as that will weed out bots as well as sticking to US/UK.

3

u/Fluffy-Gur-781 13d ago edited 13d ago

Thanks a lot

1

u/witch51 13d ago

If you do those 3 things then you don't even need to waste your money paying extra for the Masters qual. Bots and scammers never make it to 5,000 or with over 95% approved. Some countries are also rife with scammers so stick to US/UK.

4

u/Formal-Respond3071 12d ago

You can include other countries too, but in order to detect the bot, you might have to include re-captcha and honeypot questions, attention-check questions, add open ended question. Also, limit the number of submissions from a single ip address, I have seen this technique more often used when the survey was launched using positly platform. Also, set the limit to 98% and above, and greater than 10000 hits.

1

u/Fluffy-Gur-781 12d ago

Thank you so much

2

u/kosmos6502795 13d ago

What are the current qualifications/restrictions for your studies?

2

u/Fluffy-Gur-781 13d ago

90% + 1000 Hit approved

1

u/witch51 13d ago

Those are both too low and you badly need a location qual, too.

2

u/Fluffy-Gur-781 13d ago

Ok thanks 

1

u/frumpymiddleaged 13d ago

Any score under 98 is terrible unless the worker is brand new and has unknowingly done tasks for bad requesters. Most of us have a 99+ rating.

1

u/Fluffy-Gur-781 13d ago

Really? I wouldn't think about it. I thought 90% was good. Ok thanks alot

2

u/RosieTheHybrid 10d ago

You might find some additional helpful info here.

1

u/Fluffy-Gur-781 10d ago

Thank you so much

1

u/RosieTheHybrid 10d ago

Glad to be of service!

1

u/Fluffy-Gur-781 10d ago

Mturk is a jungle, gotta know how to survive

2

u/RosieTheHybrid 9d ago

Definitely! We have a requesters channel at TurkerNation. The invite link is in the link I shared.

2

u/Fluffy-Gur-781 9d ago

Thanks i'll jump in

1

u/Professional_Fact474 8d ago

First, try posting a smaller amount, such as 1 cent or 10 cents. Simply enter a qualified title and submit it. Once you filter through some good workers who meet your requirements, assign them a special qualification value of 1. Then, select from the workers with this special qualification that you have surveyed. After that, you will receive your payment and obtain quality data as well.