Yes, … we know (spam).

Hello everyone,

Yes, we know. Freesound is under a MASSIVE spammer attack. We’re discussing internally how we will handle this situation. Just deleting the spam message isn’t enough as those &*%$%! spammers are creating spam quicker than we can delete them. A few of the options we are discussing:

  • More Captchas (funny words you have to type). All users who haven’t uploaded sounds yet and want to post on the forum will need to fill in the guess-the-squiggly-words thing.
  • First-post-moderation. The first post of every new user is moderated. If that post is passed, you’re now free to post. We have enough active forum members to make this work, probably.
  • Flood-protection. Right now you’re allowed to post as many messages in a single day as you like. We should limit this to max W per X minutes, max Y per hour, max Z per day. We could get these numbers from looking at the past for our most acive forum members.
  • Akismet. Akismet is a -nonperfect- system/service that analyses a text and tells you if it’s spam or ham. Non-perfect because sometimes it flags spam as ham and vice-versa. We have this already running in freesound but right now we’re not doing anything active with the results. I.e. if a post is marked as spam we just store “hey this post is probably spam” but don’t stop the user from posting more.
  • No posting more than X url’s in your first post. We used to have this in freesound “1″, we could revisit this idea.
  • No posts until you have downloaded at least one sound.
  • Some kind of flagging of spam posts + moderation.
  • [Post your realistic ideas in the comments here!]

We know that some of these are more work than others, and none of them are perfect solutions; for each of these options you can always come up with a “yes, but ….”. So, we will have to make a choice and implement some of them. This should also apply to sound comments, because I fear the day those $%&#$%# discover those.

Hang on people, this is going to take a while :(

- Bram

PS: On a more personal note: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaargh $&%^@#$$$$!

This entry was posted in spam. Bookmark the permalink.

24 Responses to Yes, … we know (spam).

  1. AlienXXX says:

    As with many things, I guess what will work best is a combination of various techniques.
    Using an automated tool to work out what is spam is dangerous, as Bram points out it is not perfect. On the other hand, manual checking of every post will not only result in a frustrating delay to users as it may be a considerable load on people moderating these posts (I am guessing here, I do not know the number of posts from new members on a daily/weekly basis).

    Would it work to have a combined method. For example:
    1) Automated tool checks post and classes it as spam
    2) Only if the user posts more than 3 suspect spam messages in a given period does the tool act to ‘block’ or hinder the user. – several options here: lock the account (meaning not allowed to post) so that a mod has to check and unlock it, automatically delete the user and his posts (dangerous!) or, my personal favourite: flag the account for moderation AND force the user to go through a captcha until account has been given the ‘all clear’ by a mod.

    I do not know how easy these things are to implement. Most of the time, it is only new users we have to worry about, so maybe the automated spam checker could be only applied to new users or users with less than X number of posts…

  2. Implement a must-do-2 feature with many different filters to choose from. This means that a commenter must do at least 2 or the following things (incomplete list)

    - Wait 1 week
    - Download more than 1 sound
    - Visited at least 10 pages between signup and comment
    - (other prerequisites here…)

    In addition, users should always have the following limitations
    - Prove themselves human with registration confirmation, clickable through email
    - No urls in first comment at all
    - Captcha’s on all comments
    - Can comment only on sounds they have downloaded. This is the App Store rule, and it works great.

    Freesound should block all posts auto-detected as spam automatically, and put a link in the rejection notice to contact you if the blockage was made in error. Spam bots won’t read the error and won’t click on the link.

  3. nemoDaedalus says:

    You could have a look at the stopforumspam.com API. That alone may also not be perfect and have it’s share of false positives/negatives. But together with above ideas, this should help a bit. If a post is flagged as spam by a few systems, for example, their IP is in the stopforumspam database, they have words that generally occur in spam in the message and there are already 3 posts by the same user in the last minute, then it is very likely spam.

    I’m a webmaster for a small site and have implemented a very basic spam prevention system in the contact form. If someone tries to send a message, it is checked for a few things; properly formed e-mail address, no spam-specific words in the body, hidden input fields with session id, etc. When a message is flagged as spam, the IP is blocked from sending any new message for a certain time. This at the very least prevents flooding. And when I look at the data, I see spam messages are occasionally attempted, but rarely sent. And if the spammers retry, I add their IP to the .htaccess file, to block them completely.

  4. Strangely_gnarled says:

    Updated suggestion that I made in the forum some time age:

    Put “report spam” button on post page.

    If reports from 2 (or more) users, who have been active members for more than 6 months (say) then quarantine “spam” poster. ie flag the user and his posts, hide the posts from the forum and block, restrict or “capcha” further posts from this user until a moderator has made judgement.

    Develop moderator/admin tool to delete all posts and the user with single stroke.

    Since there are usually a hundred plus users on line at any time, many of them long standing, I think spam would normally be caught within minutes of being posted, and the the spammer quarantined (and flagged for the mods) before any damage is done. I would certainly do my bit with enthusiasm as would most serious members. I tend to log on 2 or 3 times most days just to see what’s going on.

    The above could be fine tuned later. For instance a database of trusted “non moderators” could be developed so that a singe “report spam” from one of them would activate the quarantine immediately. The numbers/length of membership etc. criteria in the count reports filter can be fine tuned and made more sophisticated with experience.

    I think that using all long standing members as persistant policemen will ultimately prove simpler to impliment and be more effective and reliable than automatic spam detectors. Fine tune the filter rules on the spam reports and malicious or accidental quarantining can be eliminated, and there is NO extra inconvenience being security checked every time you try to post.

    (okay…. heart’s pounding, lets see if I can get this capcha right!!!!)

  5. bdejong says:

    Already some really good ideas we hadn’t come up with! Keep’m coming guys!

  6. afleetingspeck says:

    As AlienXXX mentioned, a combination (I like all) of these would work best.
    One other option is the feature to report a post as spam and if it is done so by a moderator, the post should be hidden altogether (I guess I am just repeating a user’s suggestion above).

  7. Timbre says:

    I suggest using the type of picatcha below rather than the conventional capatcha which I and others find difficult to read.

    https://lh4.googleusercontent.com/7q_lZMc7Bn9m4PEhfXjStAVq2O9wd4XQOSlrNcEZmIhLzBQqzaAjyzBGnkisTA5JwZorBnISW5PmdsXte6bOvTSffw9yEs4nOP8=w1600

  8. Suva says:

    I am using the stopforumspam together with Akismet on plenty of sites. Works wonderful. 99% of spammers get blocked by the stopforumspam and other blacklists, which also helps to keep the traffic under control. Akismet takes care of the rest, making sure there is reasonable amount of caught spam, so some admin can ocasionally check for false positives without having to spend entire day sorting through viagra ads.

  9. dobroide says:

    as a first, provisional urgency-measure I would not allow any new users to register within the next… say 48h. Then, don’t allow URLs in the first post (permanently or for period of time). Then… whatever
    The other day I saw an interesting approach, don’t know if it can be useful or not:
    http://www.urusoft.net/emailthis.php?7082=406&lang=1

    D

  10. qubodup says:

    At FreeGameDev Forums, we do two things:

    1. Before a new user’s first post is visible, it has to be authorized by a moderator. The moderator has the choice between accepting the post, rejecting it and giving reason and deleting the user with giving reason. New users see a piece of text while they have <1 posts at the top of the site so they know this.
    2. On our registration page, we let the human spammers know that everything gets moderated.

    It’s pretty effective and (2) is important.

  11. tekgnosis says:

    I agree with dobroide: an immediate stop-gap is required to stop the spammers from getting in. So I would stop all forms of registration for the next N hours.

    Once they stop registering, they need to be blocked from posting. This is where requiring a picatcha would be best. Captcha was hacked a couple years ago, and only provides a basic-level of security…going one-up might help in the short term, until a new solution such as first-post moderation can be implemented.

  12. Timbre says:

    qubodup says:

    1. Before a new user’s first post is visible, it has to be authorized by a moderator. The moderator has the choice between accepting the post

    If that system were in place currently the moderators would be flooded with spam: it would accumulate faster than they could delete it, with new Freesound accounts created automatically by spambots. Maybe the human botmaster would realize that nothing was appearing in the forum and give up attempting to spam on the Freesound Forum, but how long would that take ?

    [ PS hopefully below is a working link to the picatcha thing I mentioned above ...
    https://docs.google.com/open?id=0B4otMEAHEkEWNTIyNzBhN2MtYzMxNy00MzlkLWJlNmQtMjg4MzM4MmJmZGNh ]

  13. juskiddink says:

    Captchas using bold letters and numbers can be read by spam programmes
    http://security.goldsby.com/2009/07/11/x-rumer-5-0-spam-tool-pure-evil-busts-captcha-registration-etc/
    Wouldn’t using picatcha’s together with a “must download one sound before posting on forum” be enough?
    Using a time-lapse is restrictive,it shouldn’t take a week to prove you’re human.

  14. Kane says:

    Hi, I’m not a member here, yet, but I just had a peep after hearing about the site on tthe credits of a podcast. I run a forum of around 40,000 and in the past we’ve had some spam blitz’s and like ‘touch wood’, we have erradicated them totally now so thought I’d mention how in case it helps your situation.

    At my forum we run phpBB I’m not sure what forum software you’re using here or what Admin options you have access to but two things that have a big impact on spambots are stopping registrations from web-based email accounts, especially atm, gmail and yahoo, but the biggest success we had was making registrations ‘admin approved’, we did have the software set so that anyone could register and then once they receive an emailed link to click, can post away, now I have it set to admin activation it’s so much easier.

    I have it setup so that each morning I have a look through the new registrations and can with a bit of experience, spot a spambot quite easily now, I just select them all and do a mass delete, they never get to post on the forums at all. I go through every registration left and check to make sure IPs are legit and match the country that the email address refers to and that pretty much stops 100% of spambots, human spammers are pretty rare these days unless it’s someone with a grudge but they’re pretty easy to stop too by banning IPs/email addy’s and if you also implemement ISP email addresses only they’re stuffed too. :)

    All of the other methods will work to some extent and work together but stopping auto registration works the best.

    All the best

    Kane

    (aka STeALtH – Admin @ http://www.disqworld.net)

  15. afleetingspeck says:

    Just to quickly add, some sort of user validation (answering some sort of a puzzle or such) at the user registration level (to make sure the sign-up is a human) might help, too.
    It’s probable that the spammer is just one person and so he is using different proxies, but limiting sign up based on the IP (although I am not sure if it has already been suggested) might be another help.
    I second justkiddink, though, one week is a huge restriction.

  16. Jesse says:

    How about something interesting and themed?

    Sound recognition system for new user signups!
    we all got mics

  17. Steve Lawson says:

    Within hours of signing up on FreeSound I received a number of SPAM emails (all from “jane35″) that said that she had fallen in love with my profile and wanted me to send her my email address so we could correspond. The funny thing is, if she really did check my profile, all she saw was whitespace, because I haven’t written anything, yet!

    So, the fact that the email came so quickly after I signed up, suggest three possibilities:
    1. Every member gets a notification when someone joins.
    2. “She” checks the “People” section of the site regularly for new members.
    3. There’s some kind of worm running on the server that hosts the FreeSound.org website (a worm that sends her notifications.)

    So, if #1 isn’t true, then I would seriously check for worms (and then try to answer the question of how that worm got there in the first place).

    And, then, consider adding a “This member sent me SPAM” button next to the “From” field on each message in the Messages section. Each button press would be a vote (one per voting member) to get this potential SPAMer kicked off the site (or perhaps, give them a warning, first).

  18. Why not implement “Achievement points” ?

    Which could give achievement points:

    - Succesfull and verified uploads
    - regular answers to questions, regular posts
    - Helping users
    - Making alternative editions of sounds
    - Topic starters with positive reactions

    A self-regulated system which involves some interaction of the users. Combine this with IP & cooking tracking, with a bit of moderation (mark as abuse/spam) and you might have a neat system, to track those that got the best intention with the freesound community..

  19. me says:

    this is very inetersting, since it does not burden the user to write some funny letters (I have it running on my wp for some time now);
    http://wordpress.org/extend/plugins/wp-captcha-free/

  20. Don Engel says:

    Isn’t there a way to instantly disconnect an incoming spam, by treating all the errors a spam makes as a sudden disconnect from your website..? It’ll upset those who make errors while connecting.. so what…

    Or is there a way to recognize incoming spam, and direct it to a decoy website..?
    Let the goofy spam do its thing in the “toilet website”, whilst recording its IP, which gets blacklisted from your real website, and from all your projects and links…

  21. jericho says:

    Hey man, captcha’s not worth anything. The bot’s have .api’s with programs that solve captcha.

    Pretty simple solution though, set your settings to manual question, write something like… Are you a robot? Answer: no

    Pretty simple, just need to change the question every few months.

    If that doesn’t work, then you need to change the text on the registration page when it says:

    I agree to the rules: and gives you the option to check the

  22. jericho says:

    sorry, didn’t finish my sentence.
    Give you the option to check the ‘agree’ box.

    Essentially, the bots are programmed to look for that text, so if you CHANGE letters around or re-write that sentence, then most bots will be stopped before they even sign up.

    I’m an seo specialist, so I know how it all works.

  23. Uranoxyd says:

    Because the site turns around samples, how about a “sound-Captcha”. Choose a random sound from the database and the user is asked to specify the tag assignet to it, it will be 2-3 right and some wrong tags available. Or build a small stand-alone database, and give only 1 right and 4 wrong choices. e.g. it plays the sound of a doorbell and asked “What did you hear?” A doorbell, a crying baby, a dying swan or a pig in space :–)

  24. da bishop says:

    #1: flagging button
    #2: puzzle game, preferably audio based.