Freesound 2.0 – A.k.a. Nightingale

So, we have finally succombed to the pressure of web 2.0, we have been tagged, we tagged, we dugg, opened a flickr.com account and … created a blog for Freesound. Or should I say Nightingale? Well, let’s only use Nightingale codename for what I devised it for: the new and improved Freesound two point ooooh.

Xavier and I have talked multiple times about rewriting parts of Freesound, but when Google gave us a research grant, it was a godsend! He asked me if I would like to come “back” (not being able to move back to spain – staying in the same place – but in a digital kind of way anyway) to work for MTG on a freelance basis to work on Freesound. I thought about it, but being the good father to Freesound I try to be, how could I say no??

Mind you, that version of Freesound is still very far away! But we already know some of the things we really want in it. Here are the things I had said in the forum with some additional information added where needed.

• All the code will be public and released under the gnu public license (v2 or 3, not sure yet). The code will be hosted on MTG’s code repository. You could ask, why not google code or sourceforge… well, I guess we could have done that but have opted for an environment we can easily control. As Jordi (our fearless sysadmin) said: what if you want to change to GIT (another “version control” tool for development) in a few months. Before anyone asks, the open source release will not include the “search similar” features of the website: this is a closed source technology, developed by MTG and sold by BMAT, MTG’s first spinoff.
• Obviously we will add all the highly wanted features that we have been talking about for so long. This includes the support of 3 different licenses (public domain, attribution and attribution non-commercial), embedding of the sounds in other pages (add your freesound files to … your blog), adding images to your sounds etc etc. We will do another pass for “officially wanted features”, I’ll also go through the forum to collect everything in there.
• The forum will be replaced by a hand-coded forum. This is good and bad: bad because it means more work, good because we don’t need to depend on phpBB anymore. PhpBB is great if you run a forum, but we run a website, and the forum is taking too prominent a place right now, it governs the whole site (the login system in freesound is based on phphBB). PhpBB being as omnipresent as it is is also highly plagued by spam. Another problem is that if you add hacks to phpBB (like… antispam) you need to re-implement your changes or do a very careful merge of the code. You could say, yes, but if you make a forum you will not have as many features and not as thoroughly tested software as phpBB. True, but … we don’t need all those features of phpBB and a smaller code base is easier to bugfix and maintain!! 🙂
• All development will be done using the python framework called django. Aah, this is a difficult one. I’ve had many discussions with many people saying plone or typo3 or insert-favorite-cms-here is better. Frameworks versus content-management-systems is not my favorite subject, but I will say this: if you try to do things with a CMS that it wasn’t really made for you will end up fighting the CMS. And fighting that CMS will take you just as long as writing something from zero, in a framework with plenty of features (think RoR, django, turbogears, …). I just happen to know django, like python and I’ve got some experience with it.
• Freesound has grown out of its initial parameters, so we will try to make the website more easily scalable. Freesound is taking up 2TB of bandwidth per month, and our database server is… suffering. We need some caching and we need it now. Greg has been working on some changes in the current freesound code to have more distributed bandwidth consumption over various squid servers all over the web. The plan is to roll out that changeset tomorrow (March 4th)! Dobroide has set up a test squid server in Sevilla. We will be blasting his computer into high gear in a few days. Poor CPU 🙂
• A new and fresh skin/design most likely created in collaboration with oneDotOnly but this is still in discussion. Whatever the skin is, it will need to be cleaner than the mess we have right now. When my friend Javi did the initial skin for Freesound, who would know that we would abuse it so badly in the future. 3 years of adding features with a skin that was designed to last 1 year. Auch, sorry Javi!
• Sound playback with soundmanager2. About 2 and a half years ago I found a bug in Mozilla’s flash plugin. I submitted the bug. 2.5 years have passed. Time to move on. Using this javascript + 1 flash instance per page plugin solves the problem (and in a very nice way!). It allows us to add as many sounds per page as we want without the flash player stopping playback. Have you ever noticed all sound goes nuts when you have 2 Freesound tabs open? That will be fixed.
• Search capability with Apache Solr. Me and lately Gerard as well have been experimenting with Solr. Jordi wants to damn me to hell and back for wanting to use a java based server (“you’re going to make me install Tomcat? Grrr!” – maybe he is not so fearless!), but this thing is just too nice. A highly scalable search engine that will take care of all the annoying bits for us. In splicemusic we used SQL triggers to update data in a big search-table (which was probably not the smartest way either), but writing sql triggers can sometimes be… annoying. With django we can make a simple priority queue for documents that need updating (“hey, this document changed, you need to look at it again, Mr. search engine”). I’ve been looking around for other solutions but none seem as powerful (while staying easy-to-use) as Solr. Especially the faceting seems to be well worth it! Faceting is what you get in some shops: you searched for camera we found digital cameras (150), analog cameras (2), tripods (15), etc. Imagine this for freesound: you searched for bass drum, we found samples with samplerate 44100 (15), 44800 (10), samples in packs Bram’s Bassdrum Pack (28), Jovica Heavy Hitters Pack (58), …

So, to conclude: a lot of plans, a “todo” from here to the moon and back, but… it will be well worth it!