Wikia, Inc. and Intellisophic, Inc. announced this week that Intellisophic, which publishes subject-specific taxonomy, thesaurus and vocabulary products for the search, categorization and text-mining industry, will be making its software available via open source in conjunction with the Search Wikia project.
Categorization is the process of organizing information based on a common set of characteristics or concepts, which in terms of Internet searching helps to increase accuracy. By understanding the vocabularies, terminology and relationships of concepts in different categories of content such as finance, gaming, or news, the quality of search results for the related keywords can be significantly increased.
This semantic step by the Search Wikia movement follows their acquisition of the Grub crawler last week which is a shared web crawler which was the ‘original’ shared search compilation project masterminded by LookSmart’s Michael Grubb, who visualized shared search well ahead of its time back in 2000.
The open sourcing of Intellisophic technology and Grub will lend a new rejuvinated breath in open search, which has been spearheaded by the Nutch effort for years. Wikia is taking an interesting leading role in the spread of open source search, based upon these values :
1. Transparency – Openness in how the systems and algorithms operate, both in the form of open source licenses and open content + APIs.
2. Community – Everyone is able to contribute in some way (as individuals or entire organizations), strong social and community focus.
3. Quality – Significantly improve the relevancy and accuracy of search results and the searching experience.
4. Privacy – Must be protected, do not store or transmit any identifying data.
It will be even more interesting to see which existing search companies step up to help fund or build a partnership with the Search Wikia project, as some of the budding start-ups (Facebook) or established companies (Yahoo) may have major interest in the future of transparent and user influenced & governed search. Om Malik suggested Yahoo! on his video cast the other day as the company has “little to lose and everything to gain.” Yahoo would not be a bad idea for a company to get behind and possibly enable open source search within their network, especially with the socially driven element behind Yahoo.
Let’s also remember it was Yahoo who backed Google, and thrust them into the spotlight when Yahoo contracted Google to serve their web results. If Search Wikia could produce a better and more accepted alternative to Google (there are better search engines out there, as better is usually subjective and dependent on the needs of the end user), would it be in Yahoo’s interest to back the project, promote it and perhaps benefit from it?
