Sunday, August 3, 2008

Search Engine and Artificial Intelligence!

Search Engines encompass a variety of sub domains or specialized fields of computer science. Starting from Information Retrieval (as they were in the primitive form) to being highly related to Artificially Intelligence (check out the few options mentioned below). Search engines are ideally to be self learning machines that can rightfully decipher the queries provided to them as Natural Languages, also understanding the intent and context of the searcher viz. differentiating between Jaguar - the car and Jaguar - the animal.

Anatomy of a Search Engine,
Generally indexing comprises of
  • Porting/Crawling (to for a data source)
  • Mapping, Cleanup
  • Indexing

And searching

  • Index reader
  • Query formation
  • Searcher (for selecting data)
  • Scoring mechanism (for ordering data)

Other ideas that are of use while designing a search engine:

  • Mapping
  • Clustering
  • Wisdom of crowd for
    • Relevance scoring
    • Spam filtering
    • Suggestion Engine
  • Natural Language Processing for
    • Query formation
    • Intent determination
    • One box searching - G****E like!

There's a lot more that could be of use, but lets start off with this for the moment!