Anatomy of a Search Engine,
Generally indexing comprises of
- Porting/Crawling (to for a data source)
- Mapping, Cleanup
- Indexing
And searching
- Index reader
- Query formation
- Searcher (for selecting data)
- Scoring mechanism (for ordering data)
Other ideas that are of use while designing a search engine:
- Mapping
- Clustering
- Wisdom of crowd for
- Relevance scoring
- Spam filtering
- Suggestion Engine
- Natural Language Processing for
- Query formation
- Intent determination
- One box searching - G****E like!
There's a lot more that could be of use, but lets start off with this for the moment!