Solr is Searching On Lucene w/Replication
Specifically, Solr is a scalable, ready-to-deploy enterprise search engine that’s optimized to search large volumes of text-centric data and return results sorted by relevance.
- Scalable—Solr scales by distributing work (indexing and query processing) to multiple servers in a cluster.
- Ready to deploy—Solr is open source, is easy to install and configure, and provides a preconfigured example to help you get started.
- Optimized for search—Solr is fast and can execute complex queries in subsecond speed, often only tens of milliseconds.
- Large volumes of documents—Solr is designed to deal with indexes containing many millions of documents.
- Text-centric—Solr is optimized for searching natural-language text, like emails, web pages, resumes, PDF documents, and social messages such as tweets or blogs.
- Results sorted by relevance—Solr returns documents in ranked order based on how relevant each document is to the user’s query.
Search engines like Solr are optimized to handle data exhibiting four main characteristics:
- Text-centric
- Read-dominant
- Document-oriented
- Flexible schema
You also want to consider which fields in your documents must be stored in Solr and which should be stored in another system, such as a database. A search engine isn’t the place to store data unless it’s useful for search or displaying results.
Building a web-scale inverted index
It might surprise you that search engines like Google also use an inverted index for searching the web. In fact, the need to build a web-scale inverted index led to the invention of MapReduce.
MapReduce is a programming model that distributes large-scale data-processing operations across a cluster of commodity servers by formulating an algorithm into two phases: map and reduce. With its roots in functional programming, MapReduce was adapted by Google for building its massive inverted index to power web search.
Using MapReduce, the map phase produces a unique term and document ID where the term occurs. In the reduce phase, terms are sorted so that all term/docID pairs are sent to the same reducer process for each unique term. The reducer sums up all term frequencies for each term to generate the inverted index.
------------------------------------------------------------------------------------------------------------------------------------
Diagram of the main components of Solr 4
