Class IndexSearcher
- java.lang.Object
-
- org.apache.lucene.search.IndexSearcher
-
- Direct Known Subclasses:
QueryProfilerIndexSearcher
,SuggestIndexSearcher
public class IndexSearcher extends java.lang.Object
Implements search over a single IndexReader.Applications usually need only call the inherited
search(Query,int)
method. For performance reasons, if your index is unchanging, you should share a single IndexSearcher instance across multiple searches instead of creating a new one per-search. If your index has changed and you wish to see the changes reflected in searching, you should useDirectoryReader.openIfChanged(DirectoryReader)
to obtain a new reader and then create a new IndexSearcher from that. Also, for low-latency turnaround it's best to use a near-real-time reader (DirectoryReader.open(IndexWriter)
). Once you have a newIndexReader
, it's relatively cheap to create a new IndexSearcher from it.NOTE: The
search(org.apache.lucene.search.Query, int)
andsearchAfter(org.apache.lucene.search.ScoreDoc, org.apache.lucene.search.Query, int)
methods are configured to only count top hits accurately up to1,000
and may return alower bound
of the hit count if the hit count is greater than or equal to1,000
. On queries that match lots of documents, counting the number of hits may take much longer than computing the top hits so this trade-off allows to get some minimal information about the hit count without slowing down search too much. TheTopDocs.scoreDocs
array is always accurate however. If this behavior doesn't suit your needs, you should create collectors manually with eitherTopScoreDocCollector.create(int, int)
orTopFieldCollector.create(org.apache.lucene.search.Sort, int, int)
and callsearch(Query, Collector)
.NOTE:
instances are completely thread safe, meaning multiple threads can call any of its methods, concurrently. If your application requires external synchronization, you should not synchronize on theIndexSearcher
IndexSearcher
instance; use your own (non-Lucene) objects instead.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
IndexSearcher.LeafSlice
A class holding a subset of theIndexSearcher
s leaf contexts to be executed within a single thread.private static class
IndexSearcher.ShortcutHitCountCollector
static class
IndexSearcher.TooManyClauses
Thrown when an attempt is made to add more thanIndexSearcher.TooManyClauses.getMaxClauseCount()
clauses.static class
IndexSearcher.TooManyNestedClauses
Thrown when a client attempts to execute a Query that has more thanIndexSearcher.TooManyClauses.getMaxClauseCount()
total clauses cumulatively in all of it's children.
-
Field Summary
Fields Modifier and Type Field Description private static QueryCachingPolicy
DEFAULT_CACHING_POLICY
private static QueryCache
DEFAULT_QUERY_CACHE
private static Similarity
defaultSimilarity
private java.util.concurrent.Executor
executor
protected java.util.List<LeafReaderContext>
leafContexts
private IndexSearcher.LeafSlice[]
leafSlices
used with executor - each slice holds a set of leafs executed within one threadprivate static int
MAX_DOCS_PER_SLICE
Thresholds for index slice allocation logic.private static int
MAX_SEGMENTS_PER_SLICE
(package private) static int
maxClauseCount
private QueryCache
queryCache
private QueryCachingPolicy
queryCachingPolicy
(package private) IndexReader
reader
protected IndexReaderContext
readerContext
private Similarity
similarity
The Similarity implementation used by this searcher.private SliceExecutor
sliceExecutor
private static int
TOTAL_HITS_THRESHOLD
By default we count hits accurately up to 1000.
-
Constructor Summary
Constructors Constructor Description IndexSearcher(IndexReader r)
Creates a searcher searching the provided index.IndexSearcher(IndexReaderContext context)
Creates a searcher searching the provided top-levelIndexReaderContext
.IndexSearcher(IndexReaderContext context, java.util.concurrent.Executor executor)
Creates a searcher searching the provided top-levelIndexReaderContext
.IndexSearcher(IndexReaderContext context, java.util.concurrent.Executor executor, SliceExecutor sliceExecutor)
IndexSearcher(IndexReader r, java.util.concurrent.Executor executor)
Runs searches for each segment separately, using the provided Executor.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description CollectionStatistics
collectionStatistics(java.lang.String field)
ReturnsCollectionStatistics
for a field, ornull
if the field does not exist (has no indexed terms)int
count(Query query)
Count how many documents match the given query.Weight
createWeight(Query query, ScoreMode scoreMode, float boost)
Creates aWeight
for the given query, potentially adding caching if possible and configured.Document
doc(int docID)
Sugar for.getIndexReader().document(docID)
Document
doc(int docID, java.util.Set<java.lang.String> fieldsToLoad)
Sugar for.getIndexReader().document(docID, fieldsToLoad)
void
doc(int docID, StoredFieldVisitor fieldVisitor)
Sugar for.getIndexReader().document(docID, fieldVisitor)
Explanation
explain(Query query, int doc)
Returns an Explanation that describes howdoc
scored againstquery
.protected Explanation
explain(Weight weight, int doc)
Expert: low-level implementation method Returns an Explanation that describes howdoc
scored againstweight
.static QueryCache
getDefaultQueryCache()
Expert: Get the defaultQueryCache
ornull
if the cache is disabled.static QueryCachingPolicy
getDefaultQueryCachingPolicy()
Expert: Get the defaultQueryCachingPolicy
.static Similarity
getDefaultSimilarity()
Expert: returns a default Similarity instance.java.util.concurrent.Executor
getExecutor()
Returns this searchers executor ornull
if no executor was providedIndexReader
getIndexReader()
Return theIndexReader
this searches.java.util.List<LeafReaderContext>
getLeafContexts()
Expert: returns leaf contexts associated with this searcher.static int
getMaxClauseCount()
Return the maximum number of clauses permitted, 1024 by default.private static QueryVisitor
getNumClausesCheckVisitor()
Returns a QueryVisitor which recursively checks the total number of clauses that a query and its children cumulatively have and validates that the total number does not exceed the specified limit.QueryCache
getQueryCache()
Return the query cache of thisIndexSearcher
.QueryCachingPolicy
getQueryCachingPolicy()
Return the query cache of thisIndexSearcher
.Similarity
getSimilarity()
Expert: Get theSimilarity
to use to compute scores.private static SliceExecutor
getSliceExecutionControlPlane(java.util.concurrent.Executor executor)
Return the SliceExecutionControlPlane instance to be used for this IndexSearcher instanceIndexSearcher.LeafSlice[]
getSlices()
Returns the leaf slices used for concurrent searching, or null if noExecutor
was passed to the constructor.IndexReaderContext
getTopReaderContext()
Returns this searchers the top-levelIndexReaderContext
.Query
rewrite(Query original)
Expert: called to re-write queries into primitive queries.private Query
rewrite(Query original, boolean needsScores)
protected void
search(java.util.List<LeafReaderContext> leaves, Weight weight, Collector collector)
Lower-level search API.TopDocs
search(Query query, int n)
Finds the topn
hits forquery
.TopFieldDocs
search(Query query, int n, Sort sort)
Search implementation with arbitrary sorting.TopFieldDocs
search(Query query, int n, Sort sort, boolean doDocScores)
Search implementation with arbitrary sorting, plus control over whether hit scores and max score should be computed.void
search(Query query, Collector results)
Lower-level search API.<C extends Collector,T>
Tsearch(Query query, CollectorManager<C,T> collectorManager)
Lower-level search API.private <C extends Collector,T>
Tsearch(Weight weight, CollectorManager<C,T> collectorManager, C firstCollector)
private TopFieldDocs
searchAfter(FieldDoc after, Query query, int numHits, Sort sort, boolean doDocScores)
TopDocs
searchAfter(ScoreDoc after, Query query, int numHits)
Finds the topn
hits forquery
where all results are after a previous result (after
).TopDocs
searchAfter(ScoreDoc after, Query query, int n, Sort sort)
Finds the topn
hits forquery
where all results are after a previous result (after
).TopFieldDocs
searchAfter(ScoreDoc after, Query query, int numHits, Sort sort, boolean doDocScores)
Finds the topn
hits forquery
where all results are after a previous result (after
), allowing control over whether hit scores and max score should be computed.static void
setDefaultQueryCache(QueryCache defaultQueryCache)
Expert: set the defaultQueryCache
instance.static void
setDefaultQueryCachingPolicy(QueryCachingPolicy defaultQueryCachingPolicy)
Expert: set the defaultQueryCachingPolicy
instance.static void
setMaxClauseCount(int value)
Set the maximum number of clauses permitted per Query.void
setQueryCache(QueryCache queryCache)
Set theQueryCache
to use when scores are not needed.void
setQueryCachingPolicy(QueryCachingPolicy queryCachingPolicy)
Set theQueryCachingPolicy
to use for query caching.void
setSimilarity(Similarity similarity)
Expert: Set the Similarity implementation used by this IndexSearcher.protected IndexSearcher.LeafSlice[]
slices(java.util.List<LeafReaderContext> leaves)
Expert: Creates an array of leaf slices each holding a subset of the given leaves.static IndexSearcher.LeafSlice[]
slices(java.util.List<LeafReaderContext> leaves, int maxDocsPerSlice, int maxSegmentsPerSlice)
Static method to segregate LeafReaderContexts amongst multiple slicesTermStatistics
termStatistics(Term term, int docFreq, long totalTermFreq)
ReturnsTermStatistics
for a term.java.lang.String
toString()
-
-
-
Field Detail
-
maxClauseCount
static int maxClauseCount
-
DEFAULT_QUERY_CACHE
private static QueryCache DEFAULT_QUERY_CACHE
-
DEFAULT_CACHING_POLICY
private static QueryCachingPolicy DEFAULT_CACHING_POLICY
-
TOTAL_HITS_THRESHOLD
private static final int TOTAL_HITS_THRESHOLD
By default we count hits accurately up to 1000. This makes sure that we don't spend most time on computing hit counts- See Also:
- Constant Field Values
-
MAX_DOCS_PER_SLICE
private static final int MAX_DOCS_PER_SLICE
Thresholds for index slice allocation logic. To change the default, extendIndexSearcher
and use custom values- See Also:
- Constant Field Values
-
MAX_SEGMENTS_PER_SLICE
private static final int MAX_SEGMENTS_PER_SLICE
- See Also:
- Constant Field Values
-
reader
final IndexReader reader
-
readerContext
protected final IndexReaderContext readerContext
-
leafContexts
protected final java.util.List<LeafReaderContext> leafContexts
-
leafSlices
private final IndexSearcher.LeafSlice[] leafSlices
used with executor - each slice holds a set of leafs executed within one thread
-
executor
private final java.util.concurrent.Executor executor
-
sliceExecutor
private final SliceExecutor sliceExecutor
-
defaultSimilarity
private static final Similarity defaultSimilarity
-
queryCache
private QueryCache queryCache
-
queryCachingPolicy
private QueryCachingPolicy queryCachingPolicy
-
similarity
private Similarity similarity
The Similarity implementation used by this searcher.
-
-
Constructor Detail
-
IndexSearcher
public IndexSearcher(IndexReader r)
Creates a searcher searching the provided index.
-
IndexSearcher
public IndexSearcher(IndexReader r, java.util.concurrent.Executor executor)
Runs searches for each segment separately, using the provided Executor. NOTE: if you are usingNIOFSDirectory
, do not use the shutdownNow method of ExecutorService as this uses Thread.interrupt under-the-hood which can silently close file descriptors (see LUCENE-2239).
-
IndexSearcher
public IndexSearcher(IndexReaderContext context, java.util.concurrent.Executor executor)
Creates a searcher searching the provided top-levelIndexReaderContext
.Given a non-
null
Executor
this method runs searches for each segment separately, using the provided Executor. NOTE: if you are usingNIOFSDirectory
, do not use the shutdownNow method of ExecutorService as this uses Thread.interrupt under-the-hood which can silently close file descriptors (see LUCENE-2239).- See Also:
IndexReaderContext
,IndexReader.getContext()
-
IndexSearcher
IndexSearcher(IndexReaderContext context, java.util.concurrent.Executor executor, SliceExecutor sliceExecutor)
-
IndexSearcher
public IndexSearcher(IndexReaderContext context)
Creates a searcher searching the provided top-levelIndexReaderContext
.- See Also:
IndexReaderContext
,IndexReader.getContext()
-
-
Method Detail
-
getDefaultSimilarity
public static Similarity getDefaultSimilarity()
Expert: returns a default Similarity instance. In general, this method is only called to initialize searchers and writers. User code and query implementations should respectgetSimilarity()
.
-
getLeafContexts
public java.util.List<LeafReaderContext> getLeafContexts()
Expert: returns leaf contexts associated with this searcher. This is an internal method exposed for tests only.
-
getDefaultQueryCache
public static QueryCache getDefaultQueryCache()
Expert: Get the defaultQueryCache
ornull
if the cache is disabled.
-
setDefaultQueryCache
public static void setDefaultQueryCache(QueryCache defaultQueryCache)
Expert: set the defaultQueryCache
instance.
-
getDefaultQueryCachingPolicy
public static QueryCachingPolicy getDefaultQueryCachingPolicy()
Expert: Get the defaultQueryCachingPolicy
.
-
setDefaultQueryCachingPolicy
public static void setDefaultQueryCachingPolicy(QueryCachingPolicy defaultQueryCachingPolicy)
Expert: set the defaultQueryCachingPolicy
instance.
-
getMaxClauseCount
public static int getMaxClauseCount()
Return the maximum number of clauses permitted, 1024 by default. Attempts to add more than the permitted number of clauses causeIndexSearcher.TooManyClauses
to be thrown.- See Also:
setMaxClauseCount(int)
-
setMaxClauseCount
public static void setMaxClauseCount(int value)
Set the maximum number of clauses permitted per Query. Default value is 1024.
-
setQueryCache
public void setQueryCache(QueryCache queryCache)
Set theQueryCache
to use when scores are not needed. A value ofnull
indicates that query matches should never be cached. This method should be called before starting using thisIndexSearcher
.NOTE: When using a query cache, queries should not be modified after they have been passed to IndexSearcher.
- See Also:
QueryCache
-
getQueryCache
public QueryCache getQueryCache()
Return the query cache of thisIndexSearcher
. This will be either thedefault query cache
or the query cache that was last set throughsetQueryCache(QueryCache)
. A return value ofnull
indicates that caching is disabled.
-
setQueryCachingPolicy
public void setQueryCachingPolicy(QueryCachingPolicy queryCachingPolicy)
Set theQueryCachingPolicy
to use for query caching. This method should be called before starting using thisIndexSearcher
.- See Also:
QueryCachingPolicy
-
getQueryCachingPolicy
public QueryCachingPolicy getQueryCachingPolicy()
Return the query cache of thisIndexSearcher
. This will be either thedefault policy
or the policy that was last set throughsetQueryCachingPolicy(QueryCachingPolicy)
.
-
slices
protected IndexSearcher.LeafSlice[] slices(java.util.List<LeafReaderContext> leaves)
Expert: Creates an array of leaf slices each holding a subset of the given leaves. EachIndexSearcher.LeafSlice
is executed in a single thread. By default, segments with more than MAX_DOCS_PER_SLICE will get their own thread
-
slices
public static IndexSearcher.LeafSlice[] slices(java.util.List<LeafReaderContext> leaves, int maxDocsPerSlice, int maxSegmentsPerSlice)
Static method to segregate LeafReaderContexts amongst multiple slices
-
getIndexReader
public IndexReader getIndexReader()
Return theIndexReader
this searches.
-
doc
public Document doc(int docID) throws java.io.IOException
Sugar for.getIndexReader().document(docID)
- Throws:
java.io.IOException
- See Also:
IndexReader.document(int)
-
doc
public void doc(int docID, StoredFieldVisitor fieldVisitor) throws java.io.IOException
Sugar for.getIndexReader().document(docID, fieldVisitor)
- Throws:
java.io.IOException
- See Also:
IndexReader.document(int, StoredFieldVisitor)
-
doc
public Document doc(int docID, java.util.Set<java.lang.String> fieldsToLoad) throws java.io.IOException
Sugar for.getIndexReader().document(docID, fieldsToLoad)
- Throws:
java.io.IOException
- See Also:
IndexReader.document(int, Set)
-
setSimilarity
public void setSimilarity(Similarity similarity)
Expert: Set the Similarity implementation used by this IndexSearcher.
-
getSimilarity
public Similarity getSimilarity()
Expert: Get theSimilarity
to use to compute scores. This returns theSimilarity
that has been set throughsetSimilarity(Similarity)
or the defaultSimilarity
if none has been set explicitly.
-
count
public int count(Query query) throws java.io.IOException
Count how many documents match the given query. May be faster than counting number of hits by collecting all matches, as the number of hits is retrieved from the index statistics when possible.- Throws:
java.io.IOException
-
getSlices
public IndexSearcher.LeafSlice[] getSlices()
Returns the leaf slices used for concurrent searching, or null if noExecutor
was passed to the constructor.
-
searchAfter
public TopDocs searchAfter(ScoreDoc after, Query query, int numHits) throws java.io.IOException
Finds the topn
hits forquery
where all results are after a previous result (after
).By passing the bottom result from a previous page as
after
, this method can be used for efficient 'deep-paging' across potentially large result sets.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
search
public TopDocs search(Query query, int n) throws java.io.IOException
Finds the topn
hits forquery
.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
search
public void search(Query query, Collector results) throws java.io.IOException
Lower-level search API.LeafCollector.collect(int)
is called for every matching document.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
search
public TopFieldDocs search(Query query, int n, Sort sort, boolean doDocScores) throws java.io.IOException
Search implementation with arbitrary sorting, plus control over whether hit scores and max score should be computed. Finds the topn
hits forquery
, and sorting the hits by the criteria insort
. IfdoDocScores
istrue
then the score of each hit will be computed and returned. IfdoMaxScore
istrue
then the maximum score over all collected hits will be computed.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
search
public TopFieldDocs search(Query query, int n, Sort sort) throws java.io.IOException
Search implementation with arbitrary sorting.
-
searchAfter
public TopDocs searchAfter(ScoreDoc after, Query query, int n, Sort sort) throws java.io.IOException
Finds the topn
hits forquery
where all results are after a previous result (after
).By passing the bottom result from a previous page as
after
, this method can be used for efficient 'deep-paging' across potentially large result sets.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
searchAfter
public TopFieldDocs searchAfter(ScoreDoc after, Query query, int numHits, Sort sort, boolean doDocScores) throws java.io.IOException
Finds the topn
hits forquery
where all results are after a previous result (after
), allowing control over whether hit scores and max score should be computed.By passing the bottom result from a previous page as
after
, this method can be used for efficient 'deep-paging' across potentially large result sets. IfdoDocScores
istrue
then the score of each hit will be computed and returned. IfdoMaxScore
istrue
then the maximum score over all collected hits will be computed.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
searchAfter
private TopFieldDocs searchAfter(FieldDoc after, Query query, int numHits, Sort sort, boolean doDocScores) throws java.io.IOException
- Throws:
java.io.IOException
-
search
public <C extends Collector,T> T search(Query query, CollectorManager<C,T> collectorManager) throws java.io.IOException
Lower-level search API. Search all leaves using the givenCollectorManager
. In contrast tosearch(Query, Collector)
, this method will use the searcher'sExecutor
in order to parallelize execution of the collection on the configuredleafSlices
.- Throws:
java.io.IOException
- See Also:
CollectorManager
-
search
private <C extends Collector,T> T search(Weight weight, CollectorManager<C,T> collectorManager, C firstCollector) throws java.io.IOException
- Throws:
java.io.IOException
-
search
protected void search(java.util.List<LeafReaderContext> leaves, Weight weight, Collector collector) throws java.io.IOException
Lower-level search API.LeafCollector.collect(int)
is called for every document.
NOTE: this method executes the searches on all given leaves exclusively. To search across all the searchers leaves use
leafContexts
.- Parameters:
leaves
- the searchers leaves to execute the searches onweight
- to match documentscollector
- to receive hits- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
rewrite
public Query rewrite(Query original) throws java.io.IOException
Expert: called to re-write queries into primitive queries.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
rewrite
private Query rewrite(Query original, boolean needsScores) throws java.io.IOException
- Throws:
java.io.IOException
-
getNumClausesCheckVisitor
private static QueryVisitor getNumClausesCheckVisitor()
Returns a QueryVisitor which recursively checks the total number of clauses that a query and its children cumulatively have and validates that the total number does not exceed the specified limit. ThrowsIndexSearcher.TooManyNestedClauses
if the limit is exceeded.
-
explain
public Explanation explain(Query query, int doc) throws java.io.IOException
Returns an Explanation that describes howdoc
scored againstquery
.This is intended to be used in developing Similarity implementations, and, for good performance, should not be displayed with every hit. Computing an explanation is as expensive as executing the query over the entire index.
- Throws:
java.io.IOException
-
explain
protected Explanation explain(Weight weight, int doc) throws java.io.IOException
Expert: low-level implementation method Returns an Explanation that describes howdoc
scored againstweight
.This is intended to be used in developing Similarity implementations, and, for good performance, should not be displayed with every hit. Computing an explanation is as expensive as executing the query over the entire index.
Applications should call
explain(Query, int)
.- Throws:
IndexSearcher.TooManyClauses
- If a query would exceedgetMaxClauseCount()
clauses.java.io.IOException
-
createWeight
public Weight createWeight(Query query, ScoreMode scoreMode, float boost) throws java.io.IOException
Creates aWeight
for the given query, potentially adding caching if possible and configured.- Throws:
java.io.IOException
-
getTopReaderContext
public IndexReaderContext getTopReaderContext()
Returns this searchers the top-levelIndexReaderContext
.- See Also:
IndexReader.getContext()
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
termStatistics
public TermStatistics termStatistics(Term term, int docFreq, long totalTermFreq) throws java.io.IOException
ReturnsTermStatistics
for a term.This can be overridden for example, to return a term's statistics across a distributed collection.
- Parameters:
docFreq
- The document frequency of the term. It must be greater or equal to 1.totalTermFreq
- The total term frequency.- Returns:
- A
TermStatistics
(never null). - Throws:
java.io.IOException
-
collectionStatistics
public CollectionStatistics collectionStatistics(java.lang.String field) throws java.io.IOException
ReturnsCollectionStatistics
for a field, ornull
if the field does not exist (has no indexed terms)This can be overridden for example, to return a field's statistics across a distributed collection.
- Throws:
java.io.IOException
-
getExecutor
public java.util.concurrent.Executor getExecutor()
Returns this searchers executor ornull
if no executor was provided
-
getSliceExecutionControlPlane
private static SliceExecutor getSliceExecutionControlPlane(java.util.concurrent.Executor executor)
Return the SliceExecutionControlPlane instance to be used for this IndexSearcher instance
-
-