Package org.apache.lucene.index
Class IndexingChain
- java.lang.Object
-
- org.apache.lucene.index.IndexingChain
-
- All Implemented Interfaces:
Accountable
final class IndexingChain extends java.lang.Object implements Accountable
Default general purpose indexing chain, which handles indexing all types of fields.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
IndexingChain.FieldSchema
A schema of the field in the current document.private static class
IndexingChain.IntBlockAllocator
private class
IndexingChain.PerField
NOTE: not static: accesses at least docState, termsHash.
-
Field Summary
Fields Modifier and Type Field Description private java.util.function.Consumer<java.lang.Throwable>
abortingExceptionConsumer
private ByteBlockPool.Allocator
byteBlockAllocator
(package private) Counter
bytesUsed
private IndexingChain.PerField[]
docFields
(package private) ByteBlockPool
docValuesBytePool
private IndexingChain.PerField[]
fieldHash
(package private) FieldInfos.Builder
fieldInfos
private IndexingChain.PerField[]
fields
private boolean
hasHitAbortingException
private int
hashMask
private int
indexCreatedVersionMajor
private LiveIndexWriterConfig
indexWriterConfig
private InfoStream
infoStream
private long
nextFieldGen
(package private) StoredFieldsConsumer
storedFieldsConsumer
(package private) TermsHash
termsHash
(package private) TermVectorsConsumer
termVectorsWriter
private int
totalFieldCount
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Constructor Description IndexingChain(int indexCreatedVersionMajor, SegmentInfo segmentInfo, Directory directory, FieldInfos.Builder fieldInfos, LiveIndexWriterConfig indexWriterConfig, java.util.function.Consumer<java.lang.Throwable> abortingExceptionConsumer)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) void
abort()
private void
finishStoredFields()
Calls StoredFieldsWriter.finishDocument, aborting the segment if it hits any exception.(package private) Sorter.DocMap
flush(SegmentWriteState state)
java.util.Collection<Accountable>
getChildResources()
Returns nested resources of this class.private LeafReader
getDocValuesLeafReader()
(package private) DocIdSetIterator
getHasDocValues(java.lang.String field)
private IndexingChain.PerField
getOrAddPerField(java.lang.String fieldName, IndexableFieldType fieldType)
Returns a previously createdIndexingChain.PerField
, absorbing the type information fromFieldType
, and creates a newIndexingChain.PerField
if this field name wasn't seen yet.private IndexingChain.PerField
getPerField(java.lang.String name)
Returns a previously createdIndexingChain.PerField
, or null if this field name wasn't seen yet.private void
indexDocValue(int docID, IndexingChain.PerField fp, DocValuesType dvType, IndexableField field)
Called from processDocument to index one field's doc valueprivate void
initializeFieldInfo(IndexingChain.PerField pf)
private Sorter.DocMap
maybeSortSegment(SegmentWriteState state)
private void
onAbortingException(java.lang.Throwable th)
private void
oversizeDocFields()
(package private) void
processDocument(int docID, java.lang.Iterable<? extends IndexableField> document)
private boolean
processField(int docID, IndexableField field, IndexingChain.PerField pf)
Index each field Returnstrue
, if we are indexing a unique field with postingslong
ramBytesUsed()
Return the memory usage of this object in bytes.private void
rehash()
private void
startStoredFields(int docID)
Calls StoredFieldsWriter.startDocument, aborting the segment if it hits any exception.private static void
updateDocFieldSchema(java.lang.String fieldName, IndexingChain.FieldSchema schema, IndexableFieldType fieldType)
private void
validateIndexSortDVType(Sort indexSort, java.lang.String fieldToValidate, DocValuesType dvType)
private static void
verifyUnIndexedFieldType(java.lang.String name, IndexableFieldType ft)
private void
writeDocValues(SegmentWriteState state, Sorter.DocMap sortMap)
Writes all buffered doc values (called fromflush(org.apache.lucene.index.SegmentWriteState)
).private void
writeNorms(SegmentWriteState state, Sorter.DocMap sortMap)
private void
writePoints(SegmentWriteState state, Sorter.DocMap sortMap)
Writes all buffered points.private void
writeVectors(SegmentWriteState state, Sorter.DocMap sortMap)
Writes all buffered vectors.
-
-
-
Field Detail
-
bytesUsed
final Counter bytesUsed
-
fieldInfos
final FieldInfos.Builder fieldInfos
-
termsHash
final TermsHash termsHash
-
docValuesBytePool
final ByteBlockPool docValuesBytePool
-
storedFieldsConsumer
final StoredFieldsConsumer storedFieldsConsumer
-
termVectorsWriter
final TermVectorsConsumer termVectorsWriter
-
fieldHash
private IndexingChain.PerField[] fieldHash
-
hashMask
private int hashMask
-
totalFieldCount
private int totalFieldCount
-
nextFieldGen
private long nextFieldGen
-
fields
private IndexingChain.PerField[] fields
-
docFields
private IndexingChain.PerField[] docFields
-
infoStream
private final InfoStream infoStream
-
byteBlockAllocator
private final ByteBlockPool.Allocator byteBlockAllocator
-
indexWriterConfig
private final LiveIndexWriterConfig indexWriterConfig
-
indexCreatedVersionMajor
private final int indexCreatedVersionMajor
-
abortingExceptionConsumer
private final java.util.function.Consumer<java.lang.Throwable> abortingExceptionConsumer
-
hasHitAbortingException
private boolean hasHitAbortingException
-
-
Constructor Detail
-
IndexingChain
IndexingChain(int indexCreatedVersionMajor, SegmentInfo segmentInfo, Directory directory, FieldInfos.Builder fieldInfos, LiveIndexWriterConfig indexWriterConfig, java.util.function.Consumer<java.lang.Throwable> abortingExceptionConsumer)
-
-
Method Detail
-
onAbortingException
private void onAbortingException(java.lang.Throwable th)
-
getDocValuesLeafReader
private LeafReader getDocValuesLeafReader()
-
maybeSortSegment
private Sorter.DocMap maybeSortSegment(SegmentWriteState state) throws java.io.IOException
- Throws:
java.io.IOException
-
flush
Sorter.DocMap flush(SegmentWriteState state) throws java.io.IOException
- Throws:
java.io.IOException
-
writePoints
private void writePoints(SegmentWriteState state, Sorter.DocMap sortMap) throws java.io.IOException
Writes all buffered points.- Throws:
java.io.IOException
-
writeDocValues
private void writeDocValues(SegmentWriteState state, Sorter.DocMap sortMap) throws java.io.IOException
Writes all buffered doc values (called fromflush(org.apache.lucene.index.SegmentWriteState)
).- Throws:
java.io.IOException
-
writeVectors
private void writeVectors(SegmentWriteState state, Sorter.DocMap sortMap) throws java.io.IOException
Writes all buffered vectors.- Throws:
java.io.IOException
-
writeNorms
private void writeNorms(SegmentWriteState state, Sorter.DocMap sortMap) throws java.io.IOException
- Throws:
java.io.IOException
-
abort
void abort() throws java.io.IOException
- Throws:
java.io.IOException
-
rehash
private void rehash()
-
startStoredFields
private void startStoredFields(int docID) throws java.io.IOException
Calls StoredFieldsWriter.startDocument, aborting the segment if it hits any exception.- Throws:
java.io.IOException
-
finishStoredFields
private void finishStoredFields() throws java.io.IOException
Calls StoredFieldsWriter.finishDocument, aborting the segment if it hits any exception.- Throws:
java.io.IOException
-
processDocument
void processDocument(int docID, java.lang.Iterable<? extends IndexableField> document) throws java.io.IOException
- Throws:
java.io.IOException
-
oversizeDocFields
private void oversizeDocFields()
-
initializeFieldInfo
private void initializeFieldInfo(IndexingChain.PerField pf) throws java.io.IOException
- Throws:
java.io.IOException
-
processField
private boolean processField(int docID, IndexableField field, IndexingChain.PerField pf) throws java.io.IOException
Index each field Returnstrue
, if we are indexing a unique field with postings- Throws:
java.io.IOException
-
getOrAddPerField
private IndexingChain.PerField getOrAddPerField(java.lang.String fieldName, IndexableFieldType fieldType)
Returns a previously createdIndexingChain.PerField
, absorbing the type information fromFieldType
, and creates a newIndexingChain.PerField
if this field name wasn't seen yet.
-
updateDocFieldSchema
private static void updateDocFieldSchema(java.lang.String fieldName, IndexingChain.FieldSchema schema, IndexableFieldType fieldType)
-
verifyUnIndexedFieldType
private static void verifyUnIndexedFieldType(java.lang.String name, IndexableFieldType ft)
-
validateIndexSortDVType
private void validateIndexSortDVType(Sort indexSort, java.lang.String fieldToValidate, DocValuesType dvType) throws java.io.IOException
- Throws:
java.io.IOException
-
indexDocValue
private void indexDocValue(int docID, IndexingChain.PerField fp, DocValuesType dvType, IndexableField field)
Called from processDocument to index one field's doc value
-
getPerField
private IndexingChain.PerField getPerField(java.lang.String name)
Returns a previously createdIndexingChain.PerField
, or null if this field name wasn't seen yet.
-
ramBytesUsed
public long ramBytesUsed()
Description copied from interface:Accountable
Return the memory usage of this object in bytes. Negative values are illegal.- Specified by:
ramBytesUsed
in interfaceAccountable
-
getChildResources
public java.util.Collection<Accountable> getChildResources()
Description copied from interface:Accountable
Returns nested resources of this class. The result should be a point-in-time snapshot (to avoid race conditions).- Specified by:
getChildResources
in interfaceAccountable
- See Also:
Accountables
-
getHasDocValues
DocIdSetIterator getHasDocValues(java.lang.String field)
-
-