Package org.apache.lucene.index
Class BufferedUpdatesStream
- java.lang.Object
-
- org.apache.lucene.index.BufferedUpdatesStream
-
- All Implemented Interfaces:
Accountable
final class BufferedUpdatesStream extends java.lang.Object implements Accountable
Tracks the stream ofFrozenBufferedUpdates
. When DocumentsWriterPerThread flushes, its buffered deletes and updates are appended to this stream and immediately resolved (to actual docIDs, per segment) using the indexing thread that triggered the flush for concurrency. When a merge kicks off, we sync to ensure all resolving packets complete. We also apply to all segments when NRT reader is pulled, commit/close is called, or when too many deletes or updates are buffered and must be flushed (by RAM usage or by count).Each packet is assigned a generation, and each flushed or merged segment is also assigned a generation, so we can track which BufferedDeletes packets to apply to any given segment.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description (package private) static class
BufferedUpdatesStream.ApplyDeletesResult
private static class
BufferedUpdatesStream.FinishedSegments
Tracks the contiguous range of packets that have finished resolving.(package private) static class
BufferedUpdatesStream.SegmentState
Holds all per-segment internal state used while resolving deletions.
-
Field Summary
Fields Modifier and Type Field Description private java.util.concurrent.atomic.AtomicLong
bytesUsed
private BufferedUpdatesStream.FinishedSegments
finishedSegments
private InfoStream
infoStream
private long
nextGen
private java.util.concurrent.atomic.AtomicInteger
numTerms
private java.util.Set<FrozenBufferedUpdates>
updates
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Constructor Description BufferedUpdatesStream(InfoStream infoStream)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) boolean
any()
private boolean
checkDeleteStats()
(package private) void
clear()
Only used by IW.rollback(package private) void
finished(FrozenBufferedUpdates packet)
Called by indexing threads once they are fully done resolving all deletes for the provided delGen.(package private) void
finishedSegment(long delGen)
(package private) long
getCompletedDelGen()
All frozen packets up to and including this del gen are guaranteed to be finished.(package private) long
getNextGen()
(package private) int
getPendingUpdatesCount()
(package private) int
numTerms()
(package private) long
push(FrozenBufferedUpdates packet)
long
ramBytesUsed()
Return the memory usage of this object in bytes.(package private) boolean
stillRunning(long delGen)
Returns true if this delGen is still running.private void
waitApply(java.util.Set<FrozenBufferedUpdates> waitFor, IndexWriter writer)
(package private) void
waitApplyAll(IndexWriter writer)
Waits for all in-flight packets, which are already being resolved concurrently by indexing threads, to finish.(package private) void
waitApplyForMerge(java.util.List<SegmentCommitInfo> mergeInfos, IndexWriter writer)
Waits only for those in-flight packets that apply to these merge segments.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
-
-
-
Field Detail
-
updates
private final java.util.Set<FrozenBufferedUpdates> updates
-
nextGen
private long nextGen
-
finishedSegments
private final BufferedUpdatesStream.FinishedSegments finishedSegments
-
infoStream
private final InfoStream infoStream
-
bytesUsed
private final java.util.concurrent.atomic.AtomicLong bytesUsed
-
numTerms
private final java.util.concurrent.atomic.AtomicInteger numTerms
-
-
Constructor Detail
-
BufferedUpdatesStream
BufferedUpdatesStream(InfoStream infoStream)
-
-
Method Detail
-
push
long push(FrozenBufferedUpdates packet)
-
getPendingUpdatesCount
int getPendingUpdatesCount()
-
clear
void clear()
Only used by IW.rollback
-
any
boolean any()
-
numTerms
int numTerms()
-
ramBytesUsed
public long ramBytesUsed()
Description copied from interface:Accountable
Return the memory usage of this object in bytes. Negative values are illegal.- Specified by:
ramBytesUsed
in interfaceAccountable
-
waitApplyAll
void waitApplyAll(IndexWriter writer) throws java.io.IOException
Waits for all in-flight packets, which are already being resolved concurrently by indexing threads, to finish. Returns true if there were any new deletes or updates. This is called for refresh, commit.- Throws:
java.io.IOException
-
stillRunning
boolean stillRunning(long delGen)
Returns true if this delGen is still running.
-
finishedSegment
void finishedSegment(long delGen)
-
finished
void finished(FrozenBufferedUpdates packet)
Called by indexing threads once they are fully done resolving all deletes for the provided delGen. We track the completed delGens and record the maximum delGen for which all prior delGens, inclusive, are completed, so that it's safe for doc values updates to apply and write.
-
getCompletedDelGen
long getCompletedDelGen()
All frozen packets up to and including this del gen are guaranteed to be finished.
-
waitApplyForMerge
void waitApplyForMerge(java.util.List<SegmentCommitInfo> mergeInfos, IndexWriter writer) throws java.io.IOException
Waits only for those in-flight packets that apply to these merge segments. This is called when a merge needs to finish and must ensure all deletes to the merging segments are resolved.- Throws:
java.io.IOException
-
waitApply
private void waitApply(java.util.Set<FrozenBufferedUpdates> waitFor, IndexWriter writer) throws java.io.IOException
- Throws:
java.io.IOException
-
getNextGen
long getNextGen()
-
checkDeleteStats
private boolean checkDeleteStats()
-
-