Package org.apache.lucene.misc.store
Class DirectIODirectory
- java.lang.Object
-
- org.apache.lucene.store.Directory
-
- org.apache.lucene.store.FilterDirectory
-
- org.apache.lucene.misc.store.DirectIODirectory
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public class DirectIODirectory extends FilterDirectory
ADirectory
implementation for all Unixes and Windows that uses DIRECT I/O to bypass OS level IO caching during merging. For all other cases (searching, writing) we delegate to the provided Directory instance.See Overview for more details.
WARNING: this code is very new and quite easily could contain horrible bugs.
This directory passes Solr and Lucene tests on Linux, OS X, and Windows; other systems should work but have not been tested! Use at your own risk.
@throws UnsupportedOperationException if the operating system, file system or JDK does not support Direct I/O or a sufficient equivalent.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
DirectIODirectory.DirectIOIndexInput
private static class
DirectIODirectory.DirectIOIndexOutput
-
Field Summary
Fields Modifier and Type Field Description private int
blockSize
static int
DEFAULT_MERGE_BUFFER_SIZE
Default buffer size before writing to disk (256 KB); larger means less IO load but more RAM and direct buffer storage space consumed during merging.static long
DEFAULT_MIN_BYTES_DIRECT
Default min expected merge size before direct IO is used (10 MB):(package private) static java.nio.file.OpenOption
ExtendedOpenOption_DIRECT
Reference tocom.sun.nio.file.ExtendedOpenOption.DIRECT
by reflective class and enum lookup.(package private) boolean
isOpen
private int
mergeBufferSize
private long
minBytesDirect
-
Fields inherited from class org.apache.lucene.store.FilterDirectory
in
-
-
Constructor Summary
Constructors Constructor Description DirectIODirectory(FSDirectory delegate)
Create a new DirectIODirectory for the named location.DirectIODirectory(FSDirectory delegate, int mergeBufferSize, long minBytesDirect)
Create a new DirectIODirectory for the named location.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
Closes the directory.IndexOutput
createOutput(java.lang.String name, IOContext context)
Creates a new, empty file in the directory and returns anIndexOutput
instance for appending data to this file.protected void
ensureOpen()
Ensures this directory is still open.private static java.nio.file.OpenOption
getDirectOpenOption()
java.nio.file.Path
getDirectory()
IndexInput
openInput(java.lang.String name, IOContext context)
Opens a stream for reading an existing file.protected boolean
useDirectIO(java.lang.String name, IOContext context, java.util.OptionalLong fileLength)
Determines if direct IO should be used for a file.-
Methods inherited from class org.apache.lucene.store.FilterDirectory
createTempOutput, deleteFile, fileLength, getDelegate, getPendingDeletions, listAll, obtainLock, rename, sync, syncMetaData, toString, unwrap
-
Methods inherited from class org.apache.lucene.store.Directory
copyFrom, getTempFileName, openChecksumInput
-
-
-
-
Field Detail
-
DEFAULT_MERGE_BUFFER_SIZE
public static final int DEFAULT_MERGE_BUFFER_SIZE
Default buffer size before writing to disk (256 KB); larger means less IO load but more RAM and direct buffer storage space consumed during merging.- See Also:
- Constant Field Values
-
DEFAULT_MIN_BYTES_DIRECT
public static final long DEFAULT_MIN_BYTES_DIRECT
Default min expected merge size before direct IO is used (10 MB):- See Also:
- Constant Field Values
-
blockSize
private final int blockSize
-
mergeBufferSize
private final int mergeBufferSize
-
minBytesDirect
private final long minBytesDirect
-
isOpen
volatile boolean isOpen
-
ExtendedOpenOption_DIRECT
static final java.nio.file.OpenOption ExtendedOpenOption_DIRECT
Reference tocom.sun.nio.file.ExtendedOpenOption.DIRECT
by reflective class and enum lookup. There are two reasons for using this instead of directly referencing ExtendedOpenOption.DIRECT:- ExtendedOpenOption.DIRECT is OpenJDK's internal proprietary API. This API causes un-suppressible(?) warning to be emitted when compiling with --release flag and value N, where N is smaller than the the version of javac used for compilation. For details, please refer to https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8259039.
- It is possible that Lucene is run using JDK that does not support ExtendedOpenOption.DIRECT. In such a case, dynamic lookup allows us to bail out with UnsupportedOperationException with meaningful error message.
This reference is
null
, if the JDK does not support direct I/O.
-
-
Constructor Detail
-
DirectIODirectory
public DirectIODirectory(FSDirectory delegate, int mergeBufferSize, long minBytesDirect) throws java.io.IOException
Create a new DirectIODirectory for the named location.- Parameters:
delegate
- Directory for non-merges, also used as reference to file system path.mergeBufferSize
- Size of buffer to use for merging.minBytesDirect
- Merges, or files to be opened for reading, smaller than this will not use direct IO. SeeDEFAULT_MIN_BYTES_DIRECT
anduseDirectIO(java.lang.String, org.apache.lucene.store.IOContext, java.util.OptionalLong)
.- Throws:
java.io.IOException
- If there is a low-level I/O error
-
DirectIODirectory
public DirectIODirectory(FSDirectory delegate) throws java.io.IOException
Create a new DirectIODirectory for the named location.- Parameters:
delegate
- Directory for non-merges, also used as reference to file system path.- Throws:
java.io.IOException
- If there is a low-level I/O error
-
-
Method Detail
-
getDirectory
public java.nio.file.Path getDirectory()
- Returns:
- the underlying file system directory
-
ensureOpen
protected void ensureOpen() throws AlreadyClosedException
Description copied from class:Directory
Ensures this directory is still open.- Overrides:
ensureOpen
in classFilterDirectory
- Throws:
AlreadyClosedException
- if this directory is closed.
-
useDirectIO
protected boolean useDirectIO(java.lang.String name, IOContext context, java.util.OptionalLong fileLength)
Determines if direct IO should be used for a file. By default this tests if it is a merge context and if the merge or file length extends the minimum size (seeDEFAULT_MIN_BYTES_DIRECT
). Subclasses may override method to enforce direct IO for specific file types.- Parameters:
name
- file name (unused by default implementation)context
- information about merge sizefileLength
- if available, gives the file length. Will be empty when requesting anIndexOutput
.- Returns:
true
if direct IO should be used;false
if input/output should be requested from delegate directory.
-
openInput
public IndexInput openInput(java.lang.String name, IOContext context) throws java.io.IOException
Description copied from class:Directory
Opens a stream for reading an existing file.This method must throw either
NoSuchFileException
orFileNotFoundException
ifname
points to a non-existing file.- Overrides:
openInput
in classFilterDirectory
- Parameters:
name
- the name of an existing file.- Throws:
java.io.IOException
- in case of I/O error
-
createOutput
public IndexOutput createOutput(java.lang.String name, IOContext context) throws java.io.IOException
Description copied from class:Directory
Creates a new, empty file in the directory and returns anIndexOutput
instance for appending data to this file.This method must throw
FileAlreadyExistsException
if the file already exists.- Overrides:
createOutput
in classFilterDirectory
- Parameters:
name
- the name of the file to create.- Throws:
java.io.IOException
- in case of I/O error
-
close
public void close() throws java.io.IOException
Description copied from class:Directory
Closes the directory.- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Overrides:
close
in classFilterDirectory
- Throws:
java.io.IOException
-
getDirectOpenOption
private static java.nio.file.OpenOption getDirectOpenOption()
-
-