Class LogBuffer
- java.lang.Object
-
- org.objectweb.howl.log.LogObject
-
- org.objectweb.howl.log.LogBuffer
-
- Direct Known Subclasses:
BlockLogBuffer
abstract class LogBuffer extends LogObject
Classes used as buffers in LogBufferManager must implement this interface.This abstract class implements methods common to all LogBuffer sub-classes.
-
-
Field Summary
Fields Modifier and Type Field Description (package private) int
bsn
buffer sequence number.(package private) java.nio.ByteBuffer
buffer
(package private) int
bytesUsed
Number of used data bytes in the buffer.(package private) java.util.zip.Adler32
checksum
(package private) byte[]
checksumBuffer
Local buffer used to compute checksums.(package private) boolean
doChecksum
switch to enable computation of checksum.(package private) boolean
forceNow
set true by LogFileManager if the buffer must be forced due to file switch.(package private) int
index
buffer number used by owner (LogBufferManager) to workerID into an array of buffers.(package private) java.io.IOException
ioexception
IOException from last write(package private) int
iostatus
results of last write.(package private) LogFile
lf
LogFile associated with the current LogBuffer.(package private) java.lang.String
name
name of this buffer object.(package private) boolean
rewind
set true if this LogBuffer should issue a rewind on the FileChannel before writing ByteBuffer.(package private) long
tod
currentTimeMillis that buffer was initialized.protected int
waitingThreads
number of waiting threads.(package private) java.lang.Object
waitingThreadsLock
mutex for synchronizing access to waitingThreads
-
Constructor Summary
Constructors Constructor Description LogBuffer(Configuration config)
default constructor.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description (package private) int
checksum()
Computes a checksum over the the entire byte buffer backing this LogBuffer object.(package private) java.lang.String
getName()
May be used in traces or other purposes for debugging.(package private) abstract java.lang.String
getStats()
returns statistics for this LogBuffer object.(package private) int
getWaitingThreads()
returns the number of threads currently waiting for the buffer to be forced to disk.(package private) abstract LogBuffer
init(int bsn, LogFileManager lfm)
initialize members for LogBuffer implementation class for reuse.(package private) abstract long
put(short type, byte[][] data, boolean sync)
puts a data record into the buffer and returns a token for record.(package private) abstract LogBuffer
read(LogFile lf, long position)
read a block of data from the LogFile object provided in the lf parameter starting at the position specified in the postiion parameter.(package private) int
release()
decrements count of waiting threads and returns updated value.(package private) abstract boolean
shouldForce()
returns true if the buffer should be forced to disk.(package private) void
sync()
park threads that are waiting for the ByteBuffer to be forced to disk.(package private) abstract void
write()
write ByteBuffer to the LogFile.
-
-
-
Field Detail
-
buffer
final java.nio.ByteBuffer buffer
-
index
int index
buffer number used by owner (LogBufferManager) to workerID into an array of buffers.Actual use of workerID is determined by the buffer manager implementation.
-
tod
long tod
currentTimeMillis that buffer was initialized.Implementations of LogBuffer should provide some means to persist tod to the file to allow recovery operations to determine when a buffer was written.
Used during replay situations to validate block integrity. The TOD field from the block header is compared with the TOD field of the block footer to verify that an entire block was written.
-
waitingThreads
protected int waitingThreads
number of waiting threads.Always synchronized on (waitingThreadsLock). Design Note:
Originally this field was volatile. It was changed at the suggestion of David Jencks to protected.We tried using (this) to protect access, but this caused some additional lock contention and performance fell off about 10%.
-
waitingThreadsLock
final java.lang.Object waitingThreadsLock
mutex for synchronizing access to waitingThreads
-
iostatus
int iostatus
results of last write.Value must be one of the constants defined in LogBuffer interface.
-
bsn
int bsn
buffer sequence number.LogBufferManager maintains a sequence number of buffers written. The sequence number is stored in the block header of each log block.
Initialized to zero.
Set to -1 by read() if bytes read is -1 (end of file)
-
rewind
boolean rewind
set true if this LogBuffer should issue a rewind on the FileChannel before writing ByteBuffer.
-
forceNow
boolean forceNow
set true by LogFileManager if the buffer must be forced due to file switch.BUG: 300505
-
ioexception
java.io.IOException ioexception
IOException from last write
-
name
java.lang.String name
name of this buffer object.
-
lf
LogFile lf
LogFile associated with the current LogBuffer.The LogBufferManager will have a pool of LogBuffers. If the containing Logger is managing a pool of files, then it is possible that some period of time, some buffers will be written to one file, while other buffers are written to another file. To allow writes and forces to separate files to be performed in parallel, each LogBuffer must keep track of its own LogFile.
-
doChecksum
boolean doChecksum
switch to enable computation of checksum.Since it takes some CPU to compute checksums over a buffer, it might be useful to disable the checksum, at least for performance measurements.
When doChecksum is true then an implementation class should compute a checksum and store the value in the buffer during the write() method.
Use of checksums is optional and depends on the actual implementation class.
-
bytesUsed
int bytesUsed
Number of used data bytes in the buffer.This is different than buffer capacity(). The bytes used is the number of bytes of usable data in a buffer. Bytes between the bytes used count and the buffer footer are undefined, possibly uninitialized or residue.
set by operations that read data from files into the buffer such as read().
checked by operations that retrieve logical records from the buffer get().
-
checksumBuffer
byte[] checksumBuffer
Local buffer used to compute checksums. The initial implementation of HOWL used the ByteBuffer.hashCode() method to compute a checksum. Since each vendor's implementation of the JVM is not guaranteed to generate the same value for hashCode, this approach is not portable, and could result in journal files that appear to be invalid if restarted using a JVM by a vendor that is different than the JVM that was used to write the journal initially. The problem is solved by implementing our own checksum method. The checksumBuffer member is used to obtain a local copy of the ByteBuffer contents for purposses of computing the checksum. Instead of obtaining each byte individually, the entire buffer is retrieved into checksumBuffer for computing the checksum.
-
checksum
final java.util.zip.Adler32 checksum
-
-
Constructor Detail
-
LogBuffer
LogBuffer(Configuration config)
default constructor.after creating a new instance of LogBuffer the caller must invoke config().
-
-
Method Detail
-
release
final int release()
decrements count of waiting threads and returns updated value.- Returns:
- number of threads still waiting after the release.
-
getWaitingThreads
final int getWaitingThreads()
returns the number of threads currently waiting for the buffer to be forced to disk.- Returns:
- current value of waitingThreads
-
sync
final void sync() throws java.io.IOException, java.lang.InterruptedException
park threads that are waiting for the ByteBuffer to be forced to disk.The count of waiting threads ( waitingThreads ) has been incremented in put() .
- Throws:
java.io.IOException
java.lang.InterruptedException
-
checksum
int checksum()
Computes a checksum over the the entire byte buffer backing this LogBuffer object.- Returns:
- the computed checksum.
-
getName
java.lang.String getName()
May be used in traces or other purposes for debugging.- Returns:
- name of LogBuffer object.
-
init
abstract LogBuffer init(int bsn, LogFileManager lfm) throws LogFileOverflowException
initialize members for LogBuffer implementation class for reuse.LogBufferManager maintains a pool of LogBuffer objects. Each time a LogBuffer is allocated from the pool for use as the current collection buffer, the init() routine is called. After performing necessary initialization, the LogBuffer invokes the LogFileManager to obtain a LogFile for use when writing and forcing the buffer. If a file switch occurrs, the LogFileManager will store a file header record into this newly initialized buffer.
- Parameters:
bsn
- Logical Block Sequence Number of the buffer. LogBufferManager maintains a block sequence number to ensure correct order of writes to disk. Some implementations of LogBuffer may include the BSN as part of a record or block header.lfm
- LogFileMaager to call to obtain a LogFile.- Returns:
- this LogBuffer
- Throws:
LogFileOverflowException
-
read
abstract LogBuffer read(LogFile lf, long position) throws java.io.IOException, InvalidLogBufferException
read a block of data from the LogFile object provided in the lf parameter starting at the position specified in the postiion parameter.Used by LogFileManager implementations to read blocks of data that are formatted by a specific LogBuffer implementation.
The LogFileManager uses LogBufferManager.getLogBuffer() method to obtain a buffer that is used for reading log files.
- Parameters:
lf
- LogFile to read.position
- within the LogFile to be read.- Returns:
- this LogBuffer reference.
- Throws:
java.io.IOException
InvalidLogBufferException
-
shouldForce
abstract boolean shouldForce()
returns true if the buffer should be forced to disk.The criteria for determining if a buffer should be forced is implementation dependent. Typically, this method will return true if there are one or more threads waiting for the buffer to be forced and the amount of time the threads has been waiting has been longer than some implementation defined value.
- Returns:
- true if buffer should be forced immediately.
-
put
abstract long put(short type, byte[][] data, boolean sync) throws LogRecordSizeException
puts a data record into the buffer and returns a token for record.PRECONDITION: caller holds a bufferManager monitor.
The caller must set the sync parameter true if the thread will ultimately call sync() after a successful put(). This strategy allows the waitingThreads counter to be incremented while the current thread holds the bufferManager monitor.
Implementations should return a token that can be used later for replay, and for debugging purposes.
The data record is passed as a byte[][] allowing callers to construct data records from individual bits of information. The arrays are concatenated into a single log record whose size is the sum of the individual array sizes. Each array is preceded by the size of the individual array. The entire record is preceded by the size of all arrays including the individual array size fields.
The record format is as follows:
+------+------------+----------------+---------+ +----------------+---------+ | type | total size | data[0].length | data[0] | . . . | data[n].length | data[n] | +------+------------+----------------+---------+ +----------------+---------+
During replay the entire record is returned as a single byte[]. The ReplayListener is responsible for decomposing the record into the original array of byte[].
- Parameters:
type
- short containing implementation defined record type information. The type is stored as the first field of the log record.data
- byte[][] to be written to log.sync
- true if thread will call sync following the put. Causes count of waitingThreads to be incremented.- Returns:
- a long that contains the physical position of the record is returned. The value returned by put() is an encoding of the physical position. The format of the returned value is implementation specific, and should be treated as opaque by the caller. Returns 0 if there is no room for the record in the current buffer.
- Throws:
LogRecordSizeException
- if the sum of all data[] array sizes is larger than the maximum allowed record size for the configured buffer size.
-
write
abstract void write() throws java.io.IOException
write ByteBuffer to the LogFile.updates the buffer header with the number of bytes used. Based on configuration, some implementations may compute a hash code or other integrety value and include the value in some implementation defined header information.
The buffer is written using the LogFile.write() method to allow the LogFile to manage file position for circular journals.
forcing and notification of waiting threads is the responsibility of the LogBufferManager that owns this LogBUffer.
- Throws:
java.io.IOException
- rethrows any IOExceptions thrown by FileChannel methods.QUESTION: should waiters be interupted if IO Error occurs?
- See Also:
init(int, LogFileManager)
,LogFile.write(LogBuffer)
-
getStats
abstract java.lang.String getStats()
returns statistics for this LogBuffer object.information is returned in the form of an XML node that can be included as a nested element in a larger document containing stats for multiple LogBuffer objects, and any containing objects.
- Returns:
- statistics for this buffer as XML
-
-