Class ByteBlockPool

java.lang.Object
org.apache.lucene.util.ByteBlockPool
All Implemented Interfaces:
Accountable

public final class ByteBlockPool extends Object implements Accountable
This class enables the allocation of fixed-size buffers and their management as part of a buffer array. Allocation is done through the use of an ByteBlockPool.Allocator which can be customized, e.g. to allow recycling old buffers. There are methods for writing (append(BytesRef) and reading from the buffers (e.g. readBytes(long, byte[], int, int), which handle read/write operations across buffer boundaries.
  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    Abstract class for allocating and freeing byte blocks.
    static final class 
    A simple ByteBlockPool.Allocator that never recycles.
    static class 
    A simple ByteBlockPool.Allocator that never recycles, but tracks how much total RAM is in use.
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
     
    private static final long
     
    byte[]
    Current head buffer.
    private byte[][]
    Array of buffers currently used in the pool.
    private int
    index into the buffers array pointing to the current buffer used as the head
    static final int
    Use this to find the position of a global offset in a particular buffer.
    static final int
    Use this to find the index of the buffer containing a byte, given an offset to that byte.
    static final int
    The size of each buffer in the pool.
    int
    Offset from the start of the first buffer to the start of the current buffer, which is bufferUpto * BYTE_BLOCK_SIZE.
    int
    Where we are in the head buffer.

    Fields inherited from interface org.apache.lucene.util.Accountable

    NULL_ACCOUNTABLE
  • Constructor Summary

    Constructors
    Constructor
    Description
     
  • Method Summary

    Modifier and Type
    Method
    Description
    void
    append(byte[] bytes)
    Append the provided byte array at the current position.
    void
    append(byte[] bytes, int offset, int length)
    Append some portion of the provided byte array at the current position.
    void
    append(ByteBlockPool srcPool, long srcOffset, int length)
    Append the bytes from a source ByteBlockPool at a given offset and length
    void
    Appends the bytes in the provided BytesRef at the current position.
    private void
    appendBytesSingleBuffer(ByteBlockPool srcPool, long srcOffset, int length)
     
    byte[]
    getBuffer(int bufferIndex)
    Retrieve the buffer at the specified index from the buffer pool.
    long
    the current position (in absolute value) of this byte pool
    void
    Allocates a new buffer and advances the pool to it.
    long
    Return the memory usage of this object in bytes.
    byte
    readByte(long offset)
    Read a single byte at the given offset
    void
    readBytes(long offset, byte[] bytes, int bytesOffset, int bytesLength)
    Reads bytes out of the pool starting at the given offset with the given length into the given byte array at offset off.
    void
    reset(boolean zeroFillBuffers, boolean reuseFirst)
    Expert: Resets the pool to its initial state, while optionally reusing the first buffer.
    (package private) void
    setBytesRef(BytesRefBuilder builder, BytesRef result, long offset, int length)
    Fill the provided BytesRef with the bytes at the specified offset and length.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.lucene.util.Accountable

    getChildResources
  • Field Details

    • BASE_RAM_BYTES

      private static final long BASE_RAM_BYTES
    • BYTE_BLOCK_SHIFT

      public static final int BYTE_BLOCK_SHIFT
      Use this to find the index of the buffer containing a byte, given an offset to that byte.

      bufferUpto = globalOffset >> BYTE_BLOCK_SHIFT

      bufferUpto = globalOffset / BYTE_BLOCK_SIZE

      See Also:
    • BYTE_BLOCK_SIZE

      public static final int BYTE_BLOCK_SIZE
      The size of each buffer in the pool.
      See Also:
    • BYTE_BLOCK_MASK

      public static final int BYTE_BLOCK_MASK
      Use this to find the position of a global offset in a particular buffer.

      positionInCurrentBuffer = globalOffset & BYTE_BLOCK_MASK

      positionInCurrentBuffer = globalOffset % BYTE_BLOCK_SIZE

      See Also:
    • buffers

      private byte[][] buffers
      Array of buffers currently used in the pool. Buffers are allocated if needed.
    • bufferUpto

      private int bufferUpto
      index into the buffers array pointing to the current buffer used as the head
    • byteUpto

      public int byteUpto
      Where we are in the head buffer.
    • buffer

      public byte[] buffer
      Current head buffer.
    • byteOffset

      public int byteOffset
      Offset from the start of the first buffer to the start of the current buffer, which is bufferUpto * BYTE_BLOCK_SIZE. The buffer pool maintains this offset because it is the first to overflow if there are too many allocated blocks.
    • allocator

      private final ByteBlockPool.Allocator allocator
  • Constructor Details

  • Method Details

    • reset

      public void reset(boolean zeroFillBuffers, boolean reuseFirst)
      Expert: Resets the pool to its initial state, while optionally reusing the first buffer. Buffers that are not reused are reclaimed by ByteBlockPool.Allocator.recycleByteBlocks(byte[][], int, int). Buffers can be filled with zeros before recycling them. This is useful if a slice pool works on top of this byte pool and relies on the buffers being filled with zeros to find the non-zero end of slices.
      Parameters:
      zeroFillBuffers - if true the buffers are filled with 0. This should be set to true if this pool is used with slices.
      reuseFirst - if true the first buffer will be reused and calling nextBuffer() is not needed after reset iff the block pool was used before ie. nextBuffer() was called before.
    • nextBuffer

      public void nextBuffer()
      Allocates a new buffer and advances the pool to it. This method should be called once after the constructor to initialize the pool. In contrast to the constructor, a reset(boolean, boolean) call will advance the pool to its first buffer immediately.
    • setBytesRef

      void setBytesRef(BytesRefBuilder builder, BytesRef result, long offset, int length)
      Fill the provided BytesRef with the bytes at the specified offset and length. This will avoid copying the bytes if the slice fits into a single block; otherwise, it uses the provided BytesRefBuilder to copy bytes over.
    • append

      public void append(BytesRef bytes)
      Appends the bytes in the provided BytesRef at the current position.
    • append

      public void append(ByteBlockPool srcPool, long srcOffset, int length)
      Append the bytes from a source ByteBlockPool at a given offset and length
      Parameters:
      srcPool - the source pool to copy from
      srcOffset - the source pool offset
      length - the number of bytes to copy
    • appendBytesSingleBuffer

      private void appendBytesSingleBuffer(ByteBlockPool srcPool, long srcOffset, int length)
    • append

      public void append(byte[] bytes)
      Append the provided byte array at the current position.
      Parameters:
      bytes - the byte array to write
    • append

      public void append(byte[] bytes, int offset, int length)
      Append some portion of the provided byte array at the current position.
      Parameters:
      bytes - the byte array to write
      offset - the offset of the byte array
      length - the number of bytes to write
    • readBytes

      public void readBytes(long offset, byte[] bytes, int bytesOffset, int bytesLength)
      Reads bytes out of the pool starting at the given offset with the given length into the given byte array at offset off.

      Note: this method allows to copy across block boundaries.

    • readByte

      public byte readByte(long offset)
      Read a single byte at the given offset
      Parameters:
      offset - the offset to read
      Returns:
      the byte
    • ramBytesUsed

      public long ramBytesUsed()
      Description copied from interface: Accountable
      Return the memory usage of this object in bytes. Negative values are illegal.
      Specified by:
      ramBytesUsed in interface Accountable
    • getPosition

      public long getPosition()
      the current position (in absolute value) of this byte pool
    • getBuffer

      public byte[] getBuffer(int bufferIndex)
      Retrieve the buffer at the specified index from the buffer pool.