Package org.apache.hadoop.hbase

Class KeyValueUtil

java.lang.Object
org.apache.hadoop.hbase.KeyValueUtil
@Private public class KeyValueUtil extends Object
static convenience methods for dealing with KeyValues and collections of KeyValues

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final org.slf4j.Logger
    LOG
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    KeyValueUtil()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static int
    appendKeyTo(Cell cell, byte[] output, int offset)
     
    static int
    appendKeyTo(Cell cell, ByteBuffer buf, int offset)
     
    static int
    appendTo(Cell cell, ByteBuffer buf, int offset, boolean withTags)
    Copy the Cell content into the passed buf in KeyValue serialization format.
    static int
    appendToByteArray(Cell cell, byte[] output, int offset, boolean withTags)
    copy key and value
    static void
    appendToByteBuffer(ByteBuffer bb, KeyValue kv, boolean includeMvccVersion)
     
    (package private) static String
    bytesToHex(byte[] buf, int offset, int length)
     
    (package private) static void
    checkKeyValueBytes(byte[] buf, int offset, int length, boolean withTags)
     
    private static int
    checkKeyValueTagBytes(byte[] buf, int offset, int length, int pos, int endOffset)
     
    static ByteBuffer
    copyKeyToNewByteBuffer(Cell cell)
    The position will be set to the beginning of the new ByteBuffer
    static byte[]
    copyToNewByteArray(Cell cell)
     
    static KeyValue
    copyToNewKeyValue(Cell cell)
    copy the cell to create a new keyvalue
    static KeyValue
    create(int length, DataInput in)
    Create a KeyValue reading length from in
    static KeyValue
    create(DataInput in)
    Where to read bytes from.
    static KeyValue
    createFirstOnRow(byte[] row)
    Create a KeyValue that is smaller than all other possible KeyValues for the given row.
    static KeyValue
    createFirstOnRow(byte[] row, byte[] family, byte[] qualifier)
    Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row,family,qualifier.
    static KeyValue
    createFirstOnRow(byte[] buffer, byte[] row, byte[] family, byte[] qualifier)
    Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
    static KeyValue
    createFirstOnRow(byte[] row, byte[] f, byte[] q, long ts)
    Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
    static KeyValue
    createFirstOnRow(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength)
    Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
    static KeyValue
    createFirstOnRow(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength)
    Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier.
    static KeyValue
    createFirstOnRow(byte[] row, int roffset, short rlength)
    Create a KeyValue that is smaller than all other possible KeyValues for the given row.
    static KeyValue
    createFirstOnRow(byte[] row, long ts)
    Creates a KeyValue that is smaller than all other KeyValues that are older than the passed timestamp.
    static KeyValue
    createKeyValueFromInputStream(InputStream in, boolean withTags)
    Create a KeyValue reading from the raw InputStream.
    static KeyValue
    createKeyValueFromKey(byte[] b)
    Returns a KeyValue made of a byte array that holds the key-only part.
    static KeyValue
    createKeyValueFromKey(byte[] b, int o, int l)
    Return a KeyValue made of a byte array that holds the key-only part.
    static KeyValue
    createKeyValueFromKey(ByteBuffer bb)
    Return a KeyValue made of a byte buffer that holds the key-only part.
    static KeyValue
    createLastOnRow(byte[] row)
    Creates a KeyValue that is last on the specified row id.
    static KeyValue
    createLastOnRow(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength)
    Create a KeyValue for the specified row, family and qualifier that would be larger than or equal to all other possible KeyValues that have the same row, family, qualifier.
    static KeyValue
    ensureKeyValue(Cell cell)
    Deprecated.
    without any replacement.
    static List<KeyValue>
    ensureKeyValues(List<Cell> cells)
    Deprecated.
    static int
    getSerializedSize(Cell cell, boolean withTags)
     
    private static int
    keyLength(short rlen, byte flen, int qlen)
     
    static int
    keyLength(Cell cell)
    Returns number of bytes this cell's key part would have been used if serialized as in KeyValue.
    static int
    length(short rlen, byte flen, int qlen, int vlen, int tlen, boolean withTags)
    length
    static int
    lengthWithMvccVersion(KeyValue kv, boolean includeMvccVersion)
     
    static KeyValue
    nextShallowCopy(ByteBuffer bb, boolean includesMvccVersion, boolean includesTags)
    Creates a new KeyValue object positioned in the supplied ByteBuffer and sets the ByteBuffer's position to the start of the next KeyValue.
    static int
    oswrite(Cell cell, OutputStream out, boolean withTags)
     
    static KeyValue
    previousKey(KeyValue in)
    Decrement the timestamp.
    static KeyValue
    toNewKeyCell(Cell cell)
    Copies the key to a new KeyValue
    static int
    totalLengthWithMvccVersion(Iterable<? extends KeyValue> kvs, boolean includeMvccVersion)
     
    static long
    write(KeyValue kv, DataOutput out)
    Write out a KeyValue in the manner in which we used to when KeyValue was a Writable.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • LOG

      private static final org.slf4j.Logger LOG

  • Constructor Details

  • Method Details

    • length

      public static int length(short rlen, byte flen, int qlen, int vlen, int tlen, boolean withTags)
      length

    • keyLength

      public static int keyLength(Cell cell)
      Returns number of bytes this cell's key part would have been used if serialized as in KeyValue. Key includes rowkey, family, qualifier, timestamp and type.
      Returns:
      the key length

    • keyLength

      private static int keyLength(short rlen, byte flen, int qlen)

    • lengthWithMvccVersion

      public static int lengthWithMvccVersion(KeyValue kv, boolean includeMvccVersion)

    • totalLengthWithMvccVersion

      public static int totalLengthWithMvccVersion(Iterable<? extends KeyValue> kvs, boolean includeMvccVersion)

    • copyToNewKeyValue

      public static KeyValue copyToNewKeyValue(Cell cell)
      copy the cell to create a new keyvalue

    • copyKeyToNewByteBuffer

      public static ByteBuffer copyKeyToNewByteBuffer(Cell cell)
      The position will be set to the beginning of the new ByteBuffer
      Returns:
      the Bytebuffer containing the key part of the cell

    • toNewKeyCell

      public static KeyValue toNewKeyCell(Cell cell)
      Copies the key to a new KeyValue
      Returns:
      the KeyValue that consists only the key part of the incoming cell

    • copyToNewByteArray

      public static byte[] copyToNewByteArray(Cell cell)

    • appendKeyTo

      public static int appendKeyTo(Cell cell, byte[] output, int offset)

    • appendToByteArray

      public static int appendToByteArray(Cell cell, byte[] output, int offset, boolean withTags)
      copy key and value

    • appendTo

      public static int appendTo(Cell cell, ByteBuffer buf, int offset, boolean withTags)
      Copy the Cell content into the passed buf in KeyValue serialization format.

    • appendKeyTo

      public static int appendKeyTo(Cell cell, ByteBuffer buf, int offset)

    • appendToByteBuffer

      public static void appendToByteBuffer(ByteBuffer bb, KeyValue kv, boolean includeMvccVersion)

    • nextShallowCopy

      public static KeyValue nextShallowCopy(ByteBuffer bb, boolean includesMvccVersion, boolean includesTags)
      Creates a new KeyValue object positioned in the supplied ByteBuffer and sets the ByteBuffer's position to the start of the next KeyValue. Does not allocate a new array or copy data.

    • previousKey

      public static KeyValue previousKey(KeyValue in)
      Decrement the timestamp. For tests (currently wasteful) Remember timestamps are sorted reverse chronologically.
      Returns:
      previous key

    • createLastOnRow

      public static KeyValue createLastOnRow(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength)
      Create a KeyValue for the specified row, family and qualifier that would be larger than or equal to all other possible KeyValues that have the same row, family, qualifier. Used for reseeking. Should NEVER be returned to a client. row key row offset row length family name family offset family length column qualifier qualifier offset qualifier length
      Returns:
      Last possible key on passed row, family, qualifier.

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] row, int roffset, short rlength)
      Create a KeyValue that is smaller than all other possible KeyValues for the given row. That is any (valid) KeyValue on 'row' would sort _after_ the result.
      Parameters:
      row - - row key (arbitrary byte array)
      Returns:
      First possible KeyValue on passed row

    • createLastOnRow

      public static KeyValue createLastOnRow(byte[] row)
      Creates a KeyValue that is last on the specified row id. That is, every other possible KeyValue for the given row would compareTo() less than the result of this call.
      Parameters:
      row - row key
      Returns:
      Last possible KeyValue on passed row

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] row)
      Create a KeyValue that is smaller than all other possible KeyValues for the given row. That is any (valid) KeyValue on 'row' would sort _after_ the result.
      Parameters:
      row - - row key (arbitrary byte array)
      Returns:
      First possible KeyValue on passed row

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] row, long ts)
      Creates a KeyValue that is smaller than all other KeyValues that are older than the passed timestamp.
      Parameters:
      row - - row key (arbitrary byte array)
      ts - - timestamp
      Returns:
      First possible key on passed row and timestamp.

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] row, byte[] family, byte[] qualifier)
      Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row,family,qualifier. Used for seeking.
      Parameters:
      row - - row key (arbitrary byte array)
      family - - family name
      qualifier - - column qualifier
      Returns:
      First possible key on passed row, and column.

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] row, byte[] f, byte[] q, long ts)
      Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.
      Parameters:
      row - - row key (arbitrary byte array)
      f - - family name
      q - - column qualifier
      ts - - timestamp
      Returns:
      First possible key on passed row, column and timestamp

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength)
      Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.
      Parameters:
      row - row key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - family offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      Returns:
      First possible key on passed Row, Family, Qualifier.

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] buffer, byte[] row, byte[] family, byte[] qualifier) throws IllegalArgumentException
      Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.
      Parameters:
      buffer - the buffer to use for the new KeyValue object
      row - the value key
      family - family name
      qualifier - column qualifier
      Returns:
      First possible key on passed Row, Family, Qualifier.
      Throws:
      IllegalArgumentException - The resulting KeyValue object would be larger than the provided buffer or than Integer.MAX_VALUE

    • createFirstOnRow

      public static KeyValue createFirstOnRow(byte[] buffer, int boffset, byte[] row, int roffset, int rlength, byte[] family, int foffset, int flength, byte[] qualifier, int qoffset, int qlength) throws IllegalArgumentException
      Create a KeyValue for the specified row, family and qualifier that would be smaller than all other possible KeyValues that have the same row, family, qualifier. Used for seeking.
      Parameters:
      buffer - the buffer to use for the new KeyValue object
      boffset - buffer offset
      row - the value key
      roffset - row offset
      rlength - row length
      family - family name
      foffset - family offset
      flength - family length
      qualifier - column qualifier
      qoffset - qualifier offset
      qlength - qualifier length
      Returns:
      First possible key on passed Row, Family, Qualifier.
      Throws:
      IllegalArgumentException - The resulting KeyValue object would be larger than the provided buffer or than Integer.MAX_VALUE

    • ensureKeyValue

      @Deprecated public static KeyValue ensureKeyValue(Cell cell)
      Deprecated.
      without any replacement.
      Returns:
      cell if it is an object of class KeyValue else we will return a new KeyValue instance made from cell Note: Even if the cell is an object of any of the subclass of KeyValue, we will create a new KeyValue object wrapping same buffer. This API is used only with MR based tools which expect the type to be exactly KeyValue. That is the reason for doing this way.

    • ensureKeyValues

      @Deprecated public static List<KeyValue> ensureKeyValues(List<Cell> cells)
      Deprecated.

    • write

      public static long write(KeyValue kv, DataOutput out) throws IOException
      Write out a KeyValue in the manner in which we used to when KeyValue was a Writable.
      Returns:
      Length written on stream
      Throws:
      IOException
      See Also:

    • bytesToHex

      static String bytesToHex(byte[] buf, int offset, int length)

    • checkKeyValueBytes

      static void checkKeyValueBytes(byte[] buf, int offset, int length, boolean withTags)

    • checkKeyValueTagBytes

      private static int checkKeyValueTagBytes(byte[] buf, int offset, int length, int pos, int endOffset)

    • createKeyValueFromInputStream

      public static KeyValue createKeyValueFromInputStream(InputStream in, boolean withTags) throws IOException
      Create a KeyValue reading from the raw InputStream. Named createKeyValueFromInputStream so doesn't clash with create(DataInput)
      Parameters:
      in - inputStream to read.
      withTags - whether the keyvalue should include tags are not
      Returns:
      Created KeyValue OR if we find a length of zero, we will return null which can be useful marking a stream as done.
      Throws:
      IOException

    • createKeyValueFromKey

      public static KeyValue createKeyValueFromKey(byte[] b)
      Returns a KeyValue made of a byte array that holds the key-only part. Needed to convert hfile index members to KeyValues.

    • createKeyValueFromKey

      public static KeyValue createKeyValueFromKey(ByteBuffer bb)
      Return a KeyValue made of a byte buffer that holds the key-only part. Needed to convert hfile index members to KeyValues.

    • createKeyValueFromKey

      public static KeyValue createKeyValueFromKey(byte[] b, int o, int l)
      Return a KeyValue made of a byte array that holds the key-only part. Needed to convert hfile index members to KeyValues.

    • create

      public static KeyValue create(DataInput in) throws IOException
      Where to read bytes from. Creates a byte array to hold the KeyValue backing bytes copied from the steam.
      Returns:
      KeyValue created by deserializing from in OR if we find a length of zero, we will return null which can be useful marking a stream as done.
      Throws:
      IOException

    • create

      public static KeyValue create(int length, DataInput in) throws IOException
      Create a KeyValue reading length from in
      Returns:
      Created KeyValue OR if we find a length of zero, we will return null which can be useful marking a stream as done.
      Throws:
      IOException

    • getSerializedSize

      public static int getSerializedSize(Cell cell, boolean withTags)

    • oswrite

      public static int oswrite(Cell cell, OutputStream out, boolean withTags) throws IOException
      Throws:
      IOException