Using this library in MapReduce jobs

In Mapper

If your MapReduce job is reading from an HBase table, in your map() method, HBase's Result object can be converted to object of your bean-like class using below method:

T readValue(ImmutableBytesWritable rowKey, Result result, Class<T> clazz)

where T is your bean-like class that extends this library's HBRecord interface (e.g. Citizen class above).

For example:

Citizen e = hbObjectMapper.readValue(key, value, Citizen.class);

In Reducer

If your MapReduce job is writing to an HBase table, in your reduce() method, object of your bean-like class can be converted to HBase's Put (for row contents) and ImmutableBytesWritable (for row key) using below methods:

ImmutableBytesWritable getRowKey(T record)

Put writeValueAsPut(T record)

where T is your bean-like class that extends this library's HBRecord interface (e.g. Citizen class above).

For example, below code in Reducer writes your object as one HBase row with appropriate column families and columns:

Citizen citizen = new Citizen(/*details*/);
context.write(hbObjectMapper.getRowKey(citizen), hbObjectMapper.writeValueAsPut(citizen));

In unit-test for Mapper

If your MapReduce job is reading from an HBase table, you would want to unit-test your map() method as below.

Object of your bean-like class can be converted to HBase's Result (for row contents) and ImmutableBytesWritable (for row key) using below methods:

ImmutableBytesWritable getRowKey(T record)

Result writeValueAsResult(T record)

where T is your bean-like class that extends this library's HBRecord interface (e.g. Citizen class above).

Below is an example of unit-test of a Mapper using MRUnit:

Citizen citizen = new Citizen(/*params*/);
citizenMapDriver
.withInput(
  hbObjectMapper.getRowKey(citizen),
  hbObjectMapper.writeValueAsResult(citizen)
)
.withOutput(
  hbObjectMapper.toIbw("key"),
  new IntWritable(citizen.getAge())
)
.runTest();

In unit-test for Reducer

If your MapReduce job is writing to an HBase table, you would want to unit-test your reduce() method as below.

HBase's Put object can be converted to your object of you bean-like class using below method:

T readValue(ImmutableBytesWritable rowKey, Put put, Class<T> clazz)

where T is your bean-like class that extends this library's HBRecord interface (e.g. Citizen class above).

Below is an example of unit-test of a Reducer using MRUnit:

Pair<ImmutableBytesWritable, Mutation> reducerResult = citizenReduceDriver
  .withInput(hbObjectMapper.toIbw("key"), inputList)
  .run()
.get(0);
CitizenSummary citizenSummary = hbObjectMapper.readValue(
  reducerResult.getFirst(),
  (Put) reducerResult.getSecond(),
  CitizenSummary.class
);