日期:2014-05-16  浏览次数:20430 次

hfile为什么会把偏移信息放在hfile的尾部?

今天会上大家争论了hfile为什么会把偏移信息放在hfile的尾部?

quora上有个解释如下:

If it's at the end, you can efficiently compute the contents of the index while writing the file out. This index often has a lot of byte offsets in it, and figuring out what these offsets are can be done easily as you write the file.

Having the index at the end doesn't make reading it inefficient: you can read the very end of the file to find out where the index starts, and then read the index. This is about the same as reading the index out of the start of the file.?

如果再做compression,figure out offset就更方便了。memstore flash时,keyvalue compress完后就写hfile比compress完暂留在内存并等所有的keyvalue计算完后才写hfile能充分利用cpu、内存和磁盘io。

?

欢迎指正。