Apache HBase is a database that runs on a Hadoop cluster. Clients can access HBase data through either a native Java API, or through a Thrift or REST gateway, making it accessible by any language.
Some of the key properties of HBase include:
- NoSQL: HBase is not a traditional relational database (RDBMS). HBase relaxes the ACID (Atomicity, Consistency, Isolation, Durability) properties of traditional RDBMS systems in order to achieve much greater scalability. Data stored in HBase also does not need to fit into a rigid schema like with an RDBMS, making it ideal for storing unstructured or semi-structured data.
- Wide-Column: HBase stores data in a table-like format with the ability to store billions of rows with millions of columns. Columns can be grouped together in "column families" which allows physical distribution of row values onto different cluster nodes.
- Distributed and Scalable: HBase group rows into "regions" which define how table data is split over multiple nodes in a cluster. If a region gets too large, it is automatically split to share the load across more servers.
- Consistent: HBase is architected to have "strongly-consistent" reads and writes, as opposed to other NoSQL databases that are "eventually-consistent". This means that once a write has been performed, all read requests for that data will return the same value.
The hierarchy for tables in HBase are as follows:
- Column Families
- Column Families
When a table is created, one or more column families are defined as high-level categories for storing data corresponding to an entry in the table. As is suggested by HBase being "column-oriented", column family data for all table entries, or rows, are stored together. For a given (row, column family) combination, multiple columns can be written at the time the data is written. Therefore, two rows in an HBase table need not necessarily share the same columns, only column families. For each (row, column-family, column) combination HBase can store multiple cells, with each cell associated with a version, or timestamp corresponding to when the data was written. HBase clients can choose to only read the most recent version of a given cell, or read all versions.
HBase Architecture Components
- HMaster: The HBase HMaster is a lightweight process responsible for assigning regions to RegionServers in the Hadoop cluster to achieve load balancing.
- RegionServer: HBase RegionServers are the worker nodes that handle read, write, update, and delete requests from clients. The RegionServer process typically runs on each Hadoop node in the cluster.
HBase Table Operations
- Reads: Client read requests are directed to the proper RegionServer by the ZooKeeper service. Clients can read all columns for a given row, or read an entire column or column family for a range of rows.
- Writes: An HBase write to a single row is atomic, meaning the whole operation either succeeds or fails, even if the write occurs across column families. A write operation to multiple rows however is not atomic, some row writes may succeed while others fail.
- Updates: Each Cell in Hbase is capable of storing multiple values, each with an associated version, or timestamp corresponding to the time the value was written. Users can specify time to live values for cells, instructing HBase to delete old cells at a given interval.
- Deletes: When an HBase client wishes to delete a row, it is not immediately removed from the table. Instead, HBase writes a tombstone marker to the blocks of data storing the row. The data is permanently removed from storage during the next major compaction, described below.
HBase Table Maintenance
- Minor Compactions: When data is written to HBase, it is first written to an in-memory structure called a memstore for performance. Intermittently, when the memstore reaches a certain size, the data is written to a store-file on disk and marked read-only. When the number of storefiles reaches a configured threshold, a minor compaction occurs to merge multiple storefiles.
- Major Compactions: Periodically, default every 24 hours, a major compaction runs to merge all storefiles together into a single storefile on each RegionServer. In addition, the RegionServer walks its tables to find any rows that were marked with a tombstone, meaning a delete was requested, and those rows are purged at this time.
Compactions, especially major compactions, can take a toll on utilization of a RegionServer. Client requests made during a compaction will experience latency and jitter as a result of resource contention.