Using HBase/Thrift through the Rackspace Load Balancer
Using the binary Thrift protocol through a load balancer can be a little tricky. While it works out of the box in HAProxy, you may run into some slight trouble running it through other load balancers. By default, when trying to access Thrift through the Rackspace Cloud Load Balancers, you may get the following exception:
The issue turned out to be a setting on the Rackspace load balancer.
The default TCP protocol, called simply “TCP”, did not work. By changing the setting to “TCP (Client First)”, HappyBase was able to connect via Thrift.
What does this setting do? The Rackspace documentation simply says “This protocol is similiar to TCP, but is more efficient when a client is expected to write the data first”. This would seem to hint that the regular TCP option attemps to read from the server socket when a client connects, where-as as an RPC protocol, Thrift expects the client to send the first data packet.