Ruby client for Apache Spark SQL and Apache Hive

Add this line to your application’s Gemfile:

gem "hexspace"

Create a client

client = Hexspace::Client.new

Execute queries

client.execute("SELECT COUNT(*) FROM users")

There are a number of connection options available.

Hexspace::Client.new(
  host: "localhost",
  port: 10000,
  username: "user",
  password: "secret",
  database: "default",
  mode: :sasl,
  timeout: 10
)

Supported modes are :sasl, :nosasl, :http, and :https. Please create an issue if you need Kerberos.

The timeout is in seconds and only applies to :sasl and :nosasl.

Set a timeout

client.execute(statement, timeout: 10)

Get a Hexspace::Result object instead of an array of hashes

result = client.execute(statement, result_object: true)
result.rows
result.columns
result.column_types
result.to_a

Download Apache Spark and start the Thift server.

./sbin/start-thriftserver.sh

Download Apache Hive and initialize the schema.

./bin/schematool -dbType derby -initSchema

Then start the HiveServer2.

./bin/hiveserver2

It can take a minute or two to be ready. To debug, pass --hiveconf hive.root.logger=DEBUG,console.

View the changelog

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone https://github.com/ankane/hexspace.git
cd hexspace
bundle install
# create a database
beeline -u jdbc:hive2://localhost:10000 -e 'CREATE DATABASE hexspace_test;'
# run the tests
bundle exec rake test

Resources

• Spark SQL Reference
• Hive Language Manual
• Thrift SASL Spec