LMDB is a fully transactional embedded key-value store that can be concurrently shared between different threads and processes. This page covers many of the issues to be aware of when accessing LMDB from different threads or processes.

For LMDB to operate correctly, the file system being used to store the LMDB database must provide proper locking support. This rules out file systems such as NFS.

Assuming the lockable file system requirement just mentioned is met, different processes may safely open the same LMDB database. It is possible for these different processes to be written in different programming languages, although you will need to pay added attention to the interoperability of the data (refer to the Keys and Value Serialisation and Compression pages).

It is important that a process does not open a specific LMDB database if it has already opened that database and not closed it. Nothing stops a process from opening as many LMDB databases as it requires.

Every LMDB operation requires a transaction, which is provided by the Txn object in LmdbJava. Transactions are usually obtained from Env.txnRead() or Env.txnWrite(), with the former being a read-only transaction and the latter being a read-write transaction. There’s also an Env.txn(Txn, TxnFlags) method for more specialised cases, but we’ll ignore that in this introductory discussion.

Only a single write transaction is allowed at a time. This means a call to Env.txnWrite() (or equivalent in other languages) will block until a transaction is available. It is possible to obtain other read transactions during this time, however those transactions will only see the committed data as existed at the time the read transaction was commenced.

Due to the way LMDB uses MVCC, opening a write transaction while there is a concurrent read transaction will cause the database file to grow. Care should therefore be taken to avoid long-lived read transactions, as the extra space can only be recovered by rebuilding the database file (eg Env.copy(File, CopyFlags.MDB_CP_COMPACT).

By default the LMDB native library use thread-local storage for transactions. This means you must not acquire a Txn and pass it to a different thread. If you wish to disable LMDB from using thread-local storage, open the Env with EnvFlags.MDB_NOTLS. This means you can pass transactions between threads, or a given thread can open multiple, parallel read transactions.

Read-only transactions offer additional flexibility. In particular read-only Txn object allocations can be minimised by using Txn.release() and Txn.renew(). A related allocation-minimising approach is to reuse read-only cursors by presenting a new, active transaction withCursor.renew(Txn).

While LMDB allows concurrent access, often it’s a single Java process that will using an LMDB database. In that case one option is to move all the concurrency control to Java. This provides a few benefits:

• Limiting the duration a thread will block waiting a write transaction
• Additional visibility into blocking (eg jstack)
• Blocking write transaction acquisition until all read transactions have concluded
• Passing transactions between threads

To fully control transactions from Java, open the Env with EnvFlags.MDB_NOLOCK.

In certain cases it may also be feasible to use a single, long-lived transaction. This provides maximum performance while avoiding transactional complexities or MVCC-related file growth. A constructor or open method may create a read-write transaction and an AutoCloseable method (or other convenient checkpoint) may commit the write transaction and obtain a new one. This obviously means different threads aren’t isolated from each other and data may be lost before the deferred commit, but this is often not an issue where the source data can be reloaded.

The simplest way to use LMDB is to open the Env without any special flags and allow it to manage thread-local transactions. Just use the transactions on the same thread as requested them, and be aware that any attempt to start a later write transaction will be blocked until an earlier write transaction is released. If you have more advanced requirements, you can reuse transactions or even switch off LMDB locking if prepared to handle this yourself.