Module org.cojen.tupl

Package org.cojen.tupl

Interface Database

All Superinterfaces:

AutoCloseable, CauseCloseable, Closeable, Flushable

public interface Database
extends CauseCloseable, Flushable

Primary database interface, containing a collection of transactional indexes. Call open to obtain a Database instance. Examples:

Open a non-durable database, limited to a max size of 100MB:

var config = new DatabaseConfig().maxCacheSize(100_000_000);
Database db = Database.open(config);
Index data = db.openIndex("mydata");

Open a regular database, with a fixed cache size, and a weak durability mode for the best transactional commit performance.

var config = new DatabaseConfig()
   .baseFilePath("/var/lib/tupl/myapp")
   .cacheSize(100_000_000)
   .durabilityMode(DurabilityMode.NO_FLUSH);

Database db = Database.open(config);
Index data = db.openIndex("mydata");

The following files are created by the above example:

• /var/lib/tupl/myapp.db – primary database file
• /var/lib/tupl/myapp.lock – lock file to ensure that at most one process can have the database open
• /var/lib/tupl/myapp.redo.0 – first transaction redo log file

New redo log files are created by checkpoints, which also delete the old files. When snapshots are in progress, one or more numbered temporary files are created. For example: /var/lib/tupl/myapp.temp.123.

See Also:

• DatabaseConfig

Method Summary

Modifier and Type	Method	Description
void	applyCachePrimer(InputStream in)	Prime the cache, from a set encoded earlier.
Snapshot	beginSnapshot()	Support for capturing a snapshot (hot backup) of the database, while still allowing concurrent modifications.
default long	capacityLimit()	Returns the current capacity limit, rounded down by page size.
default void	capacityLimit(long bytes)	Set a soft capacity limit for the database, to prevent filling up the storage device.
default void	capacityLimitOverride(long bytes)	Set capacity limits for the current thread, allowing it to perform tasks which can free up space.
void	checkpoint()	Durably sync and checkpoint all changes to the database.
default void	close()	Closes the database, ensuring durability of committed transactions.
void	close(Throwable cause)	Closes the database after an unexpected failure.
Lock	commitLock()	Returns the checkpoint commit lock, which can be held to prevent checkpoints from capturing a safe commit point.
boolean	compactFile(CompactionObserver observer, double target)	Compacts the database by shrinking the database file.
static Database	connect(SocketAddress addr, SSLContext context, long... tokens)	Establish a remote connection to a database which is running a server.
void	createCachePrimer(OutputStream out)	Writes a cache priming set into the given stream, which can then be used later to prime the cache.
CustomHandler	customWriter(String name)	Returns a handler instance suitable for writing custom redo and undo operations.
Runnable	deleteIndex(Index index)	Fully closes and deletes the given index, but does not immediately reclaim the pages it occupied.
static Database	destroy(DatabaseConfig config)	Delete the contents of an existing database, and replace it with an empty one.
boolean	failover()	If the database instance is currently acting as a leader, attempt to give up leadership and become a replica.
Index	findIndex(byte[] name)	Returns the given named index, returning null if not found.
default Index	findIndex(String name)	Returns the given named index, returning null if not found.
default <R> Table<R>	findTable(Class<R> type)	Convenience method which returns a Table that uses the index named by the row type itself.
void	flush()	Flushes all committed transactions, but not durably.
default Index	indexById(byte[] id)	Returns an index by its identifier, returning null if not found.
Index	indexById(long id)	Returns an index by its identifier, returning null if not found.
View	indexRegistryById()	Returns an unmodifiable View which maps all available index identifiers to names.
View	indexRegistryByName()	Returns an unmodifiable View which maps all available index names to identifiers.
boolean	isClosed()	Returns true if database was explicitly closed, or if it was closed due to a panic.
boolean	isLeader()	Returns true if the database instance is currently the leader.
Server	newServer()	Returns an object for enabling remote access into this database.
Sorter	newSorter()	Returns a new Sorter instance.
Index	newTemporaryIndex()	Creates a new unnamed temporary index.
default Transaction	newTransaction()	Returns a new Transaction with the default durability mode.
Transaction	newTransaction(DurabilityMode durabilityMode)	Returns a new Transaction with the given durability mode.
static Database	open(DatabaseConfig config)	Open a database, creating it if necessary.
Index	openIndex(byte[] name)	Returns the given named index, creating it if necessary.
default Index	openIndex(String name)	Returns the given named index, creating it if necessary.
default <J> Table<J>	openJoinTable(Class<J> joinType, String spec)	Convenience method which joins tables together, opening them if necessary.
default <R> Table<R>	openTable(Class<R> type)	Convenience method which returns a Table that uses the index named by the row type itself.
long	preallocate(long bytes)	Preallocates pages for immediate use.
PrepareHandler	prepareWriter(String name)	Returns a handler instance suitable for preparing transactions.
void	renameIndex(Index index, byte[] newName)	Renames the given index to the one given.
default void	renameIndex(Index index, String newName)	Renames the given index to the one given.
static Database	restoreFromSnapshot(DatabaseConfig config, InputStream in)	Restore from a snapshot, into the data files defined by the given configuration.
void	resumeCheckpoints()	Resume automatic checkpoints after having been temporarily suspended.
void	shutdown()	Cleanly closes the database, ensuring durability of all modifications.
DatabaseStats	stats()	Returns a collection of database statistics.
void	suspendCheckpoints()	Temporarily suspend automatic checkpoints and wait for any in-progress checkpoint to complete.
void	sync()	Durably flushes all committed transactions.
void	uponLeader(Runnable acquired, Runnable lost)	Registers the given task to start in a separate thread when the database instance has become the leader.
boolean	verify(VerificationObserver observer)	Verifies the integrity of the database and all indexes.

Method Details

open

static Database open(DatabaseConfig config)
              throws IOException

Open a database, creating it if necessary.

Throws:

IOException

destroy

static Database destroy(DatabaseConfig config)
                 throws IOException

Delete the contents of an existing database, and replace it with an empty one. When using a raw block device for the data file, this method must be used to format it. When database is replicated, calling destroy only affects the local replica.

Throws:

IOException

connect

static Database connect(SocketAddress addr,
 SSLContext context,
 long... tokens)
                 throws IOException

Establish a remote connection to a database which is running a server.

Parameters:

context - optionally pass a context to open a secure connection

Throws:

IllegalArgumentException - if not given one or two tokens

IOException

openIndex

Index openIndex(byte[] name)
         throws IOException

Returns the given named index, creating it if necessary.

Returns:

shared Index instance

Throws:

IOException

openIndex

default Index openIndex(String name)
                 throws IOException

Returns the given named index, creating it if necessary. Name is UTF-8 encoded.

Returns:

shared Index instance

Throws:

IOException

findIndex

Index findIndex(byte[] name)
         throws IOException

Returns the given named index, returning null if not found.

Returns:

shared Index instance; null if not found

Throws:

IOException

findIndex

default Index findIndex(String name)
                 throws IOException

Returns the given named index, returning null if not found. Name is UTF-8 encoded.

Returns:

shared Index instance; null if not found

Throws:

IOException

indexById

Index indexById(long id)
         throws IOException

Returns an index by its identifier, returning null if not found.

Returns:

shared Index instance

Throws:

IllegalArgumentException - if id is reserved

IOException

indexById

default Index indexById(byte[] id)
                 throws IOException

Returns an index by its identifier, returning null if not found.

Parameters:

id - big-endian encoded long integer

Returns:

shared Index instance

Throws:

IllegalArgumentException - if id is malformed or reserved

IOException

openTable

default <R> Table<R> openTable(Class<R> type)
                        throws IOException

Convenience method which returns a Table that uses the index named by the row type itself.

Returns:

shared Table instance

Throws:

IOException

See Also:

• Index.asTable(java.lang.Class<R>)

findTable

default <R> Table<R> findTable(Class<R> type)
                        throws IOException

Convenience method which returns a Table that uses the index named by the row type itself.

Returns:

shared Table instance; null if not found

Throws:

IOException

See Also:

• Index.asTable(java.lang.Class<R>)

openJoinTable

default <J> Table<J> openJoinTable(Class<J> joinType,
 String spec)
                            throws IOException

Convenience method which joins tables together, opening them if necessary.

Parameters:

spec - join specification

Throws:

NullPointerException - if any parameters are null

IllegalArgumentException - if join type or specification is malformed

IOException

See Also:

• Table.join(java.lang.Class<J>, java.lang.String, org.cojen.tupl.Table<?>...)

renameIndex

void renameIndex(Index index,
 byte[] newName)
          throws IOException

Renames the given index to the one given.

Parameters:

index - non-null open index

newName - new non-null name

Throws:

ClosedIndexException - if index reference is closed

IllegalStateException - if name is already in use by another index

IllegalStateException - if index belongs to another database instance

IOException

renameIndex

default void renameIndex(Index index,
 String newName)
                  throws IOException

Renames the given index to the one given. Name is UTF-8 encoded.

Parameters:

index - non-null open index

newName - new non-null name

Throws:

ClosedIndexException - if index reference is closed

IllegalStateException - if name is already in use by another index

IllegalStateException - if index belongs to another database instance

IOException

deleteIndex

Runnable deleteIndex(Index index)
              throws IOException

Fully closes and deletes the given index, but does not immediately reclaim the pages it occupied. Run the returned task in any thread to reclaim the pages.

Once deleted, accesing the index causes exceptions to be thrown. A new index by the original name can be created, which will be assigned a different unique identifier. Any transactions still referring to the old index will not affect the new index.

If the deletion task is never started or it doesn't finish normally, it will resume when the database is re-opened. All resumed deletions are completed in serial order by a background thread.

Parameters:

index - non-null open index

Returns:

non-null task to call for reclaiming the pages used by the deleted index

Throws:

ClosedIndexException - if index reference is closed

IllegalStateException - if index belongs to another database instance

IOException

See Also:

• EventListener
• Index.drop

newTemporaryIndex

Index newTemporaryIndex()
                 throws IOException

Creates a new unnamed temporary index. Temporary indexes never get written to the redo log, and they are deleted when the database is re-opened. Temporary indexes should be explicitly deleted when no longer needed, rather than waiting until the database is re-opened.

Throws:

IOException

indexRegistryByName

View indexRegistryByName()
                  throws IOException

Returns an unmodifiable View which maps all available index names to identifiers. Identifiers are long integers, big-endian encoded.

Throws:

IOException

indexRegistryById

View indexRegistryById()
                throws IOException

Returns an unmodifiable View which maps all available index identifiers to names. Identifiers are long integers, big-endian encoded.

Throws:

IOException

newTransaction

default Transaction newTransaction()

Returns a new Transaction with the default durability mode.

newTransaction

Transaction newTransaction(DurabilityMode durabilityMode)

Returns a new Transaction with the given durability mode. If null, the default is used.

customWriter

CustomHandler customWriter(String name)
                    throws IOException

Returns a handler instance suitable for writing custom redo and undo operations. A corresponding recovery instance must have been provided when the database was opened, via the customHandlers config method.

Returns:

new writer instance

Throws:

IllegalStateException - if no recovery instance by the given name is installed

IOException

prepareWriter

PrepareHandler prepareWriter(String name)
                      throws IOException

Returns a handler instance suitable for preparing transactions. A corresponding recovery instance must have been provided when the database was opened, via the prepareHandlers config method.

Returns:

new writer instance

Throws:

IllegalStateException - if no recovery instance by the given name is installed

IOException

newSorter

Sorter newSorter()

Returns a new Sorter instance. The standard algorithm is a parallel external mergesort, which attempts to use all available processors. All external storage is maintained in the database itself, in the form of temporary indexes.

preallocate

long preallocate(long bytes)
          throws IOException

Preallocates pages for immediate use. The actual amount allocated varies, depending on the amount of free pages already available.

Returns:

actual amount allocated

Throws:

IOException

capacityLimit

default void capacityLimit(long bytes)

Set a soft capacity limit for the database, to prevent filling up the storage device. When the limit is reached, writes might fail with a DatabaseFullException. No explicit limit is defined by default, and the option is ignored by non-durable databases. The limit is checked only when the database attempts to grow, and so it can be set smaller than the current database size.

Even with a capacity limit, the database file can still slowly grow in size. Explicit overrides and critical operations can allocate space which can only be reclaimed by compaction.

Parameters:

bytes - maximum capacity, in bytes; pass -1 for no limit

capacityLimit

default long capacityLimit()

Returns the current capacity limit, rounded down by page size.

Returns:

maximum capacity, in bytes; is -1 if no limit

capacityLimitOverride

default void capacityLimitOverride(long bytes)

Set capacity limits for the current thread, allowing it to perform tasks which can free up space. While doing so, it might require additional temporary storage.

Parameters:

bytes - maximum capacity, in bytes; pass -1 for no limit; pass 0 to remove override

beginSnapshot

Snapshot beginSnapshot()
                throws IOException

Support for capturing a snapshot (hot backup) of the database, while still allowing concurrent modifications. The snapshot contains all data up to the last checkpoint. Call the checkpoint method immediately before to ensure that an up-to-date snapshot is captured.

To restore from a snapshot, store it in the primary data file, which is the base file with a ".db" extension. Make sure no redo log files exist and then open the database. Alternatively, call restoreFromSnapshot, which also supports restoring into separate data files.

During the snapshot, temporary files are created to hold pre-modified copies of pages. If the snapshot destination stream blocks for too long, these files keep growing. File growth rate increases too if the database is being heavily modified. In the worst case, the temporary files can become larger than the primary database files.

Returns:

a snapshot control object, which must be closed when no longer needed

Throws:

IOException

restoreFromSnapshot

static Database restoreFromSnapshot(DatabaseConfig config,
 InputStream in)
                             throws IOException

Restore from a snapshot, into the data files defined by the given configuration. All existing data and redo log files at the snapshot destination are deleted before the restore begins.

Parameters:

in - snapshot source; does not require extra buffering; auto-closed

Throws:

IOException

createCachePrimer

void createCachePrimer(OutputStream out)
                throws IOException

Writes a cache priming set into the given stream, which can then be used later to prime the cache.

Parameters:

out - cache priming destination; buffering is recommended; not auto-closed

Throws:

IOException

See Also:

• DatabaseConfig.cachePriming(boolean)

applyCachePrimer

void applyCachePrimer(InputStream in)
               throws IOException

Prime the cache, from a set encoded earlier.

Parameters:

in - caching priming source; buffering is recommended; auto-closed

Throws:

IOException

See Also:

• DatabaseConfig.cachePriming(boolean)

newServer

Server newServer()
          throws IOException

Returns an object for enabling remote access into this database. As long as the server is still open, the JVM won't exit. Closing the database will also close the server.

If the database is configured with replication, remote access is already enabled, and so a server doesn't need to be created.

Throws:

IOException

See Also:

• connect

stats

DatabaseStats stats()

Returns a collection of database statistics.

flush

void flush()
    throws IOException

Flushes all committed transactions, but not durably. Transactions committed with no-flush effectively become no-sync durable.

When the database is replicated, the no-flush mode is identical to the no-sync mode. Calling this method on a replicated database has no effect.

Specified by:

flush in interface Flushable

Throws:

IOException

sync

void sync()
   throws IOException

Durably flushes all committed transactions. Transactions committed with no-flush and no-sync effectively become sync durable.

Throws:

IOException

checkpoint

void checkpoint()
         throws IOException

Durably sync and checkpoint all changes to the database. In addition to ensuring that all committed transactions are durable, checkpointing ensures that non-transactional modifications are durable. Checkpoints are performed automatically by a background thread, at a configurable rate.

Throws:

IOException

suspendCheckpoints

void suspendCheckpoints()

Temporarily suspend automatic checkpoints and wait for any in-progress checkpoint to complete. Suspend may be invoked multiple times, but each must be paired with a resume call to enable automatic checkpoints again.

Throws:

IllegalStateException - if suspended more than 2³¹ times

resumeCheckpoints

void resumeCheckpoints()

Resume automatic checkpoints after having been temporarily suspended.

Throws:

IllegalStateException - if resumed more than suspended

commitLock

Lock commitLock()

Returns the checkpoint commit lock, which can be held to prevent checkpoints from capturing a safe commit point. By holding the commit lock, multiple modifications can be made to the database atomically, even without using a transaction.

The commit lock should only ever be held briefly, because it prevents checkpoints from starting. Holding the commit lock can stall other threads trying to make modifications, if a checkpoint is trying to start. In addition, a thread holding the commit lock must not attempt to issue a checkpoint, because deadlock is possible.

compactFile

boolean compactFile(CompactionObserver observer,
 double target)
             throws IOException

Compacts the database by shrinking the database file. The compaction target is the desired file utilization, and it controls how much compaction should be performed. A target of 0.0 performs no compaction, and a value of 1.0 attempts to compact as much as possible.

If the compaction target cannot be met, the entire operation aborts. If the database is being concurrently modified, large compaction targets will likely never succeed. Although compacting by smaller amounts is more likely to succeed, the entire database must still be scanned. A minimum target of 0.5 is recommended for the compaction to be worth the effort.

Compaction requires some amount of free space for page movement, and so some free space might still linger following a massive compaction. More iterations are required to fully complete such a compaction. The first iteration might actually cause the file to grow slightly. This can be prevented by doing a less massive compaction first.

Parameters:

observer - optional observer; pass null for default

target - database file compaction target [0.0, 1.0]

Returns:

false if file compaction aborted

Throws:

IllegalArgumentException - if compaction target is out of bounds

IllegalStateException - if compaction is already in progress

IOException

verify

boolean verify(VerificationObserver observer)
        throws IOException

Verifies the integrity of the database and all indexes.

Parameters:

observer - optional observer; pass null for default

Returns:

true if verification passed

Throws:

IOException

isLeader

boolean isLeader()

Returns true if the database instance is currently the leader.

uponLeader

void uponLeader(Runnable acquired,
 Runnable lost)

Registers the given task to start in a separate thread when the database instance has become the leader. If already the leader, the task is started immediately. The task is started at most once per registration, and a second task is called when leadership is lost. It too is called in a separate thread, and at most once per registration. Note that if leadership is quickly lost, the second task might run before the first task.

Parameters:

acquired - called when leadership is acquired (can be null)

lost - called when leadership is lost (can be null)

failover

boolean failover()
          throws IOException

If the database instance is currently acting as a leader, attempt to give up leadership and become a replica. If the database is a replica, or if failover is successful, true is returned. When false is returned, the database is likely still the leader, either because the database isn't replicated, or because no replicas exist to failover to.

Throws:

IOException

close

default void close()
            throws IOException

Closes the database, ensuring durability of committed transactions. No checkpoint is performed by this method, and so non-transactional modifications can be lost.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

See Also:

• shutdown()

close

void close(Throwable cause)
    throws IOException

Closes the database after an unexpected failure. No checkpoint is performed by this method, and so non-transactional modifications can be lost.

Specified by:

close in interface CauseCloseable

Parameters:

cause - if non-null, delivers a panic event and future database accesses will rethrow the cause

Throws:

IOException

See Also:

• shutdown()

isClosed

boolean isClosed()

Returns true if database was explicitly closed, or if it was closed due to a panic.

shutdown

void shutdown()
       throws IOException

Cleanly closes the database, ensuring durability of all modifications. A checkpoint is issued first, and so a quick recovery is performed when the database is re-opened. As a side effect of shutting down, all extraneous files are deleted.

Throws:

IOException