SwayDB

Persistent and in-memory key-value storage engine for JVM that scales on a single machine.

Description

SwayDB is a type-safe & non-blocking key-value storage library for single/multiple disks and in-memory storage.

It's an implementation of Log-structured merge-tree written in Scala with asynchronous Leveled Compaction based on push-pull strategy built on the Actor model.

It supports configurable Graph like file format for faster reads and has compression support for both memory & persistent databases.

Documentation: http://swaydb.io

Programming language: Scala

License: Apache License 2.0

Tags: Database     Tools     Type-Safe    

Latest version: v0.16.2

README

SwayDB

Persistent and in-memory key-value storage engine for the JVM aimed at high performance & resource efficiency.

Small footprint: around 7.1 MB jar size. No external core dependency (#307).

Scalable on a single machine: Distribute data to multiple local SSDs. Allocate single or multiple Threads for reads, caching & compaction.

Branches: This is the development branch. See master for stable version.

Status: Under testing & performance optimisations. See project status.

Documentation

Sponsors

Thank you JetBrains & JProfiler for full-featured open-source licences to their awesome development tools!

[Become a sponsor]

Overview

• Simple data types - Map, Set, Queue, SetMap & MultiMap with native Java and Scala collections support.
• Conditional updates using any pure JVM function - No query language.
• Atomic updates and inserts with Transaction API.
• Non-blocking core with configurable APIs for blocking, non-blocking and/or reactive use-cases.
• Single or multiple disks persistent, in-memory or eventually persistent.
• Streams - Async & sync forward and reverse data streaming/iteration.
• TTL - non-blocking, non-polling asynchronous auto expiring key-values.
• Range operations to update, remove & expire a range of key-values.
• Key only iterations (Lazily fetched values).
• Configurable compression with LZ4 & Snappy
• Configurable core internals to support custom workloads.
• Duplicate values elimination with compressDuplicateValues.

Use cases

Highly configurable to suit different workloads. Some known use-cases are:

• General key-value storage
• Message queues
• Time-series or Events data
• Caching
• Application logs
• Archiving data or cold storage with high file level compression

Quick start

• Java - Quick start
• Scala - Quick start
• Kotlin - Quick start

Contributing

Contributions are encouraged and welcomed. We are here to help and answer any questions.

Code of conduct - Be nice, welcoming, friendly & supportive of each other. Follow the Apache foundation's COC.

• Contributing to data management API

  • Build new data structures extending existing data structures. See MultiMap for example which is an extension of Map.
    • Graph
    • List
    • Geospatial
    • Logs
    • Observables
    • Full-text search
    • etc
  • Add support for external serialising, streaming & compression libraries.
  • Test and improve existing data structures.

• Contributing to core API

  • See issues labelled good first issue .

• Contributing to core internals

  • See code marked TODO.
  • Pick any small section to improve. You will find that everything is a function and can be unit-tested independently, so you can easily pick anything to work on.

Project status

Under testing & performance optimisations.

Your feedback and support is very important to get to production. Please get involved via chat, discussion, issues or by becoming a sponsor.

Future releases might not be backward compatible until we are production ready.

See tasks labelled Production release that are required before becoming production ready.

Related GitHub projects

• SwayDB.java.examples - Java examples demonstrating features and APIs.
• SwayDB.kotlin.examples - Kotlin examples demonstrating features and APIs.
• SwayDB.scala.examples - Scala examples demonstrating features and APIs.
• SwayDB.benchmark - Performance benchmarks.