Description
akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using Google Cloud Datastore.
It uses the official Google Java Dependency to talk with the datastore.
Scala 2.12 & 2.13, Java 8 & Java 11, akka 2.6.X are supported.
The plugin supports the following functionality:
GCP Datastore Akka Persistence Plugin alternatives and similar packages
Based on the "Database" category.
Alternatively, view GCP Datastore Akka Persistence Plugin alternatives based on common mentions on social networks and blogs.
-
Slick
Modern database query and access library for Scala. -
Elastic4s
A scala DSL / reactive client for Elasticsearch -
Quill
Compile-time Language Integrated Query for Scala -
doobie
Pure functional JDBC layer for Scala. -
PostgreSQL and MySQL async
Async database drivers to talk to PostgreSQL and MySQL in Scala. -
ScalikeJDBC
A tidy SQL-based DB access library for Scala developers. -
scala-redis
A Scala library for connecting to a redis server, with clustering support -
Phantom
Reactive type safe Scala driver for Apache Cassandra. -
ReactiveMongo
Reactive Scala Driver for MongoDB. -
rediscala
Non-blocking, Reactive Redis driver for Scala (with Sentinel support) -
Squeryl
A Scala DSL for talking with databases with minimum verbosity and maximum type safety. -
Casbah
Officially supported Scala driver for MongoDB -
Salat
ORM for MongoDB. A related Play-plugin is also available. -
mongo-scala-driver
A modern idiomatic MongoDB Scala Driver. -
gremlin-scala
Scala wrapper for Apache TinkerPop 3 Graph DSL -
Scanamo
A library to make using DynamoDB with Scala simpler and less error-prone. -
Activate
Pluggable object persistence in Scala. -
Scala ActiveRecord
ORM library for scala, inspired by ActiveRecord of Ruby on Rails. -
Relate
Lightweight, blazing-fast database access layer for Scala that abstracts the idiosyncricies of the JDBC while keeping complete control over the SQL. -
SwayDB
Type-safe, non-blocking, back-pressured key-value storage library for single/multiple disks & in-memory - www.SwayDB.io -
scredis
Non-blocking Redis client built on top of Akka IO (used by Livestream) -
Scala-Forklift
Type-safe database migration for Slick, Git, etc. -
AnormCypher
Neo4j Scala library based on Anorm in the Play Framework -
Troy
Type-safe and Schema-safe Scala wrapper for Cassandra driver -
Scruid
Scruid (Scala+Druid) is an open source library that allows you to compose Druid queries easily in Scala. -
rethink-scala
Scala Driver for RethinkDB -
neotypes
Pure functional driver for neo4j. -
Shade
Memcached client for Scala, based on Spymemcached -
longevity
A Persistence Framework for Scala and NoSQL with a Domain Driven Design Orientation -
Clickhouse-scala-client
Reactive client for Clickhouse -
scala-sql
Yet another SQL-based DB access library for scala language -
Tepkin
Reactive MongoDB Driver for Scala built on top of Akka IO and Akka Streams. -
Morpheus
Reactive type safe Scala Driver for MySQL/Postgres. -
CouchDB-Scala
Purely functional Scala client for CouchDB -
laserdisc
A Future-free, fs2 native pure FP Redis client -
ReactiveCouchbase
Reactive Scala Driver for Couchbase. Also includes a Play plug-in. An official plug-in is also in development. -
ScalaRelational
Type-Safe framework for defining, modifying, and querying SQL databases. -
ReactiveNeo
Reactive type-safe Scala driver for Neo4J. -
lucene4s
Light-weight convenience wrapper around Lucene to simplify complex tasks and add Scala sugar. -
Couchbase
Official Couchbase client for Scala. -
Memcontinuationed
Memcached client for Scala. -
d4s
"Dynamo DB Database done Scala way". A library that allows accessing the DynamoDB in a purely functional way. -
scala-migrations
Database migrations written in Scala -
neo4akka
Neo4j Scala client using Akka HTTP with compile-time query interpolation, case class support, true non-blocking IO, and much more. -
etcd4s
Scala etcd client implementing V3 APIs -
MapperDao
An ORM library for oracle, mysql, mssql, and postgresql
Get performance insights in less than 4 minutes
Do you think we are missing an alternative of GCP Datastore Akka Persistence Plugin or a related project?
README
GCP Datastore Akka Persistence Plugin
akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using Google Cloud Datastore. It uses the official Google Java Dependency to talk with the datastore.
Scala 2.12 & 2.13, Java 8 & Java 11, akka 2.6.X are supported.
The plugin supports the following functionality:
- serialization of events and snapshots with play-json
- peristence-query api
- custom serialization
Related Blogposts
- Using Google Datastore with akka-persistence
- Was ist Event Sourcing? (de)
- Wie lässt sich das Lesen beim Event Sourcing durch CQRS optimieren? (de)
Usage & Setup
Versions: The table below lists the versions and their main dependencies
Version to use | Scala 2.12 | Scala 2.13 | Scala 3 / Dotty | Akka | play-json | google-cloud-datastore |
---|---|---|---|---|---|---|
1.0.2 | ✓ | ✓ | ? | 2.6.x | 2.8.x | 1.102.x |
Dependency
You just need to add the following dependency to you sbt dependencies
libraryDependencies += "de.innfactory" %% "akka-persistence-gcp-datastore" % "X.Y.Z"
Configuration
Take a look at reference.conf under src/main/resources We forked the cqrs cassandra lightbend example with necessary changes for gcp-datastore (@ Demo Test Project available based on the CQRS Example from Lightbend https://github.com/innFactory/akka-persistence-gcp-datastore-example)
Add the following to your application.conf for a basic configuration:
akka {
# use google cloud datastore as journal and snapshot store
persistence {
journal {
plugin = "gcp-datastore-journal"
auto-start-journals = ["gcp-datastore-journal"]
}
snapshot-store {
plugin = "gcp-datastore-snapshot"
auto-start-snapshot-stores = ["gcp-datastore-snapshot"]
}
}
}
Datastore Configuration
- Google Cloud Project with Datastore or FireStore in Datastore mode enabled
Create a index.yml file with content bolow in the project that will use this plugin:
indexes: - kind: journal properties: - name: persistenceId - name: sequenceNr direction: desc - kind: snapshot properties: - name: persistenceId - name: timestamp direction: desc - kind: snapshot properties: - name: persistenceId - name: timestamp - kind: journal properties: - name: tagsKey - name: timestamp - kind: journal properties: - name: persistenceId - name: sequenceNr
index.yml
Open terminal and execute
gcloud app deploy index.yaml
This is telling the GCP Datastore to build indexes for the plugin based on the yaml file
Create a service account for read and write to datastore. Download the json and add it to the project
src/main/resources/datastore.json
Persistence query API
The plugin supports the Persistence query APi, mostly used in CQRS applications to transform/migrate the events from the write side to the read side.
The ReadJournal is retrieved via the akka.persistence.datastore.journal.read.DatastoreScaladslReadJournal
and akka.persistence.datastore.journal.read.DatastoreJavadslReadJournal
. There is also a DatastoreReadJournalProvider.
import akka.persistence.datastore.journal.read.DatastoreScaladslReadJournal
import akka.persistence.query.{ EventEnvelope, PersistenceQuery }
val system = ??? //ActorSystem akka-classic or akka-typed then system.toClassic is needed. see the example.
val readJournal =
PersistenceQuery(system).readJournalFor[DatastoreScaladslReadJournal]("gcp-datastore-query")
Supported Queries
All queries are live streams and they are not completed when they reaches the end of the currently stored events, but continue to push new events when new events are persisted.
eventsByTag
eventsByTags is used for retrieving events that were marked with a given tag.
eventsByPersistenceId
eventsByPersistenceId is used for retrieving events for a specific PersistentActor identified by its persistenceId
Testing
To test this plugin
(Source: https://cloud.google.com/datastore/docs/tools/datastore-emulator)
gcloud components install cloud-datastore-emulator
gcloud beta emulators datastore start --no-store-on-disk --consistency=1.0
Set Env Variable
DATASTORE_TESTHOST=http://<host>:<port>
of datastore emulatorExecute
sbt run
Before executing test reset datastore data:
curl -X POST http://<host>:<port>/reset
There is a shell script under .circle ci which runs all of these tests. cqrs tests are outsourced in the example project.
Contribution policy
Contributions via GitHub pull requests are gladly accepted from their original author. Along with any pull requests, please state that the contribution is your original work and that you license the work to the project under the project's open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project's open source license and warrant that you have the legal authority to do so.
Credits
innFactory GmbH is a lightbend partner from germany. We are experts for Apps, BigData & Cloud Computing. If you need help with your next project, feel free to ask for our support.
*Note that all licence references and agreements mentioned in the GCP Datastore Akka Persistence Plugin README section above
are relevant to that project's source code only.