« Changelog History


Scanamo v1.0.0-M11 Release Notes

Release Date: 2019-09-22 // over 4 years ago

  • Notable changes

    There are a couple of breaking changes that must happen now that we don't really care about binary/source compatibility.

    No more Symbol

    🔨 Scanamo used to heavily rely on symbols in its DSL: table.get('id -> 123). The syntactic sugar has always been problematic in IDEs, that's why it is going away in the next Scala version, and will probably be replaced with something like sym"id"; at which point it becomes a nuisance in our DSL. So I've removed them in favour of simple strings: table.get("id" -> 123). They get exactly the same treatment and are still quite limiting. In an upcoming version, AttributeName will be refactored to support the full breadth of expression supported by DynamoDB, and a custom string context will be provided, provisionally a"My.Array[$i].$SubProperty".

    Streaming in constant space

    This has been bugging me for long and many users have complained about the fact that table.scan will load the full result set in memory, even if one only needs to ever process a subset of results. There are now four new APIs in table: scanM, scanPaginatedM, queryM and queryPaginatedM which will load one subset at a time. The return type is slightly different than the other APIs: ScanamoOpsT[M, List[Either[DynamoReadError, T]]].

    The M type parameter identifies an underlying monad that has additional capabilities given by the MonoidK constraint: lazy streams such a ZStream from the ZIO library are one example, so you could write something along the lines of table.scanM[Stream[AmazonDynamoDBException, *]]. You can mix "regular" API calls with these ones in the same computation; all you have to do is lift them, for example:

    val ops = for { \_ \<- items.putAll(list.toSet).toFreeT[Iterant[IO, \*]] list \<- items.scanPaginatedM[Iterant[IO, \*]](1) } yield list

    In order to interpret your program, use the client.execT function which takes two parameters:

    • hoist: IO[AmazonDynamoDBException, *] ~> M which lifts an effect into the destination monad
    • op: ScanamoOpsT[M, A] your program

    I wrote some obvious versions of hoist for:

    • ScanamoZio.ToStream for ZStream (ZIO)
    • ScanamoCats.ToIterant for Iterant (Monix)
    • ScanamoCats.ToStream for Stream (FS2)

    The above example can be interpreted with client.execT(ScanamoCats.ToIterant)(ops).

    I haven't used this extensively yet, so any feedback on that new feature will be greatly appreciated.

    0️⃣ Removal of DynamoFormat[A].default

    DynamoFormat is a typeclass witnessing that your type can be encoded/decoded to/from a fancy JSON value. However, DynamoDB comes with some idiosyncratic restrictions wrt the values that it will accept or not in some APIs. For instance, empty strings are forbidden. These peculiar rules have nothing to do with the JSON encoding, they are an implementation detail that leaks into DynamoDB APIs. Besides, it only ever affected a very strict subset of all the types for which there is a DynamoFormat instance, all of which are defined in the library.

    0️⃣ The default member of DynamoFormat was created to solve that problem, where e.g. during decoding if a value is supposed to be a string and it is missing, we can produce the empty string in its stead. There is another way to do exactly that without having to pollute the API, and that's what I've done. You shouldn't notice any difference, as it is unlikely that default ever leaked into your code, but that simplification allows for future improvements.

    👌 Improvements to the retry policy for the Alpakka interpreter

    🚀 The previous release introduced the ability to control the behaviour of scanamo whenever an error occurs in DynamoDB, using the alpakka interpreter. While the DSL is sound, its interpretation was flawed in several ways 😅. These issues have all been addressed now.

    👍 Working on this, I realised other interpreters delegate a lot to the underlying DynamoDB client without much fanfare. I think we should not use these clients at all—they're just fancy HTTP clients—so that we can better control the behaviour in case of errors and be consistent across interpreters.

    ⚡️ Updates

    By contributors

    By @scala-steward

    • ⚡️ Update akka-stream-alpakka-dynamodb to 1.1.1 #478
    • ⚡️ Update cats-core, cats-free to 2.0.0 #503
    • ⚡️ Update cats-effect to 2.0.0 #504
    • ⚡️ Update fs2-core to 2.0.1
    • ⚡️ Update joda-time to 2.10.4 #512
    • ⚡️ Update monix to 3.0.0 #506
    • ⚡️ Update refined to 0.9.10 #508
    • ⚡️ Update sbt to 1.3.2 #513
    • 🚀 Update sbt-ci-release to 1.3.2 #510
    • ⚡️ Update sbt-doctest to 0.9.5 #437
    • ⚡️ Update sbt-scalafmt to 2.0.5 #509
    • ⚡️ Update sbt-scoverage to 1.6.0 #439
    • ⚡️ Update scalacheck-toolbox-datetime to 0.2.6 #487
    • ⚡️ Update scalafmt to 2.0.1 #483
    • ⚡️ Update scalatest to 3.0.8 #431
    • ⚡️ Update tut-plugin to 0.6.12 #427
    • ⚡️ Update zio, zio-streams to 1.0.0-RC12-1 #498
    • ⚡️ Update zio-interop-cats to 2.0.0.0-RC3 #499

    Coming next

    🚀 The next release will be the last one before a first release candidate. First?! Hmm, I may have lost count 😅

    • 👍 Better package organisation: the scanamo package makes implementation decisions on the user's behalf. Instead, I plan on moving the core library features into scanamo-core, the generic derivation into scanamo-generic and the interpreters into sub-packages as they are now.
    • Shapeless will be replaced with Magnolia for derivation; this should speed up compilation substantially. My initial experiment failed because I couldn't find a way to handle automatic derivation properly, but I found a solution for that.
    • Separate encoders and decoders, as many apps only require one of them. That will also give us the flexibility to make manual encoding a breeze, easy enough that people can avoid using scanamo-generic most of the time.
    • Condition expressions can be encoded using an initial encoding, rather than going through the current implicit machinery. This will make the API cleaner and also more scalable.
    • Some more API cleanup: some operations can return values, this is an opt-in feature of DynamoDB. At the moment operations like put make a decision that may not fit everyone, so I'm relaxing that constraint and generalising the API.
    • 📚 Documentation: the website publishing process is kind of broken and as a result its content has drifted from the most recent API changes. This has confused more than one user, me included. We need to fix this once and for all, so it doesn't happen again.