Scio v0.10.0-beta2 Release Notes
Release Date: 2020-11-19 // over 4 years ago-
๐ฑ ๐ Enhancements
- ๐ improve smb auto-splittability (#3485) @ClaireMcGinty
- โ Add RedisDoFn (#3461) @regadas
- โ Add byte size limit option to ElasticsearchIO writer (#1829) (#3437) @alexclare
- โ Add RedisType implicit error message (#3456) @regadas
- Override Parquet block size and GCS connector FADVISE (#3452) @nevillelyh
- โ Add multiple key range read for BigTableIO (#3442) @syodage
- ๐ Distributed HyperLogLog++ support with ZetaSketch (#3440) @syodage
๐ฑ ๐ Bug Fixes
- ๐ Fix scio version warning in tests (#3489) @regadas
- ๐ Fix getRecord should not throw on a missing field (#3490) @sfines-clgx
- ๐ Fix logging in ApproxFilter (#3475) @martinbomio
- โก๏ธ Update gRPC to 1.32.1 (#3467) @regadas
- โช Revert read/write test param removal (#3451) @regadas
- โ Add projection & predicate in test ID for BigQueryStorage (#3466) @moscowart
- ๐ fixup! fix doc error in Tap.scala (#3487) @regadas
๐ ๐ Documentation
- ๐ Fix README Scio logo path (#3468) @viktorjonsson
- Clarify smb auto-parallelism behavior in GH site (#3449) @ClaireMcGinty
- โ Added documentation for typedBigQuery (#3448) @jasmineytchen
๐ ๐๏ธ Build Improvements
๐จ ๐ง Refactorings
- ๐ฆ Rework
com.spotify.scio.transforms
package (#3454) @regadas - โ Add RedisDoFn (#3461) @regadas
- ๐จ Refactor sketching package (#3443) @syodage
โก๏ธ ๐ฑ Dependency Updates
- โก๏ธ Update mysql-socket-factory to 1.2.0 (#3491) @scala-steward
- โก๏ธ Update protobuf-java to 3.14.0 (#3476) @scala-steward
- โก๏ธ Update elasticsearch to 7.10.0 (#3472) @regadas
- โก๏ธ Update gRPC to 1.32.1 (#3467) @regadas
- โก๏ธ Update scalatest to 3.2.3 (#3463) @scala-steward
- โก๏ธ Update scalacheck to 1.15.1 (#3460) @scala-steward
- โก๏ธ Update cassandra-all to 3.11.9 (#3453) @scala-steward
- โก๏ธ Update scalacheck to 1.15.0 (#3444) @scala-steward
๐ Contributors to this release
@ClaireMcGinty, @alexclare, @dependabot, @dependabot[bot], @jasmineytchen, @martinbomio, @moscowart, @nevillelyh, @regadas, @scala-steward, @sfines-clgx, @syodage and @viktorjonsson
Previous changes from v0.10.0-beta1
-
๐ See v0.10.0 Migration Guide for detailed instructions.
๐ Improvements
- โ Add scio-redis (#3386)
- โ Add RedisMutation support for RedisIO (#3331) (#3391)
- โ Add ApproxDistinctCount trait and two HLL++ beam extension implementations. (#3361)
- โ Add Predicate for SMB source, fix #3398 (#3402)
- ๐ Allow SortedBucketPreKeyedSink key validation with null keys (#3439)
- โ Add missing sortMergeGroupByKey API with custom parallelism (#3430)
- ๐ Unblock MergeAndWriteBucketsSource while in progress (#3407)
- โ Add SColletion
withResource
functions (#3389) - ๐ท Make BQ inherit labels from Beam/Dataflow job (#3375)
- Simplify Bigquery Format typeclass and bubble up Avro coder (#3401)
- ๐ Support $LATEST replace for BQ Table type (#3376)
- โ Integrate docs in CI (#3392)
- โ Add 0.10.0 migration guide (#3436)
- โ Add scalafix rules for 0.10 (#3432)
- ๐ Clarify extra.csv.CsvIO docs (#3435)
- โก๏ธ Update scio.g8 steps
- โ Add install instructions for sbt. (#3433)
๐ Bug Fixes
- Avoid Iterable backed by List when running locally (#3408)
- โ Remove Coder context bound from read (#3384)
- Avoid fallback Coder on GenericRecord read (#3385)
- โ Add missing pubsub coders to avoid fallback (#3382)
- โ Remove prompt before interp is created (#3404)
- ๐ Fix repl windows character escape (#3405)
- Sanitize staged file path string (#3406)
- โ Remove unused fields in AvroIO (#3383)
- ๐ Fix JavaAsyncLookupDoFn doc link (#3381)
โก๏ธ Dependency Updates
- โก๏ธ Update Beam to 2.25.0 (#3438)
- โก๏ธ Update sbt-avro to 3.2.0 (#3434)
- โก๏ธ Update magnolify-avro, magnolify-bigtable, ... to 0.3.0 (#3431)
- โก๏ธ Update elasticsearch-rest-client, ... to 7.9.3 (#3422)
- โก๏ธ Update sbt-explicit-dependencies to 0.2.15 (#3417)
- โก๏ธ Update joda-time to 2.10.8 (#3424)
- โก๏ธ Update sbt-mima-plugin to 0.8.1 (#3414)
- โก๏ธ Update sbt to 1.4.1 (#3415)
- โก๏ธ Update scalafmt-core to 2.7.5 (#3410)
- โก๏ธ Update mysql-connector-java to 8.0.22 (#3412)
- โก๏ธ Update sbt-mdoc to 2.2.10 (#3413)
- โก๏ธ Update junit to 4.13.1 (#3394)
- โก๏ธ Update caffeine to 2.8.6 (#3395)
- โฌ๏ธ Bump Featran to 0.7.0, re-enable failing TF tests (#3380)