Scio v0.10.0-beta2 Release Notes
Release Date: 2020-11-19 // over 2 years ago-
🍱 🚀 Enhancements
- 👌 improve smb auto-splittability (#3485) @ClaireMcGinty
- ➕ Add RedisDoFn (#3461) @regadas
- ➕ Add byte size limit option to ElasticsearchIO writer (#1829) (#3437) @alexclare
- ➕ Add RedisType implicit error message (#3456) @regadas
- Override Parquet block size and GCS connector FADVISE (#3452) @nevillelyh
- ➕ Add multiple key range read for BigTableIO (#3442) @syodage
- 👍 Distributed HyperLogLog++ support with ZetaSketch (#3440) @syodage
🍱 🐛 Bug Fixes
- 🛠 Fix scio version warning in tests (#3489) @regadas
- 🛠 Fix getRecord should not throw on a missing field (#3490) @sfines-clgx
- 🛠 Fix logging in ApproxFilter (#3475) @martinbomio
- ⚡️ Update gRPC to 1.32.1 (#3467) @regadas
- ⏪ Revert read/write test param removal (#3451) @regadas
- ➕ Add projection & predicate in test ID for BigQueryStorage (#3466) @moscowart
- 🛠 fixup! fix doc error in Tap.scala (#3487) @regadas
📚 📗 Documentation
- 🛠 Fix README Scio logo path (#3468) @viktorjonsson
- Clarify smb auto-parallelism behavior in GH site (#3449) @ClaireMcGinty
- ➕ Added documentation for typedBigQuery (#3448) @jasmineytchen
🏗 🏗️ Build Improvements
🔨 🔧 Refactorings
- 📦 Rework
com.spotify.scio.transformspackage (#3454) @regadas - ➕ Add RedisDoFn (#3461) @regadas
- 🔨 Refactor sketching package (#3443) @syodage
⚡️ 🌱 Dependency Updates
- ⚡️ Update mysql-socket-factory to 1.2.0 (#3491) @scala-steward
- ⚡️ Update protobuf-java to 3.14.0 (#3476) @scala-steward
- ⚡️ Update elasticsearch to 7.10.0 (#3472) @regadas
- ⚡️ Update gRPC to 1.32.1 (#3467) @regadas
- ⚡️ Update scalatest to 3.2.3 (#3463) @scala-steward
- ⚡️ Update scalacheck to 1.15.1 (#3460) @scala-steward
- ⚡️ Update cassandra-all to 3.11.9 (#3453) @scala-steward
- ⚡️ Update scalacheck to 1.15.0 (#3444) @scala-steward
🚀 Contributors to this release
@ClaireMcGinty, @alexclare, @dependabot, @dependabot[bot], @jasmineytchen, @martinbomio, @moscowart, @nevillelyh, @regadas, @scala-steward, @sfines-clgx, @syodage and @viktorjonsson
Previous changes from v0.10.0-beta1
-
👀 See v0.10.0 Migration Guide for detailed instructions.
👌 Improvements
- ➕ Add scio-redis (#3386)
- ➕ Add RedisMutation support for RedisIO (#3331) (#3391)
- ➕ Add ApproxDistinctCount trait and two HLL++ beam extension implementations. (#3361)
- ➕ Add Predicate for SMB source, fix #3398 (#3402)
- 👍 Allow SortedBucketPreKeyedSink key validation with null keys (#3439)
- ➕ Add missing sortMergeGroupByKey API with custom parallelism (#3430)
- 🔀 Unblock MergeAndWriteBucketsSource while in progress (#3407)
- ➕ Add SColletion
withResourcefunctions (#3389) - 👷 Make BQ inherit labels from Beam/Dataflow job (#3375)
- Simplify Bigquery Format typeclass and bubble up Avro coder (#3401)
- 👌 Support $LATEST replace for BQ Table type (#3376)
- ↔ Integrate docs in CI (#3392)
- ➕ Add 0.10.0 migration guide (#3436)
- ➕ Add scalafix rules for 0.10 (#3432)
- 📄 Clarify extra.csv.CsvIO docs (#3435)
- ⚡️ Update scio.g8 steps
- ➕ Add install instructions for sbt. (#3433)
🐛 Bug Fixes
- Avoid Iterable backed by List when running locally (#3408)
- ✂ Remove Coder context bound from read (#3384)
- Avoid fallback Coder on GenericRecord read (#3385)
- ➕ Add missing pubsub coders to avoid fallback (#3382)
- ✂ Remove prompt before interp is created (#3404)
- 🛠 Fix repl windows character escape (#3405)
- Sanitize staged file path string (#3406)
- ✂ Remove unused fields in AvroIO (#3383)
- 🛠 Fix JavaAsyncLookupDoFn doc link (#3381)
⚡️ Dependency Updates
- ⚡️ Update Beam to 2.25.0 (#3438)
- ⚡️ Update sbt-avro to 3.2.0 (#3434)
- ⚡️ Update magnolify-avro, magnolify-bigtable, ... to 0.3.0 (#3431)
- ⚡️ Update elasticsearch-rest-client, ... to 7.9.3 (#3422)
- ⚡️ Update sbt-explicit-dependencies to 0.2.15 (#3417)
- ⚡️ Update joda-time to 2.10.8 (#3424)
- ⚡️ Update sbt-mima-plugin to 0.8.1 (#3414)
- ⚡️ Update sbt to 1.4.1 (#3415)
- ⚡️ Update scalafmt-core to 2.7.5 (#3410)
- ⚡️ Update mysql-connector-java to 8.0.22 (#3412)
- ⚡️ Update sbt-mdoc to 2.2.10 (#3413)
- ⚡️ Update junit to 4.13.1 (#3394)
- ⚡️ Update caffeine to 2.8.6 (#3395)
- ⬆️ Bump Featran to 0.7.0, re-enable failing TF tests (#3380)