All Versions
22
Latest Version
Avg Release Cycle
28 days
Latest Release
13 days ago

Changelog History
Page 1

  • v0.9.4

    September 10, 2020

    "Deletrius"

    ๐Ÿš€ There are no breaking changes in this release, but some were introduced with v0.9.0:

    ๐Ÿ‘€ See v0.9.0 Migration Guide for detailed instructions.

    ๐Ÿ‘Œ Improvements

    • โž• Add SCollection#filterNot (#3291)
    • ๐Ÿ‘Œ Improve filterValues doc (#3290)
    • โž• Add support for JDBC sharding by UUID encoded as string (#3307)
    • โž• Add optimised coder derivation for AnyVal (#3296)
    • ๐Ÿ‘Œ Support BigQuery Avro Format (#3221)
    • ๐Ÿ‘Œ Support sparkey compression, fix #3210 (#3295)
    • Warn if sparkey is bigger than memory, #3280
    • (fix #3278) Warn on chained .groupByKey.join (#3297)
    • [SMB] delete file early in NativeFileSorter (#3274)
    • ๐Ÿ”„ Change default SMB codec to Deflate to match Scio (#3247)
    • โž• Add java.time LocalDate, LocalDateTime, LocalTime, Period, Duration coders(#3238)

    ๐Ÿ› Bug Fixes

    • โœ‚ Remove duplicate ShardedSparkeyReader
    • ๐Ÿ‘‰ Use andThen for future side effect ops (#3275)
    • ๐Ÿ›  Fix Bigquery IT test
    • โž• Add state to exception message when pipeline is cancelled (#3270)
    • Avoid scala.jdk.CollectionConverters implicit import in Avro macro (#3250)
    • Avoid scala.jdk.CollectionConverters implicit import in Bigquery macro (#3240)
    • ๐Ÿ›  fix(avro-bq): added EnumSymbol case for matching avro types to BQ TableRow (#3226) (#3232)
    • โœ‚ Remove uneeded LowPriority implicit (#3239)
    • โœ‚ Remove coders deprecation warns (#3242)
    • ๐Ÿ‘‰ Use ## and support consistent Array hashCode (#3246)
    • ๐Ÿ‘Œ Improve SMB error handling (#3253)
    • โ†ช Workaround for sorter memory limit #3260 (#3269)

    โšก๏ธ Dependency Updates

    • โฌ†๏ธ Bump sparkey to 3.2.0
    • โœ‚ Remove unused imports (#3243)
    • โšก๏ธ Update case-app, case-app-annotations, ... to 2.0.4 (#3256)
    • โšก๏ธ Update cassandra-all to 3.11.8 (#3281)
    • โšก๏ธ Update cassandra-driver-core to 3.10.2 (#3276)
    • โšก๏ธ Update commons-io to 2.8.0 (#3310)
    • โšก๏ธ Update elasticsearch, ... to 7.9.1 (#3301)
    • โšก๏ธ Update elasticsearch, ... to 6.8.12 (#3264)
    • โšก๏ธ Update flink runner to 1.10.1 (#3249)
    • โšก๏ธ Update magnolia to 0.17.0 (#3262)
    • โšก๏ธ Update magnolify-avro, magnolify-bigtable, ... to 0.2.3 (#3263)
    • โšก๏ธ Update parquet-avro, parquet-column, ... to 1.11.1 (#3251)
    • โšก๏ธ Update protobuf-generic to 0.2.9 (#3227)
    • โšก๏ธ Update protobuf-java to 3.13.0 (#3257)
    • โšก๏ธ Update sbt-avro to 3.0.0 (#3252)
    • โšก๏ธ Update sbt-bloop to 1.4.4 (#3287)
    • โšก๏ธ Update sbt-buildinfo to 0.10.0 (#3245)
    • โšก๏ธ Update sbt-java-formatter to 0.6.0 (#3259)
    • โšก๏ธ Update sbt-jmh to 0.4.0 (#3288)
    • โšก๏ธ Update sbt-mdoc to 2.2.7 (#3311)
    • โšก๏ธ Update sbt-mima-plugin to 0.8.0 (#3305)
    • โšก๏ธ Update sbt-scalafix to 0.9.20 (#3298)
    • โšก๏ธ Update scalactic to 3.2.2 (#3271)
    • โšก๏ธ Update scalafmt-core to 2.7.0
    • โšก๏ธ Update scalatest to 3.2.2 (#3272)
    • โšก๏ธ Update transport to 6.8.12 (#3265)
    • ๐Ÿ‘‰ Use beam-runners-flink-1.10 when using BEAM_RUNNERS (#3303)
  • v0.9.3

    August 05, 2020

    "Petrificus Totalus"

    ๐Ÿš€ There are no breaking changes in this release, but some were introduced with v0.9.0:

    ๐Ÿ‘€ See v0.9.0 Migration Guide for detailed instructions.

    ๐Ÿ‘Œ Improvements

    • ๐Ÿ‘ Allow user-supplied filename prefix for smb writes/reads (#3215)
    • โ™ป๏ธ Refactor SortedBucketTransform into a BoundedSource + reuse merge logic (#3097)
    • โž• Add keyGroupFilter optimization to scio-smb (#3160)
    • โž• Add error message to BaseAsyncLookupDoFn preconditions check (#3176)
    • โž• Add Elasticsearch 5,6,7 add/update alias on multiple indices ops (#3134)
    • โž• Add initial update alias op to ES7(#2920)
    • โž• Add ScioContext#applyTransform (#3146)
    • ๐Ÿ‘ Allow SCollection#transform name override (#3142)
    • ๐Ÿ‘ Allow setting default name through SCollection#applyTransform (#3144)
    • ๐Ÿ“š Update 0.9 migration doc and add Bigquery Type read schema documentation(#3148)

    ๐Ÿ› Bug Fixes

    • ๐Ÿ“‡ AvroBucketMetadata should validate keyPath (fix #3038) (#3140)
    • ๐Ÿ‘ Allow union types in non leaf field for key (#3187)
    • ๐Ÿ›  Fix issue with union type as non-leaf field of smb key (#3193)
    • ๐Ÿ›  Fix ContextAndArgs#typed overloading issue (#3199)
    • ๐Ÿ›  Fix error propagation on Scala Future onSuccess callback (#3178)
    • ๐Ÿ›  Fix ByteBuffer should be readOnly (#3220)
    • ๐Ÿ›  Fix compiler warnings (#3183)
    • ๐Ÿ›  Fix JdbcShardedReadOptions.fetchSize description (#3209)
    • ๐Ÿ›  Fix FAQ typo (#3194)
    • ๐Ÿ›  Fix scalafix error in SortMergeBucketScioContextSyntax (#3158)
    • โž• Add scalafix ExplicitReturnType and ProcedureSyntax rules (#3179)
    • Cleanup a few more unused and unchecked params (#3223)
    • ๐Ÿ—„ Use GcpOptions#getWorkerZone instead of deprecated GcpOptions#getZone (#3224)
    • ๐Ÿ‘‰ Use raw coder in SCollection#applyKvTransform (#3171)
    • โž• Add raw beam coder materializer (#3164)
    • Avoid circular dep between SCollection and PCollectionWrapper (#3163)
    • โœ‚ Remove unused param of internal partitionFn (#3166)
    • โœ‚ Remove unused CoderRegistry (#3165)
    • โœ‚ Remove defunct scio-bench (#3150)
    • Reuse applyTransform (#3162)
    • ๐Ÿ‘‰ Make multijoin.py python3
    • ๐Ÿ‘‰ Use TextIO#withCompression (#3145)

    โšก๏ธ Dependency Updates

    • โšก๏ธ Update Beam SDK to 2.23.0 (#3197)
    • โšก๏ธ Update dependencies to be inline with 2.23.0 (#3225)
    • โšก๏ธ Update to scala 2.12.12 (#3157)
    • โšก๏ธ Update auto-value to 1.7.4 (#3147)
    • โšก๏ธ Update breeze to 1.1 (#3211)
    • โšก๏ธ Update cassandra-all to 3.11.7 (#3186)
    • โšก๏ธ Update cassandra-driver-core to 3.10.0 (#3195)
    • โšก๏ธ Update commons-lang3 to 3.11 (#3161)
    • โšก๏ธ Update commons-text to 1.9 (#3185)
    • โšก๏ธ Update contributing guidelines with current tools (#3149)
    • โšก๏ธ Update elasticsearch-rest-client, ... to 7.8.1 (#3192)
    • โšก๏ธ Update elasticsearch, ... to 6.8.11 (#3188)
    • โšก๏ธ Update jackson-module-scala to 2.10.5 (#3169)
    • โšก๏ธ Update jna to 5.6.0 (#3156)
    • โšก๏ธ Update magnolify to 0.2.2 (#3154)
    • โšก๏ธ Update mysql-connector-java to 8.0.21 (#3153)
    • โšก๏ธ Update pprint to 0.6.0 (#3203)
    • โšก๏ธ Update protobuf version to 3.11.4 (#3200)
    • โšก๏ธ Update sbt-scalafix to 0.9.18 (#3138)
    • โšก๏ธ Update sbt-sonatype to 3.9.4 (#3136)
    • โšก๏ธ Update scalafmt-core to 2.6.2 (#3139)
    • โšก๏ธ Update scalafmt-core to 2.6.3 (#3152)
    • โšก๏ธ Update scalafmt-core to 2.6.4 (#3167)
    • โšก๏ธ Update sparkey to 3.1.0 (#3204)
    • ๐Ÿ›  Fix conflicting gcsio dependency (#3180)
  • v0.9.2

    July 02, 2020

    "Alohomora"

    ๐Ÿš€ There are no breaking changes in this release, but some were introduced with v0.9.0:

    ๐Ÿ‘€ See v0.9.0 Migration Guide for detailed instructions.

    ๐Ÿ‘Œ Improvements

    • โž• Add more info to RateLimiterDoFn javadoc (#3054)
    • โž• Add OnTimeBehaviour window option (#3063)
    • โž• add small SMB sink optimizations (#3079)
    • โž• Add support for PipelineOptions in ContextAndArgs (#3121)
    • โž• Add support for read only ensureTable (#3114)
    • โž• Add support for sharded JDBC reads from tables (#3020)
    • โž• Add support for String/ByteString TF Bucket key types (#3098)
    • Ensures the future returned by addCallback will complete (#3099)
    • โšก๏ธ Update BigQueryType and AvroType compileTimeOnly msg (#3107)
    • Cleanup scio-repl classloader (#3118)
    • ๐ŸŽ‰ Initialize ScioContext on :reset (#3109)
    • โœ‚ Remove unnecessary cache of client interceptors (#3091)
    • โ™ป๏ธ Refactor scio-repl underlying ILoop (#3108)
    • ๐Ÿ‘‰ Make SMB source a splittable BoundedSource (#3005)
    • More Coder doc (#3025)
    • ๐Ÿ‘‰ Use a sampled subset of files to compute estimated SMB source size (#3037)
    • ๐Ÿ‘‰ Use IndexRecord upper bound (#3028)
    • ๐Ÿ‘‰ Use MonoidAggregator when available (#3051)
    • Wrap skewedJoin internals in transform (#3104)
    • โšก๏ธ Update deprecated .toIterator with .iterator (#3053)
    • โšก๏ธ Update to use testId instead of toString (#3115)

    ๐Ÿ› Bug Fixes

    • ๐Ÿ›  Fix additional exception wrappings for Flink (#3035)
    • ๐Ÿ›  Fix classpath issue when running from IntelliJ or Bloop (#3087)
    • ๐Ÿ›  Fix containValue serialization (#3128)
    • ๐Ÿ›  Fix doc site edit page link (#3062)
    • ๐Ÿ›  Fix javafmt from #3054
    • ๐Ÿ›  Fix repl-generated classfiles output directory (#3106)
    • ๐Ÿ›  Fix scio-repl scala specific repl settings (#3089)
    • ๐Ÿ›  fix site homepage formatting
    • ๐Ÿ›  fix SMB doc formatting (#3078)
    • ๐Ÿ›  Fix tuplecoders script
    • ๐Ÿ›  Fix: Add extra schema encoding step if coder already set (#3027)
    • ๐Ÿ›  Fix: propagate PipelineOptions in job test (#3026)
    • ๐Ÿ›  Fix: support mutable zeroValue in aggregate/combine/fold (#3033)
    • โž• Add missing Dataflow runner to scio-repl (#3081)

    โšก๏ธ Dependency Updates

    • โšก๏ธ Update auto-value, auto-value-annotations to 1.7.3 (#3042)
    • โšก๏ธ Update Beam to 2.22.0 (#3039)
    • โšก๏ธ Update caffeine to 2.8.5 (#3112)
    • โšก๏ธ Update case-app, case-app-annotations, ... to 2.0.3 (#3094)
    • โšก๏ธ Update elasticsearch-rest-client, ... to 7.8.0 (#3067)
    • โšก๏ธ Update magnolify-avro, magnolify-bigtable, ... to 0.2.1 (#3127)
    • โšก๏ธ Update scala to 2.13.3 (#3130)
    • โšก๏ธ Update scalactic to 3.2.0 (#3072)
    • โšก๏ธ Update scalatest to 3.2.0 (#3073)
    • โšก๏ธ Update transport to 6.8.10 (#3047)
  • v0.9.1

    June 03, 2020

    "Aberto"

    ๐Ÿš€ There are no breaking changes in this release, but some were introduced with v0.9.0:

    ๐Ÿ‘€ See v0.9.0 Migration Guide for detailed instructions.

    ๐Ÿ‘Œ Improvements

    • โž• Add option to disable colors when pretty printing (#2897)
    • โž• Add SCollection reify methods (#2987)
    • โž• Add scio-extra BigQuery saveAvroAsBigQuery IT test (#3004)
    • Unify sharded & non sharded sparkey write (#3012)
    • Write empty sparkey shards when no data (#3008)
    • โž• Add SCollection#groupMap and SCollection#groupMapReduce operations (#2996)
    • โž• Add BigQuery section to 0.9.0 migration guide (#2991)
    • โšก๏ธ Update default SMB sorter memory to 1GB (#2993)
    • SCollection.withWindowFn with upper bound type (#2975)
    • Implement SMB "least bucket replication" for SortedBucketSource (#2953)
    • โž• Add Grpc-specific Kryo serializers (fix #2963) (#2964)
    • Document bigquery Query type (#2950)
    • โœ‚ Remove unsused Coder and ClassTag (#2934)
    • โœ‚ Remove unused declared vals
    • ๐Ÿ†• New method mapKeys for PairScollection (#2922)
    • 0๏ธโƒฃ Use scala 2.13 as default (#2927)
    • ๐Ÿ†• New inLatePane matcher for SCollection (#2921)
    • โž• Add java.sql.Timestamp coder (#2907)

    ๐Ÿ› Bug Fixes

    • ๐Ÿ‘‰ Use scalaVersion semantic selector (#3022)
    • โž• Add -release 8 scalac option (#3006)
    • ๐Ÿ›  Fix scio-extra BigQuery IT test
    • ๐Ÿ‘‰ Use base64 encoding (#3003)
    • ๐Ÿ‘‰ Use BigQuery client project consistently (#2995)
    • Propagate targetParallelism from SortedBucketIO (#2997)
    • โœ‚ Remove mercator override (#2992)
    • ๐Ÿ›  Fixup Grpc Kryo serializers (#2967)
    • โช Revert "Fix: generate tree eagerly before checking for private constructors (#2846)" (#2943)
    • ๐Ÿ›  Fix tcnative and gcsio dep conflicts (#2948)
    • ๐Ÿ“‡ Rename BigQuery Beam Schema based op (#2932)
    • ๐Ÿ›  Fix creation of local sharded Sparkey files (#2930)
    • Don't require a List for Bigtable admin operations. (#2931)
    • ๐Ÿ“‡ Don't write SMB metadata if prior steps fail (fix #2895) (#2899)
    • ๐Ÿ›  Fix dependency conflicts (#2924)
    • ๐Ÿ›  Fix docs public datasets
    • ๐Ÿ›  Fix circe deprecation warn (#2910)
    • ๐Ÿ›  Fix: Parser with custom Formatter not always in scope (#2909)
    • ๐Ÿ›  Fix ScalaMatcher serialization in isEqualTo (#2904)
    • ๐Ÿ›  Fix: temp table location changed back to US (#2905)
    • ๐Ÿ›  Fix deprecation warns (#2966)

    โšก๏ธ Dependency Updates

    • โšก๏ธ Update kantan.csv to 0.6.1 (#3016)
    • โšก๏ธ Update caffeine to 2.8.4 (#2999)
    • โšก๏ธ Update magnolify-avro, magnolify-bigtable, ... to 0.2.0 (#2986)
    • โšก๏ธ Update caffeine to 2.8.3 (#2983)
    • โšก๏ธ Update elasticsearch 6/7 (#2974) (#2970)
    • โšก๏ธ Update auto-value, auto-value-annotations to 1.7.2 (#2973)
    • โšก๏ธ Update auto-service to 1.0-rc7 (#2972)
    • โšก๏ธ Update scala to 2.13.2 (#2951)
    • โšก๏ธ Update scalactic to 3.1.2 (#2959)
    • โšก๏ธ Update scalatest to 3.1.2 (#2961)
    • โšก๏ธ Update algebird-core, algebird-test to 0.13.7 (#2954)
    • โšก๏ธ Update auto-value, auto-value-annotations to 1.7.1 (#2940)
    • โšก๏ธ Update jackson-annotations, jackson-core, ... to 2.10.4 (#2938) (#2939)
    • โšก๏ธ Update magnolia to 0.16.0 (#2923)
    • โšก๏ธ Update cassandra-driver-core to 3.9.0 (#2925)
    • โšก๏ธ Update mysql-socket-factory to 1.0.16 (#2911)
    • โšก๏ธ Update mysql-connector-java to 8.0.20 (#2918)
    • โšก๏ธ Update joda-time to 2.10.6 (#2912)
    • โšก๏ธ Update featran-core, featran-scio, ... to 0.6.0 (#2896)
    • โšก๏ธ Update scala-collection-compat to 2.1.6 (#2894)
  • v0.9.0

    April 17, 2020

    "Furnunculus"

    ๐Ÿ’ฅ Breaking changes

    • ๐Ÿ‘€ See v0.9.0 Migration Guide for detailed instructions
    • โœ‚ Remove deprecated elasticsearch2 (#2800)
    • โœ‚ Remove deprecated cassandra2 (#2801)
    • โœ‚ Remove deprecated tensorflow saveAsTfExampleFile (#2798)
    • โœ‚ Remove toEither from ScioUtil (#2799)
    • โœ‚ Remove ReflectiveRecordIO (#2856)
    • โœ‚ Remove context close in favor of run (#2858)
    • โœ‚ Remove deprecated ScioContext Future references (#2859)
    • ๐Ÿ“ฆ Rework implicits/syntax for scio-extra bigquery package (#2844)
    • โœ‚ Remove implicit Coder requirement for .saveAsSortedBucket (#2839)
    • โฌ‡๏ธ Drop scala 2.11 support (#2619)
    • ๐Ÿ“œ Re-vamp Bloom filter and sparse-transforms (#2651)
    • โœ‚ Remove deprecated bigQuery, typedBigQuery and saveAsBigQuery (#2806)

    ๐Ÿ‘Œ Improvements

    • โž• Add scala 2.13 support (#2619)
    • โž• Add queryAsSource to BigQueryType (#2804)
    • ๐Ÿ—„ Deprecate BigQueryType query in favor of queryRaw (#2857)
    • ๐Ÿ‘‰ Make OptionCoder extends from AtomicCoder (#2882)
    • ๐Ÿ‘‰ Make iterable and traversable coders buffered (#2881)
    • ๐Ÿ‘ Better support for alternative runners in tests (#2877)
    • ๐Ÿ‘‰ Use UUID in SMB temp directory (#2849)
    • Reuse ApproxFilter (#2817)
    • ๐Ÿ‘Œ Support metadata in AvroFileOperations (fix #2832) (#2834)
    • โž• Add --help command line support for custom PipelineOptions (#2840)(#2843)
    • โž• Add covary method to lift SCollection to the specified type (#2808)
    • โœ… Customize equality in unit tests and better failure message (#2733)
    • โž• Add more convinience methods that support default transform names (#2805)

    ๐Ÿ› Bug Fixes

    • ๐Ÿ›  Fix create scio-spanner it clients lazy (#2889)
    • ๐Ÿ›  Fix generate tree eagerly before checking for private constructors (#2846)
    • ๐Ÿ›  Fix missing-bucket case when Sink collection is empty (#2869)
    • โœ‚ Remove uneeded caffeine dep in scio-bigquery (#2861)
    • ๐Ÿ›  Fix Sharded Sparkey string hashing behaviour for strings longer than one character. (#2826)
    • โž• Add magnolify BigtableType usage examples to scio-examples #2789 (#2816)
    • ๐Ÿ‘ท Check jobReference.location for query location (#2845)
    • ๐Ÿ›  Fix NPE in BaseAsyncLookupDoFn.Try#hashCode() (#2841)
    • ๐Ÿ›  Fix: cancel job on waitUntilFinish timeout (#2823)
    • ๐Ÿ›  Fix: full camelCase typed args support (#2777)

    โšก๏ธ Dependency Updates

    • โšก๏ธ Update magnolify to 0.1.7
    • โšก๏ธ Update magnolia to 0.14.5 (#2886)
    • โšก๏ธ Update beam-runners-core-construction-java, ... to 2.20.0 (#2887)
    • โšก๏ธ Update scala-collection-compat to 2.1.5 (#2885)
    • โšก๏ธ Update gcs-connector to hadoop2-2.1.2 (#2842)
    • โšก๏ธ Update algebra to 2.0.1 (#2821)
    • โšก๏ธ Update es6 transport to 6.8.8 (#2830)
    • โšก๏ธ Update es7 elasticsearch-rest-client, ... to 7.6.2 (#2829)
    • โšก๏ธ Update cats-kernel to 2.1.1 (#2822)
    • โšก๏ธ Update PPrint to 0.5.9 (#2793)
  • v0.8.4

    March 30, 2020

    "Expecto Patronum"

    ๐Ÿ”‹ Features

    • ๐ŸŽ‰ Initial flink runner support #2785
    • โž• Add support for default transform names #2760 #2803
    • โž• Add fromStorage doc #2792

    ๐Ÿ› Bug fixes & improvements

    • ๐Ÿ›  Fix offset in SlidingWindows #2790
    • โฌ‡๏ธ Reduce Consumer impl leak in SortedBucketTransform (#2795)
    • โช Revert "make CoGbkResult Iterable slightly lazy (#2749)" #2791
  • v0.8.3

    March 19, 2020

    "Draconifors"

    ๐Ÿ”‹ Features

    • โž• Add CsvIO (#2701)
    • Implement hashSubtractByKey (#2769)
    • โž• Add directory treatment and compression option to readFiles (#2756)
    • โž• Add FileNaming and prefix support to BinaryIO (#2759)
    • โž• Add Header and Footer support to TextIO (#2758)
    • โž• Add GenericJson coder (#2742)
    • โž• Add ProtobufUtil for Message<->Avro conversions (#2743)

    ๐Ÿ› Bug fixes & improvements

    • โž• Add schema cache support in BigQueryType (#2775)
    • โšก๏ธ Optimize KeyGroupIterator (#2779)
    • ๐Ÿ‘‰ Make CoGbkResult Iterable slightly lazy (#2749)
    • ๐Ÿ‘‰ Make java Enum coder deterministic #2761 #2764
    • ๐Ÿ›  Fix saveAsTypedBigQuery deprecation message (#2763)
    • Patch SortValues doFn to avoid re-enconding bytes (#2748)
    • ๐Ÿ›  Fix: remove redundant withExtendedErrorInfo in dynamic BigQuery (#2744)
  • v0.8.2

    March 02, 2020

    "Capacious Extremis"

    ๐Ÿ”‹ Features

    • โšก๏ธ Update Beam to 2.19.0 (#2665)
    • โž• Add SCollection#tap (#2713)
    • โž• Add SortedBucketTransform (#2693)
    • โž• Add API bindings for SortedBucketTransform (#2712)
    • โž• Add counters to SMB read transforms (#2704)
    • โž• Add ParquetExampleIO (#2696)
    • ๐Ÿ‘ Allow setting MIME type on RemoteFileUtil.upload (#2711)
    • โž• Add BigQuery macro Avro Schema and toAvro support (#2646)

    ๐Ÿ› Bug fixes & improvements

    • โšก๏ธ Ignore updates on google libs that we need to keep in sync with beam (#2728)
    • โž• Add all scio compile deps (#2634)
    • โฌ†๏ธ Bump circe to 0.13.0 (#2717)
    • ๐Ÿ›  Fix BigQuery Storage default params (#2692)
    • ๐Ÿ›  Fix: TableRow#getRecord returns Map (#2710)
    • ๐Ÿ›  Fix: eval coder eagerly for sealead traits (#2699)
    • โž• Add readFiles alternative to deprecated readAll (#2697)
    • โšก๏ธ Minor update topByKey comments (#2695)
    • โœ‚ Remove duplicate stage files (#2678)
    • ๐Ÿ‘‰ Make files to stage list mutable (#2676)
    • ๐Ÿ‘‰ Make Coder[Set] throw NonDeterministicException (#2674)
    • Prevent sharded-sparkey shards from having negative indices. (#2667)
    • โฌ‡๏ธ Reduce SMB Dataflow graph size for many-bucketed joins (#2666)
    • โž• Add test for #2649, Avro map field in SMB sink (#2664)
    • ๐Ÿ‘ฎ Force GenericRecord writer in SMB Avro, fix 2649 (#2650)
    • ๐Ÿ›  Fix: Add missing toAvro to BigQueryType (#2657)
    • โœ‚ Remove patched bigquery rows iterator (#2647)
  • v0.8.1

    January 31, 2020

    "Bombarda Maxima"

    ๐Ÿ”‹ Features

    • โšก๏ธ Update Beam to 2.18 (#2631)
    • โž• Add BigQuery Geography support (#2620)
    • โž• Add simple sorter method (#2542)
    • โž• Add Avro BigQuery logical types conversion (#2602)

    ๐Ÿ› Bug fixes & improvements

    • โž• Add Map/MultiMap singleton side input, fix #2628 (#2629)
    • ๐Ÿ›  Fix ClassCastEx in PubSubIO (#2583)
    • ๐Ÿ›  Fix sharded Sparkey side input with missing values. (#2617)
    • โž• Add specialized coders for Elasticsearch, fix #2626 (#2637)
    • ๐Ÿ‘‰ Make KryoAtomicCoder.instanceId an incrementing integer, not a UUID (#2589)
    • ๐Ÿ›  Fix BigQuery caching, by using dry-run when extracting tables (#2613)
    • ๐Ÿ›  Fix recursive coders (#2605)
    • Avoid NPE when finding rt.jar under jdk11 (#2610)
    • โœ… More tests on Coders (#2639)
    • ๐Ÿคก Consolidate IT mock data generation (#2624)
    • ๐Ÿ›  Fix sql script query generator
    • Avro Message record from scio-schemas could hide protobuf Message (#2609)
    • ๐Ÿ“‡ Rename metricsLocation option parameter (#2603)
    • โž• Add comment on scio-extra support level
    • โž• Add test case for avro conversion with bytes (#2594)
    • โž• Add lisp-case arg parsing removal (#2592) (#2221)
    • โœ‚ Remove unused ClassTag (#2590)
    • ๐Ÿ›  Fix BigQuery deprecated method message (#2586)
    • โž• Add scalafmt to scalafix rules (#2491), rename join names (#2455), rename named args: that to rhs (#2468)
    • ๐Ÿ›  Fix BigQuery deprecated method message (#2586)
    • โž• Add elasticsearch example (#2388)
    • โœ‚ Remove mdoc annotation for macro-based Parquet.md examples (#2581)
    • โšก๏ธ Update doc and migration guides (#2585)
    • ๐Ÿ‘‰ Use weakTypeOf in scio-bigquery (#2641)
    • โฌ‡๏ธ Drop FastEval in scio-sql (#2622)
  • v0.8.0

    July 16, 2019

    ๐Ÿ”‹ Features

    • โšก๏ธ Update beam to 2.13.0 (#1952)
    • โž• Add partitionToMap function (#1968)
    • ๐Ÿ†• New async DoFns (#1971)
    • ๐Ÿ”ฆ Expose Beam's AvroIO.parseGenericRecords (#1973)(#2051)
    • โž• Add batchByKey function (#2009)
    • โž• Add elasticsearch7 module (#2004)
    • โž• Add support for recursive Coders (#2032)(#2048)
    • โž• Adding filterValues function (#1964)

    ๐Ÿ› Bug Fixes & Improvements

    • ๐Ÿ›  Fix equality for mutable bf sparse instances (#1972)
    • โž• Add bq not implemented tap when write append (#2042)
    • ๐Ÿ›  Fix schema inference from BigQueryType (#2027)
    • Explicitly name the transform in close() that initializes counters (#2047)
    • Rework ClosedScioContext into ScioRunner (#2036)(#2055)
    • โž• Add incorrect table aliases hint (#2053)
    • ๐Ÿ‘ Allow app/job name override in scio-repl (#2046)
    • ๐Ÿ‘‰ Use BigtableOptions param for BigTable utils (#2003)
    • ๐Ÿ‘ป InMemorySink will now create an empty collection in the cache so empty SCollections also return an iterable instead of throwing an exception (#2039)
    • โž• Add WrappedArray coder (#1945)
    • ๐Ÿ›  Fix sql interpolator tag index (#1961)
    • ๐Ÿ“œ Use composite transform for sparse transforms (#1947)
    • ๐Ÿ›  Fix benchmarks (#1976)(#2005)
    • ๐Ÿ‘ท Set location when fetching Job in JobOps (#1966)
    • Cleanup materialize and set coder (#2016)
    • โ™ป๏ธ Refactor applyPerKey (#2011)(#1978)
    • ๐Ÿ›  Fix BigtableDoFn (#2001)
    • ๐Ÿ‘Œ Support passing Assertions directly to JobTest output (#1993)(#1995)
    • โšก๏ธ Update docs (#2012)
    • โž• Add deprecation dates to Apache-Beam.md
    • โœ‚ Remove deprecation warnings (#2041)(#2045)
    • โœ‚ Remove uneed implicit context and duplicated logger (#2018)
    • ๐Ÿš€ Warn only about latest stable release (non prerelease, draft) (#1980)
    • โšก๏ธ Dependency updates (#1962)(#1960)(#1956)(#1957)(#1948)(#1965)(#1990)(#2023)(#2033)(#1977)(#1998(#2052)