All Versions
26
Latest Version
Avg Release Cycle
27 days
Latest Release
1225 days ago

Changelog History
Page 3

  • v0.7.2 Changes

    March 04, 2019

    "Ursus t. Ussuricus"

    ๐Ÿ”‹ Features

    • โšก๏ธ Update Beam to 2.10 (#1674, #1676)
    • Clearer Coder exceptions (#1672)
    • ๐Ÿ‘‰ Use new HadoopFormatIO (#1675)
    • โž• Add spanner MutationGroup coder (#1704)
    • โšก๏ธ Optimize CombineFn's (speeds up aggregate-, reduce-, and combine-based operations!) (#1699)
    • ๐Ÿ‘‰ Use list side input on cross product (#1691)
    • ๐Ÿ›  Fix DistinctBy serialization for Scala Classes (#1710, #1715)
    • โœ‚ Remove deprecation warning on tfRecordExampleFileWithSchema (#1714)
    • Cleanup around scio context (#1679)
    • ๐Ÿ”– Version bumps: cassandra-all -> 2.2.14 (#1677), 3.11.4 (#1678); Sparkey -> 3.0.0 (#1690), ES5 -> 5.6.15, ES6 -> 6.6.1 (#1700); tensorflow -> 1.13.1 (#1707); scalatest -> 3.0.6 (#1709); featran-* -> 0.3.0 (#1713)

    ๐Ÿ› Bug fixes

    • ๐Ÿ›  Fix Magnolia generated tree annotations removal to ensure Derived coders are serializable (#1673)
  • v0.7.1 Changes

    February 08, 2019

    "Taxidea Taxus"

    ๐Ÿ”‹ Features

    • ๐Ÿ†• New HashCode-based partitioning method for keyed SCollections (#1654)
    • ๐Ÿ†• New Coder for java.lang.ArrayList (#1649), and more space-efficient coders for small ADTs like Either and Try (#1652)
    • ๐Ÿ†• new BinaryIO output (#1663)
    • Simpler, clearer toString method for Coders (#1671)
    • ๐Ÿ“ฆ Custom Assertions for unit testing Coders added to scio-test package (#1642)
    • ๐Ÿ†• New SideMap and SideSet SideInput types, usable in hashFullOuterJoin, hashIntersectByKey, and hashFilter methods
    • Library version bumps: mysql-connector-java -> 8.0.15 (#1653), mysql-socket-factory -> 1.0.12 (#1627), protobuf-java -> 3.6.1 (#1633), hadoop-client -> 2.7.7 (#1634), jackson-module-scala -> 2.9.8 (#1632), parquet-avro -> 1.10.1 (#1648), kantan.csv -> 0.5.0 (#1647)

    ๐Ÿ› Bug fixes & Improvements

    • โšก๏ธ Optimized Bloom filter aggregations in sparse joins (#1644)
    • ๐Ÿ“ฆ Spanner-specific Coders repackaged from scio-core to scio-spanner (#1630)
    • ๐Ÿšš Fallback coder always uses Kryo (#1668) and RichCoderRegistry is removed (#1670)
  • v0.7.0 Changes

    January 18, 2019

    "Suricata suricatta"

    ๐Ÿ’ฅ Breaking changes

    • ๐Ÿ‘€ See v0.7.0 Migration Guide for detailed instructions
    • ๐Ÿ†• New Magnolia based Coders derivation replaces ClassTag and Kryo
    • ๐Ÿ†• New ScioIO replaces TestIO[T] to simplify IO implementation and stubbing in JobTest
    • โšก๏ธ Update dynamic file destination API #1305
    • โœ‚ Remove deprecated TensorFlow graph prediction method #1370
    • Object file IO is no longer backwards compatible due to coder changes
    • ๐Ÿ”จ Refactor bigquery client (#1439)

    ๐Ÿ”‹ Features

    ๐Ÿ› Bug fixes & Improvements

    • ๐Ÿ‘‰ Make PTransform names unique #1355 #1387
    • Fail for unknown args in ContextAndArgs.typed[T] (#1413)
    • ๐Ÿ›  Fix verifyNondeterministic exception in coders (#1418)
    • ๐Ÿ›  Fix BigQueryType on refined types (#1424)
    • ๐Ÿ›  Fix mergeAccumulators crash (#1428)
    • โœ… Set timestamp attribute in JobTest for PubSubIO (#1417)
    • Rework Coder's implicit not found message (again) (#1469)
    • ๐Ÿ›  Fix KryoRegistrar scope widenning (#1462)
    • ๐Ÿ‘‰ Make compression options in ExtractOps typed (#1449) (#1457)
    • โž• Add back BigQuery schema caching, regression of #1439 (#1458)
    • โœ… Register default file systems in Scio test context (fix #1455) (#1463)
    • 0๏ธโƒฃ Use coherent defaults accross IO (#1478)
    • ๐Ÿ›  Fix scio-repl to use refactored BigQuery client (#1459)
    • ๐Ÿ“œ Typed argument parsing is broken when name contains camelCase. (#1460)
    • Pubusb topic name was not being set for Messages (#1568)
    • ๐Ÿ›  Fix macro generated class directory (#1558)
    • ๐Ÿ›  Fix stack overflow when maxByKey is used with explicit ordering (#1560)
    • ๐Ÿ›  Fix id and timestamp attributes not being passed in saveAsPubsub (#1559)
    • ๐Ÿ›  Fix flatten type inference changing the coder context bound to an implicit parameter(#1551)
    • ๐Ÿ›  Fix: use CodeMaterializer in SideOutputCollections (#1548)
    • โš  Default to disabled warning on coders (#1588)
    • ๐Ÿ—„ Use alternative to deprecated write method (#1592)
    • ๐Ÿ“œ Simplify BigQueryType query method arg type parsing (#1585)
    • โž• Add rules for TextIO, AvroIO, PubsubIO and BigQueryIO (#1577)
    • #1587: Fix sideoutput potentialy missing coder (#1598)
    • โž• Add region to DataflowResult (#1479)
    • โœ‚ Remove unused autovalue dependency (#1575)
  • v0.7.0-beta3 Changes

    January 08, 2019

    ๐Ÿ› Bug fixes & Improvements

    • โš  Default to disabled warning on coders (#1588)
    • ๐Ÿ—„ Use alternative to deprecated write method (#1592)
    • ๐Ÿ“œ Simplify BigQueryType query method arg type parsing (#1585)
    • โž• Add rules for TextIO, AvroIO, PubsubIO and BigQueryIO (#1577)
    • #1587: Fix sideoutput potentialy missing coder (#1598)
    • โšก๏ธ Update beam-runners-direct-java, ... to 2.9.0 (#1580)
    • โšก๏ธ Update annoy4s to 0.8.0 (#1579)
    • โšก๏ธ Update zoltar-api, zoltar-tensorflow to 0.5.1 (#1578)
    • โšก๏ธ Update circe-core, circe-generic, ... to 0.11.0 (#1586)
    • โšก๏ธ Update guava to 25.1-jre (#1589)
    • โœ‚ Remove unused autovalue dependency (#1575)

    ๐Ÿ”‹ Features

    • โž• Add elasticsearch 6 (#1572)
    • โž• Add Numeric type support in scio-bigquery (#1599)
  • v0.7.0-beta2 Changes

    December 06, 2018

    ๐Ÿ› Bug Fixes

    • Pubusb topic name was not being set for Messages (#1568)
    • ๐Ÿ›  Fix macro generated class directory (#1558)
    • ๐Ÿ›  Fix stack overflow when maxByKey is used with explicit ordering (#1560)
    • ๐Ÿ›  Fix id and timestamp attributes not being passed in saveAsPubsub (#1559)
    • ๐Ÿ›  Fix flatten type inference changing the coder context bound to an implicit parameter(#1551)
    • ๐Ÿ›  Fix: use CodeMaterializer in SideOutputCollections (#1548)

    ๐Ÿ”‹ Features

  • v0.7.0-beta1 Changes

    December 06, 2018

    ๐Ÿ› Bug fixes

    • Rework Coder's implicit not found message (again) (#1469)
    • ๐Ÿ›  Fix KryoRegistrar scope widenning (#1462)
    • ๐Ÿ‘‰ Make compression options in ExtractOps typed (#1449) (#1457)
    • โž• Add back BigQuery schema caching, regression of #1439 (#1458)
    • โœ… Register default file systems in Scio test context (fix #1455) (#1463)
    • 0๏ธโƒฃ Use coherent defaults accross IO (#1478)
    • ๐Ÿ›  Fix scio-repl to use refactored BigQuery client (#1459)
    • ๐Ÿ“œ Typed argument parsing is broken when name contains camelCase. (#1460)

    ๐Ÿ”‹ Features

    • โž• Add Google Spanner package (#1491)
    • โž• Add BigQuery TimePartitioning support, fix #1419 (#1466)
    • โž• Add subscription function to PubSubAdmin(#1483)
    • โฌ†๏ธ Bump Beam to 2.8.0 (#1493)
    • โšก๏ธ Update dependencies (#1489)
    • โž• Add scalafix rules (#1435)(#1464)(#1474)(#1468)(#1470)
    • โž• Add BigQueryType typesafe args (#1476)
    • โž• Add region to DataflowResult (#1479)
    • ๐Ÿ”ฆ Expose transform function (#1492)(#1487)
    • ๐Ÿ‘ Allow creating DataflowResult from df Job (#1481)
    • โœ‚ Remove Future.failed in IOs (#1482)
    • โž• Add better error messages when missing sys.props (#1488)(#1461)