« Changelog History


DynaML v1.5.3 Release Notes

Release Date: 2018-11-20 // over 5 years ago

  • ➕ Additions

    Data Set API

    The DataSet family of classes helps the user to create and transform potentially large number of data instances.
    👉 Users can create and perform complex transformations on data sets, using the DataPipe API or simple Scala functions.

    import \_root\_.io.github.mandar2812.dynaml.probability.\_import \_root\_.io.github.mandar2812.dynaml.pipes.\_import io.github.mandar2812.dynaml.tensorflow.\_val random\_numbers = GaussianRV(0.0, 1.0) :\* GaussianRV(1.0, 2.0) //Create a data set.val dataset1 = dtfdata.dataset(random\_numbers.iid(10000).draw) val filter\_gr\_zero = DataPipe[(Double, Double), Boolean]( c =\> c.\_1 \> 0d && c.\_2 \> 0d) //Filter elementsval data\_gr\_zero = dataset1.filter(filter\_gr\_zero) val abs\_func: (Double, Double) =\> (Double, Double) = (c: (Double, Double)) =\> (math.abs(c.\_1), math.abs(c.\_2)) //Map elementsval data\_abs= dataset1.map(abs\_func)

    Find out more about the DataSet API and its capabilities in the user guide.

    Tensorflow Integration

    📦 Package dynaml.tensorflow

    Batch Normalisation

    Batch normalisation is used to standardize activations of convolutional layers and
    to speed up training of deep neural nets.

    Usage

    import io.github.mandar2812.dynaml.tensorflow.\_val bn = dtflearn.batch\_norm("BatchNorm1")

    Inception v2

    The Inception architecture, proposed by Google is an important
    building block of convolutional neural network architectures used in vision applications.

    📄 inception

    In a subsequent paper, the authors introduced optimizations in the Inception
    architecture, known colloquially as Inception v2.

    In Inception v2, larger convolutions (i.e. 3 x 3 and 5 x 5) are implemented in a factorized manner
    to reduce the number of parameters to be learned. For example the 3 x 3 convolution is expressed as a
    combination of 1 x 3 and 3 x 1 convolutions.

    📄 inception

    Similarly the 5 x 5 convolutions can be expressed a combination of two 3 x 3 convolutions

    📄 inception2

    DynaML now offers the Inception cell as a computational layer.

    Usage

    import io.github.mandar2812.dynaml.pipes.\_import io.github.mandar2812.dynaml.tensorflow.\_import org.platanios.tensorflow.api.\_//Create an RELU activation, given a string name/identifier.val relu\_act = DataPipe(tf.learn.ReLU(\_)) //Learn 10 filters in each branch of the inception cellval filters = Seq(10, 10, 10, 10) val inception\_cell = dtflearn.inception\_unit( channels = 3, num\_filters = filters, relu\_act, //Apply batch normalisation after each convolution use\_batch\_norm = true)(layer\_index = 1)

    Dynamical Systems: Continuous Time RNN

    Continuous time recurrent neural networks (CTRNN) are an important class of recurrent neural networks. They enable
    the modelling of non-linear and potentially complex dynamical systems of multiple variables, with feedback.

    ➕ Added CTRNN layer: dtflearn.ctrnn

    ➕ Added CTRNN layer with inferable time step: dtflearn.dctrnn.

    ➕ Added a projection layer for CTRNN based models dtflearn.ts_linear.

    Training Stopping Criteria

    Create common and simple training stop criteria such as.

    Stop after fixed number of iterations dtflearn.max_iter_stop(100000)

    Stop after change in value of loss goes below a threshold. dtflearn.abs_loss_change_stop(0.0001)

    Stop after change in relative value of loss goes below a threshold. dtflearn.rel_loss_change_stop(0.001)

    🏗 Neural Network Building Blocks

    • Added helper method dtlearn.build_tf_model() for training tensorflow models/estimators.

    Usage

    import io.github.mandar2812.dynaml.tensorflow.\_import org.platanios.tensorflow.api.\_import org.platanios.tensorflow.data.image.MNISTLoaderimport ammonite.ops.\_val tempdir = home/"tmp"val dataSet = MNISTLoader.load( java.nio.file.Paths.get(tempdir.toString()) )val trainImages = tf.data.TensorSlicesDataset(dataSet.trainImages)val trainLabels = tf.data.TensorSlicesDataset(dataSet.trainLabels)val trainData = trainImages.zip(trainLabels) .repeat() .shuffle(10000) .batch(256) .prefetch(10)// Create the MLP model.val input = tf.learn.Input( UINT8, Shape( -1, dataSet.trainImages.shape(1), dataSet.trainImages.shape(2)) )val trainInput = tf.learn.Input(UINT8, Shape(-1))val architecture = tf.learn.Flatten("Input/Flatten") \>\> tf.learn.Cast("Input/Cast", FLOAT32) \>\> tf.learn.Linear("Layer\_0/Linear", 128) \>\> tf.learn.ReLU("Layer\_0/ReLU", 0.1f) \>\> tf.learn.Linear("Layer\_1/Linear", 64) \>\> tf.learn.ReLU("Layer\_1/ReLU", 0.1f) \>\> tf.learn.Linear("Layer\_2/Linear", 32) \>\> tf.learn.ReLU("Layer\_2/ReLU", 0.1f) \>\> tf.learn.Linear("OutputLayer/Linear", 10)val trainingInputLayer = tf.learn.Cast("TrainInput/Cast", INT64)val loss = tf.learn.SparseSoftmaxCrossEntropy("Loss/CrossEntropy") \>\> tf.learn.Mean("Loss/Mean") \>\> tf.learn.ScalarSummary("Loss/Summary", "Loss")val optimizer = tf.train.AdaGrad(0.1)// Directory in which to save summaries and checkpointsval summariesDir = java.nio.file.Paths.get( (tempdir/"mnist\_summaries").toString() )val (model, estimator) = dtflearn.build\_tf\_model( architecture, input, trainInput, trainingInputLayer, loss, optimizer, summariesDir, dtflearn.max\_iter\_stop(1000), 100, 100, 100)(trainData)
    • 🏗 Build feedforward layers and feedforward layer stacks easier.

    Usage

    import io.github.mandar2812.dynaml.tensorflow.\_import org.platanios.tensorflow.api.\_//Create a single feedforward layerval layer = dtflearn.feedforward(num\_units = 10, useBias = true)(id = 1)//Create a stack of feedforward layersval net\_layer\_sizes = Seq(10, 5, 3) val stack = dtflearn.feedforward\_stack( (i: Int) =\> dtflearn.Phi("Act\_"+i), FLOAT64)( net\_layer\_sizes)

    3D Graphics

    📦 Package dynaml.graphics

    Create 3d plots of surfaces, for a use case, see the jzydemo.sc and tf_wave_pde.sc

    Library Organisation

    • ✂ Removed the dynaml-notebook module.

    🐛 Bug Fixes

    • 🛠 Fixed bug related to scalar method of VectorField, innerProdDouble and other inner product implementations.

    👌 Improvements and Upgrades

    • ⬆️ Bumped up Ammonite version to 1.1.0
    • RegressionMetrics and RegressionMetricsTF now also compute Spearman rank correlation as
      🐎 one of the performance metrics.

Previous changes from v1.5.3-beta.2

  • ➕ Additions

    3D Graphics

    📦 Package dynaml.graphics

    Create 3d plots of surfaces, for a use case, see the jzydemo.sc and tf_wave_pde.sc

    Tensorflow Utilities

    📦 Package dynaml.tensorflow

    Training Stopping Criteria

    Create common and simple training stop criteria such as.

    • Stop after fixed number of iterations dtflearn.max_iter_stop(100000)
    • Stop after change in value of loss goes below a threshold. dtflearn.abs_loss_change_stop(0.0001)
    • Stop after change in relative value of loss goes below a threshold. dtflearn.rel_loss_change_stop(0.001)

    🏗 Neural Network Building Blocks

    • Added helper method dtlearn.build_tf_model() for training tensorflow models/estimators.

    Usage

    val dataSet = MNISTLoader.load(java.nio.file.Paths.get(tempdir.toString())) val trainImages = tf.data.TensorSlicesDataset(dataSet.trainImages) val trainLabels = tf.data.TensorSlicesDataset(dataSet.trainLabels) val trainData = trainImages.zip(trainLabels) .repeat() .shuffle(10000) .batch(256) .prefetch(10) // Create the MLP model.val input = tf.learn.Input(UINT8, Shape(-1, dataSet.trainImages.shape(1), dataSet.trainImages.shape(2))) val trainInput = tf.learn.Input(UINT8, Shape(-1)) val architecture = tf.learn.Flatten("Input/Flatten") \>\> tf.learn.Cast("Input/Cast", FLOAT32) \>\> tf.learn.Linear("Layer\_0/Linear", 128) \>\> tf.learn.ReLU("Layer\_0/ReLU", 0.1f) \>\> tf.learn.Linear("Layer\_1/Linear", 64) \>\> tf.learn.ReLU("Layer\_1/ReLU", 0.1f) \>\> tf.learn.Linear("Layer\_2/Linear", 32) \>\> tf.learn.ReLU("Layer\_2/ReLU", 0.1f) \>\> tf.learn.Linear("OutputLayer/Linear", 10) val trainingInputLayer = tf.learn.Cast("TrainInput/Cast", INT64) val loss = tf.learn.SparseSoftmaxCrossEntropy("Loss/CrossEntropy") \>\> tf.learn.Mean("Loss/Mean") \>\> tf.learn.ScalarSummary("Loss/Summary", "Loss") val optimizer = tf.train.AdaGrad(0.1) // Directory in which to save summaries and checkpointsval summariesDir = java.nio.file.Paths.get((tempdir/"mnist\_summaries").toString()) val (model, estimator) = dtflearn.build\_tf\_model( architecture, input, trainInput, trainingInputLayer, loss, optimizer, summariesDir, dtflearn.max\_iter\_stop(1000), 100, 100, 100)(trainData)
    • 🏗 Build feedforward layers and feedforward layer stacks easier.

    Usage

    //Create a single feedforward layerval layer = dtflearn.feedforward(num\_units = 10, useBias = true)(id = 1)//Create a stack of feedforward layersval stack = dtflearn.feedforward\_stack( (i: Int) =\> dtflearn.Phi("Act\_"+i), FLOAT64)( net\_layer\_sizes.tail)

    📦 Package dynaml.tensorflow.layers

    Dynamical Systems: Continuous Time RNN

    • ➕ Added CTRNN layer with inferable time step: DynamicTimeStepCTRNN.
    • ➕ Added a projection layer for CTRNN based models FiniteHorizonLinear.

    Activations

    • ➕ Added cumulative gaussian distribution function as an activation map dtflearn.Phi("actName").
    • ➕ Added generalised logistic function as an activation map dtflearn.GeneralizedLogistic("actName")

    🐛 Bug Fixes

    • 🛠 Fixed bug related to scalar method of VectorField, innerProdDouble and other inner product implementations.

    👌 Improvements and Upgrades

    • ⬆️ Bumped up Ammonite version to 1.1.0