All Versions
8
Latest Version
Avg Release Cycle
14 days
Latest Release
62 days ago

Changelog History

  • v0.5.2 Changes

    March 16, 2021

    โž• Added

    • Handle multiple BERT pooling strategies (e.g. CLS_TOKEN, REDUCE_MEAN) in nlp.transformers.bert.server_encode.go.
  • v0.5.1 Changes

    March 07, 2021

    โž• Added

    ๐Ÿ”„ Changed

    • ๐Ÿ‘Œ Improve nlp.transformer.generation algorithms:
      • optimize Generator.getTopKScoredTokens().
      • optimize Generator.updateTokensScores().
    • Simplify mat32.Dense.Mul when doing Matrix-Vector multiplication.
    • โ™ป๏ธ Refactor math32 functions using chewxy/math32 functions.
    • ๐Ÿ‘Œ Improve ag.Graph efficiency:
      • Use pre-computed cache doing ag.Graph.groupNodesByHeight().
      • Use sync.pool to reduce allocations of graph's operators.

    ๐Ÿ›  Fixed

    • ๐Ÿ›  Fix past key-values usage on self-attention and cross-attention
  • v0.5.0 Changes

    February 15, 2021

    โž• Added

    • Implement a beam-search algorithm for conditional generation:
      • nlp.transformer.generation package.
    • โž• Add implementation of the Sentence-Piece tokenizer:
      • nlp.tokenizers.sentencepiece package.
    • BART improvements:
      • gRPC and HTTP API to perform Text Generation.
      • Add support for "Marian" architecture (used for translation tasks).
      • Add sinusoidal positional encoder (used by Marian).
      • Add "head" for conditional generation:
      • nlp.transformers.bart.head.conditionalgeneration package.
    • โž• Add nn.Closer interface (e.g. embeddings.Model needs to close the underlying key-value store).
    • โž• Add Swish act. function without trainable parameters.
    • โž• Add SiLU act. function (it is just an alias for Swish).
    • ๐Ÿ†• New pe.SinusoidalPositionalEncoder (this implementation replaces unused pe.PositionalEncoder and pe.AxialPositionalEncoder)

    ๐Ÿ”„ Changed

    • โšก๏ธ Update urfave/cli to v2
    • โšก๏ธ Update dgraph-io/badger to v3.
    • ๐Ÿ‘‰ Make the BART positional encoder an interface to support various encoding (i.e. trainable vs static).
    • Rename to fn.NewSwish into fn.NewSwishB as this was the Swish variant with trainable parameters (B).
    • ๐Ÿ˜Œ Relax ag.GetOpName to match operator names in lower-case.
    • ๐Ÿ‘ Allow arbitrary activation function on BART encoder/decoder layers.
    • ๐Ÿ‘‰ Use precomputed "keys" and "values" in self-attention, multi-head attention and BART decoder.

    โœ‚ Removed

    • In relation to the aforementioned positional encoding changes:
      • pe.PositionalEncoder and related functions
      • pe.AxialPositionalEncoder and related functions

    ๐Ÿ›  Fixed

    • ๐Ÿ›  Fix causal-mask used by nn.ScaledDotProductAttention
  • v0.4.1 Changes

    January 22, 2021

    โž• Added

    • ๐Ÿ†• New function ReleaseMatrix to packages mat32 and mat64.
    • ๐Ÿ†• New methods to Matrix interface, from mat32 and mat64: Minimum, Maximum, MulT, Inverse, DoNonZero. However, the implementation on sparse matrices is not implemented yet (it always panics).

    ๐Ÿ”„ Changed

    • ๐Ÿ“œ Prefer handling Matrix interface values over specific Dense or Sparse matrices, also avoiding unnecessary type casts. Relevant changes to the public API are listed below.
      • mat(32|64).Stack function's arguments and returned value are now Matrix interfaces, instead of explicit Dense matrices.
      • Dense.Minimum and Dense.Maximum, from packages mat32 and mat64, return a Matrix interface, instead of a specific Dense type.
      • The return values of fofe.EncodeDense, fofe.Encode, and fofe.BiEncode are slices of Matrix values, instead of Dense or Sparse.
      • The z argument of the function fofe.Decode is of type Matrix, instead of Dense.
      • ml.optimizers.de (Differential Evolution optimizer) API was changed handling Matrix values, instead of specific Dense matrices. Changes include: Member.TargetVector, Member.DonorVector, ScoredVector.Vector, the vector argument of NewMember function, the solution argument of score and validate functions passed to NewOptimizer.
      • PositionalEncoder.Cache and AxialPositionalEncoder.Cache are slices of Matrix, instead of slices of Dense.
      • AxialPositionalEncoder.EncodingAt returns a Matrix value, instead of Dense.
      • nn.DumpParamsVector returns a Matrix value, instead of Dense.
      • The vector argument of the function nn.LoadParamsVector is a Matrix, instead of Dense.
      • The value argument of the method embeddings.Model.SetEmbedding is of type Matrix, instead of Dense.
      • The type of the struct field evolvingembeddings.WordVectorPair.Vector is Matrix, instead of Dense.
  • v0.4.0 Changes

    January 17, 2021

    โž• Added

    • โœ… Various new test cases (improving the coverage).
    • ๐Ÿ“ฆ nlp.embeddings.syncmap package.
    • ml.nn.recurrent.srnn.BiModel which implements a bidirectional variant of the Shuffling Recurrent Neural Networks ( SRNN).
    • ๐Ÿ‘€ Configurable timeout and request limit to all HTTP and gRPC servers (see also commands help).

    ๐Ÿ”„ Changed

    • โ™ป๏ธ All CLI commands implementation has been refactored, so that the docker-entrypoint can reuse all other cli.App objects, instead of just running separate executables. By extension, now the Dockerfile builds a single executable file, and the final image is way smaller.
    • โฌ†๏ธ All dependencies have been upgraded to the latest version.
    • Simplify custom error definitions using fmt.Errorf instead of functions from github.com/pkg/errors.
    • Custom binary data serialization of matrices and models is now achieved with Go's encoding.gob. Many specific functions and methods are now replaced by fewer and simpler encoding/decoding methods compatible with gob. A list of important related changes follows.
      • utils.kvdb.KeyValueDB is no longer an interface, but a struct which directly implements the former "badger backend".
      • utils.SerializeToFile and utils.DeserializeFromFile now handle generic interface{} objects, instead of values implementing Serializer and Deserializer.
      • mat32 and mat64 custom serialization functions (e.g. MarshalBinarySlice, MarshalBinaryTo, ...) are replaced by implementations of BinaryMarshaler and BinaryUnmarshaler interfaces on Dense and Sparse matrix types.
      • PositionalEncoder.Cache and AxialPositionalEncoder.Cache fields (from ml.encoding.pe package) are now public.
      • All types implementing nn.Model interface are registered for gob serialization (in init functions).
      • embeddings.Model.UsedEmbeddings type is now nlp.embeddings.syncmap.Map.
      • As a consequence, you will have to re-serialize all your models.
    • Flair converter now sets the vocabulary directly in the model, instead of creating a separate file.
    • sequencelabeler.Model.LoadParams has been renamed to Load.

    โœ‚ Removed

    • In relation to the aforementioned gob serialization changes:
      • nn.ParamSerializer and related functions
      • nn.ParamsSerializer and related functions
      • utils.Serializer and utils.Deserializer interfaces
      • utils.ReadFull function
    • sequencelabeler.Model.LoadVocabulary

    ๐Ÿ›  Fixed

    • ๐Ÿณ docker-entrypoint sub-command hugging-face-importer has been renamed to huggingface-importer, just like the main command itself.
    • ๐Ÿณ docker-entrypoint sub-command can be correctly specified without leading ./ or / when run from a Docker container.
    • ๐Ÿ’ฅ BREAKING: mat32.Matrix serialization has been fixed, now serializing single values to chunks of 4 bytes (instead of 8, like float64). Serialized 32-bit models will now be half the size! Unfortunately you will have to re-serialize your models (sorry!).
  • v0.3.0 Changes

    January 10, 2021

    โž• Added

    • ๐Ÿ‘• Static analysis job (golint and gocyclo) to Go GitHub workflow.
    • You can set a limit for concurrent heavyweight Graph computations (e.g. forward and backward steps) - see ml.ag.ConcurrentComputations() (GraphOption) and ml.ag.Graph.ConcurrentComputations(). If no option is specified, by default the limit is set to runtime.NumCPU().
    • You can set a limit for concurrent heavyweight computations of ml.optimizers.gd.GradientDescent (e.g. params update step).
    • ๐Ÿ†• New package utils.processingqueue.
    • ๐Ÿ“ฆ mat32 package, which operates on float32 data type.
    • 0๏ธโƒฃ It's possible to switch between float32 and float64 as default floating-point data type, using the script change-float-type.sh
    • โœ… Go GitHub workflow has been adapted to run tests using both float32 and float64 as main floating-point data type.
    • This CHANGELOG file.
    • Pull and convert Hugging Face models automatically if not found locally when starting BERT or BART server.
    • ๐Ÿ“ฆ Move content from GitHub Wiki to README in related package folders.

    ๐Ÿ”„ Changed

    • ml.ag.ConcurrentComputations (GraphOption) expects the maximum number of concurrent computations handled by heavyweight Graph operations (e.g. forward and backward steps).
    • ml.nn.linear.Model and ml.nn.convolution.Model read the concurrent computations limit set on the model's Graph, thus SetConcurrentComputations() methods have been removed.
    • mat has been renamed to mat64 and some functions have been renamed.
    • The whole project now works with float32 floating-point data type by default, by using the package mat32.
    • ๐Ÿ“ฆ When imported, the new package mat32 is always aliased as mat. Then, explicit usages of float64 type have been replaced with mat.Float. Moreover, bitsize-specific functions have been made more generic (i.e. operating with mat.Float type) or split into separate implementation, in mat32 and mat64. In this way, switching the whole project between float32 and float64 is just a matter of changing all imports, from mat32 to mat64, or vice-versa (see also the new file change-float-type.sh).
    • โšก๏ธ Update internal links to pre-trained NER models to float32 versions.
    • nlp.sequencelabeler.Convert() now loads and converts original Flair models, instead of pre-processed dumps.
    • ๐Ÿ”„ Change command line arguments to make them more consistent; please refer to the help messages of each command.
    • โšก๏ธ Update Dockerfile using a new base building image and adding bart server.

    ๐Ÿ›  Fixed

    • โž• Added dedicated package names to different protocol buffers definition files to avoid name conflicts.
  • v0.2.0 Changes

    December 31, 2020

    โž• Added

    • ๐Ÿ‘Œ Support for BART model (tested on Natural Language Inference task).
    • BART API to perform Zero-Shot Text Classification.

    ๐Ÿ”„ Changed

    • Significant reduction of boilerplate code through the unification of nn.Model and nn.Processor interfaces:
      • there is now a single nn.Model interface that can be reified to become a neural processor - see nn.Reify();
      • there was no compelling reason to have a Forward method in the nn.Model interface, so it has been removed, gracefully increasing flexibility in the implementation of a model.
  • v0.1.0 Changes

    December 09, 2020

    ๐Ÿš€ First tagged release!