gotokenizer v1.1.0 Release Notes

Release Date: 2018-10-17 // about 6 years ago
    • โž• Add Support Custom word Filter (MM RMM BMM)
    • โž• Add Support filter stop tokens (MM RMM BMM)
    • โž• Add tests
    • ๐Ÿ›  Fixed test error
    • โšก๏ธ Update ReadMe

Previous changes from v1.0.0

    • ๐Ÿ‘Œ Support Maximum Matching Method
    • ๐Ÿ‘Œ Support Minimum Matching Method
    • ๐Ÿ‘Œ Support Reverse Maximum Matching
    • ๐Ÿ‘Œ Support Reverse Minimum Matching
    • ๐Ÿ‘Œ Support Bidirectional Maximum Matching
    • ๐Ÿ‘Œ Support Bidirectional Minimum Matching
    • ๐Ÿ‘Œ Support using Stop Tokens