gotokenizer v1.1.0 Release Notes
Release Date: 2018-10-17 // over 5 years ago-
- ➕ Add Support Custom word Filter (MM RMM BMM)
- ➕ Add Support filter stop tokens (MM RMM BMM)
- ➕ Add tests
- 🛠 Fixed test error
- ⚡️ Update ReadMe
Previous changes from v1.0.0
-
- 👌 Support Maximum Matching Method
- 👌 Support Minimum Matching Method
- 👌 Support Reverse Maximum Matching
- 👌 Support Reverse Minimum Matching
- 👌 Support Bidirectional Maximum Matching
- 👌 Support Bidirectional Minimum Matching
- 👌 Support using Stop Tokens