Popularity
4.8
Growing
Activity
8.8
Declining
147
8
7

Programming language: Go
Tags: Machine Learning    
Latest version: v0.6.0

Goptuna alternatives and similar packages

Based on the "Machine Learning" category

Do you think we are missing an alternative of Goptuna or a related project?

Add another 'Machine Learning' Package

README

Goptuna

Software License GoDoc Go Report Card

Distributed hyperparameter optimization framework, inspired by Optuna [1]. This library is particularly designed for machine learning, but everything will be able to optimize if you can define the objective function (e.g. Optimizing the number of goroutines of your server and the memory buffer size of the caching systems).

Key features:

State-of-the-art algorithms Optuna compatible RDB backend

Supported algorithms:

Goptuna supports various state-of-the-art Bayesian optimization, Evolution strategy and Multi-armed bandit algorithms. These algorithms are implemented in pure Go and continuously benchmarked on GitHub Actions.

  • Random search
  • TPE: Tree-structured Parzen Estimators [2]
  • CMA-ES: Covariance Matrix Adaptation Evolution Strategy [3]
  • IPOP-CMA-ES: CMA-ES with increasing population size [4]
  • BIPOP-CMA-ES: BI-population CMA-ES [5]
  • Median Stopping Rule [6]
  • ASHA: Asynchronous Successive Halving Algorithm (Optuna flavored version) [1,7,8]

Projects using Goptuna:

Installation

You can integrate Goptuna in wide variety of Go projects because of its portability of pure Go.

$ go get -u github.com/c-bata/goptuna

Usage

Goptuna supports Define-by-Run style API like Optuna. You can dynamically construct the search spaces.

5 steps to use Goptuna.

  1. Define an objective function which returns a value you want to minimize.
  2. Define the search space via Suggest APIs.
  3. Create a study which manages each experiment.
  4. Evaluate your objective function.
  5. Print the best evaluation parameters.

Furthermore, I recommend you to use RDB storage backend for following purposes.

  • Continue from where we stopped in the previous optimizations.
  • Scale studies to tens of workers that connecting to the same RDB storage.
  • Visualize parameters on Jupyter notebook using Optuna.
package main

import (
    "log"
    "math"

    "github.com/c-bata/goptuna"
    "github.com/c-bata/goptuna/tpe"
)

func objective(trial goptuna.Trial) (float64, error) {
    x1, _ := trial.SuggestFloat("x1", -10, 10)
    x2, _ := trial.SuggestFloat("x2", -10, 10)
    return math.Pow(x1-2, 2) + math.Pow(x2+5, 2), nil
}

func main() {
    study, err := goptuna.CreateStudy(
        "goptuna-example",
        goptuna.StudyOptionSampler(tpe.NewSampler()))
    if err != nil { ... }

    err = study.Optimize(objective, 100)
    if err != nil { ... }

    v, _ := study.GetBestValue()
    p, _ := study.GetBestParams()
    log.Printf("Best value=%f (x1=%f, x2=%f)",
        v, p["x1"].(float64), p["x2"].(float64))
}

Advanced usages

Distributed optimization using RDB storage backend with MySQL

There is no complicated setup for distributed optimization but all Goptuna workers need to use the same RDB storage backend. First, setup MySQL server like following to share the optimization result.

$ cat mysql/my.cnf
[mysqld]
bind-address = 0.0.0.0
default_authentication_plugin=mysql_native_password

$ docker pull mysql:8.0
$ docker run \
  -d \
  --rm \
  -p 3306:3306 \
  --mount type=volume,src=mysql,dst=/etc/mysql/conf.d \
  -e MYSQL_USER=goptuna \
  -e MYSQL_DATABASE=goptuna \
  -e MYSQL_PASSWORD=password \
  -e MYSQL_ALLOW_EMPTY_PASSWORD=yes \
  --name goptuna-mysql \
  mysql:8.0

Then, create a study object using goptuna CLI

$ goptuna create-study --storage mysql://goptuna:password@localhost:3306/yourdb --study yourstudy
yourstudy
$ mysql --host 127.0.0.1 --port 3306 --user goptuna -ppassword -e "SELECT * FROM studies;"
+----------+------------+-----------+
| study_id | study_name | direction |
+----------+------------+-----------+
|        1 | yourstudy  | MINIMIZE  |
+----------+------------+-----------+
1 row in set (0.00 sec)

Finally, run the Goptuna workers which contains following code.

package main

import ...

func main() {
    db, _ := gorm.Open("mysql", "goptuna:password@tcp(localhost:3306)/yourdb?parseTime=true")
    storage := rdb.NewStorage(db)
    defer db.Close()

    study, _ := goptuna.LoadStudy(
        "yourstudy",
        goptuna.StudyOptionStorage(storage),
        ...,
    )
    _ = study.Optimize(objective, 50)
    ...
}

The schema of Goptuna RDB storage backend is compatible with Optuna's one. So you can check optimization result with Optuna's dashboard like following:

$ pip install optuna bokeh mysqlclient
$ optuna dashboard --storage mysql+mysqldb://goptuna:password@127.0.0.1:3306/yourdb --study yourstudy
...

[shell script to reproduce this](./_examples/simple_rdb/check_mysql.sh)

Receive notifications of each trials

You can receive notifications of each trials via channel. It can be used for logging and any notification systems.

package main

import ...

func main() {
    trialchan := make(chan goptuna.FrozenTrial, 8)
    study, _ := goptuna.CreateStudy(
        ...
        goptuna.StudyOptionIgnoreObjectiveErr(true),
        goptuna.StudyOptionSetTrialNotifyChannel(trialchan),
    )

    var wg sync.WaitGroup
    wg.Add(2)
    go func() {
        defer wg.Done()
        err = study.Optimize(objective, 100)
        close(trialchan)
    }()
    go func() {
        defer wg.Done()
        for t := range trialchan {
            log.Println("trial", t)
        }
    }()
    wg.Wait()
    if err != nil { ... }
    ...
}

[full source code](./_examples/trialnotify/main.go)

References:

Blog posts:

Status:

License

This software is licensed under the MIT license, see [LICENSE](./LICENSE) for more information.


*Note that all licence references and agreements mentioned in the Goptuna README section above are relevant to that project's source code only.