Optimize ML model with optuna and visualize the result with MLFlow #informatics #machine learning

As you know Optuna is very useful and powerful package for machine learning. I often use the package in my own task. And MLFLOW is also useful package. I posted about mlflow before. MLflow has many functions for visualize experiment results and manage models.

https://iwatobipen.wordpress.com/2018/11/14/tracking-progress-of-machine-learning-machinelearning/

I think it will be useful if models can be optimized optuna and be managed with mlflow. Fortunately new version of optuna integrates mlflow ;)

It sounds nice doesn’t it. I used the function with simple Iris dataset.

At first, I optimized SVC model with optuna. The code is below. To integrate optuna and mlflow, MLflowCallback should be called from optuna.integration.mlflow. And pass the object to optimize method.

import os
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score
import optuna
from optuna.integration.mlflow import MLflowCallback

iris = load_iris()
trainx, testx, trainy, testy = train_test_split(iris.data, iris.target, test_size=0.2)

def objective(trial):
    gamma = trial.suggest_loguniform('gamma', 1e-3, 3.0)
    C = trial.suggest_loguniform('C', 1e+0, 1e+2/2)
    kernel = trial.suggest_categorical('kernel', ['linear','rbf','sigmoid'])
    svc = SVC(gamma=gamma, C=C, kernel=kernel)
    svc.fit(trainx, trainy)
    predy = svc.predict(testx)
    accuracy = accuracy_score(testy, predy)
    return accuracy

if __name__=='__main__':

    mlflc = MLflowCallback(tracking_uri='ml_exp',
                      metric_name='accuracy')
    study = optuna.create_study(study_name='iris_test')
    study.optimize(objective, n_trials=50, callbacks=[mlflc])

The code above optimizes kernel, gamma and C of SVC. After optimization, all experiments data is stored in ‘ml_exp’ folder where is defined in MLflowCallback.

Then run mlflow ui command to check the results. And open localhost:5000 from web browser.

$  mlflow ui --backend-store-uri file:///home/user/dev/mlflowsample/test_optuna/ml_exp/

Flow webbrowser, user can check performance of each model and relation ship between model accuracy and hyper parameters.

MLflow not only visualize model performance but also serve model. Today I showed simple example of model performance visualization only but I’ll post more examples related chemoinformatics topics near the feature.

Advertisement