As you know Optuna is very useful and powerful package for machine learning. I often use the package in my own task. And MLFLOW is also useful package. I posted about mlflow before. MLflow has many functions for visualize experiment results and manage models.
I think it will be useful if models can be optimized optuna and be managed with mlflow. Fortunately new version of optuna integrates mlflow ;)
It sounds nice doesn’t it. I used the function with simple Iris dataset.
At first, I optimized SVC model with optuna. The code is below. To integrate optuna and mlflow, MLflowCallback should be called from optuna.integration.mlflow. And pass the object to optimize method.
import os import numpy as np from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.svm import SVC from sklearn.metrics import accuracy_score import optuna from optuna.integration.mlflow import MLflowCallback iris = load_iris() trainx, testx, trainy, testy = train_test_split(iris.data, iris.target, test_size=0.2) def objective(trial): gamma = trial.suggest_loguniform('gamma', 1e-3, 3.0) C = trial.suggest_loguniform('C', 1e+0, 1e+2/2) kernel = trial.suggest_categorical('kernel', ['linear','rbf','sigmoid']) svc = SVC(gamma=gamma, C=C, kernel=kernel) svc.fit(trainx, trainy) predy = svc.predict(testx) accuracy = accuracy_score(testy, predy) return accuracy if __name__=='__main__': mlflc = MLflowCallback(tracking_uri='ml_exp', metric_name='accuracy') study = optuna.create_study(study_name='iris_test') study.optimize(objective, n_trials=50, callbacks=[mlflc])
The code above optimizes kernel, gamma and C of SVC. After optimization, all experiments data is stored in ‘ml_exp’ folder where is defined in MLflowCallback.
Then run mlflow ui command to check the results. And open localhost:5000 from web browser.
$ mlflow ui --backend-store-uri file:///home/user/dev/mlflowsample/test_optuna/ml_exp/
Flow webbrowser, user can check performance of each model and relation ship between model accuracy and hyper parameters.
MLflow not only visualize model performance but also serve model. Today I showed simple example of model performance visualization only but I’ll post more examples related chemoinformatics topics near the feature.