Platfrom-as-a-Service for Deep Learning.

Yesterday, I enjoyed mishima.syk #10. I uploaded my presentation and code to mishimasyk repo.
I introduced briefly about a PaaS for DL named ‘Floyd’. I think the service is interesting because I can run DL on cloud with GPU!

So, I describe very simple example to start DL with “FLOYD” 😉
At first, Make account from the site.
Next, install command line tools. Just type pip install -U floyd-cli!

# Install floyd-cli
$ pip install -U floyd-cli

Third step, login the floyd.

# from terminal
$ floyd login

Then web browser will launch and the page provides authentication token. Copy and paste it.
Ready to start!
Let’s play with floyd.
Fist example is iris dataset classification using sklearn.

import numpy as np
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.cross_validation import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
dataset = load_iris()
X = dataset.data
y = dataset.target

trainx, testx, trainy, testy = train_test_split( X, y, test_size=0.2,random_state = 123 )

svc = SVC( kernel='rbf' )
svc.fit( trainx, trainy )

rfc = RandomForestClassifier()
rfc.fit( trainx, trainy )

predsvc = svc.predict( testx )
predrf = rfc.predict( testx )

print( classification_report(testy, predsvc ))

Use floyd run command to start the code after initialize the project.

$ mkdir test_pj
$ cd test_pj
$ floyd init
$ floyd run 'python svc_rf_test.py'
Creating project run. Total upload size: 168.9KiB
Syncing code ...
[================================] 174656/174656 - 00:00:02
Done
RUN ID                  NAME                     VERSION
----------------------  ---------------------  ---------
xxxxxxxx  iwatobipen/test_pj:10         10

To view logs enter:
    floyd logs xxxxxxxx

I could check the status via web browser.

Next run the DNN classification model.
It is very very simple example. not so deeeeeeeeeeeep.

mport numpy as np
from sklearn.datasets import load_iris
from sklearn.cross_validation import train_test_split
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.utils import np_utils

dataset = load_iris()
X = dataset.data

xdim = 4
y = dataset.target
y = np_utils.to_categorical( y, 3 )
trainx, testx, trainy, testy = train_test_split( X, y, test_size=0.2,random_state = 123 )

model = Sequential()
model.add( Dense( 16, input_dim = xdim  ) )
model.add( Activation( 'relu' ))
model.add( Dense( 3 ))
model.add( Activation( 'softmax' ))
model.compile( loss = 'categorical_crossentropy',
               optimizer = 'rmsprop',
               metrics = ['accuracy'])

hist = model.fit( trainx, trainy, epochs = 50, batch_size = 1 )
classes = model.predict( testx, batch_size = 1 )

print( [ np.argmax(i) for i in classes ] )
print( [ np.argmax(i) for i in testy ] )
loss, acc = model.evaluate( testx, testy )

print( "loss, acc ={0},{1}".format( loss, acc ))

To run the code in the same manner.

iwatobipen$ floyd run 'python dnn_test.py'
Creating project run. Total upload size: 168.9KiB
Syncing code ...
[================================] 174653/174653 - 00:00:02
Done
RUN ID                  NAME                     VERSION
----------------------  ---------------------  ---------
xxxxxxx  iwatobipen/test_pj:11         11

To view logs enter:
    floyd logs xxxxxxx

Check the log from web site.

2017-07-09 01:51:37,703 INFO - Preparing to run TaskInstance <TaskInstance: iwatobipen/test_pj:11 (id: Uus7cp996732cBWdgt3nz3) (checksum: 144078ab50a63ea6276efee221669d13) (last update: 2017-07-09 01:51:37.694913) [queued]>
2017-07-09 01:51:37,723 INFO - Starting attempt 1 at 2017-07-09 01:51:37.708707
2017-07-09 01:51:38,378 INFO - adding pip install -r floyd_requirements
2017-07-09 01:51:38,394 INFO - Executing command in container: stdbuf -o0 sh command.sh
2017-07-09 01:51:38,394 INFO - Pulling Docker image: floydhub/tensorflow:1.1.0-py3_aws.4
2017-07-09 01:51:39,652 INFO - Starting container...
2017-07-09 01:51:39,849 INFO -
################################################################################

2017-07-09 01:51:39,849 INFO - Run Output:
2017-07-09 01:51:40,317 INFO - Requirement already satisfied: Pillow in /usr/local/lib/python3.5/site-packages (from -r floyd_requirements.txt (line 1))
2017-07-09 01:51:40,320 INFO - Requirement already satisfied: olefile in /usr/local/lib/python3.5/site-packages (from Pillow->-r floyd_requirements.txt (line 1))
2017-07-09 01:51:43,354 INFO - Epoch 1/50
2017-07-09 01:51:43,460 INFO - 1/120 [..............................] - ETA: 8s - loss: 0.8263 - acc: 0.0000e+00
 58/120 [=============>................] - ETA: 0s - loss: 1.5267 - acc: 0.6552
115/120 [===========================>..] - ETA: 0s - loss: 1.2341 - acc: 0.6522
120/120 [==============================] - 0s - loss: 1.2133 - acc: 0.6583
2017-07-09 01:51:43,461 INFO - Epoch 2/50
..........................
 57/120 [=============>................] - ETA: 0s - loss: 0.1135 - acc: 0.9649
115/120 [===========================>..] - ETA: 0s - loss: 0.1242 - acc: 0.9739
120/120 [==============================] - 0s - loss: 0.1270 - acc: 0.9750
2017-07-09 01:51:48,660 INFO - Epoch 50/50
2017-07-09 01:51:48,799 INFO - 1/120 [..............................] - ETA: 0s - loss: 0.0256 - acc: 1.0000
 57/120 [=============>................] - ETA: 0s - loss: 0.0911 - acc: 0.9825
114/120 [===========================>..] - ETA: 0s - loss: 0.1146 - acc: 0.9737
120/120 [==============================] - 0s - loss: 0.1161 - acc: 0.9750
2017-07-09 01:51:48,799 INFO - [1, 2, 2, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 2, 2, 2, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 2, 2, 0]
2017-07-09 01:51:48,800 INFO - [1, 2, 2, 1, 0, 2, 1, 0, 0, 1, 2, 0, 1, 2, 2, 2, 0, 0, 1, 0, 0, 2, 0, 2, 0, 0, 0, 2, 2, 0]
2017-07-09 01:51:48,800 INFO - 30/30 [==============================] - 0s
2017-07-09 01:51:48,800 INFO - loss, acc =0.23778462409973145,0.8666666746139526

The following software packages (in addition to many other common libraries) are available in all the environments:
h5py, iPython, Jupyter, matplotlib, numpy, OpenCV, Pandas, Pillow, scikit-learn, scipy, sklearn

Also, user can install additional packages from pypi. ( Not anaconda … 😦 ) To install that, put file named ‘floyd_requirements.txt’ in the project folder.

In summary, Floyd is very interesting service. Easy to set up DL environment and use GPU on cloud.
I want to support anaconda in FLOYD, because I want to use chemoinformatics package like RDKit, Openbabel etc…

広告

コメントを残す

以下に詳細を記入するか、アイコンをクリックしてログインしてください。

WordPress.com ロゴ

WordPress.com アカウントを使ってコメントしています。 ログアウト / 変更 )

Twitter 画像

Twitter アカウントを使ってコメントしています。 ログアウト / 変更 )

Facebook の写真

Facebook アカウントを使ってコメントしています。 ログアウト / 変更 )

Google+ フォト

Google+ アカウントを使ってコメントしています。 ログアウト / 変更 )

%s と連携中