Pytorch Sample

Requirements

• Authenticated to gcloud (gcloud auth application-default login)

This notebook demonstrate how to create and deploy PyTorch model to Merlin. It uses IRIS classifier model as example.

!pip install --upgrade -r requirements.txt > /dev/null

import merlin
import warnings
import os

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable

from merlin.model import ModelType
from sklearn.datasets import load_iris
warnings.filterwarnings('ignore')

1. Initialize

1.1 Set Merlin Server

merlin.set_url("localhost:8080/api/merlin")

1.2 Set Active Project

project represent a project in real life. You may have multiple model within a project.

merlin.set_project(<project_name>) will set the active project into the name matched by argument. You can only set it to an existing project. If you would like to create a new project, please do so from the MLP console at http://localhost:8080/projects/create.

merlin.set_project("sample")

1.3 Set Active Model

model represents an abstract ML model. Conceptually, model in Merlin is similar to a class in programming language. To instantiate a model you’ll have to create a model_version.

Each model has a type, currently model type supported by Merlin are: sklearn, xgboost, tensorflow, pytorch, and user defined model (i.e. pyfunc model).

model_version represents a snapshot of particular model iteration. You’ll be able to attach information such as metrics and tag to a given model_version as well as deploy it as a model service.

merlin.set_model(<model_name>, <model_type>) will set the active model to the name given by parameter, if the model with given name is not found, a new model will be created.

merlin.set_model("pytorch-sample", ModelType.PYTORCH)

2. Train Model

2.1 Prepare training data

iris = load_iris()
y = iris['target']
X = iris['data']

train_X = Variable(torch.Tensor(X).float())
train_y = Variable(torch.Tensor(y).long())

2.2 Create PyTorch Model

class PyTorchModel(nn.Module):
    # define nn
    def __init__(self):
        super(PyTorchModel, self).__init__()
        self.fc1 = nn.Linear(4, 100)
        self.fc2 = nn.Linear(100, 100)
        self.fc3 = nn.Linear(100, 3)
        self.softmax = nn.Softmax(dim=1)

    def forward(self, X):
        X = F.relu(self.fc1(X))
        X = self.fc2(X)
        X = self.fc3(X)
        X = self.softmax(X)

        return X

2.3 Train and Check Prediction

net = PyTorchModel()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(net.parameters(), lr=0.01)

for epoch in range(10):
    optimizer.zero_grad()
    out = net(train_X)
    loss = criterion(out, train_y)
    loss.backward()
    optimizer.step()

predict_out = net(train_X)
predict_y = torch.max(predict_out, 1)
predict_y

3. Deploy Model

3.1 Serialize Model

In this step, we will serialize the model and create an archive file for torchserve, then move the archive file (.mar) into the model-store folder.

model_dir = "pytorch-model"
model_path = os.path.join(model_dir, "model.pt")

torch.save(net.state_dict(), model_path)

!torch-model-archiver --model-name model \
                     --version 1.0 \
                     --model-file pytorch-model/model.py \
                     --serialized-file pytorch-model/model.pt \
                     --handler pytorch-model/iris_handler.py

!mv model.mar pytorch-model/model-store/

3.2 Create Model Version and Upload

merlin.new_model_version() is a convenient method to create a model version and start its development process. It is equal to following codes:

v = model.new_model_version()
v.start()
v.log_pytorch_model(model_dir=model_dir)
v.finish()

# Create new version of the model
with merlin.new_model_version() as v:
    # Upload the serialized model to Merlin
    merlin.log_pytorch_model(model_dir=model_dir)

3.3 Deploy Model

Each of a deployed model version will have its own generated url

endpoint = merlin.deploy(v)

3.4 Send Test Request

%%bash -s "$endpoint.url"
curl -v -X POST $1 -d '{
  "instances": [
    [2.8,  1.0,  6.8,  0.4],
    [3.1,  1.4,  4.5,  1.6]
  ]
}'

3.5 Delete Deployment

merlin.undeploy(v)