The newb guide to Google Cloud Machine Learning (ML) Engine - Episode Two

By Michael Yee

I have mixed feeling about cliffhangers and I hope you do not hate me for leaving you with one in the last blog. In this episode, we will look at the last piece of the puzzle for predictions using Cloud Machine Learning Engine REST APIs.

Answer Part 3: Cloud Machine Learning Engine REST APIs

The following code describes how to use the Google API client library to easily make calls to the Cloud Machine Learning Engine REST APIs and is designed to familiarize yourself with with the following ideas:

  • Use application default credentials in your Python applications
  • Get a Python representation of the Cloud ML Engine services
  • Use that representation to create a model in your project
import json

from googleapiclient import discovery
from googleapiclient import errors
from oauth2client.client import GoogleCredentials
from import file_io

# Google Storage locations
BUCKET                  = "my-ml-project-888"
GS_BUCKET               = "gs://my-ml-project-888/"
INPUT_FOLDER            = "input/"
OUTPUT_FOLDER           = "output/"
MODEL_FOLDER            = "regression"
INPUT_FILENAME          = "ML_engine_feed_dict.json"
OUTPUT_FILENAME         = "ML_engine_result.json"

def read_bucket_json(filename):
    # reads a json file from Google Storage
    with file_io.FileIO(GS_BUCKET+INPUT_FOLDER+filename, "r") as json_file:
        json_data = json.load(json_file)

    return json_data

def write_bucket_json(json_data, filename):
    # writes a json file to Google Storage
    with file_io.FileIO(GS_BUCKET+OUTPUT_FOLDER+filename, "w") as json_file:
        json.dump(json_data, json_file)

# store the project ID as a variable in the format the API requires
projectID = "projects/{}/models/{}".format(BUCKET, MODEL_FOLDER)

# get application default credentials
credentials = GoogleCredentials.get_application_default()

# build a representation of the Cloud ML API
ml ="ml", "v1", credentials=credentials)

# read the input file from Google Storage
feed_dict = read_bucket_json(INPUT_FILENAME)

# create a dictionary with the fields from the request body
request_dict = {"instances": [feed_dict]}

# create a request to call projects.models.create
request = ml.projects().predict(name=projectID, body=request_dict)

    # make the prediction request!
    response = request.execute()
    # print the result to screen

    # write the result to Google Storage
    write_bucket_json(response, OUTPUT_FILENAME)

except errors.HttpError as err:
    # something went wrong!
    print("There was an error!  Here are the details:")

The above code will make a request to Google Cloud Machine Learning (ML) Engine using data and model located on Google Storage.

Input instances (data) are passed for online prediction as the message body for the projects.predict call. Make each instance an item in a list and name the list member instances. Example: {“instances”: [{‘x’: [1.0, 2.0, 3.0]}]}

The output will print the result {‘predictions’: [{‘y’: [1.5, 2.0, 2.5]}]} to the screen and to an output file located on Google Storage.

The End?

Written on August 23, 2017