Configuration file for Training

On this page, explain about training.yaml which is setting file that created at init page under the TRAINING COMMAND category.

Format

training.yaml is written in YAML format.

 Setting

training.yaml have those items as below.

items	Overview
name	Training job definition name
handler	Specify path for call function
image	Specify image for training model
environment	Environment variables. The format is key:value. For more information on user-specifiable environment variables, see here
params	[DEPRECATED] User parameter. The format is key:value. This item has been deprecated. Please use environment instead.
datasets	Using for training model. The format is key:value
ignores	Specify excluding Directly or File for not to update when execute create-version command

name

Training job definition name.

This is required item.

handler

Specify path for function to call from ABEJA Platform to get started training.

When it is train:handler, Call a handler function that has been defined in the train.py.

When it is src.train:handler, Call a handler function that has been defined in the train.py under the src directory.

This is required item.

image

Specify environment image for training model. Please select it in accordance with your propose from below.

• abeja-inc/all-gpu:18.10
• abeja-inc/all-cpu:18.10

This is required item.

environment

Specify environment variables as a key:value format to pass in a program when executing training. When specifying more than one, Please specified params 1 item per line as a sample below.

environment:
  key1: value1
  key2: value2

It is available to read at training program because that specified parameter is passing as an environment variable at train execution environment.

This is an optional item.

params

DEPRECATED this item has been deprecated, please use environment instead.

Specify parameter as a key:value format to pass in a program when executing training. When specifying more than one, Please specified params 1 item per line as a sample below.

params:
  key1: value1
  key2: value2

It is available to read at training program because that specified parameter is passing as an environment variable at train execution environment.

This is an optional item.

datasets

Specify dataset using for model training. Specify alias name for using training program on the key of key:value format and specify dataset ID on the value. When specifying more than one, Please specified params 1 itme per line as a sample below.

datasets:
  alias1: 1111111111111
  alias2: 1111111111112

Specified dataset is passing as a context params when execute training and available to retrieve it.

def handler(context):
    dataset_aliases = context.datasets
    dataset_id = dataset_aliases['alias1']

This is an optional item. When omit specified command, context.datasets will be empty dict.

ignores

Specify excluding Directly or File for not to update when execute create-version command.

In case of specifying Directory/File under the Directory, Specify relative path from current directory. When specifying more than one, Please specified params 1 item per line as a sample below.

ignores:
  - tests
  - data/ignore_one

This is an optional item. When omit this option, Only .git directory will be excluded.

Ex.

name: training-config-sample
handler: train:handler
image: abeja-inc/all-gpu:18.10
environment:
  ENV_VAL1: value1
  ENV_VAL2: value2
params:
  ENV_VAL1: value1
  ENV_VAL2: value2
datasets:
  alias1: 1234567890123
  alias2: 1234567890234
ignores:
  - .gitignore
  - tests
  - README.md