Upload file to Datalake using CLI

Introduction

This documentation explains how to upload a file to Datalake using ABEJA Platform CLI.

Install CLI

Please confirm installation method of ABEJA Platform CLI and install it.

Upload files in a directory

Directory tree as follows

$ tree
.
├── cats
│   ├── cat-1.jpeg
│   ├── cat-2.jpeg
│   └── cat-3.jpeg
└── dogs
    ├── dog-1.jpeg
    ├── dog-2.jpeg
    └── dog-3.jpeg
2 directories, 6 files

Confirmation and configure of authentication information

ABEJA Platform CLI is authenticated by user authentication. Check the user ID, personal access token, and your organization name (or ID). Please confirm CONFIGURATION COMMAND’s init and configure it.

$ abeja config init
User ID              : 0123456789123
Personal Access Token: qwertyuiopasdfghjklzxcvbnm
Organization ID      : 1020304050234

Upload files to Datalake

Execute the following commands to upload files in the directory to Datalake. (If you specify directory as a path, you need specify option --recursive also)

$ abeja datalake upload ./cats --channel_id 1234567890123 --recursive --metadata label:cat
$ abeja datalake upload ./dogs --channel_id 1234567890123 --recursive --metadata label:dog

Confirm Uploaded files

You can check uploaded files on the Datalake details page.

Upload files to Datalake specifing the metadata individually

If you use option --file-list, you can upload files to Datalake specifing the metadata individually.

[
  {
    "file": "cats/cat-1.jpeg",
    "metadata": {
      "family": "cat",
      "form": "Tiger-cat"
    }
  },
  {
    "file": "dogs/dog-1.jpeg",
    "metadata": {
      "family": "dog",
      "form": "Akita",
    }
  }
]

$ abeja datalake upload --channel_id 1234567890123 --file-list ./upload.json