Managing Apache CloudStack API and automation

Managing Apache CloudStack API and automation

Apache CloudStack – API

The CloudStack programmer guide is the reference guide on how to create API requests and how to interpret resulting the responses. Programming against the API requires familiarity with HTTP GET/POST methods, query strings, and either XML or JSON and how to generate those in the programming language of your choice. 

In addition to programming requests and custom integration, the API is also used by a number of cloud infrastructure management, provisioning, and orchestration tools, and it can even be accessed from a command line. Examples of those include:

API endpoint URLs

The API endpoints are specific for each data center location from which Leaseweb offers Apache CloudStack Elastic Compute.

Currently available are:

Data Center location API URL Service
AMS-01 (Amsterdam)

https://csrp01nl.leaseweb.com/client/api

Elastic Compute Premium
AMS-01 (Amsterdam)   https://csrp07nl.leaseweb.com/client/api Elastic Compute
AMS-01 (Amsterdam) https://csrp08nl.leaseweb.com/client/api Elastic Compute Premium
FRA-01 (Frankfurt)

https://csrp02de.leaseweb.com/client/api

Elastic Compute Premium
WDC-02 (Washington D.C.)

https://csrp03us.leaseweb.com/client/api

Elastic Compute Premium
WDC-02 (Washington D.C.)

https://csrp09us.leaseweb.com/client/api

Elastic Compute
SIN-01 (Singapore)

https://csrp04sg.leaseweb.com/client/api

Elastic Compute Premium
LON-01 (UK) https://csrp06uk.leaseweb.com/client/api Elastic Compute Premium
MTL-02 (CA) https://csrp10ca.leaseweb.com/client/api Elastic Compute

Set up API access

To use the API, a domain administrator will need to associate a user account with a generated key pair containing both an API and the Secret Key.

It is strongly recommended that the domain administrator doesn’t use the account Leaseweb provided for API access and will create additional CloudStack accounts first. Once the additional account(s) have been created, the domain administrator can use the web interface and the menu Accounts → your-domain → Users  →  Account Name to generate a new API and Secret Key pair for that Account. The Account type Domain Administrator or User determines which API functionality will be available.

CloudStack will generate a unique key pair with both a Secret and an API key. 

Information

The clipboard icon next to the Secret Key and API Key should be used to select the keys for copying because the web interface won’t display the complete key and selecting the key with a mouse may otherwise result in missing characters.

82575393

82575392

Install cloudmonkey – cmk

Cloudmonkey – cmk is a front-end to the API that offers both an interactive shell and a command line tool that allows you to quickly get started with automation, without the need for programming.  

The original cloudmonkey was written in Python but starting from version 6.0.0, the modern cloudmonkey cmk is a fast and simplified Go-port of the original tool with some backward incompatibilities and a reduced feature set. It ships as a standalone 64-bit executable binary for several platforms such as Linux, Mac, and Windows.
For more information on cloudmonkey and to get binary please visit the link cloudstack-cloudmonkey.

Install on Linux and Mac OSX

Download the binary for Linux or Mac based on your platform and move it to a directory that is on $PATH such as /usr/local/bin.
For example, we are downloading cmk version 6.2.0 for X86-64 platform:

install cloudmonkey on OSX
sudo wget https://github.com/apache/cloudstack-cloudmonkey/releases/download/6.2.0/cmk.linux.x86-64 -O /usr/local/bin/cmk
sudo chmod +x /usr/local/bin/cmk

Configure cloudmonkey for first use

The default server profile has the name localcloud with defaults (such as URL and credentials) set to a locally running CloudStack server. Start cmk, create a new server profile, configure your CloudStack server’s API endpoint and run sync, for example:

Before cloudmonkey can be used, it needs to be configured with the correct settings and credentials.

We start by launching cloudmonkey from a terminal prompt, and from the interactive cloudmonkey shell, creating a profile named “LeaseWeb” with default values. 

cloudmonkey – create profile
~> cmk
Apache CloudStack CloudMonkey 6.2.0
Report issues: https://github.com/apache/cloudstack-cloudmonkey/issues

(LeaseWeb)  > set profile LeaseWeb
Loaded in-built API cache. Failed to read API cache, please run 'sync'.
Loaded server profile: LeaseWeb
Url:         http://localhost:8080/client/api
Username:    admin
Domain:      /
API Key:     
Total APIs:  590

Then update the default values with the correct settings and personal keys to manage the private cloud:

cloudmonkey – update profile
(LeaseWeb)  > set username your-username
(LeaseWeb)  > set domain your-domain
(LeaseWeb)  > set apikey 1234567890abcdefghijklmnopQWERTYASDFGHJJJ
(LeaseWeb)  > set url https://csrp01nl.leaseweb.com/client/api
(LeaseWeb)  > set secretkey 0987654321ponmlkkihgfedcbaSECRETHHHHGF

The URL depends on which data center is hosting the private cloud and can be found in the table at the top of this page.  

You can also use the set command to set the various server profile-specific options. The cloudmonkey configuration is stored on ~/.cmk/config file. The ~/.cmk directory also contains the API cache at ~/.cmk/profiles.

cloudmonkey – sync and test profile
(LeaseWeb)  > sync 
Discovered 610 APIs

Using cloudmonkey

Cloudmkonkey can be used as an interactive shell with autocomplete and history functionality by starting it from the command line:

Usage – interactive shell
$ cmk
 (LeaseWeb)  > list users
count = 2
user:
+--------------+---------+--------------------------------------+-----------+--------------------------+----------+----------------------------------------------------------------------------------------+---------+--------------------------------+----------------------------------------------------------------------------------------+---------------------+---------+-------------+------------------+--------------------------------------+-----------+--------------------------------------+
|   username   | account |               domainid               | firstname |         created          | lastname |                                         apikey                                         |  domain |             email              |                                       secretkey                                        | iscallerchilddomain |  state  | accounttype |     timezone     |                  id                  | isdefault |              accountid               |
+--------------+---------+--------------------------------------+-----------+--------------------------+----------+----------------------------------------------------------------------------------------+---------+--------------------------------+----------------------------------------------------------------------------------------+---------------------+---------+-------------+------------------+--------------------------------------+-----------+--------------------------------------+
|     LSW      |   LSW   | e7de501e-c5c7-460c-8063-d1ac82154dd1 |  Leaseweb | 2015-10-27T13:42:48+0100 |   Test   | apikey-1234567890-abcdefghijklmnopqrstuvwxyz-9876543210-ABCDEFGHIJKLMNOPQRSTUWXYZ-xxyz | LSWxxxx  |    cloudstack@leaseweb.tld     | secretkey-1234567890-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |        False        | enabled |      2      |                  | 80b23484-2573-4ce4-b7c6-145be7c34426 |   False   | a6acf153-6522-4c86-9b24-559ed50cfedc |
|   john.doe   |   LSW   | e7de501e-c5c7-460c-8063-d1ac82154dd1 |    John   | 2017-10-31T09:43:33+0100 |   Doe    | apikey-1234567890-abcdefghijklmnopqrstuvwxyz-9876543210-ABCDEFGHIJKLMNOPQRSTUWXYZ-xxxx | LSWxxxx  |  john.doe@global.example.tld   | secretkey-1234567890-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |        False        | enabled |      2      |                  | 433ec38f-d6e5-4351-b0c7-3fee9e5e11b4 |   False   | a6acf153-6522-4c86-9b24-559ed50cfedc |
+--------------+---------+--------------------------------------+-----------+--------------------------+----------+----------------------------------------------------------------------------------------+---------+--------------------------------+----------------------------------------------------------------------------------------+---------------------+---------+-------------+------------------+--------------------------------------+-----------+--------------------------------------+
(LeaseWeb)  >

Alternatively use it as a command line utility:

Usage – command line utility
$ cmk list users
count = 2
user:
+--------------+---------+--------------------------------------+-----------+--------------------------+----------+----------------------------------------------------------------------------------------+---------+--------------------------------+----------------------------------------------------------------------------------------+---------------------+---------+-------------+------------------+--------------------------------------+-----------+--------------------------------------+
|   username   | account |               domainid               | firstname |         created          | lastname |                                         apikey                                         |  domain |             email              |                                       secretkey                                        | iscallerchilddomain |  state  | accounttype |     timezone     |                  id                  | isdefault |              accountid               |
+--------------+---------+--------------------------------------+-----------+--------------------------+----------+----------------------------------------------------------------------------------------+---------+--------------------------------+----------------------------------------------------------------------------------------+---------------------+---------+-------------+------------------+--------------------------------------+-----------+--------------------------------------+
|     LSW      |   LSW   | e7de501e-c5c7-460c-8063-d1ac82154dd1 |  Leaseweb | 2015-10-27T13:42:48+0100 |   Test   | apikey-1234567890-abcdefghijklmnopqrstuvwxyz-9876543210-ABCDEFGHIJKLMNOPQRSTUWXYZ-xxyz | LSWxxxx |    cloudstack@leaseweb.tld     | secretkey-1234567890-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |        False        | enabled |      2      |                  | 80b23484-2573-4ce4-b7c6-145be7c34426 |   False   | a6acf153-6522-4c86-9b24-559ed50cfedc |
|   john.doe   |   LSW   | e7de501e-c5c7-460c-8063-d1ac82154dd1 |    John   | 2017-10-31T09:43:33+0100 |   Doe    | apikey-1234567890-abcdefghijklmnopqrstuvwxyz-9876543210-ABCDEFGHIJKLMNOPQRSTUWXYZ-xxxx | LSWxxxx |  john.doe@global.example.tld   | secretkey-1234567890-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |        False        | enabled |      2      |                  | 433ec38f-d6e5-4351-b0c7-3fee9e5e11b4 |   False   | a6acf153-6522-4c86-9b24-559ed50cfedc |
+--------------+---------+--------------------------------------+-----------+--------------------------+----------+----------------------------------------------------------------------------------------+---------+--------------------------------+----------------------------------------------------------------------------------------+---------------------+---------+-------------+------------------+--------------------------------------+-----------+--------------------------------------+
 

Cloudmonkey can also be scripted. Simply create a file with any number of commands on individual lines:

Usage – script
$ cat file-with-cmds
list users
list zones
$ cmk < file-with-cmds

For both the command line (with the –d switch) as well as in interactive mode, you can define how the output should get displayed JSON, XML, tabular, CSV, etc:

Usage - set output format
(LeaseWeb)  > set output table

A filter can be used to limit which fields will get returned :

Usage - filter to limit output
(LeaseWeb)  > list users filter=username,firstname,lastname,created
 
count = 2
user:
+--------------+----------+-----------+--------------------------+
|   username   | lastname | firstname |         created          |
+--------------+----------+-----------+--------------------------+
|    lswtest   |   Test   |  Leaseweb | 2015-10-27T13:42:48+0100 |
|   john.doe   |   Doe    |    John   | 2017-10-19T10:55:34+0200 |
+--------------+----------+-----------+--------------------------+

Certain API queries take optional request parameters. For instance, the listUsers API call allows among others to list of users by username. The % symbol allows wildcard matching:

Usage - matching to limit output
(LeaseWeb)  > list users username=john% filter=username,firstname,lastname,created
 
count = 1
user:
+--------------+----------+-----------+--------------------------+
|   username   | lastname | firstname |         created          |
+--------------+----------+-----------+--------------------------+
|   john.doe   |   Doe    |    John   | 2017-10-19T10:55:34+0200 |
+--------------+----------+-----------+--------------------------+

Automating common tasks 

Below are some code snippets, cloudmonkey, and shell scripts that illustrate how to perform common tasks.

Display what resources are assigned to a private cloud domain, what is currently in use, and how much is still available:

Automation - usage reporting
(LeaseWeb)  > list domains filter=cpulimit,memorylimit,primarystoragelimit
count = 1
domain:
+---------------------+----------+-------------+
| primarystoragelimit | cpulimit | memorylimit |
+---------------------+----------+-------------+
|         500         |    20    |    51200    |
+---------------------+----------+-------------+
(LeaseWeb)  > list domains filter=cputotal,memorytotal,primarystoragetotal
count = 1
domain:
+-------------+----------+---------------------+
| memorytotal | cputotal | primarystoragetotal |
+-------------+----------+---------------------+
|     1536    |    2     |          22         |
+-------------+----------+---------------------+
(LeaseWeb)  > list domains filter=cpuavailable,memoryavailable,primarystorageavailable
count = 1
domain:
+-------------------------+-----------------+--------------+
| primarystorageavailable | memoryavailable | cpuavailable |
+-------------------------+-----------------+--------------+
|           478           |      49664      |      18      |
+-------------------------+-----------------+--------------+
(LeaseWeb)  >

Reporting on users:

Automation - report on users
(LeaseWeb)  > list users filter=username,firstname,lastname,email,state,id
(LeaseWeb)  > list users state=disabled filter=username,firstname,lastname,email
(LeaseWeb)  > list users state=enabled filter=username,firstname,lastname,email

Create a new user under an existing Account, change the password, create API keys, and disable and remove an account. 

The createUser API function requires that the account is accompanied by a domain ID, so that will need to be retrieved first.

Automation - maintain users
(LeaseWeb)  > list accounts filter=domain,domainid,name
  count = 1
  account:
  +---------+--------------------------------------+------+
  |  domain |               domainid               | name |
  +---------+--------------------------------------+------+
  | LSWxxxx | e7de501e-c5c7-460c-8063-d1ac82154dd1 | LSW  |
  +---------+--------------------------------------+------+

(LeaseWeb)  > create user email='jane.doe@example.com' firstname='Jane' lastname='Doe' password='V3ry-S3cr3t' username='jane.doe' account=LSW  domainid=e7de501e-c5c7-460c-8063d1ac82154dd1
  user:
  id = c1bc02dd-9608-49a5-b124-1187b65debfb
  account = LSW 
  accountid = a6acf153-6522-4c86-9b24-559ed50cfedc
  accounttype = 2
  created = 2017-11-28T13:47:34+0100
  domain = LSWxxxx
  domainid = e7de501e-c5c7-460c-8063-d1ac82154dd1
  email = jane.doe@example.com
  firstname = Jane
  iscallerchilddomain = False
  isdefault = False
  lastname = Doe
  state = enabled
  username = jane.doe

(LeaseWeb)  > update user password='new-password' id=c1bc02dd-9608-49a5-b124-1187b65debfb

  user:
  id = c1bc02dd-9608-49a5-b124-1187b65debfb
  account = LSW 
  accountid = a6acf153-6522-4c86-9b24-559ed50cfedc
  accounttype = 2
  created = 2017-11-28T13:47:34+0100
  domain = LSWxxxx
  domainid = e7de501e-c5c7-460c-8063-d1ac82154dd1
  email = jane.doe@example.com
  firstname = Jane
  iscallerchilddomain = False
  isdefault = False
  lastname = Doe
  state = enabled
  username = jane.doe

(LeaseWeb)  > register userkeys id=c1bc02dd-9608-49a5-b124-1187b65debfb

  userkeys:
  apikey = v7rdS-Rh_eTTghrVLwdzMETpf5PpQYIPg6olRw-fLIVVBH64Jx9NH9KeH1mpx1wlSWEAqO0FJ0tOuav3ro0rrg
  secretkey = ynKZ5sOjHol-OLbv8mmB-oMO3NVceMBKRq1fVMuCn391UR4TWYxmPxmm_tvYFS6GaDwIMVPraToOXntcf-h3cA

(LeaseWeb)  > disable user id=c1bc02dd-9608-49a5-b124-1187b65debfb
  accountid = a6acf153-6522-4c86-9b24-559ed50cfedc
  cmd = org.apache.cloudstack.api.command.admin.user.DisableUserCmd
  created = 2017-11-28T13:48:34+0100
  jobid = 298a1000-1f08-4604-a38f-cf219f7f388d
  jobprocstatus = 0
  jobresult:
  user:
  id = c1bc02dd-9608-49a5-b124-1187b65debfb
  account = LSW 
  accountid = a6acf153-6522-4c86-9b24-559ed50cfedc
  accounttype = 2
  apikey = v7rdS-Rh_eTTghrVLwdzMETpf5PpQYIPg6olRw-fLIVVBH64Jx9NH9KeH1mpx1wlSWEAqO0FJ0tOuav3ro0rrg
  created = 2017-11-28T13:47:34+0100
  domain = LSWxxxx
  domainid = e7de501e-c5c7-460c-8063-d1ac82154dd1
  email = jane.doe@example.com
  firstname = Jane
  iscallerchilddomain = False
  isdefault = False
  lastname = Doe
  secretkey = ynKZ5sOjHol-OLbv8mmB-oMO3NVceMBKRq1fVMuCn391UR4TWYxmPxmm_tvYFS6GaDwIMVPraToOXntcf-h3cA
  state = disabled
  username = jane.doe
  jobresultcode = 0
  jobresulttype = object
  jobstatus = 1
  userid = ab6fc0c7-dfb8-44ec-a74e-1ebb6e566bc7

(LeaseWeb)  > delete user id=c1bc02dd-9608-49a5-b124-1187b65debfb

  success = true

(LeaseWeb)  >


Keywords
You can click on any of the keywords below this article to see all related articles for that keyword