Rally 是一套 OpenStack benchmark tool。關於 Rally 的介紹本文就不再贅述。本篇安裝時是以 OpenStack 官方 Wiki 的教學為參考內容，紀錄安裝與使用 Rally 的筆記。

實驗環境

Item	Value
Operating System	Ubuntu 12.04 x64
Python version	Python 2.7
OpenStack version	Grizzly (2013)

Install Rally

一開始是想要安裝在 OpenStack controller node 上，但是一直裝不起來，看起來似乎是 python module 的版本問題，不過我沒有仔細研究到底是哪個 module 的哪個版本卡住。

直接在 OpenStack 上開了一個新的 VM，一樣是選擇 Ubuntu 12.04，就可以直接安裝。(不過現在回想起來，應該要用 virtualenv 來裝就可以解決，也不會影響原本的系統。)

PS: Rally 可以跟 DevStack 一起部署。參考: Rally with DevStack all in one installation

Step:

下載 source code

root@rally:~# git clone https://git.openstack.org/stackforge/rally

執行 script

root@rally:~# ./rally/install_rally.sh

安裝並執行 tox (當前版本可以跳過此步驟)

root@rally:~# pip install 'tox==1.6.1'
root@rally:~# tox

Deployment initialization

安裝完 Rally 之後，再進行下一步之前，必須要確定該 host 可以使用 OpenStack API

建立 OpenStack credentials (openrc)，然後看看 keystone client 是否可已使用:

建立 RC file:

export OS_TENANT_NAME=admin
export OS_USERNAE=admin
export OS_PASSWORD=myStackAdmin
export OS_AUTH_URL="http://myopenstack:5000/v2.0/"
export OS_SERVICE_ENDPOINT="http://myopenstack:35357/v2.0"
export OS_SERVICE_TOKEN=myOpenStackAdminToken28143

測試 keystone 指令是否能使用: (keystone client 已經在 install_rally.sh 的時候就安裝了，不需要額外裝)

root@localhost:~# source openrc
root@localhost:~# keystone tenant-list
... (省略) ...

接下來要設定 deployment

Step 1

首先，複製 samples 裡面的 existing.json，並修改參數

root@rally:~# cp rally/samples/deployments/existing.json ./

{
    "type": "ExistingCloud",
    "auth_url": "http://myopenstack:5000/v2.0/",
    "region_name": "RegionOne",
    "endpoint_type": "public",
    "admin": {
        "username": "admin",
        "password": "myStackAdmin",
        "tenant_name": "benchmark"
    }
}

type 的部分不能換改
auth_url 這個參數只要使用 public url 的就可以了
User 一定要是具有 admin role 的 user；tenant 則不一定要特地建立專門使用的，因為測試的過程不會使用到自己的 tenant。

接著執行命令 rally deployment create --filename=<json file> --name=<name of Openstack deployment>

root@rally:~# rally deployment create --filename existing.json --name myStack
+--------------------------------------+----------------------------+---------+------------------+--------+
| uuid                                 | created_at                 | name    | status           | active |
+--------------------------------------+----------------------------+---------+------------------+--------+
| 3070593f-c583-4bcf-85a8-7dc7345ad146 | 2015-02-24 03:45:37.088393 | myStack | deploy->finished |        |
+--------------------------------------+----------------------------+---------+------------------+--------+
Using deployment: 3070593f-c583-4bcf-85a8-7dc7345ad146
~/.rally/openrc was updated

HINTS:
* To get your cloud resources, run:
	rally show [flavors|images|keypairs|networks|secgroups]

* To use standard OpenStack clients, set up your env by running:
	source ~/.rally/openrc
  OpenStack clients are now configured, e.g run:
	glance image-list
root@rally:~#

這時候執行 rally show [option] 可以看到 Openstack 資訊

root@rally:~# rally show images
+--------------------------------------+------------------------------+------------+
| UUID                                 | Name                         | Size (B)   |
+--------------------------------------+------------------------------+------------+
| 53c59f00-509a-4056-b5b2-ff1984c9323f | Ubuntu 14.04 (Cloud Image)   | 256836096  |
| 75426fa5-f73b-46a4-84fa-4a44f9cbc9fb | Ubuntu 12.04 (Cloud Image)   | 261423616  |
| 7bd19a5c-0eba-439c-a627-599df9d9ad4d | CentOS 7                     | 418688512  |
+--------------------------------------+------------------------------+------------+
root@rally:~#

Step 2

接下來，執行 rally deployment check

root@rally:~# rally deployment check
keystone endpoints are valid and following services are available:
+----------+----------+-----------+
| services | type     | status    |
+----------+----------+-----------+
| cinder   | volume   | Available |
| ec2      | ec2      | Available |
| glance   | image    | Available |
| keystone | identity | Available |
| nova     | compute  | Available |
| quantum  | network  | Available |
+----------+----------+-----------+
root@rally:~#

Benchmarking OpenStack

接下來跑一個 benchmark 的範例

Step 1

從 sample 中複製 benchmark task 範例

cp samples/tasks/scenarios/nova/boot-and-delete.json ./task-1.json

修改 task 內容

{
    "NovaServers.boot_and_delete_server": [
        {
            "args": {
                "flavor": {
                    "name": "m1.tiny"
                },
                "image": {
                    "name": "CirrOS 0.3.3"
                },
                "force_delete": false
            },
            "runner": {
                "type": "constant",
                "times": 10,
                "concurrency": 2
            },
            "context": {
                "users": {
                    "tenants": 3,
                    "users_per_tenant": 2
                }
            }
        }
    ]
}

Flavor name、image name 可以用 rally show [option] 去抓取。另外，根據 OpenStack Wiki 的資料， flavor 與 image 可以改用提供 id、uuid 的方式。

"args": {
   "flavor_id": 42,
    "image_id": "73257560-c59b-4275-a1ec-ab140e5b9979"
},

Step 2

接下來，執行 rally -v task start <task json file> --tag <task name>

最後會出現:

root@rally:~# rally -v task start task-1.json --tag "Task 1"

... (過程省略) ...

--------------------------------------------------------------------------------
Task 9148f0ee-d5d6-42a7-a44f-d8cd81b6d309: finished
--------------------------------------------------------------------------------

test scenario NovaServers.boot_and_delete_server
args position 0
args values:
{u'args': {u'flavor': {u'name': u'm1.tiny'},
           u'force_delete': False,
           u'image': {u'name': u'CirrOS 0.3.3'}},
 u'context': {u'users': {u'project_domain': u'default',
                         u'resource_management_workers': 30,
                         u'tenants': 3,
                         u'user_domain': u'default',
                         u'users_per_tenant': 2}},
 u'runner': {u'concurrency': 2, u'times': 10, u'type': u'constant'}}
+--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
| action             | min (sec) | avg (sec) | max (sec) | 90 percentile | 95 percentile | success | count |
+--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
| nova.boot_server   | 6.708     | 13.289    | 37.405    | 34.299        | 35.852        | 100.0%  | 10    |
| nova.delete_server | 2.347     | 3.872     | 5.509     | 4.932         | 5.22          | 100.0%  | 10    |
| total              | 9.408     | 17.16     | 39.752    | 38.566        | 39.159        | 100.0%  | 10    |
+--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
Load duration: 85.8980619907
Full duration: 97.1042778492

HINTS:
* To plot HTML graphics with this data, run:
	rally task report 9148f0ee-d5d6-42a7-a44f-d8cd81b6d309 --out output.html

* To get raw JSON output of task results, run:
	rally task results 9148f0ee-d5d6-42a7-a44f-d8cd81b6d309

Using task: 9148f0ee-d5d6-42a7-a44f-d8cd81b6d309
root@rally:~#

跑完之後就會出現如上方範例的簡易報告，如果想要詳盡的報告內容，可以按照提示執行 rally task report <uuid> --out <html file> 就可以產生 html 檔案。

Benchmark 參數說明請參考: https://wiki.openstack.org/wiki/Rally/Concepts

另外，Rally 官方也提供了安裝在 docker 的方法，以及如果 Rally 是安裝在 OpenStack controller 時，也有幾個比較簡易的步驟可以參考: http://rally.readthedocs.org/en/latest/tutorial.html

Troubleshooting

Schema validation error

執行命令 rally deployment create --filename=<json file> --name=<name of Openstack deployment> 時發生以下錯誤:

root@rally:~# rally deployment create --filename=mystack.json --name=myStack
2015-02-24 03:11:33.579 23212 ERROR rally.api [-] Deployment 59dc5f2c-a83a-42d8-bf49-777ff5e32783: Schema validation error.
Config schema validation error: {'admin': {'username': 'admin', 'password': 'myStackAdmin', 'tenant': 'benchmark'}, 'auth_url': 'http://myopenstack:5000/v2.0/', 'type': 'ExistingCloud'} is not valid under any of the given schemas

Failed validating 'anyOf' in schema:
    {'anyOf': [{'properties': {'admin': {'$ref': '#/definitions/user'}},
                'required': ['type', 'auth_url', 'admin']},
               {'required': ['type', 'auth_url', 'users'],
                'users': {'items': {'$ref': '#/definitions/user'},
                          'type': 'array'}}],
     'definitions': {'user': {'oneOf': [{'properties': {'tenant_name': {'type': 'string'}},
                                         'required': ['username',
                                                      'password',
                                                      'tenant_name']},
                                        {'properties': {'project_domain_name': {'type': 'string'},
                                                        'project_name': {'type': 'string'},
                                                        'user_domain_name': {'type': 'string'}},
                                         'required': ['username',
                                                      'password',
                                                      'project_name']}],
                              'properties': {'password': {'type': 'string'},
                                             'username': {'type': 'string'}},
                              'type': 'object'}},
     'properties': {'auth_url': {'type': 'string'},
                    'endpoint_type': {'enum': ['admin',
                                               'internal',
                                               'public'],
                                      'type': 'string'},
                    'region_name': {'type': 'string'},
                    'type': {'type': 'string'}},
     'type': 'object'}

On instance:
    {'admin': {'password': 'myStackAdmin',
               'tenant': 'benchmark',
               'username': 'admin'},
     'auth_url': 'http://myopenstack:5000/v2.0/',
     'type': 'ExistingCloud'}.
root@rally:~#

這時候執行 rally deployment list 可以看到 deployment 的狀態是 failed

root@rally:~# rally deployment list
+--------------------------------------+----------------------------+---------+----------------+--------+
| uuid                                 | created_at                 | name    | status         | active |
+--------------------------------------+----------------------------+---------+----------------+--------+
| 59dc5f2c-a83a-42d8-bf49-777ff5e32783 | 2015-02-24 03:11:33.195680 | myStack | deploy->failed |        |
+--------------------------------------+----------------------------+---------+----------------+--------+
root@rally:~#

雖然這時候，執行 rally deployment recreate --deployment <uuid of deployment> 就會看似恢復正常。

root@rally:~# rally deployment recreate --deployment 59dc5f2c-a83a-42d8-bf49-777ff5e32783
root@rally:~# rally deployment list
+--------------------------------------+----------------------------+---------+------------------+--------+
| uuid                                 | created_at                 | name    | status           | active |
+--------------------------------------+----------------------------+---------+------------------+--------+
| 59dc5f2c-a83a-42d8-bf49-777ff5e32783 | 2015-02-24 03:11:33.195680 | myStack | deploy->finished |        |
+--------------------------------------+----------------------------+---------+------------------+--------+
root@rally:~# rally use deployment --deployment 59dc5f2c-a83a-42d8-bf49-777ff5e32783
Using deployment: 59dc5f2c-a83a-42d8-bf49-777ff5e32783
~/.rally/openrc was updated

HINTS:
* To get your cloud resources, run:
	rally show [flavors|images|keypairs|networks|secgroups]

* To use standard OpenStack clients, set up your env by running:
	source ~/.rally/openrc
  OpenStack clients are now configured, e.g run:
	glance image-list
root@rally:~#

但是要做其他動作就會發生錯誤，例如:

root@rally:~# rally deployment check
Authentication Issues: user rally doesn't have 'admin' role.

這是因為OpenStack Wiki 教學文裡，deployment.json 少了一些參數。如果是直接從 source code 裡面的 sample 中複製並修改，就不會發生這個問題。

ConnectionRefused: Unable to establish connection OpenStack API endpoint

root@rally:~# rally deployment check
keystone endpoints are valid and following services are available:
Command failed, please check log for more info
2015-02-24 03:48:00.930 24945 CRITICAL rally [-] ConnectionRefused: Unable to establish connection to http://quanta-42.sslab.cs.nthu.edu.tw:35357/v2.0/OS-KSADM/services
2015-02-24 03:48:00.930 24945 TRACE rally Traceback (most recent call last):
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/bin/rally", line 10, in <module>
2015-02-24 03:48:00.930 24945 TRACE rally     sys.exit(main())
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/rally/cmd/main.py", line 42, in main
2015-02-24 03:48:00.930 24945 TRACE rally     return cliutils.run(sys.argv, categories)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/rally/cmd/cliutils.py", line 318, in run
2015-02-24 03:48:00.930 24945 TRACE rally     ret = fn(*fn_args, **fn_kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "<string>", line 2, in check
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/rally/cmd/envutils.py", line 66, in default_from_global
2015-02-24 03:48:00.930 24945 TRACE rally     return f(*args, **kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/rally/cmd/commands/deployment.py", line 248, in check
2015-02-24 03:48:00.930 24945 TRACE rally     for service in client.services.list():
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/v2_0/services.py", line 32, in list
2015-02-24 03:48:00.930 24945 TRACE rally     return self._list("/OS-KSADM/services", "OS-KSADM:services")
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py", line 113, in _list
2015-02-24 03:48:00.930 24945 TRACE rally     resp, body = self.client.get(url, **kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 164, in get
2015-02-24 03:48:00.930 24945 TRACE rally     return self.request(url, 'GET', **kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 200, in request
2015-02-24 03:48:00.930 24945 TRACE rally     resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 89, in request
2015-02-24 03:48:00.930 24945 TRACE rally     return self.session.request(url, method, **kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
2015-02-24 03:48:00.930 24945 TRACE rally     return func(*args, **kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 369, in request
2015-02-24 03:48:00.930 24945 TRACE rally     resp = send(**kwargs)
2015-02-24 03:48:00.930 24945 TRACE rally   File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 412, in _send_request
2015-02-24 03:48:00.930 24945 TRACE rally     raise exceptions.ConnectionRefused(msg)
2015-02-24 03:48:00.930 24945 TRACE rally ConnectionRefused: Unable to establish connection to http://quanta-42:35357/v2.0/OS-KSADM/services
2015-02-24 03:48:00.930 24945 TRACE rally
root@rally:~#

我會發生這個問題是因為，我將 Rally 安裝在 OpenStack 之外，而 rally deployment check 會 call internal url (或是 admin url 我不是很確定)而非 public url。不過解決方法很簡單，只要將 IP 與 hostname 對應設定到 /etc/hosts 上即可。

Install Rally 筆記 (OpenStack benchmark)

實驗環境

Install Rally

Deployment initialization

Step 1

Step 2

Benchmarking OpenStack

Step 1

Step 2

Troubleshooting

Schema validation error

ConnectionRefused: Unable to establish connection OpenStack API endpoint