注意要 按顺序 搭建服务，在 CentOS 7 安装 Openstack Rocky 版本 - 环境搭建 的基础上安装服务。

Object Storage service - 对象存储服务（Swift）

在 控制节点 和 对象存储节点 安装

对象存储节点（管理网络）

  • object1：10.0.0.51/24
  • object2：10.0.0.52/24

控制节点

For simplicity, this guide installs and configures the proxy service on the controller node.

身份认证和 API 配置

创建用户 swift 并添加到 admin 角色

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 更新环境变量
. admin-openrc

# 创建用户，设置密码为 swift
openstack user create --domain default --password-prompt swift
# User Password:
# Repeat User Password:
# +---------------------+----------------------------------+
# | Field               | Value                            |
# +---------------------+----------------------------------+
# | domain_id           | default                          |
# | enabled             | True                             |
# | id                  | 0b89a0baf12a4e9a9d944dc4f04e850f |
# | name                | swift                            |
# | options             | {}                               |
# | password_expires_at | None                             |
# +---------------------+----------------------------------+


# 将用户添加到 admin 角色
openstack role add --project service --user swift admin
# 无输出

创建 swift 服务实体和端点（endpoint）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# 创建服务实体
openstack service create --name swift --description "OpenStack Object Storage" object-store
# +-------------+----------------------------------+
# | Field       | Value                            |
# +-------------+----------------------------------+
# | description | OpenStack Object Storage         |
# | enabled     | True                             |
# | id          | cd7bec7ce2e34ab894977827cb87f064 |
# | name        | swift                            |
# | type        | object-store                     |
# +-------------+----------------------------------+


# 创建服务 API 端点（endpoint）
openstack endpoint create --region RegionOne object-store public http://controller:8080/v1/AUTH_%\(project_id\)s
# +--------------+-----------------------------------------------+
# | Field        | Value                                         |
# +--------------+-----------------------------------------------+
# | enabled      | True                                          |
# | id           | a9bc53548cec4e85bd58cafc736104ce              |
# | interface    | public                                        |
# | region       | RegionOne                                     |
# | region_id    | RegionOne                                     |
# | service_id   | cd7bec7ce2e34ab894977827cb87f064              |
# | service_name | swift                                         |
# | service_type | object-store                                  |
# | url          | http://controller:8080/v1/AUTH_%(project_id)s |
# +--------------+-----------------------------------------------+


openstack endpoint create --region RegionOne object-store internal http://controller:8080/v1/AUTH_%\(project_id\)s
# +--------------+-----------------------------------------------+
# | Field        | Value                                         |
# +--------------+-----------------------------------------------+
# | enabled      | True                                          |
# | id           | e8ddbfbd46dd4b81932abaafd71f52e3              |
# | interface    | internal                                      |
# | region       | RegionOne                                     |
# | region_id    | RegionOne                                     |
# | service_id   | cd7bec7ce2e34ab894977827cb87f064              |
# | service_name | swift                                         |
# | service_type | object-store                                  |
# | url          | http://controller:8080/v1/AUTH_%(project_id)s |
# +--------------+-----------------------------------------------+


openstack endpoint create --region RegionOne object-store admin http://controller:8080/v1
# +--------------+----------------------------------+
# | Field        | Value                            |
# +--------------+----------------------------------+
# | enabled      | True                             |
# | id           | 33287f8172a645a884ab5931fa2ef61c |
# | interface    | admin                            |
# | region       | RegionOne                        |
# | region_id    | RegionOne                        |
# | service_id   | cd7bec7ce2e34ab894977827cb87f064 |
# | service_name | swift                            |
# | service_type | object-store                     |
# | url          | http://controller:8080/v1        |
# +--------------+----------------------------------+

安装和配置 swift 客户端

巨坑 x 2

安装软件包

1
2
3
4
# 安装（完整的 OpenStack 环境已经包含其中的一部分包）
yum install openstack-swift-proxy python-swiftclient \
  python-keystoneclient python-keystonemiddleware \
  memcached -y

下载配置文件

1
2
# 从对象存储源存储库获取代理服务配置文件
curl -o /etc/swift/proxy-server.conf https://opendev.org/openstack/swift/raw/branch/stable/rocky/etc/proxy-server.conf-sample

编辑配置文件 /etc/swift/proxy-server.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift

[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server

[app:proxy-server]
use = egg:swift#proxy
account_autocreate = true

[filter:keystoneauth]
use = egg:swift#keystoneauth
# user -> myrole
operator_roles = admin,myrole

[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
www_authenticate_uri = http://controller:5000
# 35357 -> 5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = swift
password = swift
delay_auth_decision = True

[filter:cache]
use = egg:swift#memcache
memcache_servers = controller:11211

对象存储节点

For simplicity, this configuration references two storage nodes, each containing two empty local block storage devices. The instructions use /dev/sdb and /dev/sdc, but you can substitute different values for your particular nodes.

在 所有对象存储节点 执行相同操作。

创建存储设备

添加 2 块新硬盘

安装软件包

1
2
# 安装
yum install xfsprogs rsync -y

格式化硬盘，并创建挂载点

1
2
3
4
5
6
7
# 将设备格式化为 XFS 文件系统
mkfs.xfs /dev/sdb
mkfs.xfs /dev/sdc

# 创建挂载点目录结构
mkdir -p /srv/node/sdb
mkdir -p /srv/node/sdc

编辑 /etc/fstab 文件系统信息

1
2
3
# 添加内容
/dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
/dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2

挂载设备

1
2
3
# 挂载
mount /srv/node/sdb
mount /srv/node/sdc

编辑配置文件 /etc/rsyncd.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# 添加配置，注意修改 IP 地址
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 10.0.0.51

[account]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/account.lock

[container]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/container.lock

[object]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/object.lock

启用同步服务

1
2
3
# 启用服务
systemctl enable rsyncd.service
systemctl start rsyncd.service

安装和配置 swift

安装软件包

1
2
# 安装
yum install openstack-swift-account openstack-swift-container openstack-swift-object -y

下载配置文件

1
2
3
4
# 下载
curl -o /etc/swift/account-server.conf https://opendev.org/openstack/swift/raw/branch/stable/rocky/etc/account-server.conf-sample
curl -o /etc/swift/container-server.conf https://opendev.org/openstack/swift/raw/branch/stable/rocky/etc/container-server.conf-sample
curl -o /etc/swift/object-server.conf https://opendev.org/openstack/swift/raw/branch/stable/rocky/etc/object-server.conf-sample

编辑账户服务配置文件 /etc/swift/account-server.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 注意修改 IP 地址
[DEFAULT]
bind_ip = 10.0.0.51
bind_port = 6202
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = true

[pipeline:main]
pipeline = healthcheck recon account-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

编辑容器服务配置文件， /etc/swift/container-server.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 注意修改 IP 地址
[DEFAULT]
bind_ip = 10.0.0.51
bind_port = 6201
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = true

[pipeline:main]
pipeline = healthcheck recon container-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

编辑对象服务配置文件 /etc/swift/object-server.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 注意修改 IP 地址
[DEFAULT]
bind_ip = 10.0.0.51
bind_port = 6200
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = true

[pipeline:main]
pipeline = healthcheck recon object-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock

设置目录权限

1
2
3
4
5
6
7
# 设置挂载点目录的权限
chown -R swift:swift /srv/node

# 创建 recon 目录并设置权限
mkdir -p /var/cache/swift
chown -R root:swift /var/cache/swift
chmod -R 775 /var/cache/swift

创建和分发初始化 rings

在 控制节点 上执行操作

创建账户 ring

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 切换目录
cd /etc/swift

# 创建 account.builder
swift-ring-builder account.builder create 10 3 1

# 将每个存储节点的存储设备添加到 ring 中
swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.0.0.51 --port 6202 --device sdb --weight 100
swift-ring-builder account.builder add --region 1 --zone 1 --ip 10.0.0.51 --port 6202 --device sdc --weight 100
swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.0.0.52 --port 6202 --device sdb --weight 100
swift-ring-builder account.builder add --region 1 --zone 2 --ip 10.0.0.52 --port 6202 --device sdc --weight 100

# 验证 ring 的内容
swift-ring-builder account.builder
# account.builder, build version 4, id 9cea989e0c1e41f3a801ae80bd5906bf
# 1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
# The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining)
# The overload factor is 0.00% (0.000000)
# Ring file account.ring.gz not found, probably it hasn't been written yet
# Devices:   id region zone ip address:port replication ip:port  name weight partitions balance flags meta
#             0      1    1  10.0.0.51:6202      10.0.0.51:6202   sdb 100.00          0 -100.00

#             1      1    1  10.0.0.51:6202      10.0.0.51:6202   sdc 100.00          0 -100.00

#             2      1    2  10.0.0.52:6202      10.0.0.52:6202   sdb 100.00          0 -100.00

#             3      1    2  10.0.0.52:6202      10.0.0.52:6202   sdc 100.00          0 -100.00


# 平衡 ring
swift-ring-builder account.builder rebalance
# Reassigned 3072 (300.00%) partitions. Balance is now 0.00.  Dispersion is now 0.00

创建容器 ring

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 切换目录
cd /etc/swift

# 创建 container.builder
swift-ring-builder container.builder create 10 3 1

# 将每个存储节点的存储设备添加到 ring 中
swift-ring-builder container.builder add --region 1 --zone 1 --ip 10.0.0.51 --port 6201 --device sdb --weight 100
swift-ring-builder container.builder add --region 1 --zone 1 --ip 10.0.0.51 --port 6201 --device sdc --weight 100
swift-ring-builder container.builder add --region 1 --zone 2 --ip 10.0.0.52 --port 6201 --device sdb --weight 100
swift-ring-builder container.builder add --region 1 --zone 2 --ip 10.0.0.52 --port 6201 --device sdc --weight 100

# 验证 ring 的内容
swift-ring-builder container.builder
# container.builder, build version 4, id 06866745c9ad488eaeeb7919e03d15a8
# 1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
# The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining)
# The overload factor is 0.00% (0.000000)
# Ring file container.ring.gz not found, probably it hasn't been written yet
# Devices:   id region zone ip address:port replication ip:port  name weight partitions balance flags meta
#             0      1    1  10.0.0.51:6201      10.0.0.51:6201   sdb 100.00          0 -100.00

#             1      1    1  10.0.0.51:6201      10.0.0.51:6201   sdc 100.00          0 -100.00

#             2      1    2  10.0.0.52:6201      10.0.0.52:6201   sdb 100.00          0 -100.00

#             3      1    2  10.0.0.52:6201      10.0.0.52:6201   sdc 100.00          0 -100.00


# 平衡 ring
swift-ring-builder container.builder rebalance
# Reassigned 3072 (300.00%) partitions. Balance is now 0.00.  Dispersion is now 0.00

创建对象 ring

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 切换目录
cd /etc/swift

# 创建 object.builder
swift-ring-builder object.builder create 10 3 1

# 将每个存储节点的存储设备添加到 ring 中
swift-ring-builder object.builder add --region 1 --zone 1 --ip 10.0.0.51 --port 6200 --device sdb --weight 100
swift-ring-builder object.builder add --region 1 --zone 1 --ip 10.0.0.51 --port 6200 --device sdc --weight 100
swift-ring-builder object.builder add --region 1 --zone 2 --ip 10.0.0.52 --port 6200 --device sdb --weight 100
swift-ring-builder object.builder add --region 1 --zone 2 --ip 10.0.0.52 --port 6200 --device sdc --weight 100

# 验证 ring 的内容
swift-ring-builder object.builder
# object.builder, build version 4, id 6a9fa933150246c1a98d214a4272a771
# 1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
# The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining)
# The overload factor is 0.00% (0.000000)
# Ring file object.ring.gz not found, probably it hasn't been written yet
# Devices:   id region zone ip address:port replication ip:port  name weight partitions balance flags meta
#             0      1    1  10.0.0.51:6200      10.0.0.51:6200   sdb 100.00          0 -100.00

#             1      1    1  10.0.0.51:6200      10.0.0.51:6200   sdc 100.00          0 -100.00

#             2      1    2  10.0.0.52:6200      10.0.0.52:6200   sdb 100.00          0 -100.00

#             3      1    2  10.0.0.52:6200      10.0.0.52:6200   sdc 100.00          0 -100.00


# 平衡 ring
swift-ring-builder object.builder rebalance
# Reassigned 3072 (300.00%) partitions. Balance is now 0.00.  Dispersion is now 0.00

分发配置文件

将 /etc/swift 目录中的以下文件复制到每个对象存储节点的 /etc/swift 目录

1
2
3
account.ring.gz
container.ring.gz
object.ring.gz

使用 scp 复制

1
2
3
#
scp account.ring.gz container.ring.gz object.ring.gz root@object1:/etc/swift
scp account.ring.gz container.ring.gz object.ring.gz root@object2:/etc/swift

完成安装

在 控制节点 上执行操作

下载配置文件

1
2
# 下载
curl -o /etc/swift/swift.conf https://opendev.org/openstack/swift/raw/branch/stable/rocky/etc/swift.conf-sample

编辑配置文件 /etc/swift/swift.conf

1
2
3
4
5
6
7
8
# HASH 为唯一值
[swift-hash]
swift_hash_path_suffix = HASH_PATH_SUFFIX
swift_hash_path_prefix = HASH_PATH_PREFIX

[storage-policy:0]
name = Policy-0
default = yes

将该配置文件复制到每个对象存储节点的 /etc/swift 目录

1
2
scp /etc/swift/swift.conf root@object1:/etc/swift
scp /etc/swift/swift.conf root@object2:/etc/swift

在 所有节点 上设置 swift 配置文件目录的权限（如果有）

  • 控制节点、对象存储节点
1
chown -R root:swift /etc/swift

在 控制节点 启用对象存储代理服务

1
2
systemctl enable openstack-swift-proxy.service memcached.service
systemctl restart openstack-swift-proxy.service memcached.service

在所有 对象存储节点 启用服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 账户
systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service openstack-swift-account-reaper.service openstack-swift-account-replicator.service

systemctl start openstack-swift-account.service openstack-swift-account-auditor.service openstack-swift-account-reaper.service openstack-swift-account-replicator.service

# 容器
systemctl enable openstack-swift-container.service openstack-swift-container-auditor.service openstack-swift-container-replicator.service openstack-swift-container-updater.service

systemctl start openstack-swift-container.service openstack-swift-container-auditor.service openstack-swift-container-replicator.service openstack-swift-container-updater.service

# 对象
systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service

systemctl start openstack-swift-object.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service

验证

在 控制节点 上执行操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# 更新环境变量
. demo-openrc

# 查看服务状态
swift stat
#                Account: AUTH_eb296de9a0e64b2a9243ad58f0805746
#             Containers: 0
#                Objects: 0
#                  Bytes: 0
#        X-Put-Timestamp: 1608975906.47918
#            X-Timestamp: 1608975906.47918
#             X-Trans-Id: txbe36e2a0c3e8402b9530b-005fe70622
#           Content-Type: text/plain; charset=utf-8
# X-Openstack-Request-Id: txbe36e2a0c3e8402b9530b-005fe70622


# 创建 container1 容器
openstack container create container1
# +---------------------------------------+------------+------------------------------------+
# | account                               | container  | x-trans-id                         |
# +---------------------------------------+------------+------------------------------------+
# | AUTH_eb296de9a0e64b2a9243ad58f0805746 | container1 | tx5c9f73c7f8d540f2a9308-005fe70635 |
# +---------------------------------------+------------+------------------------------------+


# 创建测试文件
echo 'Hello OpenStack!' > test.txt

# 上传测试文件到 container1 容器中
openstack object create container1 test.txt
# +----------+------------+----------------------------------+
# | object   | container  | etag                             |
# +----------+------------+----------------------------------+
# | test.txt | container1 | f398d3c700c2fb2c13d2714660691b97 |
# +----------+------------+----------------------------------+


# 列出 container1 容器中的文件
openstack object list container1
# +----------+
# | Name     |
# +----------+
# | test.txt |
# +----------+


# 从 container1 容器下载文件
openstack object save container1 test.txt