前言

这段时间都在读 OpenStack 组件源码,主要依赖官方文档和《OpenStack 设计与实现》,目前这部分的工作告一段落,稍微整理一下阅读源码的方法,主要是如何找到程序的入口。

Kolla-Ansible 搭建环境使用的是 Victoria 版本的源码,主要包含以下几个组件:

  • Keystone
  • Glance
  • Nova
  • Neutron
  • Heat

源码阅读

实际上,程序的入口从 setup.cfg 文件就可以看出来了,如何处理请求会涉及到 paste.ini 配置文件。有些组件比较复杂(比如 nova、neutron),组件本身包含多个子组件,有 wsgi 应用、OS-Ken 应用等,启动方式也并不统一,因此需要深入源码才能找到真正的启动位置和启动方式。

此外,组件不仅有对外提供的 RESTful API 接口,组件内部和组件之间还有 RPC 调用,会涉及消息队列(一般是 rabbitmq)和 socket 通信,而这也是需要深入源码才能理清的。

setup.cfg

上述 OpenStack 组件都包含 setup.cfg 文件,Setuptools 工具使用该配置文件设置包的元数据和其他选项,具体的配置项可以在 文档 中查阅。

这里主要关注的是 entry_points 小节,可以找到代码的入口点,组件启动的方式包括 console_scriptswsgi_scripts ,分别表示命令行脚本和 wsgi 脚本,通常 wsgi 脚本通过 Apache + mod_wsgi 调用。

以 glance 为例,glance-api 可以使用命令行脚本启动也可以使用 wsgi 脚本启动,不过官方建议在生产环境中使用命令行脚本启动。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[entry_points]
console_scripts =
glance-api = glance.cmd.api:main
glance-cache-prefetcher = glance.cmd.cache_prefetcher:main
glance-cache-pruner = glance.cmd.cache_pruner:main
glance-cache-manage = glance.cmd.cache_manage:main
glance-cache-cleaner = glance.cmd.cache_cleaner:main
glance-control = glance.cmd.control:main
glance-manage = glance.cmd.manage:main
glance-replicator = glance.cmd.replicator:main
glance-scrubber = glance.cmd.scrubber:main
glance-status = glance.cmd.status:main
wsgi_scripts =
glance-wsgi-api = glance.common.wsgi_app:init_app

等号右边可以理解为调用的函数,以 glance-api = glance.cmd.api:main 为例,定位源码 glance/cmd/api.py 中的 main 函数。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
def main():
try:
config.parse_args() # 读取配置
config.set_config_defaults() # 设置默认配置
wsgi.set_eventlet_hub() # 设置 eventlet.hub
logging.setup(CONF, 'glance') # 日志
gmr.TextGuruMeditation.setup_autorun(version)
notifier.set_defaults()

if CONF.profiler.enabled: # OSProfiler
osprofiler.initializer.init_from_conf(
conf=CONF,
context={},
project="glance",
service="api",
host=CONF.bind_host
)

# NOTE(danms): Configure system-wide threading model to use eventlet
glance.async_.set_threadpool_model('eventlet') # 设置同步线程池模型

# NOTE(abhishekk): Added initialize_prefetcher KW argument to Server
# object so that prefetcher object should only be initialized in case
# of API service and ignored in case of registry. Once registry is
# removed this parameter should be removed as well.
initialize_prefetcher = False
if CONF.paste_deploy.flavor == 'keystone+cachemanagement':
initialize_prefetcher = True
server = wsgi.Server(initialize_glance_store=True, # wsgi 应用
initialize_prefetcher=initialize_prefetcher)
server.start(config.load_paste_app('glance-api'), default_port=9292) # 启动 wsgi 应用
server.wait() # 等待启动完成
except Exception as e:
fail(e)

再来看一下 wsgi 脚本 glance-wsgi-api = glance.common.wsgi_app:init_app,定位源码 glance/common/wsgi_app.py 的 init_app 函数。重点关注的是最后使用 Paste Deploy 加载 wsgi 应用。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
def init_app():
config.set_config_defaults()
config_files = _get_config_files()
CONF([], project='glance', default_config_files=config_files)
logging.setup(CONF, "glance")

# NOTE(danms): We are running inside uwsgi or mod_wsgi, so no eventlet;
# use native threading instead.
glance.async_.set_threadpool_model('native')
atexit.register(drain_threadpools)

# NOTE(danms): Change the default threadpool size since we
# are dealing with native threads and not greenthreads.
# Right now, the only pool of default size is tasks_pool,
# so if others are created this will need to change to be
# more specific.
common.DEFAULT_POOL_SIZE = CONF.wsgi.task_pool_threads

if CONF.enabled_backends:
if store_utils.check_reserved_stores(CONF.enabled_backends):
msg = _("'os_glance_' prefix should not be used in "
"enabled_backends config option. It is reserved "
"for internal use only.")
raise RuntimeError(msg)
glance_store.register_store_opts(CONF, reserved_stores=RESERVED_STORES)
glance_store.create_multi_stores(CONF, reserved_stores=RESERVED_STORES)
glance_store.verify_store()
else:
glance_store.register_opts(CONF)
glance_store.create_stores(CONF)
glance_store.verify_default_store()

run_staging_cleanup()

_setup_os_profiler()
return config.load_paste_app('glance-api') # Paste Deploy

paste.ini

paste.ini 文件是 wsgi 应用的配置文件,根据该文件可以知道应用程序是如何映射 URL 以及如何处理请求。

paste-ini 配置文件类似 ini 配置,每个 Section 的格式均为 [type:name] ,有以下几个小节

  • composite:收到请求后通过的第一个 Section,表示需要将 HTTP URL Request 调度到一个或者多个应用中
  • app:实现主要功能的具体应用
  • pipeline:过滤器管道,最后一个必须是 app 类型
  • filter:实现过滤器功能的中间件,用于过滤请求和响应

仍然以 glance 为例,启动 glance 服务时需要指定 paste.ini 配置文件,源码中的 etc/glance-api-paste.ini 配置文件如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
# Use this pipeline for no auth or image caching - DEFAULT
[pipeline:glance-api]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler unauthenticated-context rootapp

# Use this pipeline for image caching and no auth
[pipeline:glance-api-caching]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler unauthenticated-context cache rootapp

# Use this pipeline for caching w/ management interface but no auth
[pipeline:glance-api-cachemanagement]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler unauthenticated-context cache cachemanage rootapp

# Use this pipeline for keystone auth
[pipeline:glance-api-keystone]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler authtoken context rootapp

# Use this pipeline for keystone auth with image caching
[pipeline:glance-api-keystone+caching]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler authtoken context cache rootapp

# Use this pipeline for keystone auth with caching and cache management
[pipeline:glance-api-keystone+cachemanagement]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler authtoken context cache cachemanage rootapp

# Use this pipeline for authZ only. This means that the registry will treat a
# user as authenticated without making requests to keystone to reauthenticate
# the user.
[pipeline:glance-api-trusted-auth]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler context rootapp

# Use this pipeline for authZ only. This means that the registry will treat a
# user as authenticated without making requests to keystone to reauthenticate
# the user and uses cache management
[pipeline:glance-api-trusted-auth+cachemanagement]
pipeline = cors healthcheck http_proxy_to_wsgi versionnegotiation osprofiler context cache cachemanage rootapp

[composite:rootapp]
paste.composite_factory = glance.api:root_app_factory
/: apiversions
/v2: apiv2app

[app:apiversions]
paste.app_factory = glance.api.versions:create_resource

[app:apiv2app]
paste.app_factory = glance.api.v2.router:API.factory

[filter:healthcheck]
paste.filter_factory = oslo_middleware:Healthcheck.factory
backends = disable_by_file
disable_by_file_path = /etc/glance/healthcheck_disable

[filter:versionnegotiation]
paste.filter_factory = glance.api.middleware.version_negotiation:VersionNegotiationFilter.factory

[filter:cache]
paste.filter_factory = glance.api.middleware.cache:CacheFilter.factory

[filter:cachemanage]
paste.filter_factory = glance.api.middleware.cache_manage:CacheManageFilter.factory

[filter:context]
paste.filter_factory = glance.api.middleware.context:ContextMiddleware.factory

[filter:unauthenticated-context]
paste.filter_factory = glance.api.middleware.context:UnauthenticatedContextMiddleware.factory

[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
delay_auth_decision = true

[filter:gzip]
paste.filter_factory = glance.api.middleware.gzip:GzipMiddleware.factory

[filter:osprofiler]
paste.filter_factory = osprofiler.web:WsgiMiddleware.factory
hmac_keys = SECRET_KEY #DEPRECATED
enabled = yes #DEPRECATED

[filter:cors]
paste.filter_factory = oslo_middleware.cors:filter_factory
oslo_config_project = glance
oslo_config_program = glance-api

[filter:http_proxy_to_wsgi]
paste.filter_factory = oslo_middleware:HTTPProxyToWSGI.factory

可以看到 pipeline 最后的总是 rootapp 应用,paste.composite_factory 设置应用的工厂函数

1
2
3
4
[composite:rootapp]
paste.composite_factory = glance.api:root_app_factory
/: apiversions
/v2: apiv2app

定位源码 glance/api.py 的 root_app_factory 函数,显然是用于设置 url 映射的。

1
2
def root_app_factory(loader, global_conf, **local_conf):
return paste.urlmap.urlmap_factory(loader, global_conf, **local_conf)

由于我主要关注的是服务的启动方式,因此 URL 映射具体是如何实现的并没有了解。通过 paste.ini 文件可以知道请求到达真正的应用前经过了什么中间件(过滤器)的处理,这些中间件的源码也可以看一看。

参阅