As part of the onboarding process to the Packstack team, I had to understand how the Packstack internals work. There is no official documentation describing it, so when I learnt that some other colleagues would help us writing new plugins, I thought it would be a good idea to create something in written form. Using a series of blog posts by E.Nakai (available here) and some of my own reverse-engineering as a starting point, I have created this blog post.
Big thanks to Martin Mágr for his review and corrections to the post. Please do not hesitate to send me any corrections or comments!
Understanding Packstack flow
The following diagram shows a high-level, simplified flow of how Packstack works.
Packstack has a plugin-based architecture. The main module creates a controller object, which takes care of managing the whole deployment process and stores several important variables.
On start, it will load all available plugins from a specified directory (/usr/lib/python2.6/site-packages/packstack-<version>/packstack/plugins), and check their validity. Each plugin must have two functions to be considered valid:
- initConfig(controller): this function defines the plugin configuration (more on that later).
- initSequences(controller): this function will define the sequence of actions to be executed for this plugin (again, more on that later).
Once all plugins have been correctly initialized, command-line arguments will be parsed and handled. Alternatively, if we are running an interactive installation, Packstack will ask the user to enter the configuration options in the terminal. Either way, configuration options are validated and/or processed, if required by the option definition.
After getting all options, each plugin will be called to initialize its sequence of actions, which basically defines a series of plugin functions to be executed. The plugin functions will take care of whatever configuration steps are required, usually involving the creation of a Puppet manifest for each host to be installed, based on the configuration options.
A special plugin (puppet_950) will copy all generated Puppet manifests to the target host(s) and execute them.
Finally, any messages generated during the Packstack execution will be displayed to the user, and cleanup actions will be started (removing temporary variables and such).
Writing a new plugin
1 – First steps
The first step in writing a new plugin is making sure we have the required Puppet modules to configure the components we want to install, or alternatively create our own. Puppet-openstack – OpenStack can be a good starting point to find the appropriate modules.
Once we have the modules and we have cloned the Packstack repository locally (git clone git://github.com/stackforge/packstack.git), we can go to the modules directory and create our plugin file. The naming convention for plugins is name_XXX.py, where XXX is a 3-digit sequence number that defines the plugin execution order.
2 – Imports and basic plugin definitions
This is an example of the imports used by the AMQP plugin, together with the basic plugin definitions.
# -*- coding: utf-8 -*- ''' Installs and configures amqp ''' import logging import uuid import os from packstack.installer import validators from packstack.installer import processors from packstack.installer import basedefs from packstack.installer import utils from packstack.modules.common import filtered_hosts from packstack.modules.ospluginutils import (getManifestTemplate, appendManifestFile) #------------------ oVirt installer initialization ------------------ PLUGIN_NAME = 'AMQP'; PLUGIN_NAME_COLORED = utils.color_text(PLUGIN_NAME, 'blue')
The modules imported from packstack.installer and packstack.modules will likely differ depending on your needs, but most of these are widely used.
Then, there are two variables to define:
Variable | Description |
---|---|
PLUGIN_NAME | A simple string defining the plugin name |
PLUGIN_NAME_COLORED | A colored string to define the plugin name, with a color |
3 – Plugin configuration
To define the plugin configuration, we need to create function initConfig(), and there we will define all configuration options using a pre-defined syntax. Let’s use the following example to understand the way it works:
def initConfig(controller): params = [ {"CMD_OPTION": "os-horizon-ssl", "USAGE": "To set up Horizon communication over https set this to 'y'", "PROMPT": "Would you like to set up Horizon communication over https", "OPTION_LIST": ["y", "n"], "VALIDATORS": [validators.validate_options], "DEFAULT_VALUE": "n", "MASK_INPUT": False, "LOOSE_VALIDATION": True, "CONF_NAME": "CONFIG_HORIZON_SSL", "USE_DEFAULT": False, "NEED_CONFIRM": False, "CONDITION": False}, ] group = {"GROUP_NAME": "OSHORIZON", "DESCRIPTION": "OpenStack Horizon Config parameters", "PRE_CONDITION": "CONFIG_HORIZON_INSTALL", "PRE_CONDITION_MATCH": "y", "POST_CONDITION": False, "POST_CONDITION_MATCH": True} controller.addGroup(group, params)
We have two entities:
- Configuration options: These options can be used as part of the sequence execution, or as variables for the Puppet templates. We will group them together in a list, and then use that list to define the configuration group. Each configuration option is a dictionary, with the following options:
Option | Description |
---|---|
CMD_OPTION | Option to be used when invoked from the command-line |
USAGE | Usage description for the option. It is included in the answer file as a comment |
PROMPT | During interactive execution, which prompt to show the user |
OPTION_LIST | Valid options. For free-text entries, you can use [] (empty list) or just remove this configuration option. |
VALIDATORS | A list of validator functions. Validator functions will check if the user input is correct, and are defined in packstack/installer/validators.py. Some useful validator functions are:
|
DEFAULT_VALUE | The default value for the option. |
PROCESSORS | A list of processor functions. Processor functions perform some post-processing on the user entered input, and can even change the value. Processor functions are defined in packstack/installer/validators.py. Some useful functions are:
|
MASK_INPUT | Set to True if the user input should be masked (e.g. for passwords), otherwise False |
LOOSE_VALIDATION | Set to True to ask user to use the entered configuration value even if validation failed for it |
CONF_NAME | Configuration option name in the answer file. It is also used to find the configuration option in the controller.CONF dictionary. |
USE_DEFAULT | If set to True, do not ask the user for input on this variable when running in interactive mode, and use DEFAULT_VALUE instead. |
NEED_CONFIRM | Set to True if the user input needs confirmation (usually for passwords), otherwise False |
CONDITION | In theory, it should be a condition to enable/disable the option. In practice, it is just ignored, and always set to False |
DEPRECATES | A list of deprecated CONF_NAME options. Useful when reusing an answer file from a previous version |
- Configuration groups: they group together several related configuration options, and can be made optional based on the value of another configuration option, defined in the same plugin or another one. Again, each configuration group is a dictionary, with the following options:
Option | Description |
---|---|
GROUP_NAME | A group name, should be unique. |
DESCRIPTION | A group description. It is displayed when showing the configuration group usage after running “packstack –help” |
PRE_CONDITION | It can be either a configuration option that should have a desired value, or a function, that will receive a single parameter (the controller.CONF config dictionary). If set to False, the configuration group is always enabled. |
PRE_CONDITION_MATCH | The required value for the PRE_CONDITION configuration option, if we want to have this configuration group enabled. If PRE_CONDITION is a function, it must match the value returned by the function. |
POST_CONDITION | A configuration option that should have a desired value to allow the configuration group to be considered as valid after all its parameters have been entered. If set to False, no post-condition checks are executed. No plugin is currently making use of post-conditions. |
POST_CONDITION_MATCH | The required value for the POST_CONDITION configuration option, to consider the configuration group as valid. |
You can define multiple configuration options in a single plugin, and remember to use the following call to add the configuration group to the controller object:
controller.addGroup(group, params)
4 – Plugin sequence
Each plugin will need to define a sequence of functions to call. For most plugins, those functions will involve the creation of a Puppet manifest file from a template, and filling it based on the variables received from the user.
Let’s have a look at how the Ceilometer plugin creates its sequence:
def initSequences(controller): if controller.CONF['CONFIG_CEILOMETER_INSTALL'] != 'y': return steps = [{'title': 'Adding MongoDB manifest entries', 'functions': [create_mongodb_manifest]}, {'title': 'Adding Ceilometer manifest entries', 'functions': [create_manifest]}, {'title': 'Adding Ceilometer Keystone manifest entries', 'functions': [create_keystone_manifest]}] controller.addSequence('Installing OpenStack Ceilometer', [], [], steps)
It is simply defining a list, where each item is a dictionary with two entries:
Entry | Description |
---|---|
title | A simple string defining the plugin name |
functions | A list of the functions to be executed. These functions should belong to the plugin, and always get called with two parameters:
|
Finally, the call to controller.addSequence() adds the plugin steps to the list of sequences to be executed. The second and third parameters are not used by any plugin, so it is a safe bet to keep them as empty lists.
5 -Working with manifest templates
Our final steps will involve creating one (or more) Puppet manifests based on the templates and supplied information. We will use some helper functions for that, as we can see in the following example:
def create_mongodb_manifest(config, messages): manifestfile = "%s_mongodb.pp" % config['CONFIG_MONGODB_HOST'] manifestdata = getManifestTemplate("mongodb.pp") config['FIREWALL_ALLOWED'] = "'%s'" % config['CONFIG_CONTROLLER_HOST'] config['FIREWALL_SERVICE_NAME'] = 'mongodb-server' config['FIREWALL_PORTS'] = "'27017'" config['FIREWALL_PROTOCOL'] = 'tcp' manifestdata += getManifestTemplate("firewall.pp") appendManifestFile(manifestfile, manifestdata, 'pre')
- First, we will generate a manifest file name, using the host IP to make it unique.
- Then, the manifestdata variable will be filled out, using getManifestTemplate(). You can see in the example that we are combining two templates here, to add any required firewall rule.
- Finally, appendManifestFile() will create the manifest, using the two variables and a marker. And what are markers for?
When plugin puppet_950.py applies the Puppet manifest on a target host, it does not necessarily wait until one manifest is finished before launching the next one. When applyPuppetManifest() finds a marked manifest, it will wait until all the previous manifests are applied, and then proceed with the marked one. If the next manifest has the same marker, it will be applied in parallel.
So we can use markers to make sure that previous manifests are completed before running ours, for example if we need to make sure MariaDB is running before doing anything else.
The general rule for attaching markers is then:
- If you are sure that the manifest can be applied in parallel with any previous manifests, do not attach a marker.
- If you are not sure that the manifest can be applied in parallel with previous manifests, you should attach a unique marker to it.
- If you are sure that the manifest can be applied in parallel with the previous one, attach the same marker as the previous one. This is generally the case when you apply the same manifest to multiple servers.
Additional details to keep in mind
a) Documenting added options
Make sure you document any added option in docs/packstack.rst. This file will be used to generate the packstack man page, and you definitely want the new options to be in there.
b) Global options
In some cases, your new plugin will be optional, so you will want to have a global option controlling whether it should be used or not. The best way to implement it is to add a single boolean option to prescript_000, using the Heat boolean as an example:
{"CMD_OPTION": "os-heat-install", "USAGE": ( "Set to 'y' if you would like Packstack to install " "OpenStack Orchestration (Heat)" ), "PROMPT": ( "Should Packstack install OpenStack Orchestration (Heat)" ), "OPTION_LIST": ["y", "n"], "VALIDATORS": [validators.validate_options], "DEFAULT_VALUE": "n", "MASK_INPUT": False, "LOOSE_VALIDATION": False, "CONF_NAME": "CONFIG_HEAT_INSTALL", "USE_DEFAULT": False, "NEED_CONFIRM": False, "CONDITION": False},
Remember that your plugin configuration group should then add a pre-condition based on the new option, as seen in heat_750.py:
... group = {"GROUP_NAME": "Heat", "DESCRIPTION": "Heat Config parameters", "PRE_CONDITION": "CONFIG_HEAT_INSTALL", "PRE_CONDITION_MATCH": "y", "POST_CONDITION": False, "POST_CONDITION_MATCH": True} controller.addGroup(group, parameters)
Also, keep in mind that PRE_CONDITION can be a function, if you need to check some more complex scenarios.
c) Copying required Puppet modules to other systems
One of the tasks performed by puppet_950 is copying the generated Puppet manifests to the rest of the systems being installed. However, it also copies certain Puppet modules, that are required by our Puppet modules. Thus, if you need to have additional Puppet modules distributed, you will need to modify puppet_950.py under the plugins directory. Look at the copy_puppet_modules function:
def copy_puppet_modules(config, messages): os_modules = ' '.join(('apache', 'ceilometer', 'certmonger', 'cinder', 'concat', 'firewall', 'glance', 'heat', 'horizon', 'inifile', 'keystone', 'memcached', 'mongodb', 'mysql', 'neutron', 'nova', 'nssdb', 'openstack', 'packstack', 'qpid', 'rabbitmq', 'remote', 'rsync', 'ssh', 'stdlib', 'swift', 'sysctl', 'tempest', 'vcsrepo', 'vlan', 'vswitch', 'xinetd', 'openstacklib'))
And add any required Puppet module. Keep in mind that if you use any third-party Puppet module (i.e. not present in the openstack-puppet-modules package), you need to copy that module into /usr/share/openstack-puppet/modules.
d) Password handling
Password options will likely need to have the following options:
MASK_INPUT: True, USE_DEFAULT: False, NEED_CONFIRM: True,
so you can ensure input is masked, and a confirmation is requested from the user. In addition to this, bz#1108742 added a requirement to have a global –default-password option. If you want your password option to be overriden by the global password, add the following options:
DEFAULT_VALUE: 'PW_PLACEHOLDER', PROCESSORS: [processors.process_password],
The default value of “PW_PLACEHOLDER” will be read by the processors.process_password function, and the following logic will be followed:
- If there is a user-entered password, use it
- Otherwise, check for a global default password, and use it if available
- If there is no user-entered or global default password, generate a random password using uuid.uuid4().hex[:16] (this was the common random password used before).