Upgrading to vRA 7.3 – Adding Puppet as a First Class Citizen
With the release notes for vRA 7.3 in hand we finally have details around how VMware is making configuration management tooling a first class citizen in the platform! I typically leave the deep level blogging around CMP’s to my buddy Eric Shanks, but this latest release jumps into one of the areas that I have some expertise in, configuration management with Puppet. With vRA 7.3 we’re now able to use a vRO plugin to execute workflows to install our puppet agent and assign a node role to our newly provisioned Virtual Machines. In this post we’ll go over what the upgrade process looks like for a minimal vRA install and then how to take advantage of the new vRO endpoint for Puppet. By the end of this post, we should have a fully functional apache webserver on CentOS 7 (built by Puppet) deploying from vRA.
Some details around upgrading your minimal vRA environment from 7.2 to 7.3
VMware provides a pretty good bit of guidance on the upgrade process here. Make sure you read and understand the caveats of upgrading your system (are you a distributed vs minimal install, do you have codestream installed?).
Actual runbooks for the upgrade are available here. Take some time to read over your specific section prior to hitting the upgrade button.
Feel free to follow the documentation for getting your vRA 7.x environment upgraded from 7.x to 7.3. it worked for me. I’ve found that with my minimal deployment that I only needed to ensure that my IaaS machine’s management agent service was up and running (spoiler alert, it wasn’t and required me to reboot the windows server that it was on to get things going… I’m no windows expert). From there (I’m on 7.2) I executed the upgrade from the vRA appliance interface. To follow along (get some meaningful feedback) with the upgrade process as it executes, login to the vRA appliance with an ssh client and execute the following to bring up a realtime log stream of what’s happening as it happens:
tail -f /opt/vmware/var/log/vami/*log
When the upgrade wraps up on the appliance a reboot will be required (you should have a window to re-login to the vRA appliance web interface and notice that the appliance asks you to reboot). Go ahead and reboot. At this point I was able to login to my newly updated 7.3 instance of vRA. I ran a quick build test in vRA of previously working blueprints and ended up a with a failure around certificates related to my vSphere endpoint (it’s a lab, I use self-signed certs). You’ll need may need to re-authorize your vCenter endpoints cert under endpoints if you run into the same. Because I’ve worked previously with both the v1 and 2.x puppet plugins, I have some additional cleanup work to do on the vRO side which is somewhat involved, we’ll get into that next.
Cleaning out past Puppet plugins for vRO
First and foremost, you won’t need to follow this step if you’ve not leveraged the puppet vro plugin in the past. This section is geared towards cleaning up the remnants of the earlier Puppet plugins for vRO. For my usecase this was the most difficult part. If you’re brand new to Puppet in your vRA environment skip ahead to installing the new vRO plugin.
I had some difficulty with the vRO configuration web-based tool, so I recommend this approach that brings together making all of the changes that you need to make on your vRA appliance into a single shell script. Puppet Labs has kindly posted the source for you over on Github.
When executing your script on the vRA application the script does the following:
- stops the vRO service
- waits for 10 seconds to give the service time to stop
- removes the puppet plugin file: /usr/lib/vco/app-server/plugins/o11nplugin-puppet.dar
- removes any lines with Puppet in them from /etc/vco/app-server/plugins/_VSOPluginInstallationVersion.xml
- restarts the vRO configurator service
- restarts vRO
Wait for the service to come back up and login to orchestrator using the orchestrater client.
First head over to the Plugins section… Here the Puppet plugin should not be installed/visible. Here’s what it looks like when it is:
If you see Puppet listed here in the plugins screen. Go back and work on removing it. If you don’t see it, GREAT, let’s remove the other bits that may still be lying around our orchestrator instance:
First let’s adjust a setting. Click the Tools/Prefs button in the upper righthand corner of the orchestrator client window and select the “delete non empty folder allowed” button… We’ll be removing old workflows in bulk.
Next let’s jump over to our packages tab and remove all of the packages with puppet in the name:
We’ll do the same with the Puppet folder under the Workflows section. Just remove it completely.
And the same with all of the Actions with Puppet in the name:
At this point we’re done removing the traces of the old Puppet plugins from here we’re back to building stuff! Let’s start by getting the new Puppet vRO plugin installed.
Installing the vRO puppet plugin:
The next phase is installing the new plugin. Grab the latest plugin .vmoapp file from the VMware solutions exchange here.
After downloading the bits login to the vco-control center at https://<VRO-SERVER-IP-ADDRESS>:8283/vco-controlcenter
Head over to the plugins screen and install the .vmoapp file. After that, restart the orchestrator server.
Now on to some of the work on our Puppet Server.
Installing the Puppet-vRO-Starter-Content on your Puppet Master:
Puppet was nice enough to work with VMware to build out some starter content to demo Puppetized blueprints in vRA. The best part is that you can leverage this work to help bring some extra services to your cloud as a customer today. Included are classes that configure webservers and database services for you by the end of this post hopefully you’re using vRA to provision some more meaningful blueprints from your vRA instance. To get going you’ll need a Puppet Enterprise master server that’s all setup. More on getting that done and your 10 free licenses for Puppet Enterprise over on Puppet’s website.
Okay, I have to admit we need a BIG RED WARNING sign here. As all of the documentation for puppet states you should only run the Puppet-vRO-Starter_Content install script code on a Puppet Enterprise server that isn’t using code manager and also isn’t home to existing production stuff. If you have ANY doubt take a snapshot prior to this section of your Puppet server as this part WILL ALTER how puppet pushes code to nodes that are registered to it. Okay, sure that you’re not about to break production? Carry on:
Let’s get started installing the Puppet-vRO-Starter-Content on our Puppet Enterprise Master Server:
- Login via ssh to your Puppet Master server.
- Become a user that has rights to the Puppet Master service (in a lot of lab cases this is root) sudo su – root
- Execute a git clone https://github.com/puppetlabs/puppet-vro-starter_content (clones the git repo down to your Puppet server)
- Execute cd puppet-vro-starter_content/scripts (jumps you into the starter content directory)
- Okay last warning… execute a less vra_nc_setup.sh – see that line in the script where it’s moving all of the prod code on your puppet server another backup directory? Don’t just do this on your prod puppet server or any puppet server you care about for that matter without thinking about it and talking to your puppet admin. Executing this script will break your current environment.
- If you’re sure that you’re ready, execute ./vra_nc_setup.sh
- After the script finishes execute a puppet agent -t to finalize the changes on the puppet master
Configure the vRO Plugin.
Comparatively this next part, setting up the Puppet vRO plugin, is pretty easy. We’re going to login to the vRO appliance with the vRO client and run a workflow to register our newly Puppet master.
- Login to your vRO appliance
Go to Workflows > Puppet > Configuration > Add a Puppet Enterprise Master and run it.
Fill out the form as requested:
The workflow should execute successfully. If it doesn’t, check the fqdn and the login credentials for your Puppet Master server. I’ve used root because this is a lab, if you’re using a non-root user make sure to add that user to the sudoers file with no-passwd on the Puppet Enterprise server.
We’ve now setup communication with the vRO server and Puppet. Let’s tie this into vRA and complete a blueprint.
vRA setup for the Puppet vRO Endpoint
- Click Administration > vRO Configuration > Endpoints and click New
- From the plugin screen you should now be able to select Puppet from the dropdown
- Give your Endpoint a Name and a description
- On the next screen give your puppet master server a name, enter it’s FQDN or ip, and input the name of a user that can ssh into the box to execute puppet commands. For me that user is root because I’m using a lab. Here’s what my form looked like:
At this point we can build out a Puppet Based blueprint in the vRA blueprint designer. We cover off on that in the next section.
Building a Puppet Blueprint for a CentOS Webserver.
While how to use the blueprint designer is out of scope for this post, i’d like to start with a simple CentOS blueprint and clone it. If you need help building out a CentOS blueprint, I’d recommend having a look at Eric Shank’s fabulous walkthrough on his blog, or better yet, check out his AWESOME video course on PluralSight.
Let’s start by copying an existing linux blueprint (I’m using CentOS):
Edit the blueprint to open the blueprint designer.
As you can see for this post, I’ve built a VERY basic VM blueprint that clones and attaches a CentOS 7 virtual machine to an existing vSphere network and applies a customization spec to the box.
Let’s add a Puppet role, by selecting configuration management in the Categories pane.
Drag the Puppet object onto the Virtual Machine object to begin assigning Puppet info to it.
In the Server section you’ll need to select the Puppet Master that you just configured, for environment select dev, and for role select role::linux:webserver For right now, leave the shared secret section blank. The shared secret is used to autosign your puppet machine in the puppet server. As of this moment the documentation on how that’s implemented is just a bit thin, but I’ll work on having an update here on how to enable that.
For the managed node section you’re going to enter a username and password that is capable of logging into the VM that you’re going to build with this blueprint (In this example the root password for your CentOS 7 template). This info is needed to install the Puppet agent as a part of the post provisioning process.
Almost done, let’s test
Publish you blueprint and run it to test it out… I like to follow my builds watching, vCenter for the clone:
vRO for the start of the Puppet work and then logging into my puppet master to accept the new nodes certificate requests during the build. The first should be self explanatory, for vRO, login using the client and monitor the Install PE Agent with Role workflow under the Puppet > vRA_Integration folder in the workflows tab.
After seeing this login to your puppet master console and click nodes > unsigned certificates. After a machine’s first execution of puppet (you should see that in the vRO workflow log while it waits) click the button to accept the certificate request. (This is the part that we’ll automate with a secret in the future)
At this point you should be able to monitor the vRO workflow until it completes successfully. When it does, open a web browser and point it at the ip of the Virtual Machine that you just provisioned and you should see a shiny new webserver built with puppet:
There’s still a lot to cover and I hope to do that in part two of this series where I talk about automating the certificate signing for your Puppet builds with vRA.
Until then, thanks for reading!