Automate Server Patching with Puppet (Part 2)

In the first blog post I made about this, I showed how to patch windows 3rd party software like, notpad++, windirstat and other software utilizing puppet and chocoloatey on windows.  However, the next part of the solution was to deploy windows patches as the name suggests.  This part of the equation was a bit more challenging, but well worth the effort.

For help setting up puppet and chocolatey refer to the previous blog post. 

When patching windows security patches, most of them are not on the chocolatey website ready to go, so I choose to build my own repository of patches using a internal NuGet server.  Getting NuGet up and running requires Visual Studio.  In my case I utilized the community edtion (Free).  During the install you have to pick which components you want to install.  See below:

Once you get visual studio prepare, your ready to head over to the microsoft project site and build the package on your own IIS instance.  I wont go into much detail here, I didn't run into any issues with the steps provided by Microsoft.  However, after the install, I did have an issue with packages being picked up quickly.  I tuned the APP Pool to recycle every 5 minutes, which addressed the issue.

https://docs.microsoft.com/en-us/nuget/hosting-packages/nuget-server

Once you get your nuget server setup and running the next step is to start creating and added packages to the nuget server.

1. Download the patch you want to package from microsoft:  https://www.catalog.update.microsoft.com/Search.aspx
2. From a system with chocolatey installed run the command "choco new "KBNUMBER" -file="PATH TO KB.MSU" -build-package.  This command will create a new folder with the basic package structure.  Before we convert it into something that the nuget server can host, we need to tweak the package.
3. To prepare the deployment package first edit the nuspec file that the command above created.  You can tweak this to fit your own needs.  In my case I edited the following lines, then saved the file.
   

   Replace Line 29	<version>1.0</version>
   Replace Line 38	<authors>MHHCC.ORG</authors>
   Delete Line 40	<projectUrl>https://_Software_Location_REMOVE_OR_FILL_OUT_</projectUrl>
   Replace Line 50	<tags>YOURKBNUMBER</tags>
   Replace Line 51	<summary>YOURKBNUMBER</summary>
   Replace Line 52	<description>YOURKBNUMBER</description>

4. I also bundled the MSU files into the nuget package, so copy the MSU file you downloaded from microsoft into the "Tools" folder that the choco command created.
5. Next while we are in the tools folder edit the "chocolateyinstall.ps1" file.  In my case I edited the following lines and saved the file.
   

   Replace & Uncomment Line 13	Example:	$fileLocation = Join-Path $toolsDir 'windows10.0-kb4103720-x64_c1fb7676d38fffae5c28b9216220c1f033ce26ac.msu'
   Replace Line 24		fileType      = 'MSU'
   Uncomment Line 27		file         = $fileLocation
   Replace Line 41		silentArgs    = "/quiet /norestart /log"

6. The final step to create the package is to run the command choco pack.  You need to run this from the inside the folder that the first choco -build-package command created.  This will basically pack all of the files we just edited and the MSU file into a .NUPKG file that the NuGet server can understand.
7. Now that we have a NUPKG prepared and created you can copy it to the IIS server where NuGet is running.  In my case the path was "C:\inetpub\wwwroot\NuGet\Packages"  Once you place the file there browse to your nuget server http://localhost/NuGet and click on "Here" to view your packages.  This should trigger NuGet to create the required files and place them int he proper folders.  You will see the file you placed in the "Packages" folder get moved into a folder with name of the package you specified in the nuspec file.  Once this happens your package is ready to be deployed with puppet.
8. In my case I wanted to keep my packages seperated by OS version.  So I created a new folder and class object on the puppet server.
1. mkdir /etc/puppetlabs/code/enviroments/production/modules/server2016
2. mkdir /etc/puppetlabs/code/enviroments/production/modules/server2016/manifests
3. vi /etc/puppetlabs/code/enviroments/production/modules/server2016/manifests/init.pp
• class serverupdates {
• include chocolatey
• 
  
• package { 'KB4103723x64':
• ensure => present,
• provider => 'chocolatey',
• source => "http://<YOURNUGETSERVERNAME>/NuGet/nuget"
• }
• }
9. Login to foreman at this point and import your class from the "Configure->Classes->"Import environments from "your puppet server" button.
10. Edit a host group and add the class to it
11. From a host that you just added the class to restart the puppet service.
12. Monitor the c:\programdata\chocolatey\logs\ folder for the process
13. Monitor the application event viewer for the puppet logs

While the first few times going through this process seemed to take a bit of time, the last few packages I've created have only taken about 5 minutes.  I have created all the packages needed to upgrade Server 2016 to current.  Once we build a new 2016 server we install choco and puppet, add it to the host group, and it becomes compliant within a few minutes.