Keeping the Distribution Point (DP) servers' Lights On

Point-of-View: from a ConfigMgr Administrator to his Manager, discussing how he/she spends their day managing a ConfigMgr environment.
This is the third part of my four-part story that details what we did in our company (in a previous life) to help reduce the DP server management nightmare.
Part one of this story (trying to troubleshoot a ConfigMgr distribution problem) can be found here.
Part two of this story (investigations into possible solutions) can be found here.
A little background first: we had 140 distribution points in our ConfigMgr environment and too often we had package distribution problems. I’ve previously described how I spent an entire day troubleshooting a failed package distribution. My manager at the time asked me to investigate other possible solutions, and we chose to evaluate the 1E Nomad software. 1E presented the overall solution to us as well as some of the tech within the product, and we were given a direct line to the local 1E Solutions Engineer (aka the experienced techie) who could help with any questions that might arise and they could even come to my office to help speed along the knowledge-learning process.
The introductory meeting with 1E was great and we soon had a good understanding of the basics. Now I needed more information on Nomad. I telephoned the 1E Solutions Engineer and we spoke about some of the other benefits of introducing Nomad into the ConfigMgr environment. We also discussed some formal testing. The following steps describe that process from start to finish.

Step 1: Decide what the evaluation would involve

This meant getting all my ducks in a row. What would I need to test that would ensure Nomad would actually prevent or resolve the original content distribution issue? Was that really my only problem? Would Nomad introduce other problems? Would Nomad resolve other issues I didn’t know I even had? Maybe I didn’t know as much as I thought I knew. So many questions! It was great to be able to discuss our queries with an experienced SE who has travelled down this path before.

Step 2: Determine how to measure the evaluation to confirm we had successfully met our requirements

Our normal measure of prioritizing requirements and defining the success criteria is based on the “MoSCoW method”. The MoSCoW method is a technique used to reach a common understanding with stakeholders on the importance they place on the delivery of each requirement. We determined this method would work well for this project.
Further information on the MoSCow method can be found here: https://en.wikipedia.org/wiki/MoSCoW_method

Step 3: Define requirements, both business and technical

After several meetings with internal sponsors, we finalised our business and technical requirements for the proposed solution. We prioritised each requirement using the MoSCoW method:

Requirement ID Design Principles and High-Level Requirements MoSCoW Method
REQ01 Do not introduce additional risk in software distribution design or client-side software. No single-point-of-failure should be introduced into any feature of ConfigMgr by any integration of third-party solutions. M
REQ02 The design should seek opportunities to gain efficiencies in administration by simplifying the installation, operational processes and integration with ConfigMgr, to provide the most efficient software, patching and OSD deployment delivery M
REQ03 Ensure that LAN-connected (wired) clients at remote sites or those with slow WAN links have the ability to receive deployed software distributions during business hours without an adverse impact to the WAN M
REQ04 Ensure that wireless clients at remote sites or those with Internet links have the ability to receive deployed software distributions during business hours without an adverse impact to the WAN S
REQ05 Regardless of corporate network connection type (WAN, LAN, VPN, home/hotel, WiFi, WiMax, 2G/3G/4G/etc) the client computer must be able to download any ConfigMgr content without impacting the computer, user, or the local network connection M
REQ06 The solution should provide for increased efficiencies and automatic resilience of standard software distributions and OS Deployment, ensuring that local content is always used rather than multiple WAN transfers, and should be fully transparent to the end-users of the machine. S
REQ07 Simplify the Windows Image Deployment (OSD) tasks and processes;

  • Every office location must be able to support the imaging of computers
M
REQ08
  • Be able to image more than a dozen computers at the same time and without any reduction in speed
S
REQ09
  • OSD: The ability to successfully deploy operating systems to any office location without the requirement of server infrastructure at that location
M
REQ10
  • OSD: The ability to successfully deploy operating systems to any office location reconfiguring any (network) infrastructure at that location
S
REQ11
  • OSD: The ability to successfully deploy operating systems to any office location without impacting the network bandwidth availability requirements of any business applications
M
REQ12 The ability to successfully deploy software to any office location without the requirement of server infrastructure at that location M
REQ13 The ability to successfully deploy software to any office location without impacting the network bandwidth availability requirements of any business applications M
REQ14 Reduce to as few ConfigMgr servers as possible without creating a single point of failure M
REQ15 De-risk the software distribution process from the current state S
REQ16 Minimise the total amount of content that needs to be transferred over the network M
REQ17 The ability to successfully import existing ConfigMgr content (from the local client cache) into the proposed solutions’ cache, without being forced to re-transfer (previously downloaded) content over the WAN C

Step 4: Prepare the testing/evaluation environment

My lab environment was going to need some fundamental changes in order to correctly evaluate 1E Nomad with ConfigMgr. For example, I would need multiple IP subnets and a number of client computers in each subnet. Obviously no ConfigMgr servers would exist at any of the “remote” subnets where the client computers exist.
I also requested the network team’s assistance to ensure the virtual WAN links were true representations of our production network environment (i.e.: slow links, congested links, and high-latency links). And the network team was requested to configure monitoring of each port for bandwidth utilization, etc.
I created the following Visio infrastructure diagram which should allow us to test and ensure our requirements are met:
1E Basic Test Lab Diagram

Step 5: Setup and Run the test cases to prove functionality

With assistance from the 1E Solutions Engineer I was able to create a table that would detail our business and technical requirements alongside the actual tests and the expected results. I was so happy to receive this information and it really helped speed up our testing. I’ve included a screenshot of our test document below:
Plan Brent
Then it was just the matter of running through all the test cases and ensuring everything worked as expected and produced successful results. Additionally, we would need to check in with the network team to ensure the bandwidth utilization and throttling technology were within our allowances.

Step 6: Document the results and report back to management

It is very important to provide evidence-based facts when reporting information back to the senior management team. Remembering that some people like data and others like graphs, we showed both.
You need hard evidence to prove the outcome of any tests, and one good method is to already have a baseline that you can compare any future tests with. When monitoring the bandwidth utilization of Nomad we needed evidence to prove that there was a reduction of content being transferred over the network as well as proof that there was zero impact to the users, computers, and business. I’ve included a couple of screenshots from our report below:
Test Case Report
Test Case Report
This is part three of a four-part story I experienced in a past life as a ConfigMgr Administrator. The fourth and final part of this story will describe our actual implementation of 1E Nomad, our experiences and some gotchas!