Roberts Blog

The House of SCCM and Intune on System Center Street

Tag: Build

Windows 10 Automation–Changing Language–B1903

A customer of mine is in the process of bringing the image factory back in-house, leveraging their ConfigMgr installation, hosted in Azure, to deliver Windows 10 task sequences (build and upgrade) to intranet and eventually via their CMG, internet based devices.

The quality bar is relatively basic from the MSP that is responsible for purchasing, preparing and shipping their devices to their end-users, so it hasn’t taken long to spin up OSD in ConfigMgr, and match the build results, producing a better tooled, more customised build that meets their needs and goes several steps further, while they handle purchasing via the MSP and do the delivery themselves.

Managing the Windows 10 image factory using ConfigMgr is an interim measure for this specific customer, at some point they will swing towards using AutoPilot as part of their modernisation and cost reduction plan that we’ve come up with, which includes the ultimate objective that a company can have nowadays, or an IT pro can have on their radar, the biggy, winding up Active Directory.

Part of this customers image factory requirements is that the build starts out life in English (en-GB), so that their build engineers can customise the OS further with a bunch of tasks that haven’t been brought into the task sequence at this point, due to time restraints or Windows 10 B1903 related bugs (VPN settings annoyances when setup in SYSTEM context, SCCM delivered Wifi profile password woe’s …). Finally the customer wants to be able to switch the builds language to that required for the target user, just before they close the lid and begin shipping.

In this post I’m going to show how I handled the customers language requirements in Windows 10 using SCCM OSD, leaving some footprints on ground already well-trodden by notable others.

Straight out of the gate I was experiencing issues setting the language reliably in Build 1903.

I tried to approach using the unattended setup file, that ‘trusty’ old horse, and when that wouldn’t play ball, I turned to using brutality with DISM and PowerShell applets at the tail end of the task sequence, in an attempt to coerce the operating system into doing my bidding. Failure is a spur towards success for the less weak-of-heart, is what I say when things just don’t work. Surely there has to be a way.

I cruised the net. Saw much chatter about language issues in various Windows 10 builds, most of it seemingly unrelated noise, I noticed a post by Dan Padgett where he uses a different method, RUNDLL32 and an XML file (or two), passed it by, I recall at the time thinking that it most likely was for an older version of Windows 10 and looked pig ugly Smile

At my whit’s end, I reached out to Paul Winstanley, who promptly pointed me back at Dan’s post as the only reliable way he could get it all to work at present.

Dan’s post is actually quite comprehensive and is in part a derivative of some of the ground work carried out by Nicolas Lacours, there isn’t much more for me to add if anything, a Stirling job indeed, instead I’ll show how I leveraged the proposed method to switch languages during and after OSD.

So yeah, I implemented Dan’s write-up on using the RUNDLL32 method, and viola, after a bit of tinkering to match up with the task sequence variables in use, and after ironing out SillinessFromMe™, I was able to produce a build in any of the list of languages the customer needed.

Now that language in the newly built OS was controllable (thanks Dan and Nicolas, and Paul for circling me back there!) the next step was to force it to build with en-GB, while storing away in the registry what was chosen as the destination language when UI++ launches, so that it can be read in and processed another time to do the final language switch.

At this point the build engineer has an en-GB build to log into, and do whatever they want in readiness for the user.

The next piece was the final language switch, I used another task sequence, with all the language steps from the main task sequence copied across and some additional bits added, and then deployed as Available to the OSD build collection.

This then showed up in Software Center, and could be run as the final task before the device is powered off and shipped.

I’ll now go over the OSD build parts where it differs from Dan’s, and has notes worth pointing out, as I said there wasn’t much need for any change from what he has already etched out.

At the front-end of the task sequence, UI++ runs and interviews the build engineer:

  1. Launch UI++, buzz the engineer for build details and store selections in task sequence variables
  2. Stored the resulting OSDUILanguage value in a new task sequence variable called StoredOSDUILanguage, which is then used at the tail-end of the task sequence as part of the branding\tattooing (not MDT tattoo) of the device
  3. OSDUILanguage is forced to become en-GB to model the experience needed, this is the override that will force all builds to be en-GB initially

After the Setup Windows and ConfigMgr step, we break into the steps to handle the language.

Pretty much how Dan does it. I think the only difference is that I put the Language pack logic on the steps, and added the UK keyboard instead of US.

The final part of the solution in my task sequence runs just before the task sequence finishes up, and is used to poke the value stored away in the task sequence variable StoredOSDUILanguage, into the registry for later use alongside a few other settings.

Aside from my modifications, if you follow Dan’s guide, and you’ll get perfect a result every time. Very nice.

The task sequence to do the final language switch is part-clone of the OSD build task sequence steps, along with some customisations.

As you can see the structure of the change language task sequence is a copy\pasta of the OSD build task sequence with some added bits:

What’s happening:

  1. OSDUILanguage, the task sequence variable doing all the language donkey work, is set to en-GB as a default in case anything goes awry
  2. The registry key OSDUILanguage is retrieved from the registry and poked into OSDUILanguage, I do this using a PowerShell script stored in the same package hosting the language injection script from Dan
  3. A task sequence variable that I use to confirm if the language can be changed, BeginProcessing, is initialised as False
  4. OSDUILanguage is used to drive the dynamic variable step, each rule sets the BeginProcessing variable to True

5. The Begin group has logic on it that will skip the group if BeginProcessing isn’t true, which essentially ends the task sequences execution.

The rest is identical to the OSD build task sequence, the language is laid down, three reboots occur, and bosh the language has changed for new users (who have not already logged in). I will no doubt finesse this out a bit more to do error handling and an existential check on the registry key to trigger an abort if missing.

The PoSh to retrieve the registry key, which most likely can be done better, is here:

$tsenv = New-Object -COMObject Microsoft.SMS.TSEnvironment
$regValue = Get-ItemProperty -Path ‘HKLM:\Software\<COMPANY NAME HERE>\Branding’ -Name OSDUILanguage | Select OSDUILanguage
$tsenv.Value(“OSDUILanguage”) = $regValue.OSDUILanguage
Write-host (“Found language ” + $regValue.OSDUILanguage)

We can probably collapse that into a single line executed using a Run Command Line step and do away with the script. Be ideal if ConfigMgr let us read directly from a ‘repository’ such as the registry as a task sequence step, and poke the value into a task sequence variable.

With this mechanism in place, simply changing the registry key from say fr-FR to de-DE and running the task sequence from Software Center will swap languages for new users only.

I couldn’t get it to switch the language for any profiles that were already created on the device.

The build engineers profile remains en-GB, however the destination user once they log in for the first time will have the correct language set.

Now the customer has a handle on their Windows 10 builds, and language is slotted into place to fit their needs, initially configured as en-GB, then configured with one of several other languages supported by the company in readiness for delivery.

All in all a job well done I thought.

I might take a look into changing the language for all users not just new users, and whittle up a post on it at some point if its doable without the user having to be logged in.

ConfigMgr CB1906–Management Insights–NTLM Fallback

I remember a few years back at an MVP Summit watching a PG member showing us the mock up’s they had prepared for how Management Insights would “look” in the console, while gauging our response and taking in feedback.

The feature certainly has come along way from then, if’ you’ve not paid much attention to Management Insights, now would be a good time to visit the feature and see what insights it gives you for your site.

From what I can recall the motivating reason for Management Insights was driven by the desire to make administrators lives easier overall, bringing to light the “house chores” needed to keep SCCM running fluidly, highlighting or giving insights into operational capability of a site (Empty collections, Fast Evaluation rules etc), and it has extended out to highlight best practices for certain parts of the product (example being the Site’s current Client Push NTLM Fallback state).

There’s a new Management Insight (MI) in CB 1906, called “NTLM Fallback disabled” which I’ll quickly run over now.

This MI will check the ConfigMgr Site, to see if Client Push Installation property Allow connection fallback to NTLM is enabled:

image

Enabled, the MI will report Action Needed:

image

When Allow connection fallback to NTLM is disabled in Client Installation properties, and when the MI is re-evaluated (right click) the MI reports a Completed state, which means we’re compliant:

image

The reason why you would disable Client Push attempts using NTLM is to force site to client authentication to take place using Kerberos, so as to fall in place with modern security practices, which see NTLM as insecure (rightly so) and something we should all be drifting away from, as partially noted in the docs:

 image

When using the client push method of installing the Configuration Manager client, the site can require Kerberos mutual authentication. This enhancement helps to secure the communication between the server and the client. For more information, see How to install clients with client push.

At a lower lever you can disable NTLM fallback for the Operating System itself, with consequences that should be thought out first, using either domain or local GPO settings. This isn’t something you do without wising up on the consequences.

The new Management Insight is not checking the OS, it isn’t checking IIS or SQL for relevance and state, which have their own options for handling NTLM fallback.

Here’s a shot of local GPO on the site server for tinkering with restricting NTLM:

Note that GPO changes are made, remote devices attempting RDP to the site server that are not patched may encounter the “Encryption Oracle Remediation” issue.

Changing any NTLM setting requires some preparation work, at the least an understanding of what might break in your environment.

High Availability–Failing over to a new Passive

I’ve been very much interested in the High Availability feature since it inception.

I’ve followed its evolution in technical preview, kicking its tyres, discussing it with the product group and MVP peers, and seeing the feature come out of the development dock, technical preview, to sail around the sea in the form of current branch, is exciting.

Exciting because the High Availability feature is a high-value design element for ConfigMgr Hierarchies, something architects have been waiting a very long time for.

We’re use to high availability for most roles now, with a few single instance roles causing ripples, but the crown jewel is the site server role.

With all the redundancy in place, business can continue, but nothing new can be authored without that site server role, and critically, client registrations would cease to function causing OSD failures.

High Availability in its first forms, will make the Site server role highly available, with a small lead-time between transitions, mostly due to it being a manual exercise in Build 1806, and because there is an up to 30 minute delay before the transition is completed. This will ease out to become instant when automatic failover is introduced. And as this feature iterates, we’ll see a reduction in the server count.

There’s more that High Availability can be used for, in the future we’re going to be able to build multiple primaries to leverage elastic computing, reducing as demand tails off, and right now we can use High Availability to completely remove the need to do an in-place OS upgrade on a primary ever again.

I put together a quick poll on Twitter to gauge interest in the feature, 427 votes is surely not representative, but it gives me an idea of interest none the less.

Over half of the participants do not have High Availability on the radar. The remaining half breaks down almost equally between not interested and implementing soon, with a small percentage implementing right now.

I’ll do another poll later in the year, I’m sure by that time we’ll see far more adoption as the penny drops.

In this post, I’m going to focus on showing you how to recover from an unrecoverable primary site failure, using an already built High Availability ConfigMgr hierarchy.

At some point I’ll rattle out a HA build-out guide, it is pretty straight forward, but I see that others have done this already, and alongside my now-ancient guide on SQL AlwaysOn, I don’t see what more I could contribute beyond the good work they have already laid down, I may have a go at it none the less.

Right then. Let’s define the Lab I’m using to make tracking what is about to happen easier.

The server and role manifest for the lab hierarchy looks like this as of Build 1806:

  • Node 1 – L3CMN1 – ConfigMgr (Active)
  • No roles other than Site server role
  • Node 2 – L3CMN3 – ConfigMgr (Passive)
  • No roles other than Site server role
  • Node 3 – L3CMSS1 – Site system
  • SMS Provider
  • MP, DP, SUP, FSP, SCP, needed SMB Shares for Content Library, SQL AlwaysOn database transfer and the Windows Cluster File Share Witness
  • Node 4 – L3CMSS2 – Site system
  • MP, DP, SUP, FSP
  • Node 5 – L3CMSQL1 – SQL Server (Clustered)
  • Reporting Services
  • Reporting Point
  • WSUS Database (in an AG!)
  • SCCM Database (AG)
  • Reporting Services Database #1
  • Node 6 – L3CMSQL2 – SQL Server (Clustered)
  • Reporting Services
  • Reporting Point
  • WSUS Database (in an AG!)
  • SCCM Database (AG)
  • Reporting Services Database #2
  • This is a lab build-out and not for production, not all site roles are installed or referenced. Their inclusion and placement is a matter of simple consideration and action, for example the Reporting Point and Reporting Services elements can be located on the site systems instead of on the SQL servers, Asset Intelligence Synchronisation point can be put on the Site systems.

    In the above mock-up I’ve placed the SMS Providers onto the Site systems instead of the Primaries, placing the SMS Provider onto the primaries is a supported option in Build 1806.

    When Build 1810 arrives, we’ll be able to condense this solution down to 4 nodes by placing the SQL AlwaysOn and Windows Cluster onto the two primary site servers and jettisoning remote SQL.

    At some point, we may very well arrive at a 2 node solution, time will tell, there are a few interesting technical barriers that need to be overcome before such a design can be realised, making it unrealistic in the relatively short term. I figure that 4 nodes to make a rock-solid highly available hierarchy is a cost worth paying today.

    So let’s have a play with my HA lab, literally switching off a passive site server to check out how unplanned events are handled.

    We could have switched off the active primary, would have incurred a time penalty while we wait for the transition to take place, the passive to switch to becoming an active, I might do that in the end of this guide let’s see.

    Handling an unrecoverable passive Site server

    My two Primaries are called:

    1. L3CMN1 (Active)
    2. L3CMN3 (Passive)

    Node 2 ‘took one for the team’ during testing.

    I’ve turned off L3CMN3.Lab1.com, so as to simulate the complete and unrecoverable failure of one of the two nodes of a High Availability cluster.

    As you can see in the shot below, L3CMN3.Lab1.com is listed as a Site system, which has the Site Server role installed:


    * That lower-case server entry is annoying me too, sorry

    If you try to remove the site system by right-clicking on it in the top panel, and selecting delete, it’ll complain that there is a role there:

    So clearly the first step is to remove that role from the site system, the role is shown in the Site System Roles panel, right-click the role there and select Delete:

    Give it a few moments, then select the site system in the top panel and select Delete:

    L3CMN3.Lab1.com is gone:

    We’re currently running with one primary, so we’re going to need to build ourselves a new passive primary at some point. We’d obviously schedule this for out of hours, although there is barely if any operational down-time while building a new passive primary.

    Building a new passive Primary

    Ready up a new OS, give it a name, fixed IP, install IIS + ADK in accordance with a Primary site’s needs, permission so that the active primary and everyone else can say hello, then create a new Site system in the ConfigMgr console, here I am creating L3CMN4.Lab1.com:

    And I’m going to add the Site server in passive mode role:

    Let it copy the files to the server, and install to C:\Program Files\Microsoft Configuration Manager

    The new passive primary L3CMN4 now shows in the Servers and Site System Roles list:

    Note that by default an SMS Provider is not installed with the passive primary. I recommend not placing them there if you have site systems to hand for client-facing roles, as it will complicate and extend recovery times. Maybe in the future when we’re down to 2 or 3 nodes, it may be more appropriate to house everything on the primaries, everything, maybe.

    Keep an eye on the FailOverMgr log, and visit the new Primary and check out its setup log. I recommend using LogLauncher and punching in the servers that make up your HA setup, use it to visit\open logs, it will make life easier.

    While we’re waiting for the build to complete, we can button mash the refresh button in the console, if it fails it is most likely due to permissions or prerequisites, the FailOverMgr log and site setup logs will help you shore the issues up and a Retry can be performed (right select the node in Sites > (Site server) > Nodes tab).

    L3CMN4 is in place as a passive Primary, and we’re back to high availability of the Site server role:

    With minimal roles deployed to the Primary, the option to give up on classical DR and build out a new primary site server on failure becomes a reality, and a good course of action to take.

    If you have Roles installed onto the Primaries, things get a little bit more complicated, especially for the SMS Provider, and when Build 1810 arrives and SQL is co-located with the Primaries, building a new passive node will require a lot more to achieve.

    All very doable though.

    I figure I could build a passive and get it back into a Windows Cluster, then install SQL using a pre-configured unattended file, get the availability group feature running and join it back in to the existing AG being held alive by the active primary node in time for tea, well, over a few hours if things are well-connected and its all running good equipment (not laggy servers). But there will be down-time, due to a site reset being necessary to complete the SMS Provider removal process which has to take place before a new primary can be built.

    In a moment I’m going to nuke the Active Primary by turning it off.

    As you can guess from the serialisation, L3CMN1 is the very first primary built, so offing it is like cutting the cord in a big way, we’re not swinging from an artificial primary back and forth between a real primary, each primary in a High Availability configuration is a real primary, who cares not if its originator still exists, just whether it can reach the database and site systems.

    Let’s step back a moment and ponder things.

    How many of you are fixated on keeping the primary alive at all costs, treating it like an irreplaceable treasure chest?

    Isn’t it a strange feeling to begin treating a primary like a throw-away lighter, something that can easily be replaced?

    That’s new, exciting as I said.

    High Availability is a game-changer, it is literally forcing us to look at a primary, which we’ve treated with affection and protected at all costs in the past, as something easily replaced and ‘disposable’. Almost.

    3rd Party product integration with primaries may require some tending, which would complicate a High Availability design or recovery procedure. I’m exploring that.

    Ok let’s have some more fun.

    Making life complicated, a dead primary with SMS Provider and Roles

    I’ve now installed the SMS Provider and a Management Point onto L3CMN1, before I sacrifice it to the Hyper-V gods for this guide.