A customer of mine is planning to consolidate several Hyper-V clusters into one or two big ones.
In order to do that, I had to report on the amounts of memory currently offered by the Hyper-V hosts and the memory configurations of the virtual machines.

When I would use MAP, yet another tool would be introduced into the environment… and only to report on the memory configuration? It seemed to me that this would be a bit overkill.
But then there is SCOM. This customer already has SCOM implemented and configured… but since I’m a n00b when it comes to SCOM this would not have been my favorite solution for such a simple task.

So I turned yet again to my buddy PowerShell

I created a small one-liner to provide an output of all the VM’s, their respective nodes, the amount of memory offered by the nodes and the memory configuration of the VM’s.

Note: This one-liner looks at static memory configurations, not dynamic memory configurations and is based on the SCVMM PowerShell cmdlets!

Get-VM -vmmserver SCVMM01 | select name,memory,vmhost,@{n=’TotalMemoryOfHost(GB)’;e={"{0:N0}" -f ((Get-VMHost &_.VMHost).TotalMemory/1MB)}} | sort VMHost | convertto-html > D:\Temp\MemoryConfig.html

This will give you an output looking like this:

Sometimes, the status of a VM in SCVMM gets stuck although that status is faulty.
In my case, it got stuck to “failed” and when viewing the job in SCVMM, I saw a nice little error:
Error (402)
Library server is not associated with this Virtual Machine Server.

Big fat note: Making manual changes in the SCVMM database is not supported. When you decide to do it anyway, please make sure you have a recent backup before you start making changes!!!

Last week I was at the Hyper-V.nu event at Microsoft Netherlands HQ in Amsterdam.
Ronald Beekelaar (MVP Virtual Machine aka MVP Hyper-V) gave a Data Deduplication Deep Dive session.
This was a very good and highly technical session, which got me thinking… and I decided to write a little article about this new technology in Windows 8.

Introducing: Deduplication in Windows 8
With Windows Server 8 Microsoft introduces a built-in software based data deduplication (dedupe) solution. Where several storage providers offer such solutions, Microsoft has taken another approach by providing a solution for duplicate data from an operating system level instead of a storage level. Where some deduplication solutions provide their services file-based, the deduplication offered in Windows Server 8 does this block-based. More on that later on…

Now, let’s take a few pointers before we start looking at dedupe in Windows Server 8:

• Only available in Windows Server 8.
• Is cluster aware.
• Based on a filter driver per volume.
• Not supported on boot- or system volumes, only intended for data storage volumes.
• Does not work on compressed or NTFS encrypted files.
• Dedupe requires an NTFS file system and is not supported for the new ReFS file system which is introduced in Windows 8.
• Does not work with Cluster Shared Volumes.
• Does not work with encrypted files, files smaller than 64KB, re-parse points or files with extended attributes.
• Not configurable through Group Policy.
• It is a post-process deduplication process.
• Windows caching is dedupe aware.

How does it work?
For me this is always the most fun question to ask… because when you know how it works, you can understand the use case and the possible gotcha’s when designing an environment that makes use of this technology.
Dedupe looks at the storage from a block-based point of view and divides the storage on ‘chunks’ which are typically somewhere between 32 and 128 KB in size with an average of 80K, although smaller chunks are possible.
To understand dedupe in Windows Server 8, we first have to understand the concept of ‘hard links’.

When data is stored on a file system, the actual bits and bytes are stored on a single location. So, if some bits are the same… why save it multiple times? By using hard links you can refer to bits which can be used by multiple files.
Let’s clarify that one a little… When you have hundreds of *.docs files created by your HR department, they probably use some templates. This means that a lot bits and bytes in the files is exactly the same!
Since dedupe views the storage in chunks, it will notice a lot of those being exactly the same. So, instead of saving the bits and bytes multiple times, it saves the chunk only one time and creates hard links on all locations so they refer to the same data.

When you view the properties of the Program Files folder, you will probably notice that the values behind “Size” and “Size on disk” differ from each other.

This is because some hard links are used for files in this folder. So “Size on disk” involves the accumulated amount of bits and bytes by the files in this folder and “Size” equals the accumulates amount of bits and bites on the disk minus the bits and bites that are replaced by hard links.
Note that the example provided above is file based, where dedupe is block based and provides a far better utilization of the available storage.

I found the diagram below which clearly explains the basic concept of dedupe.

As you can see, some chunks (A, B and C) are used by both files.
By using a technology similar to hard links, but on block-level, all files can access the correct bits and bytes where they only need to be stored once instead of multiple times.

The dedupe process works through scheduled tasks, but can be run interactively by using PowerShell. More about that command later on…

Why use data deduplication?
A valid question… what benefits does dedupe provide? A lot of my customers require massive amounts of storage.
The purpose of dedupe is the better utilize the storage capacity which is available to you.
Microsoft has done some research in their dedupe technology and come up with some numbers on the storage savings dedupe provided:

These numbers come straight from a vendor and these tests may be somewhat optimized for better results.
* As an economy teacher of mine always said:”You give me raw data and the results you want to come out of it, and I’ll provide you a calculation that offers the results you want…”.

Nevertheless these are some pretty impressive numbers! I would love to test this in a production environment and hopefully see the eyes of some IT guys grow, aswell as the smiles of IT managers, when they see the storage savings in their environment

But what about the performance? Any dedupe technology causes some sort of a performance hit, right?
Yeah, that’s true… also with the dedupe in Windows Server 8.
Microsoft has offered some information about this.
Write actions have no direct performance hit since the dedupe process is done in the background when the system is idle.
Read actions do have a performance hit, around 3% when the file is not in cache.

The components of deduplication
Drivers are always ‘fun’ to troubleshoot and since the entire technology of deduplication in Windows Server 8 is based on the filter driver, some understanding of the thing may be useful.

To do this, we have to look at the technology from an architectural view. Where the management of dedupe can be done by Server Manager, PowerShell and WMI, it only manages the dedupe service which in its turn manages the dedupe jobs.
Those dedupe jobs are the ones that talk to the dedupe filter driver that does the actual handling of the chunks of data on the file system. But when data is only stored once, the files will have to know where their data has gone to. That’s where the metadata comes in to play since this is where the location of all the bits and bytes is stored.
With ‘normal files the metadata will only have references to the regular storage. But when a file is affected by the dedupe process, the metadata will not only refer to the regular storage but also to some chunks in the chunk store.
The dedupe service can be scheduled or can run in a background mode while it waits for the system to enter an idle mode so that the system will not experience a negative performance in production hours. This is also called a post-process dedupe mode.

Dedupe and the GUI
The basic management features for dedupe are available in the GUI. Let’s do a quick walkthrough for enabling and configuring the dedupe feature in Windows Server 8.

After installing the File Services role, add the Data Deduplication feature to that role:

Next, you can configure dedupe on a volume:

Now we get the option to configure some settings in dedupe, such as files and folders to exclude… but more interesting is the setting for the minimum amount of days a file must not been changed for the dedup process to pick up this file:

Dedupe and PowerShell
To enable dedupe we have to use my favorite tool: PowerShell.
The first task is add the deduplication feature which is part of the file system role. This can be done by using the Server Manager (GUI)… but where’s the fun in that? You can’t automate that… but by using PowerShell you can
To enable the deduplication feature by using (elevated) PowerShell commands:

Import-Module ServerManager
Add-WindowsFeature -name FS-Data-Deduplication

Now that the deduplication feature has been enabled, we can start configuring.
First, as with any other PowerShell module, we have to load the module. You can do this with the following command:

Import-Module Deduplication

To configure the dedupe feature on volume E on a device:

Enable-DedupVolume E:

Now that dedupe has been enabled and configured on a volume, we want to know some statistics such as what amount of storage we actually saved by using dedupe:

Get-DedupStatus

By default, the dedupe process will only affect files that have not been changed for 30 days. Especially in demo environments this can be a nasty gotcha… you probably don’t want to wait 30+ days for dedupe to start doing its thing…
So, to change this value to 0 (process the file a.s.a.p.) you can use the following command:

Set-DedupVolume E: -MinimumFileAgeDays 0

Normally the dedupe process is done through scheduled tasks in the Windows operating system… but you can start this process manually with PowerShell:

Start-DedupJob E: –Type Optimization

However, this job runs in the background and may take some time. To view the status of that job, the following command can be used:

Get-DedupJob

HELP!!! I’ve done something wrong and I have to disable dedupe on this volume!!
Don’t get your nickers in a twist… again, this can be done by using PowerShell Use this command to un-dedupe the volume:

Start-DedupJob -Volume E: -Type Unoptimization

If you are as enthusiastic about this feature as I am you can read the help for the dedupe PowerShell cmdlets by using this command:

Help Dedup

Problem
Symptom 1: Microsoft Outlook (2010) does not remember the mail profile when logging on for a second time.
Symptom 2: Every time Outlook starts for the first time in a session, it comes with a “First run”-wizard.

Cause
Outlook can be forced to run the “First run”-wizard every first time Outlook is started. When using mandatory profiles, the configuration chance to the user profile will not be saved and when a user starts a new session, Outlook will think the “First run”- wizard has not been run yet.

Workaround
Don’t use mandatory profiles.

Resolution
Remove the following registry keys from your installation:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Office\14.0\User Settings\{C22A96AB-A00C-4A3D-9545-42B4C0005809}\Create\software\microsoft\office\14.0\outlook\Setup
“ImportPRF”=”C:\\PROGRA~2\\MICROS~1\\custom14.prf”
”First-Run”=”hex:00,00,00,00,00,00,00,00”

The registry key for “custom14.prf” is also described in KB242642 in the RES Software Knowledge Base, but without the removal of the “First-Run” key it did not solve my customers issue.

Note: This only applies in environments where mandatory profiles are used.

Last November I went to the Experts 2 Experts Virtualization Conference in London where I met some other Hyper-V enthusiasts.
One of them was Carsten Rachfahl (MVP Virtual Server) who does frequent video interviews with leading experts in the Microsoft IT community. He asked a bunch of us if he could interview us which all of us were happy to do

First up was Aidan Finn (MVP Virtual Server). Aidan also did two presentations at the event, one about what he could tell us about Hyper-V 3.0 and one about designing your backup solution when using Hyper-V Cluster Shared Volumes (CSV) which were both very interesting… His interview is also about designing Hyper-V CSV and Backup:
http://www.it-cast.de/alle/videointerview-with-aidan-finn-about-hyper-v-csv-and-backup/

Carsten’s second victim was Didier van Hoye (MVP Virtual Server since this month… congrats Didier!). Didier did a presentation at the event about 10GB networking in Hyper-V and found this to be very educational
His interview is about Cluster Shared Volumes and Storage design in Windows 8:
http://www.it-cast.de/alle/videointerview-with-didier-van-hoye-about-csv-and-storage-design/

Third up was Ronnie Isherwood who lives in Jersey (no, not that city in the USA but that small island between the UK and France…). Ronnie also did a presentation at the event, about System Center Virtual Machine Manager (SCVMM) 2012 although I had to leave in the middle of the presentation because a colleage contacted me from back home (The Netherlands) with a critical issue and needed some advice. So, altough I was unable to see most of Ronnie’s presentation I heard it was a good one (didn’t expected anything else though ).
His interview with Carsten is about SCVMM 2012, among others it’s multi-hypervisor support:
http://www.it-cast.de/alle/videointerview-with-ronnie-isherwood-virtual-machine-manager-2012/

And the fourth person able to get in from of the camera was… me . I did a presentation at the event about the v-Alliance from Microsoft and Citrix, which was my first presentation at an international IT event.
My interview on the other hand was about the combination of my two favorites: Hyper-V and PowerShell:
http://www.it-cast.de/alle/videointerview-with-jeff-wouters-about-powershell/

And to close a few days of deep technical sessions, meeting community peers and of course a lot of fun, we did a group interview (featuring all of the above) about our favorite Hyper-V feature in Windows 8:
http://www.it-cast.de/alle/videointerview-what-is-your-favorite-windows-8-feature/

During the entire event we’ve talked with each other about a lot of things… and I’m hoping some of the ideas will come to see the light of day in 2012… I’ll keep you posted

The first time I started using Immidio Flex Profiles I immediately noticed one challenge in particular… The settings for every single application needed to be configured manually. Since most people don’t know how their applications work, and me as a consultant that is only with a customer for a short time doesn’t know anything about the internal workings of their applications, this can be a very time intrusive job…

Immidio recognized this issue, started thinking and developing… and within Flex Profiles 7.0 they came up with the Flex Profiles Assistant. Although this made the creating of the Flex Config files much easier, still a lot of manual activities were required.

Somewhere this month (January 2012) Immidio will release their Application Profiler product, which will be licensed as a component for Flex Profiles 7.5. This product makes the creating of Flex Config files much easier!
With this product the steps for creating a Flex Config file will be:

1. Start the Immidio Application Profiler.
2. Start your application from within the (user friendly) interface.
3. After the application is analyzed, save the optimized Flex Config file.
4. Import the Flex Config file into Immidio Flex Profiles.

And you’re done

But, there is more… Flex Profiles provides the feature of “predefined settings”. This means that you can configure some predefined settings for an application which will, for example, enforce the reading pane for Windows Explorer to be set to “details”.
Creating such predefined settings for applications can be a the most time consuming task of an entire Immidio implementation.
Immidio also recognized this issue and provides a solution to automate this process within the Application Profiler product. With this, you can start your application, set it to the defined settings you want, generate the file with those predefined settings and import this into Flex Profiles… much faster compared to ‘the old way’.

Interested? Take a look at the YouTube video which Immidio has placed online showing the Immidio Application Profiler product:

Lots of people that are used scripting languages, such as VBS or Kix, and start using PowerShell contact me about the infamous “Unexpected token ‘and’” error message.
I found this to be funny because I ran into that one myself in my early PowerShell days
So, lets go back in time to the point where I was just started using PowerShell and was playing around with it…

When creating a PowerShell script to find a file based on some criteria, my way of thinking would provide something like this:

$MSI = get-childitem $TargetPath –recurse | where ({$_.Extension -match “msi”} and {$_.Name -match “LiveMessenger”})

But… it will give an error stating that there is an “unexpected token ‘and’”… very annoying.
Since I was used to scripting in VBS, Batch and Kix I thought this was very strange, the syntax should have worked, right? Wrong…
PowerShell is an object based scripting language so a little bit of thinking needs to be changed, compared to what I would like to call ‘VBS thinking’.

With PowerShell it is possible to use the output of a first command as the input of the command that comes after it by using a technique called ‘piping’.
So, instead of using an ‘and’ statement, we could simply use the output of the first filter command as the input of the second filter command.
With this in mind, a change can be made. The following command uses the piping technique instead of an ‘and’ statement… and will work without that nasty “unexpected token ‘and’” error. Why? Simply because the ‘and’ statement is not being used anymore.

$MSI = get-childitem $TargetPath –recurse | where {$_.Extension -match “msi”} | where {$_.Name -match “LiveMessenger”}

And by using the piping technique, you can simply go even further and, if you would like that, you can filter even further!
Next to not getting an error, the piping technique is very CPU friendly compared to the ‘and’ statement.
The ‘and’ statement would cause the entire criteria to be tested on every single object, where the piping technique filters down the objects at every step which causes the list of objects for the second filter to be smaller… and so on. Far more elegant, right?

So, what does this all mean?
If you’re used to scripting languages like VBS or Kix, PowerShell will require a little change in your thinking. Don’t think about AND statements… instead, think about piping such as shown above