Windows Server 2016 —What’s New in Data Deduplication

Deduplication eliminates the need to repeat data to create a single instance. The creation of the single instance improves storage utility and efficiencies in a network with heavy network transfers.

Some may confuse deduplication with data compression, which identifies repeat data within single files and encodes the redundancy.

In simple terms, deduplication is a continuous process that eliminates excess copies of data; therefore, decreasing storage demands.

Data deduplication applies to Windows Server, the Semi-annual Channel, and Windows Server 2016.

Data deduplication in Windows Server 2016 is a highly optimized, manageable, and flexible process.

Here are the updated and new data deduplication features in Windows Server 2016.

The Updated Features

Here are two of the updated features.

1. Support for Large Volumes

In earlier versions, volumes were partitioned to fit data sizes that are above 10TB.

However, in Windows Server 2016, data deduplication supports volume sizes of up to 64TB.

• What is the Added Value?

The volumes in Windows Server 2012 R2 had to be appropriately portioned in the correct sizes to ensure optimization demands keep up with the rate of data transfer.

The implication here was that data deduplication could only work on volumes with data of 10TB or less. The performance also depended on existing workloads on write patterns.

• What is Different?

Windows Server 2012 R2 uses a single thread and an input and an output queue for every volume.

This is to maximize optimization and make sure jobs do not fall behind, which can affect the volume’s overall saving rate. This way, large data sets have to be broken into small volumes.

The volume size depends on the expected partition size; the maximum size is between 6 and 7TB for high volumes and 9 and 10TB for low volumes.

Windows Server 2016 has a new way of working with data deduplication: it runs on more than one thread and uses multiple inputs and outputs for every volume.

This introduces a new routine that was only possible after dividing data into small chunks.

2. Support for Large Files

In earlier versions, any file approaching the 1TB size was not eligible for deduplication.

However, Windows Server 2016 supports files with a maximum size of 1TB.

• What is the Added Value?

In Windows Server 2012 R2, you cannot deduplicate large files due to reduced performance in the deduplication process queue.

In Windows Server 2016, deduplication of files of up to 1TB is possible.

Consequently, this enables you to save a large volume of work; for example, reduplicating large backup files.

• What is Different?

Windows Server 2016 deduplication process uses new streaming and mapping structures to improve the deduplication output and its access.

Besides, the process can now be optimized when there is a failure, instead of restarting the entire process. Deduplication affects files with a capacity of 1TB.

The New Features

Here are three of the new features.

1. Support for Nano Servers

Nano servers support is a new feature that is available in any Nano Server Deployment option in Windows Server 2016.

• What is the Added Value?

Nano servers is a headless deployment in Windows Server 2016 that need a smaller system for tracing resources. It enables quick startups and needs fewer updates and restarts than the Windows Server Core Deployment version.

2. Simple Backup Support

The Windows Server 2012 R2 support Virtualized Backups, like Microsoft Data Protection Manager, after successful manual configurations.

Windows Server 2016 has some new default backups that allow for seamless data deduplication for Virtual backups.

• What is the Added Value?

For this to happen in earlier versions of the Windows Server, you needed to manually tune deduplication settings, as opposed to Windows Server 2016 that has a simplified process for its Virtualized backup applications.

Server 2016 enables deduplication for each volume, just the same way as the General Purpose File Servers.

3. Support for Clusters Operating System Rolling Upgrade

Data deduplication is capable of supporting the new Cluster OS Rolling Upgrade feature in Windows Server 2016.

• What is the Added Value?

The failover clusters in Windows Server 2012 R2 can have a mix of nodes that run deduplication alongside nodes that operate Windows Server 2016 versions of deduplication.

This improvement adds full access to the data that is being deduplicated during the rolling upgrade.

Consequently, it allows the gradual rollout of the new version of data deduplication on an existing Windows Server 2012 R2 cluster without experiencing downtimes during the upgrading process.

• What is Different?

In earlier versions of the Windows Server, a failover cluster required that all nodes in a cluster must be of the same Windows Server version.

However, in Windows Server 2016 version, the rolling upgrades allow clusters to run in mixed modes.