In this NetApp training tutorial, you’ll learn about inline and postprocess storage efficiency operations. I’ll explain what the default settings are on both AFF and FAS systems and how they differ. Moreover, I’ll also explain the various caveats about what can be configured where. Scroll down for the video and also text tutorials.
NetApp Inline vs Postprocess Storage Efficiency Video Tutorial
Charles Lawton
Just completed NetApp Storage training! I found the instructor Neil Anderson to be amazing. He’s extremely competent, has spent time in the industry, and knows the platform very well. He colors the materials with some real world examples, which is always helpful to understand the differences between doing something in the lab and doing it in the real world.
I manage several NetApp arrays, and found myself going to the real platform to see how we’ve implemented the concepts presented here. Very happy I picked up this course!
Inline and Postprocess Storage Efficiency
Deduplication and compression can both run both postprocess and inline. Compaction, on the other hand, is an inline operation only. The difference between postprocess and inline is that inline operations run before being written to disc, postprocess run after.
Inline Storage Efficiency
Looking at the effects that wherever you're using inline and-or postprocess can have, inline operations can reduce writes to desk prolonging the life of SSDs. Because you're squeezing the data in a smaller amount of physical space, that's lesser writes going to the SSDs and giving them a longer life.
They also increase parallelism because they're multiprocessor-capable. They take place while data is still in memory before being written to disk. Therefore, they're best suited to faster controllers, basically meaning the AFF.
Inline operations can increase latency when using each HDDs. You can enable inline operations on FAS systems, but it's generally not a good idea. If you are thinking of doing it, you must conduct a performance testing.
Storage efficiency is great because it means that you can fit the same data into less physical space. You get cost savings there, but you also need to be careful that it's not going to cause performance degradation.
It is not likely a problem on AFF systems which are very fast, but it's something that you do need to test on FAS systems before you enable it. Because of this, inline storage efficiency is enabled by default on AFF systems, since they can handle it. It's not enabled by default on FAS systems but you should test it first.
Storage Efficiency
Postprocess deduplication must be enabled on a volume before you can enable postprocess compression. You'll see this when you enable it in the GUI, you have to tick the checkbox for postprocess deduplication before you can check the checkbox for postprocess compression.
When you configure it in the command line, the first thing that you do is enable postprocess deduplication. Then you can then enable the other storage efficiency features.
Another caveat, storage efficiency is not supported on a volume which has been configured as semi-thick provisioned with the -space-slo semi-thick option. One of the requirements for overwrite guarantee for our LUNs is that deduplication and compression are turned off.
Deduplication
Let's look at deduplication, compression, and compaction in a little more detail. Deduplication can be enabled at the volume level in FAS systems. In AFF systems, it can be enabled at the volume level and the aggregate level as well.
The volumes in an aggregate must all be thin provisioned for aggregate level inline deduplication to be enabled on your AFF systems.
Inline and Postprocess Deduplication
Inline deduplication is enabled by default on AFF systems. It's not supported by default on FAS systems except for flash pool aggregates. You can turn it on on your FAS systems as well, but you need to do that at the command line. Postprocess deduplication can be run on all systems, including FAS.
Inline and Postprocess Compression on FAS
Looking at compression, FAS systems support both inline and postprocess compression. If you enable both, then you must enable postprocess compression on the volume before enabling inline compression.
Inline Compression on AFF
Only inline, not postprocess compression, is supported on AFF systems. It is enabled by default. If you disable it and then want to turn it back on again, you would need to enable postprocess compression first but, it will be skipped.
So you know how I just said that postprocess compression is not supported on AFF? It's just a funny thing because of the way that the operating system works and because this is how it works on FAS. You have to turn on postprocess compression first.
It's the same on the AFF, you have to turn on postprocess compassion first, but that will actually just be skipped. It won't actually be run. It's only the inline compression that will be run.
Data Compaction
Compaction is an inline only operation. We don't have postprocess for that. It occurs during the consistency point after inline compression and the deduplication. Compaction runs are the volume level and volumes must be thin provisioned for compaction to be enabled.
Inline Compaction
Compaction is enabled by default on AFF. You can also enable it on FAS systems.
Default AFF Inline Storage Efficiency Operations
Let's look at those defaults starting with the AFF and the default inline operation:
1. The first thing that happens is inline zero block deduplication. So basically, empty blocks will be deduplicated.
2. Next up is inline adaptive compression. If you remember when we covered compression earlier, we've got adoptive and secondary compression. Adaptive is better suited for AFF systems, so that is the default here.
3. Next, we have inline deduplication. First off, duplicates within a volume will be deduplicated, then duplicates within an aggregate if not found within a volume.
4. Finally, we have inline data compaction. Inline deduplicated blocks are not compacted.
Those are all the inline operations and that is the order that they are processed in as well. You can see that deduplication, compression, and compaction are all enabled inline by default on the AFF.
Default AFF Postprocess Storage Efficiency Operations
Postprocess deduplication is enabled by default on AFF systems as well. The operation is triggered automatically when a large enough percentage of a change log has been populated.
So since the last background deduplication, when there is a new 20% (by default) of data being written to that volume or aggregate, then the postprocess background deduplication will run again.
On the AFF, only inline compression is supported, and compaction is an inline operation. Thus, the only postprocess operation that happens in the AFF is the duplication by default.
Default FAS Storage Efficiency Operations
Next up, looking at the defaults on the FAS. Deduplication, compression, and compaction are disabled by default on FAS systems.
As I was saying earlier, this is because you want to check the impact it'll have on performance before you run them. Volume-level deduplication, compression, and compaction can all be enabled on FAS.
Aggregate-level deduplication is only available on AFF systems. Postprocess deduplication can be enabled on all volumes, and inline deduplication is not supported by default, but you can turn it on.
Both postprocess and inline compression can be enabled and data compaction is an inline operation that can be enabled.
Postprocess Operations
Now, let's have a look at how everything works for storage efficiency in a bit more detail. You can run postprocess deduplication and data compression operations by specifying a threshold percentage that waits for new data to exceed the specified percentage which is 20% by default.
The default that happens in the AFF systems is that after 20% of new data is written to the volume since the last operation, it will run again. You can also enable that on the FAS systems as well.
The other way that you can run your scans is at a scheduled time. You can set a maximum time period for it to run in as well. Finally, you can also run the operations manually on demand for your deduplication and compression.
Postprocess compression uses the same schedule as deduplication. This is why you have to enable deduplication first before you can turn on the compression. When both are enabled, the compression runs first, followed by deduplication.
Scanning Existing Data
The last thing to tell you is that storage efficiency applies to new data after it is enabled. This is important. If you've got an existing volume which has already got data in there and you didn't turn on storage efficiency when you first created the volume, the existing data that is already there is not going to be deduplicated and compressed by default.
However, you can manually scan that existing data which is in the volume. Storage efficiency must be enabled to perform a manual scan.
Additional Resources
Storage Efficiency: https://docs.netapp.com/us-en/ontap/cloud/storage-efficiency-concept.html
Storage Efficiency Features: https://library.netapp.com/ecmdocs/ECMP1368855/html/GUID-4CD5BFC7-1EEC-4F15-A638-9A803BBBBC5B.html
Efficient Storage: How NetApp Guarantees Storage Efficiency for AFF Systems: https://www.netapp.com/blog/netapp-guarantees-storage-efficiency-aff-systems/
Click Here to get my 'NetApp ONTAP 9 Storage Complete' training course.
Libby Teofilo
Text by Libby Teofilo, Technical Writer at www.flackbox.com
With a mission to spread network awareness through writing, Libby consistently immerses herself into the unrelenting process of knowledge acquisition and dissemination. If not engrossed in technology, you might see her with a book in one hand and a coffee in the other.