The Quick and Dirty Deduplication Analyzer

The best thing about being me… There are so many “me”s.

— Agent Smith, The Matrix Reloaded

One of our customers reported less than optimal space savings on XtremIO running Oracle. In order to test various scenarios with Oracle I was in search of a deduplication analysis method or tool – only to find out that there was nothing available that qualified.

TL;DR: QDDA is an Open Source tool I wrote to analyze Linux files, devices or data streams for duplicate blocks and compression estimates. It can quickly give you an idea of how much storage savings you could get using a modern All-Flash Array like XtremIO. It is safe to use on production systems and allows quick analysis of various test scenarios giving direct results, and even works with files/devices that are in use. No registration or uploading of your confidential data is required.

Thin Provisioning

Some customers ask us – not surprisingly – how they can reduce their total cost of ownership in their information infrastructure even more. In response, I sometimes ask them what the utilization is of their storage systems.

Their answer: often something like 70% – you need of course some spare capacity for sudden application growth, so close to 100% is probably not a good idea.

Overallocating storage

If you really measure the utilization you often find other numbers. And I don’t mean the overhead of RAID, replication, spare drives, backup copies etc. because I consider these as required technology – invisible from the applications but needed for protection and so on. So the question is – of each net gigabyte of storage, how much is actually used by all applications?