The Quick and Dirty Dedupe Analyzer – Part 1 – Hands on

As announced in my last blogpost, qdda is a tool that analyzes potential storage savings by scanning data and giving a deduplication, compression and thin provisioning estimate. The results are an indication whether a modern All-Flash Array (AFA) like Dell EMC XtremIO would be worth considering.

In this (lenghty) post I will go over the basics of qdda and run a few synthetic test scenarios to show what’s possible. The next posts will cover more advanced scenarios such as running against Oracle database data, multiple nodes and other exotic ones such as running against ZFS storage pools.

[ Warning: Lengthy technical content, Rated T, parental advisory required ]

Read more of this post

Featherweight Linux VNC services

This article describes how to set up a very lightweight VNC service under CentOS/Red Hat.



In Red Hat Enterprise Linux (and derivates, I use CentOS) you can run a VNC service to allow graphical connections to a linux system. I was looking for a very lightweight VNC service (no fancy desktop with all the bells and whistles, just something that lets me do some stuff that requires an X session and run an Xterm – such as installing Oracle or running Swingbench, without using another host with an X client). In other words, a typical service for virtual machines that run as servers (such as database servers, web servers, etc).

CentOS standard method

I tried the standard documented way to do this in CentOS: CentOS Virtual Network Computing using the standard tigervnc-server method, but found a few issues with the way they set it up:

  • For every user requiring VNC services, you need to customize the configuration
  • If one user deletes or corrupts his VNC password file, the whole service stops working (fix via normal SSH login but requires skilled user)
  • If a user messes up his xstartup file he is locked out (fix via normal SSH login but requires skilled user)
  • Users need 2 passwords: for their (own) VNC service, and the usual one for Linux
  • Their X window and VNC processes are always running and thus eating resources even if not used
  • If their X session hangs (i.e. window manager killed, or simple logout) it’s hard or even impossible to clean up and restart (see section 4 in the mentioned article: Recovery from a logout) without resetting the whole VNC service
  • Every user requires a separate, unique TCP port

All by all, nice and easy for a small test server with a few users, but no good for larger environments. The good thing is that the desktops are persistent, i.e. you may disconnect and reconnect later and the VNC session will be as you left it. And you can install lighter desktop environments (twm or openmotif) instead of the huge and heavy Gnome desktop.

But I was looking for something better.

Read more of this post

Debunking Oracle certification myths

Another frequently asked question I get asked a lot:

Is Oracle certified on Vmware?

There are plenty articles discussing this very topic, here’s a few examples:

oracle blog – is Oracle certified on VMware
vmware understanding oracle certification support licensing environments – oracle linux fully supported vmware esxi and hyper-v
longwhiteclouds – fight the fud oracle licensing and support on vmware vsphere/
oraclestorageguy – what the oracle vmware support statement really means and why
everything oracle @ emc – vmwares official support statement regarding oracle certification and licensing

…and yet it still seems to bother many people I talk to when showing the clear and present benefits of going all-virtual.

It seems there is a lot of confusion between the meaning of “certified”, “supported”, and even the term “validated” comes up every now and then. To make things worse, the context in which those words are used makes a big difference.
Read more of this post

ZFS and Database fragmentation

Disk Fragmentation

Disk Fragmentation – O&O technologies.
Hope they don’t mind the free advertising

Yet another customer was asking me for advice on implementing the ZFS file system on EMC storage systems. Recently I did some hands-on testing with ZFS as Oracle database file store so that I could get an opinion on the matter.

One of the frequent discussions comes up is on the fragmentation issue. ZFS uses a copy-on-write allocation mechanism which basically means, every time you write to a block on disk (whether this is a newly allocated block, or, very important, overwriting a previously allocated one) ZFS will buffer the data and write it out on a completely new location on disk. In other words, it will never overwrite data in place. Now a lot of discussions can be found in the blogosphere and on forums debating whether this is really the case, how serious this is, what the impact is on performance and what ZFS has done to either prevent, or, alternatively, to mitigate the issue (i.e. by using caching, smart disk allocation algorithms, etc).

In this post I attempt to prove how database files on ZFS file systems get fragmented on disk quickly. I will not make any comments on how this affects performance (I’ll save that for a future post). I also deliberately ignore ZFS caching and other optimizing features – the only thing I want to show right now is how much fragmentation is caused on physical disk by using ZFS for Oracle data files. Note that this is a deep technical and lengthy article so you might want to skip all the details and jump right to the conclusion at the bottom :-)

Read more of this post

Performance – The database stack

hamb-stackAs mentioned before, I frequently find myself in discussions around Oracle performance and how an Oracle database behaves on EMC storage. It turns out that often there is a lot of confusion on how the different layers interact with each other and very few people seem to understand the whole stack.

So I started a personal challenge to make a “one picture tells more than 1000 words” complete overview of the Oracle on EMC database stack.

I failed.

Turns out it’s nearly impossible to get everything in one picture without cutting corners.

So here is a simplified (and therefore incorrect) picture. It ignores certain complexities and is far from complete, and might even contain errors.

Read more of this post

Limitations of host-based mirroring for stretched clusters

For data mirroring, EMC SRDF is sometimes used in such a setup that both servers write to one location only (the “far” server writes across dark fibre links to the local storage). EMC has similar tools (Mirrorview, Recoverpoint, etc) for other storage platforms than Symmetrix.

srdf cluster

SRDF cluster with passive target

Read more of this post

Pinpointing I/O bottlenecks on Linux

Does this story sound familiar?

The end users of a database application start complaining about poor system response and long running batch jobs. The DBA team starts investigating the problem. DBA’s look at their database tools such as Enterprise Manager, Automatic Workload Repository (AWR) reports, etc. They find that storage I/O response times are too high (such as an average of 50 milliseconds or more) and involve the storage team to resolve it.

The storage guys, in turn, look at their tooling – in case of EMC this could be Navisphere Analyzer, Symmetrix Performance Analyzer (SPA) or similar tools. They find completely normal response times – less than 10 milliseconds average.

The users still complain but the storage and database administrators point to each other to resolve the problem. There is no real progress in solving the problem though.

Two Way Communications

Read more of this post

How to set disk alignment in Linux

As you might know, if disk partitions containing Oracle datafiles are not aligned with the underlying storage system, then some I/O’s can suffer from some overhead as they are effectively translated in two I/O’s.

If you want more info, google for “EMC disk alignment” and you’ll find plenty of information, explaining the issue.

Update 28-03-2013: I wrote a follow-up for this post describing the same thing for Linux (Red Hat / CentOS / OEL) versions 6. For that, you might want to jump straight to the new post as this one gets a bit outdated ;-)
Read more of this post