Questions & Answers to replatforming webcast

Q&AA list of questions being asked to my most recent webcast on Oracle replatforming together with Madora Consulting.

I decided to put the Q&A on the blog so anyone can benefit instead of just the 100+ attendees we had on the webcast (awesome!)

Disclaimer: Some Q&A are about licensing and my answers are according to the best I know, but always consult with an independent expert (like my copresenter Keith Dobbs who I reviewed the Q&A with). Also the topic is highly controversial so don’t feel offended if you disagree ;)

Question: For normal data guard should i have to pay for normal DB EE or SE licenses or no need ?

Bart: Data Guard is free but requires both primary and standby host to be fully licensed. It requires Enterprise Edition. If you use advanced features such as running read-only queries on the standby, then you need Active Data Guard licenses on both nodes. Note that there is no technical difference between “normal” and “active” Data Guard. You need to pay for the license as soon as you use the Active Data Guard features.

Question: I’d like to know about replication Virtualized Oracle DB Server using Vsphere , should I worry about consistency?

Bart: VMware (vSphere) does not buffer any IO and passes any writes directly to the underlying storage. Consistency is therefore not dependent on whether or not you are running virtualized or physical. If you use any kind of storage replication (such as Dell EMC Recoverpoint, SRDF, VPLEX etc) then you must make sure that all volumes for a database are replicated as one “consistency” group, such that after a failover you don’t get timing issues between the different database volumes. As said, this is not depending on whether you run virtual or physical.

Also, I highly recommend testing D/R failover at least every 6 months or so, to make sure your D/R strategy is working in case you really need it. I have experience with customers who added local volumes but forgot to replicate those volumes which resulted in non-recoverable D/R systems. Using Dell EMC snapshot features of your storage array allows you to perform D/R testing during normal weekdays so your primary database and DR replication stay active even while you perform D/R testing (a method that you cannot use the same way with Data Guard unless you break replication or use other workarounds).

Question: How to use VMware in a DR site and the production site has engineered system ? Without high cost

Bart: Interesting scenario. Assuming Engineered System is either Exadata (most likely) or Supercluster (where the storage is the same as Exadata), the challenge here is that Oracle does not allow connectivity of non-Oracle storage, so we cannot use classic SAN replication here. We need to discuss in more detail but a few options you may consider:

  • Backup Exadata with EMC Data Domain, replicate to another Data Domain in the D/R site and use that to recover your database (you can use it to daily update your standby using replicated backup files). This would be a very cost efficient solution but the trade-off is that it would take some time to recover after a failover (so the RTO – recovery time objective) would not be close to zero.
  • Use Data Guard (the free version i.e. not using Active DG features) to replicate to a (Dell EMC) system. There are some disadvantages here and care must be taken that you don’t use Hybrid Columnar Compression – as this would have to be de-compressed first in order to start the standby database
  • Best solution: re-platform production from Engineered System to a Dell EMC proven solution and then you can use any D/R method you like, as well as save on license cost as we have shown in the presentation

Question: How are Madora (consulting) funded / paid for ? How does the local DEll EMC (Think DELL folks) Business Units generate revenue from a Madora engagement?

Bart: I assume this question was asked by one of our Dell EMC colleagues on the call. But sharing it with everyone because no secrets…

I’ve had this question before several times so it’s an important point, and we should have that in a future version of the presentation. There is no formal partnership between Madora and Dell EMC, because Madora must be able to provide a completely independent advice to customers and not be biased because being sub-contracted by Dell EMC. So for Dell EMC sales teams there is no direct revenue stream, however, the chance of winning infrastructure deals increases because

  • customers can save money and spend it on the infrastructure that is needed to achieve those savings
  • customers can eliminate uncertainty and make sure they are – and stay – compliant and don’t have to worry about project risks
  • bringing in license expertise can eliminate the objections raised by the customer on licensing, support, audits etc.

The relationship is purely an informal one but that said, we’ve cooperated succesfully on a number of occasions.

Question: How and why can a Customer move to Standard Edition?

Bart: If you have a database on Enterprise Edition and want to move to Standard Edition (Standard Edition 2 – SE2 – in future deployments as of Oracle version 12) then you must consider if there are any Oracle features in use that are not available in SE/SE2. If that is not the case, you can install a server with SE and move the datafiles, then restart the database. Again, test the scenario before go live as the devil is often in the details. Also you need to make sure you have enough SE(2) licenses because you are not allowed to run Standard Edition using Enterprise Edition licenses (it is essentially a different product). Madora may be able to help converting or re-negotiating licenses. Oracle will probably push back because it’s a huge reduction in their revenue stream.

Question: Can you convert Exadata quarter rack to ovm and reduce the licenses of DB and reuse them in DR site ? without buying new licenses.

Bart: (assuming OVM is Oracle Virtual Machine) – As long as the total number of physical processors is licensed then that should be possible from a licensing standpoint. Note that with OVM you have many of the same (perceived or real) license issues as with VMware (but Oracle doesn’t tell you that). Note that I would not recommend running OVM for consolidation of mission critical systems as it’s not best practice and does not offer the same level of performance management, 3rd party tooling integration and features as VMware, however, on Exadata it may be your only viable option next to running physical. I haven’t seen any customers yet who run OVM on Exadata for mission-critical workloads yet (but there sure will be a few out there – let me know your experiences!) so make sure you know what you are doing before starting such a journey.

Final thought:

I get many more questions during my customer meetings. Maybe I should put a static FAQ page on the topic that I update with new questions, answers and insights… Let me know what you think!

This post first appeared on Dirty Cache by Bart Sjerps. Copyright © 2011 – 2017. All rights reserved. Not to be reproduced for commercial purposes without written permission.

Tales from the past – Disaster Recovery testing

A long time ago in a datacenter far, far away….

Turmoil has engulfed the IT landscape. Within the newly formed digital universe,
corporate empires are becoming more and more
dependent on their digital data and computer systems.
To avoid downtime when getting hit by an evil strike, the corporations are
starting to build disaster recovery capabilities in their operational architectures.

While the congress of the Republic endlessly debates whether
the high cost of decent recovery methods is justified,
the Supreme CIO Chancellor has secretly dispatched a Jedi Apprentice,
one of the guardians of reliability and availability,
to validate existing recovery plans…

Another story from my days as UNIX engineer in the late nineties. I obfuscated all company or people names to protect their reputation or disclose sensitive information, but former colleagues might recognize parts of the stories or maybe everything. Also, some of it is a long time ago and I cannot be sure all I say is factually correct. The human memory is notoriously unreliable.

oobsignIn those days, our company was still relying on tape backup as the only Disaster Recovery (DR) strategy. The main datacenter had a bunch of large tape silos, where, on a daily basis, trays of tapes were unloaded, packed and labeled in a small but strong suitcase, and sent to an off-site location (Pickup Truck Access Method) so the invaluable data could be salvaged in case our entire datacenter would go up in flames.

Read more of this post

Big Ideas; Big Tech: Continuous Operations for Oracle RAC with EMC VPLEX

Here’s an EMC video on Youtube about Oracle RAC with EMC VPLEX. Very nice, check it out!

http://www.youtube.com/watch?v=DRtl6dU2P_E

vplex-2

vplex-1

Why clone databases for firefighting

clonesAs more and more customers are moving their mission-critical Oracle database workloads to virtualized infrastructure, I often get asked how to deal with Oracle’s requirement to reproduce issues on a physical environment (especially if they use VMware as virtualization platform – as mentioned in Oracle Support Note # 249212.1).

In some cases, database engineers are still reluctant to move to VMware for that specific reason. But the discussion is not new – I remember a few years ago I was speaking in Vienna to a group of customers and partners from Eastern Europe, and these were the days we still had VMware ESX 3.5 as state-of-the-art virtualization platform. Performance was a bit limited (4 virtual CPUs max, some I/O overhead and memory limitations) but for smaller workloads it was stable enough for mission critical databases. So I discussed the “reproduce on physical in case of problems” issue and I stated that I never heared of any customer who really had to do this because of some issues. Immediately someone in the audience raised his hand and said, “well, I had to do that once!” – Duh, so far for my story…

Let’s say that very often I learn as much from my audience as (hopefully) the other way around ;-)

Later I heard of a few more occasions where customers actually were asked by Oracle support to “reproduce on physical” because of suspected problems with the VMware hypervisor. In all of the cases I am aware of, the root cause turned out to be elsewhere (Operating System or configuration) but having to create a copy in case of issues is a scary thought for many database administrators – as it could take a long time and if you have strict SLAs then this might bite back at you.

So what is my take on this?

Read more of this post

The Zero Dataloss Myth

In previous posts I have focused on the technical side of running business applications (except my last post about the Joint Escalation Center). So let’s teleport to another level and have a look at business drivers.

What happens if you are an IT architect for an organization, and you ask your business people (your internal customers) how much data loss they can tolerate in case of a disaster? I bet the answer is always the same:

“zero!”

This relates to what is known in the industry as Recovery Point Objective (RPO).

Ask them how much downtime they can tolerate in case something bad happens. Again, the consistent answer:

“none!”

This is equivalent to Recovery Time Objective (RTO).

Now if you are in “Jukebox mode” (business asks, you provide, no questions asked) then you try to give them what they ask for (RPO = zero, RTO = zero). Which makes many IT vendors and communication service providers happy, because this means you have to run expensive clustering software, and synchronous data mirroring to a D/R site using pricey data connections.

If you are in “Consultative” mode, you try to figure out what the business really wants, not just what they ask for. And you wonder if their request is feasible at all, and if so, what the cost is of achieving these service levels.

Read more of this post

Oracle snapshots and clones with ZFS

Another Frequently Asked Question: Is there any disadvantage for a customer in using Oracle/SUN ZFS appliances to create database/application snapshots in comparison with EMC’s cloning/snapshot offerings?

Oracle marketing is pushing materials where they promote the ZFS storage appliance as the ultimate method for database cloning, especially when the source database is on Exadata. Essentially the idea is as follows: backup your primary DB to the ZFS appliance, then create snaps or clones off the backup for testing and development (more explanation in Oracle’s paper and video). Of course it is marketed as being much cheaper, easier and faster than using storage from an Enterprise Storage system such as those offered by EMC.

Oracle Youtube video

Oracle White paper

In order to understand the limitations of the ZFS appliance you need to know the fundamental workings of the ZFS filesystem. I recommend you look at the Wikipedia article on ZFS (here http://en.wikipedia.org/wiki/ZFS) and get familiar with its basic principles and features. The ZFS appliance is based on the same filesystem but due to it being an appliance, it’s a little bit different in behaviour.

So let’s see what a customer gets when he decides to go for the Sun appliance instead of EMC infrastructure (such as the Data Domain backup deduplication  system or VNX storage system).

Read more of this post

Oracle RAC on VPLEX now certified

Last week EMC announced that Oracle RAC on VPLEX stretched clusters is now officially supported and certified by Oracle!

News Summary:

 

  • Oracle has certified that EMC® VPLEX™ METRO in a stretch cluster configuration can provide Oracle Real Application Clusters (Oracle RAC) customers with an easy-to-deploy, active/active solution, as they transform from single- to dual-site environments.
  • Having passed Oracle’s rigorous testing standards, the EMC VPLEX METRO solution can enable Oracle RAC to be easily configured over extended distances while enabling simultaneous access to the same data at both locations.

This is the final step in a process to help customers that have been asking for true active/active support over distance for their mission-critical Oracle Database business processes.

For those who are not yet familiar with this solution, here is a small summary:

  • Customers have been in search for ways to survive datacenter failures (i.e. “disasters”) without the need to recover and restart the databases, in such a way that any component failure or even complete site failure would not lead to database downtime
  • This was not possible before except when deploying complex configurations based on host mirroring using Oracle ASM or a 3rd party volume manager. (note that competing storage virtualization products from other storage vendors also do not offer this full capability – even though their marketing might make it seem so)
  • EMC VPLEX offers this functionality which is now completely certified and supported by Oracle, and the solution avoids risk by making the stretched cluster deployment as easy as a basic Oracle RAC install
  • The VPLEX solution offers additional benefits including better performance, better recovery from issues such as component or link failures and offers a complete solution for the whole application stack, not just Oracle
  • Note that AFAIK this solution should also work for IBM DB2 (but I haven’t confirmed)

The full news release can be found here: http://www.emc.com/about/news/press/2012/20120517-04.htm

A full series of blog posts on this solution can be found here: https://bartsjerps.wordpress.com/category/vplex/

The VPLEX witness (the final component of VPLEX that made this possible) was announced last year at EMC World 2011. Typically we see the start of market adoption between 1 to 1.5 years after bringing new technology in the market. I am working on a few customers myself who are on the edge of starting a project with this, hopefully by the end of year we have a set of good customer references!

Update: The new white paper can be found here: http://www.emc.com/collateral/software/white-papers/h8930-vplex-metro-oracle-rac-wp.pdf

Update 2: VPLEX support mentioned (briefly) on Oracle’s website: http://www.oracle.com/technetwork/database/enterprise-edition/tech-generic-linux-new-086754.html

Update 3: Demos available on EMC Demo Center:

EMC VPLEX Metro for Oracle RAC Solution Overview
Oracle RAC with VPLEX Metro Site Failure
Oracle RAC with VPLEX Metro Solution Overview
Oracle RAC with VPLEX Storage Failure

If you’re a frequent reader of my blog you might recognize familiar pictures there ;)

Data Guard protecting from EMC block corruptions?

Today I was giving a training to fellow EMC colleagues on some Oracle fundamentals. One of the things that was mentioned is something I have heard several times before: Oracle is claiming that EMC SRDF (a data mirroring function from EMC Symmetrix enterprise storage systems mainly to provide enterprise disaster recovery functions) cannot detect certain types of data corruption where Oracle Data Guard can. Ouch. The trouble with this statement is that it is half-true (and these ones are the most dangerous).
Read more of this post

POC: Piece Of Cake or Point Of Contradiction?

Every now and then I get involved in Customer Proof of Concepts. A Proof of Concept (POC) is, according to Wikipedia, something like a demonstration of feasibility of a certain idea, concept or theory.

Concept Performance Aircraft

Concept Aircraft

Read more of this post

Data Guard or Storage based replication?

A comparison between Oracle (Active) Data Guard and EMC replication for disaster recovery purposes

Panic Button
This is an article I wrote a while ago for customers’ Database Administrators (DBAs) and application managers, that helps them in selecting the right Disaster Recovery tools for their business applications.
It is slightly modified to update new insights and to make it more readable on the web.

Read more of this post