By Karl Dohm, HP Storage Architect
Today I'm taking a few minutes to respond to some of the comments regarding my initial post on Making Sense of WAFL.
Apparently in that post I unwittingly opened up a few of NetApp's old wounds which have been extensively hashed through previously in public forums. Looking through the responses, NetApp has done a nice job of trying to deflect some of these problems through releases of nice looking apparently credible documentation.
For those that are biased NetApp's way, or are enamored with the technical ways of WAFL, there may be nothing to say to convince you otherwise. But for those with an open mind, read on.
The problems we are talking about here are the core of WAFL, and are clearly not easy to fix - or they would be already fixed. NetApp is not unique is having problems of course, all array vendors have their strong and weak points. But to assert that WAFL has no weaknesses around fragmentation, performance, and capacity utilization defies common sense. The old wounds are there for a reason.
Let's take a look at the Avanade white paper. It glows with enthusiasm about how the FAS3050c performs in MS Exchange based environments. Further detail from the paper's author can be found in an interview here. Peeling back the onion a bit, we see that this paper was created shortly after creation of a business partnership between NetApp and Avanade. Evidence of this partnership can be found here.
The IOmeter baseline performance data cited in the paper is interesting and worth exploring. In the words of the white paper's author the IOmeter test against the FAS3050c had.. "two goals: to validate that our environment was set up correctly and to assess overall performance of the FAS3050c".
The report is exceptionally loose about describing the setup. The transfer size used for IOmeter are claimed to range from .5KB to 64KB in size, but there is no indication on the weight applied to portions of this range. There is no mention of percent reads/writes or percent random vs sequential. It also doesn't discuss MPIO policy or HBA queue depth setting. There is no indication whether OnTap Exchange extents are enabled. Worst of all, and unique to NetApp, it doesn't define the history of writes and therefore level of fragmentation on the LUN.
I like IOMeter because its a relatively simple test to run that is available for anyone to try since its in the public domain. Given this open invitation to compare results with Avanade, it made sense to give the described test a try and see what happens.
It turns out that no matter what combination of the unspecified test parameters I tried, I could never get into the ballpark of results claimed in this white paper.
So to illustrate an example, I decided to just keep things simple as possible. Running a typical exchange 2008 simulation load of 8KB transfer size, 80% random, 60% read, IOmeter queue depth of 128, MPIO round robin, Exchange extents enabled, HBA max queue depth of 254, 20x 15K spindle raid-dp aggregate, and letting the LUN settle through its fragmentation period, the throughput settles at 19MBs at a average latency of 52msec.
The white paper claimed the FAS3050c runs 48 MB/s at 30msec latency, which is a world of difference.
So what gives? One of several things has happened. Perhaps I could not successfully piece together how to run this test from the information given. It would be great to get clarification from NetApp on how to properly run this test and recreate the results. The other explanation is perhaps that the results are not re-creatable without some special internal-use-only tuning parameters. Or perhaps there is no way to recreate these results.
An EVA4400, run with the same workload, experiences approximately 39MB/s at 25 msec average latency. That's about twice the thoughput on a workload that is mostly random, meaning the bottleneck is supposed to be at the spindles. Apparently on the FAS the bottleneck is somewhere else.
Incidentally, this FAS 3050c LUN degraded about 10% in MB/s throughput as the fragmentation settled out. That isn't such a big number, but recall that this test is mostly random I/O. The sequential read portion, if looked at in isolation, degrades much worse. It is why NetApp introduced Exchange extents.
As in my previous post, if you don't believe what I am saying, give it a try. Unlike my colleagues at NetApp, I gave you enough information here to run the test.
Barring sound explanation from NetApp, It seems to me that there is reason to doubt the credibility of the white papers and test results that NetApp is producing.
Posted
11-08-2008 12:04 AM
by
CalvinZ