If you’re like me, you probably don’t bother reading through all the fine print before accepting software licenses.  When I’m awake at 3 AM and I need to fall asleep, I’ll install a new piece of software and read the End User Licensing Agreement (EULA) to put myself out.  Works every time.

Rock a bye baby, licensed as follows...

Buried in the fine print in arcane legalese, however, is some stuff that really matters to you: you probably can’t publish performance benchmarks or competitive reviews of software without getting approval from the vendor.

For example, here’s a quote from VMware Server’s EULA:

You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at benchmark@vmware.com to request such review.

When I first read about this, back before I came to work for Quest, I hit the roof. How dare they stop us, the community, from testing their software and writing about it?  People need to know about the exact performance overheads of virtualization before they implement it.  We need to work together to share this information, right?

The more underground reviews I read, the more uninformed opinions I got from friends, and the more I saw bad VMware implementations, the more I understood why the license prohibited benchmarks.  There are some really lazy users out there and some really shady virtualization implementations – just like there’s bad SQL Server implementations.  You know who I’m talking about – the folks who install everything on the C drive and then have to keep shrinking their log files so the OS doesn’t run out of drive space.  If those folks published performance benchmarks, they’d be worse than useless.

In VMware’s corporate blog, they talked about why their license prohibits benchmarking in their EULA, and it does a great job of laying out the problems involved with letting just anybody publish reviews.

But I’m Special!  I Should Be Allowed To Publish Benchmarks!

Some users take the time to really learn the technology, dive deeply into performance bottlenecks, and set up a specialized test rig to benchmark stuff.  I used to do this with SAN gear, and I know how much work is involved.  All it takes is one math error, one configuration problem, or one unseen hardware issue, and the results are completely garbage.

Perfect example – I’ve got SQLIO testing code up at SQLServerPedia to help people test their storage subsystems.  I had a reader tell me that when he used that code on the latest solid-state gear, he got so many IOPs that my text parsing code broke and chopped off the leading numbers, thereby giving him falsely low results.  It’s an awesome problem to have, but the only reason I found out about it was because somebody else took the time to dive into my code.  It’s not as if I tossed that code into production without testing, either – I’d tested it against dozens of SANs and hundreds of drive arrays without an issue.

Even if you’re completely qualified, someone else has to double-check your work.  The vendor has to:

• Set up an application process
• Vet the candidates
• Babysit the ones who don’t get approved
• Work with the ones who DO get approved
• Double-check their results with internal test gear

The Vendor Has to Reproduce the Results

No matter whether our software came in worst or first, we would want to reproduce it internally and then figure out how to get better.  It’s just like carmakers: you’d better believe that before each new car goes to the EPA for testing, the automaker already knows exactly what mileage to expect.  Before each car is sent to a car magazine, the maker already knows the 0-60mph times.  They know all of the possible tests that each reviewer will run, and they’ve already run ‘em.

Software and hardware testing is just as challenging.  Comparing SQL Server backup software performance on a text-filled database hosted on a single-CPU box with 1gb of memory and SATA drives is completely different than testing a binary-filled database on a HP SuperDome with 128gb of memory and an EMC SAN.  Before someone publishes a set of competitive benchmarks, we need to know that they’re not performing some obscure edge use case and saying it’s representative of the software as a whole.

This puts me in an odd position.  As a performance freak, I want to know exactly how our stuff performs – especially how it stacks up against the competition.  But that’s where it gets really crazy, and I’ll talk about that in my next post.

Hello all, we get questions from time to time asking how alarms in Spotlight on SQL Server Enterprise can be configured to match a specific customers need.  Questions such as:

1) As a database administrator, all of my critical data files are on my F and G drives.  I don’t want to monitor those drives with the same thresholds and alerts as I do my C drive.

2) I have different backup policies for different databases on my servers and I don’t want to alarm on every database the same way.

All of these use cases can be met with Spotlight on SQL Server Enterprise’s alarm configurations.  Please watch the video below to see how its done.  If anyone has any questions or comments, or you want to be walked through a use case of your own, leave a comment or post in the forums!

I live and work in San Francisco and this week, like every year around this time, the city hosts Oracle Open World. Round 35,000 people have turned up, which, while down on last year by a good margin, is still an extremely impressive number of people.

Quest is there, debuting Toad 10, which along with being its version number is also its age now. Common toads can live for around 40 years, so Toad for Oracle has some ways to go yet. We’ve also got exciting new versions of Foglight for Oracle and Spotlight on Oracle to show, as well as book signings and talks by our Oracle experts, Steven Feuerstein, Bert Scalzo and Guy Harrison.

Next month it’s off to PASS in Seattle. I’ll be blogging more about specifics next week, but if you can possibly convince your boss to let you go, it’ll be worth it. The education and networking opportunities alone make it beneficial to anyone working with SQL Server. Online education and networking are fantastic nowadays, but nothing beats the face to face contact and interaction of a trade show.

Spotlight on SQL Server Enterprise has a great feature titled “Custom Counters” that allows you to incorporate your own custom data collections into the Spotlight Enterprise framework for data collection, diagnostics, and reporting.  This post will serve as a getting started guide to assist in adding custom counters to your Spotlight deployment.

First off, a disclaimer.  Due to the “open” nature of custom counters, Quest Software makes no guarantees as to the overhead or accuracy of any custom counter.  Custom Counters should be fully tested and validated in a customer’s environment prior to loading onto a production system.

For documentation on the counter used in this screencast as well as other possible counters, head to the Articles -> Documentation -> Spotlight on SQL Server Enterprise section of this site, or click on the link below:

http://sqlserver.quest.com/kbcategory.jspa?categoryID=372

If you have any questions, please post them to the forum.

Honestly, quite a lot. When was the last time you looked through http://sqlserver.quest.com, http://sqlserverpedia.com and http://www.quest.com/sql-server/performance-management.aspx?  

Brent has posted a series on Quest’s SQL Server product management and marketing team, and I wanted to follow that up with some information on how I’m working to ensure that our SQL Server performance management products are evolving to meet your needs. By way of a re-introduction, my name is Ari Weil.  I am currently managing our suite of SQL Server performance products.  My background is pretty varied; I’ve worked as a project manager, database developer and development DBA for a web-based startup company, as a production DBA and WebSphere administrator for an international warranty services organization, as a domain expert in Quest’s Israel development lab, and finally as a Solutions Architect for Quest’s SQL Server business before assuming the role of product manager.  So since I started working with technology, I’ve gone from being in a position where I wished I had some decent performance management tools, to trying to ensure the Quest’s performance management tools are the best available.

What does that have to do with you?  Well, our tools are only as good as you perceive them to be, so we’ve been gathering (and of course will continue to gather) your feedback on what you like, what you don’t, and what you want.  Sure, we’re always studying, always following the latest releases and advances in SQL Server technology to understand where the market is headed and how companies are implementing SQL Server. But we know that the real insight comes from interacting with you, our customers.  Oh, by the way, it bears mentioning that Quest are our own customers as well; our own internal IS department uses Quest products to manage, monitor, and diagnose our own systems so you can be sure that feedback on how we’re doing is never far away! 

We have some very exciting developments coming for the rest of this year, and throughout 2010.  If you’ve ever spoken with me, or with your sales or support representative about our performance management products, I’m confident you won’t be disppointed by the enhancements you’ll see in the near future!  If you have anything to tell us, please email pow@quest.com.

In today’s installment of Meet the Questies, we’ve got Christian Hasker.  You have to be nice to him, because he’s my boss.

Christian Hasker, King of the Burgers

1. Pretend you’ve never met me. Introduce yourself with your name, your location, and where you’d rather be right now.

Good day sir – my name is Christian Hasker. I live in San Francisco, although I was born and bred in England. Hmmmm…beach or mountains, tough call, but would probably be beach…Thailand…yes, beach in Thailand.

2. What do you do at Quest, other than answer emails from me?

Answer emails from other people with a phone permanently glued to my ear; the sad thing is I am not in the slightest bit joking. I am responsible for product management and marketing of our SQL Server portfolio and it sure keeps me on my toes.

3. How long have you been with Quest, and has your job changed along the way?

Andy Grant and I go way back to Leccotech; we were grandfathered in (wow that makes us sound old), so on paper I have been with Quest for 9 years. I could list out all my jobs that I have had, but we might run out of paper. Generally I have been in sales, marketing, and product management.

4. If you could have any other profession, what would you do?

I had opportunities to become a professional musician, and am glad that I didn’t take that path, because I think it’s pretty brutal on your personal life, but parts of it would have been fun.

5. Mac or PC?

“PAC”

6. At the end of yet another grueling 80-hour workweek at Quest, how do you unwind?

Family time is so important to me. There’s nowhere I would rather be than with Tiffany, Ruby and Lily.

7. When you belly up to the bar at the PASS Summit, what will you order?

“lime and soda”

8. What music are you listening to right now?

I am on hiatus from listening to music, so I am tuning into the sounds around me.

9. What’s your favorite Quest product, and why?

SQLServerPedia, because it’s so different from anything else we have; anyone can be part of it, and anyone involved in any way with SQL Server can benefit from it.

10. Do you ever get tired of hearing Brent talk about blogs and Twitter?

Extremely, but I would never tell him that, so shhhhh…