In mid-2008, I left my job as a SQL Server database administrator and came to work for Quest Software.  I’m coming up on my two year anniversary, and I wanted to blog about the experience while I’m still junior enough to remember all too clearly what it was like BQ – Before Quest.

We Can’t Build Everything

When I first came to Quest, I poured my heart out in long Word docs about all kinds of features we should add to our products.  Developers got excited, and people scribbled frantic notes while I talked.

As things didn’t get built, I got frustrated, but I came to understand pretty quickly that Quest didn’t hire me because they were fresh out of smart people with good ideas.  There’s all kinds of brilliant people here who can recite long lists of features we should add, programs we should create, or services we should offer.  The problem is that all these things cost time, resources, and money.

If we wanted to build out every single good idea, we’d be broke in no time.  Features cost money, and customers aren’t always willing to pay for them.  Some customers want us to keep the price as low as possible, just keep the lights on without adding any additional features whatsoever.  Some would even gladly give up our support just to get a lower price.  Others want us to build in obscure features that hardly anyone would use other than them.

It sucks to stand in front of a user and say, “I know you want this feature, and I know why you want it, but can you find 1,000 other users who are all willing to chip in $100 each so we can build it?”  Software development – especially for enterprise-ready software – costs real money:

• Developers have to code it (because you want good code)
• Testers have to bang on it (because you don’t want it to break)
• Authors have to write the documentation (because not all users are as smart as you are)
• Marketers have to get the word out about it (because nothing sells itself)
• Salespeople have to handle the negotiations and contracts (because enterprise-level software isn’t sold at Staples)
• Support has to answer the phone (because sometimes users can’t figure things out, sometimes we coded it wrong, or something else is broken in the environment)
• Managers have to coordinate all this (because somebody has to approve our raises)

I’ve met users who can’t understand why we don’t add 1,000 features to every new version of LiteSpeed, yet at the same time, they grumble about having to pay any maintenance fees.  I get a chuckle out of these interactions, and it helps even more to have someone to laugh with.

We Hang Out with Other Vendors

Before I came to Quest, I thought we’d have an adversarial relationship with staff from other vendors.  I thought if I saw Red Gate’s Brad McGehee walking to a conference center, I was supposed to mow him down with my rental car, and then pin the blame on Adam Machanic of SQLBlog.com.

Now I know that nothing could be further from the truth.  I get excited about talking to people like Brad and Adam because they’re like my coworkers.  There’s not a lot of people on this planet who know what it’s like to work for a SQL Server software vendor or to run a blog syndication site. I love sharing stories with them because we can all laugh at the same users challenges.

These guys have the same pains I have, and none of us can be successful without the other.  If every other SQL Server vendor and blog site folded up tomorrow, do you think Quest would have a need for guys like me?  Lemme ask it another way – do you know any evangelists who work for the electric company?  Of course not, because they have a near monopoly.  That doesn’t mean we share secrets or plans, though, because sooner or later, we all want to have an edge that customers are willing to pay for.

We Have to See the Future

When I was a DBA, I didn’t have a lot of time to download the latest beta builds of SQL Server as soon as they came out, let alone dive deeply into them.  I would grab them every now and then, see if anything looked different, and maybe try restoring my production databases into the new version just for yuks.  I read up on new features and deprecated ones so that I could make better planning decisions, but that was about it.

As a vendor, I have to go way, way deeper.  I have to understand what’s coming as fast as possible, because I have to:

• Understand how the changes affect our products
• Read between the lines to guess how people will use the new features
• Foresee weaknesses in the new features
• Predict how we should change our tools to ease user pain points in a version that nobody’s even using yet

For example, when SQL Server 2008 Enterprise Edition introduced data compression, I recognized that few people would adopt it right away.  The setup process was awkward, SSMS didn’t surface it well in the user interface, and there would be a lot of confusion about the CPU impact.  The question became, “Will this be a big enough pain point for users that they’d shell out money to make it easier?”

Working for a vendor means knowing not just what’s coming down the road for SQL Server, but knowing users well enough to understand how it will impact their jobs.  This is probably my favorite thing about working for a systems management company – staying on the bleeding edge, and bringing users along with me for the ride.

So if you’ll excuse me, I’m off to go play with NoSQL….

I love presenting at user group meetings.  I can’t get enough of it.  If I could do it every day, I would.  I’ve got dozens of presentations I can use, and I’m continuously refining them based on attendee feedback.  I love helping people get more out of their servers and advance their careers.

There’s a few things that limit how often I can present, though.

Travel requires a lot of downtime. When I have to fly to another city, that means at least 4 hours of downtime to get there, and 4 hours to get back.  (Pack, subway, airport security, boarding, flight, taxi, etc.)  That’s a day of downtime, plus the time I spend at the user group, and during those travel windows, I can’t usually be as productive as I would be back in my home office.  If I have to take multiple flights because I can’t get there in one cheap flight from Chicago, my downtime skyrockets.

Travel is expensive. User group meetings are usually in the evenings, so I can’t fly straight back home afterwards.  That means I have to get dinner, a hotel, and breakfast the next morning.  Presenting at a user group costs around $300-$400, and that’s before I get other Quest people involved.  See, if I’m going to take the time and money to present, then Quest probably wants to have a salesperson in the room to handle questions afterwards, and the salesperson costs money too.  Hopefully we’ll make a sale in order to pay for all that expense, but it’s a gamble, and that gamble has to be chosen by the sales staff.  They decide where they want to invest their travel budgets.

Multi-city travel is risky and it sucks. I can mitigate the downtime & expense of travel by stringing together several cities in a row, but that is daaaangerous.  Read through my post on The Glamour of International Travel to see just how wrong it can go.  In one trip, I lost a bag, broke a laptop, and ran out of clean clothes.  All it takes is one delay or goof, and suddenly the whole trip is in jeopardy.  I love you guys, but I abhor going on these multi-hop trips.

Doghouses are uncomfortable. When I was young and single, working for a hotel company, I lived on the road full time.  Those days are over.  I’ve committed to Erika that I won’t average more than one week per month of travel.  For reasons I cannot comprehend, she seems to enjoy my company.  She can’t take off and travel along with me because she’s got a job.  When she takes vacations, we want to go somewhere that I don’t have to work at all.

I could avoid the travel hassles altogether if I could present remotely, but…

User groups don’t like remote presenters. The presentation experience is better when there’s an actual person in the room rather than just a talking head on a projector.  I totally understand where they’re coming from.  User group leaders have been spoiled for the last decade or so when vendors were throwing tons of money at travel, but as companies slowed down their software spending, that makes it harder to justify travel expenses.

User groups don’t have a lot of attendees. Some user groups can bring in 100 people, and that’s awesome, but most of the time it’s closer to 10-30 people.  Regional events like SQLSaturdays and SQLBits can bring in hundreds of people at once.  The more attendees I can reach, the easier it is to justify the travel money.

Help Me Help You

If you want to get out-of-town speakers at your local user group, there’s a few steps you can take to increase your odds of success.

Work with other area user groups. Coordinate schedules with .NET and SharePoint user groups within driving distance.  If you can work together to set up several meetings in a row, one night after another, all within driving distance, there’s a much higher likelihood that we can make it happen.  That opens up the possibility of me and a salesperson doing a one-week road trip and covering several groups in a row.

Schedule a regional event with vendor sessions. Set up a SQLSaturday and get the vendor to sponsor it.  Let the vendor have a lunch session – a meeting room where they can bring in a speaker to talk for an hour about their product.  Some vendors employ people like me and Kevin Kline who can talk about not just SQL Server, but about their products too.

Document your attendee history. Take pictures of your monthly meetings and show butts in seats.  Salespeople get excited when they see pictures of lively attendees.

Work with the vendor’s available speakers. When Kevin and I can’t make it to a particular user group meeting, Quest usually suggests other Quest employees.  We’re not the only ones who work with SQL Server – in a company of 3,500, we have some really SQL-savvy people who do great presentations.  User groups often turn up their nose at these guys because they’re not “famous” – but they’re not famous because they have real jobs.  If you compromise and let some of the other folks speak, then those guys may bring a good report back to the mother ship.  I’ve heard Quest people say, “The ___ user group wants a speaker.  I spoke there a few months ago and they had really good turnout – we should send Kevin or Brent this time.”  It really does work.

Work with the vendor’s salespeople. Find out who your regional salesperson is, and find out what customers they’re working with.  Salespeople want to meet more customers, and if you run a local user group, you have something the salespeople want.  If I was a user group leader, I’d talk to the salesperson and say, “So, at the last user group meeting, we talked about how to do performance monitoring, and I noticed there were a couple of DBAs from a major financial firm that were asking a lot of questions.  I don’t feel comfortable telling you exactly who they were, but maybe you should do a presentation here about monitoring.  Is there anything I can do to help make that happen?”

I wish getting in-person speakers was easier, but in today’s economy, you’ve gotta give a little to get a little.  I want to cover as many user groups and SQLSaturday events as I can in 2010, but to do it, I’ve gotta minimize my travel and maximize the return for Quest.  Being educated about that will improve your odds of getting an in-person speaker.

Of course you do. You’re a geek, and geeks love toys.

Ceci n'est pas une Kindle

As a vendor, it can be hard for us to get your attention.  If we want to deliver a message to you about our company and our products, we have to pay you back somehow.  We have a few choices:

• We can attach our message to something – like buying ads alongside content that you’re already reading like SQL Server Magazine.  This used to be the biggest way vendors got a message out.
• We can make the message itself useful to you somehow – like our wiki and blogs at SQLServerPedia.  You probably read those, and hopefully they make you associate Quest Software with quality and SQL Server expertise.
• We can pay you for your attention – by baiting you with stuff like shirts, posters, and Kindles.  If you’ve ever sat through a vendor presentation for the chance to win something, the vendor was paying you for your attention.

Let’s focus on that last one.  How much does a vendor have to give you to get your complete attention?  In the case of executives like CIOs, it’s a heck of a lot.  Vendors can’t just send a t-shirt to a CIO and get them to sit through a half hour demo.  CIOs make enough money that t-shirts aren’t a big motivator.

For that matter, DBAs even make enough money that t-shirts are less of a motivator these days.  In order to make shirts a successful giveaway, we’re constantly working on different shirts.  SQLServerPedia’s shirts for 2009 included Twitter quotes on the back and promoted Twitter almost as much as they did SQLServerPedia, but it paid off.  Twitter is seen as something hip and cool and new, and people clamored for these t-shirts.  That was a successful giveaway.

But back to our CIOs.  If we want to get their attention for half an hour – heck, even for half a minute – how do we do it?  Well, last year we tried a Kindle promotion.  We sent life-sized Kindle mockups made of plastic to CIOs in packages that looked like real Kindles.  When you get one of these in the mail, it’s pretty exciting – it feels like you’re holding a real Kindle.  The message on the “screen” said:

“How do you do more with less in IT?  Let us give you our read on it. Get a free Kindle2 when you meet with a Quest Expert.”

It worked great!  The placebo Kindle captured the attention of CIOs long enough to get them to read it, and the promise of a real Kindle got them to call us.

We can’t afford to do that kind of promotion for everybody (sorry, DBAs) but we CAN do drawings for Kindles.  Whenever we get a dozen or more people together, we can either give everybody something like a shirt or a USB key, or we can do a drawing for a single high-value prize like a Kindle.  On my own blog, I asked what DBAs wanted to win, and we’ve taken that feedback into account on our future giveaways.

We’re getting ready to do a round of testing of some of our software in combination with popular filter drivers, and I want your help.  If there’s any filter drivers or file drivers you run on your SQL Servers, we want to focus extra testing time on those.

You can help by filling out a one-question survey.  Thanks for your time!

Say 'hello' to my little friend!

We’re very excited to introduce a new addition to LiteSpeed brand – we’ve released LiteSpeed Engine for SQL Server, which is now available with LiteSpeed for SQL Server Enterprise.  Let’s nail some questions that you may be having:

• What is LiteSpeed Engine for SQL Server?  It’s a very light weight, drop and go compression and encryption ‘engine’ for SQL Server.

• Why is that important?  By offering LiteSpeed Engine for SQL Server, customers who already have a backup and recovery strategy in place using their own scripts and processes can now get the compression and encryption that is offered by LiteSpeed – without changing their current backup processes.

• What’s the difference between LiteSpeed Engine and LiteSpeed Enterprise?  LiteSpeed for SQL Server Enterprise offers the full monty – a very comprehensive backup and recovery solution that provides not just compression and encryption, but also Maintenance Plan management, Object level recovery, Fast Compression, etc.  Now, LiteSpeed Engine for SQL Server, as stated in the previous bullet, simply offers the compression and encryption capabilities of LiteSpeed which fits into an already established backup and recovery process.

• Can you give a brief usecase for this?  LiteSpeed Engine provides much more flexibility and choice to you as to how to deploy LiteSpeed.  For example, on your ‘mission/business critical’ systems, the full implementation of a backup and recovery strategy offered by LiteSpeed Enterprise will be very valuable.  On those less critical systems, or pre-production environments, or those databases that already have a backup and recovery plan in place, LiteSpeed Engine will be the missing piece that gives compression and encryption.

• How do I get LiteSpeed Engine for SQL Server?  Currently, LiteSpeed Engine is available to evaluate by downloading from our LiteSpeed for SQL Server page on Quest.com.  Take a look – we’d love your feedback.

• Is there any other resource where we can see how LiteSpeed Engine for SQL Server will work?  Yes!  We have a brief video that provides an overview of LiteSpeed Engine for SQL Server and how to configure.

We’re really excited about our introduction of LiteSpeed Engine for SQL Server and are confident that the flexibility now available in how you decide LiteSpeed fits into your backup and recovery strategy will blow you away.

Selling products that interact with SQL Server and Windows makes troubleshooting tougher: when people call in with problems, sometimes they’re not really related to our products.  Sometimes – heck, even often – the problem boils down to a Microsoft, hardware, or configuration issue totally unrelated to Quest products.

When our support team can’t get the right answer for the customer, they kick it up to the escalations team.  Escalations works with developers, product managers, and subject matter experts like me to drill deeper into issues.  This is when things get to be a little more challenging, because we have to ask questions about the customer’s environment – and sometimes we turn up answers that nobody wants to hear.

For example, I’m currently working with a customer who’s having trouble with their backups taking too long.  In the course of investigating the issue, we found that they were never purging their backup history in MSDB – ever.  The server had years and years of backup history, and as each backup job finished, SQL Server took longer and longer to update the history.  I’ve tried suggesting that native backups will also face that same problem, but the customer can’t try native backups – because they take even longer!  Ouch.

Awww, How....Cute?

I’ve blogged about this backup bottleneck before, and SQL Server MVP Geoff Hiten has blogged about making MSDB backup histories faster, but I can’t seem to find a definitive article from Microsoft explaining the importance of this issue.  If it’s not in Books Online, some folks just don’t believe it – especially when it’s coming from a third party vendor like us.  Telling someone they have problems in their SQL Server environment is like calling their baby ugly; as far as they’re concerned, it’s all just a matter of opinion.

Every year, Quest puts on a one-day virtual conference called QuestConnect.  Quest staffers and community members give seminars about all the different technologies Quest is involved in – SQL Server, other database platforms, Active Directory, Exchange, virtualization, security, identity, all kinds of stuff.

We encourage attendees to fill out a survey so that we can improve the event each year.  Here’s some highlights from some of the hundreds of responses:

We want more!  Add more days and more tracks. I hear ya – we could offer free training 24/7 around the clock, and it still wouldn’t be enough.  I liken it to my TV cable company: I’ve got hundreds of channels, but I still want more.  We want to offer as much free training as we can while still keeping it free.  At some point, adding and broadcasting live content starts to cost money.

Couldn’t figure out how to navigate the system. This is one of those pieces of feedback that make me say, “Ah, cool, it wasn’t just me.”  We’re constantly trying to improve our game, and we had a hunch that this was a weak spot.  I’m really thankful to hear this feedback.

Less sales pitches and more actual implementations. Agreed – I hate infomercials.  In the SQL Server group, all of our presentations are focused on helping you work with the native tools first, and our tools second.  Not all of the groups feel that way, though, and that’s why some of the presentations can be a little light on value.

Add a high-level track for managers. I love this idea.

Add virtual booth babes. I’ll get right on that.  (It did make me laugh though.)

Allow the on-demand webinars to be downloaded. This is a tough one as a vendor because it means the content can be given around to friends more easily.  Part of the payback for the free training is giving us your name and email address.  If you download the training, then give it out to a bunch of people, it’s harder for us to justify doing the free training as a marketing method.  I feel bad saying that, because I’m kinda like a record company, but that’s the way it is.  Instead, let’s go back to the root problem: why do you need the webinars on your local hard drive?  If the objective is to be able to watch ‘em whenever you want, that’s what SQLServerPedia’s podcasts and Quest’s webcast archives are for.

More demos and virtual hands-on labs for the attendees. We’ve got a great solution for that coming.

Add more vendors. Wow, that surprised me.  I hadn’t thought of that, but we could pitch the service to other vendors too.

Bring in more world class speakers like Brent Ozar. Okay, now you’re just pulling my leg.

Every quarter we have ‘QBRs’, which stands for Quarterly Business Reviews. I’ve been busy reviewing our SQL Server business, and planning for 2010, and on October 29th I get to journey into the lion’s den and present that plan.
The audience for the QBR is high-level executives from every area of the company – development, support, legal, sales, marketing, CEO, President – and it will be my first time presenting the whole business to them. People have made or broken their careers during these meetings, and some of the stories I have heard range from the truly painful, Gong Show-esque moments to people knocking it out of the park and cementing their place. I’d be lying if I said I wasn’t a tad nervous, but I am looking forward to the opportunity.
I am a big believer of advancing your career by getting out of your comfort zone. I’ll let you know how that goes!

If you’re like me, you probably don’t bother reading through all the fine print before accepting software licenses.  When I’m awake at 3 AM and I need to fall asleep, I’ll install a new piece of software and read the End User Licensing Agreement (EULA) to put myself out.  Works every time.

Rock a bye baby, licensed as follows...

Buried in the fine print in arcane legalese, however, is some stuff that really matters to you: you probably can’t publish performance benchmarks or competitive reviews of software without getting approval from the vendor.

For example, here’s a quote from VMware Server’s EULA:

You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at benchmark@vmware.com to request such review.

When I first read about this, back before I came to work for Quest, I hit the roof. How dare they stop us, the community, from testing their software and writing about it?  People need to know about the exact performance overheads of virtualization before they implement it.  We need to work together to share this information, right?

The more underground reviews I read, the more uninformed opinions I got from friends, and the more I saw bad VMware implementations, the more I understood why the license prohibited benchmarks.  There are some really lazy users out there and some really shady virtualization implementations – just like there’s bad SQL Server implementations.  You know who I’m talking about – the folks who install everything on the C drive and then have to keep shrinking their log files so the OS doesn’t run out of drive space.  If those folks published performance benchmarks, they’d be worse than useless.

In VMware’s corporate blog, they talked about why their license prohibits benchmarking in their EULA, and it does a great job of laying out the problems involved with letting just anybody publish reviews.

But I’m Special!  I Should Be Allowed To Publish Benchmarks!

Some users take the time to really learn the technology, dive deeply into performance bottlenecks, and set up a specialized test rig to benchmark stuff.  I used to do this with SAN gear, and I know how much work is involved.  All it takes is one math error, one configuration problem, or one unseen hardware issue, and the results are completely garbage.

Perfect example – I’ve got SQLIO testing code up at SQLServerPedia to help people test their storage subsystems.  I had a reader tell me that when he used that code on the latest solid-state gear, he got so many IOPs that my text parsing code broke and chopped off the leading numbers, thereby giving him falsely low results.  It’s an awesome problem to have, but the only reason I found out about it was because somebody else took the time to dive into my code.  It’s not as if I tossed that code into production without testing, either – I’d tested it against dozens of SANs and hundreds of drive arrays without an issue.

Even if you’re completely qualified, someone else has to double-check your work.  The vendor has to:

• Set up an application process
• Vet the candidates
• Babysit the ones who don’t get approved
• Work with the ones who DO get approved
• Double-check their results with internal test gear

The Vendor Has to Reproduce the Results

No matter whether our software came in worst or first, we would want to reproduce it internally and then figure out how to get better.  It’s just like carmakers: you’d better believe that before each new car goes to the EPA for testing, the automaker already knows exactly what mileage to expect.  Before each car is sent to a car magazine, the maker already knows the 0-60mph times.  They know all of the possible tests that each reviewer will run, and they’ve already run ‘em.

Software and hardware testing is just as challenging.  Comparing SQL Server backup software performance on a text-filled database hosted on a single-CPU box with 1gb of memory and SATA drives is completely different than testing a binary-filled database on a HP SuperDome with 128gb of memory and an EMC SAN.  Before someone publishes a set of competitive benchmarks, we need to know that they’re not performing some obscure edge use case and saying it’s representative of the software as a whole.

This puts me in an odd position.  As a performance freak, I want to know exactly how our stuff performs – especially how it stacks up against the competition.  But that’s where it gets really crazy, and I’ll talk about that in my next post.

I live and work in San Francisco and this week, like every year around this time, the city hosts Oracle Open World. Round 35,000 people have turned up, which, while down on last year by a good margin, is still an extremely impressive number of people.

Quest is there, debuting Toad 10, which along with being its version number is also its age now. Common toads can live for around 40 years, so Toad for Oracle has some ways to go yet. We’ve also got exciting new versions of Foglight for Oracle and Spotlight on Oracle to show, as well as book signings and talks by our Oracle experts, Steven Feuerstein, Bert Scalzo and Guy Harrison.

Next month it’s off to PASS in Seattle. I’ll be blogging more about specifics next week, but if you can possibly convince your boss to let you go, it’ll be worth it. The education and networking opportunities alone make it beneficial to anyone working with SQL Server. Online education and networking are fantastic nowadays, but nothing beats the face to face contact and interaction of a trade show.