SoSSE is not designed to be a full blown VMWare monitoring package.  Quest has one of those (it’s called vFoglight) if the you’re interested.  SoSSE is hoping to provide you with enough information about your virtual environment so that you can rule in, or out, VMWare as the cause of your performance issue.  By showing a you the amount of resource utilization that ESX is “stealing” from your database server, you can immediately know whether or not your performance issue is caused by your SQL Server instance, or is caused by VMWare taking your resources.  Spotlight on SQL Server Enterprise can now provide full visibility into the entire OS stack that the database instance is running on. SQL Server -> Windows OS -> VMWare ESX.

This is exactly what DBA’s have been needing since VM’s became prevalent, the ability to know whether a problem is theirs to troubleshoot, or whether they are wasting their time looking into an issue that is caused by some other machine.  Obviously with any metric showing overhead, you want the value to be as low as possible.  SoSSE can raise alarms, if you’d like, whenever this metric exceeds a user definable percentage.

In addition to this single homepage metric, SoSSE also now has two additional drilldowns to provide more detailed information.  The Virtual CPU drilldown shows a historical graph showing  CPU % (idle and used) as well as the CPU that has been stolen by ESX% (in red).  In addition to CPU %, this drill down also shows the MHz of CPU that a VM has consumed.  We discussed in the last post how critical it is to analyze these two metrics  together!

If you determine that VMWare is taking resources from your VM, you can also view a list of VM’s managed by ESX so that you can view which VM is taking your resources.  You can sort this list by host to view all VM’s you are sharing an ESX server with, and then see how much (in GHZ) each VM is consuming.

We hope you find this feature useful and, as always, would love to hear your feedback!

Historically, when we look at a physical machine, the simple metric of CPU % Used is a pretty good measure of how busy a SQL Server is.  A server that is at one time showing 20% CPU utilization and then at another time is showing 80% CPU utilization is generating 4X as much work during the later time period.  In an operating system that is being hosted by a VM, we lose the luxury of knowing exactly how much CPU horsepower we have at any given time.  The are a few factors that attribute to this “grey area”:

1. A VMWare admin has the ability to set an upper limit on the amount of CPU that is available to your VM.  They can also set a reservation to guarantee an amount of CPU to your VM.
   
   I will not see this limit in My Computer -> Properties nor will I see any representation of this limit in task manager or perfmon.  Let’s assume that throughout the morning I have 2GHz available to my VM and am showing 20% CPU utilization.  Later in the afternoon, my VMWare admin needs to free up resources so they put a 1 GHz cap on my available CPU.  Now the exact same workload will show 40% CPU utilization.  Nothing has changed on the OS or in my SQL Server workload, yet I am showing twice the CPU %.  See where this gets confusing?
2. Even if the VMWare admin hasn’t set any resource cap on your VM’s available CPU, the ESX host could simply become overloaded.  Let’s say an ESX host has 8GHz of total processing power, and that host has 5 VM’s running on it.  Normally each VM uses about 1GHz of processing power, but all of a sudden, each VM needs 2GHz.  Like fitting 10 pounds of feathers into a 5 pound bag, something has to give.  What gives, is that ESX has to dynamically scale down each VM’s available CPU to account for the increased workload.  As a result, you may see 80% CPU utilization when looking at perfmon or task manager, but you have no idea what that 100% is of.
3. A virtual machine may not be tied to a single ESX host.  For DR or performance reasons, a VMWare administrator can move your VM from ESX host to ESX host without you knowing.  These ESX hosts also need not offer the same performance as one another.  You could be chugging along just fine during the morning with 4 GHz of processing power, and then in the afternoon be switched to an ESX host with 3 GHz of processing power.  Not only did you not know that this occured, but your VM’s CPU % will go up, even though the workload is unchanged.

Because of this, it is absolutely critical that you not simply look at CPU % as a measure of how busy SQL Server is or how much CPU it is using.  Percentages are always relative to a ceiling, and when that ceiling can move up or down at will (or whenever a VMWare admin decides that your ceiling is too high), the percentage itself loses meaning.  CPU% analyzed in conjunction with GHz used will allow you to paint the full picture of a VM’s CPU requirements.  Unfortunately, this data is not available by looking purely at the OS.  You will need metrics from the virtual layer as well.  That data is readily available in the built in VMWare client tools (vSphere), but you’ll either need to have access to the ESX or vCenter instance to view them, or a tight relationship with the VMWare admin who can send them to you.

For a better way to have this information at your fingertips, stay tuned…

I wanted to show you some of what will be coming in the 7.0 release before it’s available.  First up, replication support.  In keeping with the ease of use that SoSSE brings to other aspects of SQL Server management, just identify your distribution server and SoSSE will discover your publishers and subscribers, show you the health of individual replication jobs and the overall replication scheme, and allow you to configure alarms and automated responses to them.  Add to that other key features in SoSSE like playback for up to two weeks of data (presented in the same interface and not a feature-limited report view), long-term reporting for trending and analysis, and the ability to manage your entire environment via alarm logs, timelines, or a graphical enterprise view and you can rest assured that SoSSE will have the data you need, when you need it.

Replication Home Page

Replication Drill-down

• Fully remote collection – that’s correct, no more work database or stored procedures, and because we’re proponents of eating our own proverbial dog food, we have tuned the queries to be even more efficient than the stored procedures that preceeded them.  This means that Spotlight on SQL Server Enterprise has a zero-install footprint in your monitored environments and a lower collection overhead overall.
• Comprehensive Replication Monitoring – if you had the opportunity to participate in our Spotlight on Replication beta program, thank you for your time and your feedback! We have implemented the full complement of replication data and implemented it via a new connection type in Spotlight on SQL Server Enterprise. Just connect Spotlight to your distributor, and we’ll show you your entire replication environment with comprehensive visualizations, drill-downs, alerts, and emails and automation.
• Support for Database Mirroring – SQL Server high availability management isn’t complete without support for Database Mirroring, so we’ve created a new High Availability drill-down and included mirroring visualization. Whatever your mirroring configuration and however many databases you’re mirroring in your environment, we have payed Spotlight’s architecture visualization paradign forward to help you identify the health of your mirroring configuration at a glance.
• Support for SQL Server Running on VMWare ESX – Are you virtualizing your SQL Servers and wondering whether you’re seeing the 30% overhead the experts warn you to plan for?  Our integrated virtualization overhead metrics will help you determine if that’s the case. Do you worry that another image might be causing issues with your SQL Server or the applications that rely on it?  Well now with version 7.0 all you have to do is register your ESX server and we can show you the guests running on that host, and the resource consumption for each.  Couple that with our Playback and Reporting and Trending capabilities and you have the peace of mind you’ve been seeking to ensure the health of your virtualized SQL Servers.
• Support for SQL Azure – yes, in the same product and in the same release (it’s been a long winter).  Just register your SQL Azure database with the address and login information, and then specify whether yours is Web, Business, or Enterprise edition and you’ll be monitoring the health of your SQL Azure database alongside your other SQL Server or SSAS instances.  One thing of note: Microsoft has a bug in the DMVs that they should fix any time now where it’s only possible to see one active request at a time.  Once this is fixed in SQL Azure, Spotlight will show you all of the requests.
• Other product enhancements – For those who have already used Spotlight on SQL Server Enterprise’s previous versions, we’ve taken your feedback and made the changes you’ve been asking for, including:
  • Enterprise View enhancements: Longer server labels (so you can see your fully-qualified or otherwise many character server names), layout customization persistence, group name visibility.
  • Disk Queue Length has been replaced by Max I/O Wait: Unfortunately there is still a lot of advice out there pointing folks to disk queue length to identify I/O bottlenecks.  I won’t say there’s no value in that approach, but the advice is meant to be applied in conjunction with other metrics and investigations.  To avoid confusion and to improve the data we’re presenting we’ve replaced Disk Queue with Max I/O Wait.  Now, SAN, RAID, or otherwise, the I/O information on the instance home page will indicate whether you have an I/O bottleneck or not…and then you can use the:
  • New I/O Diagnostic Drill-down: We added the CPU diagnostic drill-down in version 6.0, and now in 7.0 we’ve added a simliar dashboard to identify I/O bottlenecks. Spotlight has always been great at helping you to identify resource bottlenecks, but these diagnostic drill-downs go a step further by pre-analyzing data for you to show you areas we’ve already analyzed, a confidence factor relative to the information presented, and data in a single location to identify what you should address to resolve the issue.
  • Playback up to 14 days and do so more simply than before: you wanted the ability to playback over long weekends and holidays, so unless you’re taking a sabbatical we should have you covered. Also, starting with version 7.0, when you navigate through Playback data you’ll see that we only list the times when alarms were generated to make it simpler to find the performance issues you’re looking for. If you want to navigate to a specific period of time, just enter the date and we’ll take you there. Simple!
  • Faster and lighter than ever before: this is most noticeable in Spotlight Today and in the Alarms by Time and Alarm Log views when we’re processing all of the alarms in your environment. From front to back though, version 7.0 is the fastest, most efficient Spotlight on SQL Server to date which translates into a better experience and more time savings for you.

We are very excited about this upcoming release and look forward to hearing your feedback! If you have any questions or concerns, please don’t hesitate to contact Quest via your account manager, our world class Support organization, or you can email me directly at ariel.weil@quest.com and I’ll be happy to help.

We recently closed an important case with Microsoft support that resolves an issue where the WMI classes required to monitor default SSAS instances were missing on the host server.  What this means to you is if you want to monitor the SQL Server relational engine, Analysis Services, or just the Windows server hosting the solution,  you really should check out Spotlight on SQL Server Enterprise.  We’ve got you covered from Windows and SQL Server 2000 to the latest releases of 2008.

Following the initial rollout of this program, we will be implementing equivalent offerings for Spotlight on SQL Server Enterprise and Foglight for SQL Server.  Stay tuned for updates!