Become a Columnist Microsoft Exchange Site Microsoft Support SiteMSDN Exchange Site

   

Subscribe to OutlookExchange
Anderson Patricio
Ann Mc Donough
Bob Spurzem
Brian Veal
Catherine Creary
Cherry Beado
Colin Janssen
Collins Timothy Mutesaria
Drew Nicholson
Fred Volking
Glen Scales
Goran Husman
Guy Thomas
Henrik Walther
Jason Sherry
Jayme Bowers
John Young
Joyce Tang
Justin Braun
Konstantin Zheludev
Kristina Waters
Kuang Zhang
Mahmoud Magdy
Martin Tuip
Michael Dong
Michele Deo
Mitch Tulloch
Nicolas Blank
Pavel Nagaev
Ragnar Harper
Ricardo Silva
Richard Wakeman
Russ Iuliano
Santhosh Hanumanthappa
Steve Bryant
Steve Craig
Todd Walker
Tracey J. Rosenblath
 
 
The Scalability of Digital Dashboards Steve Bryant Page 1 | Page 2 | Page 3 | Page 4

ECMS Dashboard Cache Engine

Since the most pressing problem is the constant access to the SQL server, we will begin with the Dashboard Cache Engine product.

Here are the three presented scenarios:

User Dashboard Sessions

1. User logs on to the dashboard:
2. Dashboard.asp determines whether or not a cached version of their dashboard exists.
i. If the cached version does not exist:
a. SQL is queried for the information and the dashboard is built.
b. The dashboard is stored in an html cache file.
ii. If the cached version exists:
a. The location of the cached version is determined.
b. The user is redirected to the cached dashboard.

In this scenario, all refresh access to the SQL server is eliminated. The SQL server is only contacted if a change is made to the dashboard.

User Dashboard Reset

Since the cache is a small

If the user modifies the dashboard:
A. The cached version of the dashboard is cleared.
B. SQL is queried for the information and the dashboard is built.
C. The new modified version of the dashboard is cached.
D. The user is redirected to the cached dashboard.

Administrator Reset

An administrator modifies a web part or a global dashboard:

A. All cached dashboards will be cleared (forcing a rebuild when the user next logs on).

Process

The IIS server acting as Factory stores a copy of the constructed dashboard design for each client that connects to the server. This “Dashboard Web Server” has added customized Com+ components for redirection and process. The actual cache files are kept in a dynamic sub-directory of Dashboard named cache. In some situations, multiple web servers are required in order to process tens of thousands of users. In these situations, replication of the cache directory between IIS servers is necessary in order to maintain continuity in the user dashboards no matter the server that processes their requests.

The IIS server that processes the request will “brand” the cache file with its own name. The “branding” causes the client to seek the same server. The cache file will automatically redirect the user to the server that created the cache file. It is important to consider this when determining your cluster or load balancing techniques.

The Factory ASP engines have also been altered to first look for a web cache of the data. This is accomplished through the dashboard_const.inc file located in the dashboard directory on the server:

Before Modifications:

<%
Const stPortalServer = "http:// Portalsubscriptionserver "
Const stPortalVRoot = "Sqlwbcat"
Const stDashVRoot = "Sqlwbcat/Welcome/"
Const stPartCatalog = "Sqlwbcat/Parts/"
%>

In the default configuration, the Portal server is the local server. SQLWBCAT refers to the virtual directory in IIS that actually points to a SQL database on the same machine. By identifying a cache directory on the local server, we can direct the users to small localized “snapshots” of their dashboard settings. We also modify this file to point to backend SQL servers.

After Modifications:

<%
Const stPortalVRoot = "Sqlwbcat"
Const stDashVRoot = "Sqlwbcat/Welcome/"
Const stPartCatalog = "Sqlwbcat/Parts/"
Dim stPortalServer
stPortalServer = "http://Portalsubscriptionserver"
Dim ROOT_CACHE_PATH 'As String
Dim ROOT_CACHE_URL 'As String
Dim BASE_PREFIX 'As String
Dim BASE_URL 'As String
ROOT_CACHE_URL = "http://dashboardwebserver/Dashboard/Cache/"
BASE_URL = "http:// dashboardwebserver/Dashboard/dashboard.asp"
ROOT_CACHE_PATH = "C:\InetPub\wwwroot\Dashboard\Cache\"
BASE_PREFIX = "../../"
%>

As you can see from the settings above, we have specified that the PortalServer is http://Portalsubscriptionserver. If we had let this setting match the servers setting in the ROOT_CACHE_URL (server name), the server would act as a standalone running SQL and IIS. Here is the overall process in the ECMS two-tiered approach to the problem:

  1. The client loads http://dashboardwebserver/dashboard
  2. The Factory processes on the server access the store processes. The store.vbs file then references the dashboard_const.inc file to find the data.
  3. The data is not found on the server if the client has never accessed a dashboard from this server.
  4. The store passes the request to the portal server. The portal server assembles the page and passes the information back to the store processes.
  5. The updated store processes create the cache file on the hard drive of the server. In the Inetpub/dashboard/cache/user directory. Users are identified by their domain name plus their user name.
  6. The client is redirected to the cache version of the page. Subsequent refreshes hit the small 6K cache file on the server. Because of this the store does not make Web Server-Method requests, nor does it proxy SQL requests to the back end. Subsequent reloads execute the factory processes to load stored data.


Two-Tier Performance

As we mentioned earlier in the document, by creating a cache of the dashboard and separating the services each server provides, we can dramatically increase the capability of the server(s).

Refreshing the dashboard still impacts servers since all Dashboard calls request access to the data layer. Here are the refresh results in the Two-Tier scenario:

Digital Dashboard Server (Front end Tier)

This server represents the altered Digital Dashboard Server. This server has an added COM+ component as well as updated Factory ASP and Store pages. This server does not run SQL server. The primary purpose of this server is to provide IIS services to the client as the Factory and Store for the Dashboard System.

There are several client functions that can affect this server:

Client Dashboard Refresh

ECMS has modified the factory and store engines to reduce server impact during dashboard loads and refreshes. The following chart details the specific activity of the Dashboard Web Server during such refreshes and dashboard loads (from cache).

Users 1 2 5 20 100 1000 5000
Processor Load 0.001 0.001 0.002 0.008 0.04 0.04 2
Web Methods 5 10 25 50 250 5000 25000
Cache Refresh-bytes 8,192 16,384 40,960 16,3840 819,200 8,192,000 40,960,000


As you can see from the results listed above, the largest concern is the disk I/O needed to provide the cached dashboard to large numbers of users simultaneously. Considering the read capacity of SCSI III drives, a new server should be able to support many more than 5,000 simultaneous reads. Processor speed, in this case, is not an issue as the only processing required is to validate access to the cached file through security ACLs and token processing.

These loads also represent only the amount of data required to support the dashboard systems. Any and all additional load such as portal items should be tested in pilot and considered when determining system requirements.

The Scalability of Digital Dashboards Steve Bryant Page 1 | Page 2 | Page 3 | Page 4 | Next

Disclaimer: Your use of the information contained in these pages is at your sole risk. All information on these pages is provided "as is", without any warranty, whether express or implied, of its accuracy, completeness, fitness for a particular purpose, title or non-infringement, and none of the third-party products or information mentioned in the work are authored, recommended, supported or guaranteed by Stephen Bryant or Pro Exchange. OutlookExchange.Com, Stephen Bryant and Pro Exchange shall not be liable for any damages you may sustain by using this information, whether direct, indirect, special, incidental or consequential, even if it has been advised of the possibility of such damages.

Copyright Stephen Bryant 2008