Tech Tips

Crystal Xcelsius for Developers: Technology Overview and Introduction

Thomas W. Gonzalez Advanced Technology Consultant

Who should read this paper?

This paper was written for developers and programmers interested in learning more about using Crystal Xcelsius (CX) to build interactive data visualization and dashboard solutions. The purpose of this discussion is to give developers a deeper understanding of the Crystal Xcelsius technology and design environment so they can determine how to utilize CX for their own projects. We start with a brief overview of CX and then dive into some of the underlying technologies that make CX work. We also describe the development workflow and design paradigm in CX that allow for vastly increased developer productivity in creating data visualization solutions.

What is Crystal Xcelsius?

Crystal Xcelsius is a data visualization design tool that displays summary-level business data in a visually rich and intuitive format that allows users to analyze and interact with their data in order to make better business decisions. CX is first and foremost a visualization and presentation tool; it is not a back-end reporting tool, nor is it a sophisticated data modeling or integration package. While CX does allow for data manipulation and transformation, it is solely for the purpose of visualization and interactivity purposes, versus more advanced data transformationtype functionality, such as Data Integrator, also available from Business Objects. Most people use Crystal Xcelsius for its robust and award-winning visualization technology, not as a complete dashboard or scorecard solution. While CX most definitely can build amazing dashboards and scorecards, it represents only part of a complete dashboard or scorecard package. Most dashboard and scorecard technologies on the market today come with a complete infrastructure for managing key performance indicators and metrics, strategy mapping, collaboration, security and

navigation. If you need a complete dashboard solution and framework, I suggest checking out Business Objects Dashboard Manager. But if your interests lie in quickly creating compelling and interactive data visualization solutions for your business users or yourself, read on.

How does it work?

Crystal Xcelsius uses a unique combination of Microsoft Excel, as a data modeling tool, and Macromedia Flash Player, as the rendering engine, to compile dynamic visualizations via its proprietary Crystal Xcelsius Designer. Before you dismiss CX as a toy because it uses Excel for data modeling, let me emphasize the word DESIGNER in the previous sentence. Within Crystal Xcelsius, Excel is used solely as a data design tool to help integrate and manipulate data into a format that is easy to bind to and that is used for common charting and data visualization components. We will discuss the value of Excel as a data modeling tool a little later on, but first I want to show, from an architectural perspective, how all the technology pieces fit together. Due to the nature of Flash and the way Crystal Xcelsius leverages it, the technology architecture and deployment model are unique and not immediately obvious to the developer more accustomed to the typical client/server architecture of most BI and reporting platforms. The diagram below shows the relationship between Excel, the Crystal Xcelsius Designer, Flash, and the client. It is important to note the different role each component plays at design time versus run time.

At design time, Excel is used to model and format data while the CX Designer is used to visualize and bind Excel-modeled data to interactive visualizations via the Flash engine. Once a design is ready to be deployed or previewed, Crystal Xcelsius then does its magic and compiles the data model and visualization elements into Flash byte code in the form of a .swf file. This .swf file now contains a virtual representation of the data and its relationships as defined in the Excel model, as well as all the graphic assets and animations bound to this data. The .swf file also contains information for any XML data connections that were specified at design time. These data connections can be used to replace the existing data you designed within Excel with different sets of data at run time, thus allowing multiple queries into larger data stores.

An example would be a sales forecast dashboard that compares current sales data to specific forecast targets. In Excel you could model both the current data and the target data, knowing that at run time the current sales data would be replaced by XML data from your database, while the target data would remain static. When Crystal Xcelsius compiles the file into Flash, it puts all the data from Excel into the .swf file, along with the XML mappings that will allow the .swf file to fetch the dynamic data. As a result, at run time the user still sees the target forecast data, stored statically within the .swf file, while also viewing the updated current sales figures, which are being fetched dynamically through the XML connection. Once the Flash movie is running, you are no longer connected to the Excel file in any way; all data has been virtualized into the Flash byte code or is being fed by dynamic XML feeds. So you end up with an easy-to-use (Excel) and simple (row/column) metaphor for modeling and manipulating your data, without any of Excels limitations. Once you begin using this paradigm, you realize how powerful it is: data can be massaged much more quickly with Excel than it can be programmatically.

Will it scale?
With the CX deployment model you end up deploying to the end user a .swf file that contains the Flash byte code. This file can be hosted on a Web server or deployed directly to a client via an e-mail attachment, PowerPoint, .PDF or any other document format that supports embedded Flash Player controls. Regardless of your deployment model, when a client views a Crystal Xcelsius dashboard either directly, on the clients machine, or over the Web, the .swf file is always running locally within the clients Flash Player or plug-in. This means that any data connections the CX dashboard requires are executed directly by the client machine via the Flash Player and do NOT pass back through the Web server that hosts the CX .swf file (unless, of course, the data connection specifically points to that Web server). This is an important point, as this deployment paradigm does not put any extra burden on the server hosting the CX .swf file, and it also is not connected to the original Excel file that was used to create the CX model. Engineers frequently ask us, Will it scale? Once you realize that Excel is not part of the deployment model, the answer to that question sits outside CX specifically and rests more with how much bandwidth the Web server has and how many data connections the data warehouse or Web services will support. The only restriction on scalability is how fast the hosting server can stream the resulting CX .swf files, which range in size from about 200KB to 500KB. From the clients perspective, the only performance drawback is the initial download of the .swf file, which is dependent on the bandwidth of the clients connection. When it comes to the visualization itself, the Flash deployment model pushes the heavy lifting out to the client, who will be doing all the data processing necessary to dynamically render the visualizations.

Deep dive: The Model-View-Controller design pattern

If you are not familiar with the well-known Model-View-Controller (MVC) design pattern, popularized in the classic book Design Patterns: Elements of Reusable Object-Oriented Software by the Gang of Four, here is a brief explanation that will help to clarify exactly how CX works at an abstract-object level. In the MVC design pattern you have three main elements, each responsible for a specific set of duties. You have the model, which is responsible for storing all data and application states. You have one or more viewers responsible for displaying data within the model, and you have one or more controllers responsible for updating data within the model. These three componentsthe model, the viewers and the controllersall act in concert with each other to provide the user with an interactive, stateful user interface. In Crystal Xcelsius, the data relationships you built in Excel serve as the model, while certain charts, gauges, progress bars, etc., serve as the viewers. Where things really start to get interesting is that CX has controls that support user input (such as sliders, input boxes, and even charts themselves,

with drill-down capabilities) and these components then act as the controllers. The power of the MVC pattern is that there is one common model that stores application state that all viewers respond to and all controllers can update. In the Crystal Xcelsius architecture, the data model you built with Excel, which is internalized within the Flash engine, holds the application state as a set of complex variables with internal relationships that all visualizations are bound to and all user input controls can manipulate.

For instance, I might add a slider to my dashboard to provide a what if capability that allows me to see what would happen if I adjusted sales for the month of January. I might have several charts or components leveraging data in the January sales cell (C4.) I could then have a slider that reflects a range of inputs from 0%-100% that is inserted into an adjacent cell (C5.) I might now remap my visualizations to a third cell (C6), which takes the raw January sales data and multiplies it by the slider input, giving me a new value. The result is that any visualization connected to this new formula-driven cell would change anytime I moved my slider. Voil: MVC. The spreadsheet data cells represent the model, the charts are the viewer, and the slider is the controller. Once again, it is important to remember that Excel is just a metaphor; CX actually has its own internal representation of that data optimized for the Flash Player engine and is no way attached to Excel at run time.

How does this help my productivity as a developer?

When it comes to developer workflow and actually creating dashboards in Crystal Xcelsius, you will find that the environment is much more WYSIWYG and much less like traditional IDEs such as Visual Studio .Net or Eclipse for Java, which require writing some code by hand. The real magic lies in how CX leverages the Excel data modeling capabilities to easily format, summarize, and aggregate data and then bind that formatted data to graphic components in a powerful WYSIWYG design tool. This unique combination of Excel to manipulate data and the Crystal Xcelsius Designer to format the

visualizations results in some truly amazing gains in productivity. In less than a couple of hours working with the tool for the first time, you will be able to create some very impressive visualizations that might have taken you days with a more direct programming approach. Lets compare two approaches to developing a rich and interactive data visualization. In the first approach we will follow a more traditional programming workflow, and in the second we will walk through the Crystal Xcelsius workflow. For this example we will refer to a baseline use case that requires us to display a companys product sales for a rolling 12 months on a monthly basis within a Cartesian bar chart, and then display the sum of these sales figures in a gauge control. We will assume we have access to an internal Web service that provides a method named getRolling12MonthSales() that will return gross sales per month in an XML result set.

The traditional programming approach

Using a traditional programming approach, we would follow five basic steps to achieve our visualization: 1. 2. 3. 4. 5. Fetch data into an XML result set. Convert XML data into one or more internal variables/objects. Perform needed calculations for total sales summary information. Bind data to a visualization component. Compile and deploy the dashboard.

This is the relatively straightforward process that most programmers familiar with building XML-based intranet or extranet applications are accustomed to. To start, you would use some type of HTTP connection to execute your getRolling12MonthSales() Web method and retrieve your XML results. You now need to either leverage an internal XML parser or build your own to loop through the monthly data and assign both the sales number and the associated month name to some type of array or collection. You also need a variable that sums up the sales as you are looping so you can have a total sales number to populate your gauge control. If you are using a SOAP- or REST-enabled Web service, you will have schema information available that allows you to keep your value objects type-safe when you create your array or object collection. If you are using just raw XML, you will have to enforce this yourself. After you have transformed the XML into value objects within your code, you then need to populate your visualization components. We will assume you have already purchased some type of third-party Cartesian bar chart control and gauge component that supports automated binding. You would then have to instantiate these controls and set their data source to point to your array collection for the bar chart and the single value of total sales to your gauge. While this isnt a terribly difficult operation, I estimate that a fairly competent developer would still take about 20 minutes to write this code and test it. At this point the developer will need to compile and deploy this code, which, depending on the development platform, could be a project unto itself.

The Crystal Xcelsius approach

Now lets see how the same example works in CX. In this scenario we start with Excel, utilizing its automated WYSIWYG XML Mapping feature to create an XML map that points directly to our getRolling12MonthSales() Web method. Excel would either read the XML schema provided by the Web service or create one for us, which is very handy when dealing with raw XML. Once the XML map has been created, Excel gives us a tree view of the XML data that allows us to drag-and-drop the relevant XML nodes into our spreadsheet. Excel then performs another little piece of magic by automatically traversing and flattening the XML hierarchy to populate our two-dimensional row/column spreadsheet with the XML data values returned from getRolling12MonthSales(). In this case we would see a column of information with 12 months of sales data and the related month names in an adjacent column. We could then, with this raw monthly sales data, simply use another Excel cell to do a SUM

operation or any other of hundreds of mathematical functions supported by Excel. No looping or special logic is required to store variables, etc. If we need more complex calculations, such as performing net-present-value on the range of data, this becomes even more of a timesaver, as Excel intrinsically supports these more complex mathematical functions. The entire process of querying and formatting the XML data in Excel usually takes less than a few minutes to complete end to end, and best of all we didnt have to compile any code to test the results. After we have the data in Excel, we then go into Crystal Xcelsius and select this Excel file as our data model. Now CX has a reference to this Excel spreadsheet, so we can use it for our visualizations. At this point we drag a bar chart and a gauge component onto our workspace and through the chart and gauge property windows point their source data property to the appropriate cells in the Excel spreadsheet via the WYSIWYG lookup selector built into the Crystal Xcelsius Designer. Now both our chart and our gauge are populated with their respective data and we can actually see the data values in our components right inside our design workspace, with no need to compile the CX visualization to see this data. To deploy this dashboard, we simply hit the appropriate export button in Crystal Xcelsius and deploy the resulting visualization either directly, as a .swf file, or as a variety of document formats such as .PDF, .ppt or .doc (all of which support the interactive features of Xcelsius). This is another area where CX really shines; with the Flash deployment model you are virtually guaranteed that a client with a Flash Player can see your solution. If it runs on your dev machine, chances are 99.9% it will run on the clients. No troubleshooting ActiveX controls, .Net frameworks, JVM versions, etc. No big downloads, just one simple, thin .swf file, and the client is good to go.

400% gain in productivity

Steps to Bind 1 Data Result Fetch data into result set Convert data into bindable objects Perform looping calculations Bind data to data visualization Total Time Expended Programming Approach 510 minutes 510 minutes 510 minutes 15 minutes 1635 minutes Crystal Xcelsius Approach 3 minutes 0 minutes < 1 minute 12 minutes 47 minutes

Going back to our use case outlined above, lets do a little math and extrapolate how much time is saved by using the Crystal Xcelsius approach. Most dashboards will contain at least a half-dozen visualizations, each requiring unique data sets or views. In the above scenario we assumed that it would take on average 20 minutes, and probably quite a bit longer, to create one visualization programmatically while it takes about five minutes to do the same thing using Crystal Xcelsius. So just hooking up your data for six visualizations, ignoring any time spent to design the UI, you are looking at a little less than 30 minutes to build a dashboard with Crystal Xcelsius versus approximately 120 minutes with a more traditional approach. That is over a 400% increase in productivity. Obviously we are glossing over some of the thornier issues, such as making sure your data feeds are clean, having a deployment portal, setting up security, etc., but those issues exist regardless of which approach you use. We also are assuming that the developer is spending absolutely no time focused on the user interface and layout, which can be critical to a successful dashboard design. This is another area where CX provides huge gains in development productivity. You not only can use CX to directly connect to data sources that you will directly visualize, you also can use it as a prototyping tool to design your dashboards without having to be connected to a back-end data source.

Usually in the dashboard project life-cycle you are going to start with a set of requirements that have identified your metrics and KPIs. You might even be lucky enough to have some sample data that shows the type of data and range of values to expect. The data services needed to support those metrics and KPIs might not exist yet as fully developed SQL or Web service calls, but you still want to validate the design and interactive features you plan to support for your end users. In this case, you can simply put your sample data directly into Excel to start designing your dashboard against this sample data. Once the data becomes available from your back-end Web service, you can easily replace the sample data with your XML map and have live data streaming into your dashboard that you have already designed. This approach allows you to quickly go from a prototype design into a production dashboard with very little reworking. More importantly, this prototyping process allows you to fine-tune your design choices and validate the effectiveness of one visualization approach over another without having to do any back-end programming first.

Integrating multiple data sources

Remember that Crystal Xcelsius is a visualization tool first and data connectivity tool second. While Crystal Xcelsius does provide several ways to connect to external data sources, it is by no means a data integration tool. For the scope of this discussion we will focus on the Crystal Xcelsius Workgroup standard data integration features. The Crystal Vision product recently released by Business Objects has even deeper data integration capabilities that we will discuss at the end of this paper. Crystal Xcelsius relies upon an internal XML parser and the HTTP connectivity of the Flash engine to receive external data. This data can come in one of three formats: 1. XML provided via Excel 2003 XML Mapping 2. SOAP-based XML services 3. Raw XML via HTTP Additionally, CX is capable of SENDING data either via the HTTP call or through a SOAP XML call. Each method has its pros and cons for connecting to data, but probably the most flexible method is to use the Excel 2003 data mapping features, which give you an easy way to map the XML you are receiving directly into a format that you can manipulate for your presentation without struggling with XSLT, custom parsing, or other more programmatically intensive approaches. More in-depth articles on all three of these methods can be found at: www.xcelsius.com/Learning/Center.html When using Crystal Xcelsius for projects with dynamic data streaming from several sources that do not have a requirement for intensive data aggregation via an OLAP or data mart solution, the easiest method is to build a quick Web service in the language of your choice, such as Java or .Net. This gives you the full power of a robust OO language and the more powerful binary protocols such as ODBC or OLEDB to connect to a myriad of data sources. You can use this intermediate Web service to aggregate and do some level of normalization on the data before passing it along as formatted XML that you can then utilize in your visualizations. Once you connect to native databases, it is usually more efficient to write your own SQL or query statements than to try to WYSIWYG your way through complex Cartesian join statements. With this approach, you also can connect to other Web services or binary streams that serve up data. You also now have a very effective way to apply security to these data streams as well as validate the credentials of the user requesting the data.

Security
This brings us to security. CX allows you to leverage several security models including HTTP credentials. Because the CX deployment model relies upon the Flash engine, it also uses the built-in security context of Flash. If your dashboards are pointing at an XML feed hosted on a Web server, you have the ability both to encrypt that feed via SSL and also to require the user to authenticate against the feed. The challenge/response authentication is handled by the browser hosting the Flash

file, and it will maintain session state for that logon as long as the browser window is not closed. On the server side you have the ability not only to request the challenge response but also, depending on your Web server technology, to read the credentials of the user who has logged on to your system via the HTTP stream and validate those credentials against any custom security schema you may have created. Another approach to implementing security is providing a logon on the Web page that is hosting your .swf file and passing those logon credentials into the .swf file at run time via the Flash variables mechanism built into CX. Once you have these variables inside your .swf file at run time, you are free to use them in your XML calls. For those of you not familiar with Flash programming, Flash variables allow you to pass data into your .swf file at run time via JavaScript or the URL call to the .swf. This powerful mechanism communicates and affects the internal state of your dashboard via an external mechanism in the hosting html page at run time.

Aesthetics and interactivity

One of the most noticeable characteristics of a Crystal Xcelsius visualization is how good it looks. The Crystal Xcelsius Designer allows you to do pixel-perfect layouts to accomplish these stunning designs. You can select from a library of visualization components, each with its own set of different skins that allow you to give them the Apple Aqua look, or the Windows XP or Aero look. Once you have selected a component, you can place it wherever you want on the work stage and adjust its positioning down to the pixel. By leveraging the vector-based Flash platform as a user expands or shrinks the window, Flash dynamically scales all elements in direct proportion to the original aspect ratio with which they were created. This means that elements never change their position in relation to each other; they all grow or shrink as the user changes the size of the window. This vector scaling radically simplifies design issuesno matter at what resolution the visualization is designed, you can be assured that it will be viewable across a wide range of display devices. Another aspect of the visual design capabilities of CX is your ability to set alpha channels (transparencies) of objects and position them in relative layers. This gives you incredible flexibility in how you lay out your dashboards and data visualizations. You can layer charting elements and position background images and containers relative to other visualizations in a set of layers. In this respect the CX Designer is much more like Photoshop or Macromedia Flash Professional in that it allows you to create sophisticated visual layouts using many different design elements. If all you did was visualize data in a static manner using Crystal Xcelsius, you still would have a phenomenal design tool that allows great productivity gains in your dashboard development, but it is when you explore the more interactive features of CX that a whole new world of possibilities opens up to you. One of the more basic yet powerful interactive features is CXs ability to leverage dynamic visibility for every object you have in your workspace. Each object allows its visibility to be toggled on or off by

the value of a specific Excel cell that you set at design time. This Excel cell can contain a simple value or it can be the result of a set of complex calculations. The result is that you can build mini-state machines that turn parts of your dashboard on and off depending on what action the user has taken. For instance, you might have the user roll over or click on a gauge and have a corresponding time series chart pop into view, or vice versa. If this is beginning to sound more like the interactivity you would see in a desktop application than in a simple data report, you are correct. You could actually build a full RIA application, if you so desired, that allowed the user to view data from a persistent data store, manipulate it, AND write it back to the same data store.

Crystal Xcelsius and other Business Objects products

One of the most exciting aspects of Crystal Xcelsius is the direction the product is headed. Immediately after the acquisition of the Crystal Xcelsius technology from its original inventors at Infommersion, the Business Objects development team began to integrate Crystal Xcelsius into the rest of the product line, starting with the Crystal family of products. The release of Crystal Vision and Crystal Vision Server promises to provide a powerful marriage of the industry-leading Crystal Reports and Crystal Reports Server with the visualization and interactive capabilities of Crystal Xcelsius. What this means for developers already familiar with the Crystal family of products is that they will be able to seamlessly use their existing reports, security, and navigation structures to build exciting and interactive dashboards and data visualizations by pointing CX to the Crystal Vision server at design time. Stay tuned for additional tech docs related to this. For a developer who has not used the Crystal products before or is new to the Business Objects platform, Crystal Vision will provide a complete end-to-end reporting and data visualization infrastructure with robust data connectivity, security, and deployment options. You will have the ability to quickly and securely connect to dozens of disparate data sources through the Crystal Reports designer, create meta data and SQL statements, and then deploy those reports either via a Webbased reporting portal or as a set of static files. You can then design your Crystal Xcelsius visualizations to utilize your Crystal reports and their data structures either in their static form or dynamically, allowing a completely flexible, secure, and scalable visualization solution. You will find additional documentation on how developers can benefit from Crystal Vision and Crystal Vision Server at: www.businessobjects.com/products/crystalvision/server.asp .

Next steps
If you are interested in learning more about Crystal Xcelsius and seeing it in action for yourself, your best bet is to download a trial version of CX and roll up your sleeves. The actual product comes with several well-documented sample dashboards that quickly teach you the basics. Additionally, each object has its own property window with animated help sequences showing you how to map your data from Excel into your visualization. Once you have mastered the basics, the next step is to visit the learning center at www.xcelsius.com/Learning/Center.html where you will find many in-depth articles covering a wide variety of topics that teach you about the various design, interactivity, and integration features of Crystal Xcelsius. There is no reason why an intermediate-level developer could not have a dashboard up and running in only a couple of hours; a complete novice wouldnt need much longer. One of the joys of working with the tool is how quickly you see results and the way you can bring data to life. What used to be the dull, mundane job of creating reports for business users is now an exciting and engaging activity where you can see your efforts enabling the user to make more informed and timely decisions through your interactive and visually appealing designs. Best of all, you can now create these solutions in a fraction of the time it previously would have taken you.