Sumerics Case Study

By Geoff Arnold

Download Article

Introduction

Florian Rappl, a 28-year-old PhD student at the University of Regensburg in Germany, won the App Innovation Contest for creating Sumerics, an app that performs a complex analysis of Ultrabook™ device sensor data and displays the results in visually compelling 2D and 3D graphs. Built using existing frameworks and languages—specifically Windows* Presentation Foundation (WPF) and C#—Sumerics takes full advantage of the sensor and touch screen capabilities of Ultrabook devices running Microsoft Windows 8. While Rappl chose the well-established WPF for rendering the user interface, he relied on the relatively new open-source library MahApps.Metro to give Sumerics' UI its modern, touch-friendly look and feel.

Sumerics is available on the Intel AppUp® center.

Figure 1. A simple 3D plot rendered by Sumerics.

About the Sumerics App

Rappl designed Sumerics with two goals in mind: First, he wanted to present an intuitive touch-enabled user interface for visualizing complex data series gathered either from various static files, such as CSV, or from real-time dynamic data from Ultrabook device sensors. Second, he wanted to make it easy for users to directly access these data series as inputs in computational analysis. To enable this analysis, Rappl exposed all sensors as functions for displaying and analyzing data, a design decision that allows users to gather information about the current states of the Ultrabook device sensors for further exploration. Rappl said anyone who deals with data visualization might benefit from the app. For example, Sumerics could be used in an advanced course on acoustical physics to analyze words spoken into the Ultrabook device microphone through the use of Sumerics' fast Fourier transform algorithm, enabling users to obtain a visual representation of the frequencies for certain words by specific speakers.

Figure 2. The Sumerics "sensors" tab, where instant plots for most sensors can be created. By default this tab is activated and gives instant information about sensors, without the need for users to type a command.

Using the Ultrabook device keyboard, attached mouse, or other peripheral, a Sumerics user can input and edit data, similar to using MATLAB*, a high-level language and interactive environment for numerical computation, visualization, and programming. However, because everything is designed with touch in mind, the graphs that Sumerics displays can be rotated, translated, scaled, saved, and printed using on-screen touch commands. In this way, Sumerics takes advantage of the power and flexibility of Ultrabook devices, which Rappl said provides the killer combination of a tablet's portability and a laptop's high performance and ability to run standard software.

Challenges Faced in Creating Sumerics

During the process of creating Sumerics, Rappl faced four primary challenges.

Deciding whether to develop a Windows Store app or an app to run in Windows 8 Desktop
Deciding whether to build the app using HTML5 or WPF
Properly planning for touch and sensor implementation
Building a math parser and deciding which plotting libraries to use

The following sections describe how Rappl addressed each of these challenges.

Navigating Windows 8

Windows 8 presents a choice to end users. They can navigate the Ultrabook device in Desktop mode, which is similar to navigating PCs and laptops running earlier versions of Windows. Or they can switch to one of several flat, laterally scrolling tiles using the modern, touch-oriented UI that is already familiar to smartphone and tablet users. Consequently, developers must decide how end users will eventually access their apps.

Rappl chose to build Sumerics for the Windows 8 desktop, primarily because he already had experience building Windows desktop apps, particularly using C# with the WPF UI framework, which has been available for several years. Accordingly, many open-source libraries are available, including ones that make it easy to build touch-enabled UIs.

One disadvantage Rappl found is that, as is, the app coded in C# with WPF for the desktop UI cannot be submitted to the Windows Store. Apps in the Windows Store are generally coded in JavaScript* for UIs built around HTML5 and CSS3, and Rappl said he would have used this approach if he wanted to make sure Sumerics was distributed on Windows Store. (For now it's available on the Intel AppUp® center). Rappl noted that it is possible to use the WinRT component library to write a JavaScript wrapper for a C#-based app, which can then be distributed through the Windows Store. However, at the time Sumerics was created, Rappl said that bugs were associated with this approach, and indeed all Windows Store apps built with C#/VB/C++ and XAML, in part because of difficulties in taking advantage of the WPF binding capabilities.

Using existing tools and technologies, Rappl built a touch-enabled app that takes full advantage of Ultrabook device sensors and runs smoothly on Ultrabook devices running Windows 8, providing user access and navigation for Sumerics in the Desktop mode. Given the long and established legacy of the Windows desktop, Sumerics has a channel to reach a potentially large audience.

HTML5 versus WPF

For Rappl, a key part of navigating Windows 8 was deciding whether to build the app using HTML5 or WPF. He concluded that HTML5 undeniably has a bright future, which he expects to eventually be the de facto standard for building platform-independent apps. However, since he was already well-versed in WPF he build the UI around that graphical subsystem, which can be used to build compelling Windows 8 desktop apps that are touch- and sensor-enabled. Rappl said WPF introduces device-independent pixels, which offer a significant advantage compared to the alternative of the much older Windows Forms, in use since the early days of the Microsoft .NET Framework. He lauded the data-binding capabilities of WPF, which also fully utilize the concepts of attached properties and dependency properties.

Figure 3:The subplot feature, which allow the user to view multiple plots from the same data series simultaneously.

However, WPF has its own challenges. For example, like Windows Forms, the controls can be heavy. But because everything (other than images) is a vector and can run over DirectX*, the UI can be accelerated, thus minimizing this problem. The controls can also be completely customized, and developers can make full use of the ability to differentiate between the logical and visual tree. Plus, the 3D capabilities in WPF provide a modest productivity boost.

As for HTML5 (used in combination with CSS3 and JavaScript), Rappl thinks the markup language will in all likelihood be central for apps headed for a cross-platform touch screen future, which arguably has already arrived. But issues remain. For example, it's frustrating, he said, that while HTML5 and C# are both cross-platform, the combination of the two is not. Attempting to work around this constraint using C#-to-JavaScript, source-to-source compilers can be confusing. And any move to JavaScript, a dynamic language, means sacrificing the speed and responsiveness associated with C#, a static language. So for Rappl, who wanted a maximally responsive app for his sophisticated users, C# with WPF was the way to go.

Properly planning for touch and sensor implementation

Sumerics was the first app Rappl developed for touch screens. Rappl's three main takeaways: First, for ease of use, make each button at least 40x40 pixels—large enough to be clearly visible and easy to push or tap. Second, use tooltips (the popup text enabled by placing the mouse cursor over the element) as a complementary feature; always work to ensure that using the program is obvious without having to read the text. Third, ensure the main features of the app are accessible even without a keyboard—a simple enough rule to adhere to when designing for tablets and smartphones, but more difficult to follow when building a touch-aware app, specifically one that will be accessed on the Windows 8 desktop on an Ultrabook device.

Rappl said dealing with sensors turned out to a secondary part of his overall development work. This was because the main focus of Sumerics is data visualization of data series that can come from any source, be it an imported CSV file or an onboard accelerometer. Indeed, Sumerics works well even if it's used just to visualize data series imported from static files. Also, because it was relatively straightforward to expose sensor data as functions using the Windows Sensor and Location Platform, Rappl was able to spend most of his time working on how to best render compelling graphs and plots.

Figure 4.The Sumerics'"interaction" tab, where users can type commands, get help, or view variables currently available in the workspace (shown on the right). These variables are used to store values from previous calculations. The tab presents an overview that gives the user information about not only which variables are available, but also what kind of value is stored in them, such as a scalar, matrix, or plot. A graphical representation of the value is given when the user touches the variable.

Building a math parser and deciding which plotting libraries to use

At the heart of Sumerics is Yet Another Math Parser (YAMP*), which takes a string of characters and transforms them into mathematical expressions. When Rappl saw the Windows* 8 & Ultrabook™ App Innovation Contest advertised on CodeProject, he decided to simultaneously build both the Sumerics app and a math parser, one that he hoped could take advantage of the features of C#. By releasing the YAMP code to the open-source community, Rappl tapped their expertise during the code review process central to the open-source world.

Here's a high-level look at the Sumerics architecture, where green represents third-party libraries, orange is the YAMP external library, and blue represents the new libraries associated with Sumerics:

Sumerics obviously relies heavily on YAMP, which requires a certain format for commands. For example, if a YAMP user wants to clear certain variables, he or she must enter the following command:

clear("a", "b", "c")

With Sumerics the command is simpler:

clear a b c

This works because Sumerics has a command parser as well. Rappl added the parser to make it as easy as possible for end users to call on YAMP functions. If the command parser finds a valid command, Sumerics executes it. Otherwise the whole expression is passed to YAMP, which then parses and interprets the expression.

Rappl says writing the command parser was easy and required only an abstract base class. Below, a code snippet shows how he employed reflection to register the methods that the Sumerics command parser can use.

public static void RegisterCommands()
{
    var lib = Assembly.GetExecutingAssembly();
    var types = lib.GetTypes();

    foreach (var type in types)
    {
        if (type.IsAbstract)
           continue;

        if (type.IsSubclassOf(typeof(YCommand)))
        {
           var cmd = type.GetConstructor(Type.EmptyTypes).Invoke(null) as YCommand;
           commands.Add(cmd.Name.ToLower(), cmd);
        }
    }
}

A small subset of the classes that make some functions available from the commands parser is displayed below.

The command parser works with an arbitrary number of arguments. The basic code is similar to that described by Rappl in his article about YAMP on CodeProject.

To write the specific sensor functions in the YAMP plug-in, Rappl first started with an abstract base class, which he extended to the various sensors. Here's how Rappl implemented the acc() function:

using System;
using Windows.Devices.Sensors;

namespace YAMP.Sensors
{
    [Description("Provides access to the acceleration sensor of an Intel UltraBook™.")]
	[Kind("Sensor")]
    public class AccFunction : SensorFunction
    {
        static Accelerometer sensor;

        static AccFunction()
        {
            try
            {
                sensor = Accelerometer.GetDefault();
            }
            catch { }
        }

        protected override void InstallReadingChangedHandler()
        {
            if(sensor != null)
                sensor.ReadingChanged += OnReadingChanged;
        }

        protected override void UninstallReadingChangedHandler()
        {
            if (sensor != null)
                sensor.ReadingChanged -= OnReadingChanged;
        }

        void OnReadingChanged(Accelerometer sender, AccelerometerReadingChangedEventArgs args)
        {
            RaiseReadingChanged(args.Reading);
        }

        /// <summary>
        /// retrieves acceleration in (X,Y,Z)-direction in units of g
        /// </summary>
        /// <returns></returns>
        [Description("Retrieves acceleration in (X, Y, Z)-direction in units of g. Hence usually (no movement) the returned vector will be (0, 0, 1).")]
        [ExampleAttribute("acc()", "Returns a 3x1 matrix of accelerations in the x, y and z directions.")]
        public MatrixValue Function()
        {
            return new MatrixValue(Acceleration);
        }

        public static double[] Acceleration
        {
            get
            {
                if (sensor == null)
                    return new double[3];
                var acc = sensor.GetCurrentReading();
                return new double[] { acc.AccelerationX, acc.AccelerationY, acc.AccelerationZ };
            }
        }
    }
}

Note how Rappl provided many static functions, which allows direct access from Sumerics (for live data of the sensors), without creating an explicit instance or having YAMP interpret a fixed expression. The sensor reading AccelerometerReading is included in a nested class. If there is no sensor data, a null value is returned, though YAMP always returns data—specifically scalars and matrices with correct dimensions but without any values (so every value is 0).

Because Sumerics focuses on data visualization, it was critical to choose the appropriate libraries for displaying 2D and 3D plots. In the end the decision was between two open-source libraries: IronPlot and OxyPlot. IronPlot, which has the ability to render 3D plots, provides a suitable plotting package for IronPython (without using Mathplotlib, which would have added a dependency on Python and a dependency on the communication between C# and Python). OxyPlot is a platform-independent plotting package that includes a WPF implementation. OxyPlot, according to Rappl, is well documented, extensible, and full of great features, including the ability to annotate and track plots.

For 3D plotting, Rappl created his own library, albeit one built around OxyPlot. This library allowed for the rendering of not only rectangles, lines, and more from a data series, but also a complete image. This rendering led to a demonstrable increase in performance and made it possible to plot heat maps and other complex graphics that display well on the various Ultrabook device screens available today. (Several Ultrabook devices have screen resolutions of up to 1600x900.)

Helpful Resources

Rappl read the Intel articles that focus on using Ultrabook device sensors, including articles available at http://software.intel.com/en-us/articles/ultrabook-touch-and-sensor-resources. He also reviewed specifications for touch-aware apps (not just touch-enabled apps) and consulted external resources, including those on CodeProject. Additionally, Rappl relied on the Intel AppUp® Developer certification tool for creating simple MSI setup files. More detail about Rappl's work on Sumerics is described in an article he wrote for CodeProject: http://www.codeproject.com/Articles/472698/Sumerics. Rappl has also posted several short videos of himself using Sumerics, including one showing how to plot data captured from the Ultrabook device inclinometer, gyrometer, and compass, on his YouTube* channel: http://www.youtube.com/user/FlorianRappl.

Things to Consider

Rappl considers HTML5 to be the future for touch screen UIs and, with the experience he gained from creating Sumerics, stated that if he were starting over he would likely use HTML5 instead of WPF, particularly if he wanted to distribute his app on the Windows Store.

Conclusion

Sumerics takes data imported from static files or gathered from Ultrabook's many onboard sensors and produces compelling 2D and 3D graphs that can be easily manipulated via the touch screen. This sort of advanced data visualization might prove useful in various environments, from engineering test labs at established companies to high school physics classrooms. (In a basic course covering classical mechanics, students could set up an experiment to let an Ultrabook fall from a reasonably tall building and then use Sumerics to review data from the accelerometer to see if the device achieved weightlessness.) Despite the relative newness of the Ultrabook device hardware and Windows 8 operating system, Rappl built the award-winning app in a matter of months, working around other commitments, mostly using well-established languages and frameworks (C# and WPF) and freely available libraries.

Rappl concludes that Ultrabook devices running Windows 8 represent a relatively new category of devices, one marked by the portability and ease-of-use of touch screen tablets along with the power of a full-featured laptop. But his experience building a compelling app that evokes comparisons to industry-leading MATLAB-based products demonstrates that developers well-versed in established Microsoft technologies, particularly C# and WPF, can build a compelling touch screen app that takes full advantage of the Ultrabook device's many features—including its various sensors—while staying in the familiar territory of the Microsoft desktop, albeit one that's now touch-enabled.

About Author

Ostensibly Florian Rappl's main focus is writing his dissertation—the weighty working title is "Feynman Diagram Sampling for Quantum Chromodynamics"—in the field of theoretical particle physics. However, he's also active in computing in a way that far exceeds the work of most physics grad students. Currently he teaches graduate level courses in C# programming and in designing Web applications with HTML5, CSS3 and JavaScript. He's involved in several high-performance computing projects at the University of Regensburg, including one attempting to build a supercomputer with Intel® Many Integrated Core Architecture chips. He's an MVP at CodeProject, one of the more thriving developer communities on the Web. It turns out that winning the Windows* 8 & Ultrabook™ App Innovation Contest is just the latest in a long string of accomplishments for Rappl, though in addition to the acclaim, this accomplishment also netted him more than USD 30,000 in prize money plus a new Ultrabook device.

Portions of this document are used with permission and copyright 2012 by CodeProject. Intel does not make any representations or warranties whatsoever regarding quality, reliability, functionality, or compatibility of third-party vendors and their devices.

All products, dates, and plans are based on current expectations and subject to change without notice. Intel, the Intel logo, Intel AppUp, Intel Atom, the Intel Inside logo, and Ultrabook, are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others.

ultrabook

Windows store

sensor

Windows Presentation Foundation

Microsoft Windows* 8 Style UI