Why 2-in-1
A 2 in 1 is a PC that transforms between a laptop computer and a tablet. Laptop mode (sometimes referred to as desktop mode) allows a keyboard and mouse to be used as the primary input devices. Tablet mode relies on the touchscreen, thus requiring finger or stylus interaction. A 2 in 1, like the Intel® Ultrabook™ 2 in 1, offers precision and control with multiple input options that allow you to type when you need to work and touch when you want to play.
Developers have to consider multiple scenarios in modifying their applications to take advantage of this new type of transformable computer. Some applications may want to keep the menus and appearance nearly identical in both modes. While others, like Krita Gemini for Windows* 8 (Reference 1), will want to carefully select what is highlighted and made available in each user interface mode. Krita is a program for sketching and painting, that offers an end-to-end solution for creating digital painting files from scratch (Reference 2). This article will discuss how the Krita developers added 2 in 1 mode-awareness - including implementation of both automatic and user-selected mode switching and some of the areas developers should consider when creating applications for the 2 in 1 experience to their applications.
Introduction
Over the years, computers have used a variety of input methods, from punch cards to command lines to point-and-click. With the adoption of touch screens, we can now point-and-click with a mouse, stylus, or fingers. Most of us are not ready to do everything with touch, and with mode-aware applications like Krita Gemini, we don’t have to. 2 in 1s, like an Intel® Ultrabook™ 2 in 1, can deliver the user interface mode that gives the best experience possible, on one device.
There are multiple ways that a 2 in 1 computer can transform between laptop and tablet modes (Figure 1 & Figure 2). There are many more examples of 2 in 1 computers on the Intel website (Reference 3). The computer can transform into tablet-mode from laptop-mode by detaching the screen from the keyboard or using another means to disable the keyboard and make the screen the primary input device (such as folding the screen on top of the keyboard). Computer manufacturers are beginning to provide this hardware transition information to the operating system. The Windows* 8 API event, WM_SETTINGCHANGE and the “ConvertibleSlateMode” text parameter, signal the automatic laptop to tablet and back to laptop mode changes. It is also a good idea for developers to include a manual mode change button for users’ convenience as well.
Just as there are multiple ways that the 2 in 1 can transform between laptop and tablet modes, software can be designed in different ways to respond to the transformation. In some cases it may be desirable to keep the UI as close to the laptop mode as possible, while in other cases you may want to make more significant changes to the UI. Intel has worked with many vendors to help them add 2 in 1 awareness to their applications. Intel helped KO GmBH combine the functionality of their Krita Touch application with their popular Krita open source painting program (laptop application) in the new Krita Gemini application. The Krita project is an active development community, welcoming new ideas and maintaining quality support. The team added the mechanisms required to provide seamless transition from the laptop “mouse and keyboard” mode to the touch interface for tablet mode. See Krita Gemini’s user interface (UI) transformations in action in the short video in Figure 3.
Figure 3: Video - Krita Gemini UI Change – click icon to run
Create in Tablet Mode, Refine in Laptop Mode
The Gemini team set out to maximize the user experience in the two modes of operation. In Figure 4 & Figure 5 you can see that the UI changes from one mode to the other are many and dramatic. This allows the user to seamlessly move from drawing “in the field” while in tablet mode to touch-up and finer detail work when in laptop mode.
Figure 4:Krita Gemini tablet user interface
Figure 5: Krita Gemini laptop user interface
There are three main steps to making an application transformable between the two modes of operation.
Step one; the application must be touch aware. We were somewhat lucky in that the touch-aware step was started well ahead of the 2 in 1 activity. Usually this is a heavier lift than the transition to and from tablet mode work. Intel has published articles on adding touch input to a Windows 8 application (Reference 4).
Step two, add 2 in 1 awareness. The first part of the video (Figure 3) above demonstrates the automatic, sensor activated mode change, a rotation in this case (Figure 6). After that the user-initiated transition via a button in the application is shown (Figure 7).
Figure 6:Sensor-state activated 2 in 1 mode transition
Figure 7:Switch to Sketch transition button – user initiated action for laptop to tablet mode
Support for automatic transitions requires the sensor state to be defined, monitored, and appropriate actions to be taken once the state is known. In addition, a user initiated mode transition3 should be included as a courtesy to the user should she wish to be in the tablet mode when the code favors laptop mode. You can reference the Intel article “How to Write a 2-in-1 Aware Application” for an example approach to adding the sensor-based transition (Reference 5). Krita’s code for managing the transitions from one mode to the other can be found in their source code by searching for “SlateMode” (Reference 6). Krita is released under a GNU Public License. Please refer to source code repository for the latest information (Reference 7).
// Snip from Gemini - Define 2-in1 mode hardware states:
#ifdef Q_OS_WIN
#include <shellapi.h>
#define SM_CONVERTIBLESLATEMODE 0x2003
#define SM_SYSTEMDOCKED 0x2004
#endif
Not all touch-enabled computers offer the automatic transition, so we suggest you do as the Krita Gemini team did here and include a button in your application to allow the user to manually initiate the transition from one mode to the other. Gemini’s button is shown in Figure 7. The button-initiated UI transition performs the same functions as the mechanical-sensor-initiated transition. The screen information and default input device will change from touch & large icons in tablet mode to keyboard, mouse and smaller icons in the laptop mode. However, since the sensor path is not there the button method must perform the screen, icon, and default input device changes without the sensor-state information. Therefore, developers should provide a path for the user to change from one mode to the other with touch or mouse regardless of the state of the button-initiated UI state in case the user chooses an inappropriate mode.
The button definition - Kaction() - as well as its states and actions are shown in the code below (Reference 6):
Engineers then took on the task of handling the events triggered by the button. Checking the last known state of the system first since the code cannot assume it is on a 2-in-1 system, then changing the mode. (Reference 6): Step three, fix issues discovered during testing. While using the palette in touch or mouse mode is fairly easy, the workspace itself needs to hold focus and zoom consistent with the user’s expectations. Therefore, making everything bigger was not an option. Controls got bigger for touch interaction in tablet mode, but the screen image itself needed to be managed at a different level as to keep an expected user experience. Notice in the video (Figure 3) the image in the edit pane stays the same size on the screen from one mode to the other. This was an area that took creative solutions from the developers to reserve screen real estate to hold the image consistent. Another issue was that an initial effort had both UIs running which adversely affected performance due to both UIs sharing the same graphics resources. Adjustments were made in both UIs to keep the allotted resource requirements as distinct as possible and prioritize the active UI wherever possible. Wrap-up As you can see, adding 2 in 1 mode awareness to your application is a pretty straightforward process. You need to look at how your users will interact with your application when in one interactive mode versus the other. Read the Intel article “Write Transformational Applications for 2 in 1 Devices Based on Ultrabook™ Designs“ for more information on creating an application with a transforming user interface (Reference 8). For Krita Gemini, the decision was made to make creating drawings and art simple while in tablet mode and add the finishing touches to those creations while in the laptop mode. What can you highlight in your application when presenting it to users in tablet mode versus laptop mode? References About the Author Tim Duncan is an Intel Engineer and is described by friends as “Mr. Gidget-Gadget.” Currently helping developers integrate technology into solutions, Tim has decades of industry experience, from chip manufacturing to systems integration. Find him on the Intel® Developer Zone as Tim Duncan (Intel) Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. // Snip from Gemini - Define 2-in1 Mode Transition Button:
toDesktop = new KAction(q);
toDesktop->setEnabled(false);
toDesktop->setText(tr("Switch to Desktop"));
SIGNAL(triggered(Qt::MouseButtons,Qt::KeyboardModifiers)), q, SLOT(switchDesktopForced()));
connect(toDesktop,
SIGNAL(triggered(Qt::MouseButtons,Qt::KeyboardModifiers)), q, SLOT(switchToDesktop()));
sketchView->engine()->rootContext()->setContextProperty("switchToDesktop
sketchView->Action", toDesktop);// Snip from Gemini - Perform 2-in1 Mode Transition via Button:
#ifdef Q_OS_WIN
bool MainWindow::winEvent( MSG * message, long * result ) {
if (message && message->message == WM_SETTINGCHANGE && message->lParam)
{
if (wcscmp(TEXT("ConvertibleSlateMode"), (TCHAR *) message->lParam) == 0)
d->notifySlateModeChange();
else if (wcscmp(TEXT("SystemDockMode"), (TCHAR *)
message->lParam) == 0)
d->notifyDockingModeChange();
*result = 0;
return true;
}
return false;
}
#endif
void MainWindow::Private::notifySlateModeChange()
{
#ifdef Q_OS_WIN
bool bSlateMode = (GetSystemMetrics(SM_CONVERTIBLESLATEMODE) == 0);
if (slateMode != bSlateMode)
{
slateMode = bSlateMode;
emit q->slateModeChanged();
if (forceSketch || (slateMode && !forceDesktop))
{
if (!toSketch || (toSketch && toSketch->isEnabled()))
q->switchToSketch();
}
else
{
q->switchToDesktop();
}
//qDebug() << "Slate mode is now"<< slateMode;
}
#endif
}
void MainWindow::Private::notifyDockingModeChange()
{
#ifdef Q_OS_WIN
bool bDocked = (GetSystemMetrics(SM_SYSTEMDOCKED) != 0);
if (docked != bDocked)
{
docked = bDocked;
//qDebug() << "Docking mode is now"<< docked;
}
#endif
}
Copyright © 2013-2014 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.