XNA for Silverlight developers: Part 10 - UI Elements and Menus

This article is part 10 of the series "XNA for Silverlight developers".

This article is compatible with Windows Phone "Mango".
Latest update: Aug 1, 2011.

Games inherently have a different UI and user experience concept compared to business applications. Where you scroll through endless lists of items and enter data in greyish forms on one side, you use a game pad or touch input to control a character through a thrilling fantasy world on the other side. In some situations however, you suddenly find yourself to be in need of all these boring controls of normal desktop applications for your game too. Whenever you want to create a menu or request certain data from the user for example, you need the same kind of UI in one way or another: text boxes, combo boxes, buttons. In this article, we'll evaluate the options you have in XNA, compare this to Silverlight, and take a quick look at the future.

The situation with Silverlight

Desktop Silverlight has a strong background in business applications, and its little brother on the phone has inherited a lot of the powerful features to create rich user interfaces. Not only do we have a built-in set of all kinds of controls that offer options for almost all scenarios, things like data binding, styles and templates also make it very easy to create appealing interfaces quickly without giving up a clear separation of data and views. A sophisticated event model helps us interact with these controls and react to user input easily, and additional containers and features allow us to create nice screen layouts and dynamic arrangements within minutes. When you are coming from this background, the situation in XNA will be – gently said – shocking.

What we have in XNA

Simply put, nothing of all this. In previous parts, we have talked about some of the involved differences regarding UI elements and user input in both worlds already. I very much encourage you to read Part 1 if you haven't done yet. I explain the difference between Silverlight's event-driven programming model and XNA's polling-oriented design in detail there. In Part 5, when we were talking about touch input for the first time, I also reviewed Silverlight's concept of controls and contrasted that with XNA's much more low-level approach. I won't go into further details regarding these difference in this article again.

So, what do we have in XNA? The good news is that we have all the tools and possibilities to create a similar user experience at hand. The bad news of course is that we have to do all this manually, and depending on what your requirements are, this may result in a lot of effort and code up to a level where it is advisable to look for other options, like third-party libraries.

A look at a working sample

If you recall the sample we used in the last article you may remember that it already had an options menu that allowed the user to change some (bogus) parameters of the game:

If you inspect the code for this menu, you will see that it actually tries to modularize the involved parts. There are base classes for the menu screen, a derived class for this options menu (all of which are inherited game screens from the last part), and a separate class for the menu entries on the screen that even works with events when the user selects one. The logic to determine whether a menu entry has been selected by the user or not is contained in the "MenuScreen" class and looks like this:

 1: // look for any taps that occurred and select any entries that were tapped

 2: foreach (GestureSample gesture in input.Gestures)

 3: {

 4:     if (gesture.GestureType == GestureType.Tap)

 5:     {

 6:         // convert the position to a Point that we can test against a Rectangle

 7:         Point tapLocation = new Point((int)gesture.Position.X, (int)gesture.Position.Y);

 8:  

 9:         // iterate the entries to see if any were tapped

 10:         for (int i = 0; i < menuEntries.Count; i++)

 11:         {

 12:             MenuEntry menuEntry = menuEntries[i];

 13:  

 14:             if (GetMenuEntryHitBounds(menuEntry).Contains(tapLocation))

 15:             {

 16:                 // select the entry

 17:                 OnSelectEntry(i, PlayerIndex.One);

 18:             }

 19:         }

 20:     }

 21: }

The approach used here is that the input is handled on the screen level. The screen iterates over all controls and uses the available information about the current user input and control dimensions to decide whether a menu entry is affected by the user input or not. Here is a flow chart of how the logic works:

What the sample implementation does when a 'click' is detected is tell the menu entry to raise it's "Selected" event. This may seem a bit convoluted at first, but it actually enables derived classes (like the options menu) to do something meaningful with this event.

A different approach would be to not have any logic in the screen class itself, but simply pass through the request to handle user input to each one of the controls on the screen. Then every menu entry could decide itself what to do with the input, which would be more flexible. Not matter what alternative or variation you use, this is basically how you would create the core logic for any custom UI concept: Since there is no built-in concept of UI elements in XNA, let alone a mechanism to inform you a particular one has been selected, it really comes down to creating your own elements, and then simply iterating over all of them to see if one is affected by the current user input.

Of course all the menu entries in the example only cycle through a given set of values (including an on/off switch) or implement only a simple increment logic. But based on this rather simple structure you could already build a set of controls that is able to handle a lot of the most common requirements. The "MenuEntry" class is designed very openly for this. By overriding the "Update" and "Draw" methods you are able to change and extend the logic as well as the visual appearance of the entries.

Text input

Things are a bit different for text input you need to request from the user. While it would be possible to design and display a textbox-like control, the problem with this is the input method on the phone device. Apart from few exceptions with hardware keyboards, Windows Phone 7 relies on a SIP (Soft Input Panel). At the moment, there is no way to bring up the SIP to interact with your game, which means you would have to create your own on-screen keyboard. This is something far from being trivial (think different cultures etc.) and would most likely result in a user experience different from the rest of the phone. It also doesn't scale once additional languages are supported by the system and therefore needs adjustments once updates for this are rolled out by Microsoft.

The solution to this once again is the Guide class. It has a method "BeginShowKeyboardInput" that brings up a simple data input screen with a customizable title, description and a text box. It handles all the SIP logic for you, and once the user has finished entering their data, a callback will be invoked to notify you about the result, which you then can use for whatever you want.

This limitation has been heavily discussed in the community. It means that you cannot have a consistent design for the data input screen and your game (you cannot change its appearance), and you also don't have any direct control over the functionality (e.g. you cannot restrict input to certain characters or do other sorts of validation on that screen etc.). We'll see how this will be changed in the future below.

As an example I've extended the sample from the last article and added text input to it. For this, I simply switch to the high scores list when the user taps the game play screen to simulate a successfully finished game. In this case (coming from the game play instead of the main menu) the user is prompted for their name to create an entry in the high scores list:

 1: // is the user supposed to enter their name?

 2: if (EnterHighscore)

 3: {

 4:     // start showing the input screen

 5:     Guide.BeginShowKeyboardInput(ControllingPlayer.Value,

 6:                                  "High Score!",

 7:                                  "Please enter your name",

 8:                                  string.Empty,

 9:                                  KeyboardInputFinished,

 10:                                  null,

 11:                                  false);

 12:  

 13:     // set the flags we need

 14:     EnterHighscore = false;

 15:     _userInputInProgress = true;

 16:     return;

 17: }

The most important argument of the method is the "KeyboardInputFinished" callback, a method which is invoked automatically when the user closes the input screen. The second flag is used to pause our custom update and draw logic as long as the input screen is visible. This is necessary because the keyboard input screen works asynchronously, which means that the game continues to run in the background. The callback implementation looks like this:

 1: private void KeyboardInputFinished(IAsyncResult ar)

 2: {

 3:     // reset flag

 4:     _userInputInProgress = false;

 5:  

 6:     // get the result

 7:     string result = Guide.EndShowKeyboardInput(ar);

 8:     if (!string.IsNullOrEmpty(result))

 9:     {

 10:         // add a bogus highscore entry

 11:         string entry = string.Format("11. {0} - {1}", result, 10);

 12:         _highScores.Add(entry);

 13:     }

 14: }

As you can see, by invoking the "EndShowKeyboardInput" method on the Guide class you will receive the string the user has input. In our case this string is only used to create a bogus additional high score entry, but you should get the idea how you can use this mechanism to query input from the user.

Further resources

Apart from the game state sample that we've used here, Microsoft has also published another example in the App Hub Education section that covers UI elements and menus in XNA. It is named "User Interface Controls" and can be found here. This example contains a set of base classes for the control itself as well as a panel. Derived from this it has an image control, a page flip control to flick through items and scrolling controls to display lists of items. Even if you don't use the code as-is, it should also be possible to get some ideas from the samples to develop your own controls.

In the past there have also been some attempts to create UI frameworks for XNA, some of which looked very promising. Unfortunately a lot of them were discontinued after a while, a few even disappeared completely. One that is still actively developed (and brings a lot more than just UI elements) is the Nuclex Framework which can be found on Codeplex.

Another interesting framework is XPF, which has some strong layout features. It has not yet reached production level quality, but it has nightly debug builds you can use for testing. Apparently the licensing will include a free version for non-commercial and open source development.

Integrating XNA and Silverlight

An obvious solution to all of XNA's lacking UI features on Windows Phone 7 would be to allow mixing Silverlight with XNA graphics in the same application, so a game can make use of Silverlight's powerful features too. In fact, some people started to investigate this very early on, but of course application certification requirements of the RTM version explicitly forbid doing this, so any success with it was futile.

Things very much change with Windows Phone "Mango", where you have the chance and are allowed to do exactly that. The general application structure is significantly different when you make use of this, because then Silverlight needs to take the leading role, and using XNA is achieved by switching to it from Silverlight using a feature named "sharing mode". I discuss and demonstrate all the technical details in part 12 of this series, which of course also comes with full source code for you to explore this.

The important thing is that you can decide what parts of your game you want to implement with Silverlight, and what parts should be handled by XNA. You can use all of Silverlight's UI features to create and design your menus and other data-centric screens like highscore lists, and then switch to XNA for the actual gameplay and game content rendering. The following diagram shows a logical screen structure for such a setup, with each rectangle representing a phone application page:

But the good news doesn't stop here. You are even able to mix Silverlight and XNA content on the same page if you want. This allows you to e.g. use the more sophisticated Silverlight text rendering directly on your XNA gameplay screen. HUD overlays or other things can be handled with default Silverlight UI elements more easily, and then used in your XNA rendering process with the help of the new UIElementRenderer class.

The beauty of this approach is that Silverlight elements are not forced into a passive role by this. User input can be routed to these elements, and with a little bit of work you can even use elements like text boxes on XNA-controlled pages, and have the user enter data using the SIP. And of course all the dynamic goodies of Silverlight, for example it's excellent animation features, can be used in this scenario too.

Again, part 12 of this series shows how to do most of this by demonstrating the UIElementRenderer in combination with Silverlight animations on an application page that is rendered by XNA.

Summary

In this article we once again learned that the more low-level nature of XNA (which has a lot of advantages in game programming) comes at the cost of a lot less comfort. Especially when it comes to creating user interfaces like menus things can easily become complex and result in a lot of work, to a point where it absolutely makes more sense to use some already available commercial or free third-party libraries. However, we've also seen that the first major update to the platform ("Mango") brings exciting news about the future of XNA and Silverlight integration, which will make a lot of these things much easier. If you are interested in the updated sample code that contains the user text input code, you can get it here:

Download source code

As always, feel free to add your comments below or provide feedback to me directly.

About the author

Peter Kuhn aka "Mister Goodcat" has been working in the IT industry for more than ten years. After being a project lead and technical director for several years, developing database and device controlling applications in the field of medical software and bioinformatics as well as for knowledge management solutions, he founded his own business in 2010. Starting in 2001 he jumped onto the .NET wagon very early, and has also used Silverlight for business application development since version 2. Today he uses Silverlight, .NET and ASP.NET as his preferred development platforms and offers training and consulting around these technologies and general software design and development process questions.

He created his first game at the age of 11 on the Commodore 64 of his father and has finished several hobby and also commercial game projects since then. His last commercial project was the development of Painkiller:Resurrection, a game published in late 2009, which he supervised and contributed to as technical director in his free time. You can find his tech blog here: http://www.pitorque.de/MisterGoodcat/