Are you interested in
creating games for Windows Phone 7 but don't have XNA experience?
Watch the recording of the recent intro webinar delivered by Peter Kuhn '
XNA for Windows Phone 7'.
This article is part 6 of the series "XNA for Silverlight developers".
This article is compatible with Windows Phone "Mango".
Latest update: Aug 1, 2011.
Last time we learned about touch input and gestures, which probably is the first choice for user input in most games on the phone. However, the hardware of Windows Phone 7 devices offer more possibilities in the form of additional sensors that can be used for input, in particular the accelerometer. Starting with "Mango", this will even be extended to sensors like the compass and gyroscope (if supported by the phone's hardware).
For example, Microsoft's game "Kombo" that was available on the phone very early proved that using the accelerometer as primary input for a game is very well possible (give it a try, it's free). In this part of the series we will learn how you can use this type of data for input and take a peek at advanced topics and further resources. As always, you can download the source code at the end of the article.
Preface
With the accelerometer, for the first time in this series we are in the situation that we're working with something XNA does not have built-in support for. This is the reason there won't be a separate inspection of how you would do the same in Silverlight in this article, because it really is identical. For the XNA developer the result is that for once they got to work with events in conjunction with input data and can enjoy the comfort of being notified instead of having to poll.
Theory...
When you read values from the accelerometer, you will receive X, Y and Z values that correspond to the the following axes:
That means: if you tilt the phone so the top edge comes up (like in the drawing), you will receive negative Y values. If you tilt it so the edge that holds the buttons comes up, the Y will become positive. Tilt it to the right, so the left side comes up, and you will receive positive X values, and finally if you tilt it to the left so the right side comes up, the X values will become negative. The Z axis provides negative values when the phone's front is facing up, and the values will be positive if you flip the phone so the front is facing down.
All those values are provided as G-forces, which means that a value of 1.0 equals 9.81 m/s². This is the conventional standard value for gravitational acceleration on Earth (you can read more about it on Wikipedia if you're are not familiar with this). Ideally, if your phone lies flat on a leveled surface, you would receive values of zero for both X and Y, and -1.0 for the Z value. If the phone stands on its left edge, you would receive X = -1.0 and Y and Z both as zero. And finally if it stands on the edge where the buttons sit, the values should be Y = -1.0 and zero for both X and Z.
... and practice
In practice, you will see that the accelerometer is a very sensitive source of input. Even the slightest change produces instant reaction, and due to the nature of the sensor hardware and physics, even if the phone is not moved at all it is likely that you will see a constant noise level of changing values.
In addition, the reported values also depend on your location, in particular your current location's latitude (because the Earth is not a sphere but an ellipsoid) and the height above sea level. You'll most likely never see the ideal steady value of (-)1.0. For example, when I lie my phone down on what seems a perfectly leveled table, I see a Z value that constantly jumps between -0.97 and -0.99. Both the X and Y values jump equally in the range of 0.02-0.03. This is something you must accommodate for in your applications and games, for example by using a threshold or smoothing of the values (see below).
Setup
To use the accelerometer, an additional assembly reference is needed for your project, because it is not included by default when you create a new game solution. In particular, Microsoft.Devices.Sensors is required:
The next step is to create an instance of the Accelerometer class that can be found in the same namespace. This class has a single event ReadingChanged that is raised when new data is available. This event is raised 50 times a second, so it produces a lot of data. Because of this there's also a mechanism to turn reporting of the input data on and off, so you have the possibility to only activate it when you really need it. For example, if your game is paused or currently displaying a menu screen or similar, you should pause the accelerometer input. This can be done using the Start and Stop methods of the class.
The whole setup then looks like this:
1: // fields for the accelerometer
2: private Accelerometer _accelerometer;
3: private Vector3 _lastAccelerometerData;
4:
5: ...
6:
7: // create the accelerometer instance,
8: // e.g. in your game's constructor;
9: // hook the ReadingChanged event
10: _accelerometer = new Accelerometer();
11: _accelerometer.ReadingChanged +=
12: new EventHandler<AccelerometerReadingEventArgs>(Accelerometer_ReadingChanged);
13:
14: ...
15:
16: private void Accelerometer_ReadingChanged(object sender, AccelerometerReadingEventArgs e)
17: {
18: // simply store the reading as new vector
19: _lastAccelerometerData = new Vector3((float)e.X, (float)e.Y, (float)e.Z);
20: }
21:
22: ...
23:
24: // for example in your game's Initialize method,
25: // start the accelerometer data collection
26: try
27: {
28: _accelerometer.Start();
29: }
30: catch (Exception)
31: {
32: // this can fail if the sensor is not available etc.
33: }
Aside from the X, Y and Z values, the AccelerometerReadingEventArgs also have a property "Timestamp" that is a DateTimeOffset and indicates the time at which the reading was taken. This can be of interest if you are analyzing the change of these values over time instead of taking a momentarily snapshot.
Some important notes about "Mango"
The above mentioned ReadingChanged event is marked as obsolete in the Windows Phone "Mango" release. A new event named CurrentValueChanged has been added to replace the old one. The reason for this change is that the class hierarchy for sensor classes has been changed by the update. In the RTM release of Windows Phone, the accelerometer was a stand-alone class with no base class. In Mango, we now have a generic SensorBase class that is the base class for all other sensor classes of the phone (also the compass, gyroscope etc.). The new hierarchy looks like this:
- public abstract class SensorBase<TSensorReading> : IDisposable where TSensorReading : ISensorReading
- public sealed class Accelerometer : SensorBase<AccelerometerReading>
- public sealed class Compass : SensorBase<CompassReading>
- public sealed class Gyroscope : SensorBase<GyroscopeReading>
- public sealed class Motion : SensorBase<MotionReading>
The SensorBase class now is the place that provides general methods, properties and an event for the data retrieval, in particular:
1: virtual void Start();
2: virtual void Stop();
3: TSensorReading CurrentValue { get; }
4: bool IsDataValid { get; }
5: TimeSpan TimeBetweenUpdates { set; get; }
6: event EventHandler<SensorReadingEventArgs<TSensorReading>> CurrentValueChanged
In this process of refactoring, the existing accelerometer API has been streamlined to fit these changes. Make sure you start using the new event when you target the Mango release with your game or application. Consequently, the received arguments for this event also are slightly different too. But they still provide the same data as in the RTM release; after all, the nature of the sensor hasn't changed.
Another thing that is worth noting is the above listed TimeBetweenUpdates property of the SensorBase class. In Mango you have the possibility to influence the interval at which values are reported by changing that property. A limitation of this is that there might be differences in the capabilities of different sensors and devices. Your desired interval may not be possible to achieve, and the actual value used may be different from what you set this property to. You can query the property after you set an interval to see if changing the value actually worked.
Making use of the data
In this short example, I want to move a ball around the phone screen by using the accelerometer input. To achieve that, I add the X and Y values from the sensor to the current velocity of the ball. To work around the problem of the basic noise level that is always present even when the phone lies still, I'm using a threshold the values need to exceed to actually be considered for processing. For all this, I declare the following fields:
1: // the ball and its location + velocity
2: private Texture2D _ball;
3: private Vector2 _location;
4: private Vector2 _velocity;
5:
6: // some limits to keep the ball on screen
7: // and to not let it become too fast
8: private Vector2 _ballMaxLocation;
9: private Vector2 _ballMaxVelocity;
10:
11: // the threshold for the accelerometer readings
12: // all values lower than that are ignored
13: private const float Threshold = 0.1f;
The limit for the location of course are the screen bounds, and for the maximum velocity I simply chose an arbitrary value. Now let's take a look at the "Update" method of the game that simply takes the last sample from the accelerometer to modify the velocity:
1: // update the ball's velocity with the accelerometer values
2: if (Math.Abs(_lastAccelerometerData.Y) > Threshold)
3: {
4: _velocity.X -= _lastAccelerometerData.Y;
5: }
6:
7: if (Math.Abs(_lastAccelerometerData.X) > Threshold)
8: {
9: _velocity.Y -= _lastAccelerometerData.X;
10: }
11:
12: // limit the velocity to the maximum
13: _velocity = Vector2.Clamp(_velocity, -_ballMaxVelocity, _ballMaxVelocity);
14:
15: // add some "friction" to the ball's movement
16: _velocity = _velocity * 0.95f;
There's something really important to note about the first lines of code here. If you inspect them carefully, you can see that I'm subtracting the Y value of the accelerometer data from the X value of the velocity, and similarly use the X value for the Y component.
The reason for this is that in the sample I'm using landscape orientation (LandscapeLeft to be more precise). As my initial graphic shows, the accelerometer coordinate system uses a certain alignment of the axes relative to the device. Unlike the coordinate system of your game, which actually is aligned with respect to the current screen orientation, not the hardware of the phone, the accelerometer's coordinate system is fixed and always stays the same. This means that as long as you are using portrait mode your game's coordinate system axes and the accelerometer's axes are aligned in the the same way, but when you switch to landscape mode, you actually have to swap X and Y coordinates. Even more, you also have to perform a reversal of the sign on the retrieved values, depending on whether you are in left or right landscape mode.
A potentially cleaner solution probably would be to make this distinction in the reading changed event handler already. You could encapsulate the reading in a helper class or similar and swap the values dynamically depending on the current orientation there. That way you'd have a single point where this logic is implemented and could work with consistent values throughout the rest of your application to avoid confusion.
The last line of the above update code adds an artificial slowdown of the ball over time to simulate friction as it moves over the virtual surface. It's just an attempt to add a bit of realism to the behavior, because without that the ball would feel like moving on ice or a mirror.
Let's take a look at the rest of the Update method:
1: // clamp the location
2: var clampedLocation = Vector2.Clamp(_location, Vector2.Zero, _ballMaxLocation);
3: if (clampedLocation != _location)
4: {
5: // reset the velocity if the ball has hit a wall
6: if (clampedLocation.X != _location.X)
7: {
8: _velocity.X = 0.0f;
9: }
10:
11: if (clampedLocation.Y != _location.Y)
12: {
13: _velocity.Y = 0.0f;
14: }
15:
16: // set the corrected location
17: _location = clampedLocation;
18: }
All we do here is make sure the ball doesn't move off-screen and reset the velocity if any of the edges is hit.
The "Draw method" draws the ball at the current location and also outputs the last accelerometer readings as well as the ball's velocity to visualize how everything works together.
1: _spriteBatch.Begin();
2:
3: // draw ball
4: _spriteBatch.Draw(_ball, _location, Color.White);
5:
6: // draw readings
7: string readings = string.Format("Accelerometer readings: X:{0}, Y:{1}, Z:{2}",
8: _lastAccelerometerData.X.ToString("0.00"),
9: _lastAccelerometerData.Y.ToString("0.00"),
10: _lastAccelerometerData.Z.ToString("0.00"));
11: _spriteBatch.DrawString(_font, readings, Vector2.Zero, Color.White);
12:
13: // draw velocity
14: Vector2 readingsSize = _font.MeasureString(readings);
15: string velocity = string.Format("Ball velocity: X:{0}, Y:{1}",
16: _velocity.X.ToString("0.00"),
17: _velocity.Y.ToString("0.00"));
18: _spriteBatch.DrawString(_font, velocity, new Vector2(0.0f, readingsSize.Y), Color.White);
19:
20: _spriteBatch.End();
And here is what the final result looks like on the device:
Possible problems and pitfalls
The emulator
When you are working with the RTM release of Windows Phone, the sensor is not present in the emulator. In that case you are either forced to use a real device whenever you want to test an application or game, or revert to a custom solution to simulate the sensor, for example by mapping some keys to accelerometer input. Over time people also came up with a lot of clever external tools for this problem, including hardware solutions that make use of other accelerometer devices and route the data to the phone. There are some software projects available that use more comfortable client applications to simulate the accelerometer input and let the phone retrieve that simulated input through a web service or similar. One of these projects is the WP7 Accelerometer Emulator project, and a similar one can be found on Codeplex as the Simulator Kit. These solutions can speed up and simplify your development process if you don't have access to a real device all the time or don't want to plug it in for each test.
If you are developing for Windows Phone "Mango", things get a lot more easier and you don't have to use third-party solutions anymore. The improved emulator of the new development tools now has built-in support for the accelerometer and allows you to easily simulate and generate that kind of input data in a nice way:
As you can see a visual representation of the phone is used, and by moving around the red dot you can change the orientation of the device in space, which produces accelerometer data accordingly, without the need to change anything in your code. That is a comfortable and easy way to test your code in the emulator too:
Data noise
As I've mentioned multiple times, the data received from the sensor is really noisy. Here is a sample reading of me tilting the phone to the left, first slowly, then in a sudden movement and back:
In our example we could accommodate for that by using a simple threshold that filtered all the "ambient noise". Once the threshold was exceeded, the noise didn't hurt anymore. However, in other scenarios you might want to react to changes of the sensor values over time rather than process the momentarily values as we did here. This often requires to determine a trend in these values that ignores temporary spikes, for example. In these cases you will most likely want to perform some smoothing on the data, which in turn can introduce a new set of problems, like latency. Luckily you don't have to fight through those basics of signal processing from scratch all by yourself again. Dave Edson and Greg Hogdal from the Windows Phone 7 Applications Platform team have written an excellent article on the topic and introduce some helper classes that do the heavy lifting for you.
Screen saver
For power saving reasons, the Windows Phone 7 devices turn off the screen after a while when no user input happens. In this sense, accelerometer input is not considered user input. Obviously the device cannot distinguish between deliberate tilting of the phone in a game and e.g. changes in orientation that happen because the user carries the phone in their pocket or because they gesticulate in a discussion and are not actually using the phone. You can turn off the screen saver using the IsScreenSaverEnabled property of the Guide class to avoid that:
1: // disable automatic turning off
2: // due to no user touch input
3: Guide.IsScreenSaverEnabled = false;
Further accelerometer resources
An interesting sample application with free source is the "Level Starter Kit" that can be found in the education section of the app hub. It has the same helper classes in it as the source code of the article on data smoothing linked above and creates a virtual bubble level to demonstrate the use of the accelerometer.
Another cool library is the recently released Shake Gesture Library that provides a high-level abstraction of "shake flicks" performed by the user. This post on the Windows Phone Developer Blog gives a short introduction and explanation of the concept.
Additional sensors and combined motion API
With Windows Phone "Mango", developers will have access to additional sensors, in particular the compass and gyroscope. Unlike with the accelerometer, which was a requirement for all devices for the RTM release of Windows Phone, not all devices in the market will have a gyroscope, for example. So if you build an application or game that relies on a gyroscope, some users won't be able to use your application (they will receive a warning in the market place when they browse to your application). This is something you need to accommodate for – make sure the sensor you're trying to use is actually available on the current device to avoid errors.
Since the API for sensors has been streamlined in Mango (see above), all of them work in a similar way. For example, making use of the compass very much resembles what has been written above for the accelerometer:
1: // make sure we have a compass
2: if (Compass.IsSupported)
3: {
4: // create a new compass
5: Compass _compass = new Compass();
6:
7: // set the desired interval for value readings
8: _compass.TimeBetweenUpdates = TimeSpan.FromMilliseconds(100.0);
9:
10: // use the new event for value readings
11: _compass.CurrentValueChanged += new EventHandler<SensorReadingEventArgs<CompassReading>>(Compass_CurrentValueChanged);
12:
13: // start reading
14: _compass.Start();
15: }
16:
17: ...
18:
19: private void Compass_CurrentValueChanged(object sender, SensorReadingEventArgs<CompassReading> e)
20: {
21: // the actual data is in the "SensorReading" property,
22: // which is a "CompassReading" object in this case
23: var data = e.SensorReading;
24:
25: // do something with the read data...
26: }
A special feature of the new sensor API is the Motion class. This class combines the data of the compass, the accelerometer and gyroscope to accurately perform an analysis of the device location and its orientation in space. This is a convenient wrapper around some sophisticated math to compute that data, so you don't have to do this manually. The beauty of this is that the Motion class also inherits from the SensorBase class and hence can be used like any other sensor – this makes it very easy to get complete sensor data information on the phone.
Summary
I hope this article has made your mouth water for this alternative input method. Already existing games and applications in the market place that use this technique prove that using it in real world scenarios is not only feasible but can be a valuable addition and even create a completely new user experience. The performance and quality of the returned data even make sophisticated features like the recognition of shake patterns possible. If you have any questions or requests, feel free to leave a comment or contact me directly.
Download source code
About the author
Peter Kuhn aka "Mister Goodcat" has been working in the IT industry for more than ten years. After being a project lead and technical director for several years, developing database and device controlling applications in the field of medical software and bioinformatics as well as for knowledge management solutions, he founded his own business in 2010. Starting in 2001 he jumped onto the .NET wagon very early, and has also used Silverlight for business application development since version 2. Today he uses Silverlight, .NET and ASP.NET as his preferred development platforms and offers training and consulting around these technologies and general software design and development process questions.
He created his first game at the age of 11 on the Commodore 64 of his father and has finished several hobby and also commercial game projects since then. His last commercial project was the development of Painkiller:Resurrection, a game published in late 2009, which he supervised and contributed to as technical director in his free time. You can find his tech blog here: http://www.pitorque.de/MisterGoodcat/