OData seems to have a lot of promise in simplifying data access across multiple platforms by using fundamental technologies of HTTP and Atom/JSon. Using an OData source for data consumption/updates in your Windows Phone application?
In this short article, we take a quick look at a data caching strategy in our Windows Phone applications while using an OData data source. While we talk about a Windows Phone solution in particular, these concepts are perfectly applicable when using OData on other mobile platforms.
As always, the demo solution, along with all code samples is available for download through the link below:
Download Source Code
Introduction
The Open Data Protocol (OData) is an HTTP-based protocol for querying and updating a data source. Based on the platform or type of application we are building, we have the option of using either Atom or JSon as our feed protocol of choice and then utilize support for filtering, sorting & paging in our application. Want to learn more? http://www.odata.org/is a great place to start. OData has now built up a nice ecosystem around it’s support, with proxy-building libraries in a variety of platforms; and it obviously lends itself very nicely to usage from mobile applications.
Also, just about any data source that .NET runtime can connect to, can be exposed as an OData feed. It is essentially a two-step process: First – put Entity Framework type Object Relational Mapping (ORM) on top of our data source, in particular the ADO.NET Entity Data Model; and Second – put a WCF Data Service on top of the data entities to expose a read-only/updateable feed for the data. Want to try it yourself? Michael Crump’s “Producing & Consuming OData..” series here is a great start. Want to leverage OData as you augment your mobile application with cloud support? My last two articles here & here should be decent starting points.
The Demo
One of the best ways to learn more about OData is to start playing with a live OData service. And what better example than that of Netfllix .. we all love movies, right? 
Now, Netflix worked with MSFT to expose their entire Movie/Shows catalog as a live OData feed, one that you can hit through the URL below:
http://odata.netflix.com/v2/Catalog/
Once we look up the catalog, we can see all the supported HTTP endpoints to get various information. Head over to the official Netlfix documentation here to learn more about the supported Metadata, filters & formats. For our demo purpose, we shall choose to do a very simple lookup against the Netflix OData catalog .. just to get a list of Genres. Take a guess how many Genres Netflix knows? – Close to 400 !!
So, we make a simple HTTP Get request through our browser to the following URI to get a list of Genres, in default Atom format. Since this is not huge amount of data, there is no continuation token & we get the whole catalog of Genres back in one request:
http://odata.netflix.com/v2/Catalog/Genres
So, let’s build a simple Windows Phone application which consumes the Netflix OData feed & pulls up the list of Genres programmatically. We begin with the default template of a Windows Phone application with a portrait page. One of the first things we do is to add a reference to the root of the Netflix OData service, so that the proxies are built & we can refer to the Netflix entities in our code. Here’s how & the resulting project structure:
Next, let’s add some code to our MainPage.xaml’s code-behind file so we can load the list of Netflix Genres:
1: using NetflixODataService;
2:
3: DataServiceCollection<Genre> Genres { get; set; }
4: DataServiceContext context = new DataServiceContext(new Uri("http://odata.netflix.com/v2/Catalog/"));
5:
6: public MainPage()
7: {
8: InitializeComponent();
9: this.LoadAndStoreGenres();
10: }
11:
12:
13: public void LoadAndStoreGenres()
14: {
15: // Instantiates our collection.
16: this.Genres = new DataServiceCollection<Genre>(context);
17:
18: // Base HTTP endpoint or custom query goes here.
19: Uri uriQuery = new Uri("/Genres()", UriKind.Relative);
20:
21: // Asynchronously load the result of the query.
22: this.Genres.LoadAsync(uriQuery);
23:
24: this.Genres.LoadCompleted += (sender, args) =>
25: {
26: if (args.Error == null)
27: {
28: // Databind.
29: this.genreList.ItemsSource = this.Genres;
30: };
31: }
Now, let’s dig in, since a few interesting things happened. First, we added the reference to the Netflix Service namespace and utilized a special collection class called DataServiceCollection. A DataServiceCollection<T> “represents a dynamic entity collection that provides notifications when items get added, removed, or when the list is refreshed.” So essentially, this has the features of an ObservableCollection built-in & the INotifyPropertyChanged interface makes sure that any UI bound to such a collection is informed of changes on bound property values in the collection. In addition, the DataServiceCollection has been optimized to be the representation of an OData HTTP endpoint collection and supports features like query collections, continuation tokens & tracking changes in the collection. Find out more about the DataServiceCollection here.
Also, we used a DataServiceContext which pointed to the root of the Netflix OData feed as the base context, beyond which we can do our queries/filtering/sorting. Also, once we knew exactly the query to request (in our simple demo – “/Genres()”), we used a LoadAsync() method off of our DataServiceCollection. This, behind the scenes, wraps up our query into an HTTP request to the OData service and does so without blocking the all-important UI thread. When the remote server responds, a LoadCompleted event triggers to allow us to handle the response & bind to our UI, thus automatically marshaling data back to the UI thread.
Now, how do we display the list of Genres we are fetching? Through a very simple XAML UI as below:
1: <ListBox Margin="15,0,15,10" x:Name="genreList">
2: <ListBox.ItemTemplate>
3: <DataTemplate>
4: <StackPanel Margin="0,0,0,17" Width="432">
5: <TextBlock Text="{Binding Name}" TextWrapping="Wrap" Style="{StaticResource PhoneTextExtraLargeStyle}"/>
6: </StackPanel>
7: </DataTemplate>
8: </ListBox.ItemTemplate>
9: </ListBox>
So, we simply want to display a list of Genres as “Name” as we get back from Netflix. With just the little code above added to our MainPage.xaml page, we can have our application fetch & display Genres as follows. Please note that is for demo only .. so no fancy progress bars, which you might want to add to indicate to the user that some web request is being served.
The Problem
Now, consider this .. let’s say the source of the OData feed does not change very often or you want this demo Netflix application to work in offline mode, after the data has been fetched at least once. Both of these indicate that we should figure out a way to not make the HTTP Get request to the OData source every time! So, the first time we fetch the data, we have to cache it in the application’s Isolated Storage, to be read on subsequence application runs until something triggers a refresh of the data feed.
So, common sense suggests that we try to put the whole Genres DataServiceCollection in Isolated Storage & rehydrate our entity objects on successive needs for the data. Here’s the problem though – you will get strange error if you simply try to put the DataServiceCollection in Isolated Storage. Something along the lines of :
"An item could not be added to the collection. When items in a DataServiceCollection are tracked by the DataServiceContext, new items cannot be added before items have been loaded into the collection."
So, what’s happening? The problem is that DataServiceCollection<T> was not designed for serialization/deserialization. When we use the DataServiceCollection, the collection is responsible for tracking all the objects that it contains. Because the collection also has a reference to the DataServiceContext, it notifies the DataServiceContext of all changes to the collection and the entities in the collection. This means that we can just add,remove and update objects in the collection and when we want to persist the changes, we call SaveChanges() on the DataServiceContext. This is the crux of the problem, since Isolated Storage requires the entities to be serialized, which triggers the object being updated events. If we create a new DataServiceCollection<T>() (constructor without parameters), it creates the collection in so called auto-tracking mode. That is the collection has tracking turned on, and in order for tracking to work correctly, we must call Load on it (this will add items and connect the collection to the right DataServiceContext which will track the items). This behavior is by design.
Oops! So, is there no way to store away a DataServiceCollection for re-use? Thankfully – Yes!
The Solution
A new class called DataServiceState has been added to keep track of the state of a DataServiceCollection & it’s backend DataServiceContext. The DataServiceState simply knows how to serialize/deserialize a typed DataServiceContext with one or more DataServiceCollection objects. Essentially, during serialization through an explicit static Serialize() method, the whole DataServiceCollection is flattened into a string that is the XML serialized representation of the stored objects, along with support for nested collections.
Shall we see this in action? In our case, we have a DataServiceCollection<Genre> that we would serialize & store in Isolated Storage for future use. Also, we would add a boolean flag to indicate whether we have cached data vs when we would have to refetch. So, in our case, the moment we receive a response to our query to fetch Genres, we would serialize the collection into Isolated Storage. You may switch around where you do this for cleaner code .. but here’s the demo code that shows serialization:
1: public partial class MainPage : PhoneApplicationPage
2: {
3: #region "Data Members"
4:
5: DataServiceCollection<Genre> Genres { get; set; }
6: DataServiceContext context = new DataServiceContext(new Uri("http://odata.netflix.com/v2/Catalog/"));
7:
8: public bool AppHasFeed { get; set; }
9:
10: #endregion
11:
12: #region "Constructor"
13:
14: public MainPage()
15: {
16: InitializeComponent();
17:
18: if (IsolatedStorageSettings.ApplicationSettings.Contains("AppHasFeed"))
19: {
20: AppHasFeed = (bool)IsolatedStorageSettings.ApplicationSettings["AppHasFeed"];
21: }
22:
23: if (!AppHasFeed)
24: {
25: this.LoadAndStoreGenres();
26: }
27: else
28: {
29: this.HydrateFromIso();
30: }
31: }
32:
33: #endregion
34:
35: #region "Methods"
36:
37: public void LoadAndStoreGenres()
38: {
39: // Instantiates our collection.
40: this.Genres = new DataServiceCollection<Genre>(context);
41:
42: // Base HTTP endpoint or custom query goes here.
43: Uri uriQuery = new Uri("/Genres()", UriKind.Relative);
44:
45: // Asynchronously load the result of the query.
46: this.Genres.LoadAsync(uriQuery);
47:
48: this.Genres.LoadCompleted += (sender, args) =>
49: {
50: if (args.Error == null)
51: {
52: // Databind.
53: this.genreList.ItemsSource = this.Genres;
54:
55: // Set & store the flag indicating we have fetched data once.
56: AppHasFeed = true;
57:
58: if (IsolatedStorageSettings.ApplicationSettings.Contains("AppHasFeed"))
59: IsolatedStorageSettings.ApplicationSettings.Remove("AppHasFeed");
60:
61: IsolatedStorageSettings.ApplicationSettings.Add("AppHasFeed", AppHasFeed);
62:
63:
64: // Serialize our collection.
65: var GenresCollection = new Dictionary<string, object>();
66: GenresCollection["Genres"] = this.Genres;
67: string GenresDataFeed = DataServiceState.Serialize(context, GenresCollection);
68:
69: // Store flattened data in Isolated Storage.
70: if (IsolatedStorageSettings.ApplicationSettings.Contains("Genres"))
71: IsolatedStorageSettings.ApplicationSettings.Remove("Genres");
72:
73: IsolatedStorageSettings.ApplicationSettings.Add("Genres", GenresDataFeed);
74: }
75: };
76: }
77: #endregion
78: }
So, our boolean flag “AppHasFeed” indicates whether we have fetched the Genre collection at least once .. in your actual use, you may choose to implement an explicit way to refresh the data feed. Back in our case, if we don’t have any data, we make the one-time asynchronous request to fetch OData as Atom. Next, we utilize the DataServiceState class to serialize the entire DataServiceCollection of Genres with respect to its DataServiceContext and store the resulting string XML into Isolated Storage as a key-value pair. Voila, now we are storing state and caching OData inside our application!
One thing to note is that in our demo case, we simply put the whole DataServiceCollection in Isolated Storage for future use. If you do not want that level of persistence & are dealing with temporary data, there is a way out as well. The same techniques of serialization of a DataServiceCollection apply perfectly well when storing the resulting string in PhoneApplicationService.Current.State, in case you simply want the data to be available in case your application is tombstoned & you want you data entities to be rehydrated without making a fresh HTTP request.
So, in the code above, we saw how to serialize & store a DataServiceCollection in Isolated Storage. But we also need the same data to be deserialized & re-hydrating our data entities if we are not making another fresh request. So, in our case, if the AppHasFeed flag indicates that we have fetched data already, we simply use the following deserialization technique:
1: public void HydrateFromIso()
2: {
3: if (IsolatedStorageSettings.ApplicationSettings.Contains("Genres"))
4: {
5: // Fetch flattened data & rehydrate object model.
6: string GenresDataFeed = IsolatedStorageSettings.ApplicationSettings["Genres"].ToString();
7: DataServiceState state = DataServiceState.Deserialize(GenresDataFeed);
8: Dictionary<string, object> collections = state.RootCollections;
9:
10: // Hydrate our DataServiceCollection.
11: this.Genres = collections["Genres"] as DataServiceCollection<Genre>;
12:
13: // Databind.
14: this.genreList.ItemsSource = this.Genres;
15: }
16: }
So essentially, we deserialize the string XML from Isolated Storage through the static Deserialize() method to re-instantiate a DataServiceState. Once done, the RootCollections property holds the individual DataServiceCollection entities, which we can use to re-hydrate our data model. Easy? Again, where I do this in code is questionable, but this is for the sake of keeping the demo simple. You should be able to take the concept & add code to your application as & when needed to cache/uncache OData collections.
So, after you add the above code, go ahead & run the Netflix Demo again, but this time after turning off all network connectivity for emulator or Windows Phone device. You’ll see that we have been able to serialize/deserialize the fetched OData collection and now our applications works perfectly fine in offline mode, on successive attempts!
Summary
In this article, we talked about consumption of OData from a Windows Phone application and the subsequent serialization/deserialization of data collections. These techniques should help in building lean applications which are careful about misuse of the user’s bandwidth in fetching the same data over & over again. So, what are you waiting for? Expose just about any data store as an OData feed, put appropriate security on the data & consume it responsibly from multiple mobile platforms!
I would appreciate any comments or concerns or how things can be done better. Thanks for reading & happy coding.
Cheers SilverlightShow!
About the Author
Samidip Basu (@samidip) is a technologist & gadget-lover working as a Manager & Solutions Lead for Sogeti out of the Columbus Unit. Having worked on WP7 since CTP days, he now spends much of his time in spreading the word to discover the full potential of the Windows Phone platform & cloud-based mobile solutions in general. He passionately runs the Central Ohio Windows Phone User Group (http://cowpug.org/), labors in M3 Conf (http://m3conf.com//) organization and can be found with at-least a couple of hobbyist projects at any time. His spare times call for travel and culinary adventures with the wife. Find out more at http://samidipbasu.com/.