This article is compatible with the latest version of Silverlight.
This article is Part 2 of the series Deep Dive Into WCF.
1. Introduction
This is the second article of my series about Silverlight web services tricks. At first time I planned to write only about WCF and that is why I called my article series “Deep dive into WCF”, but with my last experience I decided to move away from mainstream and offer some new unusual stuff. That is why I have generalized name of this series to “Deep dive into Silverlight services”.
In this article I’m going to show how you can build a super fast REST web services using ASP.NET MVC and Google Protocol Buffers library.
Download source code (Requires Silverlight 3 RTM).
2. Content
2.1. When speed and size matters..
At the beginning everything is always cool... You are the only guy who uses your service… Server is located nearby you and all your services respond like a lightning bolt.
But when your web services start working in the web and thousands of people start using them every small thing matters.
One of such things are serialization speed and size.
Well known leader of fast and reliable web services is Google. And how fortunate that this giant has shared some secretes of its success with community. I’m talking about its communication protocol called protocol buffers, which provide super fast and high compressed binary serialization. Official google tutorial says that protocol buffers:
- are 3 to 10 times smaller.
- are 10 to 100 times faster.
Also you could look at Marc Gravell performance tests that prove the Google’s words.
Currently Google are not provide any official version for .Net and Silverlight, but there is at least two open source .NET ports, which worth to check out:
protobuf-csharp-port by Jon Skeet
protobuf-net by Marc Gravell
Jon Skeet’s version is close port to original Java version and Marc’s version is attempt to integrate PB with WCF.
For my experiments I’ve chosen protobuf-csharp-port, because from my point of view it is more stable and easy to use due to its API is close to well documented Java API. Also I have tired from WCF factories and proxies, which are very hard to setup =)
2.2. Four steps to start using protocol buffers.
1. The first step is to create .proto file where all you data transfer objects are described.
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
2. The second is to compile the message with the normal Protocol Buffer compiler (
protoc) to parse the file and generate a binary representation. For the build-in sample messages it would look something like this:
protoc --descriptor_set_out=addressbook.protobin --proto_path=protos --include_imports protos\tutorial\addressbook.proto
3. After that you should use ProtoGen on this binary representation to generate source code of data transfer objects.
protogen addressbook.protobin
4. Now you can use generated classes to parse stream with serialized data.
var person = Person.ParseFrom(stream);
DoSomething(person);
All generated classes are read only objects. To create new instance of data transfer object you should you use its builder:
Person.Builder personBuilder = Person.CreateBuilder();
personBuilder.Id = 1;
personBuilder.Name = "Bob";
Person person = personBuilder.Build();
2.3. Building protocol buffer REST services with ASP.NET MVC
As you may be already know ASP.NET MVC controllers can produce any kind of data and generation of html views is not the only way to use them. Generally speaking all MVC controllers are REST services, because they are providing resources (HTML views, JSON entities, Atom feeds, files) based on uri pattern. In our case they will provide stream data produced by protocol buffers data transfer objects.
The first thing we should do is a custom ActionResult, which will write our objects to output stream and will set content type to “application/x-protobuf”, which will inform clients of our services that they will deal with protocol buffers.
I called it ProtoBufResult.
public class ProtoBufResult : ActionResult
{
private IMessage message;
public ProtoBufResult(IMessage message)
{
this.message = message;
}
public override void ExecuteResult(ControllerContext context)
{
context.HttpContext.Response.ContentType = "application/x-protobuf";
message.WriteTo(context.HttpContext.Response.OutputStream);
}
}
That is all we need to create our first service. Service will perform 3 operation:
1. GetPerson - gets person by its identifier.
2. GetAllPersons - list all persons of address book.
3. UploadPerson - upload person from client side.
Lets call this service AddressBookController. For simplicity reasons I don’t want to deal with any kind of data access stuff, that is why all results would be generated in memory.
[AcceptVerbs(HttpVerbs.Get)]
public ActionResult GetPerson(int id)
{
Person.Builder personBuilder = Person.CreateBuilder();
personBuilder.Id = id;
personBuilder.Name = "Bob";
Person person = personBuilder.Build();
return new ProtoBufResult(person);
}
[AcceptVerbs(HttpVerbs.Get)]
public ActionResult GetAddressBook()
{
AddressBook.Builder addressBookBuilder = AddressBook.CreateBuilder();
for (int i = 0; i < 10; ++i)
{
Person.Builder personBuilder = Person.CreateBuilder();
personBuilder.Name = "Person #" + i;
personBuilder.Id = i;
Person person = personBuilder.Build();
addressBookBuilder.AddPerson(person);
}
return new ProtoBufResult(addressBookBuilder.Build());
}
Creating of get methods is straightforward. You simply put you proto objects to action result and that is all.
public ActionResult GetPerson(int id)
{
Person.Builder personBuilder = Person.CreateBuilder();
personBuilder.Id = id;
personBuilder.Name = "Bob";
Person person = personBuilder.Build();
return new ProtoBufResult(person);
}
To get stream with uploaded entity you should get requested input stream and ensure that its content-type is “application/x-protobuf”. For this purpose I have created following extension method:
public static class HttpRequestBaseExtensions
{
public static Stream GetProtoBufStream(this HttpRequestBase request)
{
if (!string.IsNullOrEmpty(request.ContentType) && request.ContentType == "application/x-protobuf")
{
return request.InputStream;
}
return null;
}
}
Also we need to inform our client if operation was performed successfully. For this purpose I have add another proto called OperationResult. It has the only property Error, which is string with error text. Null value of error property tells client that operation was performed successfully. This object will be returned as ProtoBufResult, which was used in GetPerson and GetAllPerson methods.
Here is the source code of created Operation result proto.
message OperationResult {
required string error = 1;
}
That is all we need to create UploadPerson method.
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult UploadPerson()
{
var stream = this.Request.GetProtoBufStream();
var resultBuilder = OperationResult.CreateBuilder();
if(stream != null)
{
try
{
var person = Person.ParseFrom(stream);
// Some logic.
}
catch(Exception e)
{
resultBuilder.Error = e.Message;
}
finally
{
stream.Close();
}
}
return new ProtoBufResult(resultBuilder.Build());
}
2.4. Building protocol buffer REST services client for Silverlight
Before we start creating service client we should share generated proto entities between .NET and Silverlight parts of our application. It can be archived by creating of Silverlight class library, where all proto related files will be added as link. Also you can use Microsoft Project Linker automate this process.
For building Silverlight service client we will use WebRequest object.
Let’s try to call and parse the result of GetPerson action.
[TestMethod]
[Asynchronous]
public void GetPerson()
{
WebRequest request = WebRequestCreator.ClientHttp.Create(
new Uri(Application.Current.Host.Source, @"../AddressBook/GetPerson/1"));
request.BeginGetResponse(
ar =>
{
WebResponse response = request.EndGetResponse(ar);
Person person = Person.ParseFrom(response.GetResponseStream());
Assert.IsNotNull(person);
EnqueueTestComplete();
}, null);
}
Also I want to take you attention to the Uri, which I have constructed: new Uri(Application.Current.Host.Source, @"../AddressBook/GetPerson/1"). Using of Application.Current.Host.Source allows us to create server address independent Uri.
Calling of UploadPerson is a bit harder. You should perform three steps to achieve that:
1. Set request method (POST) and set content-type to “application/x-protobuf” to inform server that you are sending protocol buffer stream.
2. Create request stream.
3. Get operation result.
[TestMethod]
[Asynchronous]
public void UploadPerson()
{
WebRequest request =
WebRequestCreator.ClientHttp.Create(new Uri(Application.Current.Host.Source,
@"../AddressBook/UploadPerson"));
request.Method = "POST";
request.ContentType = "application/x-protobuf";
request.BeginGetRequestStream(
ar =>
{
using (Stream requestStream = request.EndGetRequestStream(ar))
{
Person.Builder personBuilder = Person.CreateBuilder();
personBuilder.Id = 1;
personBuilder.Name = "Bob";
Person person = personBuilder.Build();
person.WriteTo(requestStream);
}
request.BeginGetResponse(
ar2 =>
{
using(var resultStream = request.EndGetResponse(ar2).GetResponseStream())
{
var operationResult = OperationResult.ParseFrom(resultStream);
}
EnqueueTestComplete();
}, null);
}, null);
}
In this tests I’ve showed direct using of web request object. However you should encapsulate all this logic and provide your teammates clear service client class. Due to WebRequest is working in asynchronous mode you should implement APM (Asynchronous Programming Model) model in your service client class.
There is two main approaches in .NET.
1. Event-based APM. It is used in generated WCF service client.
2. BeginXXX and EndXXX method. It is used in ADO.NET Data Services client.
I prefer the second. Many people afraid to implement it, because find it very hard task. And this is true. Creating of BeginXXX and EndXXX by your own power require a lot of knowledge. However Jeffrey Richter (author of a great book CLR via C#) solve this problem with his Power Threading Library.
In this library he introduce AsyncResult and AsyncResult<T> objects, which hides all “async magic”. AsyncResult is used, when EndXXX returns void and AsyncResult<T> when EndXX returns T.
When you have finished all async work inside you BeginXXX method, you should call action result SetAsCompleted with result or thrown error.
Here is the source code of BeginGetPerson and EndGetPerson methods.
public IAsyncResult BeginGetPerson(int id, AsyncCallback asyncCallback, Object userState)
{
var asyncResult = new AsyncResult<Person>(asyncCallback, userState);
WebRequest request = WebRequestCreator.ClientHttp.Create(
new Uri(
Application.Current.Host.Source,
string.Format(@"../AddressBook/GetPerson/{0}", id)
)
);
request.BeginGetResponse(
ar =>
{
try
{
WebResponse response = request.EndGetResponse(ar);
Person person = Person.ParseFrom(response.GetResponseStream());
asyncResult.SetAsCompleted(person, false);
}
catch (Exception e)
{
asyncResult.SetAsCompleted(e, false);
}
}, null);
return asyncResult;
}
public Person EndGetPerson(IAsyncResult asyncResult)
{
var result = asyncResult as AsyncResult<Person>;
return result.EndInvoke();
}
Now you can use this silverlight service client in you application:
[TestMethod]
[Asynchronous]
public void GetPerson()
{
var serviceClient = new AddressBookServiceClient();
serviceClient.BeginGetPerson(1,
ar =>
{
Person person = serviceClient.EndGetPerson(ar);
Assert.IsNotNull(person);
EnqueueTestComplete();
}, null);
}
3. Summary
Protocol buffers is a powerful toy, which can greatly increase speed and bandwidth of your rest services. Also this approach is very transparent and you won’t waste you time with configuration of some magic properties, which WCF usually require.
I hope you have enjoyed the article. If you have any questions or you have found a bug, please write a comment or e-mail me through my blog contact form.
Also I’d like to say thanks to Jon Skeet, who helped me with my first steps in protocol buffers.
4. Links
Here is some useful links, which was used while preparing of this article:
1. Protocol Buffers Google project
2. Protocol Buffers .NET port project
3. Concurrent Affairs: Implementing the CLR Asynchronous Programming Model by Jeffrey Richter
4. Create REST API using ASP.NET MVC that speaks both Json and plain Xml by Omar Al Zabir
5. Jon Skeet’s blog – author of protocol buffers net blog.