asp.net

Mapping DataTables and DataRows to Objects in C# and .NET

My group regularly uses DataSet, DataTable, and DataRow objects in many of our apps.

(What? Don't look at me like that. These apps are old.)

Anyway, we're simultaneously trying to implement good C# and object-oriented programming principles while maintaining these old apps, so we often end up having to map data from a data set to a C# object. We did this enough times that I and a coworker (we'll call her Marlena) decided to sit down and just make up a new mapping system for use with these DataTable and DataRow objects.

As always with my code-based posts, there's a GitHub project with a full working example app, so check that out too!

One Jump Ahead

So here's a basic problem with mapping from DataSet, DataTable, and DataRow objects: we don't know at compile time what columns and tables exist in the set, so mapping solutions like AutoMapper won't work for this scenario. Our mapping system will have to assume what columns exist. But, in order to make it more reusable, we will make the mapping system return default values for any values which it does not locate.

There's also another, more complex problem: the databases we are acquiring our data from use many different column names to represent the same data. 20 years of different maintainers and little in the way of cohesive naming standards will do that do a database. So, if we needed a person's first name, the different databases might use:

  • first_name
  • firstName
  • fname
  • name_first

This, as might be imagined, makes mapping anything rather difficult. So our system will also need to be able to map from many different column names.

Finally, this system wouldn't be worth much if it couldn't handle collections of objects as well as single objects, so we'll need to allow for that as well.

So, in short, our system needs to:

  1. Map from DataTable and DataRow to objects.
  2. Map from multiple different column names.
  3. Handle mapping to a collection of objects as well as a single object.

We'll need several pieces to accomplish this. But before we can even start building the mapping system, we must first acquire some sample data.

Mine, Mine, Mine

We're going to create some DataSet objects that we can test our system against. In the real world, you would use an actual database, but here (for simplicity's sake) we're just going to manually create some DataSet objects. Here's a sample class which will create two DataSet objects, Priests and Ranchers, each of which use different column names for the same data:

public static class DataSetGenerator  
{
    public static DataSet Priests()
    {
        DataTable priestsDataTable = new DataTable();
        priestsDataTable.Columns.Add(new DataColumn()
        {
            ColumnName = "first_name",
            DataType = typeof(string)
        });
        priestsDataTable.Columns.Add(new DataColumn()
        {
            ColumnName = "last_name",
            DataType = typeof(string)
        });
        priestsDataTable.Columns.Add(new DataColumn()
        {
            ColumnName = "dob",
            DataType = typeof(DateTime)
        });
        priestsDataTable.Columns.Add(new DataColumn()
        {
            ColumnName = "job_title",
            DataType = typeof(string)
        });
        priestsDataTable.Columns.Add(new DataColumn()
        {
            ColumnName = "taken_name",
            DataType = typeof(string)
        });
        priestsDataTable.Columns.Add(new DataColumn()
        {
            ColumnName = "is_american",
            DataType = typeof(string)
        });

        priestsDataTable.Rows.Add(new object[] { "Lenny", "Belardo", new DateTime(1971, 3, 24), "Pontiff", "Pius XIII", "yes" });
        priestsDataTable.Rows.Add(new object[] { "Angelo", "Voiello", new DateTime(1952, 11, 18), "Cardinal Secretary of State", "", "no" });
        priestsDataTable.Rows.Add(new object[] { "Michael", "Spencer", new DateTime(1942, 5, 12), "Archbishop of New York", "", "yes" });
        priestsDataTable.Rows.Add(new object[] { "Sofia", "(Unknown)", new DateTime(1974, 7, 2), "Director of Marketing", "", "no" });
        priestsDataTable.Rows.Add(new object[] { "Bernardo", "Gutierrez", new DateTime(1966, 9, 16), "Master of Ceremonies", "", "no" });

        DataSet priestsDataSet = new DataSet();
        priestsDataSet.Tables.Add(priestsDataTable);

        return priestsDataSet;
    }

    public static DataSet Ranchers()
    {
        DataTable ranchersTable = new DataTable();
        ranchersTable.Columns.Add(new DataColumn()
        {
            ColumnName = "firstName",
            DataType = typeof(string)
        });
        ranchersTable.Columns.Add(new DataColumn()
        {
            ColumnName = "lastName",
            DataType = typeof(string)
        });
        ranchersTable.Columns.Add(new DataColumn()
        {
            ColumnName = "dateOfBirth",
            DataType = typeof(DateTime)
        });
        ranchersTable.Columns.Add(new DataColumn()
        {
            ColumnName = "jobTitle",
            DataType = typeof(string)
        });
        ranchersTable.Columns.Add(new DataColumn()
        {
            ColumnName = "nickName",
            DataType = typeof(string)
        });
        ranchersTable.Columns.Add(new DataColumn()
        {
            ColumnName = "isAmerican",
            DataType = typeof(string)
        });

        ranchersTable.Rows.Add(new object[] { "Colt", "Bennett", new DateTime(1987, 1, 15), "Ranchhand", "", "y" });
        ranchersTable.Rows.Add(new object[] { "Jameson", "Bennett", new DateTime(1984, 10, 10), "Ranchhand", "Rooster", "y" });
        ranchersTable.Rows.Add(new object[] { "Beau", "Bennett", new DateTime(1944, 8, 9), "Rancher", "", "y" });
        ranchersTable.Rows.Add(new object[] { "Margaret", "Bennett", new DateTime(1974, 7, 2), "Bar Owner", "Maggie", "y" });
        ranchersTable.Rows.Add(new object[] { "Abigail", "Phillips", new DateTime(1987, 4, 24), "Teacher", "Abby", "y" });

        DataSet ranchersDataSet = new DataSet();
        ranchersDataSet.Tables.Add(ranchersTable);

        return ranchersDataSet;
    }
}

We'll test our system against this sample data.

Something There

Now we can build our actual mapping solution. First off, we need a way to decide what column names map to object properties. It was Marlena's idea to keep those things together, and so we came up with a class called DataNamesAttribute that looks like this:

[AttributeUsage(AttributeTargets.Property)]
public class DataNamesAttribute : Attribute  
{
    protected List<string> _valueNames { get; set; }

    public List<string> ValueNames
    {
        get
        {
            return _valueNames;
        }
        set
        {
            _valueNames = value;
        }
    }

    public DataNamesAttribute()
    {
        _valueNames = new List<string>();
    }

    public DataNamesAttribute(params string[] valueNames)
    {
        _valueNames = valueNames.ToList();
    }
}

This attribute can then be used (in fact, can only be used, due to the AttributeUsage(AttributeTargets.Property) declaration) on properties of other classes. Let's say we're going to map to a Person class. We would use DataNamesAttribute like so:

public class Person  
{
    [DataNames("first_name", "firstName")]
    public string FirstName { get; set; }

    [DataNames("last_name", "lastName")]
    public string LastName { get; set; }

    [DataNames("dob", "dateOfBirth")]
    public DateTime DateOfBirth { get; set; }

    [DataNames("job_title", "jobTitle")]
    public string JobTitle { get; set; }

    [DataNames("taken_name", "nickName")]
    public string TakenName { get; set; }

    [DataNames("is_american", "isAmerican")]
    public bool IsAmerican { get; set; }
}

Now that we know where the data needs to end up, let's start mapping out the mapper (heh).

Reflection

Our mapper class will be a generic class so that we can map from DataTable or DataRow objects to any kind of object. We'll need two methods to get different kinds of data:

public class DataNamesMapper<TEntity> where TEntity : class, new()  
{
    public TEntity Map(DataRow row) { ... }
    public IEnumerable<TEntity> Map(DataTable table) { ... }
}

Let's start with the Map(DataRow row) method. We need to do three things:

  1. Figure out what columns exist in this row.
  2. Determine if the TEntity we are mapping to has any properties with the same name as any of the columns (aka the Data Names) AND
  3. Map the value from the DataRow to the TEntity.

Here's how we do this, using just a bit of reflection:

public TEntity Map(DataRow row)  
{
    //Step 1 - Get the Column Names
    var columnNames = row.Table.Columns
                               .Cast<DataColumn>()
                               .Select(x => x.ColumnName)
                               .ToList();

    //Step 2 - Get the Property Data Names
    var properties = (typeof(TEntity)).GetProperties()
                                      .Where(x => x.GetCustomAttributes(typeof(DataNamesAttribute), true).Any())
                                      .ToList();

    //Step 3 - Map the data
    TEntity entity = new TEntity();
    foreach (var prop in properties)
    {
        PropertyMapHelper.Map(typeof(TEntity), row, prop, entity);
    }

    return entity;
}

Of course, we also need to handle the other method, the one where we can get a collection of TEntity:

public IEnumerable<TEntity> Map(DataTable table)  
{
    //Step 1 - Get the Column Names
    var columnNames = table.Columns.Cast<DataColumn>().Select(x => x.ColumnName).ToList();

    //Step 2 - Get the Property Data Names
    var properties = (typeof(TEntity)).GetProperties()
                                        .Where(x => x.GetCustomAttributes(typeof(DataNamesAttribute), true).Any())
                                        .ToList();

    //Step 3 - Map the data
    List<TEntity> entities = new List<TEntity>();
    foreach (DataRow row in table.Rows)
    {
        TEntity entity = new TEntity();
        foreach (var prop in properties)
        {
            PropertyMapHelper.Map(typeof(TEntity), row, prop, entity);
        }
        entities.Add(entity);
    }

    return entities;
}

You might be wondering just what the heck the PropertyMapHelper class is. If you are, you might also be about to regret it.

Dig a Little Deeper

The PropertyMapHelper, as suggested by the name, maps values to different primitive types (int, string, DateTime, etc.). Here's that Map() method we saw earlier:

public static void Map(Type type, DataRow row, PropertyInfo prop, object entity)  
{
    List<string> columnNames = AttributeHelper.GetDataNames(type, prop.Name);

    foreach (var columnName in columnNames)
    {
        if (!String.IsNullOrWhiteSpace(columnName) && row.Table.Columns.Contains(columnName))
        {
            var propertyValue = row[columnName];
            if (propertyValue != DBNull.Value)
            {
                ParsePrimitive(prop, entity, row[columnName]);
                break;
            }
        }
    }
}

There are two pieces in this method that we haven't defined yet: the AttributeHelper class and the ParsePrimitive() method. AttributeHelper is a rather simple class that merely gets the list of column names from the DataNamesAttribute:

public static List<string> GetDataNames(Type type, string propertyName)  
{
    var property = type.GetProperty(propertyName).GetCustomAttributes(false).Where(x => x.GetType().Name == "DataNamesAttribute").FirstOrDefault();
    if (property != null)
    {
        return ((DataNamesAttribute)property).ValueNames;
    }
    return new List<string>();
}

The other we need to define in ParsePrimitive(), which as its name suggests will parse the values into primitive types. Essentially what this class does is assign a value to a passed-in property reference (represented by the PropertyInfo class). I'm not going to post the full code on this post (you can see it over on GitHub), so here's a snippet of what this method does:

private static void ParsePrimitive(PropertyInfo prop, object entity, object value)  
{
    if (prop.PropertyType == typeof(string))
    {
        prop.SetValue(entity, value.ToString().Trim(), null);
    }
    else if (prop.PropertyType == typeof(int) || prop.PropertyType == typeof(int?))
    {
        if (value == null)
        {
            prop.SetValue(entity, null, null);
        }
        else
        {
            prop.SetValue(entity, int.Parse(value.ToString()), null);
        }
    }
    ...
}

That's the bottom of the rabbit hole, as it were. Now, we can use the DataSet objects we created earlier and our mapping system to see if we can map this data correctly.

Two Worlds

Here's a quick program that can test our new mapping system:

class Program  
{
    static void Main(string[] args)
    {
        var priestsDataSet = DataSetGenerator.Priests();
        DataNamesMapper<Person> mapper = new DataNamesMapper<Person>();
        List<Person> persons = mapper.Map(priestsDataSet.Tables[0]).ToList();

        var ranchersDataSet = DataSetGenerator.Ranchers();
        persons.AddRange(mapper.Map(ranchersDataSet.Tables[0]));

        foreach (var person in persons)
        {
            Console.WriteLine("First Name: " + person.FirstName + ", Last Name: " + person.LastName
                                + ", Date of Birth: " + person.DateOfBirth.ToShortDateString()
                                + ", Job Title: " + person.JobTitle + ", Nickname: " + person.TakenName
                                + ", Is American: " + person.IsAmerican);
        }

        Console.ReadLine();
    }
}

When we run this app (which you can do too), we will get the following output:

Which is exactly what we want!

(I mean, really, did you expect me to blog about something that didn't work?)

Go the Distance

It concerns me that this system is overly complicated, and I'd happily take suggestions on how to make it more straightforward. While I do like how all we need to do is place the DataNamesAttribute on the correct properties and then call an instance of DataNamesMapper<T>, I feel like the whole thing could be easier somehow. Believe it or not, this version is actually simpler than the one we're actually using in our internal apps.

Also, check out the sample project over on GitHub, fork it, test it, whatever. If it helped you out, or if you can improve it, let me know in the comments!

Finally, extra special bonus points will go to anyone who can figure out a) what the hell those odd section titles are about and b) where I got the sample data from.

Happy Coding!

A Simple Caching Scheme for Web API using Dependency Injection

I use Dependency Injection (DI) quite a bit in my ASP.NET projects, particularly in Web API and MVC web applications. Recently, I had a need to implement a caching layer in one of my MVC apps, and such a layer would be best used if it could be injected into my clients layer (e.g. the layer that called an API and handled responses from the same). The solution I came up with seemed to be pretty simple, so I wanted to share it here.

In this post, we're going to set up a simple MVC project that consumes a Web API and implements a caching layer.

Requirements

For this project, I'm using two of my favorite NuGet packages:

The sample project, which is over on GitHub, is a fully-functional implementation of the strategy described in this post. Feel free to check it out, branch it, download it, whatever!

Setting Up the API

First, let's take a look at our sample API. Here's the class DateNumberObject, which represents a response returned from the API to the web project:

public class DateNumberObject  
{
    public DateTime CurrentDate { get; set; }
    public int RandomNumber { get; set; }

    public DateNumberObject()
    {
        CurrentDate = DateTime.Now;
        Random rand = new Random();
        RandomNumber = rand.Next(1, 200);
    }
}

As you can see, all this class does is return the current date and time and a random number.

We can now build our API, which is also rather simple. Here's a snippet from the API's controller:

[RoutePrefix("samples")]
public class SampleController : ApiController  
{

    [HttpGet]
    [Route("date")]
    public IHttpActionResult GetDateAndNumber()
    {
        return Ok(new DateNumberObject());
    }
}

That's all our API will do: return the current date and a random number. Since there's no caching taking place at the service layer, we will need to implement caching at the web layer.

Speaking of the web layer, in our sample solution it is an MVC5 project, and we will be using my favorite Dependency Injection tool, StructureMap.

Setting up StructureMap

If you've never set up StructureMap in your MVC projects, you'll want to read this section; otherwise, skip to the next section.

The first thing we need to do is download the StructureMap.MVC5 NuGet package, which will add a DependencyInjection folder and a StructuremapMvc file to our app:

Inside the Dependency Resolution folder will be a file called DefaultRegistry.cs, which will initially look something like this:

namespace WebApiCacheDemo.Mvc.DependencyResolution {  
    using Caching;
    using StructureMap.Configuration.DSL;
    using StructureMap.Graph;

    public class DefaultRegistry : Registry {
        #region Constructors and Destructors
        public DefaultRegistry() {
            Scan(
                scan => {
                    scan.TheCallingAssembly();
                    scan.WithDefaultConventions();
                    scan.With(new ControllerConvention());
                });
        }
        #endregion
    }
}

The way StructureMap works is that it keeps instances of classes that need to be "injected" into other classes in a Container class, and uses these instances any time a particular interface is called for in another class. You'll see exactly what this means in the next section.

The Uncached Example

Let's build the uncached example first, and we can begin by setting up the MVC client. For this project we'll be using RestSharp (which I have also written about before) to consume the API responses, so be sure to download the NuGet package.

Because we're using Dependency Injection, we need our clients to be injectable into our controllers. For this reason, we will make both an interface and a class for our client, like so:

public interface ISampleClient : IRestClient  
{
    DateNumberObject GetSampleDateAndNumberUncached();
}

public class SampleClient : RestClient, ISampleClient  
{
    public SampleClient()
    {
        BaseUrl = new Uri("http://localhost:58566/");
    }

    public DateNumberObject GetSampleDateAndNumberUncached()
    {
        RestRequest request = new RestRequest("samples/date", Method.GET);
        var response = Execute<DateNumberObject>(request);
        return response.Data;
    }
}

Note that we don't need to register this client with StructureMap because the naming follows the standard conventions (ISampleClient maps to SampleClient).

Now we need our controller...

[RoutePrefix("Home")]
public class HomeController : Controller  
{
    private ISampleClient _sampleClient;

    public HomeController(ISampleClient sampleClient)
    {
        _sampleClient = sampleClient;
    }

    [HttpGet]
    [Route("Uncached")]
    [Route("")]
    [Route("~/")]
    public ActionResult Uncached()
    {
        var model = _sampleClient.GetSampleDateAndNumberUncached();
        return View(model);
    }
}

...which returns a view:

@model WebApiCacheDemo.Contracts.Samples.DateNumberObject
@{
    ViewBag.Title = "Uncached Date Sample";
}

<h2>Uncached Results</h2>  
<div class="row">  
    <span><strong>Current Date:</strong></span>
    @Model.CurrentDate
</div>  
<div class="row">  
    <span><strong>Random Number:</strong></span>
    @Model.RandomNumber
</div>  

When we run this, we can see that the view returns the newest date and time, as shown in this gif:

That's exactly what we would expect to see, of course. Now, we can get down to implementing a cache at the web layer for this.

The Cached Example

The first thing we need is our CacheService interface and implementation, which I totally stole from this StackOverflow answer:

public class InMemoryCache : ICacheService  
{
    public T GetOrSet<T>(string cacheKey, Func<T> getItemCallback) where T : class
    {
        T item = MemoryCache.Default.Get(cacheKey) as T;
        if (item == null)
        {
            item = getItemCallback();
            MemoryCache.Default.Add(cacheKey, item, DateTime.Now.AddMinutes(30));
        }
        return item;
    }
}

public interface ICacheService  
{
    T GetOrSet<T>(string cacheKey, Func<T> getItemCallback) where T : class;
}

We also need our SampleClient class updated to use this cache service, like so:

public class SampleClient : RestClient, ISampleClient  
{
    private ICacheService _cache;
    public SampleClient(ICacheService cache)
    {
        _cache = cache;
        BaseUrl = new Uri("http://localhost:58566/");
    }

    public DateNumberObject GetSampleDateAndNumber()
    {
        return _cache.GetOrSet("SampleDateAndNumber", () => GetSampleDateAndNumberUncached());
    }

    public DateNumberObject GetSampleDateAndNumberUncached()
    {
        RestRequest request = new RestRequest("samples/date", Method.GET);
        var response = Execute<DateNumberObject>(request);
        return response.Data;
    }
}

NOTE: In my structure, I am intentionally implementing the cached and uncached operations as separate methods in the same client, so that either one can be used. This may or may not be the correct usage in your application.

Further, we need to update our controller:

[RoutePrefix("Home")]
public class HomeController : Controller  
{
    ...

    [HttpGet]
    [Route("Cached")]
    public ActionResult Cached()
    {
        var model = _sampleClient.GetSampleDateAndNumber();
        return View(model);
    }
}

The last step is to register the ICacheService implementing in StructureMap's DefaultRegistry.cs class:

namespace WebApiCacheDemo.Mvc.DependencyResolution {  
    ...

    public class DefaultRegistry : Registry {
        public DefaultRegistry() {
            ...
            var inMemoryCache = new InMemoryCache();
            For<ICacheService>().Use(inMemoryCache);
        }
    }
}

Now, when we run this sample, we will see the cached values for the date and number, as shown in the following gif:

Now we've successfully implemented our cache AND used Dependency Injection in the process!

Summary

With this structure, we've successfully implemented a caching layer into our MVC application, all while using StructureMap to provide Dependency Injection. This structure allows us to potentially swap cache providers (in case we, say, move to a database-located cache for server farms) without too much trouble, as well as injecting the cache service into our existing clients.

Don't forget to check out the sample project over on GitHub, and feel free to point out how you've used this implemention or where it can be improved in the comments.

Happy Coding!

Decimal vs Double and Other Tips About Number Types in .NET

I've been coding with .NET for a long time. In all of that time, I haven't really had a need to figure out the nitty-gritty differences between float and double, or between decimal and pretty much any other type. I've just used them as I see fit, and hope that's how they were meant to be used.

Until recently, anyway. Recently, as I was attending the AngleBrackets conference, and one of the coolest parts of attending that conference is getting to be in an in-depth workshop. My particular workshop was called I Will Make You A Better C# Programmer by Kathleen Dollard, and my reaction was thus:

One of the most interesting things I learned at Kathleen's session was that the .NET number types don't always behave the way I think they do. In this post, I'm going to walk through a few (a VERY few) of Kathleen's examples and try to explain why .NET has so many different number types and what they are each for. Come along as I (with her code) attempt to show what the number types are, and what they are used for!

The Number Types in .NET

Let's start with a review of the more common number types in .NET. Here's a few of the basic types:

  • Int16 (aka short): A signed integer with 16 bits (2 bytes) of space available.
  • Int32 (aka int): A signed integer with 32 bits (4 bytes) of space available.
  • Int64 (aka long): A signed integer with 64 bits (8 bytes) of space available.
  • Single (aka float): A 32-bit floating point number.
  • Double (aka double): A 64-bit floating-point number.
  • Decimal (aka decimal): A 128-bit floating-point number with a higher precision and a smaller range than Single or Double.

There's an interesting thing to point out when comparing double and decimal: the range of double is ±5.0 × 10−324 to ±1.7 × 10308, while the range of decimal is (-7.9 x 1028 to 7.9 x 1028) / (100 to 28). In other words, the range of double is several times larger than the range of decimal. The reason for this is that they are used for quite different things.

Precision vs Accuracy

One of the concepts that's really important to discuss when dealing with .NET number types is that of precision vs. accuracy. To make matters more complicated, there are actually two different definitions of precision, one of which I will call arithmetic precision.

  • Precision refers to the closeness of two or more measurements to each other. If you measure something five times and get exactly 4.321 each time, your measurement can be said to be very precise.
  • Accuracy refers to the closeness of a value to standard or known value. If you measure something, and find it's weight to be 4.7kg, but the known value for that object is 10kg, your measurement is not very accurate.
  • Arithmetic precision refers to the number of digits used to represent a number (e.g. how many numbers after the decimal are used). The number 9.87 is less arithmetically precise than the number 9.87654332.

We always need mathematical operations in a computer system to be accurate; we cannot ever expect 4 x 6 = 32. Further, we also need these calculations to be precise using the common term; 4 x 6 must always be precisely 24 no matter how many times we make that calculation. However, the extent to which we want our systems to be either arithmetically precise has a direct impact on the performance of those system.

If we lose some arithmetic precision, we gain performance. The reverse is also true: if we need values to be arithmetically precise, we will spend more time calculating those values. Forgetting this can lead to incredible performance problems, problems which can be solved by using the correct type for the correct problem. These kinds of issues are most clearly shown during Test 3 later in this post.

Why Is Int The Default?

Here's something I've always wondered. If you take this line of code:

var number = 5;  

Why is the type of number always an int? Why not a short, since that takes up less space? Or maybe a long, since that will represent nearly any integer we could possibly use?

Turns out, the answer is, as it often is, performance. .NET optimizes your code to run in a 32-bit architecture, which means that any operations involving 32-bit integers will by definition be more performant than either 16-bit or 64-bit operations. I expect that this will change as we move toward a 64-bit architecture being standard, but for now, 32-bit integers are the most performant option.

Testing the Number Types

One of the ways we can start to see the inherent differences between the above types is by trying to use them in calculations. We're going to see three different tests, each of which will reveal a little more about how .NET uses each of these types.

Test 1: Division

Let's start with a basic example using division. Consider the following code:

private void DivisionTest1()  
{
    int maxDiscountPercent = 30;
    int markupPercent = 20;
    Single niceFactor = 30;
    double discount = maxDiscountPercent * (markupPercent / niceFactor);
    Console.WriteLine("Discount (double): ${0:R}", discount);
}

private void DivisionTest2()  
{
    byte maxDiscountPercent = 30;
    int markupPercent = 20;
    int niceFactor = 30;
    int discount = maxDiscountPercent * (markupPercent / niceFactor);
    Console.WriteLine("Discount (int): ${0}", discount);
}

Note that the only thing that's really different about these two methods are the types of the local variables.

Now here's the question: what will the discount be in each of these methods?

If you said that they'll both be $20, you're missing something very important.

The problem line is this one, from DivisionTest2():

int discount = maxDiscountPercent * (markupPercent / niceFactor);  

Here's the problem: because markupPercent is declared as an int (which in turn makes it an Int32), when you divide an int by another int, the result will be an int, even when we would logically expect it to be something like a double. .NET does this by truncating the result, so because 20 / 30 = 0.6666667, what you get back is 0 (and anything times 0 is 0).

In short, the discount for DivisionTest1 is the expected $20, but the discount for DivisionTest2 is $0, and the only difference between them is what types are used. That's quite a difference, no?

Test 2 - Double Addition

Now we get to see something really weird, and it involves the concept of arithmetic precision from earlier. Here's the next method:

public void DoubleAddition()  
{
    Double x = .1;
    Double result = 10 * x;
    Double result2 = x + x + x + x + x + x + x + x + x + x;

    Console.WriteLine("{0} - {1}", result, result2);
    Console.WriteLine("{0:R} - {1:R}", result, result2);
}

Just by reading this code, we expect result and result2 to be the same: multiplying .1 x 10 should equal .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1.

But there's another trick here, and that's the usage of the "{O:R}" string formatter. That's called the round-trip formatter, and it tells .NET to display all parts of this number to its maximum arithmetic precision.

If we run this method, what does the output look like?

By using the round-trip formatter, we see that the multiplication result ended up being exactly 1, but the addition result was off from 1 by a miniscule (but still potentially significant) amount. Question is: why does it do this?

In most systems, a number like 0.1 cannot be accurately represented using binary. There will be some form of arithmetic precision error when using a number such as this. Generally, said arithmetic precision error is not noticeable when doing mathematical operations, but the more operations you perform, the more noticeable the error is. The reason we see the error above is because for the multiplication portion, we only performed one operation, but for the addition portion, we performed ten, and thus caused the arithmetic precision error to compound each time.

Test 3 - Decimal vs Double Performance

Now we get to see something really interesting. I'm often approached by new .NET programmers with a question like the following: why should we use decimal over double and vice-versa? This test pretty clearly spells out when and why you should use these two types.

Here's the sample code:

private int iterations = 100000000;

private void DoubleTest()  
{
    Stopwatch watch = new Stopwatch();
    watch.Start();
    Double z = 0;
    for (int i = 0; i < iterations; i++)
    {
        Double x = i;
        Double y = x * i;
        z += y;
    }
    watch.Stop();
    Console.WriteLine("Double: " + watch.ElapsedTicks);
    Console.WriteLine(z);
}

private void DecimalTest()  
{
    Stopwatch watch = new Stopwatch();
    watch.Start();
    Decimal z = 0;
    for (int i = 0; i < iterations; i++)
    {
        Decimal x = i;
        Decimal y = x * i;
        z += y;
    }
    watch.Stop();
    Console.WriteLine("Decimal: " + watch.ElapsedTicks);
    Console.WriteLine(z);
}

For each of these types, we are doing a series of operations (100 million of them) and seeing how many ticks it takes for the double operation to execute vs how many ticks it takes for the decimal operations to execute. The answer is startling:

The operations involving double take 790836 ticks, while the operations involving decimal take a whopping 16728386 ticks. In other words, the decimal operations take 21 times longer to execute than the double operations. (If you run the sample project, you'll notice that the decimal operations take visibly longer than the double ones).

But why? Why does double take so much less time than decimal?

For one thing, double uses base-2 math, while decimal uses base-10 math. Base-2 math is much quicker for computers to calculate.

Further, what double is concerned with is performance, while what decimal is concerned with is precision. When using double, you are accepting a known trade-off: you won't be super precise in your calculations, but you will get an acceptable answer quickly. Whereas with decimal, precision is built into its type: it's meant to be used for money calculations, and guess how many people would riot if those weren't accurate down to the 23rd place after the decimal point.

In short, double and decimal are two totally separate types for a reason: one is for speed, and the other is for precision. Make sure you use the appropriate one at the appropriate time.

Summary

As can be expected from such a long-lived framework, .NET has several number types to help you with your calculations, ranging from simple integers to complex currency-based values. As always, it's important to use the correct tool for the job:

  • Use double for non-integer math where the most precise answer isn't necessary.
  • Use decimal for non-integer math where precision is needed (e.g. money and currency).
  • Use int by default for any integer-based operations that can use that type, as it will be more performant than short or long.

Don't forget to check out the sample project over on GitHub!

Are there any other pitfalls or improvements we should be aware of? Feel free to sound off in the comments!

Happy Coding!

Huge thanks to Kathleen Dollard (@kathleendollard) for providing the code samples and her invaluable insight into how to effectively explain what's going on in these samples. Check out her Pluralsight course for more!

The ASP.NET Web API 2 HTTP Message Lifecycle in 43 Easy Steps

Anyone who works with ASP.NET Web API should check out this poster that Microsoft created to explain the Request/Response Pipeline that Web API utilizes. It's amazing, and if you do any work in Web API you should check it out! Right now. Yes, seriously. Go ahead, I'll wait.

I love this poster, but in my opinion it doesn't do a good job of explaining the decision logic and ideas behind each step in the pipeline. Further, it doesn't explicitly tell you exactly how many things happen during this pipeline (answer: a surprisingly large number of things). In short: it's awesome, but it can be made more awesome by incorporating just a little more detail.

Here's what we're going to do in this post: using that poster, we're going to enumerate every single step involved in processing a request and receiving a response using ASP.NET Web API 2, and explain a little more about each piece of the pipeline and where we programmers can extend, change, or otherwise make more awesome this complex lifecycle. So let's get going and step through the ASP.NET Web API 2 Request Lifecycle in just 43 easy steps!

The 43 Steps

It all starts with IIS:

  1. IIS (or OWIN self-hosting) receives a request.
  2. The request is then passed to an instance of HttpServer.

  3. HttpServer is responsible for dispatching HttpRequestMessage objects.

  4. HttpRequestMessage provides strongly-typed access to the request.

  5. If one or more global instances of DelegatingHandler exist on the pipeline, the request is passed to it. The request arrives at the instances of DelegatingHandler in the order said instances were added to the pipeline.

  6. If the HttpRequestMessage passes the DelegatingHandler instances (or no such handler exists), then the request proceeds to the HttpRoutingDispatcher instance.

    • HttpRoutingDispatcher chooses which routing handler to call based on the matching route. If no such route exists (e.g. Route.Handler is null, as seen in the diagram) then the request proceeds directly to Step 10.

  7. If a Route Handler exists for the given route, the HttpRequestMessage is sent to that handler.

  8. It is possible to have instances of DelegatingHandler attached to individual routes. If such handlers exist, the request goes to them (in the order they were added to the pipeline).
  9. An instance of HttpMessageHandler then handles the request. If you provide a custom HttpMessageHandler, said handler can optionally return the request to the "main" path or to a custom end point.

  10. The request is received by an instance of HttpControllerDispatcher, which will route the request to the appropriate route as determined by the request's URL.

  11. The HttpControllerDispatcher selects the appropriate controller to route the request to.

  12. An instance of IHttpControllerSelector selects the appropriate HttpControllerDescriptor for the given HttpMessage.
  13. The IHttpControllerSelector calls an instance of IHttpControllerTypeResolver, which will finally call...
  14. ...an instance of IAssembliesResolver, which ultimately selects the appropriate controller and returns it to the HttpControllerDispatcher from Step 11.
    • NOTE: If you implement Dependency Injection, the IAssembliesResolver will be replaced by whatever container you register.
  15. Once the HttpControllerDispatcher has a reference to the appropriate controller, it calls the Create() method on an IHttpControllerActivator...
  16. ...which creates the actual controller and returns it to the Dispatcher. The dispatcher then sends the request into the Select Controller Action routine, as shown below.

  17. We now have an instance of ApiController which represents the actual controller class the request is routed to. Said instance calls the SelectAction() method on IHttpActionSelector...

  18. ...which returns an instance of HttpActionDescriptor representing the action that needs to be called.

  19. Once the pipeline has determined which action to route the request to, it executes any Authentication Filters which are inserted into the pipeline (either globally or local to the invoked action).

    • These filters allow you to authenticate requests to either individual actions, entire controllers, or globally throughout the application. Any filters which exist are executed in the order they are added to the pipeline (global filters first, then controller-level filters, then action-level filters).
  20. The request then proceeds to the [Authorization Filters] layer, where any Authorization Filters which exist are applied to the request.

    • Authorization Filters can optionally create their own response and send that back, rather than allowing the request to proceed through the pipeline. These filters are applied in the same manner as Authentication Filters (globally, controller-level, action-level). Note that Authorization Filters can only be used on the Request, not the Response, as it is assumed that if a Response exists, the user had the authorization to generate it.
  21. The request now enters the Model Binding process, which is shown in the next part of the main poster. Each parameter needed by the action can be bound to its value by one of three separate paths. Which path the binding system uses depends on where the value needed exists within the request.

  22. If data needed for an action parameter value exists in the entity body, Web API reads the body of the request; an instance of FormatterParameterBinding will invoke the appropriate formatter classes...

  23. ...which bind the values to a media type (using MediaTypeFormatter)...

  24. ...which results in a new complex type.

  25. If data needed for a parameter value exists in the URL or query string, said URL is passed into an instance of IModelBinder, which uses an IValueProvider to map values to a model (see Phil Haack's post about this topic for more info)....

  26. ...which results in a simple type.

  27. If a custom HttpParameterBinding exists, the system uses that custom binding to build the value...

  28. ...which results in any kind (simple or complex) of object being mappable (see Mike Stall's wonderful series on this topic).

  29. Now that the request is bound to a model, it is passed through any Action Filters which may exist in the pipeline (either globally or just for the action being invoked).

  30. Once the action filters are passed, the action itself is invoked, and the system waits for a response from it.

  31. If the action produces an exception AND an exception filter exists, the exception filter receives and processes the exception.

  32. If no exception occurred, the action produces an instance of HttpResponseMessage by running the Result Conversion subroutine, shown in the next screenshot.

  33. If the return type is already an HttpResponseMessage, we don't need to do any conversion, so pass the return on through.

  34. If the return type is void, .NET will return an HttpResponseMessage with the status 204 No Content.

  35. If the return type is an IHttpActionResult, call the method ExecuteAsync to create an HttpResponseMessage.

    • In any Web API method in which you use return Ok(); or return BadRequest(); or something similar, that return statement follows this process, rather than any of the other processes, since the return type of those actions is IHttpActionResult.
  36. For all other types, .NET will create an HttpResponseMessage and place the serialized value of the return in the body of that message.

  37. Once the HttpResponseMessage has been created, return it to the main pipeline.

  38. Pass the newly-created HttpResponseMessage through any AuthenticationFilters which may exist.

    • Remember that Authorization Filters cannot be used on Responses.

  39. The HttpResponseMessage flows through the HttpControllerDispatcher, which at this point probably won't do anything with it.

  40. The Response also flows through the HttpRoutingDispatcher, which again won't do anything with it.

  41. The Response now proceeds through any DelegatingHandlers that are set up to handle it. At this point, the DelegatingHandler objects can really only change the response being sent (e.g. intercept certain responses and change to the appropriate HTTP status).

  42. The final HttpResponseMessage is given to the HttpServer instance...

  43. ...which returns an Http response to the invoking client.

Tada! We've successfully walked through the entire Web API 2 request/response pipeline, and in only 43 easy steps!

Let me know if this kind of deep dive has been helpful to you, and feel free to share in the comments! Microsoft people and other experts, please chime in to let me know if I got something wrong; I intend for this post to be the definitive guide to the Web API 2 Request/Response Lifecycle, and you can't be definitive without being correct.

Happy Coding!