change-tracking

Entity Change Tracking using DbContext in Entity Framework 6

We've been searching for a way to audit changes made to our database, and one method we found was doing so via Entity Framework change tracking. Here's how to do it, what's required to set it up, and some drawbacks of this method. Let's get started!

Background

Entity Framework tracks changes made to entities when those entities are loaded from the Context. So, if we load an entity and modify it, like so:

var movie = context.Movies.Find(2);  
movie.Title = "The Great Gatsby";

context.SaveChanges()  

When the call to SaveChanges is made, Entity Framework is aware of what values have changed, what the new values are, and what the original values are. We need to access this information to accomplish the change tracking we want.

Setup

For this demo, let's assume we have the following entities and context:

public class Movie  
{
    public int Id { get; set; }
    public string Title { get; set; }
    public DateTime ReleaseDate { get; set; }
}

public class Actor  
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime DateOfBirth { get; set; }
}

public class MoviesContext : DbContext  
{
    public virtual DbSet<Movie> Movies { get; set; }
    public virtual DbSet<Actor> Actors { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        //omitted for brevity
    }
}

How can we record changes made to these entities automatically, via Entity Framework? Let's find out!

Implementing Change Tracking

For this demo, we're going to store all changes made to any entity in a single data table. Said table can be represented by this entity:

public class ChangeLog  
{
    public int Id { get; set; }
    public string EntityName { get; set; }
    public string PropertyName { get; set; }
    public string PrimaryKeyValue { get; set; }
    public string OldValue { get; set; }
    public string NewValue { get; set; }
    public DateTime DateChanged { get; set; }
}

Notice the property PrimaryKeyValue. Part of recording "which entity changed" is both which table changed (e.g. "Movie", "Actor", etc) and which specific record in that table changed (e.g. 1, 2, etc.). In our implementation, we will need both of these for proper, thorough auditing. Let's see how we can actually implement auditing using Entity Framework.

In order to get all the data we need to produce these change logs, we will follow five steps:

  1. Override the Context's SaveChanges Method
  2. Get the Change Details
  3. Get the Primary Key Values
  4. Save the Logs

Let's see what we need to accomplish, step-by-step.

Step 1: Override the Context's SaveChanges Method

public override int SaveChanges()  
{
    return base.SaveChanges();
}

This allows us to automatically generate change logs any time an entity is saved to our database.

Step 2: Get the Change Details

How can we get the actual changes that will occur for this execution of SaveChanges? We will use the ChangeTracker property of DbContext, like so:

List<DbEntityEntry> modifiedChanges = ChangeTracker.Entries().Where(x => x.State == EntityState.Modified).ToList();  

So what is DbEntityEntry? That type represents the change that will be made to a particular entity, including the original and current values and the state (Added, Modified, Deleted, etc) of that entity. What we now have is a list of all the entities being modified by this context.

Using this list of changes, we can cycle through each change and determine what values are actually changing:

public override int SaveChanges()  
{
    var modifiedEntities = ChangeTracker.Entries()
        .Where(p => p.State == EntityState.Modified).ToList();
    var now = DateTime.UtcNow;

    foreach (var change in modifiedEntities)
    {
        var entityName = change.Entity.GetType().Name;
        var primaryKey = GetPrimaryKeyValue(change);

        foreach(var prop in change.OriginalValues.PropertyNames)
        {
            var originalValue = change.OriginalValues[prop].ToString();
            var currentValue = change.CurrentValues[prop].ToString();
            if (originalValue != currentValue) //Only create a log if the value changes
            {
                //Create the Change Log
            }
        }
    }
    return base.SaveChanges();
}

That method now allows us to get most of the data we need to create a change log. However, we haven't implemented GetPrimaryKeyValue yet.

Step 3: Get the Primary Key Values

We'll use a private method in the DbContext to get the primary key values (taken from this wonderful StackOverflow answer):

object GetPrimaryKeyValue(DbEntityEntry entry)  
        {
            var objectStateEntry = ((IObjectContextAdapter)this).ObjectContext.ObjectStateManager.GetObjectStateEntry(entry.Entity);
            return objectStateEntry.EntityKey.EntityKeyValues[0].Value;
        }

In order to retrieve the primary keys, we must cast our DbContext down to IObjectContextAdapter and query the ObjectStateManager. Once we have access to that manager, we can get the primary key value (note that this method assumes a single-column primary key, which is not necessarily a good real-world scenario, see Drawbacks).

Step 4: Save the Logs

All we have to do now is add the logs to the Context and save the changes.

public override int SaveChanges()  
{
    var modifiedEntities = ChangeTracker.Entries()
        .Where(p => p.State == EntityState.Modified).ToList();
    var now = DateTime.UtcNow;

    foreach (var change in modifiedEntities)
    {
        var entityName = change.Entity.GetType().Name;
        var primaryKey = GetPrimaryKeyValue(change);

        foreach(var prop in change.OriginalValues.PropertyNames)
        {
            var originalValue = change.OriginalValues[prop].ToString();
            var currentValue = change.CurrentValues[prop].ToString();
            if (originalValue != currentValue)
            {
                ChangeLog log = new ChangeLog()
                {
                    EntityName = entityName,
                    PrimaryKeyValue = primaryKey.ToString(),
                    PropertyName = prop,
                    OldValue = originalValue,
                    NewValue = currentValue,
                    DateChanged = now
                };
                ChangeLogs.Add(log);
            }
        }
    }
    return base.SaveChanges();
}

The Complete Code

public class Movie  
{
    public int Id { get; set; }
    public string Title { get; set; }
    public DateTime ReleaseDate { get; set; }
}

public class Actor  
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime DateOfBirth { get; set; }
}

public class ChangeLog  
{
    public int Id { get; set; }
    public string EntityName { get; set; }
    public string PropertyName { get; set; }
    public string PrimaryKeyValue { get; set; }
    public string OldValue { get; set; }
    public string NewValue { get; set; }
    public DateTime DateChanged { get; set; }
}

public class MoviesContext : DbContext  
{
    public virtual DbSet<Movie> Movies { get; set; }
    public virtual DbSet<Actor> Actors { get; set; }
    public virtual DbSet<ChangeLog> ChangeLogs { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        //omitted for brevity
    }


    public override int SaveChanges()
    {
        var modifiedEntities = ChangeTracker.Entries()
            .Where(p => p.State == EntityState.Modified).ToList();
        var now = DateTime.UtcNow;

        foreach (var change in modifiedEntities)
        {
            var entityName = change.Entity.GetType().Name;
            var primaryKey = GetPrimaryKeyValue(change);

            foreach(var prop in change.OriginalValues.PropertyNames)
            {
                var originalValue = change.OriginalValues[prop].ToString();
                var currentValue = change.CurrentValues[prop].ToString();
                if (originalValue != currentValue)
                {
                    ChangeLog log = new ChangeLog()
                    {
                        EntityName = entityName,
                        PrimaryKeyValue = primaryKey.ToString(),
                        PropertyName = prop,
                        OldValue = originalValue,
                        NewValue = currentValue,
                        DateChanged = now
                    };
                    ChangeLogs.Add(log);
                }
            }
        }
        return base.SaveChanges();
    }
}

Drawbacks

There are a couple of significant drawbacks to this particular solution:

  1. No auditing for Added entities. This is because, in my system, the database is responsible for creating the primary key values (via IDENTITY columns) and therefore the primary keys do not exist before the entity is added to the database. Attempting to use the database-generated primary keys for Added entities would result in two round-trips to the database on every save.
  2. Support for single-column primary keys only. This code makes an explicit assumption that only one column per table in your database is the primary key, which is not true in the real world.

Summary

The method outlined about is a good way to track and audit changes made to existing entities. It allows you to record the entity changed, the value changed, the primary key of the changed record, and the date of change. However, it does not track the adding of entities and it does not support multiple-column primary keys. With all of that said, I feel this is a pretty good step toward having full auditing in Entity Framework for our system.

Did I miss something, or can you improve my code? Please feel free to do so, and share in the comments!

Happy Coding!

Object Change Tracking via Reflection in .NET

We have this big project we're working on (which I have written about before) and one of the things we need to do on this project is automatic logging of changes made to model objects. I've worked out a way to do this generically, for any object, and I think others might find it just as useful as we have.

Requirements

We needed to be able to compare two objects of the same type, examine their properties, and log which properties have changed value. Further, we needed to be able to mark which property represents the primary key for the instance of the object (which is not expected to change value), and we needed to be able to specify fields whose changes won't be logged. Finally, we wanted to do this generically; for any two objects of the same type, we should be able to compare them and log changes.

In short, we needed this log generator to be:

  • Able to compare two objects of any type, provided they are the same type as each other
  • Able to exclude certain properties
  • Able to determine the primary key field and record that value

Let's set up an environment where we can achieve these goals.

Program Setup

Let's say we have a model object that looks like this:

public class RetailLocation  
{
    public int Id { get; set; }
    public DateTime DateOpened { get; set; }
    public string ManagerFirstName { get; set; }
    public string ManagerLastName { get; set; }
    public bool HasLimitedMenu { get; set; }
}

Every time a change is made to this object, we want to automatically record what changes occurred.

Further, we also have the log object itself:

public class ChangeLog  
{
    public string ClassName { get; set; }
    public string PropertyName { get; set; }
    public int PrimaryKey { get; set; }
    public string OldValue { get; set; }
    public string NewValue { get; set; }
    public DateTime DateChanged { get; set; }
}

What this means is that if multiple properties change (between two objects) then we'll get multiple records of ChangeLog.

Let's set up a command-line program that will test this.

class Program  
{
    static void Main(string[] args)
    {
        //Create the old entry
        RetailLocation oldLocation = new RetailLocation()
        {
            Id = 1,
            DateOpened = new DateTime(2009, 12, 3),
            ManagerFirstName = "Steve",
            ManagerLastName = "Harkonnen",
            HasLimitedMenu = true
        };

        //Create the new entry
        RetailLocation newLocation = new RetailLocation()
        {
            Id = 1,
            DateOpened = new DateTime(2009, 12, 3),
            ManagerFirstName = "Kelly",
            ManagerLastName = "Nishimura",
            HasLimitedMenu = false
        };

        ChangeLogService service = new ChangeLogService();
        List<ChangeLog> logs = service.GetChanges(oldLocation, newLocation); //What does this do?

        foreach(var log in logs)
        {
            Console.WriteLine("Primary Key: " + log.PrimaryKey.ToString() + ", Class Name:" + log.ClassName + ", Property Name: " + log.PropertyName + ", Old Value = " + log.OldValue + ", New Value = " + log.NewValue);
        }
    }
}

Whenever we instantiate ChangeLogService and call GetChanges, we should get back a List of ChangeLogs that will contain all the changes found between the two objects. But exactly how can we get these changes?

Making the Comparison

Let's start with our GetChanges method declaration:

public List<ChangeLog> GetChanges(object oldEntry, object newEntry)  
{
    List<ChangeLog> logs = new List<ChangeLog>();
}

We'll need several steps to get all of the changes.

Step 1: Do the Types Match?

First off, we won't compare objects if they are not of the same type. So let's check the type of the two objects and return a blank list of changes if the types are different.

var oldType = oldEntry.GetType();  
var newType = newEntry.GetType();  
if(oldType != newType)  
{
    return logs; //Types don't match, cannot log changes
}

Step 2: Find the Primary Key Property

Now we come to the first real issue we have to solve: we need to record the primary key value for the record that changed.

We can make the assumption that the object will have at least one of those fields represent a primary key value. (In the real world, of course, you could have multiple fields represent the key, but in this tutorial we're assuming that the database has a single-column primary key for all tables). But, even with this assumption, how can we identify which property in the object (i.e. which column in the database) is that primary key?

We will use an attribute:

public class LoggingPrimaryKeyAttribute : Attribute  
{
}

We can then use this to decorate the RetailLocation object from earlier:

public class RetailLocation  
{
    [LoggingPrimaryKey]
    public int Id { get; set; }
    public DateTime DateOpened { get; set; }
    public string ManagerFirstName { get; set; }
    public string ManagerLastName { get; set; }
    public bool HasLimitedMenu { get; set; }
}

Step 3: Initialize Change Log Data and Get All Properties

We now need some data about each change log that we're going to create. Specifically, we need the primary key value, the date changed, and the type name. We're using the LoggingPrimaryKeyAttribute to get the first item, and the other two can be gathered from other data, like so:

var oldProperties = oldType.GetProperties();  
var newProperties = newType.GetProperties();

var dateChanged = DateTime.Now;  
var primaryKey = (int)oldProperties.Where(x => Attribute.IsDefined(x, typeof(LoggingPrimaryKeyAttribute))).First().GetValue(oldEntry);  
var className = oldEntry.GetType().Name;  

Notice the call to Attribute.IsDefined. This returns true if the given property (represented by x in this call) has an attribute of the given type defined upon it. This is how we can check if a particular attribute is defined on a particular property (and we'll be using this again in just a bit).

Better still, we now have lists of properties from the two changed objects. We can use these lists to determine which of their properties have changed.

Step 4: Ignore Specified Properties

We have now reached the step where we can compare the properties of the two objects. However, we first need a way to exclude the properties that we don't want to log changes for. To do this, we create another attribute:

public class IgnoreLoggingAttribute : Attribute  
{
}

In our sample, we'll pretend that we don't want to record changes made to the Manager's First Name, so we'll decorate RetailLocation like so:

public class RetailLocation  
{
    [LoggingPrimaryKey]
    public int Id { get; set; }
    public DateTime DateOpened { get; set; }
    [IgnoreLogging]
    public string ManagerFirstName { get; set; }
    public string ManagerLastName { get; set; }
    public bool HasLimitedMenu { get; set; }

}

Step 5: Compare the Properties

We want to walk through each of the properties on one of the objects and to see if the other object's value for that property has changed.

foreach(var oldProperty in oldProperties)  
{

}

Inside this foreach loop, we check to see if the new instance has the property:

foreach(var oldProperty in oldProperties)  
{
    var matchingProperty = newProperties.Where(x => !Attribute.IsDefined(x, typeof(IgnoreLoggingAttribute)) 
                                                    && x.Name == oldProperty.Name 
                                                    && x.PropertyType == oldProperty.PropertyType)
                                        .FirstOrDefault();
    if(matchingProperty == null)
    {
        continue; //If we don't find a matching property, move on to the next property.
    }
}

Once again we have a call to Attribute.IsDefined, only this time we want the properties that don't have the attribute [IgnoreLogging] defined upon them.

Finally, we need to compare the values of the properties and, if they are different, create a new entry of ChangeLog to record the differences. You might think about doing this:

var oldValue = oldProperty.GetValue(oldEntry);  
var newValue = matchingProperty.GetValue(newEntry);  
if(matchingProperty != null && oldValue != newValue) { //Create ChangeLog }  

However, if you do this, you'll find that the IF clause is always true. In other words, oldValue will always not be equal to newValue. This is because, in C# the == and != operators perform a reference value check on the objects being compared, which will never be equal (see Jon Skeet's answer on StackOverflow). Instead, what we need to do is change the values to string to directly compare them:

var oldValue = oldProperty.GetValue(oldEntry).ToString();  
var newValue = matchingProperty.GetValue(newEntry).ToString();  
if(matchingProperty != null && oldValue != newValue) { //Create ChangeLog }  

The Complete Method

Here is the complete code for the GetChanges method:

public List<ChangeLog> GetChanges(object oldEntry, object newEntry)  
{
    List<ChangeLog> logs = new List<ChangeLog>();

    var oldType = oldEntry.GetType();
    var newType = newEntry.GetType();
    if(oldType != newType)
    {
        return logs; //Types don't match, cannot log changes
    }

    var oldProperties = oldType.GetProperties();
    var newProperties = newType.GetProperties();

    var dateChanged = DateTime.Now;
    var primaryKey = (int)oldProperties.Where(x => Attribute.IsDefined(x, typeof(LoggingPrimaryKeyAttribute))).First().GetValue(oldEntry);
    var className = oldEntry.GetType().Name;

    foreach(var oldProperty in oldProperties)
    {
        var matchingProperty = newProperties.Where(x => !Attribute.IsDefined(x, typeof(IgnoreLoggingAttribute)) 
                                                        && x.Name == oldProperty.Name 
                                                        && x.PropertyType == oldProperty.PropertyType)
                                            .FirstOrDefault();
        if(matchingProperty == null)
        {
            continue;
        }
        var oldValue = oldProperty.GetValue(oldEntry).ToString();
        var newValue = matchingProperty.GetValue(newEntry).ToString();
        if(matchingProperty != null && oldValue != newValue)
        {
            logs.Add(new ChangeLog()
            {
                PrimaryKey = primaryKey,
                DateChanged = dateChanged,
                ClassName = className,
                PropertyName = matchingProperty.Name,
                OldValue = oldProperty.GetValue(oldEntry).ToString(),
                NewValue = matchingProperty.GetValue(newEntry).ToString()
            });
        }
    }

    return logs;
}

The completed method fulfills all of our requirements:

  • It will get and record the primary key value.
  • It will ignore specified columns.
  • It will compare the properties of two objects of the same type and return the properties that have changed values.

Check out my sample project on Github, which has all of the code we've written in this post. As always, if you notice something I could've done better with this code, let me know in the comments!

Happy Coding!