Versioning and Consistency

Consistent Reads

By default SimpleDB read operations return eventually consistent data. When you update data in SimpleDB and immediately read it back (within 1-2 seconds), you might get an old version of the item's attributes rather than the newest version. (After a few seconds your updates will have been replicated across all active SimpleDB nodes and you'll "see" only the latest version.)

Eventually consistent reads are unacceptable in some usage scenarios. To support these scenarios SimpleDB also provides "consistent reads" that guarantee the latest version of any attributes written before the read operation started. There is a performance penalty for using consistent reads so you probably will want to use them only when necessary.

Simol provides two options for using consistent reads:
  1. Use consistent reads for all operations by setting SimolConfig.ReadConsistency to ConsistencyBehavior.Immediate
  2. Use consistent reads within a limited application scope with an instance of ConsistentReadScope
To use a scoped consistent read, simply create an instance of ConsistentReadScope, and dispose it when you want to return to "normal" reads. Only get and select operations performed on the thread that created the ConsistentReadScope instance will be "consistent". Here's an example:

    using (new ConsistentReadScope())
    {
        var employeeId = new Guid("6c0a37b4-9c59-49ce-95af-d01a66f9605d");
        Employee e = simol.Get<Employee>(employeeId);

        List<Employee> allEmployees = simol.Select<Employee>("select * from Employee");
    }

    List<Employee> mostEmployees = simol.Select<Employee>("select * from Employee");

The get and select operations performed inside the using block will return "consistent" data. The final select operation will not return consistent data because it's outside our consistent read scope--the ConsistentReadScope instance is disposed as soon as we exit the using block. The fact that we've previously performed a consistent read operation using the exact same query does not matter--the second select operation can still return different data even though we haven't performed any updates between the two operations.

Important: Scoped consistent reads have no effect on asynchronous operations performed using the Simol asynchronous extension methods, including reads performed by the SimolClient.Find methods.

Reliable Writes

SimpleDB domains are completely segregated and cannot be "joined" by select queries as with a traditional relational database. This design constraint leads naturally to denormalization of data as a common pattern when designing applications for SimpleDB. But SimpleDB has no transactional mechanism for ensuring cross-domain data consistency, which means applications must take special care to maintain consistency when duplicating data or creating "relationships" between data in different domains. Reliable-writes can provide pseudo-transactionality when writing to multiple domains.

A reliable-write guarantees that you will never suffer partial data loss when writing associated data to multiple domains. System failure during a reliable-write will result in all or none of your data being stored in SimpleDB. This is accomplished using a 2-phased update. In the first phase your data is transactionally stored in a single domain. In the second phase your data is propagated to the final destination domains. If the initial propagation attempt is interrupted your data may remain in an inconsistent state for a short time. Here are some additional things to keep in mind when using reliable-writes:
  • Put, batch-put, delete, and batch-delete operations are supported
  • A maximum of 25 sub-operations may be included in a single reliable-write.
  • Conditional puts are not supported
  • A reliable-write adds about 150% overhead to each sub-operation. But parallel propagation of sub-operations allows the overall write to complete with very little additional latency.
  • Propagation of sub-operations can fail permanently if SimpleDB limits are exceeded. This condition will be very obvious and requires manual intervention to correct.
Note: Simol uses batch-put when putting multiple items at once, so you can actually put up to 625 items in a single reliable write (25 batch-put operations * 25 items per batch-put).

To use reliable-writes you must create and start a WriteMonitor at application startup:

    WriteMonitor monitor = new WriteMonitor(simol);
    monitor.Start();

This monitor runs in the background to finalize reliable-write updates that fail during the propagation phase. You must also provide a WriteMonitor instance when actually performing the write operation (see the example below). You may use the same monitor instance throughout your application or or create new ones for each reliable-write. However, you should never start more than one monitor during the life of your application.

Here's a reliable-write example that demonstrates saving new Employee and Address items while also deleting a Person item:

    using(var writeScope = new ReliableWriteScope(monitor))
    {
        var personId = new Guid("6c0a37b4-9c59-49ce-95af-d01a66f9605d");
        var employee = new Employee();
        var address = new Address();

        simol.Put(employee);
        simol.Put(address);
        simol.Delete<Person>(personId);

        writeScope.Commit();
    }

Note that you must invoke ReliableWriteScope.Commit() for these items to be written to SimpleDB. If you don't invoke Commit the new data will be discarded when the ReliableWriteScope instance is disposed. The following things happen when you invoke Commit:
  1. The put and delete requests are "committed" transactionally to the SimolSystem domain (phase 1). If this phase fails the write attempt is over and all data is discarded.
  2. The put and delete requests are propagated in parallel to their appropriate destination domains (phase 2). If Commit returns with no error then the entire write operation (including propagation) has completed successfully. If any part of the propagation phase fails, the resulting exception is thrown up to the caller of Commit.
  3. If any part of the propagation phase fails, the WriteMonitor running in the background will retry propagation until it succeeds or the failing request is removed from the SimolSystem domain. (The ReliableWriteId of failing requests is logged by Simol in case manual cleanup is required for a persistently failing operation.)

Data Validation and Constraints

You may occasionally wish to perform data validation or manipulation when loading or saving SimpleDB items with Simol. One option for this is to wrap Simol in a data repository class which performs these functions. A less intrusive method that will often suffice is attaching snippets of domain constraint logic to your data class with ConstraintAttribute.

For example, if we were storing Appointment items with a DayOfWeek property we could use a custom constraint to ensure the stored value is always in the range 1-7. To accomplish this we need to attach our custom constraint to the appointment class like this:

    [Constraint(typeof(AppointmentConstraint))]
    public class Appointment
    {
        [ItemName]
        public Guid Id { get; set; }

        [NumberFormat(1, 0, false)]
        public byte DayOfWeek { get; set; }
    }

And implement our custom domain constraint:

    public class AppointmentConstraint : DomainConstraintBase
    {
        public override void  BeforeSave(PropertyValues values)
        {
            byte day = (byte)values["DayOfWeek"];
            if (day < 1 || day > 7)
            {
                throw new InvalidOperationException("'DayOfWeek' value must be in the range 1-7");
            }
        }
   }

Note: DomainConstraintBase is a no-op convenience implementation of IDomainConstraint, which also includes methods for applying load and delete constraints.

Attempts to create and save invalid appointments will fail as demonstrated in the code below:

    var appointment = new Appointment
    {
        DayOfWeek = 20
    };
    try
    {
        simol.Put(appointment);
    } catch (InvalidOperationException ex)
    {
        // prints "'DayOfWeek' value must be in the range 1-7"
        Console.WriteLine(ex.Message);                    
    }

Versioning

Simol provides three versioning options that may be selected when you mark DateTime or int properties with the VersionAttribute:
  1. VersioningBehavior.None - The application increments and checks the version value as desired. This option will be used primarily for full-text indexed items, since full-text indexing requires a version property.
  2. VersioningBehavior.AutoIncrement - Simol automatically increments the version property each time you perform a put operation, but no stale update checks are performed.
  3. VersioningBehavior.AutoIncrementAndConditionallyUpdate - Simol automatically increments the version property and instructs SimpleDB to reject updates with stale versions.
This section explains how to use the third versioning option.

SimpleDB provides conditional put operations that cause data updates to fail if the item being updated contains unexpected attribute values. Simol can be instructed to perform conditional puts when you mark object properties with the VersionAttribute. First, let's define a simple Account object with a Version property:

    public class Account
    {
        [ItemName]
        public Guid Id { get; set; }

        [Version(VersioningBehavior.AutoIncrementAndConditionallyUpdate)]
        public DateTime ModifiedAt { get; set; }

        public decimal Balance { get; set; }
    }

Now let's create an Account instance, insert it in SimpleDB with a Balance of 1,000, and then attempt to immediately update the same account:

    Account account1 = new Account
    {
        Balance = 1000,
        Id = new Guid("50b156b6-33cb-4566-930e-0dd8e3f466de")
    };
    simol.Put(account1);

    Account account2 = new Account
    {
        Id = account1.Id,
        Balance = 100
    };
    try
    {
        simol.Put(account2);    
    }
    catch (AmazonSimpleDBException ex)
    {
        // prints "ConditionalCheckFailed"
        Console.WriteLine(ex.ErrorCode);
    }

The second update attempt will fail with an AmazonSimpleDBException because we are attempting to update the item using an out-of-date version value. Instead, to perform the second update you should first read the stored data item from SimpleDB before updating:

    Guid accountId = new Guid("50b156b6-33cb-4566-930e-0dd8e3f466de");
    Account account3 = simol.Get<Account>(accountId);

    account3.Balance = 100;
    simol.Put(account3);

Warning: Applications may never update version property values when using VersioningBehavior.AutoIncrementAndConditionallyUpdate. Doing so will always result in a "ConditionalCheckFailed" error when inserting or updating data.

Last edited May 24, 2011 at 12:43 PM by ashleytate, version 22

Comments

ashleytate Mar 2, 2010 at 12:38 PM 
That's right. To intersperse consistent reads with normal reads using async operations you would need to maintain two instances of SimpleSavant, one configured for each type of read.

SoopahMan Mar 2, 2010 at 4:15 AM 
Does the first portion imply there is no way to do a per-command Async select that is guaranteed consistent?