timestamp challenge

Dec 21, 2010 at 7:55 PM

I am contemplating porting a service I implemented on Windows Azure to Amazon. We have multiple clients writing (appending) to an Azure table (conceptually similar to SimpleDb) but we must maintain sequential ordering. To do this the clients post messages to an Azure queue and a worker process removes items from the queue and writes them to the table. Assuming there is a single worker (which in our case is sufficient to keep up with the transaction rate) then the queue provides us with the necessary serialiaziation and consistant timestamps.

I wonder if this can be implemented in SimpleDb (SimpleSavant) without the need for the worker process? The problem with Azure is that there are only two indexed fields and the internally generated timestamp isn't indexed. As I understand it, all fields in SimpleDb are indexed. If that is the case and if there is a system generated timestamp then I think this will work.

Sorry to hijack the SimpleSavant discussion but it seems like an interesting abstraction layer and folks reading this will be hopefully conversant enough in the underlying SimpleDb to answer.


Dec 21, 2010 at 11:20 PM

Yes, all fields (attributes) in SimpleDB are indexed, but there is no system-generated timestamp (assuming you mean a timestamp generated by SimpleDB on record insertion). I supposed you could still implement this without the worker process if your client clocks were synchronized or if the clients could retrieve and increment a sequence value stored in SimpleDB before adding each record. Amazon also has a queue service so you could also easily implement the current pattern in AWS.

Dec 22, 2010 at 12:28 AM

I was hoping that SimpleDB would insert a timestamp column. The clients are not time synched that is why we resort to the queue and the worker process to write an indexed timestamp. So I am no worse off with SimpleDB :-(