Get the Latest item in a domain

Mar 23, 2010 at 12:32 AM
Edited Mar 23, 2010 at 12:33 AM

The answer to this question may have more to do with SimpleDB than Simple Savant, but here goes....

I need to get the latest entry from a domain called "telemetry_events".  I log the status of a 20 server farm to SimpleDB with a "server_data" class every 5 minutes.   What's the best way to query just one Item using Savant?   I don't care if I select absolutely newest Item, but I want as close to the most recent event as possible.

I was sorting by log_date (a DateTime) descending:

string query = "select * from telemetry_events where log_date < '3000' order by log_date desc limit 1";
TelemetryEvent latest_telemetry = savant.Select<TelemetryEvent>(query).ElementAt(0);

(NOTE:  '3000' is a late enough year so I can get all Item and the sort works by log_date)

This worked fine with a few entries in the Domain, but with 1000 plus now, it hangs for like 5 min, like Savant is retrieving and processing all the Items first.   The same query straight to SimpleDB is instant, but I don't seem to get the same result every time (even though I log every 5 min).

Thanks for your help!

Coordinator
Mar 23, 2010 at 1:20 AM
Edited Mar 24, 2010 at 6:53 PM

One of the features of Savant is to transparently get all available results when issuing a select query. This lets applications easily execute long-running queries requiring multiple calls to SimpleDB (>5 sec execution time or >2500 results).

But it sounds like this feature is working against you here and fetching all available results, 1 at a time. You need to limit your results by using a SelectCommand and setting SelectCommand.MaxResultPages = 1. There's an example of this on the Advanced Select Queries page under Limiting Query Results.

To get the absolute latest telemetry event wrap your call with an instance of ConsistentReadScope. This will use the new consistent read feature of SimpleDB rather than the default "eventually consistent" read:

 

using (new ConsistentReadScope())
{
string query = "select * from telemetry_events where log_date < '3000' order by log_date desc limit 1";
TelemetryEvent latest_telemetry = savant.Select<TelemetryEvent>(query).ElementAt(0);
}