Simol (formerly Simple Savant)

New Post: Protecting attributes from being updated

AshleyTate — Wed, 15 Feb 2012 15:01:21 GMT

No this isn't done automatically because there's no way for Simol to know what attributes already exist in SimpleDB without first loading the attributes for whatever item you're updating. If you want to guarantee that certain attributes are not changed you'll need to either load the object from SimpleDb first or use the PutAttributes methods to perform a partial object update. See the docs here.

New Post: Protecting attributes from being updated

hairgami_master — Wed, 15 Feb 2012 14:41:09 GMT

I'm assuming this is already documented, and I'm just missing it, so anyways...

Is there a way to "protect" attributes from being updated? My hope is that if I create a new object and Simol.Put it, it will write every attribute. The next time around, if I'm putting an object with Simol, and it already exists on simpleDB, I'd like to make sure certain attributes from the original object creation remain untouched.

I could do this manually, by making sure the "new" object has the "old" values, but I'm just curious if there's a way to do this.

Cheers

New Post: DateTime not converting for some reason

AshleyTate — Wed, 01 Feb 2012 03:03:07 GMT

You're welcome! Glad you figured it out from the docs.

New Post: DateTime not converting for some reason

Hairgami_Master — Wed, 01 Feb 2012 02:56:01 GMT

Ha! I always do this. Post a question, and it magically causes me to find the solution on my own. For those who have a similar problem, Ashley has documented this very thing:

http://simol.codeplex.com/wikipage?title=Defining%20Object%20Mappings&referringTitle=Documentation

In a section called: Mapping and Formatting Customization

So in the above example, I had to change

public DateTime creationdate { get; set; }

to:

[CustomFormat("MM-dd-yyyy hh:mm:ss")]
public DateTime creationdate { get; set; }

Thanks Ashley! Awesome!

New Post: DateTime not converting for some reason

hairgami_master — Wed, 01 Feb 2012 02:42:56 GMT

Hi Ashley and others-
Simol worked the first time for me- no problems whatsoever! Awesome!! I am having a tiny problem- it doesn't want to convert a DateTime field that looks like this:

1/9/2012 10:57:36 AM

Here's the error I get:
String value '1/9/2012 10:57:36 AM' could not be converted to expected property type 'System.DateTime' for property 'PropertyValues[creationdate]'.

The property it's mapping to in the Class:

public DateTime creationdate { get; set; }

I did download and build the source using the latest AWS .NET dll,, but I think I was having this problem before I tried that. Any suggestions?

Thanks so much for all your hard work on this, and good luck with your Dynamo project.

Cheers!

More Detail

Simol.SimolDataException was unhandled

Message=String value '1/9/2012 10:57:36 AM' could not be converted to expected property type 'System.DateTime' for property 'PropertyValues[creationdate]'.

Source=Simol StackTrace:
at Simol.Core.MappingUtils.StringToPropertyValue(PropertyFormatter formatter, AttributeMapping attributeMapping, String valueString) in C:\code\simol-70792\trunk\src\Simol\Core\MappingUtils.cs:line 175

New Post: DynamoDB, SimpleDB and Simol

widescreen — Tue, 24 Jan 2012 17:18:27 GMT

I will slowly migrate to DynamoDB in a production application that currently uses SIMOL for access to SimpleDB and am eagerly awaiting your first code release.

A drop in replacement would be terrific, but not necessary (IMO). I'd rather re-code my repository as a tradeoff for being able to leverage DynamoDB specific functionality.

Thanks,

Hal

New Post: DynamoDB, SimpleDB and Simol

psoukal — Sat, 21 Jan 2012 23:03:00 GMT

The object persistence was clearly missing from SimpleDB and you made it relatively straightforward.

I've been using Simol for about 6 months now and have a working solution. You have my vote to provide a drop in replacement for DynamoDB.

The automatic partitioning in DynamoDB is of interest to us, so would plan to move to Dynamo in the future, which would be made much simpler if Simol supported it.

Also, I'm not clear if DynamoDB allows indexing of more than one attribute in the way Simol does.

Cheers,

Peter

New Post: DynamoDB, SimpleDB and Simol

SoopahMan — Fri, 20 Jan 2012 19:09:25 GMT

I know you have their ear, so here's what I would suggest first:

Make Dynamo a drop-in replacement for SimpleDb. If there was a way to just toggle things in the AWS console to say you want to transition to DynamoDb, and all existing code and calls that used to store things there would now store things in Dynamo, that'd be an obviously easy transition to Dynamo. Coders could then migrate their code base gradually to the new API while the storage remains consistent.

As for interfacing with Dynamo with a library, you know my suggestion... async.

Also, apologies for not being around lately - I gave up on SimpleDb and generally use simple SQL Server setups now, whose admin chores I lament every day. Leagues faster than SimpleDb though.

New Post: DynamoDB, SimpleDB and Simol

bdm — Fri, 20 Jan 2012 17:32:44 GMT

That would be awesome! Simol has done a great job making up for SimpleDB's limitations. AWS had to get something out early, and SimpleDB was their entry. Other than perhaps cost (I don't have a simple way to compare) it's hard to see a downside to this next-generation product. I would greatly appreciate any help you could give us in regard to DynamoDB, and I'm sure that many others feel the same! Thanks Ashley!

New Post: DynamoDB, SimpleDB and Simol

ashleytate — Thu, 19 Jan 2012 16:54:18 GMT

Some of you may be wondering what Amazon's announcement of DynamoDB today means for SimpleDB and Simol.

Due to my work on Simol I've been aware of Dynamo and been given preview and private beta access by Amazon since about June of last year. Amazon will continue to support SimpleDB, but I don't expect them to advance the feature set further. (They haven't announced that position "officially" but that's the impression I got in discussing it with several Amazon employees last year.)

I spent some time investigating whether it would make sense to integrate support for Dynamo into the current Simol codebase so that Simol would support the two products side-by-side. My conclusion was that this would not provide any major benefit over porting the code to a new library that would be similar to Simol but better adapted to the fundamental differences between the two services. As a result I stopped working on new features for Simol, and started working (slowly) on a new library for Dynamo.

I would appreciate any thoughts or feedback from existing Simol users on your plans for moving to Dynamo and the desire for a new library.

Regards, Ashley

New Post: SimpleDB connections (keepalive / pooling [sort of])

mtipton — Thu, 15 Dec 2011 21:48:10 GMT

Hi again, and thanks for all of the responses, Ashley!

After looking into this more the issues that I'm seeing are actually being caused by the AWS SDK library. I hadn't looked at the source for the AWS SDK library much but after looking at it... the main problem for me is that it's retry conventions are very odd. There are just some all around aspects that could be implemented much better in the AWS SDK library.

I will switch over to a single instance and see if that helps. Even when I do change to a single instance I think these underlying issue in the ASW SDK will still be relevant. Maybe a little less relevant but it's still a concern... so for now I've made some customizations to the AWS SDK. For instance one change that we've made is to tweak the backoffs for the retries... currently if there is a timeout in the AWS SDK and it has to reconnect then it can take quite a bit of time because it does some exponential backoffs with the retries when connecting. While this type of backoff is technically good practice in general their rate of back off seems a bit ridiculous. I can't remember the exact numbers but 3 retries amount to like a 2 minute wait... which is very bad. So, I've customized the backoff to be much less drastic and it has actually helped quite a bit.

Anyway, I thought I've give a quick update on the customizations that I've made in the AWS SDK. I will also get it switched over to just use a single client instance and can give an update on that as well. It might be a few days before I can do that because I'm in a release crunch currently.

Thanks again for all the advice!

New Post: SimpleDB connections (keepalive / pooling [sort of])

AshleyTate — Thu, 15 Dec 2011 21:20:50 GMT

mtipton wrote:
At one point, I do remember that I was very seriously considering creating a pool of like 10 Simol clients. Again, can't remember why :). Lol, apparently I have a bad memory. So, I may end up doing that just to be on the safe side / just in case I was having difficulties with a single reusable Simol instance.

Keep in mind that the Simol cache is tied to your client instance. So 10 or 100 or 1000 client instances means that many copies of your cached data if caching is enabled (it is by default).

There really is no need to create multiple client instances unless you require different configurations for different types operations (consistent vs eventually consistent reads, for example).

New Post: SimpleDB connections (keepalive / pooling [sort of])

AshleyTate — Mon, 12 Dec 2011 13:43:49 GMT

Keep in mind that the AWS library automatically retries failed requests (3 times by default) before throwing an exception back to the caller. So there could be a bug in the AWS retry logic or you may want to bump up the number of retries (it's a configurable property).

New Post: SimpleDB connections (keepalive / pooling [sort of])

mtipton — Mon, 12 Dec 2011 02:07:06 GMT

Cool, thanks for the input @bdm! Yes, you are correct checking before every CRUD is sort of tedious; however, it wouldn't be hard to make your own object that just wraps the Simol client and exposes a simple public property where the getter is defined to do any necessary checks before it returns the Simol client object.

New Post: SimpleDB connections (keepalive / pooling [sort of])

bdm — Mon, 12 Dec 2011 01:25:07 GMT

I use a single client instance. As for checking before every CRUD, there has to be a better way :)

New Post: SimpleDB connections (keepalive / pooling [sort of])

mtipton — Mon, 12 Dec 2011 00:32:27 GMT

This does naturally lead into he topic that @bdm has mentioned...

If I'm reusing the same Simol client then... now at this point I'm curious and wondering what is the best way to check for a state such as "closed connections". @bdm, it seems to me that to deal with the scenario that you have mentioned you would just want to do a check to make sure the connection is open before fetching any data via the Simol client. Only thing is, I'm not aware of the "best" ways to check for an open connection and reopen it if it's closed. I'll research this a bit and post what I find. I'll check the Simol and AWS Sdk documentation of course :).

Mat

New Post: SimpleDB connections (keepalive / pooling [sort of])

mtipton — Mon, 12 Dec 2011 00:14:41 GMT

Thanks for the quick response, @Ashley! I really appreciate your help.

I think you've made me aware of our main issue... At some point, we did indeed start creating new Simol client instances for each call to SimpleDB in our code. I think we started off with only one Simol client instance but then we changed it at some point :)... apparently, we shouldn't have changed it. So, yeah, it shouldn't be hard at all for us to put it back so that it just reuses the same Simol client instance.

The only thing that I'm a bit confused about now that I'm aware of this... is my own code :)... because It's been long enough that I can't actually recall why I started creating new client instances. So, there's that. It's possible that I started creating new client instances because of something similar / along the same lines as @bdm mentioned? Truth is though, the reasoning is totally erased from my memory [lots of code written since then :) ]. @bdm, I know our issues may or may not be related and this is just speculation... but I wouldn't mind knowing if you have a single Simol client instance that is reused?... or do you always create new Simol client instances?

At one point, I do remember that I was very seriously considering creating a pool of like 10 Simol clients. Again, can't remember why :). Lol, apparently I have a bad memory. So, I may end up doing that just to be on the safe side / just in case I was having difficulties with a single reusable Simol instance.

At any rate, I think you've pointed me in the right direction and I would say that looking into the number of Simol client instances that I am creating will help my situation. I wasn't thinking about that aspect for some reason.

Simol is awesome! Thanks for your help!

Mat

New Post: SimpleDB connections (keepalive / pooling [sort of])

AshleyTate — Sat, 10 Dec 2011 21:38:19 GMT

Hi Matt,

Thank you very much for the kind feedback!

I haven't run into any issues with that myself, but I can tell you that Simol is coded to minimize the chattiness with SimpleDB (caching and batch operations whenever possible) and that all communication with SimpleDB goes through the AWS SDK for SimpleDB. You might want to spend some time looking at the configuration options in the AWS SDK and pass a custom configured AmazonSimpleDBClient instance to Simol. I don't think any http connection configuration options are exposed through the SDK so you are probably on the right track looking for .NET framework global configuration hooks to improve performance.

Also, are you creating a new Simol client instance for each call to SimpleDB? You will see much better performance in a number of areas if you keep and reuse the same client instance (it's thread-safe) and that might help you http session issues as well.

Regards,

Ashley

New Post: SimpleDB connections (keepalive / pooling [sort of])

bdm — Sat, 10 Dec 2011 21:24:17 GMT

This may be related: I run into the error "System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server" on occasion when I've left the application idle for a while. I wonder if a keepalive might apply to my situation as well.

New Post: SimpleDB connections (keepalive / pooling [sort of])

mtipton — Fri, 09 Dec 2011 22:13:49 GMT

Hi,

First, I love Simol! Great work on a refreshingly intuitive library.

One of the very few issues I have ran into with SimpleDB / Simol in general is that the HTTP / HTTPs sessions are kind of expensive to create and I've ran into trying to balance out having to wait and deal with the timeouts that might occur if a new HTTP / HTTPs session happens to be needed. So, my thoughts here immediately go to keeping a SimpleDB connections open via HTTP keepalives. This seems like sort of a natural function that a library like Simol might provide; however, I don't see indication in the code or documentation that points towards Simol providing this sorts of connection specific profiling. I also unfortunately don't see anything in the SimpleDB apis that allows managing connections at this level.

So, my reaction is that I probably am missing something. Does anyone know if I am perhaps looking in the wrong place. Is this possibly just a setting in the .Net ServicePointManager? I'm now investigating stuff like this. If it is most easily managed at the runtime level then I probably will stumble on a quick fix and it's just a matter of time before I find the magic global .Net setting. But, just wanted to put this out there in case anyone else has any insight on this and can tell me if I'm looking in the wrong places or the right places or whatever. Also, looking for advice if it is not manageable at the runtime level then any suggestions on how to manage this?

Thanks!

Mat