Andrew Rea
-
A rough mapping of a CQRS Implementation through a Sequence Diagram
So I am currently looking at an example application implementing CQRS by Mark Nijhof. You can find and check out the example application at github, http://github.com/MarkNijhof/Fohjin
This for me is a real example of where reading code improves your own outlook and skills in coding. I feel this application is a superb example of good Architecture and encapsulates all the correct principles and practices which are encouraged and implemented through out the industry. To try and understand how it all fitted together I thought I would just jump in and create a rough sequence diagram. I used some excellent free software to create the diagram called Violet Uml Editor.
Please click to see a larger version.
In a previous post I mistakenly used the word event when I actually should have written command. After studying the code base to construct the above diagram I can now make the following assumptions about it and in turn try to describe the work flow.
- On Application Start
- All Commands and CommandHandlers are resolved and added to the MessageRouter
- All Events and EventHandlers are resolved and added to the MessageRouter
- Client creates and publishes a command to the Bus
- The message router loops through all the command handlers and invokes each wrapped inside a Transaction handler.
- The Transaction Handler gets or adds the DomainObject through the DomainRepository which indirectly gets Registered For Tracking inside the UnitOfWork.
- When the Command Handler is executed, it is done so inside a wrapper. This also invokes the Commit method on the UnitOfWork.
- The UnitOfWork will then loop through each of the Domain Objects (Event Providers) it has Registered For Tracking and do the following:
-
- Save the Domain Object (Event Provider) through the Domain Event Storage.
- Publish the events which have occurred for said Domain Object to the Bus
- The Domain Objects which are Registered For Tracking are then cleared.
- When the Events are published to the Bus, the Message Router will now loop through the EventHandlers it has for each event and invoke each.
- The Event Handlers will then interact with the Reporting Repository and update the Reporting Data Store accordingly.
This is not exact by any means as you will probably pick up on immediately, but I am gradually getting my head around what is happening with this application and CQRS in general, and liking more and ore as I go :) .
With the above in mind and the fact that I want to introduce the Service Layer using Agatha, I am thinking that the RequestHandlers in Agatha will have either of the following two responsibilities:
- Create a publish a Command to the Bus
- Interact with the Reporting Repository.
1.) Will deal with all the write requests obviously and 2.) with the read ones. Once this is done, I can see a nice layer where the client is shielded from the CQRS and literally just executes Requests as normal.
Cheers for now,
Andrew
Posted by Andrew Rea on July 11 at 9:58 AMComments (0)
- On Application Start
-
CQRS and Replaying state events from a snapshot
Still reading the blog posts from Greg Young and derived works and one of the points which I have just got my head through is the scenario where you may have 100,000 + events for a particular object. Basically, to obtain the state for said object you need to load the aggregate from the persistence layer, and the only information you have to build up the object are the commands which were executed, so you have to replay them. Replaying 100,000+ events every time you need to load up an aggregate would be slow so the Memento pattern is used. This means that you can query the last Memento of the object and replay only the Commands that were executed after that time.
I really like the idea of this way of thinking. Another point that was made is that, say you get a bug in your system. Remedy for such bug should come in the form of a Command. When I read this, i kind of tilted my head slightly whilst reading it, but it makes perfect sense.
In previous blog posts I have mentioned that I use Agatha library on my service layers and with this I was thinking that I already have the entry point for the Command Handler. Of course I have the write and read requests so I would have to make sure that the write request handlers dealt with the domain model, where as the read request handlers dealt with the traditional DAO Repositories and Finders.
IDEA : I can imagine that the types of UIs which could evolve from this style of architecture will be sweet. I am picturing an interface where you could simply find an object from the read store, and when you select this object the UI will then load in the object from the transactional write store and possibly a nice timeline look-a-like with a property grid display of the object parallel to it. You could have a slide on the timeline and each time you changed the value of the slider, you could see the information in the property grid update.
Posted by Andrew Rea on July 09 at 11:08 AMComments (0)
-
Having a think about CQRS and Eventual Consistency
I have been watching and reading a few blogs and videos recently from various sources including Udi Dahan and Greg young and other blogs which have been based on those works. I am getting my head around CQRS and the different way of thinking which arises when following an Eventual Consistency approach.
One of things which I really like is the way which the write store is constructed from commands which build up a transaction log related to objects. For example I could replay a set of transactions to build up an object for a certain point in time or all the way to current state.
The whole point of CQRS (Command Query Responsibility Segregation) is to separate the writes from the reads, and making a write does not immediately change what is read. This is where the Eventual Consistency comes in. When a write command is processed, it will emit events which describe the changes which have occurred inside the write store, and it is the responsibility of the read stores to subscribe to these change events and update their read store appropriately. One way of thinking about the read store is as a reporting data store. The read store should structure and store its data in a way which is optimized for data retrieval, i.e. denormalized to the hilt.
Testing
Whilst watching the video of Greg Young on CQRS and his implementation I heard a short part which I really liked and that was related to testing. Using the BDD way of thinking (Given When Then) he explained that:
- Given a context (Forum)
- When a command is sent (CreateForum)
- Then events are raised (ForumCreated, UserForumCountChanged, ForumCategoryChanged).
A similar approach could easily be attained from an Arrange, Act and Assert approach but I do like the Given, When, Then approach.
So what he was saying is that from one command executed against the data store may result in one or more events being raised to describe what changes have taken place. This means that the events simply contain state, as the logic which was required to build such state has been handled inside the domain model. The read store simply has to store the data which it reads from the events. There should be no business logic occurring on the read databases. The fact that an entire system could be built in this manner promotes testability in many ways, and also indication of whether the system is meeting business requirements. Accepting testing would be stream lined and simplified with this approach as we can see the direct results of executing commands.
SIDENOTE: Whilst writing this a really obvious and simple idea came into my head and that was having the transaction log serialized in JSON format. I know it is simple but I am thinking that it would be really cool seeing the applications of storing the log in such a format as opposed to binary.
Integration with Service Layer and Agatha
I was thinking about integration with a service layer and how I should implement it. From my first pass I would say that it is clearly visible where each part should fit in and more than that it guards the client from the internals of what is happening. From the client I can send read and write requests to the same service, but it is at the service layer where I would execute the commands. For the service layer I use a library called Agatha and one of the components of this library is being able to send OneWayRequests which fits in nicely with the eventual consistency approach.
Going Forward
There are a couple of frameworks out there currently for CQRS in .NET including AgrCQRS which I have had a look at and it looks really good. You can read more and have access to a few valuable resources on CQRS with the following link : http://jwbs-blog.blogspot.com/2010/03/c-cqrs-framework-agrcqrs.html .
One of the topics I am currently looking into more is where you can build up an object using a partial selection of the transaction log,
Cheers for now,
Andy
References:
Posted by Andrew Rea on July 07 at 4:07 AMComments (0)
-
Consume WCF Service InProcess using Agatha and WCF
I have been looking into this lately for a specific reason. Some integration tests I want to write I want to control the types of instances which are used inside the service layer but I want that control from the test class instance. One of the problems with just referencing the service is that a lot of the time this will by default be done inside a different process. I am using StructureMap as my DI of choice and one of the tools which I am using inline with RhinoMocks is StructureMap.AutoMocking. With StructureMap the main entry point is the ObjectFactory. This will be process specific so if I decide that the I want a certain instance of a type to be used inside the ServiceLayer I cannot configure the ObjectFactory from my test class as that will only apply to the process which it belongs to.
This is were I started thinking about two things:
- Running a WCF in process
- Being able to share mocked instances across processes
A colleague in work pointed me to a project which is for the latter but I thought that it would be a better solution if I could run the WCF Service in process. One of the projects which I use when I think about WCF Services is AGATHA, and the one which I have to used to try and get my head around doing this. Another asset I have is a book called Programming WCF Services by Juval Lowy and if you have not heard of it or read it I would definately recommend it. One of the many topics that is inside this book is the type of configuration you need to communicate with a service in the same process, and it turns out to be quite simple from a config point of view.
<system.serviceModel> <services> <service name="Agatha.ServiceLayer.WCF.WcfRequestProcessor"> <endpoint address ="net.pipe://localhost/MyPipe" binding="netNamedPipeBinding" contract="Agatha.Common.WCF.IWcfRequestProcessor"/> </service> </services> <client> <endpoint name="MyEndpoint" address="net.pipe://localhost/MyPipe" binding="netNamedPipeBinding" contract="Agatha.Common.WCF.IWcfRequestProcessor"/> </client> </system.serviceModel>You can see here that I am referencing the Agatha object and contract here, but also that my binding and the address is something called Named Pipes. THis is sort of the "Magic" which makes it happen in the same process.
Next I need to open the service prior to calling the methods on a proxy which I also need. My initial attempt at the proxy did not use any Agatha specific coding and one of the pains I found was that you obviously need to give your proxy the known types which the serializer can be aware of. So we need to add to the known types of the proxy programmatically. I came across the following blog post which showed me how easy it was http://bloggingabout.net/blogs/vagif/archive/2009/05/18/how-to-programmatically-define-known-types-in-wcf.aspx.
First Pass
So with this in mind, and inside a console app this was my first pass at consuming a service in process. First here is the proxy which I made making use of the Agatha IWcfRequestProcessor contract.
public class InProcProxy : ClientBase<Agatha.Common.WCF.IWcfRequestProcessor>, Agatha.Common.WCF.IWcfRequestProcessor { public InProcProxy() { } public InProcProxy(string configurationName) : base(configurationName) { } public Agatha.Common.Response[] Process(params Agatha.Common.Request[] requests) { return Channel.Process(requests); } public void ProcessOneWayRequests(params Agatha.Common.OneWayRequest[] requests) { Channel.ProcessOneWayRequests(requests); } }So with the proxy in place I could then use this after opening the service so here is the code which I use inside the console app make the request.
static void Main(string[] args) { ComponentRegistration.Register(); ServiceHost serviceHost = new ServiceHost(typeof(Agatha.ServiceLayer.WCF.WcfRequestProcessor)); serviceHost.Open(); Console.WriteLine("Service is running...."); using (var proxy = new InProcProxy()) { foreach (var operation in proxy.Endpoint.Contract.Operations) { foreach (var t in KnownTypeProvider.GetKnownTypes(null)) { operation.KnownTypes.Add(t); } } var request = new GetProductsRequest(); var responses = proxy.Process(new[] { request }); var response = (GetProductsResponse)responses[0]; Console.WriteLine("{0} Products have been retrieved", response.Products.Count); } serviceHost.Close(); Console.WriteLine("Finished"); Console.ReadLine(); }So what I used here is the KnownTypeProvider of Agatha to easily get all the types I need for the service/proxy and add them to the proxy. My Request handler for this was just a test one which always returned 2 products.
public class GetProductsHandler : RequestHandler<GetProductsRequest,GetProductsResponse> { public override Agatha.Common.Response Handle(GetProductsRequest request) { return new GetProductsResponse { Products = new List<ProductDto> { new ProductDto{}, new ProductDto{} } }; } }Second Pass
Now after I did this I started reading up some more on some resources including more by Davy Brion and others on Agatha. Now it turns out that the work I did above to create a derived class of the ClientBase implementing Agatha.Common.WCF.IWcfRequestProcessor was not necessary due to a nice class which is present inside the Agatha code base, RequestProcessorProxy which takes care of this for you! :-)
So disregarding that class I made for the proxy and changing my code to use it I am now left with the following:
static void Main(string[] args) { ComponentRegistration.Register(); ServiceHost serviceHost = new ServiceHost(typeof(Agatha.ServiceLayer.WCF.WcfRequestProcessor)); serviceHost.Open(); Console.WriteLine("Service is running...."); using (var proxy = new RequestProcessorProxy()) { var request = new GetProductsRequest(); var responses = proxy.Process(new[] { request }); var response = (GetProductsResponse)responses[0]; Console.WriteLine("{0} Products have been retrieved", response.Products.Count); } serviceHost.Close(); Console.WriteLine("Finished"); Console.ReadLine(); }Cheers for now,
Andy
References
Agatha WCF InProcess Without WCF
StructureMap.AutoMocking
Cross Process Mocking
Agatha
Programming WCF Services by Juval LowyPosted by Andrew Rea on June 06 at 4:49 PMComments (0)
-
IDEA for a Drop Box like application to work with the Google Docs List Data API
I have spoken a couple of times to someone at work about Google Docs and they were mentioning how good it would be if there was support similar to what you have with Drop Box and of course I agreed. So it got me thinking and I have done a little, *tiny*, research just out of sheer interest.
The language I would choose
In a perfect world I would like to say I could build this cross platform application using C++ but unfortunately my skills are not at the level where I can just jump in and code. I am thinking about using either:
- Java
- .NET (With Mono in mind for X Platform)
I am not to sure of the exact deployment requirements for both Java and .NET Mono so I shall leave this point currently at the stage of I would use either.
The service
To work in a similar fashion to Drop Box we would need functionality where by we can have a simple set up to select a folder which will be sync'd, and a service running which will detect :
- File Creation
- File Update
- File Delete
I have read somewhere about meta data on the API so it would not be a bad to reflect file meta data when they change also. This is where the deployment would differ each platform has a different way of setting up and handling services of course.
The API
http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html
As you will read the API provides the necessary methods to develop core functionality we would require to reflect that of Drop Box. Once of the main things which you will need before you can get anything of the ground is Authentication. There is a section for Client Login which is what we will need for the installed Application.
Progressing Synchronization
At any stage it needs to be possible to see the current progress of any synchronization that is ongoing, complete or has failed. My thoughts for this, and I am not a GUI GURU, was having a context menu over a system icon in the task bar which might say - Synchronization Status. I would then envisage may be a three tab interface with the tab headings of:
- In Progress
- Complete
- Failed
Each tab would contain a data grid to display the file paths and I would give the following context menus.
In Progress Tab
Cancel Menu Item
Failed Tab
Retry Menu Item
I am thinking that the complete tab would simply show the last 100 completed actions or something simply for immediate reference. So because of that would not require a context menu.
Is there anything like this currently out there. I hope so, if not? Damn cool project to open source and get collaboration on, and ultimately use!!
Cheers for now,
Andrew
Posted by Andrew Rea on May 12 at 2:46 PMComments (0)
-
AutoMapper MappingFunction from Source Type of NameValueCollection
I have had a situation arise today where I need to construct a complex type from a source of a NameValueCollection. A little while back I submitted a patch for the Agatha Project to include REST (JSON and XML) support for the service contract. I realized today that as useful as it is, it did not actually support true REST conformance, as REST should support GET so that you can use JSONP from JavaScript directly meaning you can query cross domain services. My original implementation for POX and JSON used the POST method and this immediately rules out JSONP as from reading, JSONP only works with GET Requests.
This then raised another issue. The current operation contract of Agatha and one of its main benefits is that you can supply an array of Request objects in a single request, limiting the about of server requests you need to make. Now, at the present time I am thinking that this will not be the case for the REST imlementation but will yield the benefits of the fact that :
- The same Request objects can be used for SOAP and RST (POX, JSON)
- The construct of the JavaScript functions will be simpler and more readable
- It will enable the use of JSONP for cross domain REST Services
The current contract for the Agatha WcfRequestProcessor is at time of writing the following:
[ServiceContract] public interface IWcfRequestProcessor { [OperationContract(Name = "ProcessRequests")] [ServiceKnownType("GetKnownTypes", typeof(KnownTypeProvider))] [TransactionFlow(TransactionFlowOption.Allowed)] Response[] Process(params Request[] requests); [OperationContract(Name = "ProcessOneWayRequests", IsOneWay = true)] [ServiceKnownType("GetKnownTypes", typeof(KnownTypeProvider))] void ProcessOneWayRequests(params OneWayRequest[] requests); }My current proposed solution, and at the very early stages of my concept is as follows:
[ServiceContract] public interface IWcfRestJsonRequestProcessor { [OperationContract(Name="process")] [ServiceKnownType("GetKnownTypes", typeof(KnownTypeProvider))] [TransactionFlow(TransactionFlowOption.Allowed)] [WebGet(UriTemplate = "process/{name}/{*parameters}", BodyStyle = WebMessageBodyStyle.WrappedResponse, ResponseFormat = WebMessageFormat.Json)] Response[] Process(string name, NameValueCollection parameters); [OperationContract(Name="processoneway",IsOneWay = true)] [ServiceKnownType("GetKnownTypes", typeof(KnownTypeProvider))] [WebGet(UriTemplate = "process-one-way/{name}/{*parameters}", BodyStyle = WebMessageBodyStyle.WrappedResponse, ResponseFormat = WebMessageFormat.Json)] void ProcessOneWayRequests(string name, NameValueCollection parameters); }Now this part I have not yet implemented, it is the preliminart step which I have developed which will allow me to take the name of the Request Type and the NameValueCollection and construct the complex type which is that of the Request which I can then supply to a nested instance of the original IWcfRequestProcessor and work as it should normally. To give an example of some of the urls which you I envisage with this method are:
- http://www.url.com/service.svc/json/process/getweather/?location=london
- http://www.url.com/service.svc/json/process/getproductsbycategory/?categoryid=1
- http://www.url.om/service.svc/json/process/sayhello/?name=andy
Another reason why my direction has gone to a single request for the REST implementation is because of restrictions which are imposed by browsers on the length of the url. From what I have read this is on average 2000 characters. I think that this is a very acceptable usage limit in the context of using 1 request, but I do not think this is acceptable for accommodating multiple requests chained together. I would love to be corrected on that one, I really would but unfortunately from what I have read I have come to the conclusion that this is not the case.
The mapping function
So, as I say this is just the first pass I have made at this, and I am not overly happy with the try catch for detecting types without default constructors. I know there is a better way but for the minute, it escapes me. I would also like to know the correct way for adding mapping functions and not using the anonymous way that I have used. To achieve this I have used recursion which I am sure is what other mapping function use. As you do have to go as deep as the complex type is.
public static object RecurseType(NameValueCollection collection, Type type, string prefix) { try { var returnObject = Activator.CreateInstance(type); foreach (var property in type.GetProperties()) { foreach (var key in collection.AllKeys) { if (String.IsNullOrEmpty(prefix) || key.Length > prefix.Length) { var propertyNameToMatch = String.IsNullOrEmpty(prefix) ? key : key.Substring(property.Name.IndexOf(prefix) + prefix.Length + 1); if (property.Name == propertyNameToMatch) { property.SetValue(returnObject, Convert.ChangeType(collection.Get(key), property.PropertyType), null); } else if(property.GetValue(returnObject,null) == null) { property.SetValue(returnObject, RecurseType(collection, property.PropertyType, String.Concat(prefix, property.PropertyType.Name)), null); } } } } return returnObject; } catch (MissingMethodException) { //Quite a blunt way of dealing with Types without default constructor return null; } }Another thing is performance, I have not measured this in anyway, it is as I say the first pass, so I hope this can be the start of a more perfected implementation. I tested this out with a complex type of three levels, there is no intended logical meaning to the properties, they are simply for the purposes of example. You could call this a spiking session, as from here on in, now I know what I am building I would take a more TDD approach. OK, purists, why did I not do this from the start, well I didn't, this was a brain dump and now I know what I am building I can.
The console test and how I used with AutoMapper is as follows:
static void Main(string[] args) { var collection = new NameValueCollection(); collection.Add("Name", "Andrew Rea"); collection.Add("Number", "1"); collection.Add("AddressLine1", "123 Street"); collection.Add("AddressNumber", "2"); collection.Add("AddressPostCodeCountry", "United Kingdom"); collection.Add("AddressPostCodeNumber", "3"); AutoMapper.Mapper.CreateMap<NameValueCollection, Person>() .ConvertUsing(x => { return(Person) RecurseType(x, typeof(Person), null); }); var person = AutoMapper.Mapper.Map<NameValueCollection, Person>(collection); Console.WriteLine(person.Name); Console.WriteLine(person.Number); Console.WriteLine(person.Address.Line1); Console.WriteLine(person.Address.Number); Console.WriteLine(person.Address.PostCode.Country); Console.WriteLine(person.Address.PostCode.Number); Console.ReadLine(); }Notice the convention that I am using and that this method requires you do use. Each property is prefixed with the constructed name of its parents combined. This is the convention used by AutoMapper and it makes sense.
I can also think of other uses for this including using with ASP.NET MVC ModelBinders for creating a complex type from the QueryString which is itself is a NameValueCollection.
Hope this is of some help to people and I would welcome any code reviews you could give me.
References:
Agatha : http://code.google.com/p/agatha-rrsl/
AutoMapper : http://automapper.codeplex.com/
Cheers for now,
Andrew
P.S. I will have the proposed solution for a more complete REST implementation for AGATHA very soon.
Posted by Andrew Rea on May 09 at 2:48 PMComments (0)
-
A nice method in StructureMap.AutoMocking – MockObjectFactory
Recently, while writing a test for a particular unit of code, I came across a situation where the unit in question was an object which used the StructureMap.ObjectFactory.BuildUp(this) inside the constructor to resolve a SetterProperty which it had.
The easiest way I could have solved this was to simply place the dependent property as a constructor dependency instead, but due to other circumstances it was the decision that for this particular object, the dependency stayed as a property.
It is easily possible to not use this method which I am talking about, but it would mean that inside your tests you would have to clean up the ObjectFactory to ensure you left it how you found it. Using AutoMocking, the Container will instantiated with each test meaning you will not leak any configuration into another test.
The way I have set up some test code to provide an example of what
I mean is shown on the right. I will be testing the
ConcreteClassToTest which itself has a dependency
on the interface IExampleInterface . This example
is trying to show that I want to inject a Mock implementation into
a Property Dependency during test.public class ConcreteClassToTest { [StructureMap.Attributes.SetterProperty] public IExampleInterface ExampleInterface { get; set; } public ConcreteClassToTest() { ObjectFactory.BuildUp(this); } public string OutputExampleValue() { return ExampleInterface.GetStringValue(); } }This simple class once instantiated will call on the ObjectFactory to provide an implementation for the interface IExampleInterface and inject it. In my test, I have created a Mock implementation of this and called it FakeExampleInterfaceImplementation, bit long I know…
[TestFixture] public class TestConcreteClassToTest : StructureMap.AutoMocking.RhinoAutoMocker<ConcreteClassToTest> { [SetUp] public void SetUp() { MockObjectFactory(); Container.Configure(x => x.For<IExampleInterface>().Use<FakeExampleInterfaceImplementation>()); } [Test] public void Test_AutoMocker_Container_Is_Same_As_ObjectFactory() { Assert.AreSame(Container, ObjectFactory.Container); } [Test] public void Test_Fake_Implementation_is_used() { Assert.AreEqual("TEST", ClassUnderTest.OutputExampleValue()); } }I am using NUnit as my testing framework, and in my test above I am inheriting from RhinoAutoMocker, which itself relies on RhinoMocks but is implemented inside the StructureMap project, inside the AutoMocking assembly. After the first statement is executed, any call to ObjectFactory.Container will in turn be referencing the RhinoAutoMocker Container. The method itself inside the StructureMap source code is as follows:
/// <summary> /// Use this with EXTREME caution. This will replace the active "Container" in accessed /// by ObjectFactory with the AutoMockedContainer from this instance /// </summary> public void MockObjectFactory() { ObjectFactory.ReplaceManager(_container); }The actual method ReplaceManager method is internal, so you should be using the above method to achieve the same result.
internal static void ReplaceManager(Container container) { _container = container; }
Cheers for now,
Andrew
Posted by Andrew Rea on April 07 at 12:39 PMComments (0)
-
First toe in the water with Object Databases : DB4O
I have been wanting to have a play with Object Databases for a while now, and today I have done just that. One of the obvious choices I had to make was which one to use. My criteria for choosing one today was simple, I wanted one which I could literally wack in and start using, which means I wanted one which either had a .NET API or was designed/ported to .NET. My decision was between two being:
I went for db4o for the single reason that it looked like I could get it running and integrated the quickest. I am making a Blogging application and front end as a project with which I can test and learn with these object databases. Another requirement which I thought I would mention is that I also want to be able to use the said database in a shared hosting environment where I cannot install, run and maintain a server instance of said object database. I can do exactly this with db4o. I have not tried to do this with MongoDb at time of writing. There are quite a few in the industry now and you read an interesting post about different ones and how they are used with some of the heavy weights in the industry here : http://blog.marcua.net/post/442594842/notes-from-nosql-live-boston-2010
In the example which I am building I am using StructureMap as my IOC. To inject the object for db4o I went with a Singleton instance scope as I am using a single file and I need this to be available to any thread on in the process as opposed to using the server implementation where I could open and close client connections with the server handling each one respectively. Again I want to point out that I have chosen to stick with the non server implementation of db4o as I wanted to use this in a shared hosting environment where I cannot have such servers installed and run.
public static class Bootstrapper { public static void ConfigureStructureMap() { ObjectFactory.Initialize(x => x.AddRegistry(new MyApplicationRegistry())); } } public class MyApplicationRegistry : Registry { public const string DB4O_FILENAME = "blog123"; public string DbPath { get { return Path.Combine(Path.GetDirectoryName(Assembly.GetAssembly(typeof(IBlogRepository)).Location), DB4O_FILENAME); } } public MyApplicationRegistry() { For<IObjectContainer>().Singleton().Use( () => Db4oEmbedded.OpenFile(Db4oEmbedded.NewConfiguration(), DbPath)); Scan(assemblyScanner => { assemblyScanner.TheCallingAssembly(); assemblyScanner.WithDefaultConventions(); }); } }
So my code above is the structure map plumbing which I use for the application. I am doing this simply as a quick scratch pad to play around with different things so I am simply segregating logical layers with folder structure as opposed to different assemblies. It will be easy if I want to do this with any segment but for the purposes of example I have literally just wacked everything in the one assembly. You can see an example file structure I have on the right. I am planning on testing out a few implementations of the object databases out there so I can program to an interface of IBlogRepository
One of the things which I was unsure about was how it performed under a multi threaded environment which it will undoubtedly be used 9 times out of 10, and for the reason that I am using the db context as a singleton, I assumed that the library was of course thread safe but I did not know as I have not read any where in the documentation, again this is probably me not reading things correctly. In short though I threw together a simple test where I simply iterate to a limit each time kicking a common task off with a thread from a thread pool. This task simply created and added an random Post and added it to the storage.
The execution of the threads I put inside the Setup of the Test and then simply ensure the number of posts committed to the database is equal to the number of iterations I made; here is the code I used to do the multi thread jobs:
[TestInitialize] public void Setup() { var sw = new System.Diagnostics.Stopwatch(); sw.Start(); var resetEvent = new ManualResetEvent(false); ThreadPool.SetMaxThreads(20, 20); for (var i = 0; i < MAX_ITERATIONS; i++) { ThreadPool.QueueUserWorkItem(delegate(object state) { var eventToReset = (ManualResetEvent)state; var post = new Post { Author = MockUser, Content = "Mock Content", Title = "Title" }; Repository.Put(post); var counter = Interlocked.Decrement(ref _threadCounter); if (counter == 0) eventToReset.Set(); }, resetEvent); } WaitHandle.WaitAll(new[] { resetEvent }); sw.Stop(); Console.WriteLine("{0:00}.{1:00} seconds", sw.Elapsed.Seconds, sw.Elapsed.Milliseconds); }
I was not doing this to test out the speed performance of db4o but while I was doing this I could not help but put in a StopWatch and see out of sheer interest how fast it would take to insert a number of Posts. I tested it out in this case with 10000 inserts of a small, simple POCO and it resulted in an average of:
899.36 object inserts / second.
Again this is just simple crude test which came out of my curiosity at how it performed under many threads when using the non server implementation of db4o. The spec summary of the computer I used is as follows:
With regards to the actual Repository implementation itself, it really is quite straight forward and I have to say I am very surprised at how easy it was to integrate and get up and running. One thing I have noticed in the exposure I have had so far is that the Query returns IList<T> as opposed to IQueryable<T> but again I have not looked into this in depth and this could be there already and if not they have provided everything one needs to make there own repository. An example of a couple of methods from by db4o implementation of the BlogRepository is below:
public class BlogRepository : IBlogRepository { private readonly IObjectContainer _db; public BlogRepository(IObjectContainer db) { _db = db; } public void Put(DomainObject obj) { _db.Store(obj); } public void Delete(DomainObject obj) { _db.Delete(obj); } public Post GetByKey(object key) { return _db.Query<Post>(post => post.Key == key).FirstOrDefault(); }…
Anyways I hope to get a few more implementations going of the object databases and literally just get familiarized with them and the concept of no sql databases.
Cheers for now,
Andrew
UPDATE 2010-03-18
I feel a bit stupid here as the above is great and all, but if I do not call commit on the database then the values will remain in RAM, hence the speed I reported above. In actual fact I reduced the number of inserts I am testing against to 1000 and each insert on each thread I make I also commit the object. So each thread does the following:
- Creates a Post
- Saves the Post
- Commits the changes to the file.
With the addition of adding the commit to the job the actual performance works out on the same machine spec as above as:
33.2 object insert and commit / second
I have added a Commit method onto the IBlogRepository and implement this in the concrete.
public void Commit() { _db.Commit(); }
I will also point out that omitting the commit is also great for tests where you have then an in memory implementation of your data store, where in the past when developing against an RDBMS I would use Sqlite for such a thing.
Posted by Andrew Rea on March 15 at 5:15 PMComments (0)
-
ASP.NET MVC 2 : The view 'Index' or its master was not found
I was literally setting a new project yesterday to have a play with the released version of ASP.NET MVC 2 and came across a subtle couple of things which when you know, like everything, is easy.
I am creating a small test application which is another blog one, but this time I also want to use an Object DB or many, the first being MONGO, but anyway. I added a new Area to my project being the Blog area, added the controller and a view and hit F5.
Because my area was called Blog, naturally I want the controller to be called BlogController. The cause and the solution are inside the AreaRegistration file. When you create your new area you should notice your default route inside this registration file and notice there is no default controller specified, which is correct because at time of creation you will not have a controller inside the area.
public override void RegisterArea(AreaRegistrationContext context) { context.MapRoute( "Blog_default", "Blog/{controller}/{action}/{id}", new { action = "Index", id = "" } ); }So add in the default controller and everything is back to normal. Also, if you do not add this controller and for example add a break point to the return View() of the Index Action on your controller, you will see that it does get hit, it is just the locating of the View which fails. I am sure there is more plumbing that can be done to enable the location of these views without explicitly stating the default controller but at time of writing I do not know.
public override void RegisterArea(AreaRegistrationContext context) { context.MapRoute( "Blog_default", "Blog/{controller}/{action}/{id}", new {controller="Blog", action = "Index", id = "" } ); }Cheers for now,
Andrew
Posted by Andrew Rea on March 14 at 6:48 AMComments (0)
-
Compiling examples for consuming the REST Endpoints for WCF Service using Agatha
I recently made two contributions to the Agatha Project by Davy Brion over on Google Code, and one of the things I wanted to follow up with was a post showing examples and some, seemingly required tid bits. The contributions which I made where:
- To support StructureMap
- To include REST (JSON and XML) support for the service contract
The examples which I have made, I want to format them so they fit in with the current format of examples over on Agatha and hopefully create and submit a third patch which will include these examples to help others who wish to use these additions.
Whilst building these examples for both XML and JSON I have learnt a couple of things which I feel are not really well documented, but are extremely good practice and once known make perfect sense. I have chosen a real basic e-commerce context for my example Requests and Responses, and have also made use of the excellent tool AutoMapper, again on Google Code.
Setting the scene
I have followed the Pipes and Filters Pattern with the IQueryable interface on my Repository and exposed the following methods to query Products:
IQueryable<Product> GetProducts();
IQueryable<Product> ByCategoryName(this IQueryable<Product> products, string categoryName)
Product ByProductCode(this IQueryable<Product> products, String productCode)
I have an interface for the IProductRepository but for the concrete implementation I have simply created a protected getter which populates a private List<Product> with 100 test products with random data. Another good reason for following an interface based approach is that it will demonstrate usage of my first contribution which is the StructureMap support. Finally the two Domain Objects I have made are Product and Category as shown below:
public class Product { public String ProductCode { get; set; } public String Name { get; set; } public Decimal Price { get; set; } public Decimal Rrp { get; set; } public Category Category { get; set; } }public class Category { public String Name { get; set; } }Requirements for the REST Support
One of the things which you will notice with Agatha is that you do not have to decorate your Request and Response objects with the WCF Service Model Attributes like DataContract, DataMember etc… Unfortunately from what I have seen, these are required if you want the same types to work with your REST endpoint. I have not tried but I assume the same result can be achieved by simply decorating the same classes with the Serializable Attribute. Without this the operation will fail.
Another surprising thing I have found is that it did not work until I used the following Attribute parameters:
- Name
- Namespace
e.g.
[DataContract(Name = "GetProductsRequest", Namespace = "AgathaRestExample.Service.Requests")] public class GetProductsRequest : Request { }Although I was surprised by this, things kind of explained themselves when I got round to figuring out the exact construct required for both the XML and the REST. One of the things which you already know and are then reminded of is that each of your Requests and Responses ultimately inherit from an abstract base class respectively. This information needs to be represented in a way native to the format being used. I have seen this in XML but I have not seen the format which is required for the JSON.
JSON Consumer Example
I have used JQuery to create the example and I simply want to make two requests to the server which as you will know with Agatha are transmitted inside an array to reduce the service calls. I have also used a tool called json2 which is again over at Google Code simply to convert my JSON expression into its string format for transmission. You will notice that I specify the type of Request I am using and the relevant Namespace it belongs to. Also notice that the second request has a parameter so each of these two object are representing an abstract Request and the parameters of the object describe it.
<script type="text/javascript"> var bodyContent = $.ajax({ url: "http://localhost:50348/service.svc/json/processjsonrequests", global: false, contentType: "application/json; charset=utf-8", type: "POST", processData: true, data: JSON.stringify([ { __type: "GetProductsRequest:AgathaRestExample.Service.Requests" }, { __type: "GetProductsByCategoryRequest:AgathaRestExample.Service.Requests", CategoryName: "Category1" } ]), dataType: "json", success: function(msg) { alert(msg); } }).responseText; </script>XML Consumer Example
For the XML Consumer example I have chosen to use a simple Console Application and make a WebRequest to the service using the XML as a request. I have made a crude static method which simply reads from an XML File, replaces some value with a parameter and returns the formatted XML. I say crude but it simply shows how XML Templates for each type of Request could be made and then have a wrapper utility in whatever language you use to combine the requests which are required. The following XML is the same Request array as shown above but simply in the XML Format.
<?xml version="1.0" encoding="utf-8" ?> <ArrayOfRequest xmlns="http://schemas.datacontract.org/2004/07/Agatha.Common" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"> <Request i:type="a:GetProductsRequest" xmlns:a="AgathaRestExample.Service.Requests"/> <Request i:type="a:GetProductsByCategoryRequest" xmlns:a="AgathaRestExample.Service.Requests"> <a:CategoryName>{CategoryName}</a:CategoryName> </Request> </ArrayOfRequest>It is funny because I remember submitting a question to StackOverflow asking whether there was a REST Client Generation tool similar to what Microsoft used for their RestStarterKit but which could be applied to existing services which have REST endpoints attached. I could not find any but this is now definitely something which I am going to build, as I think it is extremely useful to have but also it should not be too difficult based on the information I now know about the above. Finally I thought that the Strategy Pattern would lend itself really well to this type of thing so it can accommodate for different languages.
I think that is about it, I have included the code for the example Console app which I made below incase anyone wants to have a mooch at the code. As I said above I want to reformat these to fit in with the current examples over on the Agatha project, but also now thinking about it, make a Documentation Web method…{brain ticking} :-)
Cheers for now and here is the final bit of code:
static void Main(string[] args) { var request = WebRequest.Create("http://localhost:50348/service.svc/xml/processxmlrequests"); request.Method = "POST"; request.ContentType = "text/xml"; using(var writer = new StreamWriter(request.GetRequestStream())) { writer.WriteLine(GetExampleRequestsString("Category1")); } var response = request.GetResponse(); using(var reader = new StreamReader(response.GetResponseStream())) { Console.WriteLine(reader.ReadToEnd()); } Console.ReadLine(); } static string GetExampleRequestsString(string categoryName) { var data = File.ReadAllText(Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "ExampleRequests.xml")); data = data.Replace("{CategoryName}", categoryName); return data; } }Posted by Andrew Rea on March 09 at 4:43 PMComments (0)
