Andrew Rea

I have been wanting to have a play with Object Databases for a while now, and today I have done just that.  One of the obvious choices I had to make was which one to use.  My criteria for choosing one today was simple, I wanted one which I could literally wack in and start using, which means I wanted one which either had a .NET API or was designed/ported to .NET.  My decision was between two being:

• db4o
• MongoDb

I went for db4o for the single reason that it looked like I could get it running and integrated the quickest.  I am making a Blogging application and front end as a project with which I can test and learn with these object databases.  Another requirement which I thought I would mention is that I also want to be able to use the said database in a shared hosting environment where I cannot install, run and maintain a server instance of said object database.  I can do exactly this with db4o. I have not tried to do this with MongoDb at time of writing.  There are quite a few in the industry now and you read an interesting post about different ones and how they are used with some of the heavy weights in the industry here : http://blog.marcua.net/post/442594842/notes-from-nosql-live-boston-2010

In the example which I am building I am using StructureMap as my IOC.  To inject the object for db4o I went with a Singleton instance scope as I am using a single file and I need this to be available to any thread on in the process as opposed to using the server implementation where I could open and close client connections with the server handling each one respectively.  Again I want to point out that I have chosen to stick with the non server implementation of db4o as I wanted to use this in a shared hosting environment where I cannot have such servers installed and run.

    public static class Bootstrapper
    {
        public static void ConfigureStructureMap()
        {
            ObjectFactory.Initialize(x => x.AddRegistry(new MyApplicationRegistry()));
        }
    }

    public class MyApplicationRegistry : Registry
    {
        public const string DB4O_FILENAME = "blog123";

        public string DbPath
        {
            get
            {
                return Path.Combine(Path.GetDirectoryName(Assembly.GetAssembly(typeof(IBlogRepository)).Location), DB4O_FILENAME);
            }
        }

        public MyApplicationRegistry()
        {
            For<IObjectContainer>().Singleton().Use(
                () => Db4oEmbedded.OpenFile(Db4oEmbedded.NewConfiguration(), DbPath));

            Scan(assemblyScanner =>
            {
                assemblyScanner.TheCallingAssembly();
                assemblyScanner.WithDefaultConventions();
            });
        }
    }

So my code above is the structure map plumbing which I use for the application.  I am doing this simply as a quick scratch pad to play around with different things so I am simply segregating logical layers with folder structure as opposed to different assemblies.  It will be easy if I want to do this with any segment but for the purposes of example I have literally just wacked everything in the one assembly.  You can see an example file structure I have on the right.  I am planning on testing out a few implementations of the object databases out there so I can program to an interface of IBlogRepository

One of the things which I was unsure about was how it performed under a multi threaded environment which it will undoubtedly be used 9 times out of 10, and for the reason that I am using the db context as a singleton, I assumed that the library was of course thread safe but I did not know as I have not read any where in the documentation, again this is probably me not reading things correctly.  In short though I threw together a simple test where I simply iterate to a limit each time kicking a common task off with a thread from a thread pool.  This task simply created and added an random Post and added it to the storage.

The execution of the threads I put inside the Setup of the Test and then simply ensure the number of posts committed to the database is equal to the number of iterations I made; here is the code I used to do the multi thread jobs:

        [TestInitialize]
        public void Setup()
        {
            var sw = new System.Diagnostics.Stopwatch();
            sw.Start();
            var resetEvent = new ManualResetEvent(false);
            ThreadPool.SetMaxThreads(20, 20);
            for (var i = 0; i < MAX_ITERATIONS; i++)
            {
                ThreadPool.QueueUserWorkItem(delegate(object state)
                                                 {
                                                     var eventToReset = (ManualResetEvent)state;
                                                     var post = new Post
                                                                    {
                                                                        Author = MockUser,
                                                                        Content = "Mock Content",
                                                                        Title = "Title"
                                                                    };
                                                     Repository.Put(post);

                                                     var counter = Interlocked.Decrement(ref _threadCounter);
                                                     if (counter == 0)
                                                         eventToReset.Set();

                                                 }, resetEvent);
            }

            WaitHandle.WaitAll(new[] { resetEvent });
            sw.Stop();
            Console.WriteLine("{0:00}.{1:00} seconds", sw.Elapsed.Seconds, sw.Elapsed.Milliseconds);
        }

I was not doing this to test out the speed performance of db4o but while I was doing this I could not help but put in a StopWatch and see out of sheer interest how fast it would take to insert a number of Posts.  I tested it out in this case with 10000 inserts of a small, simple POCO and it resulted in an average of:

 899.36 object inserts / second. 

Again this is just  simple crude test which came out of my curiosity at how it performed under many threads when using the non server implementation of db4o. The spec summary of the computer I used is as follows:

With regards to the actual Repository implementation itself, it really is quite straight forward and I have to say I am very surprised at how easy it was to integrate and get up and running.  One thing I have noticed in the exposure I have had so far is that the Query returns IList<T> as opposed to IQueryable<T> but again I have not looked into this in depth and this could be there already and if not they have provided everything one needs to make there own repository.  An example of a couple of methods from by db4o implementation of the BlogRepository is below:

    public class BlogRepository : IBlogRepository
    {
        private readonly IObjectContainer _db;

        public BlogRepository(IObjectContainer db)
        {
            _db = db;
        }

        public void Put(DomainObject obj)
        {
            _db.Store(obj);
        }

        public void Delete(DomainObject obj)
        {
           _db.Delete(obj);
        }

        public Post GetByKey(object key)
        {
            return _db.Query<Post>(post => post.Key == key).FirstOrDefault();
        }

…

Anyways I hope to get a few more implementations going of the object databases and literally just get familiarized with them and the concept of no sql databases.

Cheers for now,

Andrew

UPDATE 2010-03-18

I feel a bit stupid here as the above is great and all, but if I do not call commit on the database then the values will remain in RAM, hence the speed I reported above.  In actual fact I reduced the number of inserts I am testing against to 1000 and each insert on each thread I make I also commit the object.  So each thread does the following:

1. Creates a Post
2. Saves the Post
3. Commits the changes to the file.

With the addition of adding the commit to the job the actual performance works out on the same machine spec as above as:

33.2 object insert and commit / second

I have added a Commit method onto the IBlogRepository and implement this in the concrete.

        public void Commit()
        {
            _db.Commit();
        }

I will also point out that omitting the commit is also great for tests where you have then an in memory implementation of your data store, where in the past when developing against an RDBMS I would use Sqlite for such a thing.

I was literally setting a new project yesterday to have a play with the released version of ASP.NET MVC 2 and came across a subtle couple of things which when you know, like everything, is easy.

I am creating a small test application which is another blog one, but this time I also want to use an Object DB or many, the first being MONGO, but anyway.  I added a new Area to my project being the Blog area, added the controller and a view and hit F5.

Because my area was called Blog, naturally I want the controller to be called BlogController.  The cause and the solution are inside the AreaRegistration file.  When you create your new area you should notice your default route inside this registration file and notice there is no default controller specified, which is correct because at time of creation you will not have a controller inside the area.

        public override void RegisterArea(AreaRegistrationContext context)
        {
            context.MapRoute(
                "Blog_default",
                "Blog/{controller}/{action}/{id}",
                new { action = "Index", id = "" }
            );
        }

So add in the default controller and everything is back to normal.  Also, if you do not add this controller and for example add a break point to the return View() of the Index Action on your controller, you will see that it does get hit, it is just the locating of the View which fails.  I am sure there is more plumbing that can be done to enable the location of these views without explicitly stating the default controller but at time of writing I do not know. 

        public override void RegisterArea(AreaRegistrationContext context)
        {
            context.MapRoute(
                "Blog_default",
                "Blog/{controller}/{action}/{id}",
                new {controller="Blog", action = "Index", id = "" }
            );
        }

Cheers for now,

Andrew

Today in work I had a requirement where I wanted to output binary content to the response output stream.  I also want to stay with the MVC Controller for serving this content and so wanted a result I could add my data to and return it.  To my knowledge there is a ContentResult, but with this the Content property is of type string, and is not what I wanted.  I this particular case, I am physically GZipping content and then writing out to the Response.OutputStream. 

Previous to this I have been seeing and using examples where you assign a GZipOutputStream as a Filter on the Response dynamically.  For some strange reason on the Live environment the header and filter where being ignored on the response yet on the test, builds and sandbox everything worked as expected and the dynamic resources got GZipped.

So I had to go another way and what I came up with was to physically GZip the output content and send it to the response with the accompanying header.  I cannot see why this would not work on the server as the content is physically being served already GZipped opposed to applying a filter which can obviously be removed somewhere in the pipeline.  Any way after some reading I found some evidence that a library, http://www.icsharpcode.net/OpenSource/SharpZipLib/ , yields far better results than the out of the box one from .NET, so for this example I have a dependency on this assembly.  Apart from that I have added a couple of properties which allow for the conditional Compression and also a list of headers to give a little more flexibility.  So here is the code:

    /// <summary>
    /// A content result which can accept binart data and will write to the output
    /// stream.  If GZip is set to true the content will be GZipped and the relevant
    /// header added to the response HTTP Headers
    /// </summary>
    public class BinaryContentResult : ActionResult
    {
        public byte[] Data { get; set; }
        public NameValueCollection Headers { get; set; }
        public bool Gzip { get; set; }


        public override void ExecuteResult(ControllerContext context)
        {
            foreach (string s in Headers.Keys)
            {
                context.HttpContext.Response.AddHeader(s, Headers[s]);
            }

            if (Gzip)
            {
                using (var os = new GZipOutputStream(context.HttpContext.Response.OutputStream))
                {
                    os.Write(Data, 0, Data.Length);
                }
                context.HttpContext.Response.AddHeader("Content-Encoding", "gzip");
                context.HttpContext.Response.AddHeader("X-Compressed-By", "Custom-Compressor");
            }
            else
            {
                context.HttpContext.Response.BinaryWrite(Data);
            }

            context.HttpContext.Response.End();
        }
    }

I had a requirement on a small home project I am working on, where I need to add some more information to a content result than was there.  More specifically I had to add the status code and also have access to the response header collection.  I did come across an interesting thing with this, where by accessing and adding to the response header collection in a certain way will actually raise an exception and inform you that this is only support when IIS Integrated Pipeline is enabled.  I did some looking about and found a post by Phil Haack showing how he had achieved what I was looking for when he made the Download result class. 

So the following causes an exception:

public override void ExecuteResult(ControllerContext context)
        {
            context.HttpContext.Response.StatusCode = (int)StatusCode;
            foreach (string s in Headers.Keys)
            {
                context.HttpContext.Response.Headers.Add(s,Headers[s]);
            }
            base.ExecuteResult(context);
        }

And the following works fine, notice the subtle difference in how the header is added now:

        public override void ExecuteResult(ControllerContext context)
        {
            context.HttpContext.Response.StatusCode = (int)StatusCode;
            foreach (string s in Headers.Keys)
            {
                context.HttpContext.Response.AddHeader(s,Headers[s]);
            }
            base.ExecuteResult(context);
        }

So the full code implementation of the result is below. 

 public class ExtendedContentResult : ContentResult
    {
        public NameValueCollection Headers { get; set; }
        public HttpStatusCode StatusCode { get; set; }

        public ExtendedContentResult()
        {
            Headers = new NameValueCollection();
        }

        public override void ExecuteResult(ControllerContext context)
        {
            context.HttpContext.Response.StatusCode = (int)StatusCode;
            foreach (string s in Headers.Keys)
            {
                context.HttpContext.Response.AddHeader(s,Headers[s]);
            }
            base.ExecuteResult(context);
        }
    }

I am currently deep into TDD with ASP.NET MVC and Moq as the mocking framework I have chosen.  I have been pondering on some valid methods in order to integrate both page index and also page size into the request. Page index and page size will be unique to each action for the purposes of this blog post and example.  I wanted to have the page index in the url, but the actual page size stored in session.  I have seen in some demonstration apps that the page size has been made constant inside the actual action and unfortunately I feel this is no good as a nice UI feature is being able to see more items per page of a list of items. 

So in the session I have put it and so I wanted to test this out and check that I am infact getting the expected results.  First off I made a simple contract for the paged data called IPagedModel and also an implementing base class called PagedModel:

    public interface IPagedModel
    {
        int TotalCount { get; }
        int TotalPages { get; }
        int PageIndex { get; }
        int PageSize { get; }
    }

    public class PagedModel : IPagedModel
    {
        private int _totalCount;
        private int _pageIndex;
        private int _pageSize;

        public PagedModel(int pageIndex, int pageSize, int totalCount)
        {
            _totalCount = totalCount;
            _pageIndex = pageIndex;
            _pageSize = pageSize;
        }

        #region IPagedModel Members

        public int TotalCount
        {
            get { return _totalCount; }
        }

        public int TotalPages
        {
            get { return (int)Math.Ceiling((decimal)_totalCount / (decimal)_pageSize); }
        }

        public int PageIndex
        {
            get { return _pageIndex; }
        }

        public int PageSize
        {
            get { return _pageSize; }
        }

        #endregion
    }

And the name of the test is this:

[TestMethod]
public void IndexAction_For_Space_1_Page_2_PageSize_2_Should_Have_PageIndex_1_PageSize_2_TotalCount_9()

I want to submit the required page number to the controller using a base 1 index and then using it inside the controller action using a zero based index. The method for the Index action which I will be testing expects an id for the object which it will be targeting but also a page number which is a nullable type in case it is not found inside the url and so will default to 0.  It is in this action where I am using the Session object to get the desired page size for the action. 

        public ActionResult Index(int id, int? pageIndex)
        {
            if (pageIndex == null)
                pageIndex = 0;

            if (pageIndex > 0)
                pageIndex = pageIndex - 1;

            int pageSize = Session["SpaceController!Index!PageSize"] == null ? 10 : Convert.ToInt32(Session["SpaceController!Index!PageSize"]);

            var space = _repository.GetSpace(id);
            if (space == null)
                return View("NotFound");

            long count;

            var forums = _repository.Get(space, pageIndex ?? 0, pageSize, out count);

            return View(new SpaceIndexViewModel(forums, pageIndex ?? 0, pageSize, (int)count));
        }

First thing is to check if the pageIndex is null and if so make it 0.  Next we want to make the conversion from a base 1 index to a base zero index which we will then use against the repository’s paging method, so basically if the page index is greater than 0, we want to minus 1 from it and use that.  We then check the session object for a valid pageSize for the action, specific to the controller.  From there on in I perform repository specific code and send the model onto the view.  To test this I want to be able to mock the controller context but also the session state and also I want to control the value return when a get on the session value "SpaceController!Index!PageSize" is made.  From looking at the Nerd Dinner application I have made a controller method to return me an instance on a controller, a test repository together with pre made test data I know and expect.  I have extended the method slightly so that I can supply both a username to indicate the HttpContext is authenticated but also session name value pairs.  I have used the Pair object for this but the NameValuePair would probably have been better.

        SpaceController CreateSpaceControllerAs(string userName, List<Pair> sessionValues)
        {
            var mock = new Mock<ControllerContext>();
            var mockSession = new Mock<HttpSessionStateBase>();
            mock.Setup(cts => cts.HttpContext.Session).Returns(mockSession.Object);
            mock.SetupGet(p => p.HttpContext.User.Identity.Name).Returns(userName);
            mock.SetupGet(p => p.HttpContext.Request.IsAuthenticated).Returns(true);

            foreach (Pair sessionValue in sessionValues)
            {
                mock.SetupGet(p => p.HttpContext.Session[sessionValue.First as string]).Returns(sessionValue.Second);
            }

            // Arrange
            var controller = CreateSpaceController();
            controller.ControllerContext = mock.Object;

            return controller;
        }

This tells the controller context to use the mock session object and also to use the supplied values for the expected get calls.  The actual test then consumes this builder method like so.

        [TestMethod]
        public void IndexAction_For_Space_1_Page_2_PageSize_2_Should_Have_PageIndex_1_PageSize_2_TotalCount_9()
        {
            var controller = CreateSpaceControllerAs(FakeForumData.FakeUser.UserName, new List<Pair>(new[]{
                 new Pair{
                      //Session Name
                      First = "SpaceController!Index!PageSize",
                      //Session Value
                      Second = 2
                 }
            }));
            // Act
            //Get first page
            ViewResult result = (ViewResult)controller.Index(1,2);

            IPagedModel model = (IPagedModel)result.ViewData.Model;

            // Assert
            Assert.AreEqual(1, model.PageIndex);
            Assert.AreEqual(2, model.PageSize);
            Assert.AreEqual(9, model.TotalCount);
        }

I am not saying I think this is the best method for storing varying page size personalization by the user, but it is something which has got me mocking the session state successfully so I am happy with that!  I hope this is of some use to others making their way though ASP.NET MVC!

Cheers

Andrew