CQRS Experiences So Far

So, I'm always looking for new tech, methodologies, anything that can help me do things better. The latest approach I've gone for is CQRS; that is Command-Query Responsibility Segregation.

The first hurdle is in figuring out exactly what CQRS is. As defined, it's pretty simple. Commands are on one object, Queries are on another. However, much of the information out there is inter-mixed with a lot of other architectural concepts and strategies that fit well with CQRS, like messaging and event sourcing. It gets pretty tough to see the big picture with so many concept details in the mix.

Eventually, I settled (perhaps wrongly) on a CQRS/ES/Messaging architecture for an enterprise web app. I say wrongly because the app is already very data-centric, and the users are accustomed to it being that way. However, the data-centric nature of the app had some major drawbacks, such as inscrutable business logic code, business logic being pushed to the UI (due to complex inter-relations and validation between fields on the record), and so forth.

The first hurdle I ran into was how to communicate commands and events. For this web app, a typical message bus (MSMQ or RabbitMQ) wasn't needed, at least not for now. Not only that, but this app is already a configuration nightmare that I didn't need to complicate further. I experimented with receiving commands through WCF (which I abandoned due to the difficulty of calling from Javascript), then .NET 4.5 Web API (which I abandoned because it's in beta and I ran into technical problems with it).

For commands, I settled on using vanilla MVC 3 controllers with a custom Model Binder to convert POST content to .NET command objects. (Custom because the MVC action argument is an interface, and the commands have neither a default constructor nor public setters. So the only way to instantiate the object is through reflection.) On the other side of the coin, I wrote a converter to turn all the .NET commands into Javascript objects so they were available in a Javascript library to the client and also had conversion hints (like the type's .NET full name). The library also includes some basic shortcut methods to execute commands, so the client doesn't manually call $.ajax(...). Instead, they call Command.Execute(myCommand) or Command.Validate(myCommand). I tried to use T4 to generate the library from the .NET command objects, but found it to be inadequate (due to locking the dll among other things). Instead, I have a separate controller which generates and returns the Javascript library when called. All this sounds complicated, but it actually wasn't that time consuming to develop. The time consuming part was researching and evaluating WCF, Web API, T4 before abandoning them.

For events, I also didn't need a message bus for now. Instead, I settled on an in-memory bus of sorts. Instead of the event handler applications subscribing to the event bus, I have an event publisher object that searches for event handlers using reflection (much the same way that my command dispatcher searches for command handlers), and adds their actions to the thread pool when an event is received.

I can't do full event sourcing just yet with this application since it has an existing database that is currently in use, and I'm not able to take 3 years to do a complete re-work. Instead, I have to integrate the new stuff while retaining the existing functionality of what hasn't yet been replaced. What I did do was in my domain repository, I convert the database information (in DataRow format) into an big Created event so that my domain objects don't have any special "load" code paths. The constructor just takes events as arguments to replay them. Hopefully that will make moving to full event sourcing seamless on the domain side.

The read model is also implemented as an MVC controller with get actions for each read model. I tried to do a WCF data service (odata), but our database was just too non-standard to generate an Entity Data Model without a lot of manual XML manipulation. And if I'm going to have to manually generate the data classes anyway, I didn't see the point of being constrained to whatever WCF data service requires / provides.

The UI has it's own ecosystem, since it uses HTML5 / Javascript and MVVM. But basically, it's job is to take the user's choices and make a command out of it. The UI uses various read models to guide the user's choices (e.g. to populate a drop-down list).

So here is an end-to-end look at what happens in this system.

• The UI calls various read models and ultimately generates a command and sends it to the command API (via a jQuery $.ajax POST with the Javascript command object as the 'data' parameter).
• The command API constructs the .NET command object from the POST data (using a custom model binder), then sends the command to the dispatcher.
• The dispatcher looks up the handler for the given command and then calls it with the command
• Note, the command also has a validate method which is called before the command handler is exercised. If the validate method returns an error, execution halts and the validation error it is passed back to the client. Otherwise command execution proceeds.
• Note, domain exceptions are captured here and returned to the client as command errors with the domain exception message.
• The command handler loads the necessary domain aggregate and calls the appropriate method(s) on the it.
• The aggregate performs business logic and (perhaps) generates events.
• The command handler polls the aggregate for events and then passes those events to the event dispatcher. (No event store to pass them to yet.)
• The event dispatcher looks up the handlers for those events and puts them on the thread pool to run.
• At this point, command execution returns to the client as successful since there were no domain exceptions along the way.
• Example event handlers execution process:
• Database
• The database event handler runs. It handles ALL events. It looks for a SQL embedded resource file that matches the event name. If no resource file is found, the event is ignored.
• If a matching resource is found, it's contents are loaded as the query string.
• The event's properties are extracted as name/value pairs and used as parameters to the query.
• The query is executed.
• Note: In this way, the only thing necessary to add a database integration for new event is to add an new .sql embedded resource file to the solution with the event's name. The .sql file contains the query to run when the event is handled.

So, as you may notice, my commands are NOT asynchronous, fire and forget, or even queued. They return back to the UI either Success or Error with Message. My architectural choice was to be able to immediately get the command result back to the user with domain errors. This design lacks presently concurrency handling, but the existing system also lacks this and doesn't suffer much for it. Once a command executes successfully, the event handlers do operate asynchronously. At first I was concerned that the database writes wouldn't happen fast enough to display to the user on the next page hit, but this turned out not to be a problem so far.

So a couple of properties I have noted about this design. My first observation is that there are a lot of steps to add a feature (not even counting tests).

• Generate the command and event
• Add what's necessary to the read model to support the UI
• (maybe) create queries
• (maybe) add to read model
• Add what's necessary to the UI to generate the command
• Views
• Models
• UI Logic
• Add the aggregate method for the domain logic
• Add any needed value or entity objects
• Extend repositories as needed
• Add the aggregate Apply method for the event
• Add the command handler method
• Add the event handler methods / resources for all the integration pieces

"A lot of steps" seems like a pure disadvantage, but the disconnectedness of each program area means that multiple people truly can work on the same feature independently. For a lone developer doing features in a vertical slice, this design seems like a lot of ceremony and context switching. But for a team, I think it chops up the work in a horizontal fashion rather cleanly. The separation points seem to be around commands, events, and the read model. 

Since I am the lone developer on this for the moment, I've found that this design breaks most tasks up to the point where they are somewhat boring, and I end up copying and pasting a lot (especially with regard to commands and events). That could just be the subject matter, though. The interesting parts for me have been in the development of the architecture. The domain holds some challenge at times, but mainly the challenge there is in defining what is needed for the workflow. Sagas can be pretty interesting.

Performance is pretty good in my estimation. With a debug build, I am able to get about 1000 ops/s. Were I to batch operations on the same aggregate instead of loading it from the database for each command, that would probably improve. But at the moment I don't see a need to do that.

MMO Structures That Need to Go Away Pt 1

Servers
The concept of servers (a.k.a. shards or realms) is an antiquated concept that hearkens back to the first days of MMOs when they figured out that 1 server could not handle every player at once.

Servers are a dividing point between players. They create issues of population problems (perceived and real) and server transfers. Having everyone unified together only creates issues for the game designer, not for the player. Some issues that come to mind: Name collisions and Auction House performance or segregation.


One game that has done this server-less concept more or less successfully over the years is Eve Online. In Eve Online, each "zone" (solar system in Eve, planet in SWTOR, zone in other games) is a server, so when you change zones, you are switching servers, and every player currently in that zone is on the same server. Auction Houses are divided across regions so that it's not one massive AH, and to provide opportunities of buying low and selling high across regions. Names in Eve can have 2 names (e.g. "John Smith"), which provides a larger variety than a single-name system. However, zone overpopulation is a real problem and bogs down performance in Eve (e.g. Jita on a Sunday afternoon).


This works pretty well for the type of game that Eve is, but the implementation could be different for other games. In SWTOR for example, each server could house all the planets in one instance. So, Instance 1 of Carrick Station is on the same server as Instance 1 of Coruscant. The difference being that players could change instances on the fly like we currently can when we are grouped with other players in different instances. Several instances could then be in a "collection", which would share the same GTN (auction house) and general chat channels. That would avoid a game-wide, super-massive, slow-performing auction house or chat, while also providing entrepreneurs with some gameplay options. When you log into the game each time, the game could check your friends and guild lists and place you in the same instance with them (or at least the same collection). Collections could also be dynamic in that they automatically scale up or down based on how many people are playing at that moment, so that the zones never appear too sparsely populated. In this type of model, the servers are not tied to your character. I believe this type of model could be more easily migrated to by games which are currently using a typical server model. Name collision could be handled by a random 4-digit code like we've seen with Real ID, Steam, and others. You could still send tells without the 4-digit code if the person you are sending to's name was unique to inside your guild list, friend's list, instance, collection, or just online people. Or perhaps you could send a tell to [Name] [Legacy Name]. In other words, alternatives exist to mitigate name collision. I know I really hate it when my name is taken, and it often is.

MySQL Shortcomings

I have been using MySQL with ASP .NET at work for about the last 3 years now.

I've been stewing on this for quite some time, hoping MySQL would release changes and fixes. As it stands, I cannot recommend MySQL's use to anyone. Even simple sites have a way of becoming complex over time, so don't think you won't eventually run into these very serious problems.

As of this writing, I'm using 5.0 as it's what comes packaged with our linux system. The current version is 5.1. However, these problems or deficiencies are on both versions.

Here are the main issues that I run into, which are ridiculous oversights:

• (Update 2011.03.15) Use of temporary tables breaks replication (!!!). Reference link.
• (Update 2011.02.02) No Check constraints (!!!). Reference link.
• (Update 2011.02.02) No Full Outer join (!!!). Reference link.
• Foreign key cascades do not fire triggers (!!!). Reference link.
• You cannot disable triggers (!!!). Reference link.
• You can't use the same temp table twice in the same SQL statement (!!!). Reference Link.
• There is no method for using the results of stored procedures or functions as a result set for other queries (!!!):
• Can't create or insert into a temp table with sproc results: E.g. "create temporary table mytemptable call my_sproc();" doesn't work. Neither does "insert into mytemptable call my_sproc();"
• Can't use a stored procedure as a table alias: E.g. "select * from (call my_sproc());" doesn't work
• Can't create a view from stored procedure results: E.g. "create view myview as call my_sproc();" doesn't work
• There is no table type in MySQL, so functions can't return a table and also can't be used to pass result rows to other queries. E.g. "select * from my_func();" doesn't work
• These things are not explicitly stated in the manual, but questions regarding this kind of result reusability do come up from time to time. E.g. here.
• No array support. This means that you cannot pass an array to, nor receive an array from, a stored routine or function. Reference link.
• Stored functions can only return a single value. Not explicitly stated, but implicit in the stored function syntax. Reference link.
• Unintelligible error messages. Reference link (older but still indicative).
• No user-raisable error messages. Reference link.
• Integration with ASP: Entity Framework stored procedure inserts/updates/deletes are not configurable from the GUI -- they must be configured by hand. Reference link.

I can't even wrap my head around a company releasing a database and calling it production ready with some of these issues, particularly the first several half-dozen.

Note that there are work-arounds to nearly all of these issues, but they all entail a certain amount of extra programming, and configuration management (at least having to remember why you had to use those hacky, unclear SQL statements). If you use the database extensively, several of these problems can join forces to make the developer / admin experience very frustrating.

Personally, I will not choose MySQL 5.x again for any new projects. I'll even switch away from it on existing projects if given the opportunity. Hopefully MySQL 6 will be better.

So instead, I'll use SQL Server (even Express if I have to) when integrating with .NET applications. Otherwise, I'll use Postgres, which is free and open source. I'm sure those systems have their problems as well, but at least some of the more serious ones above are absent or mitigated.

Keyboard Layouts

As you may know, type-writers originally had more or less alphabetic key ordering. Typing was a mechanical process -- essentially pressing a key caused block letter to smack the paper, printing that letter. It was soon discovered that certain very common letter combinations (e.g. ST) would physically cause the machine to jam.

The QWERTY keyboard layout was devised by Christopher Sholes to minimize jamming. The commercial success of his typewriter in 1898, featuring the QWERTY layout, led many manufacturers to adopt it. Over time, it became the standard layout for keyboards, and it has changed only slightly since then. By and by computers came along, and the QWERTY layout has come to include some things specifically for the computer -- function keys, arrow keys, etc.

So a professor named Dvorak decides to investigate efficient text entry methods. After a bit of research on hand physiology and letter / diagraph frequencies, he came up with a keyboard layout specifically designed to minimize finger travel. It was patented in 1936 and is commonly known as the Dvorak keyboard. The layout, being an ANSI standard, is available but hidden away on most modern operating systems. The layout, once learned, promises faster data entry and reduced risk of CTS / RSI over the QWERTY layout. Of particular note is that 70% of English words can be typed using just the ten keys on the home row of the Dvorak keyboard.

However, the Dvorak keyboard never saw widespread acceptance. There are many cited reasons for this. Of particular note was the fact that workers would need retraining, and new equipment would have to be purchased (before the computer age). On top of that, you might recall that the 30s was a particularly bad time for the world economy. Even today, few studies on QWERTY vs. Dvorak exist, therefore it is difficult to empirically justify a switch from the established standard.

Part of the purpose of this entry was to explore the Dvorak keyboard, and in fact I wrote all of the above with a Dvorak-based layout, before editing. (I can see why people are reticent to switch... it is very frustrating to relearn to type!) However, I can't help but wonder if, after the training obstacle is overcome, if it won't be better, easier, less strain, maybe even faster. Perhaps I'll give it a try.

Regardless of what I do, I doubt that Dvorak will ever become the standard, even if it was one day empirically found to be superior. Learning to type is no small task (something you take for granted until you have to relearn it). The established labor pool has a QWERTY mindset. Heck even the staggered rows that modern keyboards have are owed to the "ancient" typewriter. It had to stagger them so that all the keys could mechanically connect to every corresponding letter block. Typing is such a fundamental function of computer use today, and QWERTY is so entrenched, that probably the only thing that will change it is to make the keyboard obsolete as a data entry method.

Avoid Javascript if you can!

If you're developing a rich web control, then -- no two ways about it -- you are stuck using JavaScript. If you're developing a business web app, avoid JavaScript at (nearly) all costs.

Now before you assume that I'm an ignorant wretch who's never really used JavaScript, let me tell you this. I've done a bit of client-side in my time. Including: A Mac OS X widget written with no JS framework, a jQuery plugin for filtering search results (written for and included in jqGrid since 3.5), and lots of custom code for various projects using jQuery or Prototype.

This is what clinched it for me: When I was doing a refactor of a particular piece of my main work project, I decided to fully embrace jQuery and use its UI components, themes, plugins, etc. The long and the short of it is that I will never do this again.

I mean, it's cool that you can create a snazzy jQuery UI widget with a div and a line of javascript, but making that work with your server-side language requires javascript glue. Even with jQuery making things easier, I had to write an inane amount of JS code to glue everything together (check or toggle css classes, setup and handle events like clicking, make ajax requests). My application was a tad complicated, and this was not helped by the fact that I now had to debug both client and server code (and the server code being the much easier part). Sometimes it's fun to make your own glue, but it's a massive expense just to have a pretty client-side control.

The take-away from this is that as an app developer (not a control developer) it's best to avoid using client-side code (Javascript) when possible. I can get around this in ASP .NET for the most part by using pre-made AJAX server controls from the Ajax Control Toolkit, or by using normal server-side controls in UpdatePanels. UpdatePanels potentially add a good bit of overhead, but for internal projects (which is most of what I do) having slightly better performance is not nearly worth the time it takes to make high performance glue.

Well, that's my perspective anyway. ;)

Microsoft showers students with software

First I found out about DreamSpark. As long as you are enrolled in a participating college or high school and have an email address ending in .edu, you can download the following full versions of MS software (and Product Keys if they are needed) completely free:

• Visual Studio 2008 Pro
• Visual Studio 2005 Pro
• Expression Studio 3
• SQL Server 2008 Developer
• Windows 2008 R2
• Windows 2008
• Windows 2003 R2
• Robotics Developer Studio (Standard I assume?)
• Other free software (Visual Web Dev Express, SQL Express, etc.)

No media is available. Most of these are for both 32 and 64 bit systems. 2008 R2 is only for 64 bit. (EDIT: The next bit is only if you try to install it on a Mac.) Burning the 2008 R2 IMG file requires special software beyond a standard ISO burner. On my first go, I got the 1. 2. prompt. I found this blog, which explain the why and how to fix it. It's a lengthy process, but I muddled through.

Then I found out about The Ultimate Steal where you can get more software from MS on the CHEAP. As long as you have a .edu email address, you can download ISOs, get CD keys, and get shipped the actual media for the following:

• Media for all of these costs an additional $13.00 per product
  
  • The media is NOT the full retail package
  • The media takes a long time to arrive (a month perhaps)
  
  
• Windows 7 Pro Upgrade (32 or 64 bit) - $29.99
  
  • You do NOT get both 32 and 64 bit versions. So know what you need beforehand. 32-bit should work for any machine that can run Windows 7. 64-bit is better if you have the hardware for it.
  
  
• Office 2007 Ultimate - $59.95
• Visio 2007 Pro - $55.95
• Non-English language packs for Office/Visio - $9.95 each (no media available)

The agreement reads that if they discover you are not an enrolled student, they can charge you the difference between what you paid and full price. Just a fair warning.

The download process is more annoying than at DreamSpark. There are 2 things you can download. One of them is a downloader for the ISO, and the other is for the setup files themselves, presumably for an in-place upgrade. I couldn't finish the latter to find out since I was running XP (and 32-bit to boot). Regardless, at least get the ISO and burn yourself a DVD to keep, just in case!

After you download the small .exe file, you have to run it to start the download of the actual product. The Digital River downloader caused a lot of problems on my machine. I got out of buffer errors and the entire machine became completely unstable, lost network connection, wouldn't start programs, etc. Then when I shut down, it blue screened. I believe this was due to AVG. After I turned the real-time scanner off, the problems seemed to stop. After that fiasco, I uninstalled AVG and put Windows Security Essentials in its place. Just to test, I redownloaded the entire thing post-WSE with no problems.

AVG has generally been solid for me, but I guess it just couldn't handle the strain of scanning my 12 M/s download. Thumbs down to AVG.

Anyway, hope this helps all you students out there.

Business and Data Layers

What follows in this entry is a way that works well for me to architect a Business Logic Layer (BLL) and Data Access Layer (DAL) into my projects. This may not be the best way, so any constructive feedback is welcome. I love to learn new and more efficient ways of doing things. (Read: I'm lazy, and if I can make it easier on myself, I will.)

Naming

DAL

I tend to like to organize my DAL in a structure similar to the database itself. The namespace ends up being something like:

com.project.DAL.DbName.TableName.Entity.Select.All()

com.project.DAL is the namespace for the DAL project

DbName.TableName is a folder structure

Entity is a class

Select is a nested class

All is a static method

(compare the latter two parts to doing a static method called SelectAll under the Entity class)

If there are specialized procedures that don't map to specific tables, I might make a "Procedures" class under the database folder to encapsulate those.

BLL

The BLL class, I tend to name based on functionality rather than database organization. Something like:

com.project.BLL.CRM.Customer.Get.All()

Interfaces

DAL

The DAL only returns DataTables or DataSets. I don't even bother with typed data sets because of the added hassle of regenerating the XML files each time I make a change.

Returning DataTables has a number of advantages.

For one thing, it's an abstraction over the database. I could even take DataTables pulled from completely different servers, put them in the same DataSet, and use DataRelations to query them in memory to get exactly what I need.

For another thing, if no extra business logic is needed (perhaps the table is only for display, not for editing), then I can skip creation of C# objects at the business layer and bind the DataTable from the DAL directly to web controls. Out of the box, DataTables already support all the niceties of web controls like paging and sorting.

BLL

The business layer objects are Plain Old CLR (C# or VB .NET) Objects (POCOs). They look something like this:

    public class Customer : IInitable  // my interface to make an Init method
  {
      public int customer_id { get; set; }
      public string name { get; set; }
      public CustomerTypeEnum type { get; set; }
      public DateTime since { get; set; }
      public List<History> HistoryEntries  { get; private set; }

      // note absence of constructor... CLR makes a default constructor this way

      #region IInitable Members
      public void Init()
      {
          ... // set the HistoryEntries with something like this.HistoryEntries = History.Get.ByCustId(this.customer_id);
      }
      #endregion

      public static class Get // nested class to handle fetch operations
      {
          public static List<Customer> All()
          {
              DataTable customer_table = DAL.DbName.Customers.Select.All();
              List<Customer> ret_val = Converter.Fill<Customer>(customer_table);
              return ret_val;
          }
      }
  }

By convention, I use the same field names from the database as my object property names. That makes converting the data from the database to a C# object easy with reflection. I use a method similar to what I blogged about a while back that does this for me. Something like this:

    public static void Fill(object obj, DataRow row)
  {
     // get object's writable properties into dictionary by name
     ...
     // loop through each column of the datarow
     ...
         // if the prop name is the same as the column name
         // check/handle: null values, enum types, type conversion
         // assign obj to property
  }

When I need to convert a whole DataTable into a list of objects, I wrote another convenience method:

    public static List<T> Fill(DataTable Table) where T: IInitable, new()
  { // where part means that T implements IInitable interface as has a default constructor
      List<T> ret_val = new List<T>();
      foreach (DataRow row in Table.Rows)
      {
          T obj = new T();
          Fill(obj, row);
          obj.Init(); // IInitable defines this method in case the object needs initialization after filling.
          ret_val.Add(obj);
      }
      return ret_val;
  }

IInitable is just a one-method interface that I wrote that makes the C# BLL class implement an Init method that returns void. Because I have to use empty constructors for uniformity, this Init method allows me to still initialize parts that would normally be initialized in the constructor, but after the data is filled in.

The CUD part of CRUD operations, at the BLL layer can take in an object and then use that object's properties for parameters of the DAL method.

However, to be really lazy, I am considering making an overload method for my DAL operations so that the BLL can just pass in the object directly and the overload method will use reflection to pull the properties out of the object and pass them into the method with typed parameters. I can probably encapsulate this with a separate static converter method so that the overload will essentially be a one-line call to the converter method (and therefore not much coding overhead), which invokes the real method. Once I get around to trying this, I'll post something about it.