21 November 2014

Install EventStore as a service in Windows

Install Chocolatey
(optional, you can manually download NSSM)

Start a command prompt as administrator and run (copied from the website):
@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

Install NSSM

From command prompt:
choco install NSSM

Download EventStore

Get Event Store for Windows
Unzip and copy to C:\EventStore (or a location you prefer)

Note: EventStore is available from Chocolatey as well, but as of this writing it wouldn't install for me, and it’s slightly behind current version.

Install EventStore as a service

From command prompt:
nssm install EventStore C:\EventStore\EventStore.ClusterNode.exe --db ./db --log ./logs

Note: This uses the default ports and runs single node. The db and logs directories will be automatically created on startup if they don't exist.

29 October 2014

Video encoding

I was reading about VP8 and H.264 because my application will have to handle video, and I wanted to be familiar with what's going on. As part of that, I considered a method to "encode" a video in a way that is highly parallel, maybe low on space, and with little-to-no loss. Being just a simple man from the South, I'm sure someone a lot smarter than me has already thought of this, but here are my ideas.

Moving pictures

In simplest terms (at least in my limited knowledge of the video industry), raw video is an ordered sequence of pictures (frames). So each frame is a 2-dimensional grid of pixels. From video gaming, at least 60 frames per second is considered roughly optimal to make the human eye unable to perceive frame changes (it just looks like motion). 30 fps and up actually works for this, but an occasional frame change is still observable. At 1080p (1920x1080 resolution) there are 2,073,600 pixels in each frame if you look at it as an individual picture. A pixel's color is represented by a combination of color values. From the web, I know that one way to represent RGB colors is by using 6 hexadecimal digits; 2 for red, 2 for green, and 2 for blue. 6 hex digits takes up roughly 3 bytes. So that adds up to 5.9MB per frame to represent all the pixels. At 60 frames per second, that's 355MB per second of space. A 2 hr movie would be 42.7GB in raw form, not counting audio, delimiters, and format info.

Maths

If you look at an RBG color as a vector, then each frame is vector field. Though not quite, because RGB coordinates would always be integers. It'd be more like a vector ring, but probably assuming it's a real number would be sufficient. So anyway, if we extend that further, we could theoretically find a formula to exactly fit the changes of one pixel over time. (A small part of me wishes I had not dropped Vector Analysis in college.)

With the way most movies are actually edited, a single function would probably be pretty hairy, take a long time to generate, and take a lot of calculation to get the value for each pixel. But inside of a single shot, the function would probably look pretty smooth and have a fairly easy formula. Some exceptions are likely (like a night scene with gunshots, causing wide swings of color and intensity). So the one end of the spectrum you have one monolithic formula per pixel, and on the other end is a piece-wise function, with formulas for each shot.

Formula-fitting each pixel is an inherently parallel process, with each pixel considered separately. Optimization can occur afterwards. Video cards could be used (both encode and decode) for this since their computational power is all about parallelization.

Considering there are over 2 million pixels at 1080p, would this method actually save any space? A movie like The Matrix with over 2300 shots and using an accurate piece-wise function could be 10x larger than the original (my rough guesstimate based on a 50 ASCII character formula per shot per pixel). However, there are some "nerd knobs" that can be tweaked to make it theoretically small enough to be consumable. For one you can always increase the margin of error on the function, which will generally make the functions smaller and simpler while sacrificing accuracy. There are also optimizations you could make like sharing a particular formula across multiple pixels. Consider a night shot where many of the pixels will be the same shade of grey. Or consider that many pixels for a given scene will share a similar intensity, but will just be color shifted from one another. Or vice versa, same color, but different intensities (black and white film, for instance). By identifying those formulas which are simple transformations of others, there is an opportunity to conserve space. Although the sheer number of addressable pixels creates a lot of overhead for such optimizations. This volume of data is as much an exercise in organizational efficiency as anything.

Anyway, it is an interesting though experiment. Looking at a video as a series of vector fields is something I hadn't considered before.

10 October 2014

Picking a client platform for a new system

What follows is my journey to picking a platform for a particular client in a new software system. It may not be the best answer for your case, but the considerations may still be relevant.

When tasked to develop a new business system one of the (many) choices to be made is the platform internal clients will run on. It is an important decision that will have impact for years to come.


The Desktop


Our first inclination was to create a Windows desktop app. All workstations are Windows machines, our team skillset is the .NET platform, and there were a number of local resources that need to be accessed (specialized printing, camera access, signature pad, and accounting software integration). So then I began to look at .NET desktop platforms: WinForms, WPF, and WinRT. WinForms was dismissed right away because of it's lack of extensibility and modern feature support (hardware acceleration, data binding, and flexible controls to name a few).


WPF


The harder decision was between WPF and WinRT. As near as I can tell from my research, WPF and WinRT apps are not all that different to develop (XAML/.NET, although WinRT has other options too). I even started a prototype app using MahApps.Metro and began digging into learning WPF. However, there is a cloud of doubt surrounding the future of WPF. There were some new WPF bits in .NET 4.5, but overall there has been very little activity or advancement in WPF for many years. Things that were tedious generations ago (that is, computer generations) are still tedious, with no official sign that it will improve. Considering it's age and stagnation, it's hard to pick this as the platform of the future for the new clients.


WinRT


I also did some research on WinRT. As far as I've found, you can make a .NET desktop app by starting with a Windows Store project template and manually modifying the project to enable desktop usage. You can also manually add references to WinRT libraries from other project types. My main issue with WinRT is that much of its design revolves around "metro" and the Windows Store, which is generally not being embraced by the industry. To me this makes WinRT's future speculative at best. Not to mention that WinRT will only run on Windows 8 right now, which the IT department is skipping. The impending Windows 10 release probably next year and a cooperative IT department willing to upgrade would ordinarily make this choice not so bad. But the larger question of whether the Windows Store underpinning will make it as a desktop app platform (which doesn't seem to be WinRT's primary consideration) gives me great pause. Microsoft has managed to leave an unstable vacuum in desktop development, which makes me concerned about significantly investing in that space.


Now what?


So where does that leave me? Well, my background is the web, and although I was looking forward to broadening my skill set to a desktop technology, it doesn't appear (from a platform perspective) that there is an good one to pick. I was also not very encouraged by my foray into WPF. It felt like almost a backwards step from HTML5 as a front-end technology. Don't get me wrong, HTML5 has a lot of room for improvement. I always like to say (with some embellishment) that Javascript is the worst possible tool for the job, but it is the only tool that can do its job. (This is sortof a twist on the Python creator's comments about PERL.) But I will say that XAML markup is quite verbose, especially since a lot of the changes I wanted to make required custom implementations of the entire control (even if mostly pasted from the default template). And the code required for data binding is crazy verbose compared to something like knockoutjs's ko.observable(). A lot of styling that is pretty straightforward in CSS feels weird in XAML... e.g. hover/active color changes. CSS3 animations are also amazingly simple. And considering that HTML5 is actually experiencing a LOT of improvement of late, and more in the future, it seems like a good client platform choice for moving data.


Web Issues


However, there are a couple of problems with this choice. Firstly, let's talk about browser compatibility. This is the main thing which holds back HTML5/CSS3 as a platform, but that mainly concerns web pages out in the wild. Consider that with a desktop app, IT would be required to install my app on machines. Now instead of that, IT will be required to install a different app on machines -- a modern/HTML5 browser. And there's more than one to choose from. What about cross-platform compatibility? You can install recent versions of Chrome of Firefox on a broad range of OSs and versions, probably on any workstation your enterprise runs.


Signatures


Then there is local resource access, the primary reason we wanted a desktop app. With some changes to our workflow, we are able to simplify the process for customers and also get rid of the need to interface with a signature pad -- its function will be integrated into a tablet which will be used for other parts of the process. Using a browser that supports getUserMedia, our app can also access the web cam. So the main difficulties left are the specialized printing and integration with accounting, which do require local resource access that browsers cannot provide.


Direct Printing


Normal printing is not really a problem from the browser, but we must print to specialized devices that may use their own printing languages (like zebra printers). Before the HTML5 discussions, we had already decided to make a Windows service for printing which would directly talk to the printer so prints can be automatically triggered based on system events. Since the service will already be directly talking to the printer, it can handle manually triggered printing from the web app as well. So that problem was already solved.


Accounting integration


I don't have a solution designed for the accounting software, because we don't even know what we will be using yet. (They are extremely unhappy with their current accounting software and want to change away from it regardless of my effort.) In any case, all the client will be able to do is make the request (e.g. to create invoices), and the server will take the process from there; running it directly if the accounting software supports it, or delegating it to a custom service on the accountant's machine in the worst case.


Chrome app?


I also looked at developing this as a Chrome app, which provides limited access to local resources. However, I really didn't like the idea of making proprietary modifications that make my HTML5 app not run elsewhere. It also seemed like for every local access I got, some normal web access was restricted due to sandboxing. That, and really the only benefit I would get is printing, which I had already resolved. The accounting software would probably still not be accessible from the Chrome app.

So anyway, that was my decision process. Going forward, we are looking at using Angular as our UI technology, and Bootstrap as a front-end (wanted to use Foundation, but Bootstrap was easier to integrate). On the server side, it's .NET Web API.

29 September 2014

Disabling Text Selection / Right click on Websites is Stupid

So, I run into this every once in a while, and it is a minor thing that majorly irritates me: Websites who attempt to disable text selection or right-click.

First of all, this is utterly futile. HTML on a web page can't reasonably be protected from copying. The source of a web page can be viewed and manipulated (client side) at any time, even without right-click. Rather than just copying your text anyone can completely grab layout, style, script, etc. To illustrate this point to co-workers, I will go to a well-known website and change the main page by adding their name to it, for instance. It only shows up on their local browser until they reload the page, but it makes the point that HTML content is not readily securable and wasn't designed to be. If you want that control, use another format.

Second, there are valid use cases for highlighting text, as it has become a normal and expected thing in browsers. One such case is citation where you include the link, but the page's text is lengthy, and you want to block quote a passage to highlight it. There is also reporting a problem with the website. There are also accessibility/usability cases for text selection (e.g. text clipped in container). Personally, I sometimes highlight text subconsciously as I read, which is actually what led to this blog post when I found I could not do that.

Thirdly, I respect people's copyrighted / licensed work, but that respect ends with trying to hijack/disable basic functionality on my browser. Sorry, but find another way. Maybe inject a page link when someone copies text so they get a proper citation with no extra steps?

So, I made this post (where you can freely select the text if you like) to document the ways I have found to defeat this asinine infringement of browser functionality.

Javascript Selection disabling

The one I ran into most recently was a WordPress plugin called "Blog Protector". This style is very easy to disable. This should cover text selection on most browsers (not fully tested). On Chrome, the only line I needed to run was the first one.

Open a Javascript console (Chrome is Ctrl-Shift-J, IE is F12 then Console) and run this:

if(document.body.onselectstart) document.body.onselectstart = null;
if(document.body.style.MozUserSelect) document.body.style.MozUserSelect = null;
if(document.body.onmousedown) document.body.onmousedown = null;


I'll update this post if I run into more. This is mainly for my own reference, so I won't be taking requests, sorry.

06 July 2014

Using Modern UI Icons

In my research, I stumbled across the excellent Modern UI Icons.

I downloaded them and put the XAML files in a test application. Eventually, I got the the point of adding them to a view. That's when the problems started. The XAML files can't be used as sources for an Image. And here I was thinking it was an image format like SVG.

Eventually, I was able to setup a useful style for a ContentControl. This can be resized and the color-changed as well.

Here is what I ended up with:

    <Style TargetType="ContentControl" x:Key="icon">
        <Setter Property="Template">
            <Setter.Value>
                <ControlTemplate TargetType="ContentControl">
                    <Viewbox Height="{TemplateBinding Height}" Stretch="Uniform">
                        <ContentPresenter />
                    </Viewbox>
                </ControlTemplate>
            </Setter.Value>
        </Setter>
    </Style>
    <Canvas x:Shared="False" x:Key="icon-calculator"  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" x:Name="appbar_calculator" Width="76" Height="76" Clip="F1 M 0,0L 76,0L 76,76L 0,76L 0,0">
        <Path Width="32" Height="42" Canvas.Left="22" Canvas.Top="17" Stretch="Fill" Fill="{Binding RelativeSource={RelativeSource Mode=FindAncestor, AncestorType=ContentControl}, Path=Foreground}" Data="..."/>
    </Canvas>

I left the data out of this example for brevity. The Path data is in the XAML files for the modern UI icon. The yellow stuff is what I had to add.

1) Setting x:Shared to False is necessary otherwise WPF will only display the icon once, no matter how many times you reference it.
2) Settings x:Key is also necessary so you can reference it.
3) The Fill binding allows me to change the brush color

So here's how to use the icon.

<ContentControl Style="{StaticResource icon}" Content="{StaticResource icon-calculator}" Height="32" Foreground="Black"/>

I could probably shorten this with a custom control. However, the only thing it would save is the Style declaration. So I haven't bothered yet.

05 July 2014

Bootstrap style WPF buttons

In my upcoming job, I will be writing a desktop app using WPF. Most of my work has previously been on the web, where there are many established libraries for layout and general behavior. One of those is called Bootstrap.

Some people dislike Bootstrap because many websites out there use the starting Bootstrap template, hence the notion that all Bootstrap sites look the same. Let me just say, they don't. It's probably fairer to say that website looks alike if their designers are lazy. Bootstrap itself is a great tool.

I commonly use Bootstrap buttons, because they come with good default styles which can give hints to the users. I mainly use the primary, default, and danger styles to indicate the kind of actions that are being performed. So today, I will be discussing how to get those Bootstrap-style buttons in WPF.

First, what do they look like? Here is a sample from the website (using Chrome):


This style matches pretty well with a so-called "modern ui" style.

Here is what my WPF attempt at these buttons looks like:


Here is the XAML style for these buttons, including disable, hover and click effects.

    <Style x:Key="btn" TargetType="Button">
        <Setter Property="FontFamily" Value="Helvetica Neue,Helvetica,Arial,sans-serif"/>
        <Setter Property="FontSize" Value="14"/>
        <Setter Property="Padding" Value="12,8"/>
        <Setter Property="BorderThickness" Value="1"/>
        <Setter Property="Template">
            <Setter.Value>
                <ControlTemplate TargetType="{x:Type ButtonBase}">
                    <Border Name="border" CornerRadius="4" BorderBrush="{TemplateBinding BorderBrush}" BorderThickness="{TemplateBinding BorderThickness}" Background="{TemplateBinding Background}" SnapsToDevicePixels="True">
                        <Grid>
                            <Border Name="dropShadowBorder" CornerRadius="4" BorderBrush="Transparent" BorderThickness="0" Visibility="Hidden">
                                <Border.Background>
                                    <LinearGradientBrush StartPoint="0,0" EndPoint="0,0.16">
                                        <GradientStop Color="#22000000" Offset="0"/>
                                        <GradientStop Color="#00000000" Offset="1"/>
                                    </LinearGradientBrush>
                                </Border.Background>
                            </Border>
                            <ContentPresenter Name="contentPresenter" ContentTemplate="{TemplateBinding ContentTemplate}" Content="{TemplateBinding Content}" ContentStringFormat="{TemplateBinding ContentStringFormat}" Focusable="False" HorizontalAlignment="{TemplateBinding HorizontalContentAlignment}" Margin="{TemplateBinding Padding}" RecognizesAccessKey="True" SnapsToDevicePixels="{TemplateBinding SnapsToDevicePixels}" VerticalAlignment="{TemplateBinding VerticalContentAlignment}"/>
                        </Grid>
                    </Border>
                    <ControlTemplate.Triggers>
                         <!--default button highlight--> 
                        <Trigger Property="Button.IsDefaulted" Value="True">
                            <Setter Property="BorderBrush" TargetName="border" Value="{DynamicResource {x:Static SystemColors.HighlightBrushKey}}"/>
                        </Trigger>
                         <!--inner drop shadow when pressed / checked--> 
                        <Trigger Property="IsPressed" Value="True">
                            <Setter Property="Visibility" TargetName="dropShadowBorder" Value="Visible"/>
                        </Trigger>
                        <Trigger Property="ToggleButton.IsChecked" Value="True">
                            <Setter Property="Visibility" TargetName="dropShadowBorder" Value="Visible"/>
                        </Trigger>
                        <Trigger Property="IsEnabled" Value="False">
                            <Setter Property="Opacity" TargetName="border" Value="0.60"/>
                        </Trigger>
                    </ControlTemplate.Triggers>
                </ControlTemplate>
            </Setter.Value>
        </Setter>
    </Style>
    <Style x:Key="btn-default" TargetType="Button" BasedOn="{StaticResource btn}">
        <Setter Property="Foreground">
            <Setter.Value>
                <SolidColorBrush Color="#333"/>
            </Setter.Value>
        </Setter>
        <Setter Property="Background">
            <Setter.Value>
                <SolidColorBrush Color="#fff"/>
            </Setter.Value>
        </Setter>
        <Setter Property="BorderBrush">
            <Setter.Value>
                <SolidColorBrush Color="#ccc"/>
            </Setter.Value>
        </Setter>
        <Style.Triggers>
            <Trigger Property="IsMouseOver" Value="True">
                <Setter Property="Background" Value="#e6e6e6"/>
                <Setter Property="BorderBrush" Value="#adadad"/>
            </Trigger>
            <Trigger Property="IsPressed" Value="True">
                <Setter Property="Background" Value="#e6e6e6"/>
                <Setter Property="BorderBrush" Value="#adadad"/>
            </Trigger>
            <Trigger Property="ToggleButton.IsChecked" Value="True">
                <Setter Property="Background" Value="#e6e6e6"/>
                <Setter Property="BorderBrush" Value="#adadad"/>
            </Trigger>
        </Style.Triggers>
    </Style>
    <Style x:Key="btn-primary" TargetType="Button" BasedOn="{StaticResource btn}">
        <Setter Property="Foreground">
            <Setter.Value>
                <SolidColorBrush Color="#fff"/>
            </Setter.Value>
        </Setter>
        <Setter Property="Background">
            <Setter.Value>
                <SolidColorBrush Color="#428bca"/>
            </Setter.Value>
        </Setter>
        <Setter Property="BorderBrush">
            <Setter.Value>
                <SolidColorBrush Color="#357ebd"/>
            </Setter.Value>
        </Setter>
        <Style.Triggers>
            <Trigger Property="IsMouseOver" Value="True">
                <Setter Property="Background" Value="#3071a9"/>
                <Setter Property="BorderBrush" Value="#285e8e"/>
            </Trigger>
            <Trigger Property="IsPressed" Value="True">
                <Setter Property="Background" Value="#3071a9"/>
                <Setter Property="BorderBrush" Value="#285e8e"/>
            </Trigger>
            <Trigger Property="ToggleButton.IsChecked" Value="True">
                <Setter Property="Background" Value="#3071a9"/>
                <Setter Property="BorderBrush" Value="#285e8e"/>
            </Trigger>
        </Style.Triggers>
    </Style>
    <Style x:Key="btn-success" TargetType="Button" BasedOn="{StaticResource btn}">
        <Setter Property="Foreground">
            <Setter.Value>
                <SolidColorBrush Color="#fff"/>
            </Setter.Value>
        </Setter>
        <Setter Property="Background">
            <Setter.Value>
                <SolidColorBrush Color="#5cb85c"/>
            </Setter.Value>
        </Setter>
        <Setter Property="BorderBrush">
            <Setter.Value>
                <SolidColorBrush Color="#4cae4c"/>
            </Setter.Value>
        </Setter>
        <Style.Triggers>
            <Trigger Property="IsMouseOver" Value="True">
                <Setter Property="Background" Value="#449d44"/>
                <Setter Property="BorderBrush" Value="#398439"/>
            </Trigger>
            <Trigger Property="IsPressed" Value="True">
                <Setter Property="Background" Value="#449d44"/>
                <Setter Property="BorderBrush" Value="#398439"/>
            </Trigger>
            <Trigger Property="ToggleButton.IsChecked" Value="True">
                <Setter Property="Background" Value="#449d44"/>
                <Setter Property="BorderBrush" Value="#398439"/>
            </Trigger>
        </Style.Triggers>
    </Style>
    <Style x:Key="btn-info" TargetType="Button" BasedOn="{StaticResource btn}">
        <Setter Property="Foreground">
            <Setter.Value>
                <SolidColorBrush Color="#fff"/>
            </Setter.Value>
        </Setter>
        <Setter Property="Background">
            <Setter.Value>
                <SolidColorBrush Color="#5bc0de"/>
            </Setter.Value>
        </Setter>
        <Setter Property="BorderBrush">
            <Setter.Value>
                <SolidColorBrush Color="#46b8da"/>
            </Setter.Value>
        </Setter>
        <Style.Triggers>
            <Trigger Property="IsMouseOver" Value="True">
                <Setter Property="Background" Value="#31b0d5"/>
                <Setter Property="BorderBrush" Value="#269abc"/>
            </Trigger>
            <Trigger Property="IsPressed" Value="True">
                <Setter Property="Background" Value="#31b0d5"/>
                <Setter Property="BorderBrush" Value="#269abc"/>
            </Trigger>
            <Trigger Property="ToggleButton.IsChecked" Value="True">
                <Setter Property="Background" Value="#31b0d5"/>
                <Setter Property="BorderBrush" Value="#269abc"/>
            </Trigger>
        </Style.Triggers>
    </Style>
    <Style x:Key="btn-warning" TargetType="Button" BasedOn="{StaticResource btn}">
        <Setter Property="Foreground">
            <Setter.Value>
                <SolidColorBrush Color="#fff"/>
            </Setter.Value>
        </Setter>
        <Setter Property="Background">
            <Setter.Value>
                <SolidColorBrush Color="#f0ad4e"/>
            </Setter.Value>
        </Setter>
        <Setter Property="BorderBrush">
            <Setter.Value>
                <SolidColorBrush Color="#eea236"/>
            </Setter.Value>
        </Setter>
        <Style.Triggers>
            <Trigger Property="IsMouseOver" Value="True">
                <Setter Property="Background" Value="#ec971f"/>
                <Setter Property="BorderBrush" Value="#d58512"/>
            </Trigger>
            <Trigger Property="IsPressed" Value="True">
                <Setter Property="Background" Value="#ec971f"/>
                <Setter Property="BorderBrush" Value="#d58512"/>
            </Trigger>
            <Trigger Property="ToggleButton.IsChecked" Value="True">
                <Setter Property="Background" Value="#ec971f"/>
                <Setter Property="BorderBrush" Value="#d58512"/>
            </Trigger>
        </Style.Triggers>
    </Style>
    <Style x:Key="btn-danger" TargetType="Button" BasedOn="{StaticResource btn}">
        <Setter Property="Foreground">
            <Setter.Value>
                <SolidColorBrush Color="#fff"/>
            </Setter.Value>
        </Setter>
        <Setter Property="Background">
            <Setter.Value>
                <SolidColorBrush Color="#d9534f"/>
            </Setter.Value>
        </Setter>
        <Setter Property="BorderBrush">
            <Setter.Value>
                <SolidColorBrush Color="#d43f3a"/>
            </Setter.Value>
        </Setter>
        <Style.Triggers>
            <Trigger Property="IsMouseOver" Value="True">
                <Setter Property="Background" Value="#c9302c"/>
                <Setter Property="BorderBrush" Value="#ac2925"/>
            </Trigger>
            <Trigger Property="IsPressed" Value="True">
                <Setter Property="Background" Value="#c9302c"/>
                <Setter Property="BorderBrush" Value="#ac2925"/>
            </Trigger>
            <Trigger Property="ToggleButton.IsChecked" Value="True">
                <Setter Property="Background" Value="#c9302c"/>
                <Setter Property="BorderBrush" Value="#ac2925"/>
            </Trigger>
        </Style.Triggers>
    </Style>

Then to utilize these, you only have to set the style on the button. It's style name matches the class name for the equivalent bootstrap button. For example:

<Button Content="Default" Style="{StaticResource btn-default}"/>

Notes: Xaml styling is not as robust as CSS, and is a lot more verbose. You can't set more than one style, but this capability wasn't needed in this case.

12 May 2014

More on CQRS and Actors

So today, I discovered the Orleans project. I also watched the excellent presentation on it from Build 2014. The actor model has a lot of appeal for easier creation of scalable, distributed systems. In trying to work out how it would resolve with a DDD/CQRS system, I came up with some observations.

In the Build example, the presenter listed statistics as an actor (aka "grain"). As described, this report-type actor would have the same downsides as a report from an N-tier system. In other words, their described system did not have the hallmark separation between reads and writes of a CQRS system. The report data updates are destructive, history is lost, and the reports can't be proven correct (replayed).
If doing actors in a CQRS system, the actors would only be used on the write side. Using event sourcing as the actor persistence mechanism, denormalizers would still come into play for the read side, not actors.

Actors seems to fit really well for aggregates, but what about Process Managers or Sagas? These are a harder fit because they are not necessarily explicitly called by a user or another actor. They are created and run as a result of events in the system. They are system automations, if you will. The best way that I can think of to implement these event-driven components as actors is to have a separate event listener that forwards the appropriate events to them.

To my thinking, the main purpose of actors in a DDD/CQRS system is to give aggregates (and maybe process managers) a structure that can be interacted with for computation. Ideally, the actor subsystem comes with the necessary architecture to abstract away technical concerns such as message dispatch, scale, distribution, etc. The Orleans project seems to provide a lot of that and apparently does it pretty well for the Halo 4 team.

The main thing that I find really weird about it is their "message passing" method. I like to store the commands I send to the system as well as the events they produce for traceability. Orleans doesn't allow this because it hides the message passing from you. Instead, you load a faux actor and call a proxy method on it. The system takes care of serializing your method call with parameters and delivering that message to the real actor on the server.

I'm eager to see what comes of Orleans. I hope Microsoft decides to bring it out of research and turns it into a proper product. It has the potential for great things in the DDD/CQRS space.

07 March 2014

Software Architecture Traps

As developers, we like to solve problems with code. Sometimes business problems aren't hard enough, and without even thinking about it, we expand the problem so it becomes harder. We want to solve all problems in the same general category as that problem. Once and for all, so we can reuse it later. Maybe we want to build a framework or integrate multiple frameworks (ORM, IOC, AOP, etc.) to solve our problems once and for all. Well, I wish I had run across this video sooner.

http://www.infoq.com/presentations/8-lines-code-refactoring

Sometimes the pain I experience while coding something doesn't indicate that I need to bring in (or create) another framework to avoid that pain. Maybe that pain is simply telling me that there is a non-technical problem (i.e. analysis problem) that a framework isn't going to solve (but might hide).

I realized I had done this with my LocalBus project. If I wasn't trying to fit a square peg in a round hole, then the "overhead" of writing wire-up code wouldn't have been so painful. I would never have needed to write it.

I also really liked how this video shows handler wire-up with dependencies (e.g. repositories). It's manual, but it is also customizable (varying dependencies between use cases), and (using functional language features) it's pretty low overhead.

06 February 2014

No Goldilocks Serializers

So in any architecture that uses messaging for communication, one of the key pieces is serialization. Along those lines, I tried several different serializers and benchmarked the ones that would actually work in my scenario. I borrowed a lot of the serializer test code from various examples on stack overflow (e.g. this one).

Here are ones that I tried:

protobuf-net
Avro
MsgPack
fastJSON
XSockets (ServiceStack.Text)
JSON.NET

I didn't test ServiceStack on purpose, but instead was using the XSockets serializer, which turned out to just be ServiceStack. I didn't care to test ServiceStack at first because of its AGPL license, which is incompatible with my desired freedoms. The XSockets version of ServiceStack is licensed differently as part of the XSockets project. (Since they don't mention AGPL in their license page.)

Here are my requirements for messages I want to serialize:
  1. No attributes (not even Serializable in my tests)
  2. All public members are read only
These requirements killed Avro, MsgPack, and fastJSON right out of the gate. The .NET libraries for Avro and MsgPack wouldn't even instantiate a serializer. fastJSON crashed when trying to serialize. I didn't investigate why.

To get it to work with the XSockets JsonSerializer (which comes with XSockets Core on nuget and again is just ServiceStack.Text), I had to change my readonly fields to properties with private setters. Apparently it ignores fields, even public ones. Later, I figured out how to make it pick up fields, but it didn't really affect the test.

For both ServiceStack and JSON.Net, I serialized to/from an interface to force it to embed the type. This is important for my use case.

Protobuf-net doesn't support type embedding per se, so I had to send two messages. The first (header) message contained the type of the subsequent message. This test includes the time it took protobuf-net to serialize and deserialize both messages for fairness. I do know about the prefix in which I can send an integer to represent the message type, but try consistently hashing and arbitrary type to the same integer value and tell me how that works for you...

Protobuf-net has the added wrinkle that I have to dynamically add the classes to its serializer at runtime (remember, no attributes). This is a huge negative. It's not too bad for simple messages, but if you have any polymorphic properties, this code could get complicated. E.g. Find all inheritors for this type; Find all implementors of this interface; Oh, and if those objects also have reference type properties, recurse. It would be so much easier if I could just embed the type Marc... not that you will ever see this. I write this for myself, you know.

Ultimately, my conclusion for messaging is that there's not an absolute win for any one serializer. Nothing was "just right". Protobuf-net  was the fastest, but the hardest to deal with. ServiceStack.Text is the fastest JSON serializer for my use case, but the AGPL license destroys too many of my freedoms. JSON.Net performed the poorest (but still light years ahead of the default .NET serializers). Deserialization was its weakness. For now I will try out the XSockets version of ServiceStack.Text since I'm planning on using it with XSockets anyway, and I can actually stomach the XSocket license it comes under (BSD 2-clause).

Below are the results for 100000 iterations.

protobuf-net (header + message)
Length: 285
Serialize: 213
Deserialize: 897

XSockets Serializer
Length: 508
Serialize: 933
Deserialize: 1606

JSON.Net BSON
Length: 404
Serialize: 1127
Deserialize: 3417

JSON.Net JSON String
Length: 540
Serialize: 928
Deserialize: 3644

Here are the classes used:

    public class Header
    {
        public Type T { get; private set; }
        public Header(Type t)
        {
            T = t;
        }

    }

    public class TestMessage : IMessage

    {
        public Guid SenderId { get; private set; }
        public Guid ReferenceId { get; private set; }
        public Guid OrderId { get; private set; }
        public Guid CustomerId { get; private set; }
        public LineItem[] LineItems { get; private set; }
        public DateTime OrderedOn { get; private set; }
        public TestMessage(Guid senderId, Guid referenceId, Guid orderId, Guid customerId, LineItem[] lineItems, DateTime orderedOn)
        {
            SenderId = senderId;
            ReferenceId = referenceId;
            OrderId = orderId;
            CustomerId = customerId;
            LineItems = lineItems;
            OrderedOn = orderedOn;
        }
    }

    public class LineItem

    {
        public Guid ProductId { get; private set; }
        public int Quantity { get; private set; }
        public LineItem(Guid productId, int quantity)
        {
            ProductId = productId;
            Quantity = quantity;
        }

    }

    public interface IMessage { }




31 January 2014

Asynchronously Synchronous Commands

I have typically be against asynchronous commands for the simple fact that it leaves the user hanging. However, when considering responsiveness under load, there is something to be said for making this asynchronous.

My current line of thinking is to send the command asynchronously with the expectation that the result of that command (success for failure) will eventually be sent back to the client. That way, at least the client knows if there is a problem. This is what I'd call asynchronously synchronous.

This brings up a couple of interesting issues. One is what to do with the UI while that is going on. Do I show a spinner and make the user wait? The response will happen pretty quickly, so I'm leaning toward this initially. Maybe instead, I should just keep track of running commands and notify the user if one fails. How do they recover from the failure in this case? There are interesting opportunities for design here. Some of the right answer depends on the user's workflow.

So now the other issue is brought up here: Getting a positive command result doesn't mean that the read models have been updated, since this also happens asynchronously. This introduces the idea of the client being able to subscribe to read model changes. Ultimately, the user only wants one of two things to happen when they submit a command. 1) Ideally, the command succeeds and their view is updated (to verify). 2) Their command fails and they are given enough information to resolve the problem. Therefore the the only 2 things the client program will be interested in listening for are: command failure and read model updates.

That obviously doesn't cover some edge cases like network failure. After all if the network fails while I am blocking for it, then I get notified about it, but if I just never receive a message that I was expecting, then I need to account for that.

29 January 2014

CQRS and Actors

In looking through the DDD/CQRS google group, I came across a discussion about using the Actor model as the computational model for DDD/CQRS.

This immediately struck me as interesting, and even more so after listening to the episode of the Being the Worst podcast. Part of the overhead of CQRS/Messaging/ES, and a regular pain for me, is the whole handler piece. Most handler methods seem pretty boilerplate. Load something (e.g. aggregate or process manager), collect resources for it, do something with it, save it. All of this is pretty dull, and it must be written for every message. I think the actor model could have some good ideas to offer here.

The handler has been the catch-all for any infrastructure concerns related to delivering a message, but has typically been observed doing the above-mentioned 4 steps. Load and Save are exactly the same for every event sourced object, so these could be pushed higher in the infrastructure. Ignoring resources for now, the execute step could be characterized as executing a method on the aggregate with the message contents as parameters. The aggregate can be changed to just receive the message directly. This could be implemented similarly to the typical actor semantics of tell(message). There now just needs to be a way to provide resources to the aggregate at the infrastructure level, so the domain stays isolated per DDD.

So perhaps we should formalize the arrangement of there being a resource factory for a particular aggregate type. This factory could be directly used as needed by the aggregate via an interface and dependency injected by the infrastructure (not that you need a DI or IoC container for this). The resource factory is not exactly a repository, since there are no writes to it, only reads, and so there are still no persistence entanglements in the domain.

So now we have increased the burden of the infrastructure code. Namely, matching resources with actors. But we have eliminated most of the need for handlers. (You could also say that we increased the burden with load and save code, but this code was duplicated so much in handlers, that I see it as a net savings.) There may still be a case I can't think of for having handlers for one-off purposes, but now they don't have to be created perfunctorily in every case. We have also formalized another infrastructural role for servicing the domain, the Resource Factory. This role already existed informally in handlers.

I will be experimenting with this approach.

06 January 2014

Mass Effect: How Not to End a Series

To say the Mass Effect series was good is a drastic understatement in my opinion. The series as a whole is in fact one of the best I've ever played. However, when the folks at BioWare decided they were tired of making Mass Effect games, they took it out on us fans.

To set the stage for my utter disappointment, I must tell you that I played all the Mass Effect games, and played ME3 right after release. That was before the extended endings were released, although they had little to no effect in "fixing" anything.

First, I go through the hybrid ending (green), figuring it would be an optimal ending. After seeing it, it twinged as slightly evil because you were forcing a fundamental change on the entire universe that no one asked for. Then I do the "good" (blue) ending thinking it would be... well... good. However, the "good" ending realized the goal of a power-hungry villain and leads your character to not only sacrifice himself, but also his moral center. Also, the cut scenes were the EXACT SAME with minor animation differences and a different color explosion. So I thought, "oh the colors must just be reversed or something". So then I did the red ending, thinking it would be the good ending. It also had the EXACT SAME cut scenes with minor animation differences and a different color explosion. The red ending was also bad and a temporary solution, but it seemed the only ending to fit with themes from the previous titles in the series. If you have enough war assets, it's also the only ending with even a hint of the main character surviving. There's also a fail condition that you could call an ending that simply kills every one and resets technology back to zero. And none of the endings made any sense in context with the rest of the series. They were explained away in the extended endings, but still didn't really seem to fit the way the previous games ended. The fact remained that in ME3, BioWare had decided to take a different direction with the series (i.e. push it off a cliff).

So basically, the series ends on the main character only having selfish choices which also force him to die for completely arbitrary reasons (because that's the way the star kid designed it, and the main character suddenly lacks the capability to come up with independent solutions). And since BioWare seemingly intended to converge all the endings around you being a jerk and dying, they only bothered to make 1 ending cut scene, with minor animation tweaks and color changes between them. The extended ending didn't change that, it just added more filler.

All this leads me to believe the Mass Effect team was tired of making Mass Effect games and sabotaged the ending. This way, they theoretically have less fans begging for them to make a sequel (the main character died and ended up being a tool anyway). However, this hit me as a betrayal by the characters and universe I had come to love. The "mass effect" for me has been to no longer be a fan of BioWare. Their games have some of the greatest story, which makes the series-ending betrayal all the more bitter.