X3:TC Best Player Ship

So I admit that I like to min-max things a bit when I play games. This tends to involve some research to find the "best" thing for a particular purpose. In my quest to discover the best player ship, the first ship I came across is the Springblossom. It is an amazing ship. What's not to love about 360m/s top speed?

However, after completing a play-through with the Springblossom and doing a little more research, I have discovered a better ship: The overtuned Hyperion, only available from the Poisoned Paranid start (which is only available after completing the Tormented Teladi start mission (which is only available after achieving a certain trade rank with another start)). You can also get the Hyperion through boarding with any type of start, but it's the slower variety with only 169m/s max speed. The overtuned variant can have 230+m/s max speed, which is Good Enough(tm) speed for a player ship, especially with bonus pack turbo boost.

While losing speed over the Aldrin SB, the Hyperion gains so much utility for the player. Here are some pain points that I came across with using the SB and how the Hyperion solves them.

Station Building
When building stations with mines, I always had to jump into a separate ship to tow mines. This is an extra hassle that the Hype removes, because unlike the SB, it can mount a tractor beam. Goodbye tow-truck Dragon.

Exploration
I typically keep 1 docked Kestrel outfitted for Exploration and Asteroid Scanning. In TC, there's really no faster ship for exploration.

Claimed Ships
I lieu of having a 2nd extra jump drive, I typically leave one of my docking bays empty in case I get a bailed fighter or Claim My Ship mission for a fighter. Fighters often either can't use a jump drive or don't have enough cargo to jump to the destination, so transporting them directly in my ship is faster.

For non-fighter ships which are claimed, I have a spare Jump Drive on my exploration Kestrel.

Mission Running
Cargo space is also a benefit not to be under-estimated for mission running... especially those missions that want you to transport things like Radioactive Waste or Entertainment Chips.

Long Range Death
The Hype can use Wraith missiles, and an assortment of other missile types to fit the situation. The Springy can only use Poltergeist and Spectre, the latter of which can't be player-produced. I suppose that is not much of a problem, since I have often not been able to kill ships with Spectres, being as they fire one at a time and some ships can shoot them all down. Poltergeists are also pretty slow vs M5s. My particularly disappointing experience was when doing the Balance of Power plot and defending a transport. I tried with the Springy many times, and could never destroy the fighters before they reached and blew up the transport. The Poltergeists would mostly loop around behind their target and trail it for a while before catching up. With no other missile options in the SB, I ended up having to jump in my Cobra and Flail everything to death.

Combat
Just from a personal preference standpoint, the Springy has never felt quite right to me. It behaves kindof like a forklift with a jet engine. The vast majority of my deaths have been collisions in the Springy, both undocking / scraping stations and trying to strafe larger ships. This is why I am okay with giving up some of the SB's speed for the other benefits.

Although the Springy mounts PMAMLs, a pretty powerful weapon, I found them a bit difficult to use. Against smaller ships, the projectile is too slow to be able to land hits consistently while the opponent is not heading in a straight line. Against larger ships, by the time I got into firing range, the SB was taking a pretty good pounding. Approaching at the correct angle and doing strafe runs minimizes the damage, but the ship is so fast that window of opportunity to fire is pretty small in order to avoid collisions. Slowing down is an option, but I often use Tab and Backspace to control speed. :) The main thing I was able to kill with the PMAMLs were other M6s, by strafing from longer distances. I could kill some M2s, as long as their missile defense wasn't that great, and I had enough Spectres. The cargo space is a limiting factor there.

The PSSCs on the Springblossom are categorically awesome. Fighters pretty much melt, and I often find myself not getting a chance to fire on them directly for the turrets killing them. So I will definitely miss that about the Springy.

Another obvious battle advantage for the Hype are the two docked fighters.

Summing Up

So, for future play-throughs, I'm going to do Poisoned Paranid to get that overtuned Hyperion. It seems to me that Egosoft must have designed the Hype to be the player ship. No other ship has its combination of capabilities.

Now if they just had a better version of the Cobra with more cargo space...

X3:TC Lessons Learned

Wish I'd known about the game sooner
The game is deep and immensely satisfying, and I wish I'd know about it sooner! At first, I was afraid I made a mistake in buying it because it seemed hopelessly complicated. But I watched some of CmrDave's tutorial videos, then started to discover parts of the game on my own, and eventually found it staggeringly fun. I can also play it like a simulation game and choose to leave it running, letting my empire take care of itself while I sleep or go on a date with the wife.

Download the Bonus Pack
The bonus pack is signed by the X3 developer, Egosoft, so it does not mark your game as modified. It adds some amazing functionality, and I wouldn't play X3 without it. (I tried at first, and I regretted it!) I'll just briefly mention the CLS/CAGs in the Bonus Pack below:

CLS1 = the Trade -> "Start internal commodity logistics" command, from the Commodity Logistics Mk1 software
CLS2 = the Trade -> "Start external commodity logistics" command, from the Commodity Logistics Mk2 software
CAG = the Trade -> "Start commercial representation" command

Like everything in the X3, these take some experimentation to setup just right, but their functionality is game-changing. I use CLS2 extensively for resupply and one-off deliveries. I use CAGs for most of my money-making and station/complex maintenance.

Setup stations early
I thought UTs were great when I first started using them, but little did I know that stations/complexes with CAGs are even better! Complexes in high-sec space are solid and safe money-makers, and you can totally set and forget them (with a CAG or CLS freighter). They make great storage dumps for energy or other resources they consume. You can add stations onto a complex later, and deactivate individual stations. Thus you can repurpose your station over time. Complexes are valuable in many dimensions (e.g. docking, resupply from excess resources), not just for profit. I don't use STs/UTs at all now.

Start training marines early
Even if you don't do much ship capturing, you need marines for some plots and special ships. Marine training takes ages, so start early. Every time you find a station with marines, look for any that have 3+ stars in fighting, buy them, and start them training. Everything but fighting is trainable, so the only important factor for purchasing them is fighting. Keep training them until they are 5 stars in everything else. Eventually, you will need them. I would recommend collecting up to 20 5-star marines over time. This happens to be the amount an M7M holds.

Put complex hubs near gates
This makes it quick for CAGs and resupply ships to dock at the complex. My first complex was built 150km+ from the nearest gate to avoid pirate traffic, and I had to assign it a lot of extra CAGs because each ship was spending more time flying to it than making me money. And then the game spawns pirates to attack your station anyway if you use afk SETA.

Notice that I said "near" gates, not on top of gates. And notice that I said the complex hub, which you can place a little bit away from the stations. With the latest (huge) stations that I build, I have the hub within 20-25km of the gate, but the bulk of the stations are out of the field of normal view. I do this by placing 10 or 20 of the stations pretty far apart and in a line going up out of view. Then the rest of the stations are layered above that. This is a more advanced technique that takes some practice, because the complex hubs don't always get placed where you tell them. So make sure you save before you start!

In Memory Message Bus Issues

So, I've written a small in-memory message bus.

Today I realize that under certain circumstances, it will still drop messages and create a race condition. For example, a normal workflow might be:

1. User (via UI) sends command based on read model data
2. Command is processed, generates events
3. Events are saved to an event store
4. Events published to listeners (read models, integrations, etc.)

When a power failure occurs between 3 and 4, then the read model doesn't get updated even though the event store records the event. After a restart, this creates a situation where the read model doesn't show the last change, but it's in the event store. The user, seeing the read model data, will likely try to make the change again, but either a) nothing will happen because no changes get made to the aggregate or b) a concurrency exception will get thrown because the command was issued against an old version of the aggregate.

The easiest way to work around this would be to rebuild the read models on a dirty startup, but rebuilding could take a while. Or I suppose I could write some sort of Sync function that compares the read model version with the event store version and replays the difference. But that could get complicated, and creates a dependency between the read model and event store.

I could solve the problem by putting 3 and 4 in a transaction, but that is not at all ideal, performance-wise. The other alternative that I've been trying to avoid is for the listeners to keep track of the last message they have seen and be able to request catch-up messages.

The latter introduces storage dependencies, since the handler has to remember the last message it saw across restarts. And each handler becomes a bit more complicated as saving the last seen message pointer will have to be done transactionally with handling the event. At that point, the handler might as well listen straight from the event stream rather than try to use the in-memory message bus.

Update: I've ultimately realized that this is a concern for another part of the program, and not the message delivery service itself. The part responsible for feeding events into the in-memory message bus will have to manage its position in the event stream. It can actually just save an event back to the event store (in a different stream) when it updates its stream position. Then on load, it can load it's last position from the event store itself.

Simplifying Message Handlers

One thing I don't like about the message handler examples I've seen are all the interfaces that you have to implement. For instance:

public class CustomerHandlers : 
    IHandles<ConvertLeadToCustomer>,
    IHandles<CustomerCreditLine>,
    IHandles<CorrectCustomerAddress>
    .... // lots of these
{
    public void Handle(ConvertLeadToCustomer message)
    {
        ...
    }
    
    ... // lots of these also, but they actually do stuff
}

These interfaces help you to match up messages with the handler method and gives you something to cast the handler to in order to call the appropriate method. Ultimately it ends up like this:

    ((IHandles<T>)handler).Handle((T)message);

The first alternative I discovered was to use dynamic. I didn't have to implement all the interfaces, maybe just one interface on the parent class, and let the methods document for themselves the messages they handle. Assuming I know the right method exists on the handler (due to reflection), I can let the DLR figure out how to actually call it:

    ((dynamic)handler).Handle((dynamic)message);

Note that 3 calls to the DLR are actually made. One for the message, one for the handler, and one for the method call. This works, but you run into problems if you are lazy like me and have some event handlers that handle all events. For that I use a shortcut syntax.

public class DatabaseDenormalizer: IHandles<IEvent>
{
    public void Handle(IEvent message)
    {
        // get message's actual type name
        // call a stored procedure with same name (if it exists)
        // using events properties as parameters
    }
}

In that case, dynamic wouldn't work if you had both a generic handler method and a specific one. The generic one would never get called, because the DLR always goes for the most specific call. Also, dynamic has a bit of overhead as compared to the direct method call (but not anywhere near the slowness of a MethodInfo.Invoke() call).

Edit: Correction, MethodInfo.Invoke is only "slow" in simple tests. When the call actually does some work (and is in Release mode), Invoke can be just as fast as a direct method call.

So being the crazy person that I am, I kept looking for an alternative where I could minimally decorate my message handlers, but still have decent performance. I want them to look like this:

public class CustomerHandler : IMessageHandler
{
    public void Handle(ConvertLeadToCustomer message)
    {
        ...
    }

    public void Handle(RequestCustomerCreditLine message)
    {
        ...
    }
}

So after googling around for information many times before, I finally hit the magic combination of words today to bring me to this post from 4 years ago by Jon Skeet. The last code snippet (with some tweaking) pretty much solved my conundrum. It's admittedly pretty complex code, but I'm willing to accept that for improved performance, and easier setup on my objects, plus the complexity is on code that I will likely never touch again.

Asynchronous Programming

I'm always forgetting the link to this resource on multi-threaded programming, so I thought I would post it here. It's really a great resource if you're getting started with multi-threaded programming in .NET or you just want to brush up on the nuances.

http://www.albahari.com/threading/

My Next Software Architecture

I've been thinking more about how to architect my next web-based software project. I'll be honest with you, I'm not a pro at this yet, but I'm trying to figure it out and get some experience. So I'm going to bounce some ideas off of you, Internet, as well as work out some of my thought processes. Here are some choices that I have in mind. Note that these things have been around a while, and I've played with them a little, but it's fairly new to me.

Profile and Strategy

Non-distributed

Pretty much all of the web apps I write manage a business's internal workings and are not distributed (in the large-scale sense), or are at most distributed to a few satellite locations. (This has historically been handled by file replication, database replication, and private networks between locations.) Therefore, I'm not going to add extra trappings that require a lot of configuration or overhead. For example, I won't be using a durable message queue. The core of the system will be running in-memory, and the components will mostly communicate in-memory.

CQRS

Earlier in my career, I thought it was a good idea to directly use business entities for UI views. That ends up leading to your domain objects being bloated with some UI-only concerns and vice versa. The common alternative is to create different view models as projections of your live business entities. But in order to do that, you have to load your business objects and map them to view objects. So this ends up with a lot of mapping code maintenance, and the mappings can get complicated.

The CQRS strategy keeps two sets of data updated; the domain (business) object data, and the view model data. The benefit here is that each can evolve at their own pace without greatly affecting the other. It's also a bit faster because it is no longer necessary to load the domain object first -- you just load the data straight from database to client. The downside is that you have to update 2 (or more) sets of data. But overall, it eases the complexity of trying to use one set of object for 2 distinct purposes.

DDD

My problem domains tend to be, on average, moderately complex because I'm representing internal processes of a business. DDD is meant to address complexity, but it's more about behaviors and communication patterns than code patterns. Probably the one code pattern to take away is to structure the domain objects in the same way and with the same names that customers use to describe their processes. That typically means insulating the domain objects from view and persistence concerns, and let it focus on business. If not doing CQRS also, you end up with the mapping code issue.

Messaging

Using messaging brings additional overhead to the project. (As compared to direct method calls on domain objects.) But it decouples the clients from the domain and generally just bring options to the table. In my case, I'm leaning more towards a completely HTML/Javascript-based UI, so direct method calls into .NET are not an option anyway. I could write MVC actions which are coupled to domain methods, but in that case it's about as much overhead to implement as messaging (just a different kind of overhead), and the coupling still causes ripple effects and interface maintenance.

Specific Tactics/Tech

Event Sourcing

This tactic lets you represent your domain objects in persistent storage as a series of events, which are basically just classes with the appropriate contextual information. Examples: CustomerCreatedEvent, CustomerAddressCorrectedEvent, CustomerCreditLineRequestedEvent, etc. You can imagine all of these events having a customer id. The address event, you can imagine having properties related to address information.

This has a number of advantages including trace-ability, replay-ability, easy and performant persistence story. The main disadvantage I see to this is that some of the messaging concerns (events, specifically) end up leaking into your domain logic. However, anything message-related done by your domain objects is usually very simple (plain assignment statements).

But event sourcing also opens the possibility of using non-relational databases. For me, it is usually the case that the domain deals with relationships, and view models are relatively flat. When the domain is event sourced, the persistence format is flat. This opens up the doors to alternative databases (such as NoSQL) which tend to be faster and easier to work with. Which leads me to my next point...

NoSQL Database

A traditional SQL database provides a lot of capability for reporting. However, it's a real pain to work with for application data. Most of us don't think about it because it's become second nature. Running a basic SQL statement requires you to 1) have a magic string somewhere with the query or stored procedure to run, 2) wrangle objects like SqlDataAdapter, SqlConnection, etc., 3) map parameters into the query, and 4) try to run the query and interpret the results. (For select queries, there's the additional pain of mapping/casting the DataSet back to an object.). This is painful enough that most of us create abstractions around this process. The first evolution is a DAL that maps method calls to SQL statements. Later evolutions end up being a repository and/or ORM. An ORM requires that you stay abreast of the ORM-specific extensions and code around its design. A repository (or even a simple DAL) requires manual coding and maintenance. In the end, no matter what you do, dealing with SQL is a lot of work. Contrast that to the potential ease of using NoSQL databases, which could take your object as-is and store it straight to the database (e.g. db.Store(message);). That's with no custom-built abstractions, no magic string SQL statements, and no extra ORM framework to learn. That's one compelling persistence story. Even if you need a SQL database for reporting, this can just be an additional integration point as though it were another read model to update (from the CQRS story). :)

Additionally, some NoSQL databases have REST APIs, which means I wouldn't even have to implement a read-layer for the HTML5 UI. The only part that bothers me about the REST API is security. And I haven't yet researched my options there.

WebSockets
The websockets feature is one of the most exciting web technologies to come out in a while. It allows the server and client (e.g. browser) to push messages to each other in a low-impact way. Previously, I wasn't using any external bus (e.g. MSMQ), because the overhead and administration needed to use it wasn't worth it. But websockets are rather simple to get running and are still a developer concern (as opposed to an administrative concern like MSMQ). I want to use websocket connections to serve as my program's contact with the outside world. It's not a durable bus, but since my program is in-memory and not distributed, that really doesn't matter.

One other interesting point is that web sockets provide better asynchronous capabilities. If I send an AJAX request from JavaScript, to an MVC Action, both the AJAX request (on a separate browser thread) and the MVC Action are kept open and waiting until the program finishes the action. Using websockets, I have the capability to take the command, hand it off to a queue, then have a callback send a message back to that client notifying them of completion. I can also have an event websocket to allow external listeners.

The downside to WebSockets is the feature is not widely supported currently. To host WebSockets natively in IIS, you currently have to have Windows 8 or Windows 2012 Server. As far as clients, WebSockets is not supported on most browser versions aside from the current crop.

HTML5/Javascript UI w/ MVVM
HTML5/Javascript is pretty much the direction that the web has taken. I ruled out both WebForms and MVC (the design, not the project type) for the UI due to both performance and knowledge dependencies. Both approaches (traditionally) post back to the server, let the server make some UI decisions, and then send the decision (or command) on to the business layer (or domain). It's basically an extra hop (to use a network term) as compared to a purely in-browser UI. And as we all know, external communication is often the most expensive part of a given operation.

But performance alone is not enough, and in fact using a solely HTML5/Javascript UI is only possible due to some nice Javascript frameworks. My personal choice in the matter is jQuery and Kendo UI (which has controls, MVVM, data sources, etc.). With MVVM, you can completely separate the view from the model, which makes working with the code a lot easier. I end up with the following for each view: view (.html), style (.css), view helper (.js, for any extra control-related functions the view might need), and view model (.js). Then for more complex scenarios, I add more view models and/or script files which listen for view model changes and react accordingly. It's all pretty fast.

This style of development does take some getting-used-to compared to server-side UI development. But one of the reasons I like Kendo UI is because there are a lot of functions that are pre-integrated as compared to taking separate libraries for UI controls, MVVM, validation, etc. and trying to integrate them together.

Getting .NET 4.5 WebSockets Working

I just spent most of the afternoon trying to get WebSockets working with Visual Studio 2012 and an MVC project. Most of the examples seem outdated (from old RC builds) or too low level, and the few updated ones weren't complete. So, here's my little guide on how to get started with a simple echo/chat program. It just takes 2 classes and a test page.

NOTE: WebSockets on IIS only works on Windows 8. Windows 7 does not have the necessary websocket DLL that is needed by IIS. :( That wrinkle aside, WebSockets will work with IIS 8 Regular or Express editions. This demo used IIS Express.

First, you must create a new Web API project. In this example, the project is named (with great flare and creativity) "Project1". Then, you need to add the Microsoft.WebSockets package from NuGet. (Right-click the project, click Manage NuGet Packages..., search field in upper right: microsoft.websockets, select the Microsoft.WebSockets package and click Install)

First create your WebSocket Service class, and put it somewhere in the project (mine is in /Controllers):

using Microsoft.Web.WebSockets;

using System;

namespace Project1.Controllers

{

    public class ChatClient : WebSocketHandler

    {

        public readonly Guid ConnectionId = Guid.NewGuid();

        private static WebSocketCollection chatClients =

            new WebSocketCollection();

        public override void OnOpen()

        {

            chatClients.Add(this);

            chatClients.Broadcast(

                "Client joined: " + ConnectionId.ToString()

            );

        }

        public override void OnClose()

        {

            chatClients.Broadcast(

                "Client left: " + ConnectionId.ToString()

            );

            chatClients.Remove(this);

        }

        public override void OnMessage(string message)

        {

            chatClients.Broadcast(

                ConnectionId.ToString() + " said: " + message

            );

        }

    }

}

Note I'm using Guid as a connection id, and the messages end up looking pretty ugly: 0195093f-70a5-4bfe-b707-8ac96ba94c31 said: test. But you can change that for your own needs.

The next step is to setup an ApiController. This is necessary to upgrade the HTTP request to a WebSocket request.

using Microsoft.Web.WebSockets;

using System.Net;

using System.Net.Http;

using System.Web;

using System.Web.Http;

namespace Project1.Controllers

{

    public class WebSocketController : ApiController

    {

        public HttpResponseMessage Get()

        {

            HttpContext.Current.AcceptWebSocketRequest(

                new ChatClient()

            );

            return new HttpResponseMessage(

                HttpStatusCode.SwitchingProtocols

            );

        }

    }

}

As noted in another example I found, the first using statement is VERY IMPORTANT. It adds the AcceptWebSocketRequest overload that is needed for this code. The other overloads are lower level than I wanted.

That's it! But wait you say, how can I test it? Ok, here ya go. This is a simple html page I created to test the application. It doesn't use any external files (not even jQuery). You can replace the contents of Views/Home/Index.cshtml in the project with this:

@{
    Layout = null;
}

<!DOCTYPE html>

<html>
<head>
    <meta name="viewport" content="width=device-width" />
    <title>Index</title>
    <script type="text/javascript">
        var connectButton,
            disconnectButton,
            messageInput,
            sendButton,
            responseDiv,
            uriSpan,
            uri,
            webSocket;

        var connect = function () {
            connectButton.disabled = true;
            disconnectButton.disabled = false;
            sendButton.disabled = false;
            webSocket = new WebSocket(uri);
            webSocket.onmessage = function (e) {
                responseDiv.innerHTML +=
                    '<div>' + e.data + '</div>';
            };
            webSocket.onopen = function (e) {
                responseDiv.innerHTML +=
                    '<div>Connecting...</div>';
            };
            webSocket.onclose = function (e) {
                responseDiv.innerHTML +=
                    '<div>Disconnected.</div>';
            };
            webSocket.onerror = function (e) {
                responseDiv.innerHTML += '<div>Error</div>'
            };
        };

        var disconnect = function () {
            connectButton.disabled = false;
            disconnectButton.disabled = true;
            sendButton.disabled = true;
            webSocket.close();
        };

        var sendMessage = function () {
            var message = messageInput.value;
            webSocket.send(message);
            messageInput.value = '';
        };

        var setup = function () {
            connectButton = document.getElementById('connect');
            disconnectButton =
                document.getElementById('disconnect');
            messageInput = document.getElementById('message');
            responseDiv = document.getElementById('responseLog');
            sendButton = document.getElementById('sendMessage');
            uriSpan = document.getElementById('uri');
            uri = 'ws://localhost:52618/api/websocket';
            uriSpan.innerHTML = uri;
        };
    </script>
</head>
<body onload="setup()" style="font-family: sans-serif;">
    <div>
        <div>
            <span id="uri"></span>
            <button id="connect" onclick="connect()">
                Connect
            </button>
            <button id="disconnect"
                disabled="disabled"
                onclick="disconnect()">Disconnect</button>
        </div>
        <label for="message">Message</label>
        <input id="message"/>
        <button id="sendMessage"
            onclick="sendMessage()"
            disabled="disabled">Send</button>
        <hr />
        <label for="responseLog">Response</label>
        <div id="responseLog"
            style="border: 1px solid grey;
                   width: 600px; height: 400px;
                   overflow: auto;
                   font-family: monospace;">
        </div>
    </div>
</body>
</html>

NOTE: Change the uri value to match the port your project uses. Otherwise it should work as is.

And here was my test run in Chrome 21 and IE 10.