dev csharp, ios, swift, wpf, xamarin, tfl_api

Recently I discovered that Transport for London (TFL) has some great APIs that I can play around with some familiar data. It’s very to use as an API key is not even mandatory. My main goal here is to discover what I can do with this data and build a few user interfaces consuming it. All source code is available on GitHub

Tube status

The API endpoint I will use returns the current status of tube lines, an array of the following JSON objects:

{
    "$type": "Tfl.Api.Presentation.Entities.Line, Tfl.Api.Presentation.Entities",
    "id": "central",
    "name": "Central",
    "modeName": "tube",
    "created": "2015-10-14T10:31:00.39",
    "modified": "2015-10-14T10:31:00.39",
    "lineStatuses": [
        {
            "$type": "Tfl.Api.Presentation.Entities.LineStatus, Tfl.Api.Presentation.Entities",
            "id": 0,
            "statusSeverity": 10,
            "statusSeverityDescription": "Good Service",
            "created": "0001-01-01T00:00:00",
            "validityPeriods": []
        }
    ],
    "routeSections": [],
    "serviceTypes": [
        {
            "$type": "Tfl.Api.Presentation.Entities.LineServiceTypeInfo, Tfl.Api.Presentation.Entities",
            "name": "Regular",
            "uri": "/Line/Route?ids=Central&serviceTypes=Regular"
        }
    ]
}

Visualizing the data - line colours

TFL have standard colours for tube lines which are documented here. So I created a small lookup json using that reference:

[
    { "id": "bakerloo", "CMYK": { "M": 58, "Y": 100, "K": 33 }, "RGB":  { "R": 137, "G": 78, "B": 36 } },
    { "id": "central", "CMYK": { "M": 95, "Y": 100 }, "RGB":  { "R": 220, "G": 36, "B": 31 } },
    { "id": "circle",  "CMYK": { "M": 16, "Y": 100 }, "RGB":  { "R": 255, "G": 206, "B": 0 } },
    { "id": "district", "CMYK": { "C":  95, "Y": 100, "K": 27 }, "RGB":  { "R": 0, "G": 114, "B": 41 } },
    { "id": "hammersmith-city", "CMYK": { "M": 45, "Y": 10 }, "RGB":  { "R": 215, "G": 153, "B": 175 } },
    { "id": "jubilee", "CMYK": { "C": 5, "K": 45 }, "RGB":  { "R": 134, "G": 143, "B": 152 } },
    { "id": "metropolitan", "CMYK": { "C": 5, "M": 100, "K": 40 }, "RGB":  { "R": 117, "G": 16, "B": 86 } },
    { "id": "northern", "CMYK": { "K": 100 }, "RGB":  { "R": 0, "G": 0, "B": 0 } },
    { "id": "piccadilly", "CMYK": { "C": 100, "M": 88, "K": 5 }, "RGB":  { "R": 0, "G": 25, "B": 168 } },
    { "id": "victoria", "CMYK": { "C": 85, "M": 19 }, "RGB":  { "R": 0, "G": 160, "B": 226 } },
    { "id": "waterloo-city", "CMYK": { "C": 47, "Y": 32 }, "RGB":  { "R": 118, "G": 208, "B": 189 } }
]

I was hoping to map status values to colours as well (i.e. “Severe delays” to red) but there is no official guide to that. The status codes and values can be retrieved from this endpoint: https://api.tfl.gov.uk/line/meta/severity which returns a collection of objects like this:

{
    "$type": "Tfl.Api.Presentation.Entities.StatusSeverity, Tfl.Api.Presentation.Entities",
    "modeName": "tube",
    "severityLevel": 2,
    "description": "Suspended"
}

I simplified it for my purposes (just the values for tube):

[
    { "severityLevel": 0, "description": "Special Service" },
    { "severityLevel": 1, "description": "Closed" },
    { "severityLevel": 2, "description": "Suspended" },
    { "severityLevel": 3, "description": "Part Suspended" },
    { "severityLevel": 4, "description": "Planned Closure" },
    { "severityLevel": 5, "description": "Part Closure" },
    { "severityLevel": 6, "description": "Severe Delays" },
    { "severityLevel": 7, "description": "Reduced Service" },
    { "severityLevel": 8, "description": "Bus Service" },
    { "severityLevel": 9, "description": "Minor Delays" },
    { "severityLevel": 10, "description": "Good Service" },
    { "severityLevel": 11, "description": "Part Closed" },
    { "severityLevel": 12, "description": "Exist Only" },
    { "severityLevel": 13, "description": "No Step Free Access" },
    { "severityLevel": 14, "description": "Change of frequency" },
    { "severityLevel": 15, "description": "Diverted" },
    { "severityLevel": 16, "description": "Not Running" },
    { "severityLevel": 17, "description": "Issues Reported" },
    { "severityLevel": 18, "description": "No Issues" },
    { "severityLevel": 19, "description": "Information" },
    { "severityLevel": 20, "description": "Service Closed" },
]

I will keep it around but in this initial version I won’t use it as description is returned with the status query anyway. But it was still a useful exercise to figure out there is no “official” colour for status values. After all what’s the colour of “No Step Free Access” or “Exist Only”? There is a also reason field that explains the effects of any delays etc. which should ne displayed along with the severity especially when there are some disruptions in the service.

‘Nuff said about the data! Let’s start building something with it!

Core library

As I will build several API call to retrieve tube status is encapsulated in the core library which basically has sends the HTTP request, parses the JSON and returns the LineInfo list:

public class Fetcher
{
    private readonly string _apiEndPoint = "https://api.tfl.gov.uk/line/mode/tube/status?detail=true";

    public List<LineInfo> GetTubeInfo()
    {
        var client = new RestClient(_apiEndPoint);
        var request = new RestRequest("/", Method.GET);
        request.AddHeader("Content-Type", "application/json");
        var response = (RestResponse)client.Execute(request);
        var content = response.Content;
        var tflResponse = JsonConvert.DeserializeObject<List<TflLineInfo>>(content);

        var lineInfoList = tflResponse.Select(t =>
            new LineInfo()
            {
                Id = t.id,
                Name = t.name,
                Reason = t.lineStatuses[0].reason,
                StatusSeverityDescription = t.lineStatuses[0].statusSeverityDescription,
                StatusSeverity = t.lineStatuses[0].statusSeverity
            }).ToList();

        return lineInfoList;
    }
}

LineInfo class contains the current status with the description. It also contains the colour defined by TFL for that tube line:

public class LineInfo
{
    public string Id { get; set; }
    public string Name { get; set; }
    public int StatusSeverity { get; set; }
    public string StatusSeverityDescription { get; set; }
    public string Reason { get; set; }
    public RGB LineColour
    {
        get
        {
            return TubeColourHelper.GetRGBColour(this.Id);
        }
    }
}

As the line colours aren’t returned by the service I have to populate it by a helper class:

public class TubeColourHelper
{
    private static Dictionary<string, RGB> _tubeColorRGBDictionary = new Dictionary<string, RGB>();

    static TubeColourHelper()
    {
        _tubeColorRGBDictionary = new Dictionary<string, RGB>();

        string json = File.ReadAllText("./data/colours.json");
        var tubeColors = JArray.Parse(json);
        foreach (var tubeColor in tubeColors)
        {
            _tubeColorRGBDictionary.Add(tubeColor["id"].Value<string>(), new RGB(
                tubeColor["RGB"]["R"]?.Value<int>() ?? 0,
                tubeColor["RGB"]["G"]?.Value<int>() ?? 0,
                tubeColor["RGB"]["B"]?.Value<int>() ?? 0));
        }
    }
    
    public static RGB GetRGBColour(string lineId)
    {
        if (!_tubeColorRGBDictionary.ContainsKey(lineId))
        {
            throw new ArgumentException($"Colour for line [{lineId}] could not be found in RGB colour map");
        }

        return _tubeColorRGBDictionary[lineId];
    }
}

The static constructor runs only the first time it is accessed, reads the colours.json and populates the dictionary. From then on it’s just a lookup in memory.

First client: C# Console Application on Windows

Time to develop our first client and see some actual results. As it’s generally the case with console applications this one is pretty simple and hassle-free. I decided to start with that one just to see the core library is working as expected.

class Program
{
    static void Main(string[] args)
    {
        var fetcher = new Fetcher();
        var viewer = new ConsoleViewer();

        bool exit = false;

        viewer.DisplayTubeStatus(fetcher.GetTubeInfo());

        do
        {
            ConsoleKeyInfo key = System.Console.ReadKey();
            switch (key.Key)
            {
                case ConsoleKey.F5:
                    viewer.DisplayTubeStatus(fetcher.GetTubeInfo());
                    break;
                case ConsoleKey.Q:
                    exit = true;
                    break;
                default:
                    System.Console.WriteLine("Unknown command");
                    break;
            }
        }
        while (!exit);
    }
}

Displays the results when it’s first run. You can refresh by pressing F5 or quit by pressing Q. The output looks like this:

The problem with console application is that I wasn’t able to use RGB values directly as the console only supports an enumeration called ConsoleColor.

Second client: WPF Application on Windows

Now let’s look at a more graphical UI, a WPF client:

Same idea, display the results upon first run then call the service again on Refresh button’s click event.

Third client: iOS App with Xamarin

I’ve recently subscribed to Xamarin and one of the main reasons for starting this project was to see it in action. What I was mostly curious about was if I could use my C# libraries using NuGet packages on an iOS application developed with Xamarin. This would allow me build apps significantly faster.

It didn’t work out of the box because I used C# 6.0 and .NET Framework 4.5.2 on the Windows side but it wasn’t available on the Mac.

But wasn’t too hard to change the framework and make some small modifications to make it work. Good news is that it supports NuGet and most common libraries have Mono support including RestSharp and Newtonsoft.Json which I used in this project

I had to remove and add them but finally they worked fine so I didn’t have to change anything in the code.

I won’t go into implementation details as there’s not much change. The app has one table view controller and it calls the core library to get the results and assigns them to the table’s data source. It’s a relief that I could have the same functionality as Windows with just minor changes.

public override void ViewDidLoad()
{
    base.ViewDidLoad();

    var fetcher = new Fetcher();
    var lineInfoList = fetcher.GetTubeInfo();
    TableView.Source = new TubeStatusTableViewControllerSource(lineInfoList.ToArray());
    TableView.ReloadData();
}

Anyway, more on Xamarin later after I cover the Swift version.

Fourth client: iOS App with Swift

Last but not least, here comes Swift client built with XCode. Naturally this one cannot use the core library that the first 3 clients shared (which is good because I was looking for a chance to practice handling HTTP requests and parsinng JSON with Swift anyway).

I didn’t use any external libraries so the implementation is a bit long but mainly it sends the request using NSURLSession and NSSessionDataTask.

func getTubeStatus(completionHandler: (result: [LineInfo]?, error: NSError?) -> Void) {
    let parameters = ["detail" : "true"]
    let mutableMethod : String = Methods.TubeStatus
    
    taskForGETMethod(mutableMethod, parameters: parameters) { JSONResult, error in
        if let error = error {
            completionHandler(result: nil, error: error)
        } else {
            if let results = JSONResult as? [AnyObject] {
                let lineStatus = LineInfo.lineStatusFromResults(results)
                completionHandler(result: lineStatus, error: nil)
            }
       }
    }
}

Then constructs the LineInfo objects by calling the static lineStatusFromResults method:

static func lineStatusFromResults(results: [AnyObject]) -> [LineInfo] {
    var lineStatus = [LineInfo]()
    for result in results {
        lineStatus.append(LineInfo(status: result))
    }
    
    return lineStatus
}

which creates a new LineInfo and adds to resultset:

init(status: AnyObject) {

    Id = status["id"] as! String
    Name = status["name"] as! String
    StatusSeverity = status["lineStatuses"]!![0]!["statusSeverity"] as! Int
    StatusSeverityDescription = status["lineStatuses"]!![0]!["statusSeverityDescription"] as! String
    LineColour = RGB(R: 0, G: 0, B: 0)
}

JSON parsing is a bit nasy because of unwrapping the optionals. I’ll look into SwiftyJSON later on which is a popular JSON library for Swift.

Finally the controller displays the results:

override func viewWillAppear(animated: Bool) {
    super.viewWillAppear(animated)
    
    TFLClient.sharedInstance().getTubeStatus { lineStatus, error in
        
        if let lineStatus = lineStatus {
            self.lineInfoList = lineStatus
            dispatch_async(dispatch_get_main_queue()) {
                self.tableView!.reloadData()
            }
        } else {
            print(error)
        }
    }
}

And the custom cells are created when data is loaded and the text and colours are set:

override func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell {
    let cell = tableView.dequeueReusableCellWithIdentifier("TubeInfoCell", forIndexPath: indexPath) as! TubeInfoTableViewCell
    let lineStatus = lineInfoList[indexPath.row]
    
    cell.backgroundColor = colourHelper.getTubeColor(lineStatus.Id)
    
    cell.lineName?.text = lineStatus.Name
    cell.lineName?.textColor = UIColor.whiteColor()
    cell.severityDescription?.text = lineStatus.StatusSeverityDescription
    cell.severityDescription?.textColor = UIColor.whiteColor()

    return cell
}

And here’s the output:

Xamarin vs Swift

Here’s a quick overview and comparison of both platforms based on my (limited) experiences with this toy project:

  • XCode is much faster when building and deploying
  • XamarinStudio doesn’t seem to be very intuitive at times. For examples the code snippets use Java-notation
  • The more I use Swift the more I like it and it doesn’t slow me down terribly. Once you get used to it the difference is syntax more or less. For example the following two methods do the same thing:

    Xamarin:

      public override nint RowsInSection (UITableView tableview, nint section)
      {
          return lineInfoList.Length;
      }
    

    Swift:

      override func tableView(tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
              return lineInfoList.count
          }
    

    I can even argue I’d be more comfortable with the Swift version here as I have no idea what a “nint” is as it’s an input parameter in the Xamarin version.

  • The idea behind Xamarin subscription was to develop iOS apps quickly as I’m a seasoned C# developer and feel comfortable with it. But turns it, I can’t move as fast as I expected. With the Indie subscription you can only use Xamarin Studio. Enabling Visual Studio is only allowed with the business version which costs $1000/year. And Xamarin Studio is a brand new IDE for me so it definitely has a learning curve. Also I’m getting used to XCode now (besides the fact that it crashes hundred times a day on average!)

Conclusion

This was just a reconnaisance mission to explore the TFL API, iOS development with Xamarin and Swift. It was a fun exercise for me, I hope anyone who reads this can benefit from it too.

Resources

dev csharp, nancy

Recently I needed to simulate HTTP responses from a 3rd party. I decided to use Nancy to quickly build a local web server that would handle my test requests and return the responses I wanted.

Here’s the definition of Nancy from their official website:

Nancy is a lightweight, low-ceremony, framework for building HTTP based services on .Net and Mono.

It can handle DELETE, GET, HEAD, OPTIONS, POST, PUT and PATCH requests. It’s very easy to customize and extend as it’s module-based. In order to build our tiny web server we are going to need self-hosting package:

Install-Package Nancy.Hosting.Self

This would automatically install Nancy as it depends on that package.

Self-hosting in action

The container application can be anything as long it keeps running one way or another. A background service would be ideal for this task. Since all I need is testing I just created a console application and added Console.ReadKey() statement to keep it “alive”

class Program
{
    private string _url = "http://localhost";
    private int _port = 12345;
    private NancyHost _nancy;

    public Program()
    {
        var uri = new Uri( $"{_url}:{_port}/");
        _nancy = new NancyHost(uri);
    }

    private void Start()
    {
        _nancy.Start();
        Console.WriteLine($"Started listennig port {_port}");
        Console.ReadKey();
        _nancy.Stop();
    }

    static void Main(string[] args)
    {
        var p = new Program();
        p.Start();
    }
}

If you try this code, it’s likely that you’ll have an error (AutomaticUrlReservationCreationFailureException)

saying:

The Nancy self host was unable to start, as no namespace reservation existed for the provided url(s).

Please either enable UrlReservations.CreateAutomatically on the HostConfiguration provided to 
the NancyHost, or create the reservations manually with the (elevated) command(s):

netsh http add urlacl url="http://+:12345/" user="Everyone"

There are 3 ways to resolve this issue and two of which are already suggested in the error message:

  1. In an elevated command prompt (fancy way of saying run as administrator!), run

     netsh http add urlacl url="http://+:12345/" user="Everyone"
    

    What add urlacl does is

    Reserves the specified URL for non-administrator users and accounts

    If you want to delete it later on you can use the following command

     netsh http delete urlacl url=http://+:12345/
    
  2. Specify a host configuration to NancyHost like this:

     var configuration = new HostConfiguration()
     {
         UrlReservations = new UrlReservations() { CreateAutomatically = true }
     };
    	
     _nancy = new NancyHost(configuration, uri);
    

    This essentially does the same thing and a UAC prompt pops up so it’s not that automatical!

  3. Run the Visual Studio (and the standalone application when deployed) as administrator

After applying either one of the 3 solutions, let’s run the application and try the address http://localhost:12345 in a browser and we get …

Excellent! We are actually getting a response from the server even though it’s just a 404 error.

Now let’s add some functionality, otherwise it isn’t terribly useful.

Handling requests

Requests are handled by modules. Creating a module is as simple as creating a class deriving from NancyModule. Let’s create two handlers for the root, one for GET verbs and one for POST:

public class SimpleModule : Nancy.NancyModule
{
    public SimpleModule()
    {
        Get["/"] = _ => "Received GET request";

        Post["/"] = _ => "Received POST request";
    }
}

Nancy automatically discovers all modules so we don’t have to register them. If there are conflicting handlers the last one discovered overrides the previous ones. For example the following example would work fine and the second GET handler will be executed:

public class SimpleModule : Nancy.NancyModule
{
    public SimpleModule()
    {
        Get["/"] = _ => "Received GET request";

        Post["/"] = _ => "Received POST request";

        Get["/"] = _ => "Let me have the request!";
    }
}

Working with input data: Request parameters

In the simple we used underscore to represent input as didn’t care but most of the time we would. In that case we can get the request parameters as a DynamicDictionary (a type that comes with Nancy). For example let’s create a route for /user:

public SimpleModule()
{
    Get["/user/{id}"] = parameters =>
    {
        if (((int)parameters.id) == 666)
        {
            return $"All hail user #{parameters.id}! \\m/";
        }
        else
        {
            return "Just a regular user!";
        }
    };
}

And send the GET request:

GET http://localhost:12345/user/666 HTTP/1.1
User-Agent: Fiddler
Host: localhost:12345
Content-Length: 2

which would return the response:

HTTP/1.1 200 OK
Content-Type: text/html
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 10 Nov 2015 11:40:08 GMT
Content-Length: 23

All hail user #666! \m/

Working with input data: Request body

Now let’s try to handle the data posted in the request body. Data posted in the body can be accessed though this.Request.Body property such as for the following request

POST http://localhost:12345/ HTTP/1.1
User-Agent: Fiddler
Host: localhost:12345
Content-Length: 55
Content-Type: application/json

{
    "username": "volkan",
    "isAdmin": "sure!"
}

this code would first convert the request stream to a string and deserialize it to a POCO:

Post["/"] = _ =>
{
    var id = this.Request.Body;
    var length = this.Request.Body.Length;
    var data = new byte[length];
    id.Read(data, 0, (int)length);
    var body = System.Text.Encoding.Default.GetString(data);

    var request = JsonConvert.DeserializeObject<SimpleRequest>(body);
    return 200;
};

If the was posted from a form for example and sent in the following format in the body

username=volkan&isAdmin=sure!

then we could simply convert it to a dictionary with a little bit of LINQ:

Post["/"] = parameters =>
{
    var id = this.Request.Body;
    long length = this.Request.Body.Length;
    byte[] data = new byte[length];
    id.Read(data, 0, (int)length);
    string body = System.Text.Encoding.Default.GetString(data);
    var p = body.Split('&')
        .Select(s => s.Split('='))
        .ToDictionary(k => k.ElementAt(0), v => v.ElementAt(1));

    if (p["username"] == "volkan")
        return "awesome!";
    else
        return "meh!";
};

This is nice but it’s a lot of work to read the whole and manually deserialize it! Fortunately Nancy supports model binding. First we need to add the using statement as the Bind extension method lives in Nancy.ModelBinding

using Nancy.ModelBinding;

Now we can simplify the code by the help of model binding:

Post["/"] = _ =>
{
    var request = this.Bind<SimpleRequest>();
    return request.username;
};

The important thing to note is to send the data with the appropriate content type. For the form data example the request should be like this:

POST http://localhost:12345/ HTTP/1.1
User-Agent: Fiddler
Host: localhost:12345
Content-Length: 29
Content-Type: application/x-www-form-urlencoded

username=volkan&isAdmin=sure!

It also works for binding JSON to the same POCO.

Preparing responses

Nancy is very flexible in terms of responses. As shown in the above examples you can return a string

Post["/"] = _ =>
{
    return "This is a valid response";
};

which would yield this HTTP message on the wire:

HTTP/1.1 200 OK
Content-Type: text/html
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 10 Nov 2015 15:48:12 GMT
Content-Length: 20

This is a valid response

Response code is set to 200 - OK automatically and the text is sent in the response body.

We can just set the code and return a response with a simple one-liner:

Post["/"] = _ => 405;

which would produce:

HTTP/1.1 405 Method Not Allowed
Content-Type: text/html
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 10 Nov 2015 15:51:36 GMT
Content-Length: 0

To prepare more complex responses with headers and everything we can construct a new Response object like this:

Post["/"] = _ =>
{
    string jsonString = "{ username: \"admin\", password: \"just kidding\" }";
    byte[] jsonBytes = Encoding.UTF8.GetBytes(jsonString);

    return new Response()
    {
        StatusCode = HttpStatusCode.OK,
        ContentType = "application/json",
        ReasonPhrase = "Because why not!",
        Headers = new Dictionary<string, string>()
        {
            { "Content-Type", "application/json" },
            { "X-Custom-Header", "Sup?" }
        },
        Contents = c => c.Write(jsonBytes, 0, jsonBytes.Length)
    };
};

and we would get this at the other end of the line:

HTTP/1.1 200 Because why not!
Content-Type: application/json
Server: Microsoft-HTTPAPI/2.0
X-Custom-Header: Sup?
Date: Tue, 10 Nov 2015 16:09:19 GMT
Content-Length: 47

{ username: "admin", password: "just kidding" }

Response also comes with a lot of useful methods like AsJson, AsXml and AsRedirect. For example we could simplify returning a JSON response like this:

Post["/"] = _ =>
{
    return Response.AsJson<SimpleResponse>(
        new SimpleResponse()
        {
            Status = "A-OK!", ErrorCode = 1, Description = "All systems are go!"
        });
};

and the result would contain the appropriate header and status code:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 10 Nov 2015 16:19:18 GMT
Content-Length: 68

{"status":"A-OK!","errorCode":1,"description":"All systems are go!"}

One extension I like is the AsRedirect method. The following example would return Google search results for a given parameter:

Get["/search"] = parameters =>
{
    string s = this.Request.Query["q"];
    return Response.AsRedirect($"http://www.google.com/search?q={s}");
};

HTTPS

What if we needed to support HTTPS for our tests for some reason? Fear not, Nancy covers that too. By default, if we just try to use HTTPS by changing the protocol we would get this exception:

The connection to ‘localhost’ failed. System.Security.SecurityException Failed to negotiate HTTPS connection with server.fiddler.network.https HTTPS handshake to localhost (for #2) failed. System.IO.IOException Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

The solution is to add create a self-signed certificate and add it using netsh http add command. Here’s the step-by-step process:

  1. Create a self-signed certificate: Open a Visual Studio command prompt and enter the following command:
makecert nancy.cer

You can provide more properties so that it would look with a name that makes sense. Here’s an MSD page to

  1. Run mmc and add Certificates snap-in. Make sure to select Computer Account.

I selected My User Account at first and it gave the following error:

SSL Certificate add failed, Error: 1312 A specified logon session does not exist. It may already have been terminated.

In that case the solution is just to drag and drop the certificate to the computer account as shown below:

  1. Right-click on Certificates (Local Computer) -> Personal -> Certificates and select All tasks -> Import and browse to nancy.cer file created in Step 1

  2. Double-click on the certificate, switch to Details tab and scroll to the bottom and copy the Thumbprint value (and remove the spaces after copied it)

  1. Now enter the following commands. The first one is the same as before, just with HTTPS as protocol. The second command add the certificate we’ve just created.
netsh http add urlacl url=https://+:12345/ user="Everyone"

netsh http add sslcert ipport=0.0.0.0:1234 ccerthash=653a1c60d4daaae00b2a103f242eac965ca21bec appid={A0DEC7A4-CF28-42FD-9B85-AFFDDD4FDD0F} clientcertnegotiation=enable

Here appid can be any GUID.

Let’s take it out for a test drive:

Get["/"] = parameters =>
{
    return "Response over HTTPS! Weeee!";
};

This request

GET https://localhost:12345 HTTP/1.1
Host: localhost:12345


returns this response

HTTP/1.1 200 OK
Content-Type: text/html
Server: Microsoft-HTTPAPI/2.0
Date: Wed, 11 Nov 2015 10:24:58 GMT
Content-Length: 27

Response over HTTPS! Weeee!

Conclusion

There are a few alternatives when you need a small web server to test something locally. Nancy is one of them. It’s easy to configure, use and it’s lightweight. Apparently you can even host in on a Raspberry Pi!

Resources

devdesign csharp, mef, managed_extensibility_framework, plugin

In this post I will try to cover some of the basic concepts and features of MEF over a working example. In the future I’ll post more articles demonstraint MEF usage with more complex applications.

Background

Many successful and popular applications, such as Visual Studio, Eclipse, Sublime Text, support a plug-in model. Adopting a plugin-based model, whenever possible, has quite a few advantages:

  • Helps to keep the core lightweight instead of cramming all features into the same code-base.
  • Helps to make the application more robust: New functionality can be added without changing any existing code.
  • Helps to make development easier as different modules can be developed by different people simultaneously
  • Allows plugin development without distributing the main the source code

Extensibility is based on composition and it is very helpful to build SOLID compliant applications as it adopts Open/closed and Dependency Inversion principles.

MEF is part of .NET framework as of version 4.0 and it lives inside System.ComponentModel.Composition namespace. This is also the standard extension model that has been used in Visual Studio. It is not meant to replace Invesion of Control (IoC) frameworks. It is rather meant to simplify building extensible applications using dependency injection based on component composition.

Some terminology

Before diving into the sample, let’s look at some MEF terminology and core terms:

  • Part: Basic elements in MEF are called parts. Parts can provide services to other parts (exporting) and can consume other parts’ services (importing).

  • Container: This is the part that performs the composition. Most common one is CompositionContainer class.

  • Catalog: In order to discover the parts, containers use catalogs. There are various catelogs suplied by MEF such as

    • AssemblyCatalog: Discovers attributed parts in a managed code assembly.
    • DirectoryCatalog: Discovers attributed parts in the assemblies in a specified directory.
    • AggregateCatalog: A catalog that combines the elements of ComposablePartCatalog objects.
    • ApplicationCatalog: Discovers attributed parts in the dynamic link library (DLL) and EXE files in an application’s directory and path
  • Export / import: The way the plugins make themselves discoverable is by exporting their implementation of a contract. A contract is simply a common interface that the application and the plugins understand so they can speak the same language so to speak.

Sample Project

As I learn best by playing around, I decided to start with a simple project. I’ve recently published a sample project for Strategy design pattern which I blogged here. In this post I will use the same project and convert it into a plugin-based version.

IP Checker with MEF v1: Bare Essentials

At this point we have everything we need for the first version of the plugin-based IP checker. Firstly, I divided my project into 5 parts:

  • IPCheckerWithMEF.Lab: The consumer application
  • IPCheckerWithMEF.Contract: Project containing the common interface
  • Plugins: Extensions for the main application
    • IPCheckerWithMEF.Plugins.AwsIPChecker
    • IPCheckerWithMEF.Plugins.CustomIPChecker
    • IPCheckerWithMEF.Plugins.DynDnsIPChecker

I set the output folder of the plugins to a directory called Plugins at the project level.

Let’s see some code!

For this basic version we need 3 things:

  • A container to handle the composition.
  • A catalog that the container can use to discover the plugins.
  • A way to tell which classes should be discovered and imported

In this sample I used a DirectoryCatalog that points to the output folder of the plugin projects. So after adding the required parts above the main application shaped up to be something like this:

public class MainApplication
{
    private CompositionContainer _container;

    [ImportMany(typeof(IIpChecker))]
    public List<IIpChecker> IpCheckerList { get; set; }

    public MainApplication(string pluginFolder)
    {
        var catalog = new DirectoryCatalog(pluginFolder);
        _container = new CompositionContainer(catalog);

        LoadPlugins();
    }

    public void LoadPlugins()
    {
        try
        {
            _container.ComposeParts(this);
        }
        catch (CompositionException compositionException)
        {
            Console.WriteLine(compositionException.ToString());
        }
    }
}

In the constructor, it instantiates a DirectoryCatalog with the given path and passes it to the container. The container imports IIpChecker type objects found in the assemblies inside that folder. Note that we didn’t do anything about IpCheckerList. By decorating it with ImportMany attribute we declared that it’s to be filled by the composition engine. In this example we could only use ImportMany as opposed to Import which would look for a single part to compose. If we used Import we would get the following exception:

Now to complete the circle we need to export our plugins with Export attribute such as:

[Export(typeof(IIpChecker))]
public class AwsIPChecker : IIpChecker
{
    public string GetExternalIp()
    {
        // ...
    }
}

Alternatively we can use InheritedExport attribute on the interface to export any class that implements the IIpChecker interface.

[InheritedExport(typeof(IIpChecker))]
public interface IIpChecker
{
    string GetExternalIp();
}

This way the plugins would still be discovered even if they weren’t decorated with Export attribute because of this inheritance model.

Putting it together

Now that we’ve seen the plugins that export the implementation and part that discovers and imports them let’s see them all in action:

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Starting the main application");

        string pluginFolder = @"..\..\..\Plugins\";
        var app = new MainApplication(pluginFolder);

        Console.WriteLine($"{app.IpCheckerList.Count} plugin(s) loaded..");
        Console.WriteLine("Executing all plugins...");

        foreach (var ipChecker in app.IpCheckerList)
        {
            Console.WriteLine(ObfuscateIP(ipChecker.GetExternalIp()));
        }
    }

    private static string ObfuscateIP(string actualIp)
    {
        return Regex.Replace(actualIp, "[0-9]", "*");
    }
}

We create the consumer application that loads all the plugins in the directory we specify. Then we can loop over and execute all of them:

So far so good. Now, let’s try to export some metadata about our plugins so that we can display the loaded plugins to the user.

IP Checker with MEF v2: Metadata comes into play

In almost all applications plugins come with some sort of information so that the user can identify which ones have been installed and what they do. To export the extra data let’s add a new interface:

public interface IPluginInfo
{
    string DisplayName { get; }
    string Description { get; }
    string Version { get; }
}

And on the plugins we fill that data and export it using the ExportMetadata attribute:

[Export(typeof(IIpChecker))]
[ExportMetadata("DisplayName", "Custom IP Checker")]
[ExportMetadata("Description", "Uses homebrew service developed with Node.js and hosted on Heroku")]
[ExportMetadata("Version", "2.1")]
public class CustomIpChecker : IIpChecker
{
    // ...
}

In v1, we only imported a list of objects implementing IIpChecker. So how do we accommodate this new piece of information? In order to do that we have to change the way we import the plugins and use the Lazy construct:

[ImportMany]
public List<Lazy<IIpChecker, IPluginInfo>> Plugins { get; set; }

According to MSDN this is mandatory to get metadata out of plugins:

The importing part can use this data to decide which exports to use, or to gather information about an export without having to construct it. For this reason, an import must be lazy to use metadata

So let’s load and display this new plugin information:

private static void PrintPluginInfo()
{
    Console.WriteLine($"{_app.Plugins.Count} plugin(s) loaded..");
    Console.WriteLine("Displaying plugin info...");
    Console.WriteLine();

    foreach (var ipChecker in _app.Plugins)
    {
        Console.WriteLine("----------------------------------------");
        Console.WriteLine($"Name: {ipChecker.Metadata.DisplayName}");
        Console.WriteLine($"Description: {ipChecker.Metadata.Description}");
        Console.WriteLine($"Version: {ipChecker.Metadata.Version}");
    }
}

Notice that we access the metadata through [PluginName].Metadata.[PropertyName] properties. To access the actual plugin and call the exported methods we have to use [PluginName].Value such as:

foreach (var ipChecker in _app.Plugins)
{
    ipChecker.Value.GetExternalIp();
}

Managing the plugins

What if we want to add or remove plugins at runtime? We can do it without restarting the application but refreshing the catalog and calling the container’s ComposeParts method again.

In this sample application I added a FileSystemWatcher that listens to the Created and Deleted events on the Plugins folder and calls the LoadPlugins method of the application when an event fires. LoadPlugins first refreshes the catalog and composes the parts:

public void LoadPlugins()
{
    try
    {
        _catalog.Refresh();
        _container.ComposeParts(this);
    }
    catch (CompositionException compositionException)
    {
        Console.WriteLine(compositionException.ToString());
    }
}

But making this change alone isn’t sufficient and we would end up getting a CompositionException:

By default recomposition is disabled so we have to specify it explicitly while importing parts:

[ImportMany(AllowRecomposition = true)]
public List<Lazy<IIpChecker, IPluginInfo>> Plugins { get; set; }

After these changes the final version of composing class looks like this:

public class MainApplication
{
    private CompositionContainer _container;
    private DirectoryCatalog _catalog;

    [ImportMany(AllowRecomposition = true)]
    public List<Lazy<IIpChecker, IPluginInfo>> Plugins { get; set; }

    public MainApplication(string pluginFolder)
    {
        _catalog = new DirectoryCatalog(pluginFolder);
        _container = new CompositionContainer(_catalog);

        LoadPlugins();
    }

    public void LoadPlugins()
    {
        try
        {
            _catalog.Refresh();
            _container.ComposeParts(this);
        }
        catch (CompositionException compositionException)
        {
            Console.WriteLine(compositionException.ToString());
        }
    }
}

and the client app:

class Program
{
    private static readonly string _pluginFolder = @"..\..\..\Plugins\";
    private static FileSystemWatcher _pluginWatcher;
    private static MainApplication _app;

    static void Main(string[] args)
    {
        Console.WriteLine("Starting the main application");

        _pluginWatcher = new FileSystemWatcher(_pluginFolder);
        _pluginWatcher.Created += PluginWatcher_FolderUpdated;
        _pluginWatcher.Deleted += PluginWatcher_FolderUpdated;
        _pluginWatcher.EnableRaisingEvents = true;

        _app = new MainApplication(_pluginFolder);

        PrintPluginInfo();

        Console.ReadLine();
    }

    private static void PrintPluginInfo()
    {
        Console.WriteLine($"{_app.Plugins.Count} plugin(s) loaded..");
        Console.WriteLine("Displaying plugin info...");
        Console.WriteLine();

        foreach (var ipChecker in _app.Plugins)
        {
            Console.WriteLine("----------------------------------------");
            Console.WriteLine($"Name: {ipChecker.Metadata.DisplayName}");
            Console.WriteLine($"Description: {ipChecker.Metadata.Description}");
            Console.WriteLine($"Version: {ipChecker.Metadata.Version}");
        }
    }

    private static void PluginWatcher_FolderUpdated(object sender, FileSystemEventArgs e)
    {
        Console.WriteLine();
        Console.WriteLine("====================================");
        Console.WriteLine("Folder changed. Reloading plugins...");
        Console.WriteLine();
        
        _app.LoadPlugins();

        PrintPluginInfo();
    }
}

After these changes I started the application with 2 plugins in the target folder and added a 3rd one while it’s running and got this output:

It also works the same way for deleted plugins but not for updates because the assemblies are locked by .NET. Adding new plugins at runtime is painless but removing and updating would require more attention as the plugin might be running at the time.

Resources