Cloud Computing

**It’s been a while since I’ve started using Amazon Web Services (AWS) to host my sites. I think it’s a great platform as you only pay for what you use and there are lots of options. And the best part is anything you can do via their user interface (and more) can be done programmatically via their API. I’m extremely happy using AWS but still I wanted to see what its competitors are doing.

Enter RackSpace

RackSpace

So I decided to test RackSpace first. One reason for selecting it is that it has a data centre in London (the closest AWS data centre to UK is in Dublin). Also it is based on OpenStack platform which I wanted to play with for some time. I created my free account but it needs to be activated after you receive a call from a staff member. He just asked basic questions like my username and the reason I created the account. After the call the account was activated and I was ready to explore this new land.

Servers

First Impressions

This is still a work in progress actually, I cannot say I have fully covered everything about it. Here are just my first impressions and comparisons with AWS:

Pricing & Billing

Maybe I’m cheap but my first order of business was compare the prices! The cheapest Linux configuration starts from £0.030/hr. You can find the entire list here. As the site I’m planning to migrate didn’t need much resources I decided to go with the cheapest one: 1GB RAM, 1vCPU, 20GB SSD. After the migration I’m quite happy with its performance.

One interesting thing I noticed is, unlike AWS, you pay for the machine even if you stop it. Excerpt from a documentation says “Shutting down a server will NOT stop billing, since the virtual hard drives are persistent, server resources are always in use whether the servers is powered on or not.” Now that’s not cool! Actually if you are running web-based systems you never stop the machines anyway. But there are many times I preferred to keep the old machine stopped for a period until the new machine proves to functioning fully for example. It’s nice to have the chance to rollback easily if need be. Of course you can do it here too, but you just have to pay twice as much during that period.

Features

When trying to configure the machine I noticed there isn’t a feature like Security Groups of AWS. I had to update the iptables configuration on the machine. Which would make it hard to manage firewall rules in a multi-machine environment. In AWS you just add the new machine to an existing security group and forget about it because all the existing rules are applied to the new one automatically.

Programmability and API

OpenStack

Even though I haven’t developed anything for it yet, I just wanted to see what are our capabilities and how would I develop something when I needed. All I needed to do was get the NuGet package and I was ready to get the list of my machines in a a few minutes. Basically you can manage machines, images, volumes pretty much like AWS. I’ll put a pin to it for now and develop some tools for myself later.

Program

Conclusion

I think the best thing about RackSpace is that it is built on top of OpenStack. This means if you your system to another vendor your applications using the API can remain intact. Also as it is open source software you can build your own data centre if you wanted to. Of course it sounds good to geek ears but I guess in real world it doesn’t have much value as such migration of systems are quite often. Other than that I didn’t see any advantages over AWS but I’ll keep the machine running for a while and see how it goes.

Resources

Site news

I decided to switch to FeedBurner to keep better track of my RSS feed. The new address is http://feeds.feedburner.com/PlaygroundForTheMind.

Hopefully current link will be redirected automatically. (Well not exactly automatically, I installed FD FeedBurner plugin to take care of that).

If it doesn’t work it’s likely that current subscribers are not going to receive this update via RSS but I thought a notification post wouldn’t hurt anyway.

Big DataCertificationNOSQL

Online education sites have around for some time now. One of my favourites, Udacity, has recently started a new series of courses: Data Science and Big Data Track. Big Data is a fascinating subject and I’ve been wanting to learn more about it. But so far my introductions were generally short lived. This time I intend to finish all these courses and have at least a guided tutorial. Their first course in this track is Introduction to Hadoop and MapReduce.

Hadoop

Hadoop Logo

Named after the main developer’s child’s toy’s name, Hadoop is an open-source framework based on MapReduce that can run distributed data-intensive tasks. It has its own file system called Hadoop distributed file system (HDFS). It handles data redundancy by dividing the data into 64MB chunks and storing several copies of them (3 copies by default).

MapReduce

A programming model first developed at Google. It consists of 2 steps: Map and Reduce. Map function takes the input data and divides it into smaller datasets. In Reduce function takes the sub-problems as input and calculates the final output.

Udacity Course

The course they are offering is very concise and to-the-point. It doesn’t take too long to finish. It’s instructors are employees of Cloudera and they do a very good job in explaining the basic concepts in simple terms. Also, in the course they provide a download to a virtual machine fully loaded with Hadoop and tools. It also contains the example datasets and code they use throughout the course so it makes it quite easy to practice on your own.

Final Project

Final project was fun to implement. It’s based on the examples so you can develop on top of the code shown in the class. I submitted my answers to GitHub Gist. If you’re interested they’re available here. Files are named with “_xy” prefix where x is the project number (there are two parts for the final project) and y is the quesiton number.

Udacity Certification

I’m also curious about their new certification model. I haven’t enrolled to any of their paid programs. Basically the courses are still free to enroll but with paid program you have a dedicated tutor who reviews your code and gives you feedback. Also there is an exit interview and if you pass you get a verified certification. I’m not sure how that interviews is going to be conducted though. It’s not cheap ($150/month) though. You still work at your own pace but since you’re paying for it probably you’d want to finish it as soon as possible.

Resources

DevelopmentGadgetLeap Motion

TicTacToe

Like most people I got my hopes high when ordering this gizmo and again like most people I was disappointed by it. It’s not quite the mouse-replacement as I hoped it would be. Anyway, I mostly bought it to develop applications using it. It comes with an SDK and libraries for .NET so I cannot complain much about that. I wanted to develop something simple just to get the grasp of it. Recently PluralSight published a course for Leap Motion development and I thought it was a great chance to start my own little app: Tic-Tac-Toe. The course was very helpful and I’d recommend it as a starting point for Leap Motion development.

So there is still work needed on my TicTacToe but you can find below a sneak preview of the current version.

Basically it does what it’s supposed to do at the moment: draw things on screen using your finger! So I think I accomplished what I set out for. What I want to add is a custom gesture for X. Circle gesture is built-in to SDK so drawing circles is easy. But I implemented ScreenTap gesture for playing Xs which is not intuitive obviously. Also it requires precision because it’s not quite easy to target a cell while tapping. If you watched the video you may have noticed I missed the cell for Xs second move for example. So that would be the most improvement I can make apart from the basic things like player info, statistics, undo moves etc. But as they are not directly related to Leap Motion development they are not very important in this context.

Resources

GadgetGame

I used to love my Commodore 64 when I was a child. Now that we have the ability to emulate old machines and memories I decided to give it a go. Apparently creating a MAME is a popular subject. I’ve found this C64 emulator: http://www.mascal.it/rpi64_e.html.

It’s pretty straightforward. Download the rom, burn it to an SD card using a tool (I used Win32DiskImager). Then upload your ROMs to RPi and let the good times roll!

One of my favourite games was Donkey Kong so I decided to start with that.

Donkey Kong

Donkey Kong

It loaded nice and dandy but couldn’t play it with the keyboard. So either I’m going to buy an old Joystick and figure out a way to connect it to the RPi or try to dig a little deeper to find out the key mapping.

Amazon Web ServicesCloud Computing

We all know backups are good but most of the time you won’t need a backup from a year ago. Just keep enough copies to recover from a possible failure and get rid of the rest. The other day I was working on cleaning up old security camera images which become meaningless very quickly. The images are uploaded to Amazon S3. My first approach was to delete the older ones by a scheduled script but then I discovered an easier and more effective way.

Let AWS do the work!

It’s possible to loop through thousands of objects and delete them but the alternative is to set an expiration date for each object. To activate this select the folder and make sure the properties panel is visible. Expand the Lifecycle section and click Add rule. Add a number of days for the expiration. Make sure “Apply to Entire Bucket” is checked so that any newly uploaded files comply with this rule. It’s easy as that!

S3 Lifecycle

One thing to note is that this process runs once a day. So don’t expect to get your bucket cleaned up immediately. But also don’t forget to check the next to ensure everything is working as expected!

Resources

Amazon Web ServicesDevelopment

Sometimes you need a microsite with no server-side code. All you need is to display a pretty-looking entry page. In such scenarios you don’t need to use your own servers and use your precious resources on such trivial sites. The 2 ways to achieve this (that I know about) are:

  • Using Amazon Web Services S3
  • Using GitHub Pages

Both methods are very well documented. You can find the links to the official tutorials.

AWS S3

First, my favourite IT company: Amazon! S3 method requires to create a bucket with the name of your domain or subdomain. In the properties enable Static Website Hosting and point to your index document. In order for this to work, you have to use AWS Route 53 as your DNS provider. In Route 53, all you have to do is define an A record as an alias and select the S3 bucket that contains your site. If you have multiple accounts make sure that Route 53 DNS records and S3 bucket are under the same account. Otherwise you cannot point to the bucket as en endpoint.

GitHub Pages

GitHub method is also quite easy. All it takes is create a public repository, create a branch called “gh-pages” and check in your source code. To let GitHub know that you want to host a site there, you have to create a file called CNAME which includes the domain name. And in your DNS settings you have to point your site to GitHub’s IP address. The downside of this method is, obviously, your site will stop working if GitHub decides to change their IP address.

Resources

Electronics

Playing around with the development boards and gadgets I was wondering about the differences between Microcontrollers (MCU) and Microprocessors (MPU). I found a nice concise and informative article (link in the resources section). For me the highlights of the document and the comparison drawn bettern MCU and MPU are:

  • MCU uses on-chip embedded memory, this leads way to a short start-up period. Downside of embedded memory is that there is limited space. MPUs use external memory.
  • MCUs need one single power rail whereas MPUs need several (for core, DDR etc.)
  • MPUs have much more processing power than MCUs. A full operating system’s requirements are more likely to be met by MPUs.
  • For numeric-based (number-crunching scientific type) applications and heavy user interfaces MPUs are more suitable
  • MCUs consume far less power than MPUs

So in a nutshell, MCUs are generally a better choice for low-cost, low-power projects. For example, for small devices like remote controls, smart meters MCUs are preferred.MPUs perform much better in more resource-demanding systems.

Resources

Gadget

If you like playing with gadgets like me you will love this one. It’s very tiny (as the name implies) and as of version 3.0 it is Arduino-compliant. If you install Teensyduino addon on top of Arduino IDE you can compile and upload your sketches to Teensy just like Arduino. This tiny things rocks a 32-bit ARM Cortex-M4 processor.

Teensy30

Also it is quite cheap and it’s size allows it to be used in a variety of projects. Strongly recommended for gadget-lovers.

Teensyduino

Resources

Programming

I need a large dataset to practice on SQL server while preparing to exams. One idea is to generate random data but there is no fun in that. Because it is hard to create real-life scenarios but gibberish data. Then I found out that IMDB actually releases all their data and allows usage for non-commercial applications.

Data Analysis

Unfortunately, IMDB data doesn’t come in a very structured format. You have to deal with the exceptional cases and create your own data structures. I found some nice info in a Github repo called imdb-blaster (link in the resources section).

I started with the main file to parse: Movies.lst. This file contains movies, TV shows and their episodes. I want to store these in separate tables so I wanted to treat them separately. What they look like in a nutshell is as follows:

  • Movies: The Lord of the Rings: The Return of the King (2003)    2003
  • TV Shows: “Seinfeld” (1990)                    1990-1998
  • Episodes: “Seinfeld” (1990) {Male Unbonding (#1.4)}        1990

All TV Shows are enclosed with double-quotes. There are also some other optional attributes that need to be taken into account. Such as deleted items are marked with like “Seinfeld” (1990) {Good News, Bad News (#1.0)}     1989

Start parsing

I don’t like regular expressions too much as they are very hard to read. But they are very helpful and powerful in scenarios like this. The goal is validating and parsing it to groups so that it can be inserted easily to a relational database. So I developed 3 regular expression patterns for the type of data in the file. One thing I learned about regular expressions is that you can group parts by using parenthesises. This creates an unnamed group. If you want to name them you can use angle brackets and assign a name to group such as: (subexpression)

Parsing movies

As most regular expressions tend to wind up with hard-to-read complex queries this was no exception! Even if the structure is simple because of the optional parameters and exceptional cases it became very complex very soon. Here’s the sample code to run in LINQPad to parse a sample line for a movie:

string line = @"America the Punchline (2009) 		2009";
string pattern = @"(^.*)\((\d{4}|\?{4})(/I|II|III)?\)([ ]?)()(\(V\)|\(TV\))?\t*(\d{4})$";
Console.WriteLine(Regex.IsMatch(line, pattern));

MatchCollection matches = Regex.Matches(line, pattern);
foreach (Match match in matches)
{
	Console.WriteLine("Group count: {0}", match.Groups.Count);
	foreach (Group matchGroup in match.Groups)
	{
		Console.WriteLine("{0}", matchGroup.Value);
	}
}

And the output is:

True
Group count: 8
America the Punchline (2009)     2009
America the Punchline
2009

2009

The first group is the whole string. The second one is the title, the third is the year. There is an optional group because some titles come with I, II or III after the release date. I have no idea what that means but had to break it into a group to avoid an unmatch. The last two parameters are the type (V: Video, TV: Television, VG: Video game) and another date. I don’t know why the movies have two dates and they almost always match so I’ll ignore one of them. So the actual code is very simple:

if (regexMovie.IsMatch(line))
{
	Match match = regexMovie.Match(line);

	commandMovie.Parameters.Clear();
	commandMovie.Parameters.AddWithValue("@Name", match.Groups[1].Value);
	commandMovie.Parameters.AddWithValue("@ProductionYear", match.Groups[2].Value);
	commandMovie.Parameters.AddWithValue("@IsDeleted", (match.Groups[5].Value == ""));
	commandMovie.Parameters.AddWithValue("@ReleasedFor", match.Groups[6].Value);
	commandMovie.ExecuteNonQuery();
}

All the hard work is done by regular expressions already. All we have to do is to check is a line matches a movie definition and if it does just insert the data that is already broken up into groups.

Parsing the rest

I’ll publish the full source code in a Github repository and will update this post with the link. But I guess to start parsing the movies file the other two patterns suffice for now: TV Show Pattern: (^””.””) (((\d{4}|\?{4})(/I|II|III)?))\t.(\d{4}-\d{4}|\?{4})$ Episode Pattern: (^””.””) ((\d{4}|\?{4})(/I|II|III)?) {(.?)(#(\d).(\d))}\t*(\d{4}|\?{4})$

Resources

Electronics

I was trying to avoid soldering but looks like at some point it is mandatory to learn how to do it. Got myself a basic toolkit:

Soldering Tools

There are plenty of tutorials around. I enojyed Iyaz Akhtar’s KnowHow episode dedicated to soldering and EEVBlog’s tutorial. After spending some time around here’s the fruits of my efforts:

Soldered Circuit

Of course the components soldered on this board are meaningless and they don’t do anything. But it was a good practice.

I was surprised to find out that desoldering was an easy process. All you have to do is heat the iron, touch the solder and liquify it and then vacuum it using the desoldering pump.

Desoldered Resistor

Resources

ArduinoGadget

Basically the fun with playing gadgets such as Arduino, Gadgeteer and others comes from sensors. Because they are how your system interacts with the real world. And writing code that acts upon some readings from the real world is fascinating!

So let’s get started with using the motion sensor. I have a very generic sensor from Parallax and fortunate enough to find an article that explains exactly what I needed: Using motion sensor with Arduino. Check out the resources section down below to access the original article. Unlike the LCD, it’s quite simple to connect the motion sensor to Arduino. The sample code lights the onboard LED when the motion sensor detects a motion. Check out the video for a short demo. When the sensor “senses” motion, it goes red and the LED starts blinking.

I think the next step would be to connect both LCD and motion sensor to display the alarms on the display. Hope I can make that project happen without too much delay.

Resources

ArduinoGadget

I’ve been looking for a good tutorial to connect the LCD screen to Arduino. There are a lot of tutorials everywhere but Interestingly most of them just show the final version with all the cables jumbled up. After a lot of looking around I finally found a simple, step-by-step tutorial with clear illustrations: http://www.codingcolor.com/microcontrollers/connecting-a-lcd-to-arduino/

This is a great tutorial and I recommend it to anyone who wants how to connect LCD to an Arduino.

Arduino with LCD

Resources

ArduinoGadget

The most popular development boards is without a doubt Arduino. It is open-source and it’s been around since 2005. There are lots of different models and libraries. My goal for now is just get started and see it in action. To achieve this, first stop is the official site where you can download the drivers and IDE. After download simply unzip the contents to a folder.

A great place to start playing with it is Adafruit. You can find the link for Arduino tutorials in the resources section below. I simply started with Lesson 1 and changed the speed of the onboard LED.

Arduino

So it takes minutes to get started with it and comes loaded with a bunch of examples so they made it very easy for beginners.

Troubleshooting

Error: stk500_getsync() not in sync resp=0x00 This error means that Arduino couldn’t be found in the COM port. To resolve the issue:

  • Use Device Manager and make sure Arduino is connected. It should be listed under Ports node. Note the COM port number
  • Then from Tools –> Serial Port make sure the correct port is selected.

Resources

DevelopmentGame

Kodu is basically a tool for kids to develop games without writing code. You can select form a bunch of items to place in your “world” and use the visual programming language to program the actions. The best part is you don’t have to start from scratch. You can browse the existing worlds play with it and start editing it right away. For example below is a screenshot of a game developed by California Institute of Technology:

Kodu Mars Rover

You can download any game, play and modify to your liking. It’s very easy and fun way to build games. Especially if you have young children or you are one!

Resources

.NET Gadgeteer

Soon after I started playing with Gadgeteer it became a bit messy. Components were dangling by the cables and there was no way of keeping them steady. But it is about to change as I have discovered the Tamiya Universal Plate.

Although it’s main purpose is not for Gadgeteer it works great with it. I bought mine from Proto-Pic for around £10 so it’s a small price to pay to keep my design tidy.

Here’s the link for it: http://proto-pic.co.uk/tamiya-70172-universal-plate-l-210x160mm/?gclid=CLT44pym1bgCFbHKtAoddxEAjw

And here’s how I can organize my Gadgeteer designs:

Gadgeteer_Board_01

Gadgeteer_Board_02

I bought 3mm screws and nuts to pin the components but saw some plastic long pins that can be used for the same purpose and would work better as screwing all components is a bit tedious. I’ll update the post if I find those pins.

DevelopmentNetworkingSecurity

HSTS (HTTP Strict Transport Security) is a security policy where a web server instructs the client that the website they are calling can be used with HTTPS only. By using tools like Wi-Fi PineApple and sslstrip a man-in-the-middle attack can be carried out easily. The attacker can redirect the victim to HTTP version of the site they are connecting to and after the login info is submitted in clear-text and captured the victim is redirected back to HTTPS. By using HSTS the client browser always makes the calls over HTTPS hence reducing the risk of MitM attack. It’s not bullet-proof but still an improvement over not using it.

Implementation

HSTS is basically a response header. In an ASP.NET application it can be added in the web.config file as below:

  <system.webserver>
    <httpprotocol>
      <customheaders>
        <add name="Strict-Transport-Security" value="max-age=31536000" />
      </customheaders>
    </httpprotocol>
  </system.webserver>

And when you visit this site over HTTPS response looks like:

HSTS

This header tells the browser to use only HTTPS from now on for 1 year. Also, the response header can be added in the BeginRequest event handler:

protected void Application_BeginRequest(object sender, EventArgs e)
{
	protected void Application_BeginRequest(object sender, EventArgs e)
	{
		switch (Request.Url.Scheme)
		{
			case "https":
				Response.AddHeader("Strict-Transport-Security", "max-age=31536000");
				break;
			case "http":
				var path = "https://" + Request.Url.Host + Request.Url.PathAndQuery;
				Response.Status = "301 Moved Permanently";
				Response.AddHeader("Location", path);
				break;
		}
	}
}

HSTS header only works with HTTPS. So you can not observe it HTTP responses. That’s why redirecting to HTTPS is required. Because if the client uses HTTP all the time and if there is nothing forcing the user to use HTTPS, even if you have HSTS turned on, the client will never receive those responses. So at this point even if an attacker is redirecting the user to HTTP, the browser will always make the requests over HTTPS.

Shortcomings

  • Not all browsers support it. It works with Chrome and Firefox but not with IE and Safari
  • Cannot protect for the first call: If the user is making a call to a site for the first time and there is an attacker in between it wouldn’t work. At least it would start working after the attacker is finished with the attack and redirected the user back to HTTPS.

Resources

Electronics

If you’re going to do something you need the right tools. The last time I was dabbling with I didn’t quite enjoy it but that was mostly because I didn’t have the right tools. The most daunting part for me was cabling. With the jumper cable set I had it was just too painful.

Get your cables right

This time I’m prepared for all occasions: I have Male – Male, Male – Female and Female – Females cables which cover all combinations.

Cables

Essentials

Multimeter: I realized a multimeter is an essential tool if you want to build your own circuits. I added a reference for an excellent tutorial which teaches you how to use one in a few minutes. By using it, I was able to measure voltage and resistor values in no time.

LCD Display: Also, I think it’s almost impossible to build a cool project without a display. It can be used for debugging and informing the user so very helpful to have it in the mix. They are so cheap that I ordered 5 of them for about £7 on eBay. So I guess there is no excuse for not buying one!

Resistors: Real resistors! I hated my old blue resistors with 5 bands. I couldn’t read the values. Colors were not clear. But now finally I got a bunch of standard 3-band yellow ones which make it easier to read. (Actually I have a multimeter now, so I can just use it to measure the resistance anyway.)

Breadboards: I had a big nice one but I think in order to run things in parallel I’ll need a few more of the smaller ones. Also I’m trying to learn how to solder so I now have a couple of solder boards to practice. Push buttons: I think to control the flow they are definitely needed.

New Components

Let’s get crackin’

I just built a simple circuit to light a LED when the push button is pressed. I know it’s not impressive by any means but I enjoyed building something from scratch. Familiarized myself with my tool shed and looking forward to build more complicated things.

Resources

Gadget

Yes!!! Finally it arrived. It looks very tiny and elegant. This is what the package and contents look like:

Leap Motion

Installation

Good thing it comes with 2 USB cables of different lengths. One is bout half a meter, probably for laptop users and the other is about 2 meters, better suited for desktop users like myself. It doesn’t come with any instructions or user guide but a card that tells you to go to leapmotion.com/setup. And that’s where I went.

The setup file is about 56MB. As this is a new file, Norton 360 didn’t like it and issued a warning. Another interesting note is right after I plugged it in Skype icon started blinking. I checked the window and it was saying “Your video works fine”.  as I don’t normally keep my webcam plugged in, the video doesn’t work. But it became happy to detect a camera! I’m not sure if it would work with Skype though. Probably not.

App Store

After the installation you have to login to Airspace which is the app store for Leap Motion. I logged in with my LeapMotion account. It comes with a few free apps. Cut the Rope didn’t run unfortunately. Didn’t bother to give an error message either. So I tried the others. Lotus looks like a weird psychedelic app. As far as I could understand you make music with your fingers. Here’s a little demo of how it looks:

Development

What’s more interesting than downloading and using apps is developing one on my own! In order to do this you have to sign up for a developer account at: http://developer.leapmotion.com After the sign up you download their SDK. So far I managed to install the drivers, tried a few apps and downloaded the SDK and I’ll post the development experience in another post.

Final words

I’m not so sure if this gizmo will take off but as long as it’s an interface for the OS, not some custom apps, I don’t see much use for it. Keeping hands in the air for long tasks is tiring. I think it only might make some games more fun but that’s a very small market.

HardwareMobile

If you have an Android phone that supports NFC, you can save and change some settings very easily based on your location. All you need is some blank NFC tags like shown below, which are very cheap and a free app called NFC Task Launcher.

NFC Tags

NFC for the uninitiated

NFC stands for Near Field Communication. It’s a set of standards build on RFID that allows wireless communication between devices in a close proximity. It is commonly used in contactless payment systems.

Manage tasks with the task launcher

It’s a very intuitive and easy to use app. First, you select an action group (like WiFi On/Off, BlueTooth On/Off).

NFC Task Launcher

Then you configure the action (like Enable/Disable) and finally you touch to your tag to associate the task with it.

Secure your phone outside the house

After playing around with tools like WiFi PineApple I’m now even more afraid of wireless networks than ever. Thinking about it, I don’t connect to open networks on purpose and if my phone connects to one of them that means something fishy is going on. So why should I have WiFi when I’m not using my own? But of course turning it on and off every time I enter/leave the house is cumbersome. My solution is: doing it with NFC. Although it’s not completely automated, I just touch my phone to the red tag I stuck to my door when I leave the house and the WiFi is turned off. When I enter the house, I touch on the green one it is turned back on. Simple as that!

NFC Task Launcher

Resources