Cloud Computing, Development, System Administration comments edit

DevOps (Development + Operations) is one of most popular terms in the IT world recently. From what I’ve read and listened to so far, my understanding is it is all about continuous deployment (or delivery). Basically, you have to automate everything from development to deployment to practice DevOps.

Current problem

Traditionally, successful deployment is a huge challenge. It is mostly a manual and cumbersome process. Because of its sensitive nature the system admins are not huge fans of deployments. Also, another challenge is the miscommunication (or no communication in some cases) between system admin and development teams. They are generally run by different high-level executives and their priorities conflict most of the time.

Solution

On the philosophical side, DevOps is bringing these teams together and work in harmony. Having social events with both teams’ attendance is a key to build confidence among team members. As Richard Campbell (from RunAsRadio and .NET Rocks podcasts) says “Pizza and beer is a global lubricant”.

Dev…

On the development side, the key requirement is continuous integration. You have to able to run unit tests and acceptance tests automatically on build servers. This means development has to be done in short sprints in an agile way with frequent check-ins. One step further of this stage is continuous deployment.

####…Ops This is where the IT team comes into play. When the whole system is automated, deploying to production frequently and without much headache becomes possible. Cloud computing is one of the core technologies that makes DevOps possible. Ability to manage virtual machines programmatically (i.e. AWS, OpenStack) leads to a whole bunch of possibilities.

This is a fairly complex topic encompassing many disciplines and technologies. Also it’s quite dynamic and open to innovation. Definitely worth keeping an eye on.

Resources

Productivity comments edit

I’ve been trying to improve my time management techniques perpetually. This post is a small compilation of notes, tips and tools I came up with. This is an on-going process and I will keep updating these. I strongly recommend reading Getting Results the Agile Way and visiting the sites in the resources section. I learned most of the stuff below from that book. And there is plenty of more good advice in there. So here is my list:

01. Be realistic!

I think this is the most important thing I have learned so far. Yes, we all want to do more tasks and write more code, develop the killer apps we were thinking of but the reality is we have a limited time we can spare. This is not about aiming low but just don’t start everything at the same time because you are too excited about them at the time. It’s very likely that you will end up with a bunch of unfinished projects. Over time they will keep haunting you. First you will never feel the sense of accomplishment even though you spent a ton of time and effort. Also they will remain somewhere in your to-do list only making everything worse.

02. Rule of 3 rocks!

I think if there is one thing to take away from the book Getting Results the Agile Way, it is the Rule of 3. It is a very simple and effective idea. Don’t put too many things on your plate. Every month pick 3 major tasks to accomplish. Then divide them into smaller weekly tasks and even smaller daily tasks. If you only aim to complete 3 tasks every day you can steadily move toward accomplishing the greater goals at the end of the month. For me, it’s not always to plan things so clearly. Generally monthly and weekly tasks are left unfulfilled. But it is easy and very achievable to plan the next day every night and pick only 3 things. At least that one works pretty well.

03. Determine your categories (hot spots)

I have to admit I tweaked rule of 3 a little bit! I wanted to have a more balanced plan so I determined main categories in my life like health, work, career, chores etc. I’m picking 1 task per category each day. This way I believe I can make progress in every aspect of life.

04. Keep it simple

Simplify your process as much as possible. When I first started keeping lists of outcomes I created templates so detailed that it took half an hour to fill out completely. Instead of overly complicating things and spending too much time on managing just try to get rid of as much as possible. In the end the ultimate goal is to use time for things that matter. Even a blank note with 3 items written on it can be a very good way to plan the day. Just do the 3 things on the list, don’t think about anything else.

05. Keep your backlog clean and short

Learn how to let things go. This is another crucial thing. Let’s be honest we will never be realistic and we will put a ton of items to our to-do list turning it into a bottomless pit. There were times I found items in my list that didn’t even mean anything anymore. Or so outdated that it had no value anymore. The result is you keep those in your list all that time, they keep taking your time to manage also they weigh on your shoulders as they pile up. And you end up not doing them anyway. Essentially you stall and stall then delete them once they make no sense anymore. The golden rule to keep in mind is that if it’s important it will surface again! So don’t worry about forgetting a very important task. You will be reminded it in one way or another.

06. Pen & paper with colour coding helps

Keeping notes on Evernote is great but sometimes the ease of use makes it also easier to neglect or forget tasks. After all they become invisible once you minimize the window. I recently started using coloured post-its.They are coded in their order of importance. So for example I cannot do green tasks before the purple ones are completed. Also it is annoying to have a bunch of notes sticking around so trying to get rid of them is an extra motivation!

07. Use Pomodoro on small tasks

I love this technique since the day I heard about it. It’s quite simple: Just divide tasks small unbreakable units of time. The default is 25 minute work and 5 minute rest. I use Focus Booster on my desktop computer

Focus Booster

On iPad I use an app which doesn’t have a specific name and I cannot find it in the app store anymore. It looks like this:

iPad Pomodoro App

It works fine and looks stylish so I’m not planning to find another one. There are a ton of similar apps in the app store though.

I cannot use this method for large tasks. Also I cannot work non-stop with 25-5 periods. I get distracted by something else eventually. So I just use this for short 2-4 pomodoro-long tasks. For example spending 2 pomodoros a day for reading is a great way to spare time to read.

Resources

Encryption, Security comments edit

I used to wonder what different key sizes meant when dealing with SSL. Also, I noticed that SSL certificate I had purchased said “128/256 bit encryption” in its feature list which only made me more confused. What does it actually mean and why should it use 128-bit if it supports 256 anyway? I checked the website that’s running on a Linux machine and saw that it used 256-bit encryption whereas another website of mine was running with 128-bit encryption. And I bought both certificates from the same vendor so it has to do something with the server.

What’s with the naming?

For the uninitiated, TLS is the new name for the protocol. SSL name was discontinued after version 3 and after that TLS 1.0 was released. As of this writing the latest version is TLS 1.2 which was released in 2008. So technically the name of the protocol is Transport Layer Security (TLS) but many people, including me, still refer to it as SSL.

Key Sizes

SSL Key Sizes

Basically the key size (2048 bit in the image) is the public/private key pair size. This size is determined when CSR is created for the certificate. This is what determines how vulnerable the key is to brute-force attacks. Currently 2048-bit is considered to be very strong.

128/256-bit is the length of the session key. A session key is generated during the handshake. A random data (of length 128 or 256 bit) is generated by the client and encrypted using the server’s public key. The server decrypts the message with its private key. Afterwards, server and client use this session key and use symmetric encryption. RSA keys are just used in the beginning of the communication.

Let’s see it in action

I might have had a better understanding after the research but I still I had to resolve my issue. I needed to see 256-bit encryption. Since this is a rather sensitive operation I wanted to test it on a completely expandable machine. So I created two new small instances running Windows 2008 and Windows 2012. I quickly installed the IIS to both instances and checked what they looked like. As I suspected they were using 128-bit out of the box.

SSL_Key_Sizes_Win2008_Before

SSL_Key_Sizes_Win2012_Before

The problem is AES-256 option is not high in the list in the cipher suite that the server supports. This requires some registry update and group policy changes. Normally all these have to be done manually. You can find a resource below that explains how to do it (I haven’t tested it myself). Instead, I decided to use a tool which makes the whole process a lot easier and less error-prone. It’s called IISCrypto.

IIS Crypto

I just downloaded the tool and ran the best practices option. Restarted the server and here are the results:

SSL_Key_Sizes_Win2008_After

SSL_Key_Sizes_Win2012_After

Windows 2012 version prioritize TLS 1.2 over TLS 1.0 so it uses the newer version of the protocol even the browser I used was the same for both tests.

Resources

Development, NOSQL, Programming comments edit

I updated my toy project. You can find the source code and live demo for the final version below:

Source Code: https://github.com/volkanx/BeerExplorer

Live demo URL: http://beerexplorer.me

If you don’t want to bother deploying it without first seeing what it looks like, here’s a screenshot:

Beer Explorer

It’s just a simple exercise to browse Couchbase repositories. It was helpful for me and I hope you find it helpful too.

Cloud Computing comments edit

**It’s been a while since I’ve started using Amazon Web Services (AWS) to host my sites. I think it’s a great platform as you only pay for what you use and there are lots of options. And the best part is anything you can do via their user interface (and more) can be done programmatically via their API. I’m extremely happy using AWS but still I wanted to see what its competitors are doing.

Enter RackSpace

RackSpace

So I decided to test RackSpace first. One reason for selecting it is that it has a data centre in London (the closest AWS data centre to UK is in Dublin). Also it is based on OpenStack platform which I wanted to play with for some time. I created my free account but it needs to be activated after you receive a call from a staff member. He just asked basic questions like my username and the reason I created the account. After the call the account was activated and I was ready to explore this new land.

Servers

First Impressions

This is still a work in progress actually, I cannot say I have fully covered everything about it. Here are just my first impressions and comparisons with AWS:

Pricing & Billing

Maybe I’m cheap but my first order of business was compare the prices! The cheapest Linux configuration starts from £0.030/hr. You can find the entire list here. As the site I’m planning to migrate didn’t need much resources I decided to go with the cheapest one: 1GB RAM, 1vCPU, 20GB SSD. After the migration I’m quite happy with its performance.

One interesting thing I noticed is, unlike AWS, you pay for the machine even if you stop it. Excerpt from a documentation says “Shutting down a server will NOT stop billing, since the virtual hard drives are persistent, server resources are always in use whether the servers is powered on or not.” Now that’s not cool! Actually if you are running web-based systems you never stop the machines anyway. But there are many times I preferred to keep the old machine stopped for a period until the new machine proves to functioning fully for example. It’s nice to have the chance to rollback easily if need be. Of course you can do it here too, but you just have to pay twice as much during that period.

Features

When trying to configure the machine I noticed there isn’t a feature like Security Groups of AWS. I had to update the iptables configuration on the machine. Which would make it hard to manage firewall rules in a multi-machine environment. In AWS you just add the new machine to an existing security group and forget about it because all the existing rules are applied to the new one automatically.

Programmability and API

OpenStack

Even though I haven’t developed anything for it yet, I just wanted to see what are our capabilities and how would I develop something when I needed. All I needed to do was get the NuGet package and I was ready to get the list of my machines in a a few minutes. Basically you can manage machines, images, volumes pretty much like AWS. I’ll put a pin to it for now and develop some tools for myself later.

Program

Conclusion

I think the best thing about RackSpace is that it is built on top of OpenStack. This means if you your system to another vendor your applications using the API can remain intact. Also as it is open source software you can build your own data centre if you wanted to. Of course it sounds good to geek ears but I guess in real world it doesn’t have much value as such migration of systems are quite often. Other than that I didn’t see any advantages over AWS but I’ll keep the machine running for a while and see how it goes.

Resources

Site news comments edit

I decided to switch to FeedBurner to keep better track of my RSS feed. The new address is http://feeds.feedburner.com/PlaygroundForTheMind.

Hopefully current link will be redirected automatically. (Well not exactly automatically, I installed FD FeedBurner plugin to take care of that).

If it doesn’t work it’s likely that current subscribers are not going to receive this update via RSS but I thought a notification post wouldn’t hurt anyway.

Big Data, Certification, NOSQL comments edit

Online education sites have around for some time now. One of my favourites, Udacity, has recently started a new series of courses: Data Science and Big Data Track. Big Data is a fascinating subject and I’ve been wanting to learn more about it. But so far my introductions were generally short lived. This time I intend to finish all these courses and have at least a guided tutorial. Their first course in this track is Introduction to Hadoop and MapReduce.

Hadoop

Hadoop Logo

Named after the main developer’s child’s toy’s name, Hadoop is an open-source framework based on MapReduce that can run distributed data-intensive tasks. It has its own file system called Hadoop distributed file system (HDFS). It handles data redundancy by dividing the data into 64MB chunks and storing several copies of them (3 copies by default).

MapReduce

A programming model first developed at Google. It consists of 2 steps: Map and Reduce. Map function takes the input data and divides it into smaller datasets. In Reduce function takes the sub-problems as input and calculates the final output.

Udacity Course

The course they are offering is very concise and to-the-point. It doesn’t take too long to finish. It’s instructors are employees of Cloudera and they do a very good job in explaining the basic concepts in simple terms. Also, in the course they provide a download to a virtual machine fully loaded with Hadoop and tools. It also contains the example datasets and code they use throughout the course so it makes it quite easy to practice on your own.

Final Project

Final project was fun to implement. It’s based on the examples so you can develop on top of the code shown in the class. I submitted my answers to GitHub Gist. If you’re interested they’re available here. Files are named with “_xy” prefix where x is the project number (there are two parts for the final project) and y is the quesiton number.

Udacity Certification

I’m also curious about their new certification model. I haven’t enrolled to any of their paid programs. Basically the courses are still free to enroll but with paid program you have a dedicated tutor who reviews your code and gives you feedback. Also there is an exit interview and if you pass you get a verified certification. I’m not sure how that interviews is going to be conducted though. It’s not cheap ($150/month) though. You still work at your own pace but since you’re paying for it probably you’d want to finish it as soon as possible.

Resources

Book Review, Review comments edit

I’m not an early Twitter adopter but I love using it since day one. It is an fast and easy way to skim through the news of the day, get a few random tips and be notified by new articles or blog posts. I still use RSS feeds as they are not ephemeral like Twitter feed therefore more reliable to get the latest news but Twitter is a great companion to that source now.

Hatching Twitter

I heard about this book in a podcast a few days ago and Twitter being my favourite “social network” I immediately dug in. One funny thing is it makes you think about how fast things go in the Internet age. The book reads like a history of a company which, in earth years, was founded 7 years ago. Obviously we know the plot and ending of the book so there is nothing exciting about it maybe. But the storytelling is very good and riveting. My intention was to spare 2 pomodoros a day but most days I found myself extending it to 3 or 4. Long story short, it’s a very well written book about one of most fascinating tech companies of the day.

Resources

Development, Gadget, Leap Motion comments edit

TicTacToe

Like most people I got my hopes high when ordering this gizmo and again like most people I was disappointed by it. It’s not quite the mouse-replacement as I hoped it would be. Anyway, I mostly bought it to develop applications using it. It comes with an SDK and libraries for .NET so I cannot complain much about that. I wanted to develop something simple just to get the grasp of it. Recently PluralSight published a course for Leap Motion development and I thought it was a great chance to start my own little app: Tic-Tac-Toe. The course was very helpful and I’d recommend it as a starting point for Leap Motion development.

So there is still work needed on my TicTacToe but you can find below a sneak preview of the current version.

Basically it does what it’s supposed to do at the moment: draw things on screen using your finger! So I think I accomplished what I set out for. What I want to add is a custom gesture for X. Circle gesture is built-in to SDK so drawing circles is easy. But I implemented ScreenTap gesture for playing Xs which is not intuitive obviously. Also it requires precision because it’s not quite easy to target a cell while tapping. If you watched the video you may have noticed I missed the cell for Xs second move for example. So that would be the most improvement I can make apart from the basic things like player info, statistics, undo moves etc. But as they are not directly related to Leap Motion development they are not very important in this context.

Resources

Book Review, Review comments edit

The Power of Habit

I titled this post a book review but in reality I only read the 1/3 of the book! Because it was the part pertinent to my needs. Part 1 is about the habits of individuals, part 2 is about the habits of organizations and part 3 about the habits of societies. The reason I started reading this book was acquiring a few tips and tricks on managing habits and it was very helpful to analyse the nature of habits and altering them if needed. The book covers thoroughly the nature of habits. Basically it’s formulated as cue – routine and reward. So I realized if I want to develop a new habit I have to stick to this pattern. Like to have exercise daily the best approach is to do it always at the same time if possible. Right after waking up for example. And after it I have to reward myself. A little snack maybe. Small enough, of course, not to nullify all that exercise. Anyway, the book has lots of examples of success stories and scientific studies and I think it’s quite helpful to help ourselves automate mundane but necessary stuff and getting rid of some unwanted habits. I have no interest in habits of organizations or societies but judging by the first part I’m sure it must be attractive to some. Even just for Part 1 I’d recommend this book.

Resources

Gadget, Game comments edit

I used to love my Commodore 64 when I was a child. Now that we have the ability to emulate old machines and memories I decided to give it a go. Apparently creating a MAME is a popular subject. I’ve found this C64 emulator: http://www.mascal.it/rpi64_e.html.

It’s pretty straightforward. Download the rom, burn it to an SD card using a tool (I used Win32DiskImager). Then upload your ROMs to RPi and let the good times roll!

One of my favourite games was Donkey Kong so I decided to start with that.

Donkey Kong

Donkey Kong

It loaded nice and dandy but couldn’t play it with the keyboard. So either I’m going to buy an old Joystick and figure out a way to connect it to the RPi or try to dig a little deeper to find out the key mapping.

Amazon Web Services, Cloud Computing comments edit

We all know backups are good but most of the time you won’t need a backup from a year ago. Just keep enough copies to recover from a possible failure and get rid of the rest. The other day I was working on cleaning up old security camera images which become meaningless very quickly. The images are uploaded to Amazon S3. My first approach was to delete the older ones by a scheduled script but then I discovered an easier and more effective way.

Let AWS do the work!

It’s possible to loop through thousands of objects and delete them but the alternative is to set an expiration date for each object. To activate this select the folder and make sure the properties panel is visible. Expand the Lifecycle section and click Add rule. Add a number of days for the expiration. Make sure “Apply to Entire Bucket” is checked so that any newly uploaded files comply with this rule. It’s easy as that!

S3 Lifecycle

One thing to note is that this process runs once a day. So don’t expect to get your bucket cleaned up immediately. But also don’t forget to check the next to ensure everything is working as expected!

Resources

Amazon Web Services, Development comments edit

Sometimes you need a microsite with no server-side code. All you need is to display a pretty-looking entry page. In such scenarios you don’t need to use your own servers and use your precious resources on such trivial sites. The 2 ways to achieve this (that I know about) are:

  • Using Amazon Web Services S3
  • Using GitHub Pages

Both methods are very well documented. You can find the links to the official tutorials.

AWS S3

First, my favourite IT company: Amazon! S3 method requires to create a bucket with the name of your domain or subdomain. In the properties enable Static Website Hosting and point to your index document. In order for this to work, you have to use AWS Route 53 as your DNS provider. In Route 53, all you have to do is define an A record as an alias and select the S3 bucket that contains your site. If you have multiple accounts make sure that Route 53 DNS records and S3 bucket are under the same account. Otherwise you cannot point to the bucket as en endpoint.

GitHub Pages

GitHub method is also quite easy. All it takes is create a public repository, create a branch called “gh-pages” and check in your source code. To let GitHub know that you want to host a site there, you have to create a file called CNAME which includes the domain name. And in your DNS settings you have to point your site to GitHub’s IP address. The downside of this method is, obviously, your site will stop working if GitHub decides to change their IP address.

Resources

Electronics comments edit

Playing around with the development boards and gadgets I was wondering about the differences between Microcontrollers (MCU) and Microprocessors (MPU). I found a nice concise and informative article (link in the resources section). For me the highlights of the document and the comparison drawn bettern MCU and MPU are:

  • MCU uses on-chip embedded memory, this leads way to a short start-up period. Downside of embedded memory is that there is limited space. MPUs use external memory.
  • MCUs need one single power rail whereas MPUs need several (for core, DDR etc.)
  • MPUs have much more processing power than MCUs. A full operating system’s requirements are more likely to be met by MPUs.
  • For numeric-based (number-crunching scientific type) applications and heavy user interfaces MPUs are more suitable
  • MCUs consume far less power than MPUs

So in a nutshell, MCUs are generally a better choice for low-cost, low-power projects. For example, for small devices like remote controls, smart meters MCUs are preferred.MPUs perform much better in more resource-demanding systems.

Resources

Gadget comments edit

If you like playing with gadgets like me you will love this one. It’s very tiny (as the name implies) and as of version 3.0 it is Arduino-compliant. If you install Teensyduino addon on top of Arduino IDE you can compile and upload your sketches to Teensy just like Arduino. This tiny things rocks a 32-bit ARM Cortex-M4 processor.

Teensy30

Also it is quite cheap and it’s size allows it to be used in a variety of projects. Strongly recommended for gadget-lovers.

Teensyduino

Resources

Heavy Metal comments edit

After attending to both festivals this year I decided to make a list of what I liked and disliked about both of them.

Bloodstock

Bloodstock Logo

Pros

  • More friendly: Bloodstock being a small festival I find it cosier and people seem to be more friendly.
  • Cheaper: The tickets are almost 2/3 the price. Also the beer in the arena is cheaper. And best of all you have the chance to buy crates of beer inside the festival area (24 for £40 and 12 for £20).
  • Closer to London: Considering the time spent on the road is just wasted it is good that it takes less.

Cons

  • Coach on Thursday: Not a flaw of the festival but since that’s my means on transportation, I have to chalk it up on festivals cons.

Download

Download Logo

Pros

  • More bands: There are more top level bands. Of course the downside of it is that you have to choose because of the time conflicts.
  • Internet connection: A huge plus. Not very fast but still good enough to read Tweets and some newspapers

Cons

  • Too big to walk around: It took me 30 minutes to walk from the camping site to the arena. Although it’s a good exercise, I hated walking all day long on my vacation!
  • Too crowded: Even in the afternoon, there were thousand and thousands of people in front of the main stages. I watched most shows from the giant screen.
  • Logistics: There are so many people that there were queues everywhere. Want to buy a beer? Wait in line. Wanna go to toilet? Wait in line. Wanna buy food? Wait in line! If there is one thing that I hate more than walking all day on my vacation is that waiting in lines all day on my vacation.

Verdict

To me Bloodstock is definitely the better choice and if I’m around this time next year I’ll most certainly be attending to it. I’m glad I had the chance to see Download but probably that was my first and last time.

Heavy Metal comments edit

2nd Bloodstock in a row is over. I liked the line-up mostly because it had 2 big metal names: Lamb of God and Slayer. I missed some of the bands I wanted to see but overall it was great to attend this year too.

Notes from the festival

  • For the first time in a festival I met with my neighbours! Both next door tents had two friendly guys. Even though, while leaving we just ignored each other (as you might expect) it was nice.
  • I loved my tent. It takes seconds to setup. The only issue was it had black ropes which made it hard for people to see and people kept tripping to them causing me lose 4-5 metal pins.
  • There were some issues with the sound on Friday which caused some explosion-like sounds during the shows.
  • The layout of arena and festival area was the same as last year, including the foot stands which made it easy to find what I was looking for.
  • There was a tent from Wacken Foundation! I don’t know what they are doing but they had a stationary bike for people to ride. Some charity work I guess but I couldn’t find it in myself to go and ask!
  • Best food of the festival: Delikate! Next year I’m planning to eat only from them no matter how long the queue they have
  • Best new bands (for me ) of the festival: Ex-Deo and Hell. Especially Hell put up an excellent show. Hope I can make it to their show in London this November
  • Worst band: Voivod! Extremely boring and completely meaningless
  • Band with worst frontman: Amorphis. The last scrap of my love against Amorphis was destroyed after this show.
  • It felt like the number of Galaxy S phones surpassed iPhones
  • Avantasia’s female vocals looked familiar. Then I realized I saw her in Epica DVD long time ago. She is still pretty!
  • Beholder singer announced next year’s headliner (Emperor)  and he also appeared in the after party with the 4 DJs of the apocalypse.
  • To me the only conflicts were Wolfsbane vs. Anthrax and Accept vs. Scar Symmetry
  • The barrier broke during Lamb of God set. One funny remark from the show was Randy saying he calls himself Roger Brilliant in England.
  • Cursing is still not allowed on the set
  • Everybody who saw Evil Scarecrow said they were great. I wish I didn’t miss that one.
  • During Exodus show the singer started a Wall of Death and how he did it was interesting. He looks at one side and said I want this side to kill that side and looks at the other side and says the same thing to them. Sounded more violent than we generally hear on the shows.
  • Scar Symmetry guitarists looked like twins but apparently they are not related.
  • Looks like my new favourite band Ex-deo’s vocals and Kataklysm vocals are the same guy. The guy knows how to make good music.
  • The vamp girls from the after aprty and Fozzy looked like the same but couldn’t verify it yet! (Check out the videos below to compare them yourself)
  • I wonder who the old presenter who was preaching for worshipping Satan was!

Anthrax Beholder Amorphis Avantasia Accept Dark Funeral Kataklysm Fozzy Gama Bomb Hell Ex Deo King Diamond Lamb of God Ex Deo White Chapel Sabaton Scar Symmetry Municipal Waste




Heavy Metal comments edit

Wow, it’s been a year already? It’s time for another Bloodstock mayhem! This year it’s even stronger and faster: Lamb of God and Slayer are two of the headliners. ‘Nuff said!

Bloodstock 2013

I checked my notes from last year while preparing for the next one. I wrote to buy the same tent but I think I have now a better solution:

Tent

which can be bought from here: http://www.amazon.co.uk/gp/product/B00BW3XB0K

The best thing about it is it can be setup in seconds. Just open the zipper and it pops up instantly and you’re done. I had a good deal and got it for £20. At the time of this writing it is £35 on Amazon so I guess I was just lucky to seal the deal back then.

Apart from that, I’ll stick to the plan. I’m packing both suntan cream and poncho as this British weather is completely unpredictable.

Hope everything goes well and I can have a good time at BOA 2013 just like I did last year. I’ll post my reviews after the festival.

Programming comments edit

I need a large dataset to practice on SQL server while preparing to exams. One idea is to generate random data but there is no fun in that. Because it is hard to create real-life scenarios but gibberish data. Then I found out that IMDB actually releases all their data and allows usage for non-commercial applications.

Data Analysis

Unfortunately, IMDB data doesn’t come in a very structured format. You have to deal with the exceptional cases and create your own data structures. I found some nice info in a Github repo called imdb-blaster (link in the resources section).

I started with the main file to parse: Movies.lst. This file contains movies, TV shows and their episodes. I want to store these in separate tables so I wanted to treat them separately. What they look like in a nutshell is as follows:

  • Movies: The Lord of the Rings: The Return of the King (2003)    2003
  • TV Shows: “Seinfeld” (1990)                    1990-1998
  • Episodes: “Seinfeld” (1990) {Male Unbonding (#1.4)}        1990

All TV Shows are enclosed with double-quotes. There are also some other optional attributes that need to be taken into account. Such as deleted items are marked with like “Seinfeld” (1990) {Good News, Bad News (#1.0)}     1989

Start parsing

I don’t like regular expressions too much as they are very hard to read. But they are very helpful and powerful in scenarios like this. The goal is validating and parsing it to groups so that it can be inserted easily to a relational database. So I developed 3 regular expression patterns for the type of data in the file. One thing I learned about regular expressions is that you can group parts by using parenthesises. This creates an unnamed group. If you want to name them you can use angle brackets and assign a name to group such as: (subexpression)

Parsing movies

As most regular expressions tend to wind up with hard-to-read complex queries this was no exception! Even if the structure is simple because of the optional parameters and exceptional cases it became very complex very soon. Here’s the sample code to run in LINQPad to parse a sample line for a movie:

string line = @"America the Punchline (2009) 		2009";
string pattern = @"(^.*)\((\d{4}|\?{4})(/I|II|III)?\)([ ]?)()(\(V\)|\(TV\))?\t*(\d{4})$";
Console.WriteLine(Regex.IsMatch(line, pattern));

MatchCollection matches = Regex.Matches(line, pattern);
foreach (Match match in matches)
{
	Console.WriteLine("Group count: {0}", match.Groups.Count);
	foreach (Group matchGroup in match.Groups)
	{
		Console.WriteLine("{0}", matchGroup.Value);
	}
}

And the output is:

True
Group count: 8
America the Punchline (2009)     2009
America the Punchline
2009

2009

The first group is the whole string. The second one is the title, the third is the year. There is an optional group because some titles come with I, II or III after the release date. I have no idea what that means but had to break it into a group to avoid an unmatch. The last two parameters are the type (V: Video, TV: Television, VG: Video game) and another date. I don’t know why the movies have two dates and they almost always match so I’ll ignore one of them. So the actual code is very simple:

if (regexMovie.IsMatch(line))
{
	Match match = regexMovie.Match(line);

	commandMovie.Parameters.Clear();
	commandMovie.Parameters.AddWithValue("@Name", match.Groups[1].Value);
	commandMovie.Parameters.AddWithValue("@ProductionYear", match.Groups[2].Value);
	commandMovie.Parameters.AddWithValue("@IsDeleted", (match.Groups[5].Value == ""));
	commandMovie.Parameters.AddWithValue("@ReleasedFor", match.Groups[6].Value);
	commandMovie.ExecuteNonQuery();
}

All the hard work is done by regular expressions already. All we have to do is to check is a line matches a movie definition and if it does just insert the data that is already broken up into groups.

Parsing the rest

I’ll publish the full source code in a Github repository and will update this post with the link. But I guess to start parsing the movies file the other two patterns suffice for now: TV Show Pattern: (^””.””) (((\d{4}|\?{4})(/I|II|III)?))\t.(\d{4}-\d{4}|\?{4})$ Episode Pattern: (^””.””) ((\d{4}|\?{4})(/I|II|III)?) {(.?)(#(\d).(\d))}\t*(\d{4}|\?{4})$

Resources

Game comments edit

I’ve been playing this game for months now and I believe it’s time to move on. I think it is safe to say that I finished the game already. I bought all the buildings that don’t require donuts. I haven’t spent one penny playing this game so this is as far as a freeloader like me can get! No, just kidding. The prices they set are preposterous and I would never spend that much money on some imaginary donuts. Actually Leo Laporte said in one of his shows that he spend over $300 for this game! That’s how much you can spend if you want to buy all premium items.

TSTO

I’m happy with the Springfield I built. As you can see in the images below, I unlocked all the lands, bought all the buildings money can buy including the million dollar buildings. The problem with the game is there is no challenge. You just tap on buildings when they have signs on top of them and tap on characters when there is a exclamation mark on top. If this wasn’t about Simpsons I wouldn’t be playing it at all

TSTO

The beginning of the end: Krustyland!

The worst part is they started to repeat themselves. After the last update they introduced a new place called Krustyland. It is exactly like the first level of the game. A blank field and you clean up the place and build some buildings. Only this time you earn Krustyland tickets instead of money. It’s like the mini version of the same game inside it.

TSTO

So.. I’m calling it off. I don’t mind wasting a little time on games to blow off steam but not with this one anymore. Maybe I can move on to something more challenging like MindCraft.