dev fsharp

I’m not a big fan of New Year’s resolutions. I was meaning to start learning F# and since it’s the new year’s 2nd day it might a good time to finally give it a shot!

Where to start

It’s always hard to find the best resource when you are starting. Some time ago I heard about a Microsoft Research project called TryFSharp.org. It’s a tutorial website geared towards the absolute beginners. It comes with a REPL editor so no extra tools are needed to start.

From now on I’m planning to spend 2 pomodoros (around 1 hour) every day to learn F#. After my first 2 pomodoros I completed the first 3 sections and below are my notes for today’s training.

Lecture Notes

  • let keyword to bind names to values. These bindings are immutable. If you try to assign a value to a same name twice you get the following error:
let duplicated = "original value"
let duplicated = "new value"

causes the following error:

stdin(8,5): error FS0037: Duplicate definition of value 'duplicated'
  • Mutable variables can be created by explicitly specifiying mutable keyword but it should be used cautiously.
  • F# is a statically typed language like C#
  • printfn can be used to display messages. Strings can be formatted by using special characters such %d for int, %s for string such as
printfn "The answer is %d" 42
  • let can also be used bind a name to a function. The following code
let square x =
    x * x

square 4

produces this result in the output window:

> let square x =
      x * x
  
  square 4

val square : x:int -> int
val it : int = 16

> 
  • F# is whitespace-sensitive. In the function above the body of the function was denoted by indenting it 4 spaces and return values is the last line of the function.
  • In times when F# cannot determine the type on itw own, it can specified explicitly bu using type annotations. For example:
let toLeetSpeak (phrase:string) =
    phrase.Replace('t', '7').Replace('o', '0')

toLeetSpeak "root"

In the example above it needs to be specified that phrase if of type string before String.Replace method can be called.

  • Functions can be defined inside other functions:
let quadruple x =    
    let double x =
        x * 2

    double(double(x))
  • A function can be used as an argument to another function to create what’s called a higher order function.
  • Inline functions can be created such as
let square = (fun x -> x * x)

Theres are called lambda functions or lambdas.

  • Lists can be created by semi-colon separated single values or a range values with .. in between such as
let evens = [2; 4; 6; 8]
let firstHundred = [0..100]
  • Higher-order functions can be combined with other functions such as
let firstHundred = [0..100]
List.map (fun x -> x * 2) 
    (List.filter (fun x -> x % 2 = 0) firstHundred)

which produces the following output

val it : int list =
  [0; 4; 8; 12; 16; 20; 24; 28; 32; 36; 40; 44; 48; 52; 56; 60; 64; 68; 72; 76;
   80; 84; 88; 92; 96; 100; 104; 108; 112; 116; 120; 124; 128; 132; 136; 140;
   144; 148; 152; 156; 160; 164; 168; 172; 176; 180; 184; 188; 192; 196; 200]

It first filters the odd numbers out of firstHundred list and send the result to map function to double all the values.

  • Forward-pipe operator can be used to make the code easier to read when functions are chained:
[0..100]
|> List.filter (fun x -> x % 2 = 0)
|> List.map (fun x -> x * 2)
|> List.sum
  • Array indexing is zero-based.

Resources

securityawsdev network, raspberry_pi, nmap

One of the Nmap’s many usages is for asset management as it is very good at discovering devices in a network. I’m going to use it to develop a simple IDS (Intrusion Detection System). Of course IDS software is much more complex and I hope I will look into installing a proper one, like Snort, when I have the time but for now I’ll just roll out my own. My goals in this project are:

  1. Utilise the idle old Raspberry Pi: I used one to build a media server another one as a security camera. The 3rd one is one of the first releases. It has 256MB memory and failed to run various projects I tried in the past. Looks like it’s at least good enough to run Nmap so it may have a use after all.
  2. Practice more Powershell and discover XML parsing capabilities.
  3. Writing a Python version of the same script so that everything can run on Pi but I’ll defer that to a later date.

Basics

So like every Raspberry Pi project, first step is to download a distro suitable for Raspberry Pi and write to an SD/miroSD card. There is a Pi version of the (in)famous security distro Kali Linux. It’s convenient as it comes with all security tools pre-installed but for my purposes I just used a plain Raspbian as I only need Nmap.

Nmap that is installed from Linux repositories was a bit outdated (v6.00) so I decided to download the latest version (v6.47) and build it from source. Even though all I need is a simple command I like to keep my tools current!

How Does It Work

I placed the Pi near the switch so that it can use Ethernet. It gets results much faster so I recommend wired over wireless. So the initial version will work like this:

  1. A cron job runs on Pi every n minutes. Executes Nmap and uploads the results in XML format to AWS S3.
  2. A scheduled task runs on a Windows Server running Powershell. It gets the latest XML from S3 and gets the active hosts on the network.
  3. Compares the results to a device list and sends a notification if an intruder is detected.

Of course in order this to work first I need to assign static IPs to all my devices and record these addresses along with MAC addresses in the configuration.

Let’s get cracking!

I covered Nmap basics here. In this project all I need is host discovery so I’m not interested in the services, machine names, operating systems etc. I’ll just run this command:

sudo nmap -sn 172.16.1.0/24 -oX output.xml

Also I need my old friend s3cmd so I ran this

sudo apt-get install s3cmd

Then

s3cmd --configure

and entered the credentials for the new IAM user I created who has access only to a single S3 bucket.

So to put it together in a shell script I created the simple script below.

#!/bin/bash

echo "Running Nmap"
sudo nmap -sn 172.16.1.0/24 -oX /home/pi/output.xml

timestamp=$(date +%Y%m%d_%H%M%S)
s3FileName=nmap_output_$timestamp.xml

echo "Uploading the output to S3"
sudo s3cmd put /home/pi/output.xml s3://{BUCKET}/$s3FileName

sudo rm /home/pi/output.xml

Also to make the script executable so I ran this:

sudo chmod 755 my_script

Analyze and alert

So now I have a list of devices running on the network. The Nmap XML output looks something like this:

<?xml version="1.0"?>
<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 6.47 scan initiated Mon Dec 15 13:30:01 2014 as: nmap -sn -oX /home/pi/output.xml 172.16.1.0/24 -->
<nmaprun scanner="nmap" args="nmap -sn -oX /home/pi/output.xml 172.16.1.0/24" start="1418650201" startstr="Mon Dec 15 13:30:01 2014" version="6.47" xmloutputversion="1.04">
  <verbose level="0"/>
  <debugging level="0"/>
  <host>
    <status state="up" reason="arp-response"/>
    <address addr="172.16.1.10" addrtype="ipv4"/>
    <address addr="AA:BB:CC:DD:EE:FF" addrtype="mac"/>
    <hostnames>
    </hostnames>
    <times srtt="1142" rttvar="5000" to="100000"/>
  </host>
  <runstats>
    <finished time="1418650208" timestr="Mon Dec 15 13:30:08 2014" elapsed="6.40" summary="Nmap done at Mon Dec 15 13:30:08 2014; 256 IP addresses (11 hosts up) scanned in 6.40 seconds" exit="success"/>
    <hosts up="11" down="245" total="256"/>
  </runstats>
</nmaprun>

And my configuration file looks like this:

<?xml version="1.0" encoding="utf-8"?>
<assetList>
  <host name="Dev" ip="172.16.1.10" mac="12:12:12:12:12:12" />
  <host name="Raspberry Pi XBMC" ip="172.16.1.11" mac="AA:AA:AA:AA:AA:AA" />
  <host name="Printer" ip="172.16.1.12" mac="BB:BB:BB:BB:BB:BB" />
  <host name="iPad" ip="172.16.1.13" mac="CC:CC:CC:CC:CC:CC" />
</assetList>

A gem I discovered about Powershell is the Powershell ISE (Integrated Shell Environment). It supports IntelliSense-type discovery so makes it much easier and faster to write and run scripts.

Powershell ISE

Into the Powershell

The script does the following

  1. Load the configuration
  2. Get the latest asset list from S3 and load
  3. Compare and see if there are any unknown devices on the network
  4. If there is send an email notification

Since Powershell is based on .NET framework, working with XML is nothing new. I just used standard XPath queries to match the MAC and IP addresses of the discovered devices to the ones I entered to the configuration file.

Here’s the script:

Time for some action

OK let’s see how we are doing now. After I recorded all the known devices the output of the script was like below:

Script output

One interesting thing to note is Nmap cannot discover its own MAC address. I guess that’s because as it’s using ARP protocol to resolve MAC addresses on the local subnet and it doesn’t have its own MAC in its ARP table it cannot find it. I decided to skip the entry but may be a better choice to compare only the IP address if this is the case. Anyway, I will leave it as is for now.

To test it I turned on my old phone and connected to the network. Within 10 minutes I received the following notification email:

So far so good!

Conclusion

I would never trust such a thing as the ultimate defence mechanism but even so I believe it may come in handy in some situations. More importantly this was a fun little project for me as it involved bash scripting, Powershell, AWS and XML. I’m glad I finally came up with a use for the idle Raspberry Pi also happy to discover Powershell ISE.

Resources

security network, nmap

What is Nmap?

Nmap (Network Mapper) is a powerful network scanner that lets you discover the hosts and services on a network. It sends specific packets to remote hosts and analyses the responses to map the network. These packets can be standard ICMP/TCP/UDP packets as well as deliberately malformed packets to observe the hosts’ behaviour.

I believe it is very important to keep an eye on what’s going on in your network. As Nmap is one of the basic tools for this kind of job, I decided to spend some time to cover it and harness it in my own projects.

Specifying Target

First you need to specify your targets. Nmap is very flexible and accepts different notations for specifying targets:

  • Single IP: i.e. 192.168.1.15
  • Name: i.e: www.google.com
  • List: For example, 192.168.1,2.1,10 will scan 192.168.1.1, 192.168.1.10, 192.168.2.1 and 192.168.2.10. Note that the comma separated values are not ranges but single values
  • Range: i.e: 192.168.1.1-10 For ranges hyphen is used as the separator. The start and end values are inclusive. Also one octet can be omitted such as 192.168.-.1. In this case Nmap will scan all IPs from 192.168.0.1 to 192.168.255.1
  • CIDR (Classless Inter-Domain Routing): For example 192.168.1.240/29 will scan 8 IPs from 192.168.1.240 to 192.168.1.247

You can use any combinations of these values as a list separated with spaces such as: 192.168.1.42 www.google.com will scan the single LAN IP and Google. You can use -sL parameter to view the target list without scanning them.

Nmap list hosts

In complex scenarios you can use an input file to load the target list by using the -iL flag and providing the file name.

To exclude specific IP addresses –exclude flag is used with the same notations.

Port Scanning

Ports can be specified in 2 ways:

  • Using -p flag: Single value, comma-separated list, or hyphen-separated values as a range. If just hyphen is specified it scans all ports from 1 to 65535. Also protocol can be specified such as T:80 U:53
  • Using nmap-services file: You can refer to a service by name and this file is used to look it up.

Both methods can be used in combinations such as:

nmap -p http*,25,U:53 192.168.1.15

Output

While a scan is running you can an updated status by hitting enter. Also you can save the output results to a file in different formats with the values below following the -o flag:

* N: Normal
* X: XML
* G: Grepable

such as

nmap -v 172.16.1.0/24 -oG output.gnmap

Also a helpful flag is -v for verbose output

An added bonus about using output files is that you can resume a scan by using the –resume flag and specifying the output file name such as

nmap --resume output.gnmap

Basic scanning options

  • TCP SYN scan (-sS): This is the default option Nmap uses. It’s very fast and can scan thousands of ports per second. It sends a SYN packet to target and if the port is open target sends back a SYN/ACK packet. Thus far it’s just like a normal 3-way TCP handshake but in the final step instead of sending an ACK Nmap sends RST (Reset) packet and cancels the process. Since it has already acquired the information it’s looking for it doesn’t need to establish an actual connection. If the port is closed the target sends a RST packet. SYN scan is very powerful because it’s fast and quiet as it doesn’t create a session. on Linux, it requires root privileges to run it.

  • TCP connect() Scan (-sT): This one uses a full handshake and opens a session. Then sends a RST packet to close the session. The advantage of this method over SYN scan is that it doesn’t require root privileges. As it opens sessions they are logged so it’s noisies than SYN scan.

  • Ping scan (-sn formerly known as -sP): This is the quickest scan method.For local subnets it uses ARP (Address Resolution Protocol) to identify active hosts. ARP only works on local subnets so for remote subnets, it uses ICMP echo requests. It also sends a TCP ACK packet to port 80 which is completely unexpected for the host as it’s just sent out of the blue. So the host sends a RST packet to end connection (as it’s the right thing to do!) but that helps Nmap to identify there is a host up with that address. This scan is only helpful to identify hosts rather than ports and services.

  • UDP scan (-sU): This is the only scan that can identify open UDP ports. Since there’s no handshake the overhead is lower compared to TCP scan. When a port is closed the target returns ICMP port unreachable packet so this may increase the number of packets. Like SYN scan it requires privileged access.

In total there are lots of scanning options. You can find the full list here

OS and Service Version Detection

To detect version -sV flag is used. An intensity level between 0-9 can be specified. Default is 7

	nmap -sV --version-intensity 9 172.16.1.10

Versioning can be useful in some cases but also significantly increases the scan time.

For operating system detection -O flag can be used

	nmap -O -v 172.16.1.10

Nmap OS detection

Timing Categories

If you are concerned about being detected when scanning the network

  1. You might be doing something nasty!
  2. You might consider using timing categories so add some delay between each packets to evade IDSs

There are 6 categories that can be specified by T flag followed by a number from 0 to 5. Alternatively you can use the templates’ names:

  • paranoid (0)
  • sneaky (1)
  • polite (2)
  • normal (3)
  • aggressive (4)
  • insane (5)

With “Paranoid” template Nmap will wait 5 minutes between each probe making the total scan time very very long. “Insane” will speed up the process to a point that the delay is down to 5ms. So be careful which option you use. There are many more flags for tailoring the scans to your performance requirements: http://nmap.org/book/man-performance.html

Scripting engine

Nmap comes with an embedded Lua interpreter which is the core of its scripting engine.

By using -sC flag all scripts in the default category can be executed such as

nmap -sC -p www.google.com

Nmap script output

There are lots of scripts which can be found at NSE(Nmap Scripting Engine) documentation page

For example there is a script to scan OpenSSL Heartbleed vulnerability. It can be executed as follows:

nmap -p 443 --script ssl-heartbleed <target>

On my machine this script was blocked by Norton!

Norton attack block

So be careful which script to run. Your intentions may be misinterpreted if you are running them against systems that you are not authorized.

Conclusion

Nmap is one of the core tools that hackers (white or black hat) use. So it has many more options geared towards attacking and being stealthy. You can spoof your IP address, use idle stations to avoid detection etc. I left out many of those options as my intention for studying Nmap is discovering devices on my network so that I can take action if any unknown devices appear. Based on these notes I will develop a simple script/applicaton to find out if anything fishy is going on in my network. I’ll blog about it when it’s ready. Stay tuned!

Resources