George Shank

Getting Started With Erlang (Part 1 - Semantics)

Fri, 09 Jan 2015 00:00:00 +0000

I’ve recently decided to start learning Erlang and I’ve always felt the best way to learn something is to teach others. My favorite style of teaching is pairing and face to face instruction but I don’t have any mentees nearby interested in Erlang so I’ve decided to use blogging as a way of helping others and hopefully solidifying these concepts for myself. Erlang’s syntax is very interesting and a little alien so bear with me as I try to make sense and explain the mechanics. That being said, let’s get on with the post!

Installation

I won’t spend too much time talking about installing Erlang other than to say I’m using a Mac and you can install with Homebrew simply by running the brew install erlang command. Erlang should be available for most Linux distributions as well.

The REPL

Erlang comes with a REPL that you can start up with the command erl. You can run simple arithmetic like most REPL’s such as 2 + 2 or (5 + 3) * 2 but the key thing to remember is almost every Erlang statement must end with a period(.). Here’s an example of some basic commands.

%> erl
Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Eshell V6.3  (abort with ^G)
1> 2+2.
4
2> (5 + 3) * 2.
16
3>

You’ll use the REPL to compile and run your different Erlang programs. Speaking of which let’s create a new simple program now.

Modules

Each Erlang file describes a module and exports the functions necessary to interact with it. The module name needs to match the file name so if you create a hello-world.erl you’ll need to name the module hello-world in the program, like this:

-module('hello-world').

Notice the . at the end; it’s important to end all statements like that. Don’t worry if you forget though because the compiler will let you know something is funny and give you tips on how to correct it. Now you can hop in the REPL and compile your file:

1> c('hello-world').
{ok,'hello-world'}

Awesome, we got a tuple back saying ok and the name of the module we just compiled. If our file name didn’t match up with our module name the compiler would have thrown an error. If instead of -module('hello-world'). I had written -module('goodbye-world'). the compiler would have said something like this:

5> c('hello-world').
hello-world.beam: Module name 'goodbye-world' does not match file name 'hello-world'
error

When we successfully compiled the file notice how ok isn’t surrounded in single quotes? ok is what’s called an atom. In fact, so is 'hello-world'.

Atoms

Atoms are literal constants within Erlang and can be passed around as an expressive piece of information. Atoms can either be described using words starting with a lowercase letter or surrounded in single quotes to use capital letters and non-alphanumeric characters. Here are some examples of atoms:

apple
cat
'Marmalade'
'hello-world'

Atoms don’t hold a value, they simply provide a consistent expressive way for representing information. They’re closely related to enums and symbols.

Functions and Variables

So we’ve compiled our first Erlang file but it doesn’t do anything yet. Let’s create a simple function to double a number’s value. Function names, much like atoms, are defined starting with a lowercase letter with some parameters defined.

myfunc(SomeVar) ->

Variables on the other hand begin with an uppercase letter. Not only that but a variable can only be defined once. This one of many properties in Erlang that make it a functional language and is included to remove the possibility of side-effects. If you want to save the result of a calculation you’ll need to store it in a new variable.

11> MyVar = 3,
11> MyOtherVar = MyVar + 3.
6

See that comma I threw in there? When you’re building a function the period is for the very end of the function and commas are used to separate statements leading up to the final statement. We’ll talk more about that later, for now let’s go ahead and create our double function.

-module('hello-world').

double(Val) ->
	Val * 2.

That’s fairly simple isn’t it? We created a function that accepts a value and assigns it to Val. We then multiply it by 2. Notice that we don’t have to explicitly call return. Erlang implicitly returns the last evaluated expression from each function. We could even do this:

-module('hello-world').

double(Val) ->
	DoubledValue = Val * 2,
	DoubledValue.

There’s that comma again, separating the first statement from the last. As I said before, the comma says there are more statements to come, I’m not yet done processing. Also this time we return the value by simply putting the result as the last statement. Erlang will evaluate and return it just the same.

This function is looking pretty solid and all that’s left is to export it so it can be used. You’ll use the -export() statement to do that.

-module('hello-world').
-export([double/1]).

double(Val) ->
	DoubledValue = Val * 2,
	DoubledValue.

-export takes an array of function definitions you would like to export. Function names need to be accompanied by their arity which is the number of arguments they accept. Functions with the same name but different arity are considered completely separate. Now that we’ve exported double we can compile the file and use the function.

18> c('hello-world').
{ok,'hello-world'}
19> 'hello-world':double(3).
6

Success!

That’s all I’ll write for this installment but if you like what you’ve seen so far you can follow me on Twitter or subscribe to my RSS feed and you’ll know when the next blog post in this Erlang series gets posted.

Composing Your Solution

Wed, 22 Oct 2014 00:00:00 +0000

In the past couple weeks I’ve been writing a few different modules and publishing them to npm. When I’m writing new code to solve a problem I often end up coding together some functionality and releasing a singular module however this time around I tried to do things a little differently. Instead I tried to sit down and break out each individual piece of my solution (similar to what’s often referred to as the substack pattern). What pieces could I work on separately that would have value and merit on their own? The result was 3 individual modules with each one being a composition of the last. Breaking out my solution like this did a few things that I think are big wins when programming:

I made my problem’s surface area smaller by identifying tinier problems (and easier wins)
I made useful pieces of my solution available for others to compose into other potential solutions
My github activity bar is looking awesome

##The Problem The problem I set out to solve was how to keep dat (a project with the goal of streamlining data transfer between any and all data stores or file formats) in sync with MongoDB. Doing a single import had been solved by mongs but there was no existing way to keep that import up to date with changes in Mongo. Having identified my problem I broke it out into smaller problems.

##Oplog Docs The first tinier problem was translating Mongo’s oplog documents into actions that can be performed on any object. Mongo utilizes the oplog to translate changes in the primary to its secondaries keeping them in sync. The syntax of these JSON documents is little peculiar at first but they provide information about document updates, inserts, as well as deletes. Essentially everything you need to make the same changes to a document that lives outside of Mongo as well. I built oplog-transform to take oplog documents, turn them into functional expressions, and apply those changes to any JSON object you want.

##Tailing the Oplog Now that we can translate oplog docs we just need to hook up to the oplog and tail it to perform these changes in real-time. I created oplog-transform-tail to do just that. It’s a simple composition of the great mongo-oplog lib as well as oplog-transform. We’re already reaping the benefits of breaking these pieces out by being able to build them up in new useful ways.

##Wrapping up Finally we can solve the overarching problem we set out for in the beginning, syncing dat with MongoDB, and the result is dat-oplog. Dat has hooks you can plug into with your own functionality and dat-oplog makes use of listen. Now using oplog-transform-tail we can push updates to dat every time a document changes in Mongo.

The code needed to create dat-oplog was relatively small since we can leverage the previous modules. Each module being responsible for its own functionality keeps the individual code bases small which can reduce bugs and increase transparency into what is happening. Building a solution this way has a lot of advantages and I think it’s a worthwhile exercise when tackling any large problem.

How to Enable a MongoDB Replica Set on Travis

Tue, 14 Oct 2014 00:00:00 +0000

Travis is a fantastic resource for developers building open source projects since they provide free build servers to run your tests automatically. Every time you push new code to your repos on Github Travis can pull down the changes, run the tests, and update you when they fail. It also works with just about every imaginable language and provides quite a few databases out of the box.

When setting your project up for Travis a basic config can often work pretty well but sometimes you need something a little more custom. I like to use Travis for most of my projects so while I was building oplog-transform-tail (a node module that keeps your MongoDB data in sync with a secondary data store in real time) I went ahead and created a somewhat standard Travis config file for a node project using MongoDB.

That looks something like this:

#.travis.yml
language: node_js
node_js:
  - "0.11"
  - "0.10"
services:
  - mongodb

This config just says that we’re using node, we want to test on version 0.10 and 0.11, and under services we say we want access to a MongoDB database. The problem with this setup is oplog-transform-tail requires access to MongoDB’s oplog. By default MongoDB starts as a single server with no replica set and as such does not utilize the oplog.

To turn this feature on we just need to guide Travis a little bit and tell it to set our MongoDB up as a replica set. Luckily Travis has a few lifecycle entrypoints that we can take advantage of to do just that. Our use case just needs to happen before the script event (when the tests get run) so we’ll use the before_script hook to make the changes ensuring our database is set up properly for testing.

Here’s our new config for Travis:

#.travis.yml
language: node_js
node_js:
  - "0.11"
  - "0.10"
services:
  - mongodb
# make mongodb a replica set
before_script:
  - echo "replSet = myReplSetName" | sudo tee -a /etc/mongodb.conf
  - sudo service mongodb restart
  - sleep 20
  - mongo --eval 'rs.initiate()'
  - sleep 15

Let’s talk about what this does.

echo "replSet = myReplSetName" | sudo tee -a /etc/mongodb.conf Here we append a string to the MongoDB config file, telling it set the replSet name to myReplSetName. This is all that’s required for basic replica set initialization.
sudo service mongodb restart Now we restart MongoDB so it can use the new config
sleep 20 Sometimes the database can take a while to reboot, we sleep 20 seconds here to make sure it’s ready for the next step
mongo --eval 'rs.initiate()' Here we tell MongoDB to evaluate our command, in this case rs.initiate() which tells MongoDB to start up the replica set functionality.
sleep 15 Here we sleep for another 15 seconds to give MongoDB a chance to properly set up the replica set.

And there you have it, now the MongoDB instance is running as a single server replica set and will write operations to the oplog allowing the tests a chance to run properly.

Keeping Elasticsearch Springy

Tue, 10 Sep 2013 00:00:00 +0000

While elasticsearch is quick to set up the initial configuration is not quite production ready. In this post I would like to lay out some simple tweaks you can perform to get a consistent reliable search cluster up and running.

I wrote a blog post about some small performance tweaks you can do with elasticsearch.

Partial Application with JavaScript

Thu, 14 Mar 2013 00:00:00 +0000

If you haven’t heard functional programming is all the rage these days. One of the more interesting aspects of functional programming is something called partial application which essentially comes down to pre-loading arguments in a function.

But George, if I’m going to pre-load functions with variables why don’t I just use regular instance variables instead. Well, that’s silly and a good way to write more boilerplate or even duplicate code. When you’re using a language that supports higher order functions you should really take advantage of partial application (or ~~currying~~ actually, partial application and currying are not the same, Raganwald explains) to make your code more polymorphic. Essentially it gives you more value from your function. Enough gibber gabber, let’s write some code!

function sendMessageTo(name, message) {
  console.log(name + ', ' + message);
}

sendMessageTo('George', 'clean your room!'); //George, clean your room!

So what I’ve just done here in mom.js is created a function that addresses a console.log to a name with a message. Super complex stuff. I then had it tell me to clean my room (over my dead body mom). But say, what if we wanted to tell me more than one thing. Like do the dishes, mow the lawn, perhaps even a bit of laundry?

sendMessageTo('George', 'do the dishes'); //George, do the dishes
sendMessageTo('George', 'mow the lawn');  //George, mow the lawn
sendMessageTo('George', 'do the laundry');//George, do the laundry

Wow, she’s furious and there’s no way I’m doing all of that. It’s unfortunate she had to specifically say to tell George that each time too. Recalling your own son’s name is old school, computers should retain that info for us. Here comes partial application to the rescue!

var sendMessageToGeorge = sendMessageTo.bind(null, 'George');

sendMessageToGeorge('be nicer to your mother'); //George, be nicer to your mother

Hey, that’s neat. Now she doesn’t have to pass in a name each time. It’s almost like she’s got a direct line! Let’s break down what happened here.

JavaScript functions have a method called bind (at least es5 asks that most implementations do) which does 2 fancy things. The first argument I passed in was null. All this does is set the this context, a much bigger concept than the one I’m trying to convey right now. Essentially it allows you to change what this means inside the function. I passed in null because I don’t care about that in this instance, much like the laundry.

The next argument I passed in was the first argument of the function itself. It doesn’t have to stop at 1 either, you can pass in all of the variables for the function if you would like.

var sayHiToGeorge = sendMessageTo.bind(null, 'George', 'hi');
sayHiToGeorge(); //George, hi

That’s rather pointless but you can do it. You can pass in more too but they won’t go anywhere (unless you are grabbing them from the arguments hash).

So there you have it, partial application in JavaScript. It’s a neat thing and can be very useful. Functions allow us to break our code up into nice concise blocks of functionality and partial application gives us the power to customize those functions as we need to without repeating code. Next time I’ll touch on how binding this to the function can benefit you as well. Until then, happy JavaScripting!

Font Awesome

Mon, 10 Dec 2012 00:00:00 +0000

I can’t believe I’m just finding this out but there is something called Font Awesome which has free font glyphs. Font glyphs are rad because of two things.

They scale very well for high density screens
Changing the color of them is as easy as style="color:#000000;"

Github pioneered the icons as fonts area of web with their Octicons and I think it’s spot on for most needs. The only limiting factor is each icon can only have one color.

Previously I was using the free Glyphicons for my social links but quickly saw how ugly they were on retina devices (they also have fonts, just not free). I’ve swapped them out for Font Awesome glyphs and I think they look much better.

New Blog

Sun, 09 Dec 2012 00:00:00 +0000

Well, I finally bit the bullet and put my old blog to rest. It was an old Rails knockoff of Svbtle called Obtvse. The old blog was very boilerplate and I didn’t really want to delve to deep into customization with it which led to me not wanting to post very often for fear I would migrate soon.

Well, that migration has happened.

Welcome to the new and improved georgeshank.com. It’s faster, stronger, and powered by Github Pages using Jekyll. For those that don’t know Jekyll is a “blog aware” static site generator that just happens to be available on Github’s free pages offering. With a little css, html, and some YAML configuring you’re off with a free blog by Github. It’s pretty powerful stuff.

Another fun part of creating the new blog is that this one makes use of “responsive design”. Many people have different interpretations of what that means and for this blog that just means it looks good on all screens. (Except maybe retina macbooks, let me know if not!)

I’m hoping that with a new blog I’ll be encouraged to post more. Who knows, maybe I’ll post less. Either way a new year is around the corner and I’m thinking some big changes are on their way.

Npm Scripts!

Sun, 13 May 2012 00:00:00 +0000

When you’ve got a somewhat complex Node app going you can end up with a lot of necessary batch operations you need to run before actually running your app. Some people run them manually and others actually write a small script that does this for them. Either way it’s a little disconnected from actually running the app.

It turns out NPM actually has an amazing solution for this. NPM supports a scripts setting in the package.json where you can configure many items, among them being start and prestart. You simply add a command line action you’d like performed before starting the app in the prestart tag and then node app.js in the start tag and you’re ready to rock and roll. All you have to do is npm start and you’re off to the races.

I have some coffeescripts I need transpiled before each run so my package.json looks a little like this:

{
  "name": "app-name",
  "author": {
    "name": "George Shank"
  },
  "version": "0.1.1-3",
  "dependencies": {
    "express": "2.5.9",
    "jade": ">= 0.0.1"
  },
  "scripts": {
    "start": "node app.js",
    "prestart": "coffee -c public/js/*.coffee"
  }
}

There’s a ton of potential here. You could do this with all your Less files as well or anything else you need done just before run time.

Happy hacking!