How to choose the right tool for the job: awe.sm’s language journey

bennett

Steps

At awe.sm we’ve repeatedly followed a set of steps when determining the right tool for the job, so we wanted to share them with you.

  • Identify your objectives
  • Don’t be prejudiced
  • Know that it will change

This is a constant issue to tackle and is especially important when choosing a primary language to write code in.  Let me better explain each step by using our language selection as an example and then I’ll show you how we’ve put it to use over the years.

Identify your objectives

This doesn’t just mean using a tool purpose-built for each task — for example, using Node.js for event-driven problems and Erlang for distributed ones — but using the tool that provides the best tradeoffs.

We’ve boiled down our language requirements to:

  • Getting the primary objective done
  • Speed of learning
  • Extensibility
  • Libraries and frameworks

Optimally, the payoff to a successful choice is a language that helps us write faster and better code so that we can offer great products.

Don’t be prejudiced

I can dislike a language’s syntax, but code isn’t about how it looks, it’s about what we can build with it.

Know that it will change

Since our requirements and priorities will constantly evolve — much like every startup’s — we also have to acknowledge that the tool may not be appropriate in a year. Over-analyzing an important decision doesn’t make it easier.  Instead attempt something small — an MVP — to understand tradeoffs in action.

In action

We’ve changed languages a few times as the company has evolved and its objectives/resources have changed.  We were initially a Ruby shop, then a PHP one, and now we are moving pieces into Java.

2009

  • Language Choice: Ruby with Rails
  • Objective: fast development, scalability not a factor
  • Speed of learning: fast, contractors familiar with Ruby and Rails
  • Extensibility: good, Ruby can handle many problems but not all of them
  • Libraries/Frameworks: great, tons and a vibrant community

The good

This was the right choice at the time and it let the contractors get a product up and running quickly.

The bad

Customer began using the product and Ruby and Rails began to be troublesome.  Memory footprints required more servers.  An in-house team was brought in but none were Ruby experts and it was becoming hard to scale the applications.

2010

  • Language Choice: PHP with Zend Framework
  • Objective: fast development, lower server costs, address current scalability issues
  • Speed of learning: fast, high familiarity with PHP and Zend Framework
  • Extensibility: ok
  • Libraries/Frameworks: good

The good

Its lower memory footprint allowed us to cut down our servers and increase our throughput.  We were able to quickly build out tons of systems.

The bad

We didn’t design everything with testing in mind, so we encountered trouble building tests.  Also, we were having trouble sharing code since it wasn’t built with re-usability in mind and we constantly encountered the challenge of adding servers without proper management (metrics, health checks, automation) in place.  

2012

  • Language Choice: ?
  • Objective:
    • High-performance APIs
    • Better testing, re-usability, server management
  • Speed of learning: low to medium (we need experts/leaders)
  • Extensibility: great
  • Libraries/Frameworks: great

Our objective was on performance and reliability and our team had grown.  Therefore we could spare some people to work on performance over new features. Revisiting the question of which language was best suited for the job, we came up with these possibilities:

  • Ruby (we now have some resident experts)
  • Python
  • Scala
  • PHP (with improvements addressing the problems encountered)
  • Java

We tried to not be prejudiced.  We re-weighed previously discarded languages to see if they best met our new set of requirements.  Without explaining why we discarded the others, I can explain why we chose Java.

High performance APIs

Java is known to be a performant language with its multi-core CPU utilization,  multi-threading, and libraries for concurrency.

Speed of learning

Java also presents a low barrier to entry, since we have a resident expert, and the rest of the team can get up to speed quickly since they’re already familiar with PHP 5 which has a very similar syntax to Java.

Extensibility

Java is extremely extensible.  It is somewhat time-consuming to get a simple project started, but Java can span many different project types: asynchronous jobs, web services, event-driven systems, and concurrent applications. Furthermore, it’s possible to build full systems inside a Java application since you can have local queues and and caches.

Libraries/Frameworks

There are a ton of libraries and frameworks available, so many that it’s intimidating to distinguish the good from the bad — but if you want a feature, it most likely already exists.

Scala and Clojure are impressive languages, but have a much steeper learning curve. By contrast, we can hit the ground running in Java, and in the future migrate to Scala without wasting any time, thanks to both languages’ great interoperability support.

We chose to use Dropwizard after it came highly recommended for high performance APIs.  It offers that feature as well as metrics, health checks, and a good structure for many applications.

How’d it go?

After we decided on Java and Dropwizard, we chose a small project to test it out.  It was a huge success, so much that it made our other systems look bad in comparison.

High Performance APIs

The java implementation of the service can easily handle more than twice the throughput of the old system.  Most of this is obtained by using asynchronous tasks and connection pooling.  Yet, there is still a lot more room for performance that we haven’t investigated.  

Better testing, re-usability, server management

These problems can easily be addressed in any language but we need to keep that in mind from the start and design systems with these requirements in mind.

We’ve been using JUnit and Mockito for testing (suggestions from Dropwizard) and they’ve been extremely helpful at allowing us to easily test individual pieces of logic.

Dropwizard’s metric’s view combined with Graphite and Dropwizard’s extensible health checks have given us visibility to how each server is performing.

We are finding a lot of common code, which we’re extracting into common modules (which we plan to open source soon). Many of these are wrappers for caching, queueing, and background jobs.

Speed of learning

Getting setup with the new set of tools caused a short term productivity drop, but it was fairly easy to get familiar with the language and the libraries.

Extensibility

Not only are we building high performance APIs, but we are also using Java to handle some data processing, and we plan to continue rolling it out to more systems that require it.

Libraries/Frameworks

Dropwizard is extremely helpful for providing a good structure for a code base regardless of its function.  It has also been a great guide for helping narrow the huge mass of libraries to use for logging, HTTP, databases, and metrics.

Lastly

We are really happy with our choice of Java and the process we’ve been using.  If you’re interested in being part of the fun, we’re hiring!

 

This entry was posted in Engineering and tagged , , . Bookmark the permalink.

2 Responses to How to choose the right tool for the job: awe.sm’s language journey

  1. Michael Galloway says:

    Great article. Thanks for posting. I’m curious how you guys started the migration. Did you guys isolate functionality and then leverage the PHP-Java bridge or did you move entire application layers?

  2. Pingback: Rails v Express v Zend | awe.sm: the blog