I’ve been doing some HTTP benchmarking on OSX lately, using ab (ApacheBench). After a large volume of requests, I always ended up with connection timeouts. I used to blame my application and mentally filed it as “must investigate”.

I was wrong.

The problem here was OSX, which seems to only have roughly 16000 ports available for connections. A port that was used by a closed connection is only released after 15 seconds. Quick calculation shows that you can only do a sustained rate of 1000 connections per second. Try to do more and you’ll end up with timeouts.

That’s not acceptable for testing pretty much anything that scales.

 

Here’s the workaround: you can control the 15 seconds release delay with sysctl:

There’s probably a good reason why it’s in there, so you might want to revert this value once you are done testing:

 

Alternatively, you could just use Linux if you want to get some real work done.

What’s the point of releasing open-source code when nobody knows about it? In “Release Notes” I give a round-up of recent open-source activities.

Slightly calmer month, nonetheless, here are some things you might enjoy:

 

angular-debounce (New, github)

Tiny debouncing function for Angular.JS. Debouncing is a form of rate-limiting: it prevents rapid-firing of events. You can use this to throttle calls to an autocomplete API: call a function multiple times and it won’t get called more than once during the time interval you specify.

One distinct little feature I added is the ability to flush the debounce. Suppose you are periodically sending the input that’s being entered by a user to the backend. You’d throttle that with debounce, but at the end of the process, you’ll want to immediately send it out, but only if it’s actually needed. The flush method does exactly that.

Second benefit of using an Angular.JS implementation of debounce: it integrates in the event loop. A consequence of that is that the testing framework for E2E tests (Protractor) is aware of the debouncing and it can take it into account.

 

angular-gettext (Updated, website, original announcement)

A couple of small feature additions to angular-gettext, but nothing shocking. I’m planning a bigger update to the documentation website, which will describe most of these.

 

ensure-schema (New, github)

Working with a NoSQL store (like MongoDB) is really refreshing in that it frees you from having to manage database schemas. You really feel this pain when you go back to something like PostgreSQL.

The ensure-schema module is a very early work-in-progress module to lessen some of that pain. You specify a schema in code and the module ensures that your database will be in that state (pretty much what it says on the box).

It supports PostgreSQL and SQLite (for now). One thing I specifically do not try to do is database abstractions: there are other tools for that. This means you’ll have to write specific schemas for each storage type.

There’s a good reason for that: you should pick your storage type based on its strenghts and weaknesses. Once you pick one, there’s no reason to fully use all of its capabilities.

This module is being worked out in the context of the project where I use it, so things could change.

 

Testing with Angular.JS (New, article)

Earlier last month I gave a presentation for the Belgian Angular.JS Meetup group:

ngmeetup-testing.001

The slides from this presentation are now available as an annotated article. You can read it over here.

What’s the point of releasing open-source code when nobody knows about it? In “Release Notes” I give a  round-up of recent open-source activities.

Since this is the first instalment of what will hopefully be a regular thing, I’ll look back a couple months into the past. A long (and not even complete) list of things, mostly related to web technology. I hope you’ll find something useful among it.

 

angular-encode-uri (New, github)

A trivial plugin for doing URI encoding in Angular.JS view, something it oddly doesn’t do out of the box.

 

angular-gettext (Updated, website, original announcement)

The nicest way to do translations in Angular.JS is getting even nicer, with more improved coverage of strings and filetypes, built-in support for asynchronous loading and more flexibility.

But most of all: rock-solid stability. Angular-gettext is now in use for some nice production deployments and it just works.

Highlights:

  • The website is now open-source and on github.
  • There’s an ongoing effort to split the grunt plugins up into the actual grunt bits and a more generic library. There’s also a Gulp plugin coming, so you can use any tooling you want.
  • Functionality for loading translations asynchronously.
  • Now usable without jQuery loaded.
  • Better handling of translation strings in directives.

 

angular-import-scope (New, github)

Angular.JS structures your data in nested scopes. Which is great, except when page structure doesn’t work like that and you need the inner data on a much higher level (say in the navigation). With import-scope, you can import the scope of a lower-level ui-view somewhere higher up.

mockup

 

angular-select2 (New, github)

A select2 plugin for Angular.JS that actually works, with ng-options support.

 

connect-body-rewrite (New, github, DailyJS coverage)

A middleware plugin for Connect.JS that helps  you transform request bodies on the fly, typically on the result of a proxied call. Used in connect-livereload-safe and connect-strip-manifest (see below).

 

connect-livereload-safe (New, github)

A Connect.JS middleware plugin to inject livereload. What’s wrong with connect-livereload? Well, I ran into some freak issues where modifying the DOM during load breaks Angular.JS. This plugin avoids that.

 

connect-strip-manifest (New, github)

Connect.JS middleware to strip the HTML5 app cache manifest. Makes it possible to disable the caching in development, without having weird tricks in your HTML file.

 

grunt-git (Updated, github)

A pile of new git commands supported, with a much improved test suite.

 

grunt-unknown-css (New, github)

Lets you analyze HTML files to figure out which classes don’t exist anymore in the CSS. Good for hunting down obsolete style declarations.

 

grunt-usemin-uglifynew (New, github)

A plugin for grunt-usemin that reuses existing .min.js files. This speeds up compilation of web apps and lets you use the minified builds provided by library authors.

 

json-inspect (New, github)

Get JSON context information out of a string. Lets you build text editors that are aware of the structure of the JSON underneath it.

Suppose you have this:

With json-inspect you can figure out what it means if the cursor is at a given position:

 

mapbox-dist (New, github)

A compiled version of Mapbox.JS, which you can use with Bower.

 

Nested Means (New, github)

A data quantization scale that handles non-uniform distributions gracefully.

Or in human language: a Javascript module that calculates how you can make a meaningful legend for colorizing a map based on long tail data. If you’d use a linear scale, you’d end up with two colors: maximum and minimum. Nested means tries to adjust the legend to show more meaningful data.

nested-means-3

A linear scale would map everything to white and dark-green. Nested means calculates a scale that maps to many colors.

 

node-trackchange (New, github)

An experiment in using Harmony Proxies for tracking changes to objects. Here’s an example:

You can even wrap constructors. This ensures that each created instance automatically has change tracking built-in:

 

pofile (New, github)

A gettext .po parser and serializer, usable in the browser and on the backend. The angular-gettext module is powered by this library.

 

Blog it or it didn’t happen.

In a shocking move, Ubuntu decided to follow the long-strung-out decision of Debian to adopt systemd as their init system. This is a good thing: everyone can now get together and work on one great solution. I applaud them for making this move.

It’s also a good thing for those who depend on systemd for having a fantastic modern deployment environment: soon you’ll be able to depend on systemd everywhere, regardless of the distribution being used.

In this light it seems like a good idea to shamelessly mention the write-up I wrote a while back: Deploying Node.js with systemd. Everything in there is still highly relevant and relying on systemd for deploying Node.JS is still (in my humble opinion) one of the best possible setups.

Good times for Node.JS developers that also need to administer infrastructure.

Recently I’ve had to add translation support to an Angular.JS application. There’s a couple of modules around that can help you with that, but none of them felt right. Either they force you to use key/value mappings (cumbersome) or they have a very painful, broken system for plurals.

I wanted something that is:

  • Trivial to use
  • Doesn’t take any of the nice aspects of Angular.JS away from me.

Enter angular-gettext.
I’ve always been a fan of gettext, which is used extensively in the GNOME project, now there’s an MIT-licensed Angular.JS implementation.

Translating a web application with angular-gettext is as simple as adding an attribute:

<a href="/" translate>Hello {{name}}</a>

This, together with a small library and some Grunt tasks allows you to open your app up to any language you want. Interpolation just keeps working and there’s support for plurals.

There’s lots of documentation on the website, have a look at it if you want to go global: http://angular-gettext.rocketeer.be/

Posted .

Tags:

This was written some time ago, but might still be interesting to release: node-apk-parser.

If you ever tried to extract data from an Android APK file (such as the application name), you’ll have noticed that Google decided to use a very complex binary XML encoding. This format is a major pain in the ass to decode. The node-apk-parser module helps you do exactly that: get as much info out of an APK file, without the pain.

There are some modules available for this, but none handle encoded strings, so they’re generally useless. This module uses the aapt tool from the Android SDK and is thus capable of decoding all the gory details you want.

Unit-tested and battle-tested in production, now available for yours to enjoy:

I love cycling. When you are in a new area, it is really hard to figure out where the good roads are located. Smooth and nice asphalt might be right around the corner, but that is rarely indicated on a map.

The average racing bike has a ride that is hard as a rock so finding a route that’s not filled with potholes and cracks may mean the difference between a relaxing and a hellish afternoon.

Today I found this thing:

raceshape

 

A heat map of Strava rides. Makes finding great roads a breeze. Fantastic.

I’m on Strava too!

Posted .

Tags:

If you’re building Angular.JS apps, you’ll probably be using the excellent Karma test runner. I’ve been running into some troubles getting Google Chrome to work when using tmux on OSX. Here’s how to fix it.

 

The problem
I’m a fossil, I don’t use fancy new IDEs. My tools of trade, which have served me well for years now are tmux and vim. Tmux is like screen, but more powerful. It allows me to split my terminal into panes and have bash next to vim. Awesome.

Screen Shot 2013-07-04 at 12.00.23

Things go wrong when you start using the Karma test runner with Google Chrome.

You’ll be greeted by this guy:

Screen Shot 2013-07-04 at 11.58.35

 

The solution
Chrome crashes on any page, so this really doesn’t have anything to do with Karma (it just manifests itself there, because it launches a completely new Chrome instance). Turns out the daemon model of tmux is the problem here: tmux runs detached from the GUI, which means Chrome can’t find a screen to paint on.

 

There’s an easy fix!

  1. Get Homebrew, if you don’t already have it.
  2. brew install reattach-to-user-namespace
  3. Add the following line to your .tmux.conf file: set-option -g default-command "reattach-to-user-namespace -l bash"

 

Start a new tmux server and everything should work again.

 

Credit to http://robots.thoughtbot.com/post/19398560514/how-to-copy-and-paste-with-tmux-on-mac-os-x.

Stuff like this makes me sad:

Also, the github issue where TJ requests that everything gets rewritten in plain JavaScript: https://github.com/rethinkdb/rethinkdb/issues/766

 

We’ve been here before
Language discussions aren’t new (nor is vim vs. emacs). In the GNOME community we’ve seen a ton of them. Just recently there was a huge one at the DX Hackfest.

GNOME/Mono developers have certainly received their dose of crap thrown at them. But so have GNOME developers that preferred Vala, Python, JavaScript, or even just GObject/C. Whatever you seem to be using, it’s never the right thing for someone.

Have all these years of shedding words over it solved anything? Frankly: no. We are still seeing a large combination of languages being used and all of those projects have good reasons to do so.

I get TJ’s point though: by using CoffeeScript, the rethinkdb people are making it harder for the wider JS community to contribute to their project. But…

 

It really doesn’t matter
Most open-source projects (or modules) don’t have a ton of contributors. It’s usually a modest team of core maintainers/developers that do the bulk of the work. And that’s fine: the success of a project should not be measured by the number of contributors, but by the quality of the software it produces.

This smallish team of core developers will have their own good reasons for picking up a certain language. They’ll use the language that they feel most productive with for the task at hand. And that’s a good thing, they are mostly the people that move the project forward.

The biggest barrier to contributing on a project is not the language, there are plenty of projects written in unproductive languages that get a ton of contributions. Any good programmer can pick up a new language up quickly (and TJ is more than just a good programmer, he’s a fantastic one, much respect). The bigger hurdle is the specific domain knowledge involved.

Let’s all agree to disagree and have some respect for each other’s opinions, they are all valid anyway.

 

PS: I’ll be heavily moderating comments that try to turn this into a flame-war. I’m writing this to find some more respect and understanding.