printf(" SaltwaterC ");

Computing file hashes with node.js

SaltwaterC

Since node.js has the shiny crypto module which binds some stuff to the openssl library, people might be tempted to compute file hashes with node.js. At least the crypto manual page shows how to do a SHA1 for a given file (mimics sha1sum). Should people do this? The answer is: NO. Some may say because it blocks the event loop. I say: because it is as slow as molasses in January. At least compared to dedicated tools.

Let’s have a look:

var filename = process.argv[2];
var crypto = require('crypto');
var fs = require('fs');

var shasum = crypto.createHash('sha256');

var s = fs.ReadStream(filename);
s.on('data', function(d) {
  shasum.update(d);
});

s.on('end', function() {
  var d = shasum.digest('hex');
  console.log(d + '  ' + filename);
});

time node hash.js ubuntu-10.04.3-desktop-i386.iso
208fb66dddda345aa264f7c85d011d6aeaa5588075eea6eee645fd5307ef3cac ubuntu-10.04.3-desktop-i386.iso
node hash.js ubuntu-10.04.3-desktop-i386.iso 28.92s user 0.80s system 100% cpu 29.661 total

time sha256sum ubuntu-10.04.3-desktop-i386.iso
208fb66dddda345aa264f7c85d011d6aeaa5588075eea6eee645fd5307ef3cac ubuntu-10.04.3-desktop-i386.iso
sha256sum ubuntu-10.04.3-desktop-i386.iso 4.86s user 0.21s system 99% cpu 5.093 total

time openssl dgst -sha256 ubuntu-10.04.3-desktop-i386.iso
SHA256(ubuntu-10.04.3-desktop-i386.iso)= 208fb66dddda345aa264f7c85d011d6aeaa5588075eea6eee645fd5307ef3cac
openssl dgst -sha256 ubuntu-10.04.3-desktop-i386.iso 4.40s user 0.17s system 100% cpu 4.567 total

Edit: to sum up for those with little patience:

node hash.js – 29.661s
sha256sum – 5.093s
openssl dgst -sha256 – 4.567s

/Edit

That’s a ~6.5X speed boost just by invoking openssl alone instead of binding to its library. node.js does something terribly wrong somewhere since the file I/O is not to blame for the slowness:

var filename = process.argv[2];
var fs = require('fs');

var s = fs.ReadStream(filename);
s.on('data', function(d) {
  
});

s.on('end', function() {
  
});

time node read.js ubuntu-10.04.3-desktop-i386.iso
node read.js ubuntu-10.04.3-desktop-i386.iso 0.62s user 0.60s system 106% cpu 1.148 total

This little example that I hacked together shows that using child_process.exec is pretty fine:

var exec = require('child_process').exec;
exec('/usr/bin/env openssl dgst -sha256 ' + process.argv[2], function (err, stdout, stderr) {
	if (err) {
		process.stderr.write(err.message);
	} else {
		console.log(stdout.substr(-65, 64));
	}
});

time node hash2.js ubuntu-10.04.3-desktop-i386.iso
208fb66dddda345aa264f7c85d011d6aeaa5588075eea6eee645fd5307ef3cac
node hash2.js ubuntu-10.04.3-desktop-i386.iso 4.44s user 0.19s system 100% cpu 4.630 total

So you can have your cake and eat it too. The guys with the philosophy got this one right.

Exit mobile version