There’s never been a committee, board, or any other official body that directed node.js. But, in the first year of node.js, whenever a problem seemed too insurmountable to handle through email, a small group of us that, for a short time, all lived in the Bay Area would meet. I labelled our group the “Node Illuminati” as a slam against our clearly undemocratic process and lack of transparency.
At the time CouchOne had a beautiful office in recently renovated “Old Oakland” where I maintained a desk and an iMac I seldom used in favor of the IKEA rocking chair and a laptop.
In early October Matt Ranney called a new meeting, mainly to talk about Streams, which were a topic at every meeting, and some recent binary issues. When assembled this meeting consisted of me, Ryan Dahl, Paul Querna, Tim Caswell, Isaac Schlueter and Matt Ranney. Paul and Matt were the adults. They had more experience than all of us and were tremendously respected. Tim, Isaac and I were younger and had taken on a lot of the early node.js experimentation. Ryan was Ryan.
The 800 pound gorilla was "what to do about binary." Buffers (static allocations of memory outside of the v8 heap) were added in April; prior to that we were using binary encoded strings which were slow and meant we easily exceeded the then-max heap size of 1gig. From April through October, Buffers had slowly cemented themselves in to all of the APIs in node.js and their shortcomings were starting to become clear.
“What you really want is a string, most of the time,” Ryan said. The issue, of course, remained that data from the kernel had to be written to something like a Buffer and then converted to a string, requiring among other things a memcopy. Even worse, when the string ends up needing to be written back out to a file or socket it’ll require another conversion.
“Can we get at the raw memory address the underlying string is in so that we can write it out using a writev()?” I asked.
Ryan explained, “The problem is that the string is in the heap, which means at any time it can be compacted and that address in memory won’t be valid anymore and when we give that address to the kernel (this was before libuv, now we hand the address to libuv) it doesn’t write it immediately, it'll do it some time in the future.”
It wasn’t an exaggeration. The most anyone had done with Type Arrays were gaming demos and the ECMA standard was barely a wiki page. Meanwhile, Matt was pushing voice messages for a few hundred thousand users of Voxer through node.js and in a few months this would increase to millions so his experience alone put our group ahead of anyone working on standards at the time. Ryan’s protest changed our direction and it enabled us to be bolder in solving our own problems.
The eventual solution was much simpler than we had anticipated. As it turns out the slower part of Buffers at that time were waiting on
malloc and so we moved to
SlowBuffer is the old C code for
Buffer that does
SlowBuffer into smaller chunks. Today, the
Buffer object in node.js is
The longer the project goes on without Ryan being involved day to day the more you can see what his real legacy is. He left the project with a lot more than code, and it’s moments like this that I remember the most. Ryan had no patience for roadblocks. Anything (standing?) between people using node and succeeding in accomplishing the goals of their application was immediately removed. Even worse were committees and standards bodies which are the slowest to accomplish their goals and were to be avoided and even actively ignored.
Proclamations and guiding words have a short lived effect on the community. Ryan rarely commented on stuff like this publicly. What he did instead was far more effective: he changed the way the rest of us felt. Isaac and I were especially impressionable those first few years and Ryan altered our methods considerably. Isaac and I actually met through CommonJS and worked on a few standards together only to later thumb our noses and leave CommonJS to focus more exclusively on node. Now Isaac and others stand around core, greatly influencing the culture which remains very true to Ryan's early ambitions and beliefs.
If you want to leave your mark, and if you want to change something for the better, you need to change people more than code. Write all you want, prove you’re right a hundred times over, nobody will be affected unless you change how they think and enlighten them about a better way.