Part of my extracurricular habits as a software engineer include being intentional about using technology stacks other than the Scala/Lift stack I’m employed for. I think the forcing myself outside the box of what I’m familiar with is a good exercise. But sometimes, I find myself exasperated, and I’m forced to ask the question “Am I just doing it wrong?” I’m hoping you guys can help me with that.
Lately, that frustration has been directed at Node.js and Express while working on the gga-api project. Here are a few of the specific frustrations I have. Maybe you can clear them up for me.
Try as I might, I seem unable to avoid callback hell. This is particularly a pain when trying to do data process in which a series of operations need to really happen sequentially. For example, cross referencing data. The callback hell nature of files like import-committee.coffeefeels nasty. What is actually going on is obscured by the abundant use of callbacks. Though, import-members.coffee is probably worse, in thinking about it.
We’ve tried to make some use of promises to get around it, but I’m still not satisfied that the code substantially explains itself to someone reading through it. I’m also not confident that these patterns are extendable for the long term.
Particularly Express controller code. I opted for this solution for loading up all my controllers. So I require that file in my main app and pass in the app instance, the jobs instance, and the database. This feels insanely awkward. Also, my controllers tend to end up with a good bit of boilerplate. And a lot of “if err, do something then shortcut the function with a return.”
I also find a good bit of awkwardness in the job processing code. (Anything under jobs/).
It seems there’s no real convention in how this is supposed to be done. I’ve seen some threads such as this that lead me to believe I’m not completely off the beaten road, but I’m still having trouble formulating specific improvements that make me feel better about the code I’m producing. Everything I come up with seems very marginal.
Tests are obnoxious mostly because it feels like I must enter requirement / stubbing awkwardness to make them true unit tests.
Take legislationSpec.coffee as an example. This is not a unit test. This is an integration test. If I want it to be a unit test, I’d need to pull apart legislation.coffee in such a way that I stubbed req and res objects. Or perhaps move the actual database retrieval code separate of the request handling? That may be what I end up doing, but every solution I come up with in my head will require things like “require ‘../../helpers/legislation.coffee’” – and then interaction with those objects. It would seem as though I’m not really gaining much over languages such as Scala if I’m required to do that though.
Additionally, the mocha testing framework (which comes highly recommended) doesn’t seem to integrate well with Jenkins. It seems to make heavy use of line resets, with no way to turn them off. (Or even a way to tell it to cache the results and just output the results after everyone has phoned in.) So, I get output that looks like this:
I’ve tried searching around, but haven’t had much luck in finding out how to prevent this.
So, I realize this post is very stream of conciousness, but I need a gut check here: Am I completely off track in how I’ve organized this project, or is this just how Node/Express projects tend to go?
I’m genuinely interested to know what I’m (if anything) doing wrong and get specific improvement suggestions. I’ve read a good bit but there’s either
I’m at the point where I’m seriously considering burning the project to the ground and rewriting it in Scala and Lift. This will be the second Node project I decide to do that for, if that is indeed my decision, so I wanted to open myself up for some other opinions. Node and Express both have a thriving community with lots of smart people, but either I’m doing it wrong or I just genuinely don’t get the hype.
Leave me some comment love here or on the Hacker News thread.