Command-line utilities with Node.js
One often overlooked feature of node.js is the ability to create command-line utilities. Here I want to demonstrate how simple it can be to create useful command-line utilities with the help of node.js. I’m going to create a small utility which allows for quick searching of GitHub for reposities based on keywords, owners and languages. If you’d like to jump straight into the source code, it’s available here.
Understanding the command-line
Before writing anything for the Unix command-line, no matter what scripting language you are using, it’s important to understand common patterns used for input. The most basic pattern consists of three main components: the command, options and arguments.
Command
Commands are categorized into three types:
-
Internal - a command recognized and processed by the command-line which is not dependent upon any external executable file.
-
Included - a command that requires a separate executable file that is always included with the OS and generally considered part of the OS.
-
External - a command that requires an external executable file which is not part of the OS and instead added by a third party.
Options
Command-line options can be used to modify the operation of a command. Options on Unix like systems are often indicated by a hyphen and separated by a space.
Arguments
A command-line argument is an item of information passed to the command when run. Arguments are often used to identify sources of information, or alter the operation of the command.
Creating a command-line utility
Before I dive too deep into any implementation it’s worth pointing out that this tutorial has been written and tested on a unix-like operating system (OSX). Everything should work on other unix-like operating systems but this may require some extra research.
The only dependency to getting started is Node.js. You can run which node
from the command-line to find out if it’s already installed. If you already have node installed you should receive a similar response to below, else if the response is blank then node is not installed and you can download the installer from nodejs.org.
To start, create a new JavaScript file named gitsearch.js
and add a shebang to the first line of it. This informs the system which interpreter to use to run our file. In our case, we want to run the file with the node
interpreter (see this discussion on StackOverflow as to why we use /usr/bin/env node
instead of something like /usr/local/bin/node
).
Your script needs to be “executable” (so it can be run by the program loader). In order to make it executable, run chmod +x gitsearch.js
, which modifies your script’s access permissions so that the program loader can execute it.
Creating the command
The simplest way to create the command would be to call it’s file path and file name which would execute your script.
It is good practice with command-line utilities to make sure there are no other commands already in use on your system with the same name. This can be checked by using the which
command and the name of our command which commandName
. In this example we will be using the command gitsearch
, If which gitsearch
returns blank then the command is not in use.
Because this is a NodeJS script, we’re going to make this script installable using Node’s package manager, because that’s a normal thing to do if writing NodeJS scripts. Doing this will mean you can just type your script’s name, without having to worry about where you are (and where it is). Don’t worry if you don’t really know what npm is. Discussing package managers is way beyond the scope of this post - if you want to learn more about npm, read this excellent article: Package Managers: An Introductory Guide For The Uninitiated Front-End Developer
In order to make a NodeJS script installable via npm we need to create an accompanying package.json
file in the same directory as your gitsearch.js
script.
The important part here is "bin": {"gitsearch": "gitsearch.js"}
as this maps the gitsearch
command to your gitsearch.js
script. On the command-line, navigate to the directory which contains your files and install your script globally via npm (this might require using sudo).
The only draw back to this is after every change to gitsearch.js
you will need to re-run the npm install -g
command to see your changes reflected globally.
Now running the gitsearch
command will execute your script. To test this add a console.log("Hello World")
to your script, re-run npm install -g
and then run the command.
Options and Arguments
Command-line utilities are particularly useful when it comes to input and output operations. Options and arguments passed into a command can be accessed via process.argv
. Adding console.log(process.argv);
to your script and running your command with an option should return something like this:
One of Node’s most valuable features is its community of developers and the packages they have contributed. Often packages are lightweight and have been developed to perform a specific task. A great example of this is Commander, which is designed to help build command-line interfaces and provides great functionality for handling options and arguments.
On the command-line, install commander via npm npm install commander --save
(by adding the --save
option the npm
command will automatically update the dependencies in your package.json).
Now replace your script with the following:
Here we have used node’s require function to load the commander module into the script, and then started a basic structure using commander.
Commanders .args
contains just the arguments passed to a command similar to process.argv
so here we have used it to check arguments exist as this utility will require at least one argument to use as a keyword for the search.
Now running gitsearch
with an argument like gitsearch jquery
will output Keywords: jquery
(if you fail to pass an argument it will output the command’s help). Another benefit to using commander is auto generated help based on the information you provide for your options, which can also be run manually via gitsearch -h
.
Using the commands required argument we can build our GitHub search api endpoint.
As the GitHub api uses HTTP endpoints we will need to make a HTTP request. To help simplify this request we will use the Request package.
Now we can use request to make a GET
request to our base url created from the required arguments.
Note that Github’s api requires all requests to have a valid User-Agent
header which must be your username or your application name Github user-agent.
Now when you pass jquery
to your gitsearch
command it will return a json output of up to 100 repositories mentioning jquery and ordered by star count. This output contains a lot of data so to make it easier to scan we will use chalk to style our output.
For this example I have decided to loop over the response and pick out the repository name, owner, description and clone url, whilst using chalk to add the styling.
To help refine results we can also pass in more options and arguments. There are lots of options available via the Github api and I have chosen to limit by owner and language for now.
Now running the command gitsearch jquery -o jquery -l JavaScript
will return repositories that mention jquery, are owned by jquery, and use JavaScript.
Exit codes
It is important to make sure you exit your scripts correctly, once again this uses the process object. In the case of an error, the value of process.exit
must be greater than 0, whereas a successful exit should equal 0. Here I have added the exit code for the successful HTTP requests and errors. When using commanders .help()
an exit code is not needed as commanders handles the exiting for us.
Combining utilities
Finally I have added the --full
option which outputs the full response without any manipulation or styling.
This can be useful to take advantage of the other utilities available on the command-line such as grep, less and pbcopy. One easy way to combine utilties is to use the pipeline
to chain operations where the output for one command becomes the input for the next.
pbcopy
pbcopy
is an easy way to copy the output from a command. Piping the output to pbcopy
will then allow you to paste the output into other programs.
less
less
is a pager command which breaks content down into more manageable chunks, showing only one screens worth of content at a time.
grep
grep
is a utility for searching plain text data sets using regular expressions
Conclusion
Command-line utilities are great for simplifying tasks or automating repetitive operations and NodeJS can be a great stepping stone for developers looking to build commands without learning a shell script.
This was a basic example of how to get started with NodeJS on the command-line, if you want to view the full source could I have created a gist here. There are plenty of other resources available including lots of very useful tools already published on npm and GitHub.
Further reading
Find this post useful, or want to discuss some of the topics?
About The Authors
-
Glynn Phillips
Visit Glynn's personal website