Going through a few tutorials that showed how to make a Node app that from my terminal will show me the number of downloads I have had, as well as doing the math on how long raspberryCandy has been released for.
I went through this article from scotch.io which used NodeJS, Cheerio and request-promise.
- request-promise - Makes the HTTP request which supports promises. The extension of the standard request promise.
- Cheerio - Helps to traverse the DOM letting us select parts of the page we would like to extract.
Figlet is a great package for making ASCII art from text and has a library of different ways to do so. Chalk is used to colour the text of the response in the terminal which means I could keep the raspberryCandy colours being returned.
Following that I made a function which calculates how many days raspberryCandy has been out. The function works in two steps, firslty it subtracts the release date from the current date which then gives us the amount of time bwtween the dates but in Milliseconds (which isn't the most readable).
Step two of the function takes the result in milliseconds and divides it by the math that converts milliseconds to days. This is wrapped in a Math.round() function to give whole days as a return without a decimal.
Now for the main party trick, scraping the marketplace page for raspberryCandy to get the number of downloads. This is where we use the request-promise package and give it the target url which in this case is the page on the Visual Studio Marketplace.
Inspecting the page prior to building this I found that the piece I was after was a span with the class name 'downloads-text'. Using Cheerio I can target the span with the right class name from the HTML document and then return it as plain text. Below I have laid out the basic structure of what it is doing.
Now that we have the heavy lifting out of the way the structure comes into play. In my example I have wrapped each of the console.log() returns with the Chalk package allowing me to set the colour, in this case I am going with the Purple and Aqua colours of raspberryCandy.
The other extra piece is using Figlet which gives me the ASCII art, this wraps around the Cherrio function and takes the properties of what font to use, as well as the horizontal and vertical layouts.
Now that it's all pieced together the last thing left to do was to link it to my terminal as a single command rather than having to go to the file system and run the JS file. To do this, we add a line to the top of the file which will let us add our command to the package.json.
This line will let it be triggered as a Node command and run. This will also let the next part we add to the package.json work. Inside the package.json I have added:
Final step is to run npm link in the terminal while in the folder. This takes the bin command, in this case "raspberry" and then launches the command "./raspberryScraper.js". This is similar to any short hand command like npm run start or gatsby develop.