Release 0.4

For my 4th release my bug was to change the slug function in Thimble in order for it to recognize unicode characters. Although the bug required only a few lines of code to complete, it did take me a while to finish because of all the problems I ran into while attempting to test it locally. I was hoping to complete another bug for this release as well, but the frequent errors I encountered really hindered my progress.

To complete this bug I had to first figure out what code to write/use in order to account for unicode characters in a URL. The uslug function that was recommended seemed ideal, so I stuck with using that. I was hoping to simply integrate the new function in the old one (slugify), but I was advised to test it first.

Attempting to run locally led me to a bunch of errors. This can be seen in one of my previous blog posts (https://elnushaj.wordpress.com/2013/11/24/slugs-and-thimble-testing/). Apparently, I had an outdated version of Thimble and when I tried to run it it failed. This was a little odd, since I made sure to update before I started working. I had to ask for some help on IRC to figure this out since I didn’t suspect it could be due to an outdated version.

After I solved that issue I ran into another. I could not publish a make locally and could not see the outcome of the uslug function. I tried to figure out what the problem was, but I again referred back to IRC for some help. After some discussion I was told that I was missing the MakeAPI server and had to set that up in order to publish a make locally. Like before, I did not think that a missing server was the problem since I was initially informed that I did not need it to test this bug.

I tried to set up the MakeAPI server by first installing its components. However, after several attempts of trying to get MongoDB and Elasticsearch running, I could not get them to start. This can be seen in my last blog post (https://elnushaj.wordpress.com/2013/12/02/local-publishing-issues-and-slug-testing/).

I decided to alternatively code a small node application in order to test the uslug function. I was provided a reference point (https://github.com/alicoding/simple-node-express) to do this and ended up modifying some files from it in order to test out the function.

I attempted to contact someone on IRC, to see if they were satisfied with the outcome, but I did not receive a response. I tried several times to no avail. I ended up just submitting the bug instead, so that I could double check with the reviewer before the bug fix was approved.

After review, I ended up adding a test section to thimble in order for people to test out external parts of the system. I put a file in the directory, for testing the uslug function, to demonstrate that the Unicode characters were handled as required. I did not bother creating multiple tests since I was not sure what qualified as acceptable results.

Git: https://github.com/mozilla/thimble.webmaker.org/pull/310/files

Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=903143

Local Publishing Issues and Slug Testing

Publishing Locally
As you can see from my last post (https://elnushaj.wordpress.com/2013/11/24/slugs-and-thimble-testing/) I was stuck with no options for publishing a local make. I attempted to solve the problem myself by looking through the webmaker.org files to see if the files were misplaced or outdated. I wasn’t really sure why I was getting the error since I updated everything the week prior. I asked for some help on IRC, but the only person that was able to respond was aali.  He wasn’t able to spot the problem when I asked him and had to run shortly after. I had no luck pinging anyone on Tuesday or Wednesday around 10-11:59PM, since they were too busy and it was fairly late.

Later, when I got a hold of Pomax, I was told that I additionally needed the MakeAPI in order to publish locally. When I had previously asked what servers I needed to publish a local make (in order to test the new slugify function), I was told only thimble, webmaker.org and login.webmaker. Regardless, attempting to set up MongoDB and Elasticsearch (for the MakeAPI) led me to even more errors.

I had run into this same problem in my first or second bug, in which attempting to run elasticsearch would give me a “command not found”, as seen below.

The same failure occurred when attempting to run mongoDB.

Pomax suggested that I should try to set up the webmaker suite in order to bypass the issue of setting up the server independently. I did not want to waste any more time trying to set up all these servers in order to test something so seemingly simple. Therefore, I decided to set up a small node application instead.

Slugify Testing

I wasn’t really sure where to begin to write my own node app. Luckily, aali gave me a great reference point: https://github.com/alicoding/simple-node-express.

In order to test the new slugify function I ended up changing the following:

• Added the dependency “uslug”: “1.0.3” to the package.json file
• Required in the function and then used it with a string.

I wasn’t sure what the proper syntax for getting output through the app.js was, but aali provided me with a reference on how to do that.

I tested the string seen below and got an output that supports unicode characters, unlike the previous version of the slugify function.

my title$_=日本語(笑)

results to…

my-title_日本語笑

I’m not 100% sure yet  if this is the intended string required by the function. I have tried notifying someone on IRC to confirm that this is correct, but have not heard a response yet. When I get confirmation, I will finalize the same code for Thimble and create a pull request.

Slugs and Thimble Testing

Bug 903143

For my third release I switched from error processing to do some URL work. The bug I chose can be seen here: https://bugzilla.mozilla.org/show_bug.cgi?id=903143.

The basis of the bug is that I need to modify the “slugify” function currently in Thimble to handle things like foreign characters (Unicode). Without it, Thimble currently drops Unicode characters from the URL and leads to 404 errors. I was recommended to utilize uslug (https://github.com/jeremys/uslug/)  in order to get this done. I was hoping that this bug would be quick and clean so that I could get through one more, but I’m running into some setbacks.

Updating Trouble

Before I started to work on my current bug I updated everything to match the mozilla repo. However, I was getting the following problem when I tried running locally.

I had made a silly mistake and had worked on my master branch before, which I should not have done. So in order to correct this first problem I had to follow a series of steps to fix my master branch and get it up to date with that from mozilla. I am posting the steps below in order to help others who have also made a similar mistake.

Note: I end up deleting the changes on my master branch by following these git commands. The contents of the master are first saved in a temp branch, in order to check if there is anything important. Other action needs to be taken accordingly if one does not want those changes removed.

So what I did was:

• git reset –hard
• git checkout master
• git checkout -b tempbackup
• git fetch mozilla
• git checkout -B master mozilla/master

I then had to do an npm install, and a bower install like I mentioned in my previous setup blog from before (in September).

Testing Locally with No Options

After I was all updated, I ended up making a few code changes (will detail when finished) to implement uslug. However, when I tried to test the new function locally I ended up getting no options on my make, when I tried to publish it. This is what the my screen looked like…

Looking at the terminal for webmaker I had this result…

I’m not really sure what a 500 error code, with a DNS error message, means in my case. Hopefully I can figure this out soon, since the work and assignments keep piling on for my last 2 weeks of classes and things are getting hectic.

Release 0.3

For my 3rd release my bug was to create/modify an error handler in order to prevent the node process from stopping on errors. In hindsight, the bug did not seem too difficult since I already had some experience previously working with error handling in Node.js. It took a while for me to code and try to test the bug though, since I wasn’t given much information on what actually needed to be done and had too look around the code to get an idea of what was required and how to start. Besides that, I also had to devote some of my work time to the FSOSS report this time around.

To complete this bug I ended up implementing the custom error function from popcorn maker. The function took in an error code and an error message. Since this did not exist in Thimble currently it was basically just stopping node and throwing an error with some error text to explain what happened. After I added in the function I had to sift through the code to see where these errors were being thrown. This ended up being in the middleware file. I had to replace all instances of an error being made to utilize this function. Although I wasn’t sure what error code to use (none was specified) it seemed the 500 code seemed to suffice. I had to then modify the error handler I wrote from the previous bug to account for responses as text or JSON objects besides just an HTML page. Again, I looked through popcorn maker and implemented the same formatting as its own function and then integrated it in with the handler. Due to the 404 error implementation from last time there was no need to change anything in the app.js file, the error handler was already called there.

Git: https://github.com/elnushaj/thimble.webmaker.org/compare/BugFix884342

Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=884342#c8

Working Towards 3rd Release

FSOSS Paper Completion

As stated from last week I was able to finish up my FSOSS event report this week. If interested, here is the link. 

Bug 884342 Dependence 

I have not been as active on the newsletter or IRC for this bug since I tried to work a little more independently than my previous two bugs. For my previous two releases I felt that I asked a lot of questions on IRC, some of which I could potentially find the answer to myself if I looked hard enough. I didn’t really research too much since I felt that I could find the answer I was looking for from other developers much quicker than I could find it myself. Although this was useful for me to complete my bug on time I felt a little too dependant on others to finish my work.

Grasping the Bigger Picture

For this bug I wasn’t given too much of a starting point. I asked for more information as I normally do. However, as I mentioned last week, many of the developers were busy or unavailable. So I took it upon myself to figure out what needed to be done. Since I worked with error handling on the last bug and got more acquainted with the syntax of Node.js I felt more confident that I could figure out how to complete this bug alone. I tried to only ask questions if I wasn’t certain on how some things needed to be implemented. I managed to connect the dots though, once I looked around the code and got some implementation aspects figured out.

Figured Out

I still need to check with one of the developers if this is indeed what is needed and also to test it and re-factor it if necessary. Regardless, the following is some of what I ended up doing:

-I knew that a function was needed to make custom errors to catch in order to prevent unknown errors from happening and stopping the code. I was provided with sample code from bugzilla showing how popcorn maker implemented the error function, so I implemented the same function for thimble. E.g. https://github.com/mozilla/popcorn.webmaker.org/blob/master/lib/utilities.js#L34-L38

-I didn’t know where to find all the errors that were being created in thimble and needed to be replaced with this function. After searching and getting some direction I found that most of the errors were located in the middleware file. E.g. https://github.com/mozilla/thimble.webmaker.org/blob/master/lib/middleware.js#L52

-I ended up changing all the instances of error creation to utilize the new function. However, I am still unsure what type of error code needs to be passed since the function takes in two parameters, the message and the code.

– I was informed that I needed to account for different types of formatted outputs as well. So I still need to figure out what types of formats to output at what times (based off a variable).

Progressing and the Future

I seem to be on pace with this bug and should be done by this week. It doesn’t seem as painstaking as the ones before. This is probably because I have a little more experience with error handling, navigation and Node.js this time around. I also managed to get the big picture/scope of the bug earlier on, unlike the other bugs. The initial research prior to the start of the bug was very helpful and it changed the way that I will approach projects in the future.

FSOSS and Handling More Errors

I did not make a blog post last week since I was playing catch up with my other courses during the study week. This weeks post will be fairly short since most of my work has been on the FSOSS report.

Free Software and Open Source Symposium
This past week I began my write up on the FSOSS event I attended on October 25. I should have it completed either Monday or Tuesday of this week. The event was pretty interesting and covered many different types of topics. Two of the presentations I found very educational where that for MongoDB and ARM technologies. Since I extensively cover the details of the presentations in my report I will leave it at that and post a link here when it is competed. FSOSS report.

Bug 884342

While working on my report I selected a new bug to work on for my 3rd release. The specifics of the bug can be found here https://bugzilla.mozilla.org/show_bug.cgi?id=884342. What I basically need to do is create an error handler for Thimble to prevent it from stopping the node process when an error is met. Since I already created an error handler for my previous bug (the 404 error page), I need to expand on it/ revise it to encompass more than just a 404 error.

I was provided with some reference material from popcorn maker’s method for handling errors, but I am unsure of what errors I need to be checking for in Thimble and what format the responses need to be in. I also need to figure out how to alter my original error handler to deal with the different types of responses that might be needed (e.g. strings) without breaking its current function. I asked for some more details on Bugzilla and contacted one of the engineers on IRC for more details. However it seems that a few are on break after the big Mozfest event and won’t be back till later this week. Hopefully then I can get some good progress on this bug.

Release 0.2

My bug for release 0.2 was to create a custom 404 page for Thimble. This bug wasn’t too bad, but unfortunately I lost a lot of time setting up some of the environment I thought I needed for it. You can see this on my previous blog post here: https://elnushaj.wordpress.com/2013/10/14/errors-setting-up-servers-to-run-locally/.

In order to complete this bug I ended up implementing the 404 page the same as that for make-valet. I did this to maintain consistency across webmaker tools. To complete the bug I ended up transferring the code for error.html from make-valet into the thimble and changing reference paths to the correct ones. I also transferred over the necessary image needed by error.html, and also added the error.less file used for the CSS by error.html. Since the fonts from fonts.less (make-valet) did not exist in thimble I used the webfonts file instead. This was necessary since error.less imported fonts for its css.

In thimble’s app.js file I then included the same methods for the error handler as make-valet as well as the respective definitions in the middleware file. Since the error handler uses a custom filter with nunjucks I also had to add the code from that to thimble’s app.js.

The final result looks like this: http://imgur.com/MdWHHy2

Git: https://github.com/mozilla/thimble.webmaker.org/pull/257/files

Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=884351

Errors Setting Up Servers to Run Locally

Picking Up From Last Time

As you can see from my last blog post I was having difficulty setting up one of the servers (make-valet) from last week https://elnushaj.wordpress.com/2013/10/07/make-api-and-thimble-404-error-page/. I tried several different ways to fix the problem, but all of them ended up failing. Two of the methods I tried included, going to the specific location of the post install and trying to install manually. Another was changing the post install string to “.\\nodes_modules\\.bin\\bower install” in hopes that Windows could not interpret certain Linux characters properly. In the end I ended up setting up a virtual machine with the Ubuntu OS in order to avoid any future problems with windows compatibility.

Setting Up Again and Running into Problems

After readying my virtual machine (VirtualBox) I found out from one of the developers on IRC (aali) that I didn’t need make-valet server running in order to test a 404 page for Thimble. The servers I required where Thimble, Webmaker, Webmaker Login and Make API. So just like for my first bug I was forced to set up Thimble again, as well as three other servers.

Setting up Thimble in a Linux terminal was interesting and confusing at times. While setting up I ran into several errors. Three of these were based off of each other and caused me to lose a good amount of development time. The error I had was that I could not run the command “npm install -g grunt-cli”  since I did not have grunt. I could not get grunt and then grunt-cli since I did not have the latest version of Node.js.

Errors due to grunt, grunt-cli and node version

    

Several of the commands I entered to update node seemed to have been accepted correctly, but the version did not change. For my Ubuntu 13.04 OS version, following the instruction on the linked site seemed to do the trick, https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager#ubuntu-mint-elementary-os.

Solution

The instructions were as follows:

sudo apt-get install nodejs
sudo apt-get update
sudo apt-get install python-software-properties python g++ make
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

After that installing grunt and grunt-cli worked properly. The commands for installing grunt and grunt-cli on terminal were as follows:

npm install -g grunt
npm install -g grunt-cli

Although everything was set up properly, running the Thimble server lead to an even stranger problem (image below).

Object error when attempting to run the server

Solution

Speaking to a few open source engineers on IRC (credit to Humph and thanks to Pomax for his help), the way to solve this problem is as follows:

• First make sure that you have the latest code for thimble since that alone could solve the problem, otherwise continue
• Go into the thimble.webmaker folder
• Go into node_modules folder
• Open the mox-server.js file
• Replace line 52 of the file with the following          app.listen(12319, function() {

Local page is stuck on an endless load 

I no longer had a problem with the server running, but the local page was now stuck on an endless load.

Solution 

Credit for this solution goes to Vader (IRC) and his blog post, http://bzcareermongodb.blogspot.ca/2013/09/cant-run-thimble-loading-forever.html

For ease of reference here are the commands to run when in thimble.webmaker.org.

Note that they may vary based of root privileges.

npm install -g bower
bower install

Stuck on Setup

As you can see above all of these errors are preventing me from getting any work done and just like my first release I’m wasting more time setting up then actually doing some productive work.

I am currently stuck trying to run the MakeAPI server and get the following failure message when I attempt to run the mongod command before launching the server.

Similar to last week if I find a solution I will either update my post or feature it in my next blog post to help any others facing similar problems.

Make API and Thimble 404 Error Page

For my second bug I decided to further my work on Thimble since I already got Node.js up and running in the previous bug. The bug I am currently working on is changing the 404 page for Thimble into a more interactive page rather than a simple error string, Bug 884351.

Research
I was looking around the internet to see the different ways that error handling was implemented in the Express API for Node.js so that I could more easily locate the code I needed to alter and update.  Most of the code I found was structured as follows:

app.use(function(req, res, next){
    res.status(404).render('404_error_template', {title: "Sorry, page not found"});
});
source

The problem with the following function though is that it results in a string output rather than a reference link to a new page. So in my case, this doesn’t help. Looking at the Express API reference site, I saw a couple of ways to potentially use the above code in combination with other functions in order to link to a page (maybe directory() under Middleware).

Getting Started
When I was ready to begin I was shown the location in the code where to start work from, but I didn’t see anything familiar from what I had researched previously. In fact the code wasn’t even in the Thimble files, it was in one called Make. The way in which page redirection was being handled in this code was via a a function called app.use(lessMiddleware(…)). I found out that  Middleware is basically just software that allows two applications to communicate to one another. For example, middleware allows for a web application to access data from a database application upon user request and return the response for use. The line in the function of the code highlighted was “compress”. Looking at the Express API I realised that the programmers intended on using the compressed CSS and HTML and likely returning that (sort of like a string)  to display the error page.

Errors
In order for me to start changing some code I needed to download the Make API from the make-valet in a git repo. Unfortunately as I began the installation and set up process I quickly ran into a “weird” error (last 2 lines in the image below).

I looked on IRC and online to find possible solutions to this, but came up short. I tried going to the specific path to install bower, but it didn’t seem to work. I’m currently trying to resolve this problem and will update this post when I find a potential solution.  I will also include any other steps to complete the installation and set up for any of those with the same issues.

Release 0.1

In order to fix Bug 897281 (new Thimble favicon) I had to setup Node.js and understand parts of the source code of Thimble to modify and fix the problem. I then had to go through the process of  placing my work on Git  and creating a pull request as well as submitting my changes for review on Bugzilla.

I documented my work for setting up Node.js in my previous blog post here: https://elnushaj.wordpress.com/2013/09/23/setting-up-node-js-for-windows-7-and-getting-started-on-thimble-ui-bug-fixes/

After I got Node.js up and running I ran a grep command for the string “favicon”, in order to locate what document and line number the change needed to be made on. The result was the file app.js and the following line: app.use(express.favicon()). Since I was not familiar with the way Node.js applications were written, I assumed that express and app where custom classes with custom functions. I tried looking around the source code to find the definitions, but could not locate them. The fact that a default favicon was appearing without a path defined was also confusing for me.

I had an idea that the favicon function could have been predefined because of the lack of a stated path. Instead of researching for more information I ended up asking for help on IRC, which was a mistake in hindsight since the information could have been found in google with little effort (just wasn’t sure what to google at first). Luckily for me my professor , David Humphrey, provided me with more than sufficient information to resolve my bug. I used the following 2 links to change the favicon: http://www.senchalabs.org/connect/middleware-favicon.html , http://stackoverflow.com/questions/15463199/how-to-set-custom-favicon-in-node-js-express.

I ended up adding __dirname + image path to fix the bug. The result basically tells the source that from the current location of the app.js file got to the provided location and find the specified .ico file.

Below I include the results of the changes I made.

Git Hub:  https://github.com/elnushaj/thimble.webmaker.org/commit/edbaa3375804f22c1b4e77c97505d62702a70089

Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=897281