Developer Notes: Refactoring

Last year on my previous job, I was handed over a major task to orchestrate refactoring the front-end application for our subscribers that use internet to send and receive fax. The coverage area was exclusive for US region. Here I would like to share this successful experience to fellow friends especially to developers and project managers.

The Scenario

The previous front-end code modules and UI were developed using Telerik's Kendo UI framework and mixed with cluttered jQuery codes that cover the business logic and Ajax request with Asp.Net MVC backend. The backend server was depended upon some services for payment gateway (Stripe) and fax services. The online subscribers are about 200k plus with some companies that subscribed to also used simple API developed under PHP. I had been asked to lead a team to refactor the front-end application due to many issues complained by the subscribers that the application behaviours were sometimes unstable. However, the whole back-end architecture was quite sound adopting layers for separation of concerns with IoC (DryIoC) and OrM (Dapper) that tied to MSSQL database.

The Problem

As the UX designers were pushing for custom CSS for many sections even under Bootstrap CSS framework, while the front end application was, in fact, a server-side view tightly under Asp .NET MVC. It is a mixed of Razor, jQuery, and Kendo UI codes. The existing JavaScript codes were contaminated with global variables and jQuery functions created for HTTP requests were not well covered with Promise. This had created certain intermittent in Ajax request chains due to race conditions.

The previous development also manipulated DOM extensively using JavaScript and there was no templating. It was also a critical technology decision to make whether to proceed with new ReactJs or Angular framework that looks promising. But the front-end team members are more comfortable with jQuery and Bootstrap. More than 500 support calls per day meant that the application was indeed in a big trouble.

The Solution

After reviewing the front-end and backend codes and their relationship, I saw the problem could also be tackled from different architectural perspectives. On the other hand, I made the decision that we should go with our own custom framework employing jQuery, module pattern leveraging IIFE, employing jQuery deferred for Promise, using Handlebars to reduce coupling of UI with JavaScript modules. ReactJs or Angular was not an option as we are facing stringent project time frame.  The following shows one of the modules created using module-pattern with IIFE (Immediately Invoked Function Expression). Please note that this codes are constructed under JavaScript ES5 and X_CONFIG and X_HTTP are custom modules that were developed with similar pattern that were not shown here:

var ofxViewService = (function (window, document, $, X_CONFIG, X_HTTP, Handlebars, slidePanel, undefined) {

    var baseModule = {};
    var settingMenuItemId = X_CONFIG.pageSettings.settingMenuItemId;
    var pageContainerId = X_CONFIG.pageSettings.pageContainerId;
    var bread_crumb = $(X_CONFIG.pageSettings.breadCrumb);

    //loadPartialView

    baseModule.init = function () {
    };

    baseModule.loadTemplates = {
        /*  
        need to define in web.config for asp.net
        <system.webServer>
            ...
        <staticContent>
          <mimeMap fileExtension=".hbs" mimeType="text/x-handlebars-template" />
          <mimeMap fileExtension=".handlebars" mimeType="text/x-handlebars-template" />
        </staticContent>
            ...
      </system.webServer>
        */
        handleBars: function (templateUrl, templateName, callback) {
            if (Handlebars.templates === undefined || Handlebars.templates[templateName] === undefined) {
                $.ajax({
                    url: templateUrl + '/' + templateName + '.hbs', //or .handlebarsasync: true
                })
                .done(function (data) {
                    if (data === undefined) {
                        //console.error('handlebar template of ' + templateName + ' is undefined');return callback({});
                    } else {
                        //Handlebars.templates[templateName] = Handlebars.compile(data);//console.info('handlebar data of ' + templateName + ' is:' + data);return callback(data);
                    }
                   
                })
                .fail(function () {
                    //console.error('fail to get the handlebar template of ' + templateName);return callback({});
                });
            } else {
                //if precompiled using handleBars compiler with nodejsreturn Handlebars.templates[templateName];
            }
            
        },
        mustacheJS: function () { },
        html: function (htmlUrl, templateName, callback) {

            $.ajax({
                url: htmlUrl + '/' + templateName + '.html',
                async: true

            })
            .done(function (data) {
                if (data === undefined) {
                    //console.error('Html resource of ' + templateName + ' is undefined');return callback({});
                } else {
                    ////console.info('htmlview of ' + loadUrl + ' is:' + data);return callback(data);
                }

            })
            .fail(function (xhr, status, thrownError) {
                //console.error('fail to get the  Html resource of ' + templateName);
            });


        }
    }

    baseModule.loadView = function (controller, action, reqType, callback) {

        if (controller != "Settings") {
            $(settingMenuItemId).removeClass("active");
            $(settingMenuItemId).removeClass("open");
            $(settingMenuItemId + " .site-menu-item").removeClass("active");
        }

        var container = $(pageContainerId);
        var viewUrl = X_CONFIG.mvcRouteUriPattern(controller, action);

        $.ajax({
            url: viewUrl,
            cache: true,
            type: reqType,
            processData: false,
            contentType: false
        })
        .done(function (returnData) {
            container.css({
                opacity: '0.0'
            }).html(returnData).delay(50).animate({
                opacity: '1.0'
            }, 150);

            if (callback) {
                callback();
            }

        })
        .fail(function (req, status, ex) {
            //console.error('Error loading partial view ' + ex);
        });


    };

    baseModule.loadHtml = function (loadUrl, container, callback) {

        var container = $(container);

        $.ajax({

            type: "GET",
            url: loadUrl,
            async: true,
            dataType: 'html',
            cache: true, // (warning: setting it to false will cause a timestamp and will call the request twice)
            beforeSend: function () {
                container.html('');
            }
        })
        .done(function (data) {
            // dump data to container
            container.css({
                opacity: '0.0'
            }).html(data).delay(50).animate({
                opacity: '1.0'
            }, 150);

            if (callback) {
                callback();
            }

            //
        })
        .fail(function (xhr, status, thrownError) {
            //console.error('error loading Html resource');
        });


    };

    baseModule.loadDialogBox = function (url, container) {
        $.ajax({
            type: "GET",
            url: url,
            async: true,
            dataType: 'html',
            cache: true, // (warning: setting it to false will cause a timestamp and will call the request twice)
        }).done(function (html) {
            container.html(html);
        }).fail(function (xhr, status, thrownError) { //jqXHR, textStatus, errorThrown
            container.html('<h4 style="margin-top:10px; display:block; text-align:left"><i class="fa fa-warning txt-color-orangeDark"></i> Error 404! Page not found.</h4>');
            //console.error('error loading dialog box');
        });
    };

    baseModule.drawBreadCrumb = function (options) {
        var a = $("nav li.active > a"),
             b = a.length;

        bread_crumb.empty(),
        bread_crumb.append($("<li>Home</li>")), a.each(function () {
            bread_crumb.append($("<li></li>").html($.trim($(this).clone().children(".badge").remove().end().text()))), --b || (document.title = bread_crumb.find("li:last-child").text())
        });

        // Push breadcrumb manually -> drawBreadCrumb(["Users", "John Doe"]);// Credits: Philip Whitt | [email protected] (options != undefined) {
            $.each(options, function (index, value) {
                bread_crumb.append($("<li></li>").html(value));
                document.title = bread_crumb.find("li:last-child").text();
            });
        }
    };

    baseModule.attachScript = function (url, callback) {
        //var head = document.getElementsByTagName('head')[0];//prefer to use bodyvar body = document.getElementsByTagName('body')[0];

        var script = document.createElement('script');
        script.src = url;

        var done = false;
        // Attach handlers for all browsers
        script.onload = script.onreadystatechange = function () {
            if (!done && (!this.readyState ||
                    this.readyState == 'loaded' || this.readyState == 'complete')) {
                done = true;
                if (callback)
                    callback();

                // Handle memory leak in IE
                script.onload = script.onreadystatechange = null;
            }
        };

        //use body instaed of head
        body.appendChild(script);

        // We handle everything using the script element injectionreturn undefined;
    };

    baseModule.attachCss = function (url) {

        var head = document.getElementsByTagName('head')[0];
        var Css = document.createElement('link');

        var done = false;
        // Attach handlers for all browsers
        Css.onload = Css.onreadystatechange = function () {
            if (!done && (!this.readyState ||
                    this.readyState == 'loaded' || this.readyState == 'complete')) {
                done = true;
                // Handle memory leak in IE
                Css.onload = Css.onreadystatechange = null;
            }
        };
        Css.rel = "stylesheet";
        Css.type = "text/css";
        Css.href = url;
        head.appendChild(Css);
        //Format: <link href="/Assets/css/main.css" rel="stylesheet" />return undefined;
    };

    baseModule.slidePanel = {
        slidePanelHandleBarsBasic: function (contentTemplate, direction) {
            $.slidePanel.show({
                content: contentTemplate //html
            }, {
                direction: direction //'right'
            });

        },
        slidePanelHandleBarsAjax: function (dataUrl, dataType, direction) {
            $.slidePanel.show({
                url: dataUrl, //'ajax.json',
                dataType: dataType //'json'
            }, {
                contentFilter: function (data) {
                    return template(data);
                },
                loading: {
                    template: function (options) {
                        return '<div class="' + options.classes.loading + '"><div class="spinner"></div></div>';
                    }
                },
                direction: direction //'right'
            });

        },
        slidePanelHide: function () {
            $.slidePanel.hide();
        }
    };

    baseModule.hideAndShow = {
        UIMenu: {
            hide: function (menuId) {
                $(menuId).hide();
            },
            show: function (menuId) {
                $(menuId).show();
            }
        },
        UISection: {
            hide: function (sectionId) {
                $(sectionId).hide();
            },
            show: function (sectionId) {
                $(sectionId).show();
            }
        },
        UIControl: {
            hide: function (controlId) {
                $(controlId).hide();
            },
            show: function (controlId) {
                $(controlId).show();
            }
        }

    };

    baseModule.containerHeightSizing = function (viewContainer) {
        var containerHeight = $(window).height();
        containerHeight = containerHeight / 1.71;
        if (containerHeight < 320) {
            viewContainer.css('height', 320 + 'px');
        } else {
            viewContainer.css('height', containerHeight + 'px');
        }
    };


    return baseModule;

}(window, document, jQuery, ofxConfig, ofxHttpService, Handlebars, $.slidePanel));

This module actually facilitates other modules or widget on attaching JavaScript, CSS, and layout plugins such as a slide panel to the master page. It also allow for templates loading from different kind of code templates (Handlebars, Mustache, or pure HTML). The developer could choose any type of templates that consistent with jQuery or vanilla JavaScript.

On the architectural decision, I had decided to transform the existing presentation layer of MVC into a Clean Architecture using the new API that used JWT and oAuth2 (Auth and Resource server), eliminate the razor scripting and separate the front-end from any server-side dependency. The code refactoring exercise for API back-end was quite fast as we only refactor the existing controllers into the API structure with only minor changes in business and data access layer.

We actually ended up with 2 separate APIs; Internal API that can be accessed by JWT based on username-password authentication and external API that JWT is based on client id and secret key. This external API will be exclusive to companies that wanted to develop their own front-end applications. We developed a separate dashboard that mimics the PayPal API registration for client applications. As for our own front-end application, we separated codes from UI using modules, templates, and widgets, we had ensured the maintainability of the application. It was a success refactoring exercise and we could solve more than 200 bugs within the first month of our release candidate and reduce daily support calls to below 50 calls per day. 

Lesson Learned

1. Consider using alternative perspectives when examining the solution for certain requirements
2. Consider the learning curves of the team members when introducing new technology and leveraging the expertise to the highest potential.
3. Use SOLID principle for application maintainability, scalability, and modularity.
4. Ensure quality through functional testing (unit/ integration tests), automated end-to-end testing (eg. using Selenium)
5. Ensure UX requirements are consistent through manual acceptance testing before production.
6. Work closely with UX designer and project stakeholders for any issues. 
7. Measure and notify any impact (cost and time) due to new requirement(s).
8. Technology decision is critical to the success of the project. Wisely look into every angle of possibilities in term of costs, time, expertise, support, maturity, and future commitments when selecting for any proprietary or open source plugins, templates, or libraries.
9. Make formal or informal celebrations with team members to boost your team spirits.

Hope this article could give certain insights to fellow friends and those who are interested especially when making decision for application refactoring exercise.