About a week before we launched WP Migrate DB Pro 1.6, I had my heart broken by Ashley Rich in our slack channel. It went something like this:

/——dramatization——/ @bradt: Is 1.6 ready to launch? @jrgould: Yep! We are ready to go! Absolutely nothing could go wrong. @a5hleyrich: Hey @jrgould, new UI looks great but it crashes when I have a lot of media @jrgould: What? No. I’ve tested with like, 1000 media files. Get a new laptop. @a5hleyrich: I have 10k attachments and it’s locking up chrome. @jrgould: No… @a5hleyrich: Sorry, mate. @jrgould: 😢

Finding the Leak

What came next was quite a bit of speculation on everybody’s part as to what was causing browsers to slow to a crawl and often lock up when very large amounts of media files were being migrated. Clearly it’s a memory issue, but what’s causing it?

First, some background. You may have seen WP Migrate DB Pro’s new UI, but you probably haven’t gone through the code. I won’t put you through that, but for those who are interested, here’s an early “sketch” that I put together on CodePen when I was first starting to work on it:

See the Pen Backbone by JRGould (@JRGould) on CodePen.

The migration progress UI is built using Backbone models and views. Every progress bar was represented by a view and a model. Each “stage” in a migration (backup, migrate, media) is also represented by a Backbone view and model that contains and manages all of the individual progress bars. Finally, the migration as a whole is represented by a single Backbone view and model that contains and manages the stages. We’re not using Backbone to actually run the migration, so the models are being treated more like view-models and we aren’t using collections or any of Backbone’s more app-focused features like the Router or Sync modules.

The new UI shows a progress bar for each media file that we migrate which includes thumbnails, so with 10k media files that’s 30-50k progress bars that are all added to the DOM. Thinking that had to be what was using so much memory and slowing things down, I dove into the code and refactored the way my views displayed progress bars so that only a few hundred nodes would be added to the DOM at any time, the rest would get removed from the DOM and stashed away so that they could be queried. To my surprise and dismay, this did not help very much.

The next step was to blame Backbone itself. I used it because it’s baked into the WordPress admin, but it’s old and probably stupid and just using too much memory because of its geriatric stupidity.

So I rewrote Backbone.

OK, maybe I didn’t rewrite Backbone. But I did write a replacement for my implementation of the Backbone model and view – which wrapped them up into one object. This new model prototype allowed me to reuse quite a bit of the code that I’d written to extend the Backbone model and view including some basic methods like set, get, on, off, and trigger, it looked something like this:

myModel = function( userProps ) {
    var ret = {
        attributes: {
        },
        events: {},
        get: function( prop ) {
            return this.attributes[ prop ];
        },
        set: function( prop, val ) {
            this.trigger('change', prop, this.attributes[prop], val) ;
            this.trigger('change:' + prop, prop, this.attributes[prop], val) ;
            return this.attributes[ prop ] = val;
        },
        on: function( event, callback ) {
            if ( ! this.events[ event ] ) {
                this.events[ event ] = [];
            }
            this.events[ event ].push( callback )
        },
        off: function( event, callback ) {
            if ( ! this.events[ event ] ) {
                return
            }
            var callbacks = this.events[ event ];
            if ( 'function' === typeof callback ) {
                var index = callbacks.indexOf( callback );
                if ( -1 !== index ) { 
                    callbacks.splice( index, 1);
                    this.events[ event ] = callbacks;
                    return;
                }
            } else {
                this.events[ event ] = []
                return;
            }
        },
        trigger: function( event ) {
            var self = this;
            var args = Array.prototype.slice.apply( arguments, [1] );
            if( this.events[ event ] && this.events[ event ].length ) {
                var eventCallbacks = this.events[ event ].slice();
                while ( eventCallbacks.length ) {
                    var ev = eventCallbacks.shift();
                    ev.apply(self, args);
                }
            }
        },
    };
    if ( 'object' === typeof userProps ) {
        _.each( userProps, function( prop, index ){
            ret[index] = prop;
        }, this );
    }
    return ret;
};

That didn’t work either. This actually seemed to run slower than it did when using the Backbone model and view, so I just scrapped that idea and continued looking for the culprit.

This is Garbage

Since the amount of DOM nodes being rendered didn’t seem to have a huge effect, and Backbone didn’t seem to be bloating memory usage, I started to do some research on what actually causes memory leaks in JavaScript. I read through some articles written by smarter developers than myself and learned quite a bit about something called garbage collection.

Modern JavaScript engines like Chrome V8 are highly optimized and can handle much more than their predecessors (we’ve all seen the crazy particle demos). One of the tricks that they employ, though this one isn’t very new, is called garbage collection, which is a feature of the JS engine that tries to release as much memory as it can from stuff like variables and functions that were used at one point but aren’t being used anymore.

So what does this have to do with Backbone? Callbacks prevent their parent objects from being garbage collected. This is because even though a callback is called outside of the parent object, it still keeps its context. Consider the following:

var myView = Backbone.View.extend( {
    init: function() {
        this.model.on( 'change', this.render, this );
    },
    render: function() {
        /* render the view */
    }
} );  

Once that view is initialized, it registers a callback to the model’s change event, the whole object is excluded from garbage collection until the model that’s carrying the callback is able to be garbage collected. In most cases, this is fine, but what if you have 50,000 views that are bound to 50,000 models that are in turn all bound to the stage model? That’s going to cause some issues…

JavaScript: The Handy Parts

It was really nice to have each progress bar represented by its own model and view objects. This allowed each component of the migration progress UI to just sort of manage itself while everything else would react appropriately to changes made to a lower-level component. Removing those objects means that the stage model and view now have to inherit the responsibilities of the progress bar model and view respectively. It shouldn’t be too much to ask JavaScript to deal with an array of 50,000 simple objects that don’t contain any methods, but how do you keep track of changes to those objects?

Did you know that in JS, any variable that contains an object such as an Array or Object is passed by reference rather than value? Here’s an example:

function changeThings( var1, var2 ) {
    var1 = 12 * 2;
    var2.foo = 'changed';
    var2 = '';
    console.log( var1, var2 );
}

function logThings( var1, var2 ){
    console.log( var1, var2 );
}

var myVal = 12;
var myObj = { foo: 'bar', baz: 'bing' }

logThings( myVal, myObj );    // 12, Object {foo: "bar", baz: "bing"}
changeThings( myVal, myObj ); // 24, ""
logThings( myVal, myObj );    // 12, Object {foo: "changed", baz: "bing"}

This allows us to do some really nice things that don’t have a huge memory cost. After the refactor I ended up with an array containing progress bar objects, it looks something like this:

items: [
    { name: 'image001.jpg', size: 1024, transferred: 0 },
    { name: 'image002.jpg', size: 1288, transferred: 0 },
    ...
]

We also have a “lookup” object that allows me to access objects in that array by name without sacrificing the Array functionality:

itemsLookup: {
    'image001.jpg': 0, 
    'image002.jpg' 1,
    ... 
}

Now we can utilize the fact that JavaScript passes objects by reference to simplify the way our model keeps track of the state of objects. For example, here’s what it might look like to add a progress bar to the stage model:

addItem: function( itemObj ) {
    var item = _.clone( itemObj ); // Break ref to original obj
    this.get( 'items' ).push( item );
    this.get( 'itemsLookup' )[ item.name ] = this.get( 'items' ).length - 1;
    this.trigger( 'item:added', item );
}

Even though item is scoped to the addItem method (thanks to the var keyword) we can pass it directly to any callbacks that bind to the item:added event because item really just holds a pointer to the cloned object. This will allow any callbacks that bind to this event not only access to read the object, but they can modify it directly without having to get the whole items array or use the itemsLookup object to access the item within the array. This is really useful for doing stuff in the view like creating the actual DOM element:

// stageView.init() { ...
this.model.on( 'item:added', function( item ) {
    if ( !item.$el ) {
        item.$el = $( '<div />' ).addClass( 'progressBar' ).css( 'width', item.transferred / item.size + '%' );
    }
}, this );

Another interesting aspect of object pointers in JavaScript is that a single object can be contained by any number of other objects. This is useful for keeping track of things without needing to do much in the way of filtering or sorting. For example, when a progress bar is marked as complete, we’ll remove it’s element from the DOM and put it in a queue just in case we need to show it later. This is how something like that is accomplished:

// stageView.init() { ...
this.model.on( 'item:completed', function( item ) {
    item.$el.remove();
    this.itemQueue.push( item.$el );
    if ( 100 > this.$el.find( '.progressBar' ).length ) {
        var $elToShow = this.itemQueue.shift();
        this.$el.append( $elToShow );
    }
}, this );

Here, the DOM acts like the queue of “visible” elements, and itemQueue acts like a queue of hidden items. When an item completes, it’s removed from the DOM and added to the end of itemQueue while a previously hidden item is removed from the beginning of the queue and added back into the DOM if there are less than 100 visible. Each progress bar element is contained simultaneously in an object in the stage model’s items array as well as either the DOM itself or the stage view’s itemQueue array.

This is handy in the sense that it makes this sort of thing easier than it seems it should be, and it’s much more memory efficient than it would be if we had to make copies of the object or iterate over the DOM or items array to access the various moving parts.

Wrapping Up

For a seasoned programmer, garbage collection and passing variables by reference shouldn’t be new concepts. But JavaScript is often a different beast than we expect it to be, and we sometimes forget that it’s a real, actual, honest to the Flying Spaghetti Monster, programming language. Knowing about some of the lower-level intricacies and higher-level idiosyncrasies can really make or break our applications when we put JavaScript through its paces, so I’d like to know: What are your favorite (or most hated) “features” of JavaScript? What did you learn the last time your JavaScript started crashing Chrome? Let us know in the comments!