Home > Performance > Speeding up SmugMug Search

Speeding up SmugMug Search

SmugMug users have uploaded millions of awesome photos, and one of the things we work hard on is making it easy and fun for people to discover them. SmugMug Search is an important part of this, since it allows anyone on the web to search among all public photos on SmugMug. It also helps drive traffic to our Pros, many of whom make a living selling prints of their work.

Naturally we want Search to be fast, intuitive, and beautiful. But more importantly, we want to showcase our users’ gorgeous photos. When people search for photos on SmugMug, they want to see photos, not a bunch of pagination links and other user interface clutter. So, a few months ago, we launched a redesigned Search page that does just that.

SmugMug search results for "sunset"

We put those big gorgeous photos front and center, and we got rid of the ugly pagination links in favor of infinite scrolling—as you scroll down, more results are loaded automatically. This looks great and works well, but keeping the interface fast and responsive presented a lot of challenges, especially on older browsers or slower computers.

Performance Problems

The first issue we faced is that executing JavaScript as the user scrolls—which is necessary in order to implement infinite scrolling—can really bog things down if you’re not careful, especially when the browser has to render all those big beautiful images. The solution was conceptually simple: just do less stuff as the user scrolls! In practice, though, that’s not always as simple as it sounds.

Behind the scenes, the search results are loaded into a YUI Model List via XHR and rendered using YUI Views. As the user scrolls down, more results are loaded automatically, appended to the list, and rendered.

In the initial version of the search page, both a Model instance and a View instance were created for each image—a classic MVC approach. This made logical sense and made the images easy to work with in code, but it really sucked from a performance perspective. Creating all those Model and View instances as the user scrolls meant that not only was the browser trying to load and render lots of images, it was also having to execute lots of complex JavaScript at the same time. Even in modern browsers on fast machines, this could be a burden.

Even worse, as the user scrolled further and further down and more results were loaded, memory usage skyrocketed. With potentially thousands of results on the page at once and a Model and View instance for each, users without lots of RAM sometimes saw things grind to a halt as the browser was forced to rely on virtual memory. This clearly wasn’t a good experience.

Performance Improvements

To improve things, we dug deep into each of these performance issues, profiled the hell out of everything (Google Chrome’s CPU and Heap profilers were a godsend for this), and then refactored every line of JavaScript on the search page based on our findings.

Here are some of the things we did to speed stuff up:

  • We wrote a new LazyModelList class that extends YUI’s ModelList and makes it possible for the list to store and manipulate plain JavaScript objects rather than fully-instantiated Model instances. Plain objects are much cheaper to create and work with, both in terms of CPU overhead and memory usage, and LazyModelList makes it easy to revive a simple object into a full Model instance as needed, saving the work of doing it up front for every item. We contributed LazyModelList back to YUI, and it’s now available for everyone to use in YUI 3.6.0.

  • Instead of creating a View instance for each image tile, we now use a single master View instance for the entire list of results. As new results are added to the page, the master result view is responsible for rendering those new results without re-rendering any of the existing results on the page. Now, even when there are thousands of images on the page, there’s just a single view managing them all.

  • We wrote a new ScrollInfo plugin for YUI’s Node component, which provides a highly efficient, throttled wrapper around the browser’s native scroll event. Since the scroll event can fire hundreds of times per second, throttling ensures that our event handlers only run, say, once every 50 or 100 milliseconds rather than on every single event. This puts less of a burden on the browser and ensures that more system resources are available to render content and keep the page feeling responsive as the user scrolls. This plugin isn’t yet available as part of YUI, but we’ve sent a pull request and we hope they’ll accept it.

  • Our profiling revealed several memory leaks in YUI’s event system, which we were able to work around to improve memory usage even above and beyond our other improvements. The YUI team is aware of these issues. Some have already been fixed in YUI 3.6.0, and others will be addressed in an upcoming release.

  • Naturally we also took the opportunity to make lots of other minor improvements and fix lots of little bugs, but those weren’t directly related to the performance effort.

Benchmarks & Pretty Graphs

Here are some pretty graphs demonstrating the effect of the performance improvements we made. Results were gathered using Google Chrome’s profiling tools on a Mac Pro (2.8 GHz Quad-Core) running OS X 10.8.

We also created a jsPerf test for LazyModelList to demonstrate how much faster it is than ModelList.

That Ain’t All

It took some work, but now we’ve got a gorgeous search page that performs well even in older browsers and on slower machines. We’re pretty happy with it, and we hope you are too. But that’s not all! We’ve still got plenty more improvements planned, so keep your eyes peeled.

— Ryan Grove and Brian Strong, SmugMug Sorcerers

  1. August 10, 2012 at 12:09 am | #1

    Awesome work guys! Thanks for the write up and for pushing your improvements back in to YUI.

  2. August 21, 2012 at 9:06 am | #2

    Great Job Team!

    If you want to make it faster let me know! I work for SAP as a lead for In-Memory databases.

  3. August 21, 2012 at 12:41 pm | #3

    Nice! When can we start using this new search on our own Smugmug sites?

    • Dougie
      August 22, 2012 at 12:38 am | #4

      I’d like to know too please.

  4. August 22, 2012 at 10:30 am | #5

    Thanks guys! Glad you like the new Search.

    We’ll definitely be bringing the new Search hotness to your own SmugMug sites as well. There’s a lot of extra work involved since there are so many ways a SmugMug site can be customized and we want to make sure the new search experience looks and works great even on the most heavily customized sites. But don’t worry, we’re on it!

    • Dougie
      August 22, 2012 at 11:55 am | #6

      Thanks for the reply Ryan – appreciated.

  5. phillip
    October 24, 2012 at 3:34 pm | #7

    “When people search for photos on SmugMug, they want to see photos, not a bunch of pagination links and other user interface clutter.”

    Clutter can be caused by too many images on one page, and people generally like to control their Internet experience, and not to have a website control the experience for them.

    Pagination gives a person the control to skip ahead or back with no image clutter slowing the machine.

    The tab with cluttered images slows down tab switching.

    Yes, extra bytes provide clutter, and when a page has to load all of that we have clutter.

    “But Google’s doing it. It’s the Google way.”

    So what? Leave it alone.

    I went to ITT and I can run two Operating Systems. I use Linux, currently. Yet, I really can’t understand the developer’s desire, and I speak of any developer, to take something that works and blast it all to heck like this.

    It’s like Firefox, every other month people scramble all over the Internet to find out how to disable a “new feature”.

    Remember the old days when a link was blue and underlined, and you could find it? Remember when websites had black text on a white page so that so that it was legible? CSS styles screwed that over. Java script is a nasty Internet experience as well.

    Bells and whistles. Bells and whistles. Whoop ding!

  1. August 17, 2012 at 12:53 pm | #1
  2. September 12, 2012 at 11:14 am | #2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: