When Steve Ganz left to compete in the PDGA World Championships little did he know that he’d return to find a fast talking, “fashionably low-rate” robot built in his likeness; he may never take a vacation again! The “Ganzbot”, as we named it, was a collaborative effort from our Web Development team with participation from Jamie Still, Dennis Hengeveld, Scott Olson and me.

The robot reads quotes from the Ganzbot Twitter feed and an internal message queue that anyone in the office can submit text to. When he talks his face becomes expressive with moving eyebrows, lips synchronized to the audio and eyes that change color.

The idea behind Ganzbot actually goes back several months when Bryan Haggerty and Steve were joking that he needed a robot to do his work so he would have time for meetings. But frankly, we just wanted to put it together to see Steve’s reaction when he returned and I think that alone was worth the effort!

That soon caught on and Ganzbot became an office joke until being realized as a real robot while Steve was away. But who’d have thought that Engadget, Gizmodo and Make would notice too.

Want to make your own GanzBot? Check out step-by-step instructions here.

ShareThis

Tweet This Post

In Light Engineering (LED), we’re known to be multilingual – depending on the project, we’ve been known to speak Perl, Python, Java, C++, Javascript and PHP, to name a few. Our weapon of choice is still Ruby on Rails, the popular MVC framework. Out belief is that Rails makes certain types of tasks easy, and others laughably trivial. That being said, LinkedIn is still primarily a Java shop, and for good reason. Java technologies are mature, proven, and all around solid. For this reason, LED has had a very vested interest in the development work that is going into JRuby.

We started a few months ago around the time JRuby 1.1.2 went live by switching some of our Rails applications to run on Glassfish. Using Warbler, we successfully wrapped our Rails applications into WAR files and deployed on Glassfish (we’ll probably write a more detailed tutorial of this at a future date). A WAR file is completely self contained application that can be deployed simply by copying to an autodeploy directory. No more Apache/Nginx reverse proxy, no more Capistrano, no more installing gems on a production container, no more of any of that madness. This was a huge win, and we broke out the champagne bottles.

But we weren’t done. We weren’t taking advantage of many Java technologies, most notably, we weren’t taking advantage of the JDBC connection pooling capabilities of the Glassfish application server for our MySQL database.

We started by reading this tutorial by Arun Gupta of Sun. The article is fantastic, but the one criticism I have is that it was written from the perspective of a master Java engineer that learned Rails, as opposed to that of a Rails engineer approaching JRuby.

From a high level, here are the steps needed to enable JDBC connection pooling for a Rails application running in a Glassfish container:

1. Define a JDBC connection pool.
2. Define a JDBC resource with a JNDI name.
3. Download and install the MySQL connection adapter.
4. Update database.yml to use JDBC.
5. Configure ActiveRecord to disconnect after every query.

Believe it or not, there are only five steps. I have to admit, I was initially intimidated. Java allows so much power and flexibility that, to a novice, seeing a hundred configuration choices in the Glassfish admin web UI can be a deterrent. As it turns out, we only need to touch two parts of that UI. Let’s get started:

1. Define a JDBC connection pool.

Log in to your Glassfish application server. Expand Resources->Connection Pools.

Click new. You’ll be presented with three fields. The name is arbitrary, but you’ll need to know it later. Select javax.sql.DataSource as the resource type, and MySQL as the vendor.

The next screen will have more options. Change the datasource classname to com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource

Under additional properties, there should be fields configuring the database connection. Set these as appropriate.

2. Define a JDBC resource with a JNDI name.

JNDI stands for Java Naming and Directory Interface, and will allow us to create a standardized name for the JDBC connection pool we just created.

In the navigation pane, click through to Resources -> JDBC -> JDBC Resources. Click new.

In the drop down box, select the JDBC connection pool created in step 1. For the JNDI name, it’s accepted practice to name it jdbc/connection_name.

3. Download and install the MySQL connection adapter.

tar zxvf mysql-connector-java-VERSION.tar.gz
cd mysql-connector-java-VERSION
ant
cp mysql-connector-java-VERSION-bin.jar $GLASSFISH_HOME/lib

You may have to restart Glassfish for the install to work:

asadmin stop-domain DOMAIN
asadmin start-domain DOMAIN

4. Update database.yml to use JDBC.

In your config/database.yml file, we need to tell Rails to use the connection pool rather than directly connecting to the database. Here’s a snippet of our production configuration:

production:
   adapter: jdbc
   jndi: jdbc/polls
   driver: com.mysql.jdbc.Driver

That’s all you will need. Unlike standard configuration files, you do not need to specify things like the username, password or host because these are configured in the Application Server. I like this method because it means the engineer doing the deployment does not need to build the YAML file each time, check it in to SVN, or copy from a database.yml template with the production settings. It’s one less deployment step, and ultimately, one less item on the security checklist.

5. Configure ActiveRecord to disconnect after every query.

ActiveRecord maintains a persistent connection to the database. This is no longer necessary, as there is very little overhead in opening a connection to JDBC, which manages the connection persistence. We’ll need to disable this. I’m using code borrowed from Nick Sieger’s awesome presentation at RailsConf 2008:

# config/initializers/close_connections.rb
if defined?($servlet_context)
   require 'action_controller/dispatcher'

   
ActionController::Dispatcher.after_dispatch do      
      ActiveRecord::Base.clear_active_connections!
   end
end

You’re done! Now all you have to do is build the WAR file and drop it in Glassfish’s autodeploy directory.

ShareThis

Tweet This Post

Last week, we were fortunate enough to have Google’s Steve Souders give a tech talk on Even Faster Web Sites, fresh on the heels of his presentation at OSCON in Portland two weeks ago.

Steve’s name likely sounds familiar to you; indeed, if you work in web development, you’d have to work hard to avoid benefiting from his research. Not only has he spoken often at such conferences as OSCON, Rich Web Experience, Web 2.0 Expo, and The Ajax Experience; he co-founded O’Reilly’s Velocity conference; he’s also the author of the book High Performance Web Sites: Essential Knowledge for Front-End Engineers and the excellent YSlow extension for Firebug. He credits a lot of his success to adopting Harvey Penick’s methodology of keeping a notebook handy and jotting down interesting observations, researching them further, and acting on the best ones.

The tech talk centered around Steve’s "Performance Golden Rule": 80-90% of the end user response time is spent on the front end (he uses a simple user-centric metric for dividing front end and back end load time: loading the HTML document is back end; everything after that is front end). This golden rule is clearly sensible for three reasons:

1. There’s greater potential for improvement. Saving half of 90% is a much bigger win than half of 10%.
2. Front end changes are simpler than back end. The techniques you’ll use (minification, consolidation, compression, etc.) take engineer days, not weeks.
3. It’s a demonstrably effective approach. It’s not only been proven at Yahoo and elsewhere, you can likely make trivial changes to your own site and see the results immediately.

The first half of the talk was a discussion of the fourteen rules which are the basis of both his book and YSlow, which essentially automates measuring a web site’s performance against those fourteen metrics. Rather than recount it in great detail, I’ll urge you to get a copy of his book (via Amazon, O’Reilly’s Safari, or your favorite local bookstore) and read the Yahoo! Exceptional Performance Team site while you’re waiting to get the book in your hot little hands. Time and money well spent.

The second half was focused on Javascript and how it affects page loading. The load time of most modern web pages is 20-40% of the total load time, so reducing this results in noticeable improvement in page load speed. His key observations in this area are:

• Most browsers load Javascript serially by default
• Only about one-fourth of the Javascript functions that are loaded are actually needed before the onLoad event fires
• It’s important to understand how element load order affects particular browsers (e.g. Firefox will block all parallel downloads while downloading stylesheets; Internet Explorer will block if you follow a stylesheet with an inline script)

Accordingly, the top items on his hit list are splitting the initial script payload, loading
scripts in parallel, and careful attention to placement of inline
scripts — all aimed at accelerating the firing of the onLoad
event.

I could go on in some detail about how to accomplish this, but you’d be better served to read Steve’s blog and watch some of the videos of his other presentations available on his web site. Instead, here’s a preview of the guidelines which are the beginning of his next book, some of which are also found on the Yahoo! Exceptional Performance Team site:

1. Split the initial payload
2. Load scripts without blocking
3. Don’t scatter inline scripts
4. Split dominant domains
5. Make static content cookie-free
6. Reduce cookie weight
7. Minify CSS
8. Optimize images
9. Use iframes sparingly
10. To www or not to www

Finally, you should have a look at Cuzillion, Steve’s tool for modeling web page elements and tinkering with how order affects loading. His slides from the tech talk cite it for examples, and I suspect his next book will, as well. Very useful as you’re trying find the delays in loading your web pages and understand what you can do about them.

ShareThis

Tweet This Post

On Tuesday, Brad Neuberg from Google’s Developer Program visited LinkedIn’s headquarters in Mountain View to give a presentation on Gears to the engineering department as part of our series of LinkedIn Tech Talks.

Gears is an open source project that enables more powerful web applications, and provides for a better user experience with a cross-browser plugin. Gears gets installed on a user’s machine (the same way any other plugin like Adobe Acrobat Reader does), and then provides a local database, worker pool, and a local web server.

The database is a full-featured SQLite DB which provides SQL queries, views, triggers, full text search, and the ability to store gigs of user data locally. The local server can allow you to run web apps offline by storing HTML, CSS, images, etc., but in managed, versioned bundles which can be downloaded in the background, and it does not get cleared when the user clears the browser’s cache. Using it is much better than relying on the normal browser cache. The worker pool gives you the ability to send tasks for background processing off to Gears, so that JavaScript code can be run asynchronously without blocking the main script running in the browser. This is perfect for dealing with any number crunching, long DB queries, or communication with remote servers. Google has more information in their API docs.

Afterwards, Brad stayed around to discuss how different engineering groups at LinkedIn could use Gears to give our users a better user experience and more powerful features.

Although we don’t have Brad’s talk at LinkedIn online for you to see, here is another presentation on Gears that Brad gave at Google I/O 2008.

I’m impressed with Gears and some of the demos, and real applications like Remember The Milk that I’ve seen using it. I’m sure we could make use of this technology to dramatically improve the user experience for many LinkedIn users in areas like the homepage, network updates, address book, and corporate solutions.

NOTE: This entry was published with Brad Neuberg’s permission.

ShareThis

Tweet This Post

Last week, I attended O’Reilly’s Open Source Convention (a.k.a. OSCON) in Portland, Oregon. This was the 5th year I’ve attended (and spoke) at OSCON. I really like the diversity of open source projects and languages discussed at this conference. You can find always find good sessions on Perl, PHP, Python, Ruby and even Java. Of the many sessions I attended, the ones I learned the most from were Google XML Pages by Harry Heymann and Laurence Gonsalves and Even Faster Web Sites (PPT) by Steve Souders. If you didn’t get a chance to attend OSCON this year, you might try watching OSCON in 37 minutes by Gregg Pollack. This is a video where Gregg asks many speakers to summarize their talks in 30 seconds or less.

On Wednesday, I presented my talk titled Web Frameworks of the Future: Flex, GWT, Grails and Rails. I was inspired to do this talk when I first heard of SOFEA (Service Oriented Front End Architecture) and SOUI (Service Oriented UI) last fall. These acronyms suggest the same type of architecture; one where the backend serves up data (via SOAP, REST, etc.) and the frontend handles page flow, navigation, etc. SOFEA was created by Ganesh Prasad, Rajat Taneja and Vikrant Todankar in their "Life Above the Service Tier" paper, while SOUI was created by Nolan Wright and Jeff Haynie in a couple of blog posts.

To do research for this talk, I created two different versions of the same application. One was with Flex and Rails and the other was with GWT and Grails. While I didn’t complete the application before the talk, I do plan on continuing to work on it in my spare time and will hopefully have something to show later this fall. The application I’m creating is called "Rich Resume" and will allow you to edit and publish your resume online quickly and easily. If everything goes well, I’ll add the ability to import your LinkedIn Profile to get started. As far as my progress on the application, I was able to finish both backends with Grails and Rails in under 25 hours. Both framework’s out-of-the-box dynamic scaffolding didn’t support one-to-many relationships very well, so I did have to tweak some things. With Rails, I was able to use ActiveScaffold to solve my issues. With Grails, I had to generate Controllers and then manually tweak them to add one-to-many and REST support.

As far as Flex and GWT, I found them both easy to get started with. I got bit early by GWT’s Same Origin Policy, which made it difficult to develop the frontend and backend on different hosts. After moving the GWT app into Grails (using the GWT Plugin), things went much smoother. I was disappointed to find that GWT doesn’t support REST too well, but there seems to be many folks that share this same sentiment and quite a few solutions are being developed (for example, Restlet-GWT). While Flex allows for cross-domain communication, I was disappointed to find it doesn’t support zero turnaround (save/refresh) outside of an IDE. I’m sure over the coming months, I’ll develop a greater sense of affection for these frameworks, but I also think it’s important to note first impressions.

Below is my presentation for your viewing pleasure.

In October, I’ll be presenting a similar talk at the Colorado Software Summit. This time, I’ll be using Appcelerator to build my Rich Resume application. It should be a good show, especially since my colleagues Yan Pujante and Ikai Lan will be speaking. If you want to learn about how they’ve enhanced LinkedIn’s applications and architecture, you won’t want to miss their talks.

ShareThis

Tweet This Post

Last Thursday, Kevin Brown visited LinkedIn’s Mountain View office to do a presentation on Shindig, an OpenSocial Reference Implementation. Below are my notes from his talk.

In September 2007, Google started thinking about Social APIs. Google Gadgets would be better with access to Social Data … but that’s just Google. It was recognized that this is something that many others would like access to. OpenSocial was announced in November 2007. It’s an open standard for developer platforms that has a strong emphasis on “social” data. It’s based on gadgets which is now covered by The Open Social Foundation.

In November, many Googlers started working on a Google Code project based on Java and iGoogle. However, there was too much proprietary code. In December, Brian McCallister of Ning created an ASF Proposal for Shindig. It was a rough port of iGoogle but with Ning’s PHP code. This turned out to be a great starting point. It immediately got interest from Google, Hi5, MySpace and others. While most committers are still from Google, there are 12 developers that work on it full time and they’re adding 2 committers each month. Shindig is a Java/PHP implementation of OpenSocial. Open Campfire is an Apache-licensed .NET implementation that hopes to eventually merge into Shindig.

Shindig has extensive use of existing open source components, including Abdera, Jakarta Commons, Guice, ICU4J, OAuth.net, and JSON.org. It’s largely well tested (developers use TDD), especially for the newer code. There’s a heavy emphasis on DI (Guice helps tremendously) and the builds are done with Maven (Apache recommendation, Kevin hates it).

3 Parts of Shindig:

1. Gadget Rendering: Fetches/parses gadget XML documents. It also provides a robust proxy to implement gadgets.io requirements (including OAuth and Signed Fetch). It acts as a “glue” between JavaScript libraries, remote sites and social data. It’s built for medium to large deployments and has robust HTTP support. Lastly, it’s very flexible so all major components can be replaced.
2. Social API: Handles RESTful API calls and interacts with social data. It provides a backend for opensocial.* JavaScript APIs. It’s mostly a serialization/de-serialization layer that delegates to your social data. Currently evaluation two implementation options (Dave Primer’s AtomPub version or JSON version – lengthy explanation on shindig-dev). Currently, developers are creating an entirely new version because of REST and its JSON support. REST has a lot of issues when it comes to JSON. It works great with AtomPub, but AtomPub has too much verbose XML.
3. Client Code: The first issue that comes up with client code is Security.

For security, iframes cover most of the problems. Cross-domain communication requires special effort – gadgets.rpc has to be implemented using a variety of techniques: window.postMessage in HTML5, document.frameElement in Firefox and window.opener in IE (still in development). Retrieving third-party data is covered by OAuth and Signed Fetch. Getting OAuth / Signed Fetch credentials can be done by passing an encrypted blog of data to the server and treating it as a cookie equivalent. Caja is the future.

Caja (pronounced ka-ha) makes it possible to run third-party JavaScript alongside existing code. It’s not quite ready yet, but significant process has been made. It has limited support in Shindig today, which requires “taming” JavaScript APIs and DOM testing.

Summary

Full support for 0.6-0.8 gadget rendering specification (both versions). Full support for 0.6-0.8 JavaScript APIs. REST mostly done (both versions, PHP is closer). Deployed (or in progress) on many sites: Orkut, hi5, iGoogle sandbox, Hyves, CyWorld, Ning and hundreds of others. It has a very active mailing list and they’re very interested in finding folks with more Maven experience to help out.

The project’s next priority is a stable release. Shindig 1.0 will be OpenSocial 0.8 Compliant. It must have full support for REST. The Java and PHP implementations should be released around the same time. Priority #2 is graduating from Apache Incubator (goals are currently on target). One of the main things they need is more committers, especially those that aren’t from Google. Priority #3 is future enhancements, including:

• 0.9: “proxied” content type
• 0.9: OpenSocial templates
• HTTP Performance Improvements
• “out of the box” shared caching
• More social data implementations (e.g. JDBC)
• Better Documentation

We have many more Tech Talks planned in the future at LinkedIn. Please stay tuned to this blog to learn about new and exciting technologies that we’re learning about.

NOTE: This entry was published with Kevin Brown’s permission.

ShareThis

Tweet This Post

In my last post I talked about how the Spring-DM extender automatically recognizes new namespaces. In this post I’ll talk about how to configure the extender itself. First let’s talk a little bit about fragments (since it is the mechanism used by Spring-DM).

What is a fragment?
A fragment is a special kind of OSGi bundle. By itself a fragment does not do anything: it cannot be started (it is illegal to have an activator for a fragment). A fragment needs to be ‘attached’ to another bundle called the host bundle. You define a fragment bundle by adding a special header in the MANIFEST:

Fragment-Host: <Host Bundle Symbolic Name>

This is what it looks like in the Equinox console:

39	ACTIVE      org.springframework.bundle.osgi.extender_1.1.0.m2	            Fragments=5555	RESOLVED    com.linkedin.kernel-core.lispring-osgi_1.0.1.SNAPSHOT	            Master=39

55 is a fragment bundle and it attaches to its host 39. The header is defined like this:

Fragment-Host: org.springframework.bundle.osgi.extender

Note that you can have more than one fragment attached to a host (but you cannot have a fragment attached to multiple hosts).

What is the use of a fragment?
Once a fragment is attached to its host, it behaves like if it was part of it. Conceptually, the behavior is similar to what you would obtain if you were to unjar the host and the fragment, and recreate a unique bundle made up of the content of the 2 separate bundles (including manifest headers). It allows to attach behavior and resources to a bundle.

For example if you use the method Bundle.findEntries("META-INF/", "*", false) before attaching the fragment then you will see only the content of the META-INF directory in the host bundle. If you call the same method after the fragment has been attached you will get a different result with the content of the META-INF directory in the host and fragment! This is pretty powerful to load any kind of resources (for example, a localized properties file, etc.).

Here is another example using the fact that the host has now access to classes it didn’t know about before:

• I have a bundle which depends on nothing else but the JDK and OSGi.
• In the activator, I instantiate a LogFactory (internal LinkedIn class) this way:

  String logFactoryClassName = System.getProperty("org.xeril.log.Log.factory.class.name");
  
  LogFactory factory = null;
  
  // a class name was provided
  if(logFactoryClassName != null)
  {
  try
  {
  Class logFactoryClass =
  Thread.currentThread().getContextClassLoader().loadClass(logFactoryClassName);
  
  factory = (LogFactory) logFactoryClass.newInstance();
  }
  catch(Throwable th)
  {
  // we display the stack trace but we don't fail
  th.printStackTrace();
  }
  }
  
  // no class name or error
  if(factory == null)
  {
  factory = new JDKLogFactory();
  }
  

• Now I have a fragment bundle which depends on Log4j and has a Log4jLogFactory class. The system property gets set on the command line and if the fragment is attached, using reflection, the Log4j factory will be instantiated properly although the host does not depend on Log4j nor knows anything about Log4jLogFactory. If it is not attached I simply use the JDK logger.

Fragment Lifecycle
The lifecycle of fragments is a little bit different from other bundles. For example you cannot start a fragment. If you undeploy a fragment that is attached to a host, the fragment is gone from the list of bundles but it is still attached to the host. Same if you install a new one. You actually need to ‘refresh’ the host to see something happening.

Below is an example of activating a bundle with Equinox:

osgi> ss39	ACTIVE      org.springframework.bundle.osgi.extender_1.1.0.m2	            Fragments=5555	RESOLVED    com.linkedin.kernel-core.lispring-osgi_1.0.1.SNAPSHOT	            Master=39osgi> uninstall 55

osgi> ss39	ACTIVE      org.springframework.bundle.osgi.extender_1.1.0.m2	            Fragments=55

osgi> install file:/.../com.linkedin.kernel-core.lispring-osgi_1.0.1.SNAPSHOT.jar

osgi> ss39	ACTIVE      org.springframework.bundle.osgi.extender_1.1.0.m2	            Fragments=5556	INSTALLED   com.linkedin.kernel-core.lispring-osgi_1.0.1.SNAPSHOT

osgi> refresh 39osgi> ss39	ACTIVE      org.springframework.bundle.osgi.extender_1.1.0.m2	            Fragments=5656	RESOLVED    com.linkedin.kernel-core.lispring-osgi_1.0.1.SNAPSHOT	            Master=39

Spring-DM Extender
Now that you know what a fragment is, it should be easy to understand how to extend Spring-DM: you simply need to attach a fragment to the host bundle called org.springframework.bundle.osgi.extender. In your fragment you need to have a directory called META-INF/spring/extender which contains normal Spring context files. Spring-DM will automatically see those files, create an application context and instantiate the beans. It will then look for beans with some specific names to customize itself: this is described in the reference documentation.

For example, by having a bean called applicationContextCreator, you can extend or override entirely the way Spring-DM detects that a bundle is a Spring Powered Bundle. By default, Spring-DM looks for a header or the presence of a META-INF/spring directory to determine that it is a Spring Powered Bundle. You can redefine this behavior to be your own custom header or your own custom directory (or whatever other mechanism you can think of). You can also provide configuration properties like how long to wait on shutdown.

Note: As mentioned in the fragment lifecycle section above, be careful if you attach/update the fragment. You need to refresh the host which in the case of Spring-DM extender will shutdown all application contexts created previously and recreate them. My recommendation is to install both the fragment and the host, and start the host only once they are both installed so that the fragment attaches right away without having to refresh the host afterwards.

ShareThis

Tweet This Post

Despite the foreboding, Matrix-esque connotation, I was excited to become truly LinkedIn. My previous internships were interesting and educational, but I was looking forward to working in a younger, faster company. And, wow! LinkedIn delivers on that. By the end of day one I was set up with envy-inspiring equipment (fully-loaded MacBook Pro, giant Cinema display, and super comfy chair!), friendly, helpful colleagues, and all the tools I needed to start working on my first project, OpenSocial widgets.

The goal was to create a lightweight widget that grabs company information from LinkedIn via OpenSocial, and displays buzz about the user’s company on their LinkedIn home page, all as an exercise to help LinkedIn’s OpenSocial team develop and test the platform. Getting up to speed on OpenSocial was very easy. Between the Google tutorials and knowledgeable colleagues, setting up "hello world" was practically done for me. By day four I had a working, albeit ugly, version of the widget.

In short, working at LinkedIn is awesome. From the fast pace and supportive working environment (free food!) to the trust the company places in all its employees, even interns, LinkedIn is a company I am proud to work for. It’s going to be the best summer ever! Or at the very least, better than that one I spent painting houses.

Andrew Carman
LinkedIntern

ShareThis

Tweet This Post

As a podcast addict I’m constantly looking for good shows. As far as I can tell, The Java Posse (from Tor Norbye, Carl Quinn, Dick Wall, and Joe Nuxoll) is the best one out there. I’ve been listening to the show for the last two years and I’m very impressed that they’ve maintained such high quality for so long now. Shortly after joining LinkedIn I created a new LinkedIn Group for Java Posse, which has been growing rapidly.

A few days ago I heard the last episode while enjoying the ease of use of Twitters API. It was a nice surprise to hear the announcement about LinkedIn being elected the “Java Website of the Week”. It was even nicer to hear them praising the famous Linker Steve Ganz.

It seems that the announcement is a result of a blog post from the “Break it Down” blog commenting on the presentations the LinkedIn CDN team did at the last JavaOne. The post is titled LinkedIn Is Written in 99% Java, so to complete the picture I responded to the community with a message about how LinkedIn is 99% Java but 100% Mac.

The rest of the programming languages we use are C++, Ruby on Rails and Groovy/Grails. We have one team that uses Ruby on Rails top to bottom, another which heavily uses Grails, and of course XUL for the Firefox toolbar. While we enjoy using these great technologies, most of our core business logic is written in Java on a Spring/Jetty/Tomcat stack.

The 100% Mac is the development environment used by all the engineers (i.e. desktops and laptops). A new engineer that comes in gets a new MacPro with dual quad-core CPUs and 12GB of ram and a MacBook Pro. The down side of the whole deal is that you have to choose between two 23’’ or one 30’’ Apple Cinema display. Life is tough and you just can’t have it all.

If you would like to have such an epic working environment you still have a chance – we’re hiring.

ShareThis

Tweet This Post

Jim Meyer

This is the first post in a series on scalability using Ruby on Rails including an examination of the steps we took – some successful, some not so much – to scale Bumper Sticker to be one of the top four applications on Facebook. This time out, we’re going to give you a quick history of the Light Engineering Development (LED) team at LinkedIn and the creation of Bumper Sticker.

One of the best things about working for LinkedIn is the constant exposure to new technologies and challenges. While the majority of the LinkedIn infrastructure runs on Java, it’s no secret that we don’t shy away from other interesting languages and frameworks – we’re always looking for ways to make better software faster. This is the spirit that launched the LED group with the charter to see how quickly it could prototype new ideas and features using Ruby on Rails.

Interestingly enough, the most successful of LED’s Rails applications is one of its first – Bumper Sticker, the fun, viral media sharing application that allows users to express their individuality by sticking small, quirky, sometimes edgy, often humorous, rarely serious virtual stickers on each other’s (and their own) Facebook profiles.

Bumper Sticker started as a small experiment in August, 2007. Facebook had released their development platform while we were hard at work on our own. We were curious to experiment and discover some of the characteristics of an application platform built on a social network and to see what, if any, learning we could apply to our own efforts. After noticing that professional and business-related applications weren’t flourishing in the Facebook ecosystem, a few of our Product folks put their heads together while out for a run; one engineer, one week, and a few Joyent accelerators later, Bumper Sticker was born.

We’d be lying if we said that anyone was prepared for the kind of success Bumper Sticker has had since then … though we should have expected it, given the excellent Product team here at LinkedIn. Here’s a quick snapshot of Bumper Sticker statistics at this moment:

• 13.5 million installations
• 1.5 million daily active users
• 20-27 million canvas page views a day

All of this is served by:

• 13 web application servers running nginx and mongrel
• 8 static asset servers serving over 3,500,000 stickers, soon to migrate to a content distribution network (CDN)
• 4 MySQL servers in a master/slave configuration using Rick Olson’s excellent masochism plugin

Ruby on Rails is frequently criticized for lacking the ability to scale. While the road to a billion page views per month has certainly had some potholes, Bumper Sticker has clearly demonstrated that the Rails platform can scale quite well, so as long as the team behind it understands that many of the bottlenecks are exactly those faced by developers on any other database-driven web platform. From an engineering perspective, Bumper Sticker could not be more perfect for developing a set of best practices for scaling web applications that exhibit unpredicted, unexpected, explosive growth.

Stay tuned! There’s much more good stuff to come.

Quick Update: Watch Ikai and Jim describe the genesis of the LED team and Bumper Sticker

ShareThis

Tweet This Post