Forgot your password?
typodupeerror
Linux Business Social Networks The Internet IT

How Facebook Runs Its LAMP Stack 111

Posted by Soulskill
from the very-carefully-and-sometimes-not dept.
prostoalex writes "At QCon San Francisco, Aditya Agarwal of Facebook described how his employer runs its software stack (video and slides). Facebook runs a typical LAMP setup where P stands for PHP with certain customizations, and back-end services that are written in C++ and Java. Facebook has released some of the infrastructure components into the open source community, including the Thrift RPC framework and Scribe distributed logging server."
This discussion has been archived. No new comments can be posted.

How Facebook Runs Its LAMP Stack

Comments Filter:
  • Open source (Score:5, Funny)

    by Norsefire (1494323) * on Saturday April 11, 2009 @09:21AM (#27541485) Journal
    As I recall, some of their code was made open source [techcrunch.com] in 2007, although not deliberately.
  • Whatever they're doing, it's not working too well. Sure, they manage to serve the pages, but the user experience is confusing and it seems to take them forever to roll out new and improved versions.

    • by rarel (697734)
      It sheds a light on the infrastructure at least. Their implementation may not be very bright, but that's quite enlightening nonetheless. Many others would just keep us in the dark.
    • Re:whatever (Score:5, Funny)

      by AHuxley (892839) on Saturday April 11, 2009 @10:12AM (#27541771) Homepage Journal
      To quote a joke on slashdot
      "Is there anything Java cannot make slow."
      • Re:whatever (Score:5, Funny)

        by Tubal-Cain (1289912) on Saturday April 11, 2009 @12:03PM (#27542411) Journal
        Too paraphrase the answer:
        "Sun's stock price plummet.
        • by Yfrwlf (998822)
          Meh, not having an opinion on the capabilities of the Java system itself, anything which needs extra library Y that you aren't using yet is going to slow down while loading it though. I know that Java is even slower due to it being a virtual machine and not just a library of course. Most programs now days have ports for all three major OSes any way, at the same time computers keep getting faster making running things like Java less annoying, so I can't decide whether Java use will increase or not.
          • Try actually benchmarking Java http://kano.net/javabench/ [kano.net] Server-side Java is one of the fastest scalable platforms available.
          • by thsths (31372)

            >at the same time computers keep getting faster making running things like Java less annoying

            Luckily Sun has an answer for that, too: the next Java version is always more than compensating for faster computing :-(. Iremember that back in the days I timed JRE 1.0 or 1.1 to start within 3 seconds on a 486 (!). It takes longer to start JRE 1.6 on my state of the art dual core now.

      • Well, now we see who is pumping [boycottnovell.com] M$ on Slashdot.

        It looks like turfers are turning their poison pens towards Sun, just as near a decade ago they turned towards the late Novell [windowsitpro.com].

    • Well, they have the now-classic problems of needing to add new features to draw more users, to keep users from getting bored, and especially to convince investors and advertisers that there's any growth market for them. This means constantly using their infrastructure in new ways: new database manipulations, new tables, new display utilities, and retaining compatibility across a broad range of clients and older applications. This inevitably slows down new releases.
    • Re: (Score:2, Insightful)

      by theillien (984847)

      Whatever they're doing, it's not working too well. Sure, they manage to serve the pages, but the user experience is confusing and it seems to take them forever to roll out new and improved versions.

      That has little to do with the infrastructure and more to do with the site design. Please don't blame the sys engineers/admins for the poor interface design.

  • Every few days I run into whole sections of core Facebook functionality that are just plain broken for hours. Earlier this week, my main page wouldn't load for most of the day. And every couple of weeks I'm greeted with a "Sorry, you can't log in right now." message.
    • There is much more to Facebook than its LAMP stack, so you shouldn't just blame LAMP stack. And even if the LAMP stack does not work perfectly, there are still insights that developers can glean from it.

    • Every few days I run into whole sections of core Facebook functionality that are just plain broken for hours. Earlier this week, my main page wouldn't load for most of the day. And every couple of weeks I'm greeted with a "Sorry, you can't log in right now." message.

      Are you kidding me? That's not broken, those may be value added features. I'd give a kidney if I didn't need to see the results every time a "friend" - using the term loosely - took a goddamn quiz to find out they were Pablo Picasso in a previous life, yet they were born in 1972 and Pablo died in 1973.

      • by rs79 (71822)

        You do realize you can customize FB to get as much or as little as you want in terms of those notifications don't you? Look up skillfoo.

        The problem I have with FB is not that (any more) it's the fact their, as of the last rev) paged are so JS computationally complex now that rendering my FB home page cause my (fairly contemorary IBM/XP) laptop to just freeze for about 7 seconds while it's doing its JS and ajax goodness.

        Previously, only slashdots new look did this, and not for as long.

        I can get past the idea

        • Every time facebook comes up and people complain, I'm happy to point them in the bloat free direction. [facebook.com] Although this also suffers from feature creep, the "like" madness has recently infected facebook mobile... :{
        • So, what are you going to do with Sebilrazen's kidney?

        • by WillKemp (1338605)

          [......] rendering my FB home page cause my (fairly contemorary IBM/XP) laptop to just freeze for about 7 seconds while it's doing its JS and ajax goodness.

          That's probably XP then, not FB - cos it doesn't do it on my (fairly contemporary Lenovo/Linux) laptop.

    • "Every few days I run into whole sections of core Facebook functionality that are just plain broken for hours"

      What response did you get when you reported it to the Bug Reporting site [facebook.com]?
      • by kbrasee (1379057)
        Didn't submit a bug report, I never knew they had such a page. They certainly don't display it in a place where it's easily noticed.
        • "Didn't submit a bug report, I never knew they had such a page. They certainly don't display it in a place where it's easily noticed"

          I Googled on 'facebook bug report' and which lead to an article that mentioned it ..
  • A hodge podge mess (Score:1, Insightful)

    by thammoud (193905)

    As an architect, I decided to view the presentation so that I can learn new things about scalability and architecture. This presentation came across as very amateurish and lacks any serious technical depth.Facebook seems to be stitched together as a set of "solution de jour" technologies without any real architecture behind it. Too many languages, frameworks and other gems. These guys took the notion of the right language for the task to an extreme. I have to believe that code releases into production is a

    • by julesh (229690)

      These guys took the notion of the right language for the task to an extreme. I have to believe that code releases into production is a big challenge for these folks.

      It explains a lot of the complaints you here about them regularly (I'm not a user, but almost everyone I know is...), specifically that they're not rolling out new features as often as they did when they were small, and that large parts of the site are often unavailable for lengthy periods.

    • "Facebook seems to be stitched together as a set of "solution de jour" technologies without any real architecture behind it. Too many languages, frameworks and other gems. These guys took the notion of the right language for the task to an extreme. I have to believe that code releases into production is a big challenge for these folks"

      What's 'hodge podge' about a highly customized solution. It is precisely what LAMP is all about. It does seem to work for them and with Facebook supporting 200 million acti
    • by RedK (112790)
      You mean like technical depth in architecture documents ? The only people where I work that can ramble on for hours without ever saying 1 clear thing are IT architects.
    • by Gothmolly (148874)

      Declaring yourself an 'architect' doesn't mean a) you are one or b) you do anything OTHER than vague, powerpointy kinds of things. Given that Facebook is bigger and more complex than anything you have built, you should consider reserving judgment.

      • Re: (Score:1, Interesting)

        by Anonymous Coward
        You sure about that?
        http://pastebin.com/f23937796 [pastebin.com]
        Looks like shit to me
      • by thammoud (193905)

        In other news, GM is about to declare bankruptcy, yet one point in time, they produced the most number of cars in the world. I have news for you. THEY WERE SHITTY CARS and eventually that caught up with them. I have no formal mechanical or design engineering background. I bet you have no opinion on GM since you never produced cars.

    • by kv9 (697238)

      This presentation came across as very amateurish and lacks any serious technical depth.

      much like your post.

    • by mfnickster (182520) on Saturday April 11, 2009 @02:35PM (#27543519)

      Facebook seems to be stitched together as a set of "solution de jour" technologies without any real architecture behind it. Too many languages, frameworks and other gems.

      I was thinking the opposite - they have developed an architecture that is modular enough to allow them to develop different pieces using different technologies, yet they all work together pretty seamlessly. I'd say that's quite an accomplishment!

    • by rinoid (451982)

      Blah blah blah blah.

      "Facebook seems to be stitched together as a set of "solution de jour" technologies without any real architecture behind it."

      What's "de jour" about PHP or LAMP? Christ man they have been around for over a decade.

      So if this was written in Cobol would that make you happy?

      Have you developed anything with the reach, user count, view count of Facebook?

      FB doesn't seem to be buckling under the pressure. More photos are uploaded to FB than any other site man:
      http://www.techcrunch.com/2009/02/22/ [techcrunch.com]

    • by codepunk (167897)

      There are two types of people in this world, those that talk shit and those that do shit. Every so
      called "architect" I have ever met falls into the talk shit category. I also find that those that are in the
      talk shit category make up for the fact that they cannot do shit by talking shit.

      Most modern languages can be made to play rather well together, certainly in the web development arena where
      multiple languages are a must. You already have to master at least two our three languages for typical web developme

  • I need new glasses.

    Anyway, I think the answer is 'Simply by existing'?

  • One question: (Score:5, Interesting)

    by bogaboga (793279) on Saturday April 11, 2009 @09:51AM (#27541661)

    About how much has Facebook saved by using Open Source Software? I ask because I am not familiar with licensing costs from competing solutions. Thanks!

    • Re:One question: (Score:5, Informative)

      by julesh (229690) on Saturday April 11, 2009 @10:15AM (#27541779)

      About how much has Facebook saved by using Open Source Software? I ask because I am not familiar with licensing costs from competing solutions. Thanks!

      I haven't watched the presentation so don't know if this is answered there, but it's hard to pin down any numbers on precisely how many servers facebook operates. That said, an estimate of their expected power usage in their recently acquired second datacenter [datacenterknowledge.com] is 6 megawatts, placed at twice the usage in their current datacenter. Realistically, this probably equates to a cluster of around 5,000 machines in the current datacenter.

      Costs per machine are likely to be restricted to Windows Server Web Edition; other software would not be needed on all machines (depending on cluster architecture, of course) so would be a trivial cost in comparison. Retail for the web edition is $399; I think we could expect such a high profile user to qualify for a 50% discount. This would put their software costs at about $1M. Considering that they're believed to have spent over 100 times this on hardware and support costs over the last year, I doubt this would be a particular concern. Price of purchase is not a factor in why facebook does not run on proprietary software.

      • Re: (Score:3, Funny)

        by David Gerard (12369)

        Microsoft has announced the infrastructure for its cloud computing service Azure, formerly (and presently) Windows Vapor.

        "We want all open source innovation to happen on Windows. In practice, Windows is too slow, and just putting Linux underneath the same software stack triples performance. So we're running the Windows versions of the software on Linux using Wine."

        The new Microsoft Amazingly Open And Genuine Public License allows you complete freedom to use, modify and redistribute the software provided

      • This is both informative (a believable in-the-ball-park analysis of Facebook operational costs) & insightful (price is not enough to be the best solution)
        • Offhand guess they picked LAMP back when prices *were* more of a concern for them, and have no incentive to switch now.

    • That's always hard to calculate. It's always licensing costs vs. man hour costs. I've been involved in projects where people were trying to get MySQL to play nicely with EC2 without paying for MySQL enterprise. The amount of man hours it cost them would have made DB2 a much better solution. Especially with the fact that DB2 Express-C is free and already had AMI's ready to go.

    • Re: (Score:1, Informative)

      by Anonymous Coward

      I think the real question is, if they run with LAMP so much, how come they have and request for so many oracle developers?

      Palo Alto, CA

      Description
      Facebook is seeking an Oracle Applications Database Administrator to join the IT team and help build and maintain the IT application footprint. This is a full-time position based in our main office in downtown Palo Alto and will report to the manager of IT Development.

      • by BitZtream (692029)

        Because if you get someone who actually knows how to deal with Oracle, then you get someone who has a clue.

        This is in contrast to the fact that half of the high school kids in America think they are 'MySQL DBAs' because they installed it once.

        An Oracle developer will have no problem working with MySQL, though they may find themselves banging their heads against solid objects often when they are forced to deal with a hack that claims to be a database server. You won't find a high percentage of people claimi

        • Re: (Score:1, Flamebait)

          Not quite. As I understand it, Oracle is used pretty heavily in the back end. It propagates out to faster-but-looser MySQL systems for web display.

          Not that I disagree with you about the tardery of most MySQL "admins". ;-)

  • I know this should be the job of tags, but to help put this in context, remember the recent uptime [slashdot.org] comparison that showed Facebook with pretty decent availability compared to other social networking sites. I'd say it takes the admins a fair amount of disclipine and perseverance to attain those kinds of numbers. (of course, it probably has nothing to do with the uptime of their various sundry and mostly useless modules, but I'd guess that's a different set of admins than the ones that care for the core LAM

    • Re: (Score:3, Interesting)

      by BitZtream (692029)

      It takes pretty much 0 work to make LAMP continue to function. Its for all practical purposes, set it up once (properly) and forget it.

      It takes work to make the applications on top of it function continually as thats where the change occurs. LAMP isn't going down on its own, it'll appear to 'go down' because of the 'mostly useless modules' that work along with it fail, not because LAMP does.

      I would expect the admin(s) that care for 'the core LAMP platform' spend most of their time doing other stuff. In r

      • I am not a DBA (but I did stay at a holiday iinn express last night). But wouldn't it make more sense that the DBA's handle db performance and tuning since they are the ones intricately familiar with how it is set up. Isn't the whole point that the developers working on the db apps should just know what data they want not how the db is configured.
  • When I first saw the post, I though it said how Facebook RUINS its LAMP stack.

    I think that has to do with my experience with the apps and how often things timeout in that regard. It's a little frustrating and I'm sure it has nothing to do with the guys at Facebook, but it is interesting to find how that third-party experience affects my subconscious.

  • Well, I'll be - the first LAMPCJ Stack - make it too big to fail.

    For all of you fellow architect bods out there, this is how you do it:

    PHP - California, Texas and France
    C++ - New Jersey and Tibet
    Java - California, India, and Somalia

    Now, what does this variable name in Somali represent?
  • There's also Cassandra, the Java based distributed database that they've made open-source - first on google code [google.com], and now as an apache project [apache.org](in incubation).
  • Who gives a shit? (Score:2, Insightful)

    by PingXao (153057)

    I guess a story on /. with only 75 comments after 7 hours pretty much answers that question, eh?

  • by Samah (729132)
    Just to be different, I run: Linux, Lighttpd [lighttpd.net], Sqlite [sqlite.org], Kepler [keplerproject.org]

Life is difficult because it is non-linear.

Working...