Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
Novell Open Source Linux News

London Stock Exchange Tackles System Problem 237

Posted by samzenpus
from the fix-the-glitch dept.
DMandPenfold writes "The London Stock Exchange has taken steps to resolve a system problem that occurred at 4.30pm Tuesday, which saw a delay to the start of the closing auction and knocked out automatic trades during a 42 second period. The problem occurred a day after the high profile launch of its new matching engine on the main equities market, based on the SUSE Linux system from Novell."
This discussion has been archived. No new comments can be posted.

London Stock Exchange Tackles System Problem

Comments Filter:
  • Well, well, well... (Score:2, Interesting)

    by LordNacho (1909280)

    This just shows that it's hard to build these highly available, low latency, massive usergroup systems. Previously there was a lot of chatter about the platforms (.NET, MSSQL 2003, etc...)

    The problem is more likely to be internal organisation than specific platforms.

    • by _Sprocket_ (42527) on Thursday February 17, 2011 @03:09AM (#35229628)

      This just shows that it's hard to build these highly available, low latency, massive usergroup systems. Previously there was a lot of chatter about the platforms (.NET, MSSQL 2003, etc...)

      Yes. And let us not forget that a lot of that chatter came from Microsoft's PR department.

    • by the_womble (580291) on Thursday February 17, 2011 @06:40AM (#35230428) Homepage Journal

      Given that this was a showcase client for MS, it does not make them looks good.

      Given that the MS was involved in developing the TradeElect system (the Windows based one), so even is the fault lies in TradeElect rather than the MS platform, then its still at least partly MS's fault.

      Microsoft and Accenture developed a system that turned out not to be as good as the one developed by a small Sri Lankan company no-one had ever heard of.

    • Teething problems (Score:5, Insightful)

      by mangu (126918) on Thursday February 17, 2011 @07:15AM (#35230620)

      Previously there was a lot of chatter about the platforms (.NET, MSSQL 2003, etc...)

      It's one thing to have a 42 seconds glitch in the first day a totally new system is powered up. That's perfectly normal, and had been predicted [computerworlduk.com]:

      "Observers watching today's Linux-based launch will likely note that such a large change could bring about some teething problems, as with any technology overhaul."

      It's a totally different thing to have it stop for a whole day after having been in operation for three months.

      So, in conclusion, yes, it's about the platform. .NET, MSSQL 2003, etc aren't robust enough for this kind of job.

      • To be fair we don't know the exact reasons TradElec went down. It was mentioned that an upgrade went wrong. It could have been the platform.

        There was talk that the platform was slow and wasn't performing as expected. The outage was just the last straw. Like a poor performing employee, you'd like to replace him/her but until they screw up royally you don't have enough justification for HR.

        Or it could have been working fine but a public embarasment like that needed a scape goat. Someone had to be fired;

      • You do realize that it's not .NET, MSSQL and Windows 2003 alone that made up the trading system, right? It was also a boat load of custom developed software.

        There is no evidence to suggest the problems that TradeElect had were directly the problem with the technology it was based on, but more likely due to problems in the custom software developed for it. I'd venture a guess that Microsoft's own web sites, like Hotmail, MSN, etc.. generate more load and have more complexity than TradeElect, and they handl

        • True Hotmail may serve more clients per day but the type of load is vastly different. If it takes 3 seconds instead of 2 to load your mail, very few people care. Exchanges like this are all about transactional processing and execution and performance is measured in milliseconds. Volumes are chaotic in nature. On hotmail you might see a small spike during parts of the day but not an order of magnitude jump because of some news announcement. Bottom line MS has little experience with exchanges before this
          • That may be, but there is no evidence to suggest it was the basic stack that was causing the problem.

            It may simply be that the TradeElect software itself was not written to withstand the loads that were placed upon it.

            Without a thorough analysis, there is no way for us to know. Therefore, claiming it's the stack is just as valid or invalid as claiming it's the custom software.

            In any event, this is not an either/or situation. Microsoft did not write the TradeElect software, a third party did... Accenture.

            • Yes we can't really know the exact nature. However the main problem with comparison is the sample size here. I don't know of any other exchange that uses Windows as trading platform. I would say that no one else has even built one on Windows or that Windows is typically not used in cases of high performance is more of an indicator. It's kinda like this: Black & Decker is big name in tools; however, professional contractors don't use that brand as much as they use others like DeWalt. It's not Black
              • by mangu (126918)

                It's interesting that you use power tools as an analogy. I once made a presentation to managers in my company on "Why Linux?" where I had the following analogy: Windows XP on personal computers is like Black & Decker drills, cheap and easy to use, found everywhere. Windows server is like Makita drills, sturdier, more professional, and more expensive, but very much like an amateur tool on steroids.

                Linux is like an air drill [google.com]. Most people aren't even aware that such thing exists. But it's the only solution

      • Frustrations mount over London Stock Exchange price data problems http://www.computerworlduk.com/news/it-business/3261625/frustrations-mount-over-london-stock-exchange-data-interface-problems/ [computerworlduk.com]
  • by michelcolman (1208008) on Thursday February 17, 2011 @03:07AM (#35229616)
    Imagine telling a trader in the 1970s that we had a 42 second outage in the stock market, it was all over the news, and a few companies probably lost hundreds of millions in revenue.
    • by haploc (57693)

      Imagine telling a trader in the 1970s that we had a 42 second outage in the stock market, it was all over the news, and a few companies probably lost hundreds of millions in revenue.

      At least it would be The Answer to.. !

    • Was the consequences of the loss of the automatic trade?

      Everybody won? :D

    • by ignavus (213578)

      Imagine telling a trader in the 1970s that we had a 42 second outage in the stock market, it was all over the news, and a few companies probably lost hundreds of millions in revenue.

      "Hi, 1970s trader! I am just going to skip over the momentous fact that I have accomplished time travel - the ultimate cool technology - to let you know that one day people will be even more obsessive about the stock market than they are in the 1970s.

      Thought you'd like to know. Oh, and in case you are interested, I can travel through time. Now going back to talk to HG Wells about a great book idea I have."

  • Liffe[sic] the universe and everything.

    • by DamonHD (794830)

      LIFFE != LSE

      • by Luyseyal (3154)

        Restate my assumptions:

        One, Mathematics is the language of nature.

        Two, Everything around us can be represented and understood through numbers.

        Three: If you graph the numbers of any system, patterns emerge. Therefore, there are patterns everywhere in nature.

        Evidence: The cycling of disease epidemics;the wax and wane of caribou populations; sun spot cycles; the rise and fall of the Nile. So, what about the stock market? The universe of numbers that represents the global economy. Millions of hands at work, billions of minds. A vast network, screaming with life. An organism. A natural organism. My hypothesis: Within the stock market, there is a pattern as well... Right in front of me... hiding behind the numbers. Always has been.

        — Maximillian Cohen, Pi [imdb.com]

  • by PolygamousRanchKid (1290638) on Thursday February 17, 2011 @03:32AM (#35229756)

    I've seen this too often in my 25+ professional years in IT. The system test manager produces an excellent plan, that fully simulates the anticipated workload. But it requires X testers, Y test case developers and Z machines. The program manager rejects the plan, "because he is under pressure to reduce costs." The program manager says, "The testing that the developers do should be enough." He then moves on, before the system goes into production.

    The result? It always ends in tears.

    • by daid303 (843777) on Thursday February 17, 2011 @07:49AM (#35230760)

      Testing is just hard. Even with a perfect plan, and a perfect execution you're still going to miss things.

      I just spend 2 weeks finding a 6 year old bug, which only showed itself when the temperature of our product is somewhere at -5C, and then only on 75% of the total devices. Devices where tested at low as -40C, but those where lucky and kept working correctly, so nobody noticed before. Even after I found it I spend a week looking at the code to find what's really wrong.

    • Do you really think with a project this large and crucial, that the people in charge would willingly skimp on testing? I mean shit, this isn't BP we're talking about here. I'd imagine this is a pretty huge system. I'm willing to bet that whatever caused this outage was just simply overlooked when developing the test plan.
  • -- sleep(life_the_universe_and_everything);
  • 4:30. What were the sys admins doing 10 minutes before this ? ;-)

  • by molecular (311632)

    well there we have an answer to a popular question

Old programmers never die, they just branch to a new address.

Working...