twitter writes "The US Postal Service has moved its Cobol package tracking software to HP machines running GNU/Linux. 1,300 servers handle 40 million transactions a day and cost less than the last system, which was based on a Sun Solaris environment." The migration took a year. The USPS isn't spelling how big the savings are, except that they are "significant."
Really? GNU/Linux? How do we know it's not really Ruby/Apache/X.org/KDE/GNU/Linux? The article just says "Linux environment." It could very well be BSD/Linux instead of GNU/Linux.
But I digress. HP generally uses Red Hat Linux. To be semantically correct the summary should have read "Postal Service moves to Red Hat Enterprise Linux on HP Hardware"
Actually, I think that would be Ruby/Apache/KDE/QT/X.org/GNU/Linux or PHP/Apache/KDE/QT/X.org/GNU/Linux - but depending on whether they're using it for backend or desktop service (sorry, too lazy to RTFA) I'd think that either [Ruby|PHP]/Apache/GNU/Linux, or OO.org/[KDE|GNOME|XFCE]/[QT]/X.org/GNU/Linux would be more appropriate... but if they're using it for both desktop and server, that would make it
or the bogus inflation numbers the feds like to publish?
Almost certainly the "bogus" number, though every criticism I've ever heard of it claims that it is too low and that the real cost of living increase is much higher... so that would tend to make the USPS even more of a bargain.
Price them out vs. their competition and they are pretty darned competitive for a big government bureaucracy... though it is hard to compare them directly since the USPS will not guarantee a delivery date.
I paid more for a first class stamp from the Bundespost in Germany in 1976 than I pay in the U.S. today, and the service is better. The USPS is a bargain and it's better managed than people give it credit for.
There are good reasons why it is illegal to compete with the USPS. The Postal Service has requirements that other competitors do not. First, it has to offer first class service to all fifty states, even Alaska, Hawaii, and other unprofitable places. Second, it has to charge exactly the same rates domestically regardless of the distance. Third, you can mail an envelope you labeled by hand whereas UPS/FedEx would require you to barcode it.
If you allowed private enterprises to compete unchecked, they would cherrypick the most profitable routes (hubs, basically) and quickly bankrupt the Post Office. They'd also charge less than the Post Office on short routes that the Post Office would need to subsidize the longer routes. But if you had to regulate competitors to make sure they had the same disadvantages as the Post Office, what's the point?
While I'm no fan of rate hikes, getting any sort of physical entity across the country in a couple of days for under fifty cents is pretty much a modern miracle.
Yes, but do you have enough letters to be delivered from Kearny, NJ to Fresno, CA to fill that box? If not, how are you going to sort/redirect the individual letters that don't fit in conveniently-sized bins?
The USPS sorting infrastructure is just as (if not more) impressive than the actual shipping infrastructure.
They moved their package tracking system to Linux? I wonder if, when you ask it where your shipment is, it will tell you to find it yourself in a condescending manner.
When you request the location of your package, it just sneers at you and says "Google is your friend."
That's actually true.
Type/paste a tracking number from any of the major shippers into google and it will automagically figure out that is a tracking number and will show you the current status.
That was uncalled for. A Linux user will ask you very politely if the package bar code was code128 or some other zebra coding technology. Someone will pipe in that back in his day, there were no barcoded ZIPs, just hand written numbers written in brown crayon on a cardboard box. Someone else will chime in that back in his day, you were lucky if it had the country on it, much less a ZIP code. Someone else will tell you that UPS uses a system called PLD and you need to look at the 1Z label code and direct you to ups.com. Someone will call that person an idiot and say that USPS is not UPS. Someone else will ask, "Why are you trying to track your package? Tell us what you really want to accomplish."
(I kid, I kid. I'm a Linux user through and through.)
1300 servers, processing 40 million transactions a day... that's about 30,800 transactions per server. Or one transaction every 2.8 seconds or so. With an entire Linux box dedicated to it.
I work in the scan processing group at FedEx. At peak, we see over 100,000,000 transactions a day. And that's handled on 45 linux boxes, and 12 more for the database, doing upwards of 6000 transactions per second during bursts. That's a peak of about 133 transactions per second, per box. That's a little better than 0.3 TPS for the Post Office. So we have about 400 times the performance with 5% of the hardware. By that margin, I could do their processing with about 25 boxes total. That would mean another 98% savings on hardware alone.
For some reason, I fail to be really impressed that they've gone from "Crappy performance and Expensive" to "Crappy performance and less expensive."
I wonder if I can get the bazillion dollar contract to rewrite their system... No, wait, my name isn't "Boeing" or "Lockheed" or Ken Murtha.
Yeah, but how many post offices are there? Doesn't each post office need one machine to talk to the main cluster?
And do people mail stuff 24 hours per day, or is there a rush hour? Where everything spikes 10x as high?
Do these servers have to do any of that optical character recognition crap to figure out where to mail stuff, or is that handled by whatever company designed that part of the system?
There's plenty of valid reasons for why they *might* need that many servers. It could even be preparations for Christmas. Maybe they keep half of them in reserve for when they're needed?
I'm a system analyst, and I support the hardware distribution that goes into this.
This system doesn't have to help get the mail there, it just has to report when it's arrived. Mail processing is done by large machines in localized distribution centers, and then shipped to the local post offices where they get it into the hands of the carrier on the route for that day.The data entry is handled by hand held scanners that upload the data back to the LIM (local server). The central computers are most likely in one of three spots (I'm not sure, I don't know how they're configured, I just support the system they're ordered through) either in Eagan, Raleigh, or St. Louis. At least, that's where I know of major server locations in the USPS. But I'm not sure the information is sent to the local servers wirelessly. I know that the local locations have USB cradles, so I assume the data is kept on the handheld unit all day, then uploaded when the carrier returns to the office, but that's just a guess. If that's the case, the machine sits idle most of the day, then runs most of it's work in the hour or two when the carriers return to the office in a batch run.
o we have about 400 times the performance with 5% of the hardware. By that margin, I could do their processing with about 25 boxes total. That would mean another 98% savings on hardware alone.
The FedEx system doesn't handle hand written letters though, does it? You have to do a shipping label for most (all?) packages, with a digital bar code. USPS runs some very powerful OCR systems; maybe they're making the transactions so expensive. Just a thought.
Actually, FedEx still gets a lot of hand-written, photocopied or otherwise POS labels all the time. Just like the post office, they are encoded with a bar code at the first station that gets the package and are then read optically by a computer from then on. Or did you think those little squiggles on the bottom of your postal envelope (Postal Bar/Line code) were for decoration?
FedEx has a whole bunch of people whose only job is to look at scanned images of labels and type in the actual address when the machines fail to read them.
And I'm not in the part that deals with all that. That's the Tracker hardware. We just get the 60-100M scans per day, turn them from an isolated event into a "package" and then forward that information on to billing and the web interface system for tracking (and another dozen downstream systems that work with the data.) We also get fed information from about half a dozen other systems (like delay information, if there's an accident or a storm that grounds the airplanes) and use all of it to predict delivery dates for the packages. We also process point of failure information and information on commitment dates (the date that your package becomes free if we don't get it there) and containerization and consolidation information (i.e. what packages are in that bag, that got packed into that shipping container that got loaded onto that airplane, etc.) so we can pass scans done on a container down to all of the packages in that container. That's about another 20 million plus events per day.
A typical package traveling in the system ends up with 20-30 events that occur on it. Some end up with 80 or more. There's a lot more that goes on too, like clearance information, and multiple-piece shipments, COD information, and so on and so on. All of it goes through our system. With 45 boxes (actually we just bumped up to 50 with the June updates) and 12 database boxes. All HP boxes running Red Hat.
Well, you make a couple of assumptions that may or may not be valid:
1) you assume the "transactions" the USPS is doing are equivalent to the scanning FedEx does. We don't know what these "transactions" are - they could be tracking requests, they could be scanning letters, or something else.
2) FedEx can pretty much guarantee that all items start out with a valid barcode, while USPS cannot - they have to be able to handle a large number of envelopes that have nothing but a handwritten address on them: no bar code, no machine printed labels, just hand-printed (or handwritten cursive) labels. That takes quite a bit more processing.
As I mentioned above in reply to another post, we still get hand-written airbills, airbills that have been photo-copied (Gee, twenty packages, all with the same tracking number...) and you can still find airbills in the system that are total crap. Try tracking the package "444444444444" sometime... (Hey, we filtered out "TWELVEZEROES" and "NO_NUM_GIVEN")
Our transactions are things like "contract event" when you fill in an airbill on-line, pickup scans, revenue data, station outbound, ramp inbound, ramp outbound, hub inbound, revenue exceptions, hub outbound, station inbound, on-van, delivery, proof-of-delivery, package close, point-of-failure processing, etc. etc. We have over 70 types of scans, and a typical package ends up with 20-30 events on it. If you think you can see anything when you track a package, you have no idea how much more goes into it behind the scenes. Every one of those scans is 2-3K in length, these aren't just a simple 20 byte "ping" or something. And we retain all that data for nine months.
And we still do 133 transactions per second per server.
And always remember that the USPS contracts out to FedEx to move all of its "Priority Mail".
cost less than the last system, which was based on a Sun Solaris environment.
Two thoughts:
This seems to be where Linux's strength is - replacing proprietary Unix.
How lame does the Sun salesman have to be? He couldn't get the USPS to replace their Sun boxes with Linux Sun boxes (Sun makes a complete line of x86 kit that runs Linux). Instead they went to HP. There's precious little difference between an HP x86 box and a Sun x86 box....all I can think is how lame the Sun salesman must have been.
Will this allow them to improve their tracking system?
UPS has had an amazing tracking system for years. FedEx has improve theirs, to the point they have good estimated delivery dates and can show you what's going on with your package pretty well, like UPS.
When dealing with the post office, their system works but is... antiquated. When paying for any kind of fast shipping (overnight, two day) I can receive my package before the tracking number pulls up a package. It's not every time, but it's enough to make me not care much. What I really care about is estimated delivery dates. I want to know when I'll get my package. I usually don't care if my package is in NYC, Duluth, or San Antonio. It will get where it needs to go. I would rather have the step-by-step tracking information show up later and have things like estimated TOA show up fast.
I remember as a kid (I'm 26) you could order something from a catalog and you had basically no idea when it would show up from UPS, etc. Today I can find out where my UPS package was last scanned, nearly up to the minute. Very cool.
That would make a neat visualization. They should put that on their site, little packages moving along their correct routes around the country.
I think the summary and the TFA are a bit confusing. It probably would have been better had the summary linked not just to the TFA but also directly to the gcn.com article [gcn.com] which is linked to from TFA, or maybe better yet, just to the gcn.com article. What I get from that gcn.com article is this:
a) There is a mainframe that is talking to ~1300 "midrange" servers.
b) The mainframe is an IBM Z-series which has been shifted over from an IBM proprietary OS to Novell SUSE Linux.
c) The COBOL code is running on the *mainframe*, not the ~1300 servers! (TFA summary is wrong on this)
d) Because the mainframe is now running Linux, and because of a USPS IT decision to standardize on Linux (this is why OpenSolaris was never an option - sorry OpenSolaris fans), they're now converting the servers to Linux as well for better interoperability between the mainframe environment and server environments.
e) As for what this system is actually tracking:
Events are transactions that occur at the service's retail counters, such as shipping and picking up packages or the delivery of priority mail by carriers to businesses and residences. The mail is scanned to confirm delivery, and that information is sent to the PTS database....
âoeWe're inserting like 40 million events a day,â he added....
The PTS has 56 transaction types, such as acceptance scans and delivery confirmations, that have now all been migrated to Linux.
The gcn.com article has more info, but even it is confusing to me. Questions:
What is an "HP Linux Environment" (Does HP have its own version of Linux? What distro is HP using?)
Any Z-series gurus reading this want to chime in and explain what the IFL actually is (Page 2 of article)?
Yes, I know, I could Google for those answers, but I'm already worn out just doing what the original story submitter should have done. Just consider the above an "improved summary".:)
Have you ever coded in COBOL? I have. It is EXTREMELY easy. It is quite close to English and is not at all cryptic. I believe nearly any coder with any experience and a language reference guide can read through code and make changes were needed. It has been almost 10 years since I last wrote a PIC statement, but I am quite confident that not only I, but just about anyone could do it. While I think the stories about pulling old programmers out of mothballs (retirement) is rather heartening, I think they are blowing the problem out of proportion. What these companies should be doing is hiring experienced and mature coders who can learn COBOL then send them to school.
What I find disheartening is the fact that businesses are no longer able to see education and training of employees as a worthwhile investment. (I know why they probably don't see it as worthwhile and it has a lot to do with employee loyalty, but I have to insist that the problem of loyalty didn't really happen until employers started treating their employees as disposable... they have no qualms with firing and laying off people at-will and yet they expect employees to be loyal? Get real!)
I have to agree. When I was 19 and interning in the accounting department at a chemical plant, I was asked to add a new field to one of their systems because I knew FORTRAN. I looked at their code, told them that it was not FORTRAN, but I thought I understood what was going on. They said fine. After a couple of hours of monkey-see-monkey-do I had made the change and verified that the field showed up on the screen that they wanted it on, saved the values as expected and showed up in the modified reports where they wanted. It was not until over 20 years later that I looked inside a COBOL book and realized that I had been unknowingly 'tainted'. It was that easy.
they could still reap even more benefits by recoding for modern languages and coding practices
Maybe. The fact is it appears they successfully migrated the system to a new platform within a year. I have seen many "modern" systems still jerking around with UML after a year and I can't count how many were never brought fruition.
But then, this is the US Postal Service. COBOL's probably fast enough for the task.
COBOL has a lot of issues but speed isn't a big one. I'm willing to bet that on tasks that are appropriate to COBOL it would kick most "modern" scripting languages asses in terms of speed.
COBOL runs quite fast; it is a compiled language that was designed from the start for compiler optimization of business logic code. Think in terms of well written C.
COBOL's reputation for being slow has to do with writing the programs, and dates back to its earliest days when code had to be written as SET RIDICULOUSLYLONGVARIABLENAME EQUAL TO 2 PLUS 2. Development time sped up a lot when Gracie was finally convinced that RIDICULOUSLYLONGVARIABLENAME = 2 + 2 would work as well (the original approach had to do with coders handprinting the instructions on coding forms that were then given to keypunch operators who knew nothing about computer languages but could type really fast and accurately... so long as their fingers didn't have to reach too far from the home keys).
I was never found of COBOL in school, and never did much with it. But I agree with others that anyone who has worked with a couple of programming languages could probably master COBOL in a matter of hours: in retrospect, it is basically a simple and obvious language. What I probably could not do is reverse engineer some of the constructs that were used back in the day: a lot of weird code was developed to work around hardware limitations that no one born after 1980 will have ever seen. Much of that wasn't documented well, and I think that is why old COBOL programmers are in demand. They remember that this particular type of data structure was used to optimize read-write operations on dual high speed tapes... to a younger programmer it just looks like arbitrary stupidity and they don't know what to do with it.
Sure, so long as you never need to make any changes to the code. The surviving COBOL coders have gone back into comfortable retirement with the money they made fixing Y2K. So they've moved from old iron to a modern operating system; they could still reap even more benefits by recoding for modern languages and coding practices.
But then, this is the US Postal Service. COBOL's probably fast enough for the task.
So you're saying that COBOL is so hideously difficult, so byzantine, so labyrinthine in nature that no one could possibly learn it now? That programmers educated today have no possibility of understanding a language that was designed some decades ago? You realize that C is 30 years old now, right?
This sort of fear mongering through ignorance is getting stale. COBOL is just another language, and one that happened to be designed for ease of expression for less-than-stellar programmers. Legions of students have learned enough C over a weekend to code up the examples in K&R, so I'm actually quite confident that professional programmers can, without any prior experience in COBOL, learn the language, even become proficient in it, in a brief enough time to make modifications to existing code bases.
Look, we're talking about learning a computer language and modifying or maintaining code, not learning Elizabethan English well enough to write a new Shakespeare play that can pass off as an original. It just isn't that hard.
I'm sure I could get a dramatic speed improvement running Apple II 6502 code on an emulator on a Mac Pro simply because the emulator can run faster than the original hardware.
Given that it took 1400 Linux boxes to handle the load, I'd say your post is, at best ignorant, at worst, a blatant troll.
a) Just because it's COBOL, doesn't mean it was running on crappy hardware.
b) COBOL is far from dead, in that many applications running today are written in it. Believe it or not, it makes more sense to continue to run that old code than to rewrite from scratch in the latest shiny because they already know *it works*.
What are you whining about? The cost of postage has historically risen at a lower rate than inflation. Meaning that stamps do cost less, just not in face value.
Not only that, I just checked and according to fedex it costs $7.39 to mail that same letter from coast to coast for their cheapest option. That's only what, nearly 17 times more expensive? Travel times are 5 days compared to about 7 for the USPS, not much faster. I'm sure the libertarians will chime in that they could do that much cheaper if the (subsidized) USPS weren't in the way, but I suspect it would be like the way that CD prices went down after the technology became established, or the way that cable and telephone prices went down after the markets were deregulated (i.e., they didn't). Bottom line is that the USPS is an astonishingly inexpensive with a low failure rate for the price. It's a great service that our government provides. While I'm glad that they are saving this money, I'd rather that they put it to work on avoiding reductions in service or balancing their budget rather than reducing the price of postage.
"Not only that, I just checked and according to fedex it costs $7.39 to mail that same letter from coast to coast for their cheapest option. That's only what, nearly 17 times more expensive?"
factor in how much of your tax dollars when into that and then get back to us with a valid point....
Umm, the USPS is self-funded. None of your tax dollars go towards supporting their operation source [usps.com]
Umm, the USPS is self-funded. None of your tax dollars go towards supporting their operation source
That's a little misleading - it hasn't always been that way, so a lot of the USPS infrastructure is tax-payer funded. In addition, they come around every once and a while and ask for money from Congress - they are doing it this year [infowars.com] and while I am hazy on the details, I believe they did something similar about a decade ago in order to fix funding problems with their pension system. Plus, they have a monopoly on letter delivery - that's why fedex costs so much more, they have to classify and price it as something other than a letter - so that's an indirect tax by government intervention to prevent a free market.
by Anonymous Coward
on Tuesday July 14, @08:11PM (#28698751)
Wait... fedex costs more because USPS has a monopoly? Lets see... what other monopolies are out there to compare to this statement. Nope, does not compute.
Some people do not like the USPS because it represents an actual successful arm of the government. You want to know why they ask congress for money? Its to get back what Congress ultimately takes from the USPS because its the only thing besides taxes that makes money in our government system.
I say YAY USPS, one of the few "companies" where you dont have to worry about your pension, retirement, or jobs.
Let me use a "scare word" for those of you whom are still stuck in the 1980s. "Socialism" can work.
Yeah it's totally insane that we are charged a whole $0.44 to reliably send any piece of paper over 3,000 miles to it's precise recipient in a matter of days. This is the kind of technological marvel that future societies will be looking back in awe of.
You're really complaining about $.50 for the level of service you get from the USPS? For that price, you can send a standard letter anywhere in the US (including the non-continental US) usually arriving in less than 5 days with a loss rate of virtually zero. They deliver mail to (nearly) every address in the US 6 days a week, and will even come to check for outgoing if you don't have any incoming. They even manage to deliver when the roads are absolute shit and no one in their right mind would be out and about.
All for a price that has actually been decreasing over the years if you take into account inflation, let alone the increases in gas prices that have occurred over the last 10 years. Personally, I think that's pretty damn good and wouldn't complain if they raised the price to an even dollar, it would still be under priced for the service they provide.
Last week on Top Gear, they raced a standard letter sent via standard post in the UK from the south of the UK to the far north of the UK and the letter won.
Total cost of the stamp? A fraction of a pound.
The US is very similar. A little slower due to the extreme distances mail has to route to, but, i'd wager on mail versus delivering it yourself anyday. Not only that it's *cheap*
For once ... (Score:5, Funny)
Re:For once ... (Score:5, Funny)
Parent
Re:For once ... (Score:5, Funny)
No, and I'm sure there will be some people wanting to know if their mail is going to be delivered by Beowulf Cluster.
I, for one, welcome our mail-delivering, Beowulf Cluster overlords.
Parent
Re:For once ... (Score:5, Funny)
As long as my mail doesn't come covered in hot grits...
Would you object if it came wrapped in Natalie Portman's panties?
Parent
Mail server (Score:5, Funny)
Well, look on the bright side...
Now we can say, with all confidence, that the world's largest mail server runs Linux
Parent
Re:For once ... (Score:5, Informative)
Really? GNU/Linux? How do we know it's not really Ruby/Apache/X.org/KDE/GNU/Linux? The article just says "Linux environment." It could very well be BSD/Linux instead of GNU/Linux.
But I digress. HP generally uses Red Hat Linux. To be semantically correct the summary should have read "Postal Service moves to Red Hat Enterprise Linux on HP Hardware"
Parent
Re:For once ... (Score:5, Funny)
Ruby/Apache/X.org/KDE/GNU/Linux
Actually, I think that would be Ruby/Apache/KDE/QT/X.org/GNU/Linux or PHP/Apache/KDE/QT/X.org/GNU/Linux - but depending on whether they're using it for backend or desktop service (sorry, too lazy to RTFA) I'd think that either [Ruby|PHP]/Apache/GNU/Linux, or OO.org/[KDE|GNOME|XFCE]/[QT]/X.org/GNU/Linux would be more appropriate... but if they're using it for both desktop and server, that would make it
And as soon as RSF starts calling it that, I'll start calling it GNU/Linux :)
Parent
Re:For once ... (Score:4, Informative)
Having interviewed for a position at the postal training center, they used PHP on Apache and Solaris for the OS (Oracle for the DB).
I'm not sure which training location they are talking about but one of the main ones (Postal Training Center is in Norman, OK).
Parent
A year? (Score:4, Insightful)
that's pretty damn good time to move a system.
Now f they could drop tues, thurs and sat mail service they would save a bundle.
Re:A year? (Score:4, Informative)
Parent
Re:A year? (Score:4, Insightful)
or the bogus inflation numbers the feds like to publish?
Almost certainly the "bogus" number, though every criticism I've ever heard of it claims that it is too low and that the real cost of living increase is much higher... so that would tend to make the USPS even more of a bargain.
Price them out vs. their competition and they are pretty darned competitive for a big government bureaucracy... though it is hard to compare them directly since the USPS will not guarantee a delivery date.
Parent
Re:A year? (Score:5, Interesting)
Parent
Re:A year? (Score:5, Insightful)
There are good reasons why it is illegal to compete with the USPS. The Postal Service has requirements that other competitors do not. First, it has to offer first class service to all fifty states, even Alaska, Hawaii, and other unprofitable places. Second, it has to charge exactly the same rates domestically regardless of the distance. Third, you can mail an envelope you labeled by hand whereas UPS/FedEx would require you to barcode it.
If you allowed private enterprises to compete unchecked, they would cherrypick the most profitable routes (hubs, basically) and quickly bankrupt the Post Office. They'd also charge less than the Post Office on short routes that the Post Office would need to subsidize the longer routes. But if you had to regulate competitors to make sure they had the same disadvantages as the Post Office, what's the point?
Parent
Re:A year? (Score:5, Insightful)
While I'm no fan of rate hikes, getting any sort of physical entity across the country in a couple of days for under fifty cents is pretty much a modern miracle.
Parent
Re:A year? (Score:5, Insightful)
Yes, but do you have enough letters to be delivered from Kearny, NJ to Fresno, CA to fill that box? If not, how are you going to sort/redirect the individual letters that don't fit in conveniently-sized bins?
The USPS sorting infrastructure is just as (if not more) impressive than the actual shipping infrastructure.
Parent
Find It Yourself (Score:5, Funny)
They moved their package tracking system to Linux? I wonder if, when you ask it where your shipment is, it will tell you to find it yourself in a condescending manner.
Re:Find It Yourself (Score:5, Funny)
When you request the location of your package, it just sneers at you and says "Google is your friend."
Parent
Re:Find It Yourself (Score:5, Informative)
When you request the location of your package, it just sneers at you and says "Google is your friend."
That's actually true.
Type/paste a tracking number from any of the major shippers into google and it will automagically figure out that is a tracking number and will show you the current status.
Parent
Re:Find It Yourself (Score:5, Insightful)
That was uncalled for. A Linux user will ask you very politely if the package bar code was code128 or some other zebra coding technology. Someone will pipe in that back in his day, there were no barcoded ZIPs, just hand written numbers written in brown crayon on a cardboard box. Someone else will chime in that back in his day, you were lucky if it had the country on it, much less a ZIP code. Someone else will tell you that UPS uses a system called PLD and you need to look at the 1Z label code and direct you to ups.com. Someone will call that person an idiot and say that USPS is not UPS. Someone else will ask, "Why are you trying to track your package? Tell us what you really want to accomplish."
(I kid, I kid. I'm a Linux user through and through.)
Parent
Score 0: Trite (Score:5, Funny)
Submitted with the headline "Linux Penguin goes postal."
Postfix! (Score:5, Funny)
Get off my lawn (Score:5, Funny)
If you're 30-something, you rely on email.
If you're in your 20s, you use IM
If you're 13 like me, it's all Twitter, all the time. Bonus: I have no need to receive packages because I shoplift everything.
Boy, what efficiency... (Score:4, Interesting)
1300 servers, processing 40 million transactions a day... that's about 30,800 transactions per server. Or one transaction every 2.8 seconds or so. With an entire Linux box dedicated to it.
I work in the scan processing group at FedEx. At peak, we see over 100,000,000 transactions a day. And that's handled on 45 linux boxes, and 12 more for the database, doing upwards of 6000 transactions per second during bursts. That's a peak of about 133 transactions per second, per box. That's a little better than 0.3 TPS for the Post Office. So we have about 400 times the performance with 5% of the hardware. By that margin, I could do their processing with about 25 boxes total. That would mean another 98% savings on hardware alone.
For some reason, I fail to be really impressed that they've gone from "Crappy performance and Expensive" to "Crappy performance and less expensive."
I wonder if I can get the bazillion dollar contract to rewrite their system... No, wait, my name isn't "Boeing" or "Lockheed" or Ken Murtha.
Re:Boy, what efficiency... (Score:5, Interesting)
Yeah, but how many post offices are there? Doesn't each post office need one machine to talk to the main cluster?
And do people mail stuff 24 hours per day, or is there a rush hour? Where everything spikes 10x as high?
Do these servers have to do any of that optical character recognition crap to figure out where to mail stuff, or is that handled by whatever company designed that part of the system?
There's plenty of valid reasons for why they *might* need that many servers. It could even be preparations for Christmas. Maybe they keep half of them in reserve for when they're needed?
Parent
Re:Boy, what efficiency... (Score:5, Informative)
I'm a system analyst, and I support the hardware distribution that goes into this.
This system doesn't have to help get the mail there, it just has to report when it's arrived. Mail processing is done by large machines in localized distribution centers, and then shipped to the local post offices where they get it into the hands of the carrier on the route for that day.The data entry is handled by hand held scanners that upload the data back to the LIM (local server). The central computers are most likely in one of three spots (I'm not sure, I don't know how they're configured, I just support the system they're ordered through) either in Eagan, Raleigh, or St. Louis. At least, that's where I know of major server locations in the USPS. But I'm not sure the information is sent to the local servers wirelessly. I know that the local locations have USB cradles, so I assume the data is kept on the handheld unit all day, then uploaded when the carrier returns to the office, but that's just a guess. If that's the case, the machine sits idle most of the day, then runs most of it's work in the hour or two when the carriers return to the office in a batch run.
Parent
Re:Boy, what efficiency... (Score:5, Funny)
Maybe their servers are in the union?
Parent
Re:Boy, what efficiency... (Score:5, Interesting)
Parent
Re:Boy, what efficiency... (Score:5, Informative)
Actually, FedEx still gets a lot of hand-written, photocopied or otherwise POS labels all the time. Just like the post office, they are encoded with a bar code at the first station that gets the package and are then read optically by a computer from then on. Or did you think those little squiggles on the bottom of your postal envelope (Postal Bar/Line code) were for decoration?
FedEx has a whole bunch of people whose only job is to look at scanned images of labels and type in the actual address when the machines fail to read them.
And I'm not in the part that deals with all that. That's the Tracker hardware. We just get the 60-100M scans per day, turn them from an isolated event into a "package" and then forward that information on to billing and the web interface system for tracking (and another dozen downstream systems that work with the data.) We also get fed information from about half a dozen other systems (like delay information, if there's an accident or a storm that grounds the airplanes) and use all of it to predict delivery dates for the packages. We also process point of failure information and information on commitment dates (the date that your package becomes free if we don't get it there) and containerization and consolidation information (i.e. what packages are in that bag, that got packed into that shipping container that got loaded onto that airplane, etc.) so we can pass scans done on a container down to all of the packages in that container. That's about another 20 million plus events per day.
A typical package traveling in the system ends up with 20-30 events that occur on it. Some end up with 80 or more. There's a lot more that goes on too, like clearance information, and multiple-piece shipments, COD information, and so on and so on. All of it goes through our system. With 45 boxes (actually we just bumped up to 50 with the June updates) and 12 database boxes. All HP boxes running Red Hat.
Parent
Re:Boy, what efficiency... (Score:4, Interesting)
Well, you make a couple of assumptions that may or may not be valid:
1) you assume the "transactions" the USPS is doing are equivalent to the scanning FedEx does. We don't know what these "transactions" are - they could be tracking requests, they could be scanning letters, or something else.
2) FedEx can pretty much guarantee that all items start out with a valid barcode, while USPS cannot - they have to be able to handle a large number of envelopes that have nothing but a handwritten address on them: no bar code, no machine printed labels, just hand-printed (or handwritten cursive) labels. That takes quite a bit more processing.
Parent
Re:Boy, what efficiency... (Score:5, Informative)
As I mentioned above in reply to another post, we still get hand-written airbills, airbills that have been photo-copied (Gee, twenty packages, all with the same tracking number...) and you can still find airbills in the system that are total crap. Try tracking the package "444444444444" sometime... (Hey, we filtered out "TWELVEZEROES" and "NO_NUM_GIVEN")
Our transactions are things like "contract event" when you fill in an airbill on-line, pickup scans, revenue data, station outbound, ramp inbound, ramp outbound, hub inbound, revenue exceptions, hub outbound, station inbound, on-van, delivery, proof-of-delivery, package close, point-of-failure processing, etc. etc. We have over 70 types of scans, and a typical package ends up with 20-30 events on it. If you think you can see anything when you track a package, you have no idea how much more goes into it behind the scenes. Every one of those scans is 2-3K in length, these aren't just a simple 20 byte "ping" or something. And we retain all that data for nine months.
And we still do 133 transactions per second per server.
And always remember that the USPS contracts out to FedEx to move all of its "Priority Mail".
Parent
Typical (Score:5, Interesting)
cost less than the last system, which was based on a Sun Solaris environment.
Two thoughts:
Improve tracking? (Score:5, Interesting)
Will this allow them to improve their tracking system?
UPS has had an amazing tracking system for years. FedEx has improve theirs, to the point they have good estimated delivery dates and can show you what's going on with your package pretty well, like UPS.
When dealing with the post office, their system works but is... antiquated. When paying for any kind of fast shipping (overnight, two day) I can receive my package before the tracking number pulls up a package. It's not every time, but it's enough to make me not care much. What I really care about is estimated delivery dates. I want to know when I'll get my package. I usually don't care if my package is in NYC, Duluth, or San Antonio. It will get where it needs to go. I would rather have the step-by-step tracking information show up later and have things like estimated TOA show up fast.
I remember as a kid (I'm 26) you could order something from a catalog and you had basically no idea when it would show up from UPS, etc. Today I can find out where my UPS package was last scanned, nearly up to the minute. Very cool.
That would make a neat visualization. They should put that on their site, little packages moving along their correct routes around the country.
Summary not quite right... (Score:5, Informative)
I think the summary and the TFA are a bit confusing. It probably would have been better had the summary linked not just to the TFA but also directly to the gcn.com article [gcn.com] which is linked to from TFA, or maybe better yet, just to the gcn.com article. What I get from that gcn.com article is this:
a) There is a mainframe that is talking to ~1300 "midrange" servers.
b) The mainframe is an IBM Z-series which has been shifted over from an IBM proprietary OS to Novell SUSE Linux.
c) The COBOL code is running on the *mainframe*, not the ~1300 servers! (TFA summary is wrong on this)
d) Because the mainframe is now running Linux, and because of a USPS IT decision to standardize on Linux (this is why OpenSolaris was never an option - sorry OpenSolaris fans), they're now converting the servers to Linux as well for better interoperability between the mainframe environment and server environments.
e) As for what this system is actually tracking:
Events are transactions that occur at the service's retail counters, such as shipping and picking up packages or the delivery of priority mail by carriers to businesses and residences. The mail is scanned to confirm delivery, and that information is sent to the PTS database. ...
âoeWe're inserting like 40 million events a day,â he added. ...
The PTS has 56 transaction types, such as acceptance scans and delivery confirmations, that have now all been migrated to Linux.
The gcn.com article has more info, but even it is confusing to me. Questions:
What is an "HP Linux Environment" (Does HP have its own version of Linux? What distro is HP using?)
Any Z-series gurus reading this want to chime in and explain what the IFL actually is (Page 2 of article)?
Yes, I know, I could Google for those answers, but I'm already worn out just doing what the original story submitter should have done. Just consider the above an "improved summary". :)
Re:Sure, runs on GNU/Linux (Score:5, Insightful)
Your point?
There isn't anything wrong with COBOL for these kind of transactions.
Parent
Re:Sure, runs on GNU/Linux (Score:5, Interesting)
Have you ever coded in COBOL? I have. It is EXTREMELY easy. It is quite close to English and is not at all cryptic. I believe nearly any coder with any experience and a language reference guide can read through code and make changes were needed. It has been almost 10 years since I last wrote a PIC statement, but I am quite confident that not only I, but just about anyone could do it. While I think the stories about pulling old programmers out of mothballs (retirement) is rather heartening, I think they are blowing the problem out of proportion. What these companies should be doing is hiring experienced and mature coders who can learn COBOL then send them to school.
What I find disheartening is the fact that businesses are no longer able to see education and training of employees as a worthwhile investment. (I know why they probably don't see it as worthwhile and it has a lot to do with employee loyalty, but I have to insist that the problem of loyalty didn't really happen until employers started treating their employees as disposable... they have no qualms with firing and laying off people at-will and yet they expect employees to be loyal? Get real!)
Parent
Re:Sure, runs on GNU/Linux (Score:5, Interesting)
Parent
Re:Sure, runs on GNU/Linux (Score:5, Interesting)
they could still reap even more benefits by recoding for modern languages and coding practices
Maybe. The fact is it appears they successfully migrated the system to a new platform within a year. I have seen many "modern" systems still jerking around with UML after a year and I can't count how many were never brought fruition.
But then, this is the US Postal Service. COBOL's probably fast enough for the task.
COBOL has a lot of issues but speed isn't a big one. I'm willing to bet that on tasks that are appropriate to COBOL it would kick most "modern" scripting languages asses in terms of speed.
Parent
Re:Sure, runs on GNU/Linux (Score:5, Interesting)
COBOL runs quite fast; it is a compiled language that was designed from the start for compiler optimization of business logic code. Think in terms of well written C.
COBOL's reputation for being slow has to do with writing the programs, and dates back to its earliest days when code had to be written as SET RIDICULOUSLYLONGVARIABLENAME EQUAL TO 2 PLUS 2. Development time sped up a lot when Gracie was finally convinced that RIDICULOUSLYLONGVARIABLENAME = 2 + 2 would work as well (the original approach had to do with coders handprinting the instructions on coding forms that were then given to keypunch operators who knew nothing about computer languages but could type really fast and accurately... so long as their fingers didn't have to reach too far from the home keys).
I was never found of COBOL in school, and never did much with it. But I agree with others that anyone who has worked with a couple of programming languages could probably master COBOL in a matter of hours: in retrospect, it is basically a simple and obvious language. What I probably could not do is reverse engineer some of the constructs that were used back in the day: a lot of weird code was developed to work around hardware limitations that no one born after 1980 will have ever seen. Much of that wasn't documented well, and I think that is why old COBOL programmers are in demand. They remember that this particular type of data structure was used to optimize read-write operations on dual high speed tapes... to a younger programmer it just looks like arbitrary stupidity and they don't know what to do with it.
Parent
Re:Sure, runs on GNU/Linux (Score:5, Insightful)
Sure, so long as you never need to make any changes to the code. The surviving COBOL coders have gone back into comfortable retirement with the money they made fixing Y2K. So they've moved from old iron to a modern operating system; they could still reap even more benefits by recoding for modern languages and coding practices.
But then, this is the US Postal Service. COBOL's probably fast enough for the task.
So you're saying that COBOL is so hideously difficult, so byzantine, so labyrinthine in nature that no one could possibly learn it now? That programmers educated today have no possibility of understanding a language that was designed some decades ago? You realize that C is 30 years old now, right?
This sort of fear mongering through ignorance is getting stale. COBOL is just another language, and one that happened to be designed for ease of expression for less-than-stellar programmers. Legions of students have learned enough C over a weekend to code up the examples in K&R, so I'm actually quite confident that professional programmers can, without any prior experience in COBOL, learn the language, even become proficient in it, in a brief enough time to make modifications to existing code bases.
Look, we're talking about learning a computer language and modifying or maintaining code, not learning Elizabethan English well enough to write a new Shakespeare play that can pass off as an original. It just isn't that hard.
Parent
Re:Sure, runs on GNU/Linux (Score:5, Insightful)
I'm sure I could get a dramatic speed improvement running Apple II 6502 code on an emulator on a Mac Pro simply because the emulator can run faster than the original hardware.
Given that it took 1400 Linux boxes to handle the load, I'd say your post is, at best ignorant, at worst, a blatant troll.
a) Just because it's COBOL, doesn't mean it was running on crappy hardware.
b) COBOL is far from dead, in that many applications running today are written in it. Believe it or not, it makes more sense to continue to run that old code than to rewrite from scratch in the latest shiny because they already know *it works*.
Parent
Re:Now? (Score:5, Insightful)
Parent
Re:Now? (Score:5, Insightful)
Parent
Re:Now? (Score:5, Informative)
"Not only that, I just checked and according to fedex it costs $7.39 to mail that same letter from coast to coast for their cheapest option. That's only what, nearly 17 times more expensive?"
factor in how much of your tax dollars when into that and then get back to us with a valid point....
Umm, the USPS is self-funded. None of your tax dollars go towards supporting their operation source [usps.com]
Parent
Re:Now? (Score:5, Informative)
Umm, the USPS is self-funded. None of your tax dollars go towards supporting their operation source
That's a little misleading - it hasn't always been that way, so a lot of the USPS infrastructure is tax-payer funded.
In addition, they come around every once and a while and ask for money from Congress - they are doing it this year [infowars.com] and while I am hazy on the details, I believe they did something similar about a decade ago in order to fix funding problems with their pension system. Plus, they have a monopoly on letter delivery - that's why fedex costs so much more, they have to classify and price it as something other than a letter - so that's an indirect tax by government intervention to prevent a free market.
Parent
Re:Now? (Score:4, Insightful)
Wait... fedex costs more because USPS has a monopoly? Lets see... what other monopolies are out there to compare to this statement. Nope, does not compute.
Some people do not like the USPS because it represents an actual successful arm of the government. You want to know why they ask congress for money? Its to get back what Congress ultimately takes from the USPS because its the only thing besides taxes that makes money in our government system.
I say YAY USPS, one of the few "companies" where you dont have to worry about your pension, retirement, or jobs.
Let me use a "scare word" for those of you whom are still stuck in the 1980s. "Socialism" can work.
Parent
It is Socialism. (Score:5, Insightful)
Because it provides a social necessity.
If the service was not socialized then small communicates will either not be served or would have to pay more to send (or even receive) a letter.
Parent
Re:Now? (Score:4, Informative)
Gee whiz, I don't know whether I can handle the math. Somebody help me out, what's 44 cents minus zero [slate.com]?
Parent
Re:Now? (Score:5, Insightful)
Parent
Re:Now? (Score:5, Insightful)
You're really complaining about $.50 for the level of service you get from the USPS? For that price, you can send a standard letter anywhere in the US (including the non-continental US) usually arriving in less than 5 days with a loss rate of virtually zero. They deliver mail to (nearly) every address in the US 6 days a week, and will even come to check for outgoing if you don't have any incoming. They even manage to deliver when the roads are absolute shit and no one in their right mind would be out and about.
All for a price that has actually been decreasing over the years if you take into account inflation, let alone the increases in gas prices that have occurred over the last 10 years. Personally, I think that's pretty damn good and wouldn't complain if they raised the price to an even dollar, it would still be under priced for the service they provide.
Parent
Re:Now? (Score:5, Interesting)
Last week on Top Gear, they raced a standard letter sent via standard post in the UK from the south of the UK to the far north of the UK and the letter won.
Total cost of the stamp? A fraction of a pound.
The US is very similar. A little slower due to the extreme distances mail has to route to, but, i'd wager on mail versus delivering it yourself anyday. Not only that it's *cheap*
Parent