A 50 Gbps Connection With Multipath TCP 150
First time accepted submitter Olivier Bonaventure writes "The TCP protocol is closely coupled with the underlying IP protocol. Once a TCP connection has been established through one IP address, the other packets of the connection must be sent from this address. This makes mobility and load balancing difficult. Multipath TCP is a new extension that solves these old problems by decoupling TCP from the underlying IP. A Multipath TCP connection can send packets over several interfaces/addresses simultaneously while remaining backward compatible with existing TCP applications. Multipath TCP has several use cases, including smartphones that can use both WiFi and 3G, or servers that can pool multiple high-speed interfaces. Christoph Paasch, Gregory Detal and their colleagues who develop the implementation of Multipath TCP in the Linux kernel have achieved 50 Gbps for a single TCP connection [note: link has source code and technical details] by pooling together six 10 Gbps interfaces."
what's happening with SCTP? (Score:2, Insightful)
Doesn't SCTP provide for these scenarios (and many more)?
Re:what's happening with SCTP? (Score:5, Informative)
Doesn't SCTP provide for these scenarios (and many more)?
No.
SCTP supports multiple paths between endpoints, but doesn't use them simultaneously. Rather, it picks a primary path to use for data transfers and has the ability to fail over to an alternate path in the event the primary fails.
A quick glance at the MTCP RFC shows that it is essentially multiplexing packets over n separate TCP streams (called subflows). It's the responsibility of the TCP/IP stack (in the OS, generally) to make this multiplexing transparent to the application, so the application only sees one stream.
Re:what's happening with SCTP? (Score:5, Informative)
Work is underway for concurrent multipath transfer for SCTP as well. Also known as CMT-SCTP. There are significant challenges in doing this sort of thing though. SCTP wasn't designed for CMT, and probably needs much more radical changes than the current architects are proposing to do it well.
Changes like subflows with independent sequence numbers and congestion windows, to start with. SCTP is much further ahead in the connection handling and security department, but MPTCP has the odd advantage of resorting to independent subflows to begin with, and if it can handle path failure properly, it might well be ahead in the CMT game, if byte stream semantics are all you need.
Re: (Score:1)
Byte stream semantics aren't sufficient, no. I need to multiplex messages. (More downthread.)
Thanks for your insights.
Re: (Score:2)
Re:what's happening with SCTP? (Score:5, Informative)
1. SCTP - identified by a protocol number (132) - acts at the network layer. If a router along the route refuses SCTP, you are screwed; Advantage: is capable of UDP as well).
2. MPTCP - relies on pure TCP for all the connection (acts at the transport layer and fixes the protocol to TCP) and set in place conventions between client-server to discuss over multiple paths. Advantage: no sane public network will try to block it (pretty much like using http on port 80). Disadvantage: TCP only.
Re: (Score:2)
Re:what's happening with SCTP? (Score:5, Informative)
On the contrary, SCTP is a transport protocol just like TCP, except with a large number of added features. The main problem with SCTP has nothing to do with SCTP at all. It is that NAT devices do not support any transport protocol that they haven't been programmed for in advance. This makes SCTP next to impossible to deploy on a broad scale - NAT, that wart upon router-kind, is ubiquitous.
TCP would have exactly the same problem if it were a new protocol. A NAT device requires relatively deep knowledge of TCP to support it at all. It play games with both ports and addresses, keeps track of connection state, and so on. Ordinary routers do no such thing. A NAT device is a transport layer proxy by another name.
Re:what's happening with SCTP? (Score:4, Insightful)
Your comment is correct, but NAT is not the core problem. In a world without NAT people would still use stateful firewalls. Those firewalls should be configured to drop anything unknown, because as a principle whitelisting is better than blacklisting.
Re: (Score:2)
A stateful firewall doesn't need to block transport layer protocols it doesn't understand in order to provide a meaningful level of security. All it needs to do is block packets from IP addresses that corresponding interior address has not recently communicated with, with a reasonable time out. UDP is handled much the same way today.
If the developers of stateful IPv6 firewalls do not ship devices with such a reasonable configuration by default, they will block the deployment of new transport protocols ind
Re: (Score:2)
On the contrary, SCTP is a transport protocol just like TCP, except with a large number of added features. The main problem with SCTP has nothing to do with SCTP at all. It is that NAT devices do not support any transport protocol that they haven't been programmed for in advance. This makes SCTP next to impossible to deploy on a broad scale - NAT, that wart upon router-kind, is ubiquitous.
On top of that, SCTP also won't take off in the IPv6 world (where NAT isn't an issue), purely because Microsoft are on record saying they're never going to support it.
Which is a shame, because SCTP is actually a much better fit than TCP for almost all applications that currently use TCP, by virtue of the fact that it is packet based instead of stream based. Also, the multi-streaming would be very beneficial for HTTP, which is something that HTTP/2.0 is instead trying to bodge ontop of TCP.
Re: (Score:2)
1. SCTP - identified by a protocol number (132) - acts at the network layer. If a router along the route refuses SCTP, you are screwed
If your ISP is refusing to route protocols other than TCP/UDP/ICMP its time to change ISP - this will equally break GRE, AH, ESP, 6in4 and a whole host of other commonly used protocols.
The other thing SCTP has going for it is that its packet based instead of stream based, which makes it more suitable for almost everything we currently use tcp for...
Re: (Score:2)
It isn't routers you need to worry about its home gateways and firewalls. From my experiments with it so far my ISP DSL modem doesnt allow it but my office firewall supports it fine (mainly because its Linux) but the CISCO ASA we have server side supports it poorly. With the ASA, it is all or nothing. I can allow all SCTP regardless of port or I can allow no SCTP.
Re:what's happening with SCTP? (Score:4, Informative)
Request For Comments (Score:5, Informative)
RFC 6182 [ietf.org] if anyone is interested.
Re:Request For Comments (Score:4, Insightful)
The first part I read when I heard of this was the security concerns. While there's been a good attempt to address them [ietf.org] I am not 100% sold. I guess the proof will be in the pudding as the old saying goes. Anytime you make a new protocol, especially one that is more complex, you run the risk of increased vulnerability.
Re:Request For Comments (Score:5, Interesting)
What sort of security concerns are you thinking of?
An attacker who controls one of the paths can obviously modify, replace, delay or delete portions of the stream which are multiplexed onto that path. Such an attacker could probably perform a DoS that would shut down the entire stream (disclaimer: I haven't read the details). But of course ordinary TCP is subject to all the same attacks, if the attacker has control of the path that carries it. In many cases an attacker without control of the path can also execute DoS attacks against TCP (e.g. sending RSTs).
I'm not saying there aren't any new vulnerabilities exposed, but I'm not seeing where they would lie. TCP is not secure in any useful sense, so it's hard to see how MTCP could be worse.
Re: (Score:2)
Oh I didn't say it would be worse, but the more complex any kind of system gets the more uncertainty is introduced. I certainly wasn't suggesting we not evolve protocols to tackle problems as technology advances. I'm just always cautious about security since IT security is my profession. The RFC I linked to is an interesting read about some of the security concerns we need to keep in mind as this gets implemented.
Re: (Score:2)
I would think that since the data is on multiple and diverse wires it would make it harder to tap. Still security through obscurity but every bit helps.
Re: (Score:1)
What I didn't see addressed in the RFC was the possibility that a connection established over a relatively secure channel (such as a VPN or leased line) might then find and use additional paths over less secure channels, such as the public Internet.
Re: (Score:1)
What I didn't see addressed in the RFC was the possibility that a connection established over a relatively secure channel (such as a VPN or leased line) might then find and use additional paths over less secure channels, such as the public Internet.
This is why I stay far away from the IETF and committies. People like yourself thinking they are most clever wasting everyones time on utter nonsense.
Re: (Score:2)
Re: (Score:2)
It's like frequency hopping. The attacker will only have part of the message.
Right, but that would be a strength, not a vulnerability.
Re: Request For Comments (Score:1)
You could imagine doing RAID like striping of the data so that if some was lost you could recover. And/or encryption actually. Plus you can always stop using a path if it is doing something evil with the data. Its easy to imagine multi path TCP being more secure than single path TCP. Also imagine a case where you have multiple, untrusted Wifi networks to leach off - with multi path TCP you don't have to worry about them dropping or snooping packets.
Re: (Score:2)
Security concerns may or may not be relevant. A lot of places have trivial security on their iSCSI between SAN and server, because the security is applied at other levels (segregated switches / airgap, physical security).
I can think of a number of uses (SAN-server connections where you need more than gigabit) where security is irrelevant.
Re: (Score:3, Informative)
Re: (Score:2)
The old saying is, "The proof of the pudding is in the eating," not "The proof is in the pudding."
Exactly! The OLD saying was "The proof of the pudding is in the eating,"
The NEW saying is "The proof is in the pudding."
Language changes - deal.
Re: (Score:2)
"The proof is in the pudding" wins by virtue of the fact that it conjures up more amusing images.
Re: (Score:2)
Language changes - deal.
Ahh, so more junk in the steaming pile of "I could care less" and "queue in the musicians".
Re: (Score:2)
What specific security concerns do you have that don't also apply to existing protocols?
Re:Request For Comments (Score:5, Informative)
RFC 6182 [ietf.org] if anyone is interested.
I think RFC 6824 [ietf.org], linked in the summary, is the more relevant RFC.
Re: (Score:2)
I missed that one. You sir, are correct!
Also the same authors.
Support available already for most unices (Score:5, Informative)
1. Linux - either debian binaries or compiling from source. Both kernel module and UserSpace ways.
2. Virtualized Linuxes - their example is provided for Amazon EC2
3. Mac OSX - but, obviously, not on iPhone (I estimate slim chances for this to happen in the near future - it's a technology disruptive for the mobile providers income, as it makes the multi-pathing over cell/WiFi hot-spots transparent to end user)
4. Android (Opinion: see? This is one of the reasons relying on "walled gardens" is bad: you have to wait for the mercy of the garden lord to benefit from something).
cell networks already have issues (Score:2)
without every user making 3 connects to view their friends cat picture.
yes, this could have some good uses, no, dont release it to the drooling masses so 12 year old Tiffany can stream Justin Bieber videos while texting her buddy sitting right next to her "faster".
Re:cell networks already have issues (Score:4)
Sheesh, you wanna put even more people out of work? More cell bandwidth needed? Ok, more base stations, new and improved protocols, new frequency allocations, etc. etc., etc. As someone who once made a living working on cellular (phy layer) stuff, I say 12 year old Tiffany has both a Constitutional and a God given right to stream Justin Bieber videos while texting her buddy sitting right next to her. I'll even write the manifesto!
More seriously, a lot of what we take for granted started out as frivolous luxuries. I tell my daughter about days before cell phones, or PC's, and having seven channels of broadcast TV (and having to get up to change the channel!) and she's convinced I come from the age of dinosaurs. She's probably right. That was good, because I made a living changing it.
Re:cell networks already have issues (Score:5, Informative)
without every user making 3 connects to view their friends cat picture.
Rest assured: there'll be a single connection using a cell tower. A second flow will be made using the connection with nearby WiFI hot-spot, and Tiffany's chatting to her buddy sitting next to her will be really faster (without quotes); even better, the above will happen without Tiffany knowing or the extra requirement for Tiffany to have a geek father that's not lazy and does have spare time (even if one may wonder what to what good being a geek will be in the future).
Re: (Score:2)
probably one who is offended by the idea of a big dongle enough to get a couple real dorks fired
Re: (Score:2)
If speed is what you are after, the drooling masses is what you have to bet on. Nobody will invest the money needed to give you faster access to your Linux repository, but the ability to sell Joe Washup some porn or Tiffany Teenygirl some videos of the heartthrob du jour, now we're talking.
Don't even! (Score:2)
I remember getting dual-channel ISDN, which was 128k, but it was split into two 56k data channels and a 16k control channel. You could never download from any one site faster than 56k because a connection couldn't straddle more than one data channel.
Still, I could play EQ and surf at the same time on a different computer, a novel thing you young punks take for granted get off my lawn!
Re:Don't even! (Score:4, Informative)
Wow, sucks to be you. ISDN channel bonding was well known and I personally used it to achieve higher speeds than you could on a single channel even over a single TCP connection. The bonding had nothing to do with the modem/circuit actually and in reality was just a standard feature of the PPP protocol called multilink-PPP. You can still do the exact same thing today with multiple connections and pretty much any PPP client on ANY OS on the planet.
Of course, ISDN is actually 2 64kbps data channels and a 16kbps control channels, as it was meant to carry 2 voice channels, which by standard, are 64kbps data channels, so I'm guessing you really don't know that much about it in general.
Re: (Score:2)
Your phone company could handle something as fancy and high-tech as ISDN with no need to assemble every tech in the house to prop up the line? Get offa MY lawn!
Bad math? (Score:1)
Wouldn't six 10 gig connections add up to one 60 gig connection instead of one 50 gig connection?
Re: (Score:3)
Re:Bad math? (Score:4, Informative)
API support (Score:2)
One of the barriers to this technology will be API support. Many APIs provide the IP address (on both sides) with the connection object. Implementors will have to make a choice about which ip to expose and remain backward compatible.
Re: (Score:1)
No. MTCP is transparent to the application. You can use a normal socket API.
Re: (Score:2)
Re:API support (Score:5, Informative)
Re: (Score:3)
Great idea.
The fact that the protocol supports this without requiring changes to the applications is pretty impressive.
Re: (Score:2)
IPv6 should be transparent to the application as well, yet still it remains largely unimplemented.
From my point of view (Linux), it is largely implemented at the endpoints by now. We just don't have many servers to talk to, and no ISPs willing to sell it to us.
Re:API support (Score:4, Informative)
Sort of: if both WiFi and cell channels are available (think: wandering in a shopping mall with public hotspots), one's Android mobile will use both of them in the same time to manage one's plot in Farmville (or to download the MP3's using that magnet from the PirateBay, or placing whatever buy/sell orders on stock exchange); if one walks out of WiFi spot coverage, the mobile will use only what's available - the cell connection.
Why I used android in my example? Well, it's a Linux kernel, the first implementation is already available. Besides, that should be great news for Google: their "goggles" will be able to transmit what you see much faster and reliable. What I understand from the MCTCP guys' presentation [multipath-tcp.org] makes me believe MPTCP is able to cope with the use and drop from use of multiple dynamically IP addresses (are assigned to the many network devices one's mobile has): thus stepping from one hot-spot to another will not impede Google's capability to receive the data from your (their?) glasses.
Re:API support (Score:5, Interesting)
Yep. And this is a godsend, in some ways: "multipath NFS" should soon be inexplicably easier to accomplish on a high scale. I will be able to put in a single redundant/HA host with 8 1GBps NICs and not have to worry about setting up multipath on each of the individual VM heads I run. This has the significant advantage of not being stuck with immobile "SAN storage" LUNs or, for that matter, "enterprise" hardware vendors which can't bring the reliability their hardware close to anything near what generic Intel or even bcm network cards can provide.
All the better if I've got unified storage at the backend with abstracted paths (eg. lustre, unionfs).
And from the looks of it, it's designed 'forward' - it's going to be MUCH easier to do HA TCP connectivity with this than it is with misc. service level TCP (eg. heartbeat), particularly when you're dealing with (mostly) centrally assigned IPv6 addresses. Awesome.
Granted, from the looks of it, we may have to wait for switch support first, too... I didn't read that carefully.
Re: (Score:2)
Guess for what I'll use it and how I'll be testing MPTCP soon?
NFS on MTCP: was :API support (Score:2)
NFS on multipath is my interest, too.
In NFS (v4) on TCP, the endpoints frame messages on the bytestream, independently in each direction. As FireFury03 states above, we're basically (potentially large) packet based. We'd like help from new transports in framing those messages optimally, avoiding head-of-line blocking for entire messages.
In addition to solving HOL, it's been proposed that we could design message framing on SCTP so as to deliver messages and data chunks on different streams, and get some a
Re: (Score:2)
So do MTCP developers see solutions for any of these problems on the horizon?
I'm in no way affiliated or knowledgeable in MPTCP - so I may be wrong - but from what I got from their presentation [multipath-tcp.org], MCTCP is not actually designed with the NFS-like usecase in mind (even if it may be used for such), but with the more "common" usecase of a mobile device able to use either/both cellular and WiFI networking.
The second thing that I saw as peculiar: it is not even supported by a network protocol (like IP is supporting TCP/UDP/SCTP/etc), but is supported by TCP. While it will have to deal someh
Re: (Score:1)
That is, assuming the head-of-line is send on one path and that path involves a HOL-blocking, then it doesn't matter if the other paths have lower latencies, the entire original stream will be HOL-blocked
The implementation includes a solution to overcome HOL-blocking by reinjecting the blocking data-segment on the lower-latency path. Have a look at our scientific paper, which explains this mechanism: http://inl.info.ucl.ac.be/publications/how-hard-can-it-be-designing-and-implementing-deployable-multipath-tcp [ucl.ac.be]
Re: (Score:2)
The implementation includes a solution to overcome HOL-blocking by reinjecting the blocking data-segment on the lower-latency path.
Have a look at our scientific paper, which explains this mechanism: http://inl.info.ucl.ac.be/publications/how-hard-can-it-be-designing-and-implementing-deployable-multipath-tcp [ucl.ac.be]
Oh, wow! Thanks.
Re: (Score:2)
Hi,
Speed-reading the paper, there appears to be some implicit ammunition for the SCTP approach (if it worked!), at least for applications like HTTP, NFS, etc, which are characterized by multiplexing of large and small messages on the stream. I conclude this from section 5.3, which I think states MPTCP over 2 links was slower than ordinary TCP over one link, when message size was 30K.
(Apologies if I'm misreading.)
Thanks,
Matt
Re: (Score:1)
I conclude this from section 5.3, which I think states MPTCP over 2 links was slower than ordinary TCP over one link, when message size was 30K.
For very small flow-sizes (like less than 30KB), MPTCP should not try to create additional subflows. Because, the whole data fits in the initial window of the first subflow. However, at the moment the linux implementation always tries to establish new subflows. In the paper's stress-testing scenario these additional subflows just consumed CPU-cycles and thus the "bad" results for MPTCP with very small flows
An easy fix would be to delay the establishment of additional subflows until a certain threshold of da
Re: (Score:2)
Thanks, Christoph.
I think I was mis-reading, section 5.3 is discussing performance with short-lived connections, such as happens with HTTP .9 or 1.0. The question I would ask next is, how does MPTCP perform when HTTP 1.1 or similar channel multiplexing is used?
Matt
Re: (Score:1)
We have not yet done extensive experiments with HTTP1.1.
But, MPTCP is a benefit for bandwidth-intensive (thus increasing throughput) and long-lived connections (increased resilience against link-failures).
So, I would say that HTTP 1.1 would benefit from MPTCP.
Cheers,
Christoph
Re:API support (Score:4, Informative)
Use Cases? (Score:1)
I'm possibly missing the point here, but I'm struggling to understand how this would be put to use:
Servers that can pool multiple high-speed interfaces:
This capability has been around for years - port channeling/interface bonding/whatever vendors call it, do this already at L2. Aside from making it easier for those interfaces to sit on different subnets, why would you want to push this capability to L3/TCP? Seems like it just introduces additional complexity.
"Smart" devices (or any client really)
I suppose
Re:Use Cases? (Score:4, Informative)
You're missing the point. One of the big reasons to have multiple interfaces is for redundancy - with a company's internet interface, redundancy would be vastly improved by choosing two different providers, and even better with different mediums. The subnets will definitely be different.
Having both of these links acting simultaneously would be great and I could see a lot of people being excited about it.
Re: (Score:2)
Well, "both" links won't work too well.
One of the main problems with using your one redundant link as just a regular link is your bandwidth "needs" tend to grow to include some or even all of the bandwidth available from the now non-redundant link. Then not only do you not have a redundant link, but you've doubled your bandwidth requirements and made it that much harder to provision in an e
Re: (Score:2)
One of the main problems with using your one redundant link as just a regular link is your bandwidth "needs" tend to grow to include some or even all of the bandwidth available from the now non-redundant link.
Really, in most cases, there's a big difference between "bandwidth needs" and "bandwidth wants". Businesses who normally use a 10Mb connection will frequently limp along on a backup T1 when necessary, without serious problems. Being without an Internet connection is a much bigger problem.
But then the question frequently comes up: We have an 10Mb connection, and then a 1.5Mb backup connection that sits around doing nothing all day. Can we somehow combine them into a 11.5Mb connection?
And the answer is g
Re: (Score:2)
It would be great if the SSH connections from my laptop used the reliable ethernet connection when it's plugged in, and seamlessly switched to wireless LAN when it was unplugged. Even better would be if I could put the laptop to sleep and move to a different location, and that the connections would stay open (given, of course, that there was no I/O on them while the computer was moving, and that it was only moving for a reasonably short time). It seems like it *may* be possible to open a new "subflow" even
Is using all connections good? (Score:1)
Their use case uses all interfaces on a machine, then they disable some of them. Do you normally WANT to use every and all interface for your traffic? If you're a mobile device, maybe not?
Re: (Score:1)
You disable the ones you do not want to use at any given time.
API (Score:2)
Offload (Score:2)
What am I missing? (Score:1)
Re: (Score:2)
Yes, you are missing a clue.
If I got 50Gbps out of 6 x 10Gbps links I'd be ecstatic. That's pretty good efficiency considering the sheer throughput processing requited and overheads involved.
Plus it may be difficult if not impossible to get multiple high speed interfaces via one internet carrier - however what if 3 or 4 fibre providers went past your datacenter? Buy bandwidth from all of them and use them simultaneously and add redundancy.
Re: (Score:3, Informative)
You want to send a shitload of data to a destination but it takes too long? Not a problem, throw a couple quad nics in those bitches and bond them up, problem solved providing your network can support the throughput.
What am I missing?
This is layer 4 not 2. So long as both endpoints support it, it don't matter where the traffic goes. they can go over entierly different paths. This is doing what you describe, but over the internet. Transparent to the network, and the higher levels of the protocol stack.
They should kickstart an appliance (Score:1, Insightful)
Re: (Score:2)
NOOOooo...
Why would you need an appliance for something that can be done in software?!? The only "appliance" you need is an ethernet switch between your residential connection bridges and your firewall.
Tor may actually take a nice advantage here. (Score:1)
Has anyone actually thought about how The Onipn Network might take advantage of this? This could potentially, or partially, fix one of Tor's biggest problems, which is the inherent low bandwidth due to one single TCP connection via several nodes in a serial manner. If Tor could leverage multiple nodes in parallel into a single connection, this ould certainly make it faster.... And even more anonymous tha it is today.
This was fun, back in 2004 (Score:1)
I was doing TCP multipath in 2004 using iptables to get more upstream out of my box at home. I had two 1.5/384 connections and could and up with 768k upstream. All it took was a clever iptables script that marked alternating packets - even and odd if you will - and mangled odd packets to go out one interface, and even packets for the other.
Obvioulsy, an actual TCP extension for this is going to be more elegant since it's more scalable and easy to deal with, but the idea is not new.
mobile phone - YES (Score:1)
I find it annoying that my app is disrupted when I leave a Wifi and the (Android) phone needs a few seconds to connect to the #G/4G network. But then it might just to that because it needs to save the power on the network interface.
But then I'd buy the phone with double or tripple the battery over the slim one anyway. If only they would be available.
Hope Application (Score:2)
Waiting to see an apartment full of geeks and gamers rocking 10 cable modems all linked together and splitting the bill.
Re: (Score:2)
*home*
Re: (Score:2)
yes, I know etherchannel load balancing ... but maybe that would be easier to "fix" than inventing something that mostly exists. ... 10 years ago.
Do you also know the nowadays mobile devices? Wouldn't it be nice to use both WiFi and mobile wireless communication in the same time without special equipment from Cisco? Even more: transparent to you when you step from on public WiFi hot-spot coverage into another and be assigned with new IP address?
Re: Uh, I get this with lacp (Score:1)
No need for expensive Cisco equipment. I have cheap Net gear switches that are completely LACP capable, and some cheap Realtor adapters support teaming now. It's more a driver issue than anything. If cell manufacturers designed their equipment and built the right drivers, you could easily dynamically team a cell and wifi connection.
Re: (Score:3)
If cell manufacturers designed their equipment and built the right drivers
And if Apple refuses to implement it, you will still be able to grab an Android, compile/install the MPTCP stack and do it (without waiting for Apple to resist the mobile providers pressure in not supporting a feature that would hurt their bottom line. Or, for the matter, wait for the mobile providers to upgrade their towers and hurt their bottom line by themselves).
Re: (Score:1)
Yes, you can do it at layer 2 with various different technology, some vendor specific, some vendor neutral. This new method is doing it at layer 4. If you can't see why this matters then I suggest learning why the layered model exists in the first place.
Re:Uh, I get this with lacp (Score:5, Informative)
No, you dont. If I remember correctly, LACP will give you the maximum bandwidth provided by a single link, per connection. You cant just hook up LACP / LAGG / whatever your vendor calls it, fire up iSCSI, and magically have a 2gbps link to your SAN-- because iSCSI does a single connection per LUN, you will get a 1gbps connection even with LACP.
LACP gets you higher total capacity, so if you were running two iSCSI connections you could get 1gbps on each with no contention. If the summary be believed, this would give you a truly multi-gbps link off of aggregated gbit connections.
Re: (Score:2)
Then you setup your load balancing on your LACP links incorrectly for what you were trying to accomplish. Choose a different implementation for pathing and you'll get total aggregate.
The problem you're refering to is a problem with the implementation of LACP you're using, not the specification itself.
Re:Uh, I get this with lacp (Score:4, Interesting)
Not unless they changed something recently. Read http://www.ieee802.org/3/hssg/public/apr07/frazier_01_0407.pdf [ieee802.org] LACP requires that any conversation goes over only a single link at a time. Out of order packets can do some rather nasty things to tcp connections and adding buffers to correct that does nasty things to voip / other latency sensitive bits. Sure linux boxes have some non standard modes that might work if you sitting one switch away but that's not conforming to the LACP spec. They also do not scale as they require keeping state of every session running through them. What networking gear are you using?
Re:Uh, I get this with lacp (Score:5, Informative)
According to both the article which silas linked below (which is the original source for what I said), as well as a whole boatload of other documentation, thats not correct; its an 802.1ad issue.
I did find this on serverfault [serverfault.com] which indicates that ONLY balance-roundrobin can get you 2gbps on a single tcp connection; and it also notes that some protocols dont like it, which means that its not really a transparant bonding technology. All of the other methods of distributing packets rely on a hash of various values, for instance source mac and destination mac IDs, and regardless of method the hash will ALWAYS be the same on a single TCP connection, which means that the same single link will be used.
Regardless, the Linux Bonding driver is NOT the same thing as LACP, and its not something you implement on the switch.
Re: (Score:2)
LACP uses various methods to choose which link to send frames over-- for example sourceport id, source mac, etc. Regardless of what you choose, a single TCP connection will end up using the same link even when LACP is implemented on the switch.
You might try reading the linked articles in my and silas' responses before arguing; particularly as one of them is a link to the IETF spec.
Re: (Score:2)
LACP does not need to care about L3 or L4 it's free to do so. Really cheap networking kit (and some broken really expensive bits) only use mac's ever smarter kit can do whatever it wants higher up in the protocol stack. How it distributes packets is not something that's negotiated it's just fixed or a setting on each end. If you plug the dumbest of the dumb lacp switch into the smartest switch you will get good load balance in one direction and poor the other.
Re: (Score:2)
Yes, you have that total amount of bandwidth. If you were to have 4 iSCSI connections, each of them would get a full gigabit; if you had 8 connections each would get 500mbps.
However, a single connection from a single TCP port coming out of a single MAC address / IP address is going to get a single gigabit /sec of traffic; theres not really a good workaround for this.
If youve found a way to get 4gbps on a single iSCSI connection using LACP, please do share, as a LOT of people would be interested to get that
Re: (Score:2)
It's worse than that depending on how things are hashing you can have 4 connections and 2 of them use the same link (or all four) It's really dependent on the networking kit your using LACP does not specify how you do this just that you do something to insure packets for a given "conversation" only go down one path at a time. Simpler networking kit might only look at 1 mac address smarter bits go all they way up to protocol ports.
For iSCSI the "easy" fix is to run multipath it assuming your san supports it
Re: (Score:2)
Does not work with nat. Unless you could bgp anounce a single ip address.
Re: (Score:2)
On second thought not even then. Each connection has it's own ip address. You would need to find a way to terminate a single TCP connection on two ips. I assume this require change to the server and client end's software/os/
Re: (Score:2)
If you want to use multiple links all at the same time, with the packets spread over them, you're supposed to get an Autonomous System number.
This is more akin to link aggregation than it is multihomed Internet connections. Any two hosts could use this. They could be in the same autonomous system. They could be on the same subnet. There's no need to get a separate AS number for each host.
Note that one of the other use cases suggested is for smartphones.
Re: (Score:2)
When you advertise something over BGP you essentially broadcast it to every core router in the world. Having every core router know about every device is just not going to happen. Having every router know about every ISP and large company is bad enough. Also BGP is built on a mixture of trust and manually applied filters. So unless you want to open it up to every idiot breaking other peoples traffic then it's not going to handle systems that regually move arround very well.
Also internet routing looks for th
Re: (Score:2, Interesting)
or maybe we could just filter comments based on length or number of links. >1000 words or >20 links
Re: (Score:1)
Stop replying to trolls!
Re:fault tolerance (Score:4, Informative)
MPTCP has separate sequence-number spaces. One for the subflow, inside the regular TCP header. And the data sequence-numbers, included inside the TCP option-space.
This data sequence numbers include data-acks. So, this is your mentioned "cross-subflow ack machinery".