Catch up on stories from the past week (and beyond) at the Slashdot story archive


Forgot your password?
The Internet Networking Linux Technology

A 50 Gbps Connection With Multipath TCP 150

First time accepted submitter Olivier Bonaventure writes "The TCP protocol is closely coupled with the underlying IP protocol. Once a TCP connection has been established through one IP address, the other packets of the connection must be sent from this address. This makes mobility and load balancing difficult. Multipath TCP is a new extension that solves these old problems by decoupling TCP from the underlying IP. A Multipath TCP connection can send packets over several interfaces/addresses simultaneously while remaining backward compatible with existing TCP applications. Multipath TCP has several use cases, including smartphones that can use both WiFi and 3G, or servers that can pool multiple high-speed interfaces. Christoph Paasch, Gregory Detal and their colleagues who develop the implementation of Multipath TCP in the Linux kernel have achieved 50 Gbps for a single TCP connection [note: link has source code and technical details] by pooling together six 10 Gbps interfaces."
This discussion has been archived. No new comments can be posted.

A 50 Gbps Connection With Multipath TCP

Comments Filter:
  • Request For Comments (Score:5, Informative)

    by Nethead ( 1563 ) <> on Friday March 22, 2013 @09:05PM (#43253753) Homepage Journal

    RFC 6182 [] if anyone is interested.

  • by swillden ( 191260 ) <> on Friday March 22, 2013 @09:15PM (#43253805) Homepage Journal

    Doesn't SCTP provide for these scenarios (and many more)?


    SCTP supports multiple paths between endpoints, but doesn't use them simultaneously. Rather, it picks a primary path to use for data transfers and has the ability to fail over to an alternate path in the event the primary fails.

    A quick glance at the MTCP RFC shows that it is essentially multiplexing packets over n separate TCP streams (called subflows). It's the responsibility of the TCP/IP stack (in the OS, generally) to make this multiplexing transparent to the application, so the application only sees one stream.

  • by swillden ( 191260 ) <> on Friday March 22, 2013 @09:17PM (#43253821) Homepage Journal

    RFC 6182 [] if anyone is interested.

    I think RFC 6824 [], linked in the summary, is the more relevant RFC.

  • by c0lo ( 1497653 ) on Friday March 22, 2013 @09:39PM (#43253951)
    In my understanding (I might be wrong):

    1. SCTP - identified by a protocol number (132) - acts at the network layer. If a router along the route refuses SCTP, you are screwed; Advantage: is capable of UDP as well).

    2. MPTCP - relies on pure TCP for all the connection (acts at the transport layer and fixes the protocol to TCP) and set in place conventions between client-server to discuss over multiple paths. Advantage: no sane public network will try to block it (pretty much like using http on port 80). Disadvantage: TCP only.

  • by c0lo ( 1497653 ) on Friday March 22, 2013 @09:59PM (#43254049)

    without every user making 3 connects to view their friends cat picture.

    Rest assured: there'll be a single connection using a cell tower. A second flow will be made using the connection with nearby WiFI hot-spot, and Tiffany's chatting to her buddy sitting next to her will be really faster (without quotes); even better, the above will happen without Tiffany knowing or the extra requirement for Tiffany to have a geek father that's not lazy and does have spare time (even if one may wonder what to what good being a geek will be in the future).

  • by LordLimecat ( 1103839 ) on Friday March 22, 2013 @10:12PM (#43254105)

    No, you dont. If I remember correctly, LACP will give you the maximum bandwidth provided by a single link, per connection. You cant just hook up LACP / LAGG / whatever your vendor calls it, fire up iSCSI, and magically have a 2gbps link to your SAN-- because iSCSI does a single connection per LUN, you will get a 1gbps connection even with LACP.

    LACP gets you higher total capacity, so if you were running two iSCSI connections you could get 1gbps on each with no contention. If the summary be believed, this would give you a truly multi-gbps link off of aggregated gbit connections.

  • Re:API support (Score:4, Informative)

    by c0lo ( 1497653 ) on Friday March 22, 2013 @10:20PM (#43254147)
    In my understanding, this will still rely on multiple IP addresses (not using a single IP address for all the network connections). The difference: it will ride on top of multiple TCP/IP connections - assuming they are available - to multiplex their different paths into a single socket connection (that is: no API changes).

    Sort of: if both WiFi and cell channels are available (think: wandering in a shopping mall with public hotspots), one's Android mobile will use both of them in the same time to manage one's plot in Farmville (or to download the MP3's using that magnet from the PirateBay, or placing whatever buy/sell orders on stock exchange); if one walks out of WiFi spot coverage, the mobile will use only what's available - the cell connection.

    Why I used android in my example? Well, it's a Linux kernel, the first implementation is already available. Besides, that should be great news for Google: their "goggles" will be able to transmit what you see much faster and reliable. What I understand from the MCTCP guys' presentation [] makes me believe MPTCP is able to cope with the use and drop from use of multiple dynamically IP addresses (are assigned to the many network devices one's mobile has): thus stepping from one hot-spot to another will not impede Google's capability to receive the data from your (their?) glasses.

  • by c0lo ( 1497653 ) on Friday March 22, 2013 @10:55PM (#43254295)
    For those wanting to try, their install howto []. Seems supported on:
    1. Linux - either debian binaries or compiling from source. Both kernel module and UserSpace ways.
    2. Virtualized Linuxes - their example is provided for Amazon EC2
    3. Mac OSX - but, obviously, not on iPhone (I estimate slim chances for this to happen in the near future - it's a technology disruptive for the mobile providers income, as it makes the multi-pathing over cell/WiFi hot-spots transparent to end user)
    4. Android (Opinion: see? This is one of the reasons relying on "walled gardens" is bad: you have to wait for the mercy of the garden lord to benefit from something).
  • Re:Don't even! (Score:4, Informative)

    by BitZtream ( 692029 ) on Friday March 22, 2013 @11:10PM (#43254375)

    Wow, sucks to be you. ISDN channel bonding was well known and I personally used it to achieve higher speeds than you could on a single channel even over a single TCP connection. The bonding had nothing to do with the modem/circuit actually and in reality was just a standard feature of the PPP protocol called multilink-PPP. You can still do the exact same thing today with multiple connections and pretty much any PPP client on ANY OS on the planet.

    Of course, ISDN is actually 2 64kbps data channels and a 16kbps control channels, as it was meant to carry 2 voice channels, which by standard, are 64kbps data channels, so I'm guessing you really don't know that much about it in general.

  • by fleisher ( 865744 ) * <> on Friday March 22, 2013 @11:23PM (#43254435)
    The old saying is, "The proof of the pudding is in the eating," not "The proof is in the pudding."
  • Re:Use Cases? (Score:4, Informative)

    by aXis100 ( 690904 ) on Saturday March 23, 2013 @12:13AM (#43254677)

    You're missing the point. One of the big reasons to have multiple interfaces is for redundancy - with a company's internet interface, redundancy would be vastly improved by choosing two different providers, and even better with different mediums. The subnets will definitely be different.

    Having both of these links acting simultaneously would be great and I could see a lot of people being excited about it.

  • by LordLimecat ( 1103839 ) on Saturday March 23, 2013 @01:37AM (#43254967)

    According to both the article which silas linked below (which is the original source for what I said), as well as a whole boatload of other documentation, thats not correct; its an 802.1ad issue.

    I did find this on serverfault [] which indicates that ONLY balance-roundrobin can get you 2gbps on a single tcp connection; and it also notes that some protocols dont like it, which means that its not really a transparant bonding technology. All of the other methods of distributing packets rely on a hash of various values, for instance source mac and destination mac IDs, and regardless of method the hash will ALWAYS be the same on a single TCP connection, which means that the same single link will be used.

    Regardless, the Linux Bonding driver is NOT the same thing as LACP, and its not something you implement on the switch.

  • by Anonymous Coward on Saturday March 23, 2013 @01:45AM (#43254987)

    You want to send a shitload of data to a destination but it takes too long? Not a problem, throw a couple quad nics in those bitches and bond them up, problem solved providing your network can support the throughput.

    What am I missing?

    This is layer 4 not 2. So long as both endpoints support it, it don't matter where the traffic goes. they can go over entierly different paths. This is doing what you describe, but over the internet. Transparent to the network, and the higher levels of the protocol stack.

  • by butlerm ( 3112 ) on Saturday March 23, 2013 @01:52AM (#43255017)

    On the contrary, SCTP is a transport protocol just like TCP, except with a large number of added features. The main problem with SCTP has nothing to do with SCTP at all. It is that NAT devices do not support any transport protocol that they haven't been programmed for in advance. This makes SCTP next to impossible to deploy on a broad scale - NAT, that wart upon router-kind, is ubiquitous.

    TCP would have exactly the same problem if it were a new protocol. A NAT device requires relatively deep knowledge of TCP to support it at all. It play games with both ports and addresses, keeps track of connection state, and so on. Ordinary routers do no such thing. A NAT device is a transport layer proxy by another name.

  • by butlerm ( 3112 ) on Saturday March 23, 2013 @02:44AM (#43255145)

    Work is underway for concurrent multipath transfer for SCTP as well. Also known as CMT-SCTP. There are significant challenges in doing this sort of thing though. SCTP wasn't designed for CMT, and probably needs much more radical changes than the current architects are proposing to do it well.

    Changes like subflows with independent sequence numbers and congestion windows, to start with. SCTP is much further ahead in the connection handling and security department, but MPTCP has the odd advantage of resorting to independent subflows to begin with, and if it can handle path failure properly, it might well be ahead in the CMT game, if byte stream semantics are all you need.

  • by Olivier Bonaventure ( 2873367 ) on Saturday March 23, 2013 @04:43AM (#43255475)
    SCTP is cleaner than Multipath TCP, but it suffers from two drawbacks that hinder its deployment in today's Internet : - many middleboxes only support IP, ICMP and TCP and discard SCTP packets (or do not perform NAT correctly) - applications need to be modified to support SCTP Multipath TCP is an evolution to TCP that works with unmodified applications and unmodified middleboxes.
  • Re:Bad math? (Score:4, Informative)

    by Olivier Bonaventure ( 2873367 ) on Saturday March 23, 2013 @04:49AM (#43255495)
    The limit here is the CPU and on the sender and the receiver. Both servers used in the test reached 98% CPU load to achieve 52 Gbps. Note that 52 Gbps is the googput at the application and not the bandwidth used on the links (which is higher due to the various overheads)
  • Re:API support (Score:4, Informative)

    by Olivier Bonaventure ( 2873367 ) on Saturday March 23, 2013 @04:52AM (#43255511)
    The current implementation in the Linux kernel only exposes the first address used in the connection to the application. If the addresses change, the application is not informed but the TCP connection remains alive. Exposing addresses to the application is an old mistake of the socket interface. The socket interface does not expose packet losses because TCP deals with them and provides a bytestream abstraction to the application. Multipath TCP does the same, it handles all changes in address transparently to the application.
  • Re:API support (Score:5, Informative)

    by Olivier Bonaventure ( 2873367 ) on Saturday March 23, 2013 @04:53AM (#43255513)
    Multipath TCP supports transparently IPv4 and IPv6. A Multipath TCP connection can start over IPv4 and then use IPv6 without the application being aware of the utilisation of IPv6. This could help the utilisation of IPv6 paths by IPv6 unaware TCP applications.
  • Re:fault tolerance (Score:4, Informative)

    by patch11 ( 1303699 ) on Saturday March 23, 2013 @06:45AM (#43255767) Homepage

    MPTCP has separate sequence-number spaces. One for the subflow, inside the regular TCP header. And the data sequence-numbers, included inside the TCP option-space.

    This data sequence numbers include data-acks. So, this is your mentioned "cross-subflow ack machinery".

news: gotcha