Linux 3.13 Released 141
diegocg writes "Linux kernel 3.13 has been released. This release includes nftables (the successor of iptables); a revamp of the block layer designed for high-performance SSDs; a framework to cap power consumption in Intel RAPL devices; improved squashfs performance; AMD Radeon power management enabled by default and automatic AMD Radeon GPU switching; improved NUMA and hugepage performance; TCP Fast Open enabled by default; support for NFC payments; support for the High-Availability Seamless Redundancy protocol; new drivers; and many other small improvements. Here's the full list of changes."
Re: (Score:1)
To someone that has not used Linux for a while, is this just a new version of iptables or is this actually comparable with pf?
nftables on LWN (Score:5, Informative)
Absolute best technical read on the Internet. Subscribe early, subscribe often.
Dynamic Packet Filter (Score:3)
One of the comments points to DPF [mit.edu], which uses dynamic code generation to demultiplex packets. This is a very promising and surprisingly old idea. A dynamically generated classifier/filter could replace the entire network input path, and interface well with Van Jacobson's net channels [lemis.com]. In addition to providing superior performance, it would afford far greater flexibility and modularity of code.
nftables (Score:2, Interesting)
There's a compatibility wrapper, right? Right? Because nftables is an awful terrible complicated pile of needless complexity. It should be possible to set up a simple deny-inbound firewall ruleset in just a few lines, or..........I'm just not going to upgrade! Yeah. That's the idea.
Re:nftables (Score:5, Informative)
Yes, there is. [netfilter.org]
The nftables project provides a backward compatibility layer that allows you run iptables/ip6tables (using the same syntax) over the nftables infrastructure:
Re: (Score:2)
Re: (Score:2)
Yes, obsolete software is eventually abandoned. If you don't like it, use a long term support kernel.
configuration languages (Score:4, Interesting)
This release includes nftables (the successor of iptables)
Why does every network management tool include their own ugly, broken little programming language for configuring it?
Why not just use an existing language?
Like, when I get a packet from the network, I can just use Python:
if packet.origin == "127.0.0.1":
packet.drop()
elif packet.port == 80:
packet.forward(port = 1024)
etcetera.
Re:configuration languages (Score:5, Insightful)
Re:configuration languages (Score:4, Insightful)
I think the real question (and possibly what StripedCow was asking) is why not use the syntax of existing languages, to make it easier to learn and read?
Because you would need to embed an interpreter (Score:1)
If you support all the syntax of the language (so python in this case) people are going to expect to be able to interpret/run arbitrary python statements:
if packet.port in [80,443,8080]:
packet.forward(port = 1024)
Without the same semantics looking the same isn't going to be enough. It is also true that the language syntax can let you arrange for certain conditions to always hold which allows simpler backend rules to be generated (e.g. in the original example the first comparison was against an IP ad
Re: (Score:3)
I think the real question (and possibly what StripedCow was asking) is why not use the syntax of existing languages, to make it easier to learn and read?
Because it would do neither. It would turn one-liners into many-liners, which would add complexity that would only ultimately be confusing.
Re: (Score:3)
More lines does not neccessarily mean more complexity.
Re: (Score:3)
More lines does not neccessarily mean more complexity.
And breaking the output up into more lines and adding a bunch of characters and bullshit around it doesn't make it more understandable. If you're having problems understanding the rules, use a tool that makes the job simpler, like fwbuilder.
Re: (Score:3)
It most definitely can make it more readable in many cases. Have you ever tried to debug Perl written by an 'expert'. The rules are all there, but the meaning is quite obsfuscated. Breaking it up and adding characters helps a lot. I'm not saying it would definitely herlp here as I don't know the new syntax, but it's quite conceivable.
Re: (Score:2)
It most definitely can make it more readable in many cases. Have you ever tried to debug Perl written by an 'expert'. The rules are all there, but the meaning is quite obsfuscated.
That's not the case in firewall rules, where the rules are strict. The problem with perl is that the rules are loose, which is good for bashing out some code but bad for readability. We don't have that problem here. With firewall rules, the only problem is in understanding them. With perl, you also have the problem of distinguishing code from random keyboard-smashing or typos.
Re: (Score:2)
You would need to port the python interpreter to whatever VM instruction set is used. Good luck with that.
Re: (Score:3)
Probably because of the extremely high performance requirements. There's a lot of packets going through a 10Gbit interface and if you run some Python code for each of them you're gonna choke the machine.
That would be true if it were impossible to compile Python code to something efficient.
And the style of code used above (and typical network configuration scripts) would compile to something very efficient. In fact, a compiler can compile that code to the current configuration language whenever possible.
Re:configuration languages (Score:5, Insightful)
Which means that you could proibably come up with something to do this yourself. I'm sure quite a few people would be interested in it.
Re:configuration languages (Score:4, Interesting)
Probably because of the extremely high performance requirements. There's a lot of packets going through a 10Gbit interface and if you run some Python code for each of them you're gonna choke the machine.
That would be true if it were impossible to compile Python code to something efficient.
And the style of code used above (and typical network configuration scripts) would compile to something very efficient. In fact, a compiler can compile that code to the current configuration language whenever possible.
Systems like IPTABLES are based on declarative syntax. When you "program" by declaration, you are working with a limited syntax to perform limited functions. That also limits your flexibility, but the tradeoff is that the functions that you can do are so well-defined that you can be assured that anything you compile will operate in a way that minimizes surprises. It also means that you can optimize things more precisely because you don't have to support many possibilities, only a few. The #1 feature of declarative programming is that you don't have lots of loops and decisions to debug. Or optimize.
The "tables" part of IPTABLES also contributes to that functionality. There are a limited set of table types pre-defined with specific uses. By compiling the rules into tables, very efficient processing can be done. After all, optimized table handling is one of the oldest and most intensively-studied disciplines of Computer Science.
Even IPTABLES wasn't totally restricted to table-driven functions, though. As an extendable architecture, if you really wanted to, you could define your own abilities to the system, including modules incorporating user-written code. It's just that they had pre-defined and fine-tuned the most commonly-used features.
Re: (Score:1)
Re: (Score:3)
You can't compile without a compiler. Your now amended proposal is to create or fork a non-standards compliant Python compiler (You don't realize that that is your proposal, because you think "Gosh ... they already gots 'em", but read the last couple of lines at the end to see how to determine why you once again don't understand).
I'm going to go out on a limb and say that you have little or no experience designing rea
Re: (Score:1)
phew. thank goodness we only have a VM interpreter instead!
"nftables kernel engine adds a simple virtual machine into the Linux kernel, which is able to execute bytecode to inspect a network packet "
I assume its using Java bytecode?
Re: (Score:1)
No, it's a different bytecode. Very simple with a register based VM. http://www.tcpdump.org/papers/bpf-usenix93.pdf
Re: (Score:2)
Re: (Score:1)
Probably because of the extremely high performance requirements. There's a lot of packets going through a 10Gbit interface and if you run some Python code for each of them you're gonna choke the machine.
This does not seem to be an argument. Any language (maybe with some restriction) could be compiled to the appropriate byte code/assembly.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
whether network administrators really want to compile their rules every time they change them
not really necessary since usually you have to perform some kind of action to implement changes (such as executing a bash script) it would make sense to recompile the rule script when performing a flush command (such as equivalent of iptables -F) or restarting the firewall (akin to apache2ctl restart)
compiling would seem like a sensible idea even for the existing simplified syntax, since interpreting any script is much slower than executing compiled machine code
Re: (Score:2)
compiling would seem like a sensible idea even for the existing simplified syntax, since interpreting any script is much slower than executing compiled machine code
I'm quite sure the rules in the current system are stored in binary values in the kernel rather than in the script format which loads them.
Re: (Score:3)
Performance. On a firewall/filter for a major network there could be millions of packets per second. On a smaller scale routers running linux often use very low-power and thus low-performance hardware while still handling gigabit interfaces. No cycles to spare for running an interpreter or even parsing complex rules with variable-length fields.
Re: (Score:1)
No cycles to spare for running an interpreter
" nftables kernel engine adds a simple virtual machine into the Linux kernel, which is able to execute bytecode to inspect a network packet "
Wait, what, you didn't want an interpreter in your kernel?! TOO LATE.
Re:configuration languages (Score:5, Insightful)
It's faster than the scanned rule lists in iptables, and the remaining alternative is compiling to native code into a module every time you change a rule.
I know that seeing "bloat" everywhere and griping is an honored /. tradition, but someone has to add a dash of reality.
Re: (Score:1)
But to continue the basic argument: having support for native code generation could speed up many other operating system tasks that either use other interpreters (there are more of that than one could think) or cases where specialized code could be faster. Doing an infrastructure for somethi
Re: (Score:2)
No cycles to spare for running an interpreter
" nftables kernel engine adds a simple virtual machine into the Linux kernel, which is able to execute bytecode to inspect a network packet "
Wait, what, you didn't want an interpreter in your kernel?! TOO LATE.
There's likely more than one already.
Some interpreters can be extremely efficient. The threaded-code interpreter used by the original FORTH language, for example.
Re: (Score:2)
There's likely more than one already.
bpf has used one for twenty years or so; not sure when that got merged in the Linux kernel.
Re: (Score:2)
Re: (Score:2)
Re:configuration languages (Score:5, Insightful)
The problem is overheads and security.
Embedding a language at such a low level is very tricky. It has to be blinding fast and user very very little resources. python, perl, ruby are all great languages. but ill suited for the task of network management tasks. The RAM overheads are huge. This is why we are seeing a relatively constant evolution, change of embedded languages at these low levels. This is a game of resource management on the host system.
Just imagine if this host was a web server. With thousands of socket requests per second. How would Python manage to keep up with that. Without crushing the system under load even before the traffic was passed off to a process like Ngynx to handle. Python would be a performance nightmare at this level.
Another way to look at this is. What if you hammered the system with a DOS style attack. If each request had to go through a python execution stack you are basically making the system far more vulnerable to DOS than it ever was before.
Now lets look at topics around these highly extensible languages. Here you have a system that in part is supposed to improve security. But by adding in a language like python you are adding in a very extensible lnaguage at a very low level. A kernel level to be precise. So higher than root. The security implications are enormous. You are basically exposing the kernel to a far higher risk. This would be a hackers dream come true.
So there are reason for these language syntax choices. They must be managed very carefully.
Re: (Score:3)
Common syntax that results in different behaviour? That sounds more confusing to me.
Re:configuration languages (Score:5, Insightful)
Common syntax that results in different behaviour? That sounds more confusing to me.
Exactly. It seems like the only people who want firewall syntax in the form of a popular scripting language are people who should not be touching firewall rules in the first place.
Re: (Score:1)
couldn't you just use Python syntax, and write a simple interpreter that meets the efficiency demands?
you certainly could
Re: (Score:2)
Embedding a language at such a low level is very tricky.
Nobody said kernel development would be easy :)
Now lets look at topics around these highly extensible languages. Here you have a system that in part is supposed to improve security. But by adding in a language like python you are adding in a very extensible lnaguage at a very low level.
You could run it all inside a sandbox, I suppose.
Re: (Score:2)
A firewall in a sandbox?
Do you see the issue here?
Sandboxes are good for consuming applications. The firewall is not a consumer. It's a part of the command and control chain. It's a the heart of the system. Sandboxing the kernel is self defeating. As it's the kernel and everything spawns from it. So you can't really protect your child processes if your kernel is compromised.
Re: (Score:2)
My idea was that the firewall merely invokes the sandbox. It is not running inside the sandbox...
Re: (Score:1)
Re: (Score:2)
Good one. That's funny.
Python *IS POSSIBLE* in theory (Score:3)
Why does every network management tool include their own ugly, broken little programming language for configuring it?
Why not just use an existing language?
Like, when I get a packet from the network, I can just use Python:
You could use Python:
You need to write your own interpreter which takes scripts written in your special subset of python, and compile them into the special bytecode that the NFTable kernel interface uses.
The thing is, the internal representation of NFTables needs to be highly efficient (as explained by other posts here) and very likely the official NFTable bytecode isn't really feature complete or maybe not even turing-complete.
The current special language will map nicely to it. But you will probably need a
Re: (Score:3)
Why does every network management tool include their own ugly, broken little programming language for configuring it?
Why not just use an existing language?
The better solution is to fix the DSL, rather than introducing a general-purpose language.
Domain Specific Languages are really good at making code simpler, more concise and less redundant, but their most important feature is reducing bugs. A DSL can use primitives which have the possible error-modes built in; a general-purpose language would have to build that in a separate library layer, but the problems with that are 1) the general-purpose stuff is still available, which won't catch your errors 2) a decen
Re: (Score:2)
This release includes nftables (the successor of iptables)
Why does every network management tool include their own ugly, broken little programming language for configuring it?
Why not just use an existing language?
Like, when I get a packet from the network, I can just use Python:
if packet.origin == "127.0.0.1":
packet.drop()
elif packet.port == 80:
packet.forward(port = 1024)
etcetera.
The only way this could work and maintain the performance needed would be if you could write a parser that took your python code (that needed some sort of python parser to run) and converted it into a set of native nftables rules or bytecode that was efficient enough to be run at the kernel level as everyone else has described. This might actually be worth someone doing though.
I haven't played around with nftables yet but I am very familiar with ip tables (and chains) so I will just take a set of iptables r
Re: (Score:2)
Because python is twice the size of the entire kernel and all its modules. In the kernel efficiency matters.
Re: (Score:2)
Re: (Score:2)
Because a general purpose language is a bit like using a sledgehammer to open a walnut. You can do it, but there's better options.
Python / Javascript / LUA and other script languages don't have the performance needed and are needlessly more complicated than requirements.
The developers can create their own mini language which is fit for purpose and can be expressed in 10s - 100s of lines of EBNF. That can be fed into standard tools and a bunch of c++ code pops out the other end which can be used to compile o
Re: (Score:2)
Because compiler compilers exist and you could roll your own compiler that turns Pythonesque commands into proper nftables rules if you really wished to.
Re: (Score:2)
How does Python recognize the differences in example b/w IPv4 and IPv6? Like in your example, will
if packet.origin == "[::1]"
be automatically recognized as IPv6, or will it choke seeing it?
Re: (Score:2)
Re: (Score:1)
Because once a Python wraps its body around you, you are done for.
Re: (Score:2, Funny)
Maybe it's because Python is so flexible, simple and straightforward that it can be easily used for everything.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Every hipster except the Ruby on Rails hipsters. And the Java hipsters.
Does this mean the PMD for my Vaio Z2 will work? (Score:1)
Automatic GPU switching
Linux 3.12 added infrastructure support for automatic GPU switching in laptops with dual GPUs. This release adds support for this feature in AMD Radeon hardware.
(It already "works" but plug/unplug needs a lot of manual fucking around.
Or does it need more work in X?
Re: (Score:1)
http://xpressrazor.wordpress.com/2013/10/08/enable-and-use-open-source-radeon-drivers-in-a-muxless-hybrid-graphics-intelamd-setup/ [wordpress.com]
$ xrandr --listproviders
Providers: number : 2
Provider 0: id: 0x7d cap: 0xb, Source Output, Sink Output, Sink Offload crtcs: 2 outputs: 4 associated providers: 0 name:Intel
Provider 1: id: 0x57 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 6 outputs: 2 associated providers: 0 name:radeon
Then
$ xrandr --setprovideroffloadsink 0x57 0x7d
Now setting DRI_PRIME
Support for NFC payments ? In the kernel ?? (Score:5, Interesting)
misnomer perhaps? (Score:3, Informative)
nfc is a comms protocol, like bluetooth or ethernet, very timing sensitive, therefore best implemented on the kernel level. nfc is often used for payment systems, hen ce the conflation. am assuming - have no specific details.
Re:Support for NFC payments ? In the kernel ?? (Score:5, Informative)
Well, if you care to stop throwing peanuts from the gallery long enough to read about it, the patch adds driver support for interacting with a "secure element" embedded environment on the NFC hardware, from userspace, via a Netlink API. It's bascially passthrough from user space to the hardware.
Re: (Score:1)
From the newbies article:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=5ce3f32b5264b337bfd13a780452a17705307725
NFC: netlink: SE API implementation
Implementation of the NFC_CMD_SE_IO command for sending ISO7816 APDUs to NFC embedded secure elements. The reply is forwarded to user space through NFC_CMD_SE_IO as well.
Userland needs an API to talk to crypto chips
Re: (Score:1)
Re: (Score:3)
Re: (Score:2)
Re: (Score:1, Funny)
the kernel is adopting a pay-by-nfc development model in which linus will sell the kernel kernel from a street corner stand somewhere using NFC as payment. these funds will be used to further the development of linux.
in addition to receiving the linux kernel on a floppy disk (in order to remain competitive with bsd), you also get obscenities hurled at you when you question the form of payment or why you receive a floppy disk.
luckily the obscenities come at no extra charge.
Re: (Score:2)
Why are good trolls like this one kept modded -1 ?
Re: (Score:1)
Where the hell did you get that idea? I laugh my ass off at losers like you every time you make inept attempts at poking fun of Linux and OSS. I suppose the problem is that you can't seem to figure out that you are the joke. We are all laughing, we are just not laughing at what you say, but at the actual joke, to wit, you!
Re:Support for NFC payments ? In the kernel ?? (Score:5, Informative)
From TFA:
"This release implements support for the Secure Element. A netlink API is available to enable, disable and discover NFC attached (embedded or UICC ones) secure elements. With some userspace help, this allows to support NFC payments, used to implement financial transactions. Only the pn544 driver currently supports this API."
In other words, the kernel now contains the necessary API so the PC can correctly talk to a NFC Secure Element which is needed to be able to make payments over NFC, in tandem with userspace tools.
So yeah, the label is a bit misleading...
Just my luck... (Score:2, Interesting)
I FINNALY managed to wrap my head around how iptables work... Oh well, hopefully the new one will be easier.
Re: (Score:2, Troll)
#justusepfinstead
Seriously.
Re: (Score:2)
I'm with you on that one. I have to re-teach myself iptables each time I have to setup a configuration.
Re: (Score:2)
Fortunately the syntax for the new nft utility seems to be easier to understand http://kernelnewbies.org/nftables_examples [kernelnewbies.org]
Re: (Score:2)
Compared to what? Learning Japanese using Russian textbooks?
You want easy-to-understand, look at IPFW or PF. Those read like actual English sentences, not gibberish like iptables/nftables.
About the only good thing nftables does is enforce the ordering of rules elements so that everyone's rules will be written the same; and finally get rid --of --that --annoying --CLI --syntax --that --iptables --uses.
Here's hoping the devs a
I hope this fixes my black screen on boot (Score:2)
Re: (Score:2)
Re: (Score:1)
While that is an Intel CPU, it isn't using an Intel GPU. Instead it uses a PowerVR which is a brand of GPU often used in ARM devices, embedded, etc.
Long story short, there are no open source drivers in the kernel for that GPU. That means that you will have to find the PowerVR toolchain and compile your own drivers, but if it is anything like the rest of the ARM GPUs it won't work on a current kernel without a ton of patches. If you are lucky someone will have released an distro designed for your hardware
Re: (Score:2)
Re: (Score:2)
On your fanless PC, if you have a keyboard you can probably to bring up a text console. Might be easier than coming via ssh from another system, and works even without a live network. Sometimes also and etc. Note that distro's can and some do disable that text console in favor of X, but most will leave at least one console on F1.
You could also disable X from auto-starting.
no thanks (Score:1)