Canonical Drops CouchDB From Ubuntu One 93
rsk writes "Since the Ubuntu One desktop synchronization service was launched by Canonical it has always been powered by CouchDB, a popular document-oriented NoSQL data store with a powerful master-master replication architecture that runs in many different environments (servers, mobile devices, etc.). John Lenton, senior engineering manager at Canonical, announced that Canonical would be moving away from CouchDB due to a few unresolvable issues Canonical ran into in production with CouchDB and the scale/requirements of the Ubuntu One service. Instead, says Lenton, Canonical will be moving to a custom data storage abstraction layer (U1DB) that is platform agnostic as well as datastore agnostic; utilizing the native datastore on the host device (e.g. SQLite, MySQL, API layers, 'everything'). U1DB will be complete at some point after the 12.04 release."
Our sync service is not “powered by CouchDB& (Score:5, Informative)
Our structured data sync service is CouchDB, except for tomboy notes. Syncing files is a completely separate stack.
Re:Our sync service is not “powered by Couch (Score:2)
I'm sorry about that misstatement; thank you for the correction.
unresolvable issues (Score:5, Insightful)
dropped Ubuntu due to unresolvable issues with the way that they handle desktop environment migration. Liking Mint much better. Hope others are able to manage or migrate. Ubuntu is otherwise a very nice OS.
Re: (Score:2)
I use it as well. Very nice for a Debian distro and none of the hassles.
Re: (Score:2)
I upgraded recently to v10.10 and am very pleased. It's got a bit less old libraries, and PPAs let me keep current on the apps I use.
Re: (Score:2)
I upgraded recently to v10.10 and am very pleased. It's got a bit less old libraries, and PPAs let me keep current on the apps I use.
That's fine, and 11.04 will be fine as well, if you like Unity, or if you turn it off immediately after upgrading. But 11.10 is a different story, and that is mostly because of Unity or Gnome 3. It's a bad thing that they force this onto you, and I'm stuck with 11.04 as well for the moment. I hate it because we just pushed many people in our organisation to Ubuntu, and now they may have to deal with this new interface which doesn't work. For the moment I'm keeping them off this upgrade.
Re: (Score:2)
and 11.04 will be fine as well
Except for those of us who use the nvidia binary driver (for VDPAU) and are bitten by a Compiz bug that makes window movement slow as molasses.
Re: (Score:2)
11.10 is fine for me.
Then again, I use Kubuntu.
Re: (Score:2)
My desktop is now frozen at Ubuntu 11.04.
Probably a driver issue.
Re: (Score:1)
Seconded. My desktop is now frozen at Ubuntu 11.04, and i am looking for some other distro. Fedora, old love, maybe i will return to you...
Thirded. My desktops and laptops are now frozen at Ubuntu 10.04LTS.. There is FAR too much drama going on with Ubuntu since 10.04.. All the changes to new and crappy stuff. I like the new Mint/Debian, but it has one glaring issue..I use the OpenShot video editor on a daily basis, and getting it from a repo into Mint/Debian was not happening, so I'm sticking with U10.04 for now...
Re: (Score:1)
I find Mint very mint (Score:2)
I too dropped Ubuntu for Mint. Much happier now.
Re:I find Mint very mint (Score:4, Informative)
Re: (Score:1)
I feel genuinely sorry for your sake. It must suck to be rendered obsolete so easily. ;)
Re: (Score:2)
Mint 11 LXDE satisfied her completely, where Ubuntu 11.04 had left her dry and aching
Her vibrators run Linux?
Re: (Score:1)
I too dropped Ubuntu for Mint. Much happier now.
Thirded. It's probably not as mature as Ubuntu as I occasionally encounter bugs, but it's a lot better overall. Mint is the distro I give to friends who inquire about Linux. I've run FreeBSD and heaps of different distros on my desktop since way back when, I ran Ubuntu as long as possible, but now I've come to the point where I just want something with a consistent interface that works.
Re: (Score:2)
Spaceman really jumped the shark with this one.
Re: unresolvable issues with encryption on Mint (Score:2)
I'll second that. I needed to reinstall Linux on my notebook and after having fits with the most recent Ubuntu release, I was very much ready to give Linux Mint a try.
But, on all of my single user machines (desktop, notebook, netbook), I always use root-on-LVM-on-crypt (using LUKS for encryption) for my hard drive setup. This way, everything except for the small boot partition is encrypted. It works great and I've personally found the performance hit to be negligible. I even have my file server set up t
Specific Issues (Score:4, Interesting)
Re: (Score:2, Insightful)
The only "new thing" is a database abstraction layer that they should have already been using to begin with. Who in this day still writes their software heavily coupled to a single database rather than using a thin abstraction layer?
Re:Specific Issues (Score:5, Informative)
The only "new thing" is a database abstraction layer that they should have already been using to begin with. Who in this day still writes their software heavily coupled to a single database rather than using a thin abstraction layer?
we did, it's desktopcouch. Turned out to be too thin.
Re: (Score:2)
Re: (Score:3)
Yeah, and XML-encapsulated SQL, definitely needs at least one layer of XML. Definitely.
Re: (Score:3)
Just to be safe, you should store your data as XML elements in the fields of the same name:
select id from table
<id>1</id>
<id>2</id>
Re: (Score:2)
It's abstraction layers all the way down.
Re: (Score:2)
That's 80% of programming right there.
Re: (Score:2)
What do you suppose the ASF would do? The Apache Software Foundation hosts projects that meet certain standards, but they do not manage any of the projects themselves.
When I said, "reach out to the Apache project itself" I meant sending an email to the CouchDB project's mailing list or otherwise contacting contributors to the Apache CouchDB project. I didn't mean contacting the ASF.
Re: (Score:2)
Why? You're mostly just going to be contacting the same people who work for Couchbase since it's the company formed by a group of the main developers.
Re: (Score:1)
It's probably not scaling well. I've seen workloads that totally killed couchdb to a point MySQL blew it out
For instance, storing rss feeds. Couchdb seems ok for that until you try to use it.
Re: (Score:1)
also gwibber social feed aggregator, which uses it tends to crash it every other day or so.
Re: (Score:2, Interesting)
A lot of newer tech companies prefer to develop their own systems these days; there is a new culture of "dogfooding" i.e. building your tools from scratch and using your own product. There are good technical reasons for doing so when you are innovating, as existing systems will never quite meet your requirements. This is especially true of the cloud and "big data" (non-relational DBs), which are both still young and rapidly evolving.
As for specifically what went wrong: I suspect that comes under trade secre
Re: (Score:2)
Except that they aren't making their own tools. They are just rewriting their database abstraction layer so they can use a variety of didn't different backend databases rather than just couchdb.
Re: (Score:3)
Re:Specific Issues (Score:5, Insightful)
No, dogfooding is using the stuff you make yourself. It has nothing to do with WHY you made it, HOW you made it or anything else.
Its simply different than making a product to sell, and then using something entirely different internally. For instance, Microsoft selling Visual Source Safe ... but not actually using it internally for Windows because it sucked so hard. That is not dogfooding.
The term is 'eating your own dogfood', because you're eating what you made.
Linus dogfood's Linux for instance.
WTF is it with you kids fucking up phrases you don't understand. If you don't know what it means or where it came from will you fucking please stop god damn pretending you do. And stop calling all unpatched exploits zero day. They stopped being zero day within 24 hours of creation. After that they are just fucking exploits.
Re: (Score:2)
Creating your own tools for technical reasons contains the implication that you are using them because of the aforementioned technical reasons. ie, it must be "dogfooding". The rest of his post even explains why a lot of these tech shops must create their own tools -- because they are pushing innovation, so existing systems will never meet their requirements.
I think you should learn how to read before complaining about people's usages of phrases!
Re: (Score:1)
No, dogfooding is using the stuff you make yourself. It has nothing to do with WHY you made it, HOW you made it or anything else.
Its simply different than making a product to sell, and then using something entirely different internally. For instance, Microsoft selling Visual Source Safe ... but not actually using it internally for Windows because it sucked so hard. That is not dogfooding.
Additionally it implies that your own product is inferior. As I understand, it is using your own inferior product, as in "dog food is not fit for human consumption, but I eat it since I made it". I don't know about you, but I'd eat regular food instead of dog food if I had a choice. See MS and the transition of Hotmail's to MS servers for reference.
Linus dogfood's Linux for instance.
Linus probably doesn't eat his own dog food, he uses Linux because he likes it more. Now, if he would rather use MacOS or whatever, but still uses Linux because
Re: (Score:2)
Well there's one thing that never worked for me: Using UbuntuOne from behind a transparent HTTP proxy. Which means none of my Ubuntu work machines could use UbuntuOne. :)
Last time I checked (granted, a while back), they said it's a "known issue" and will be "addresed".
Or maybe it's something entirely different. I don't care what's happening in the background. All I need is for the damn solution to work
Original message from John Lenton (Canonical) (Score:5, Informative)
From the first days of Ubuntu One, before we were even in Ubuntu, we've
had a structured data storage sync service based around CouchDB.
For the last three years we have worked with the company behind CouchDB
to make it scale in the particular ways we need it to scale in our
server environment. Our situation is rather unique, and we were unable
to resolve some of the issues we came across. We were thus unable to
make CouchDB scale up to the millions of users and databases we have in
our datacentres, and furthermore we were unable to make it scale down to
be a reasonable load on small client machines.
Because of this, we are turning off most of our CouchDB-related
efforts. The contacts, notes and playlists databases will continue to
exist on our servers to support the related services, but direct
external access to the underlying databases will be shut off. Any other
databases will be deleted from our servers entirely.
For these same three years we have created and maintained desktopcouch,
which is a desktop service (and related library) to access CouchDB more
conveniently. Because we are no longer going to pursue CouchDB, we will
no longer be developing desktopcouch; in fact, if anybody wants to take
over, we'll be happy to work with you to make that official. For the
upcoming 12.04 the Ubuntu One packages will not depend on desktopcouch
nor couchdb in any way, and we'd recommend the distribution seriously
consider whether they want to continue having the package in main,
especially if no maintainer shows up.
Because we still believe there is a lot of value to our users in the
service we wanted to offer based on CouchDB, we're building something
new, based on what we've learned. It's very small, merely a layer of
abstraction and the definition of an API that will allow us and others
to build what is needed ontop of existing tools. We're calling it U1DB
for now, until it comes of age. If you're interested and techincally
inclined you can follow our progress on lp:u1db; unfortunately our
timing and resources are such that we can only promise the reference
python implementation will be ready in time for 12.04, and thus 12.04
will ship without Ubuntu One having a solid story around synchronizing
arbitrary structured data.
Thank you for reading.
https://lists.ubuntu.com/archives/ubuntu-desktop/2011-November/003474.html
Re: (Score:2, Funny)
Imagine no CouchDB. It's easy if you try.
- John Lenton
Re: (Score:2)
Re:Original message from John Lenton (Canonical) (Score:5, Informative)
Because the next release is an LTS, which will now be supported for 5 years. If couchdb is kept for the next release, it will need to be supported to another 5 years.
Re:Rolling your own (Score:4, Informative)
You did read the same thing I did, right? They *tried* using someone else's solution, and the solution did not fit their needs. If the existing solutions don't fit your needs, what else can you do other than roll your own? I guess you could drop the service/product altogether and just call it a day, but that doesn't sound like a great business model.
Re: (Score:3)
If CouchDB doesn't work, there are at least six other competing solutions that will tell you how they are a better fit for your needs than all the others. Cassandra, Hadoop/HBase, MongoDB, and more. If one doesn't work for you, you can waste as much time as you like trying the others!
Re: (Score:1)
Or they could use MySQL, SQLite, etc. instead like the summary mentions?
Re: (Score:2)
Do those other systems provide for arbitrary peer-to-peer data exchange/sync networks? Last I checked Couch was the only product in the NoSQL line up that provided robust support for distributed data networks.. Maybe I'm wrong or out of date.. Most of the others when I looked were able to sync data but it was for controlled sync among known peers, much the way MySQL and Postgres handle things -- not true/messy master-master replication among disorganized nodes spread around the internet.
Re: (Score:2)
Re: (Score:2)
Thanks - good tip on Cassandra. Riak has some capabilities that are close too. I can say that trying to do couch replication with large binary objects across unreliable networks (that is not in the same data center/peer network) is probably not a good idea anyway, even though the spec does support it..
Re: (Score:2)
why does it have to be NoSQL?
Re: (Score:2)
It doesn't - you could accomplish this with NNTP if you put enough work into it (we looked at that possibility for our implementation). Certainly you could do this with a SQL system or an rsync file distribution with custom file indices. I'm just saying that in my experience couchdb gives you more out of the box to accomplish this type of set up than anything else I've found. You can get further faster with this approach if you have the kinds of requirements I was describing, IMO.
Of course I'm open to somet
Re: (Score:3)
Lotus Notes does (maybe better). If you ignore the hate about its user interface, the server component (Lotus Domino) is very robust and scalable as a NoSQL provider.
Actually, Damien (author of CouchDB) is a former Lotus engineer and modeled his creation similar to Notes Storage File (NSF) structure.
Re: (Score:2)
Yeah - that's a good point. Damien specifically modeled Couch off of the good parts of Lotus Notes distribution (remember when IBM kept saying Lotus notes isn't really an email product - it just does email as a side effect? I now know why they were saying that). That said, it would be hard for my project to choose closed-licensed COTS when an OSS alternative like couch exists. Recognizing Lotus is almost certainly many times more robust and better engineered than couch for all these purposes. Thanks for poi
Re: (Score:2)
While not open source, it is now very competititvely priced. You might check out the licensing terms of the new IBM XWork server [edbrill.com], which is nothing but Domino server with a fixed annual cost of around $2K:
Re: (Score:2)
There are far more than 6 other vendors who will be glad to tell me how they are better and fit my needs better than the other guys ... and much like you, I'd be an idiot if I thought any of them had a clue.
Just because someone thinks their stuff kicks all ass for every situation doesn't make it actually true.
Re: (Score:1)
Re: (Score:2)
NO,NO,NO! Don't ever "roll your own" if there is something feasible already available. Too often you will end up making the same mistakes others did, possibly as long ago as 30+ years. Whenever possible take advantage of others experience (and blood, sweat, and tears) by using something existing so you don't do a poor job reinventing the wheel. And the advantage of open source is you can take someone elses work and build around it.
OK, I'll get off my soap box.
Re: (Score:1)
Huh? They are swapping their database abstraction layer so they can more easily use other backends like MySQL, SQLite, etc. instead of being tied to only couchdb. How exacty is that a bad thing? They aren't "rolling their own" since they are wanting to use already existing databases.
This is news to me (Score:1)
People use ubuntu one?
Re: (Score:2)
Free cloud storage with syncing software preinstalled with the distro. What's not to like? (Aside from the fact it's a cloud service at all- which many people seem surprisingly at ease with).
I don't use it, because I can't think of a reason to. But if you must have 5GB of data in cloud service, I can't see any reason why not Ubuntu One.
Why did syncing become so difficult? (Score:3)
I wonder what became so difficult about syncing data that it has to be re-invented all the time?
I was happy using tools like rsync, diff and unison for a long time, until the moment when even Linux desktop software is too posh to store their data in files.
Now every software uses another database, at one time even Amarok used a MySQL backend. What is better about this than just putting the data in a file? Or at least making this file the Single Point Of Truth? If you need the database for speed, you can check if the file changed since the last time and then update the database from the file's contents. But simple files have been syncing and merging and everything perfectly for ages. Now it seems like every software needs its own syncing service.
Is there any reason for this, except brading the most simple things (like copying a file), or making money with cloud storage?
Re: (Score:2)
I had the same set of thoughts. There are a ton of long standing solutions to these sorts of problems that would have served their needs far better than storing crap in a database. I don't get it either... If inode limits became a problem, there are lots of ways around that as well...
Re: (Score:2)
I wonder what became so difficult about syncing data that it has to be re-invented all the time?
I was happy using tools like rsync, diff and unison for a long time, until the moment when even Linux desktop software is too posh to store their data in files.
The indoctrination has reached you, too. It's not the software's data, it's your data, dammit. It should be in a documented format, readable by any application and friendly to rsync, version control et cetera.
Re: (Score:2)
Thanks for pointing out this freudian language glitch!
This actually sums up what I want to say. Most data could be stored and synced fine inside of files, even unicode text files most of the time, no binary. I don't need my chat logs in a database, I don't need calendar or contacts in a database. Especially if it is one that is running outside my user space and is not affected by backing up my home directory.
Re: (Score:2)
rsync has to be reinvented because its license is obnoxious.
diff and unison aren't efficient enough fi you're the poor bastard paying for bandwidth to/from the client for what is essentially a free service.
Re: (Score:3)
I would put my money on sloppy code that made the DB engine inefficient as well as ACID issues. Synching in real time in a distributed environment and staying ACID compliant is a *hard* problem. See this guy's research for some of the gory details: http://en.wikipedia.org/wiki/Jim_Gray_(computer_scientist) [wikipedia.org]
Re: (Score:2)
Syncing in real time is really something file systems don't seem to handle well. But that's more an issue with real time collaboration, not with syncing some documents and settings across a few devices.
Re: (Score:1)
Why Canonical dropped CouchDB (Score:2)
Canonical would be moving away from CouchDB due to a few unresolvable issues
Of course they'd want to drop CouchDB. It's clearly not web scale.
NIH syndrome (Score:1)