Couchbase to Amazon S3 back-up script

On my last post, I was struggling with recovering all the documents from a Couchbase wacky installation after a system crash. It was an easy task to do, but I was very, very lucky to have the results of my "view all documents" view on a browser tab before the whole thing broke down.

I'm publishing on a couple of scripts on github, based on what I used to recover my database. The exporter (exporter.py) can be used along with crontab to back-up your couchbase buckets to Amazon S3, while recover.py can be used to retrieve the data from S3 and restore it to a Couchbase installation.

I found a small tweak, already documented on Couchbase issues, but I think it is related to python's library for Couchbase and not Couchbase itself.

Enjoy. And let me know if you are using it.

Update:

Thanks to Francis Varga, from www.crowdpark.com, I discovered a faster way to retrieve all documents, based on port 8092 and not on querying a view.

Popular posts from this blog

Diseño y comités

Una conversación que mantuve ayer con Helena Solà , creativa en una de las empresas de publicidad más importantes del país, me recordó una frase que me dijeron hace bastantes años (como 10 o así): Un camello es un caballo diseñado por un comité de expertos --- UPDATE --- Vaya tela tiene el mundillo blogger. Buscando la fuente original de la frase en google, me he ido a encontrar con http://multimaniaco.wordpress.com/ y resulta que en él aparecen comentarios de laquenoencuentrasusitio :) --- END UPDATE ---
Read more

Creating an e-commerce platform with Couchbase 2.0

Image
For the last couple of months, I’ve been involved in the development of an e-commerce website named http://presive.com , focused on beautiful, unique products created by designers all over the world with a story behind. When I first met @swilera , @eoingalla and @cmgottschalk , the idea seemed so interesting to me that I got involved in the project really fast. And then, in a few weeks, we had the first release of presive.com. We created it using Python, Django, Satchmo and running on the top of a PostgreSQL database engine and it worked well for some weeks, until we started developing the first business-level report queries. The whole object-relational stack the product was based on wasn’t enough at all for our purposes, so a technological change was needed. And then, we moved to Couchbase. At first, I was quite skeptical on the power of NoSQL databases, but I soon changed my mind when I visit couchbase and read everything about it . So let’s start on the conceptual model (yes, I am
Read more

Emergency Recovery Script from Couchbase disaster

Today, I dealt with a totally broken installation of Couchbase 2.0.0 Developer Preview. I couldn't find why but, although views were returning the right results, when I tried to retrieve a single document, I was getting the error below: { "error": "badrpc", "reason": "{'EXIT',{{{badmatch,{error,closed}}, [{mc_client_binary,cmd_binary_vocal_recv,5}, {mc_client_binary,get_meta,3}, {ns_memcached,do_handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]}, {gen_server,call, [{'ns_memcached-$data-default','ns...@127.0.0.1'}, {get_meta,<>,37}, 30000]}}}" } This is taken from Network Monitor panel on Chrome. On Coucbase admin console (:8091) the only message I got was "Unknown error". I tried to upgrade the couchbase version by downloading one of the latests builds, which is said to be more stable and have less problems that Developer Preview. But, this is what happened when I tried to instal
Read more