Source

riak_search / TRANSITION.org

#+OPTIONS: author:nil timestamp:nil

Migrating to Riak 0.11 from earlier versions of Riak

* Riak 0.10 to 0.11

** Customized configuration, not switching backends

   If you have customized your configuration file (etc/app.config),
   and you will be keeping your customized configuration file, your
   transition should be as easy as stopping the 0.10 node, installing
   the 0.11 binaries, and starting the node back up.  The node should
   find its ring and data in the same location as before, and will
   continue working as if nothing changed.

** Default configuration, or switching backends

   If you're using the default Riak configuration file, you'll need to
   be aware that the default storage backend has changed from DETS to
   Basho's new Bitcask.  You'll need to choose one of two upgrade
   paths in order to migrate your data from your old storage backend
   to the new one (these steps will also work, in general, from any
   backend to any other backend).  Your options are: backup/restore,
   and online rolling upgrade.

*** Backup/Restore

    The easiest way to migrate from one storage backend to another is
    to backup the cluster as it stands, shut it down, switch the
    backend (by upgrading or editing the config file), start the
    cluster back up, and then restore the data to the cluster.  Follow
    these steps to do this:

    1. With a running cluster, use the "riak-admin backup" command to
       backup the cluster's data:

       $ riak-admin backup riak@127.0.0.1 riak riak-backup.dat all

       This script will exit when backup is complete.  If your cluster
       has a lot of data, you can backup each node individually by
       switching "all" to "node" in the command line above, and then
       running the command once for each node, changing the node name
       in the command line.

    2. Shut down the cluster.

       $ riak stop

       Run this command on each node.

    3. Upgrade each node by installing the new 0.11 package.  If
       you're just switching backends (not upgrading), modify the
       configuration file in this step.

    4. Start the cluster

       $ riak start

       Run this command on each node.  The nodes will automatically
       reconnect to each other.

    5. Reload the cluster's data

       $ riak-admin restore riak@127.0.0.1 riak riak-backup.dat

       If you used the whole-cluster ("all") backup mode, you're now
       finished.  If you backed up each node individually, you'll need
       to run this command for each of those backup files (note that
       they have node names appended to what you provided on the
       backup command line).

    Once the restore script exits, all data should be restored to the
    cluster.  After checking that this is true, you may remove the
    data directories for the old storage backend.

*** Online Rolling Upgrade

    If you need your Riak cluster to remain running during the backend
    transition, and you have spare network capacity, you can use
    Riak's easy node addition/removal process to switch backends.

    1. Configure a new node, with the new backend you want to use, and
       add it to the cluster.

       newnode$ riak start
       newnode$ riak-admin join oldclusternode@10.0.0.100

    2. Remove one of the old-backend nodes from the cluster.

       oldnode0$ riak-admin leave

    3. When the old-backend node completes handoff, it will exit (use
       'ps' or similar to watch for it to shut down).  Once it has
       exited, upgrade to 0.11 (or switch the backend) and restart the
       node.

    4. Repeat steps 2 and 3 for each other old-backend node in the
       cluster.

    5. When all nodes are running on the new backend, shutdown the new
       node you set up in step 1 (if you wish, or leave it up if you like).

    Note: You can skip step 1 if you are running a cluster of more
    than one node, and your cluster can tolerate (capacity- and
    throughput-wise) being temporarily one node smaller.  Step 1 is
    merely an attempt to add extra capacity to the system before
    beging an expensive operation.

* Riak 0.9 and earlier to 0.11

  If you are upgrading from a Riak version earlier than 0.10, your
  only option for upgrade is backup and restore.  Please use the
  backup/restore method described in the 0.10-to-0.11-transition
  section above.

  You will also be interested in the following section, which
  discusses migration from 0.9 to 0.10.


* Migrating from Riak 0.9.x to 0.10

* Overview
  Riak has undergone significant restructuring in the transition from
  version 0.9.x to version 0.10.  If you are using the binary builds,
  please skip directly to the Configuration and Clients sections.  If
  you are building from source yourself, please review the whole of
  this document.

  NOTE: If the only files you have changed in your Riak source clone
  are those underneath the "rel" directory
  ("rel/overlay/etc/app.config", for example), the safest way to
  update is to make a fresh clone of the Riak repository, and then
  skip to the Configuration and Clients sections of this document for
  details about migrating your configurations and data.

* Requirements

** Erlang/OTP R13B04

   Riak 0.10 uses new features ("NIFs") provided by the latest
   Erlang/OTP release, R13B04.  If you are building from source, you
   will need this release or a newer one.

** Mercurial

   Riak 0.10 has moved several of its components into external
   repositories.  If you are building from source, you will need
   Mercurial installed to allow the Rebar build system to retrieve
   code from these external repositories.

* Dependencies

** Mochiweb, Webmachine, Erlang_js
   mochiweb, webmachine, and erlang_js are now pulled into the "deps"
   subdirectory, instead of being included in the "apps" subdirectory.
   If you are pulling 0.10 code into a repository that formerly had
   0.9.x in it, please remove the apps/mochiweb, apps/webmachine, and
   apps/erlang_js directories from your source tree.

   There is a chance that your update will also leave an "apps/riak"
   directory hanging around.  If it does, please remove this directory
   (Riak code has moved into the "apps/riak_core" and "apps/riak_kv"
   directories).

** make deps
   The "all" make target (and, by extension, the "rel" target as
   well), depend on a new "deps" target, which handles the fetching
   the dependencies (mochiweb, webmachine, erlang_js).

* Source

** Core/KV Split
   We've drawn a line through 0.9.x Riak, and divided it into two
   things, one called "riak_core", and the other called "riak_kv".

   The things that live in riak_core are those that deal with cluster
   membership.  Ring-claiming and the like.

   The things that live in riak_kv are those that deal with storing
   data.  Get and Put FSMs, backends, etc.

** Clients
   We've also moved the clients out of the client_lib subdirectory,
   and into their own language-specific repositories on BitBucket.  At
   http://bitbucket.org/basho/, you should find:

   + riak-python-client
   + riak-php-client
   + riak-erlang-client
   + riak-java-client
   + riak-javascript-client
   + riak-ruby-client

* Configuration

** app.config

  Splitting the "riak" Erlang application into the "riak_core" and
  "riak_kv" Erlang applications means that configuration options for
  each component need to move around in etc/app.config.

  Where before etc/app.config would have contained a section like:

  {riak, [
           %% many settings here
         ]},

  Now, etc/app.config should contain two sections like:

  {riak_core, [
               %% core-specific settings
              ]},
  {riak_kv, [
             %% kv-specific settings
            ]},

  The list of settings that moved to the riak_core section are:

  + choose_claim_fun 
  + cluster_name - string, defaults to "default"
  + default_bucket_props 
  + gossip_interval - integer, defaults to 60k msec
  + ring_creation_size - integer, defaults to 64
  + ring_state_dir - string
  + target_n_val - integer, defaults to 3
  + wants_claim_fun
  + web_ip - string. Used to be "riak_web_ip"
  + web_logdir - string.
  + web_port - integer. Used to be "riak_web_port"

  IMPORTANT: Note the rename of "riak_web_*" to just "web_*"

  The list of settings that moved to the riak_kv section are:

  + add_paths - list, defaults to []
  + handoff_concurrency - integer, defaults to 4
  + js_source_dir - string
  + js_vm_count - integer
  + mapred_name - string
  + raw_name - string
  + riak_kv_stat - boolean.
  + stats_urlpath - string
  + storage_backend - atom. Backend names are now prefixed as "riak_kv_" instead of just "riak_".
  + pb_ip - string
  + pb_port - integer

  IMPORTANT: The default backend has changed names from
  riak_dets_backend to riak_kv_dets_bakend.  Other backends have
  changed names as well.  This rename does not affect you if you are
  using the Innostore backend.

  If you did not have any of these settings defined in etc/app.config,
  you still do not need to define them in your new etc/app.config.

** Ring Storage
   Periodically, Riak nodes save the state of their ring to disk.  In
   0.9, these files were named "data/ring/riak_ring.*", but in 0.10,
   they're named "data/ring/riak_core_ring.*".  Renaming the old files
   to the new scheme is all you need to do to make the switch.

   If you referenced any Riak modules in your bucket properties, you
   will also need to change those references to point to the new
   module names after your cluster is running.

** Your Data
   The rest of your cluster's data, stored in the "data" directory
   ("data/dets" or "data/innodb", for example)
   should be safe to either leave in place, or copy to your new
   install location, depending on how you upgraded.
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.