ON-DISK FORMAT

UPGRADE PATH

On-disk formats, or even data structure formats, may be changed during anupgrade. Services wishing to do so, may so do it via thePaxosService::upgrade_format() call path. There is no formalized, unifiedformat versioning; the PaxosService class keeps track of itsformat_version through a key in the store, assumed an unsigned int, butit will be the service’s responsibility to give meaning to those versions.

AUTH MONITOR

versions

versions are represented with a single unsigned int. By default, the valuezero represents the absence of a formal upgraded format. The first formatversion was introduced in Dumpling; clusters upgrading to Dumpling saw theirformat version being increased from zero to one.:

  1. 0 to 1 - introduced in v0.65, dev release for v0.67 dumpling
  2. 1 to 2 - introduced in v12.0.2, dev release for luminous
  3. 2 to 3 - introduced in mimic
  4.  
  5. 0 - all clusters pre-dumpling
  6. 1 - all clusters dumpling+ and pre-luminous
  7. 2 - all clusters luminous+ and pre-mimic
  8. 3 - all clusters mimic+
  9.  
  10. version 1: introduces new-style monitor caps (i.e., profiles)
  11. version 2: introduces mgr caps and bootstrap-mgr key
  12. version 3: creates all bootstrap and admin keys if they don't yet exist

callstack

formatversion set on _PaxosService::refresh()::

  1. - initially called from Monitor::refresh_from_paxos
  2.   - initially called from Monitor::init_paxos()
  3.     - initially called from Monitor::preinit()

AuthMonitor::upgradeformat() called by _PaxosService::_active()::

  1. - called from C_Committed callback, from PaxosService::propose_pending()
  2. - called from C_Active callback, from PaxosService::_active()
  3. - called from PaxosService::election_finished()
  4.  
  5. - on a freshly deployed cluster, upgrade_format() will be first called
  6.   *after* create_initial().
  7. - on an existing cluster, upgrade_format() will be called after the first
  8.   election.
  9.  
  10. - upgrade_format() is irrelevant on a freshly deployed cluster, as there is
  11.   no format to upgrade at this point.

boil down

  • if format_version >= current_version then format is uptodate, return.

  • if features doesn’t contain LUMINOUS then current_version = 1

  • else if features doesn’t contain MIMIC then current_version = 2

  • else current_version = 3

if format_version == 0::

  1. - upgrade to format version 1
  2.   - move to new-style monitor caps (i.e., profiles):
  3.     - set daemon profiles for existing entities
  4.     - set profile for existing bootstrap keys

if format_version == 1::

  1. - upgrade to format version 2
  2.   - for existing entities:
  3.     - add new cap for mgr
  4.   - for existing 'mgr' entities, fix 'mon' caps due to bug from kraken
  5.     setting 'allow \*', and set 'allow profile mgr' instead.
  6.   - add bootstrap-mgr key.

if format_version == 2::

  1. - upgrade to format version 3
  2.   - create all bootstrap keys if they don't currently exist