Automated backup of Atlassian Confluence Cloud is a relatively sad scene. API calls are undocumented, with only anecdotal references to be found as to which endpoints are for what. After much research, I ended up with this script for automating backups. The resulting zip file is ends up being stored in an S3 bucket, using AWS's [SSE-C](http://do…

confluence-cloud-backup

#!/usr/bin/env ruby

# This is for triggering & capturing backups of Atlassian's Confluence cloud
# service.
#
# https://docs.atlassian.com/jira/REST/cloud/
# https://bitbucket.org/atlassianlabs/automatic-cloud-backup/src/
# https://github.com/ghuntley/atlassian-cloud-backup/issues/5
# https://github.com/den-crane/my-stupid-scripts/blob/master/jira_backup.sh

require 'rest-client'
require 'aws-sdk'
require 'getoptlong'
require 'json'
require 'base64'
require_relative '../common/backup-notify'

#-------------------------------------------------------------------------------
# Script Usage
def print_usage_help
  puts <<-EOF
  #{$PROGRAM_NAME} <options>

  Where:
    -t, --target [jira|confluence]         (required) The Atlassian target for the backup
    -k, --sse-c-key <sse-c-key>            (required) AES256 key for SSE-C encryption
    -?, -h, --help                         Print this message

  Example:
    # #{$PROGRAM_NAME} --target confluence --sse-c-key 29381293adfaf102837c238b187613498
    # DEBUG=true #{$PROGRAM_NAME} --target confluence
  EOF
end

def die(message)
  alert_slack(@target, `uname -n`.chomp, 'undefined', message)
  puts " !!! ERROR: #{message}"

  exit(1)
end

#-------------------------------------------------------------------------------
# Do this if run directly
if $PROGRAM_NAME == __FILE__
  # Parse options
  opts = GetoptLong.new(
    [ '-?', '-h', '--help',  GetoptLong::NO_ARGUMENT ],
    [ '-t', '--target',      GetoptLong::REQUIRED_ARGUMENT ],
    [ '-k', '--sse-c-key',   GetoptLong::REQUIRED_ARGUMENT ]
  )

  # Default settings
  @target = nil
  ssec_key = nil

  # Process options
  opts.each do |opt, arg|
    case opt
      when '-h', '--help', '-?'
        print_usage_help
        exit(0)

      when '-t', '--target'
        @target = arg.strip.downcase

      when '-k', '--sse-c-key'
        ssec_key = arg.strip
        die('SSE-C key must be 32 characters long') unless ssec_key.length == 32
      end
  end

  die('SSE-C encryption is required for this backup') unless ssec_key
end # if $PROGRAM_NAME == __FILE__

#-------------------------------------------------------------------------------
# Set up vars
url_base = 'https://YOUR_ATLASSIAN_HOSTNAME'

prefix = case @target
when 'jira'
  '/rest'
when 'confluence'
  '/wiki/rest'
when nil
  die('Backup @target must be specified')
else
  die("Unknown Atlassian @target '#{@target}'")
end

url_login = "#{url_base}/rest/auth/1/session"
url_run_backup = "#{url_base}#{prefix}/obm/1.0/runbackup"
@url_progress = "#{url_base}#{prefix}/obm/1.0/getprogress.json"
url_download = "#{url_base}/wiki/download"
backup_dir = File.join('/data/backups/', @target)
backup_time = Time.now.strftime("%Y%m%d-%H%M%S")
backup_file_name = "#{backup_time}.zip"
local_file = "#{backup_dir}/#{backup_file_name}"
s3_bucket = 'your-backup-bucket'
s3_key = "atlassian/#{@target}/#{backup_file_name}"

# FIXME: have chef drop these off via encrypted databag
#        Could instead be stored as:
#          ~/.aws/config
#          ~/.aws/credentials

# Verify credentials are set in environment
%w[ AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY ATLASSIAN_USER ATLASSIAN_PASS ].each do |env_var|
  die(
    " !!! #{$PROGRAM_NAME} requires the '#{env_var}' to be defined!"
  ) if ENV[env_var].nil?
end

aws_access = ENV['AWS_ACCESS_KEY_ID']
aws_secret = ENV['AWS_SECRET_ACCESS_KEY']
atlassian_user = ENV['ATLASSIAN_USER']
atlassian_pass = ENV['ATLASSIAN_PASS']
@auth_string = 'Basic ' + Base64.strict_encode64("#{atlassian_user}:#{atlassian_pass}")

# Don't redirect output to logs if DEBUG is set
@debug_output = true if ENV['DEBUG']

# Redirect all output to the backup log file
log_file = File.join(backup_dir, "#{backup_time}.log")
log = File.open(log_file, File::WRONLY | File::APPEND | File::CREAT)
STDOUT.reopen(log) unless @debug_output
STDERR.reopen(log) unless @debug_output

# Backup status info
backup_success = false
backup_error = nil

#-------------------------------------------------------------------------------
# Check Jira/Confluence backup status
def check_status()
  begin
    status = RestClient::Request.execute(
      method: :get,
      url: @url_progress,
      headers: { Authorization: @auth_string }
    )
  rescue RestClient::Exception, RestClient::ExceptionWithResponse => e
    # Save error message to be saved into the log file
    error_message = "GET exception:  #{e.message}"
    error_message << " ==> #{e.response}" if e.response
    puts error_message
    raise error_message
  end

  if status.code == 204
    # Occurs when there's no current backup present
    # Define a fileName for the rest of the script to proceed as normal
    status = {
      'fileName' => 'Nothing at the moment',
      'currentStatus' => 'No Backup file present'
    }
  else
    status = JSON.parse(status.body)
  end

  # Bail out if Atlassian set a failedMessage
  die("failedMessage: #{status['failedMessage']}") if status.key?('failedMessage')

  puts status.to_s if @debug_output
  return status
end

#-------------------------------------------------------------------------------
# Trigger Atlassian Backup
puts " *** Triggering Atlassian Backup: #{@target}"
status = check_status
die(
  "Seems there is already a backup in progress!   #{status['currentStatus']}"
) unless status['alternativePercentage'] == '100%'

begin
  rest_response = RestClient::Request.execute(
    method: :post,
    url: url_run_backup,
    headers: {
      Authorization: @auth_string,
      'X-Atlassian-Token': 'no-check',
      'X-Requested-With': 'XMLHttpRequest',
      'Content-Type': 'application/json'
    },
    payload: '{"cbAttachments": "true"}'
  )
  puts "rest_response.code: #{rest_response.code}" if @debug_output

  die(
    "Triggering backup failed: #{rest_response.code},
    #{rest_response.body}"
  ) unless rest_response.code == 200
rescue RestClient::Exception, RestClient::ExceptionWithResponse => e
  # Save error message to be saved into the log file
  error_message = "POST exception:  #{e.message}"
  puts error_message if @debug_output
  error_message << " ==> #{e.response}" if e.response

  # Save error message to be saved into the log file
  backup_error = error_message

  # Sometimes things go sideways at Atlassian for this call
  # This will raise useful signal on Slack
  die(
    "Triggering backup failed: #{e.response.code}\n\n:warning: For further info:\n#{url_base}/wiki/plugins/servlet/ondemandbackupmanager/admin\n#{url_base}/wiki/rest/obm/1.0/getprogress.json"
  ) if e.response.code == 500

  raise error_message
end


#-------------------------------------------------------------------------------
# Wait for Backup Completion
puts 'Waiting for backup to complete' unless @debug_output
# Clear dummy status, set when 204 is returnd because there was no current
# backup file present.
status.clear

until status.key?('fileName') do
  puts " ... Backing up: #{status['currentStatus']}" if @debug_output
  sleep(3)
  status = check_status
end

#-------------------------------------------------------------------------------
# Download the Backup from Atlassian
source_file = status['fileName']
puts " *** Downloading zip from Atlassian: #{source_file}"

# Create backup_dir if it doesn't exist
Dir.mkdir(backup_dir) unless File.directory?(backup_dir)

# HAX: Streaming proc thing wont follow redirection, so figure out the final
# url beforehand
final_url = RestClient::Request.execute(
  method: :head,
  url: "#{url_download}/#{source_file}",
  headers: { Authorization: @auth_string }
).request.url

# This method performs the download, without loading the entire file into memory
#  http://stackoverflow.com/a/35609783/925987
File.open(local_file, 'w') {|f|
  block = proc {|response|
    response.read_body do |chunk|
      f.write chunk
    end
  }
  RestClient::Request.execute(
    method: :get,
    url: final_url,
    headers: { Authorization: @auth_string },
    block_response: block
  )
}

#-------------------------------------------------------------------------------
# Send backup file to S3
puts " *** Uploading backup zip to s3://#{s3_bucket}/#{s3_key}"

# Set up AWS S3 connection
s3_client = Aws::S3::Resource.new(region: 'us-east-1')

# Upload the file
begin
  obj = s3_client.bucket(s3_bucket).object(s3_key)
  obj.upload_file(
    local_file,
    {
      sse_customer_algorithm: 'AES256',
      sse_customer_key: ssec_key
    }
  )
rescue => e
  # Save error message to be saved into the log file
  backup_error = e

  error_message = "Error: #{e}\nat #{e.backtrace.join("\n   ")}"
  error_log_file = File.join(backup_dir, "#{backup_time}-error.log")
  File.write(error_log_file, error_message)

  # Report failure to Slack
  alert_slack(
    @target,
    `uname -n`.chomp,
    "s3://#{s3_bucket}/#{s3_key}",
    'failed'
  )

  raise error_message
end

#-------------------------------------------------------------------------------
# Looks like backup succeeded
backup_success = true

# Report success to Slack
alert_slack(
  @target,
  `uname -n`.chomp,
  "s3://#{s3_bucket}/#{s3_key}",
  "completed, encrypted"
)

# Create backup status file with the given status
# This can be used by external monitoring (Nagios) for alerting on age of
# the last backup beng over some threshold
status_file = File.join(backup_dir, backup_success ? 'backup_success.txt' : 'backup_fail.txt')
File.open(status_file, 'w') do |file|
  file.puts "Success: #{backup_success}"
  if backup_error
    file.puts "ERROR: #{backup_error}"
    file.puts "Backtrace: #{backup_error.backtrace.join("\n           ")}"
  end
end

I'm sure this could be improved in many ways, but I started learning Ruby only a couple months ago. Nonetheless, I hope this is useful to someone else, also using Atlassian Cloud, and needing to automate backups.

The latest lesson I learned about this script is that while cookie auth used to work (a few weeks ago), it no longer does. So I adjusted all calls to perform http basic auth.

I plan to replace those credential environment variables with reading a YAML file instead. This should be better for being run by cron, in that those env vars won't need to be in the cron.d file (potentially readable by anyone with machine access), but could instead be a file with 0600 permissions in the home dir of whatever user runs the script. It'd also be easier then to deploy those secrets from a config management tool (like Chef).