Class: S3Rotate::BackupRotator

Inherits:
Object
  • Object
show all
Includes:
Logging
Defined in:
lib/s3_rotate/core/backup_rotator.rb

Overview

BackupRotator Class Handles backup rotation locally and on S3

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Logging

level, logger, #logger

Constructor Details

#initialize(s3_client) ⇒ Object

Initialize a new BackupRotator instance.

Parameters:

  • s3_client —

    S3Client instance



26
27
28
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 26

def initialize(s3_client)
  @s3_client = s3_client
end

Instance Attribute Details

#s3_client ⇒ Object

attributes



17
18
19
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 17

def s3_client
  @s3_client
end

Instance Method Details

#promote(backup_name, file, type) ⇒ Object

Promote a backup into a different type of backup backup (for example, daily into weekly) This operation keeps the original daily file, and creates a new weekly backup

Parameters:

  • backup_name —

    String containing the name of the backup being updated

  • file —

    S3 File, file to be promoted

  • type —

    String representing the type the backup is being promoted into, one of “daily”, “weekly” or “monthly”

Returns:

  • created S3 Bucket File



249
250
251
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 249

def promote(backup_name, file, type)
  @s3_client.copy(backup_name, file, type)
end

#rotate(backup_name, local_backups_dir, max_local = 3, max_daily = 7, max_weekly = 4, max_monthly = 3) ⇒ Object

Rotate files (local, daily, weekly, monthly) and apply maximum limits for each type

Parameters:

  • backup_name —

    String containing the name of the backup to rotate

  • local_backups_path —

    String containing the path to the directory containing the backups

  • max_local (defaults to: 3) —

    Integer specifying the maximum number of local backups to keep

  • max_daily (defaults to: 7) —

    Integer specifying the maximum number of daily backups to keep

  • max_weekly (defaults to: 4) —

    Integer specifying the maximum number of weekly backups to keep

  • max_monthly (defaults to: 3) —

    Integer specifying the maximum number of monthly backups to keep

Returns:

  • nothing



42
43
44
45
46
47
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 42

def rotate(backup_name, local_backups_dir, max_local=3, max_daily=7, max_weekly=4, max_monthly=3)
  rotate_local(local_backups_dir, max_local)
  rotate_daily(backup_name, max_daily)
  rotate_weekly(backup_name, max_weekly)
  rotate_monthly(backup_name, max_monthly)
end

#rotate_daily(backup_name, max_daily) ⇒ Object

Rotate daily files

The rotation works as follows:

- Less than 7 days datediff between the oldest daily file and the most recent weekly file: do nothing
- More than 7 days datediff between the oldest daily file and the most recent weekly file: promote the oldest daily file to weekly file
- In both cases, apply the `max_daily`

Parameters:

  • backup_name —

    String containing the name of the backup being rotated

  • max_daily —

    Integer specifying the maximum number of daily backups to keep

    • If there are less than ‘max_daily` daily files: do nothing

    • If there are more than ‘max_daily` daily files: delete the oldest files to leave `max_daily` files

Returns:

  • nothing



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 64

def rotate_daily(backup_name, max_daily)
  # get backup files
  daily_backups  = @s3_client.remote_backups(backup_name, "daily").files
  weekly_backups = @s3_client.remote_backups(backup_name, "weekly").files

  # get most recent weekly file
  recent_weekly_file = weekly_backups.last ? weekly_backups.last.key : nil

  # look through daily backups to find which oness should be promoted
  daily_backups.each do |backup|
    # promote to weekly if applicable
    if should_promote_daily_to_weekly?(backup.key, recent_weekly_file)
      recent_weekly_file = promote(backup_name, backup, "weekly").key
    end
  end

  # cleanup old files
  if daily_backups.length > max_daily
    daily_backups.each_with_index do |backup, i|
      if i < daily_backups.length - max_daily
        logger.info("removing #{backup.key}")
        backup.destroy
      end
    end
  end
end

#rotate_local(local_backups_path, max_local) ⇒ Object

Rotate local files

Parameters:

  • local_backups_path —

    String containing the path to the directory containing the backups

  • max_local —

    Integer specifying the maximum number of local backups to keep

    • If there are less than ‘max_local` local files: do nothing

    • If there are more than ‘max_local` local files: delete the oldest files to leave `max_local` files

Returns:

  • nothing



168
169
170
171
172
173
174
175
176
177
178
179
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 168

def rotate_local(local_backups_path, max_local)
  # get backup files
  local_backups = FileUtils::files_in_directory(local_backups_path)

  # cleanup old files
  if local_backups.length > max_local
    local_backups[0..(local_backups.length - max_local - 1)].each do |backup|
      logger.info("removing #{local_backups_path}/#{backup}")
      File.delete("#{local_backups_path}/#{backup}")
    end
  end
end

#rotate_monthly(backup_name, max_monthly) ⇒ Object

Rotate monthly files

Parameters:

  • backup_name —

    String containing the name of the backup being rotated

  • max_monthly —

    Integer specifying the maximum number of month backups to keep

    • If there are less than ‘max_monthly` monthly files: do nothing

    • If there are more than ‘max_monthly` monthly files: delete the oldest files to leave `max_monthly` files

Returns:

  • nothing



143
144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 143

def rotate_monthly(backup_name, max_monthly)
  # get backup files
  monthly_backups = @s3_client.remote_backups(backup_name, "monthly").files

  # cleanup old files
  if monthly_backups.length > max_monthly
    monthly_backups.each_with_index do |backup, i|
      if i < monthly_backups.length - max_monthly
        logger.info("removing #{backup.key}")
        backup.destroy
      end
    end
  end
end

#rotate_weekly(backup_name, max_weekly) ⇒ Object

Rotate weekly files

The rotation works as follows:

- Less than 1 month datediff between the oldest weekly file and the most recent monthly file: do nothing
- More than 1 month datediff between the oldest weekly file and the most recent monthly file: promote the oldest daily file to weekly file
- In both cases, apply the `max_weekly`

Parameters:

  • backup_name —

    String containing the name of the backup being rotated

  • max_weekly —

    Integer specifying the maximum number of weekly backups to keep

    • If there are less than ‘max_weekly` weekly files: do nothing

    • If there are more than ‘max_weekly` weekly files: delete the oldest files to leave `max_weekly` files

Returns:

  • nothing



106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 106

def rotate_weekly(backup_name, max_weekly)
  # get backup files
  weekly_backups  = @s3_client.remote_backups(backup_name, "weekly").files
  monthly_backups = @s3_client.remote_backups(backup_name, "monthly").files

  # get most recent monthly file
  recent_monthly_file = monthly_backups.last ? monthly_backups.last.key : nil

  # look through weekly backups to find which oness should be promoted
  weekly_backups.each do |backup|
    # promote to monthly if applicable
    if should_promote_weekly_to_monthly?(backup.key, recent_monthly_file)
      recent_monthly_file = promote(backup_name, backup, "monthly").key
    end
  end

  # cleanup old files
  if weekly_backups.length > max_weekly
    weekly_backups.each_with_index do |backup, i|
      if i < weekly_backups.length - max_weekly
        logger.info("removing #{backup.key}")
        backup.destroy
      end
    end
  end
end

#should_promote_daily_to_weekly?(daily_file, weekly_file) ⇒ Boolean

Check whether ‘daily_file` should be promoted into a weekly file Only promote a daily file if the most recent weekly backup is one week old

Parameters:

  • daily_file —

    String, filename of the daily backup to be checked for promotion

  • weekly_file —

    String, filename of the most recent weekly backup

Returns:

  • (Boolean) —

    Boolean, True or False, whether the file should be promoted



190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 190

def should_promote_daily_to_weekly?(daily_file, weekly_file)
  # never promote if no daily file
  return false if not daily_file

  # always promote if no weekly file
  return true if not weekly_file

  # retrieve the date of each file
  begin
    date_daily_file  = FileUtils::date_from_filename(daily_file)
    date_weekly_file = FileUtils::date_from_filename(weekly_file)
  rescue
    print "Wrong date (Date.parse in should_promote_daily_to_weekly)."
    return false
  end

  # perform date comparison
  return date_daily_file - date_weekly_file >= 7
end

#should_promote_weekly_to_monthly?(weekly_file, monthly_file) ⇒ Boolean

Check whether ‘weekly_file` should be promoted into a monthly file Only promote a weekly file if the most recent monthly backup is one month old

Parameters:

  • weekly_file —

    String, filename of the weekly backup to be checked for promotion

  • monthly_file —

    String, filename of the most recent monthly backup

Returns:

  • (Boolean) —

    Boolean, True or False, whether the file should be promoted



219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
 
# File 'lib/s3_rotate/core/backup_rotator.rb', line 219

def should_promote_weekly_to_monthly?(weekly_file, monthly_file)
  # never promote if no weekly file
  return false if not weekly_file

  # always promote if no monthly file
  return true if not monthly_file

  # retrieve the date of each file
  begin
    date_weekly_file  = FileUtils::date_from_filename(weekly_file)
    date_monthly_file = FileUtils::date_from_filename(monthly_file)
  rescue
    print "Wrong date (Date.parse in should_promote_weekly_to_monthly)."
    return false
  end

  # perform date comparison
  return date_weekly_file.prev_month >= date_monthly_file
end