Creating a Scrub policy in FileFly to clean up storage space in Swarm

A Scrub policy is an administrative policy in FileFly to clean up unused data in the destination storage.

It is recommended to run a scrub policy regularly to avoid keeping unnecessary files indefinitely.

A scrub will only delete data after it has passed its grace period.

The ‘grace period’ is the period of time that FileFly will keep data to protect against accidental deletion or aid with a backup recovery.

See below from filefly--

Backup & Scrub Grace Period

A Post-Restore Revalidate policy must be run following the restore of a primary storage volume.

In order to ensure that secondary storage objects required for this revalidation are not removed prematurely, the Scrub policy includes a grace period safeguard. The grace period must be sufficient to cover the time from when a backup is taken to when the restore and Post-Restore Revalidate are complete.

For example, if backups are kept for 30 days, the grace period should be at least 35 days (allowing 5 days for restoration). For more information, please refer to the Administration Guide.

As an additional safeguard, please set a global minimum grace period that you will allow across your deployment.

High level steps -

Check the global grace period
Create a scrub policy
Run / Schedule
Extra considerations

Check the grace period

First log into the FileFly admin console.

When logged in, select the small gear icon on the top ribbon at the right.

This will take you to the settings screen.

Scroll to the bottom.

Here you can set the minimum grace period. The lowest value it can be on a production license is 1day.

So when we create our scrub policy the data must at a minimum be deleted for 24hrs before it will be deleted by FileFly.

Create a Scrub policy

First in the Admin portal go to ‘Policies’

Select ‘Create policy’

In the operation panel select ‘Scrub Destination’

Fill out the details to match your environment like below. The Name and comments are free form fields so you can enter any text in there but try make it something memorable.

Run / Schedule the Scrub.

Every policy needs a task in FileFly to actually run.

A task is just a grouping with a schedule. It would be possible run multiple scrub policies at once if they were grouped into different tasks.

To create a simple task for our new scrub policy you can go to the “Tasks” panel and select “create task”

Start filling out the form.

Double click on your scrub policy to add it to the selected panel.

You can schedule or disable the schedule here and if you have email details entered in your settings panel in FileFly you can select ‘Notify’ for an email notification.

Click save when done.

After selecting save you will be taken back to the Tasks list for further editing.

If you want to run the task right away select the scrub task from the list again.

The click on ‘Run Now’

You’ll see output like below while the task is running.

And when it finishes you will see your summary here.

The task log can be quite verbose so if there is data that is not deleted you can go to the log to check why some files weren’t deleted.

Extra considerations

Scrub policies are useful to avoid storing unneeded data.

When a file is migrated via FileFly it creates a file with metadata on Swarm / other storage. If the same file is demigrated and edited in any way the next time that file is migrated it will create a new version / new file on the destination storage.

Scrub policies clean up those extra versions also keeping the more current based on the grace period.

When used in conjunction with the stub deletion tracking tool it is possible for end users to accidentally delete files on the end storage use with caution.