Last active
June 5, 2019 21:57
Revisions
-
efrecon revised this gist
Jun 5, 2019 . 2 changed files with 24 additions and 25 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -6,11 +6,10 @@ pattern. It should be scheduled on a regular basis from the outside, e.g. with ## Options `cleaner.sh` takes a number of single or double-dashed options, possibly followed by a command to execute once the removal operations have ended. It is preferrable to separate the list of options from the command using a double-dash, to mark the definitive end of the options. The following options are recognised. @@ -19,10 +18,10 @@ The following options are recognised. The value of this option should be the number of files to keep. It defaults to an empty string, which is understood as keeping all the files. ### `-f` or `--files` The value of this option is (usually) a glob-style pattern to point at the list of files to select from when removing the oldest ones. ### `-v` or `--verbose` This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -10,8 +10,8 @@ # All (good?) defaults VERBOSE=0 KEEP= PATHS= # Dynamic vars cmdname=$(basename $(readlink -f $0)) @@ -28,14 +28,14 @@ Description: finite amount of the youngest files Usage: $cmdname [-option arg --long-option(=)arg] ([--] command) where all dash-led options are as follows (long options can be followed by an equal sign): -v | --verbose Be more verbose -k | --keep Number of files to keep, defaults to empty, meaning all -f | --files List of files to select from for removal USAGE exit "$exitcode" } @@ -49,10 +49,10 @@ while [ $# -gt 0 ]; do --keep=*) KEEP="${1#*=}"; shift 1;; -f | --files) PATHS=$2; shift 2;; --files=*) PATHS="${1#*=}"; shift 1;; -v | --verbose) VERBOSE=1; shift;; @@ -68,28 +68,28 @@ while [ $# -gt 0 ]; do esac done if [ -z "$PATHS" ]; then echo "You need to specify files to select from for removal" >& 2 usage 1 fi # Conditional logging log() { if [ "$VERBOSE" = "1" ]; then echo "$1" fi } if [ -n "${KEEP}" -a "${KEEP}" -gt "0" ]; then log "Keeping only ${KEEP} youngest file(s) of $PATHS" while [ "$(ls $PATHS -1 -t | wc -l)" -gt "$KEEP" ]; do DELETE=$(ls $PATHS -1 -t | sort -r | tail -n 1) log "Removing old copy $DELETE" rm -rf $DELETE done fi if [ $# -ne "0" ]; then log "Executing $*" exec "$@" fi -
efrecon revised this gist
Jun 4, 2019 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
Empty file. -
efrecon revised this gist
Jun 4, 2019 . 2 changed files with 64 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,35 @@ # Cleaner for Backups `cleaner.sh` is a script to keep a finite number of backup (files) matching a pattern. It should be scheduled on a regular basis from the outside, e.g. with `cron`. ## Options `cleaner.sh` takes a number of single or double-dashed options, followed with the patterns/files to clean from. It is possible to separate the list of options from the destinations using a single double-dash, to mark the definitive end of the options. This enables to point at paths which would start with a dash sign (unlikely!). The following options are recognised. ### `-k` or `--keep` The value of this option should be the number of files to keep. It defaults to an empty string, which is understood as keeping all the files. ### `-t` or `--then` The value of this option is a command that will be evaluated once the removal operations have ended. It defaults to an empty string. ### `-v` or `--verbose` When this option is given, verbosity will be increased. ## Notes This utility is tuned for maximal compatibility. It only requires a basic POSIX shell. As a result, `cleaner.sh` can easilty be used from containers or in embedded systems. This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,29 @@ BSD 3-Clause License Copyright (c) 2019, Emmanuel Frecon <efrecon@gmail.com> All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -
efrecon created this gist
Jun 4, 2019 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,95 @@ #!/bin/sh # TODO: # add a --uploads option to specify the number of latest files to uploads. Good # in case we missed some. # add options to wait before starting, similar to mirror.tcl # add option to sleep and restart automatically #set -x # All (good?) defaults VERBOSE=0 KEEP="" THEN="" # Dynamic vars cmdname=$(basename $(readlink -f $0)) appname=${cmdname%.*} # Print usage on stderr and exit usage() { exitcode="$1" cat << USAGE >&2 Description: $cmdname will find the latest files matching a pattern and only keep a finite amount of the youngest files Usage: $cmdname [-option arg --long-option(=)arg] pattern where all dash-led options are as follows (long options can be followed by an equal sign): -v | --verbose Be more verbose -k | --keep Number of files to keep, defaults to empty, meaning all -t | --then Command to execute once done USAGE exit "$exitcode" } # Parse options while [ $# -gt 0 ]; do case "$1" in -k | --keep) KEEP=$2; shift 2;; --keep=*) KEEP="${1#*=}"; shift 1;; -t | --then) THEN=$2; shift 2;; --then=*) THEN="${1#*=}"; shift 1;; -v | --verbose) VERBOSE=1; shift;; -h | --help) usage 0;; --) shift; break;; -*) echo "Unknown option: $1 !" >&2 ; usage 1;; *) break;; esac done if [ $# -eq 0 ]; then echo "You need to specify sources to copy for offline backup" >& 2 usage 1 fi # Conditional logging log() { if [ "$VERBOSE" == "1" ]; then echo "$1" fi } if [ -n "${KEEP}" -a "${KEEP}" -gt "0" ]; then log "Keeping only ${KEEP} youngest file(s) of $@" while [ "$(ls $@ -1 -t | wc -l)" -gt "$KEEP" ]; do DELETE=$(ls $@ -1 -t | sort -r | tail -n 1) log "Removing old copy $DELETE" rm -rf $DELETE done fi if [ -n "${THEN}" ]; then log "Executing ${THEN}" eval "${THEN}" fi