m/shell-scripting-templates
1
0
Fork 0
mirror of https://github.com/natelandau/shell-scripting-templates.git synced 2025-11-16 08:53:47 -05:00
Code Issues Packages Projects Releases Wiki Activity

Squashed commit of the following:

Browse Source 
commit 61bf734812cb62ba6e0ec224bc15f7928705a8a2
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Thu Oct 21 15:44:21 2021 -0400

    Major overhaul continued

     - rename templates
     - add checks utilities
     - add new array utilities
     - rename files
     - add assorted utilities
     - improve documentation

commit 546178fff3b526f492eb0eeffc63f79537e75de3
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Wed Oct 20 16:31:14 2021 -0400

    Update conventions

commit f6d0642f85518efda9c5d8472b99d1c14163e381
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Wed Oct 20 09:47:09 2021 -0400

    minor formatting changes

commit 2217612b55e3f9faf803a2d0c937ea2261206505
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Tue Oct 19 17:59:09 2021 -0400

    add new functions

commit 347ba7aa738dcd6a5ad9d70886b38da3a17dc89e
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Tue Oct 19 12:06:44 2021 -0400

    major overhaul

    - Add standaloneTemplate.sh
    - Rework README
    - Refactor inline documentation
    - Enforce coding standards
    - Remove CSV utilities
    - Add new array utilities
    - add _useGNUutils_
    - more ...

commit cd8e0d49aef25eeaf6b3e71a3c9e1f29ab9b06f5
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Sun Oct 17 09:56:08 2021 -0400

    Add debug functions

commit f7c5c0a3d19815dcc6ba80b5f5a2ebb77ef88b07
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Sat Oct 16 21:10:01 2021 -0400

    add new array functions

    _joinArray_, _isEmptyArray_, _sortArray_, _reverseSortArray_, and _mergearrays_

commit d8bc3d8cabdbcee3c479f97b43a45bdfe3bdafe0
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Fri Oct 15 17:27:12 2021 -0400

    add _columnize_

commit 2fd2ae9435f476bc3968c3eb0d793db4bf1d9eaf
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Mon Oct 11 22:17:45 2021 -0400

    _progressBar_: Fix unbound variable

commit e8933d15fc955a1acc665e9a081f131e681855d5
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Sun Oct 10 11:50:42 2021 -0400

    _alert_: header now underlined

commit c9ce894361dec7d3513c038794a155519baf26bc
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Tue Oct 5 09:49:42 2021 -0400

    _alert_: line numbers to gray

commit 4aaddd336ce613f629a7e6a62ef3b27ffc24d22d
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Fri Oct 8 15:05:20 2021 -0400

    _usage_ to stdout

commit e2372fc3122ec1f20acc27f04d29b3785f014e25
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Tue Oct 5 09:38:26 2021 -0400

    _setPATH_: remove unneeded logic

commit e60c75b6c954ac4bd146e2758252168027b9a43d
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Tue Oct 5 09:25:38 2021 -0400

    _findSource_: bugfix

commit 0e84912e1ccd7203e5beff9f8737f8374f4aa5d8
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Thu Sep 30 16:29:25 2021 -0400

    add requirements to documentation

commit 2c24843e3ada591e1868a94416e40b5ac0aa4994
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Thu Sep 30 15:34:10 2021 -0400

    _uniqueFilename_: improve extension handling

commit 08bc2dfdcc8632efee9179e9c960a574fc17cf0c
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Mon Sep 27 15:13:53 2021 -0400

    improve hooks script

commit 641918f1559d3b3aa38a9bbdf418938b2b81c176
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Fri Sep 24 08:16:52 2021 -0400

    _inArry_: case insensitivity

commit eae10f170680540fdb4a1222add7e54f8785ea63
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Mon Sep 20 18:31:44 2021 -0400

    clean up alerting

commit 700acd56f57fd57db84ef0e232ef41cdd7aee43c
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Mon Sep 20 18:22:11 2021 -0400

    refactor _execute_

commit d893f86900a9fed9d91a0c9cc06c13b6b34d9926
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Mon Sep 20 18:19:18 2021 -0400

    'fatal' replaces 'die'

commit 3326857bf127bef36cd9982246aa5b826d796d0a
Author: Nathaniel Landau <nate@natelandau.com>
Date:   Fri Sep 17 08:29:50 2021 -0400

    _execute_: ensure quiet and verbose work together
This commit is contained in:
Nathaniel Landau
2021-10-21 16:03:27 -04:00
parent 1e2d2ff287
commit 8bb89541e8
35 changed files with 5769 additions and 2791 deletions
Download Patch File Download Diff File Expand all files Collapse all files

395
README.md
View File

@@ -1,108 +1,135 @@
# Shell Scripting Templates and Utilities
A collection of shell scripting utilities and templates used to ease the creation of BASH scripts. [BATS](https://github.com/bats-core/bats-core) provides unit testing capabilities. All tests are in the `tests/` repo.
A collection of BASH utility functions and script templates used to ease the creation of portable and hardened BASH scripts with sane defaults.
## Bash Script Template Usage
## Usage
To create a new script, copy `scriptTemplate.sh` to a new file and make it executable `chmod 755 [newscript].sh`. Place your custom script logic within the `_mainScript_` function at the top of the script.
There are **two Script Templates** located in the root level of this repository. The usage of these templates is described in detail below.
### Script Template Usage
BASH **Utility functions** are located within the `utilities/` folder.
Default flags included in the base template are:
Complex `sed` find/replace operations are supported with the files located in `sedfiles/`. Read [the usage instructions](sedfiles/README.md).
- `-h`: Prints the contents of the `_usage_` function. Edit the text in that function to provide help
- `-l [level]`: Log level of the script. One of: `FATAL`, `ERROR`, `WARN`, `INFO`, `DEBUG`, `ALL`, `OFF` (Default is '`ERROR`')
- `-n`: Dryrun, sets `$DRYRUN` to `true` allowing you to write functions that will work non-destructively using the `_execute_` function
- `-v`: Sets `$VERBOSE` to `true` and prints all debug messages to stdout
- `-q`: Runs in quiet mode, suppressing all output to stdout. Will still write to log files
- `--force`: If using the `_seekConfirmation_` utility function, this skips all user interaction. Implied `Yes` to all confirmations.
**Automated testing** is provided using [BATS](https://github.com/bats-core/bats-core). All tests are in the `tests/` repo. A git pre-commit hook provides automated testing is located in the `.hooks/` directory. Read about [how to install the hook](.hooks/README.md).
## Bash Script Templates Usage
To create a new script, copy one of the script templates to a new file and make it executable `chmod 755 [newscript].sh`. Place your custom script logic within the `_mainScript_` function at the top of the script.
### Script Templates
There are two templates located at the root level of this repository.
- **`template_source_utils.sh`** - A lean template which attempts to source all the utility functions from this repository. You will need to update the path to the utilities folder sent to `_sourceUtilities_` at the bottom of the script. This template will not function correctly if the utilities are not found.
- **`template_standalone.sh`** - For portability, the standalone template does not assume that this repository is available. Copy and paste the individual utility functions under the `### Custom utility functions` line.
### Code Organization
The script templates are roughly split into three sections:
- TOP: Write the main logic of your script within the `_mainScript_` function. It is placed at the top of the file for easy access and editing. However, it is invoked at the end of the script after options are parsed and functions are sourced.
- MIDDLE: Functions and default variable settings are located just below `_mainScript_`.
- BOTTOM: Script initialization (BASH options, traps, call to `_mainScript_`, etc.) is at the bottom of the template
### Default Options
These default options and global variables are included in the templates and used throughout the utility functions. CLI flags to set/unset them are:
- **`-h, --help`**: Prints the contents of the `_usage_` function. Edit the text in that function to provide help
- **`--logfile [FILE]`** Full PATH to logfile. (Default is `${HOME}/logs/$(basename "$0").log`)
- **`loglevel [LEVEL]`**: Log level of the script. One of: `FATAL`, `ERROR`, `WARN`, `INFO`, `DEBUG`, `ALL`, `OFF` (Default is '`ERROR`')
- **`-n, --dryrun`**: Dryrun, sets `$DRYRUN` to `true` allowing you to write functions that will work non-destructively using the `_execute_` function
- **`-q, --quiet`**: Runs in quiet mode, suppressing all output to stdout. Will still write to log files
- **`-v, --verbose`**: Sets `$VERBOSE` to `true` and prints all debug messages to stdout
- **`--force`**: If using the `_seekConfirmation_` utility function, this skips all user interaction. Implied `Yes` to all confirmations.
You can add custom script options and flags to the `_parseOptions_` function.
### Script Template Functions
scriptTemplate.sh includes some helper functions to perform common tasks.
- `_alert_` Provides alerting and logging functionality. See notes below.
- `_trapCleanup_` Cleans up files on error
- `_makeTempDir_` Creates a temp directory to house temporary files
- `_acquireScriptLock_` Acquires script lock to avoid race conditions on files
- `_functionStack_` Prints the function stack in use to aid debugging
- `_parseOptions_` Parse options and take user input (`-a`, `--some-flag`, and `--some-file [filename]` supported)
- `_usage_` Prints help text when `-h` passed
- `_safeExit_` Used to exit gracefully, cleaning up all temporary files etc.
### Script Initialization
The bottom of the script template file contains a block which initializes the script. Comment, uncomment, or change the settings here for your needs
```bash
trap '_trapCleanup_ ${LINENO} ${BASH_LINENO} "${BASH_COMMAND}" "${FUNCNAME[*]}" "${0}" "${BASH_SOURCE[0]}"' \
EXIT INT TERM SIGINT SIGQUIT
set -o errtrace # Trap errors in subshells and functions
set -o errexit # Exit on error. Append '||true' if you expect an error
set -o pipefail # Use last non-zero exit code in a pipeline
# shopt -s nullglob globstar # Make `for f in *.txt` work when `*.txt` matches zero files
IFS=$' \n\t' # Set IFS to preferred implementation
# set -o xtrace # Run in debug mode
set -o nounset # Disallow expansion of unset variables
# [[ $# -eq 0 ]] && _parseOptions_ "-h" # Force arguments when invoking the script
_parseOptions_ "$@" # Parse arguments passed to script
# _makeTempDir_ "$(basename "$0")" # Create a temp directory '$tmpDir'
# _acquireScriptLock_ # Acquire script lock
_mainScript_ # Run the main logic script
_safeExit_ # Exit cleanly
trap '_trapCleanup_ ${LINENO} ${BASH_LINENO} "${BASH_COMMAND}" "${FUNCNAME[*]}" "${0}" "${BASH_SOURCE[0]}"' EXIT INT TERM SIGINT SIGQUIT SIGTERM ERR
# Trap errors in subshells and functions
set -o errtrace
# Exit on error. Append '||true' if you expect an error
set -o errexit
# Use last non-zero exit code in a pipeline
set -o pipefail
# Confirm we have BASH greater than v4
[ "${BASH_VERSINFO:-0}" -ge 4 ] || {
printf "%s\n" "ERROR: BASH_VERSINFO is '${BASH_VERSINFO:-0}'. This script requires BASH v4 or greater."
exit 1
}
# Make `for f in *.txt` work when `*.txt` matches zero files
shopt -s nullglob globstar
# Set IFS to preferred implementation
IFS=$' \n\t'
# Run in debug mode
# set -o xtrace
# Source utility functions
_sourceUtilities_ "${HOME}/repos/shell-scripting-templates/utilities"
# Initialize color constants
_setColors_
# Disallow expansion of unset variables
set -o nounset
# Force arguments when invoking the script
# [[ $# -eq 0 ]] && _parseOptions_ "-h"
# Parse arguments passed to script
_parseOptions_ "$@"
# Create a temp directory '$TMP_DIR'
# _makeTempDir_ "$(basename "$0")"
# Acquire script lock
# _acquireScriptLock_
# Source GNU utilities for use on MacOS
# _useGNUUtils_
# Run the main logic script
_mainScript_
# Exit cleanly
_safeExit_
```
# Utility Files
# Utility Functions
The files within `utilities/` contain BASH functions which can be used in your scripts. Each included function includes detailed usage information. Read the code for instructions.
The files within `utilities/` contain BASH functions which can be used in your scripts. Each included function includes detailed usage information. Read the inline comments within the code for detailed usage instructions.
## Including Utility Functions
Within the `utilities` folder are many BASH functions meant to ease development of more complicated scripts. These can be included in the template in two ways.
#### 1. Copy and Paste
#### 1. Copy and paste into standaloneTemplate.sh
You can copy any complete function from the Utilities and place it into your script. Copy it beneath the end of `_mainscript_()`
You can copy any complete function from the Utilities and place it into your script. Copy it beneath the `### Custom utility functions` line.
#### 2. Source entire utility files
#### 2. Source all the utility files by using scriptTemplate.sh
You can source entire utility files by pasting the following snippet into your script beneath `_mainScript_()`. Be sure to replace `[PATH_TO]` with the full path to this repository.
```bash
_sourceHelperFiles_() {
# DESC: Sources script helper files.
local filesToSource
local sourceFile
filesToSource=(
"[PATH_TO]/shell-scripting-templates/utilities/alerts.bash"
"[PATH_TO]/shell-scripting-templates/utilities/baseHelpers.bash"
"[PATH_TO]/shell-scripting-templates/utilities/arrays.bash"
"[PATH_TO]/shell-scripting-templates/utilities/files.bash"
"[PATH_TO]/shell-scripting-templates/utilities/macOS.bash"
"[PATH_TO]/shell-scripting-templates/utilities/numbers.bash"
"[PATH_TO]/shell-scripting-templates/utilities/services.bash"
"[PATH_TO]/shell-scripting-templates/utilities/textProcessing.bash"
"[PATH_TO]/shell-scripting-templates/utilities/dates.bash"
)
for sourceFile in "${filesToSource[@]}"; do
[ ! -f "${sourceFile}" ] \
&& {
echo "error: Can not find sourcefile '${sourceFile}'."
echo "exiting..."
exit 1
}
source "${sourceFile}"
done
}
_sourceHelperFiles_
```
`scriptTemplate.sh` contains a function to source all the utility files into the script. Beware, this will require a full path to the location of this repository and will result in a script that will not be portable to other systems.
## alerts.bash
- `_setColors_` Sets color constants for alerting (**Note:** Colors default to a dark theme.)
- `_alert_` Performs alerting functions including writing to a log file and printing to screen
- - **`_columnizeOutput_`** Creates a column output for key/value pairs with line wrapping for the right column (value)
- -**`_printFuncStack_`** Prints the function stack in use. Used for debugging, and error reporting
- **`_alert_`** Performs alerting functions including writing to a log file and printing to screen
- **`_centerOutput_`** Prints text in the center of the terminal window
- **`_setColors_`** Sets color constants for alerting (**Note:** Colors default to a dark theme.)
Basic alerting, logging, and setting color functions (included in `scriptTemplate.sh` by default). Print messages to stdout and to a user specified logfile using the following functions.
@@ -111,121 +138,183 @@ debug "some text" # Printed only when in verbose (-v) mode
info "some text" # Basic informational messages
notice "some text" # Messages which should be read. Brighter than 'info'
warning "some text" # Non-critical warnings
error "some text" # Error state warnings. (Does not stop the script)
error "some text" # Prints errors and the function stack but does not stop the script.
fatal "some text" # Fatal errors. Exits the script
success "some text" # Prints a success message
header "some text" # Prints a header element
dryrun "some text" # Prints commands that would be run if not in dry run (-n) mode
```
Set the following variables for the alert functions to work.
The following global variables must be set for the alert functions to work
- `$LOGFILE` - Location of a log file
- `$LOGLEVEL` - One of: FATAL, ERROR, WARN, INFO, DEBUG, ALL, OFF (Default is 'ERROR')
- `$QUIET` - If `true`, nothing will print to STDOUT (Logs files will still be populated)
- `$DEBUG` - If `true`, prints `debug` and `verbose` level alerts to stdout
- **`$DEBUG`** - If `true`, prints `debug` level alerts to stdout. (Default: `false`)
- **`$DRYRUN`** - If `true` does not eval commands passed to `_execute_` function. (Default: `false`)
- **`$LOGFILE`** - Path to a log file
- **`$LOGLEVEL`** - One of: FATAL, ERROR, WARN, INFO, DEBUG, ALL, OFF (Default: `ERROR`)
- **`$QUIET`** - If `true`, prints to log file but not stdout. (Default: `false`)
## arrays.bash
Common functions for working with BASH arrays.
Utility functions for working with arrays.
- `_inArray_` Determine if a value is in an array
- `_join_` Joins items together with a user specified separator
- `_setdiff_` Return items that exist in ARRAY1 that are do not exist in ARRAY2
- `_removeDupes_` Removes duplicate array elements
- `_randomArrayElement_` Selects a random item from an array
- **`_dedupeArray_`** Removes duplicate array elements
- **`_forEachDo_`** Iterates over elements and passes each to a function
- **`_forEachFilter_`** Iterates over elements, returning only those that are validated by a function
- **`_forEachFind_`** Iterates over elements, returning the first value that is validated by a function
- **`_forEachReject_`** Iterates over elements, returning only those that are NOT validated by a function
- **`_forEachValidate_`** Iterates over elements and passes each to a function for validation
- **`_inArray_`** Determine if a value is in an array
- **`_isEmptyArray_`** Checks if an array is empty
- **`_joinArray_`** Joins items together with a user specified separator
- **`_mergeArrays_`** Merges the values of two arrays together
- **`_randomArrayElement_`** Selects a random item from an array
- **`_reverseSortArray_`** Sorts an array from highest to lowest
- **`_setdiff_`** Return items that exist in ARRAY1 that are do not exist in ARRAY2
- **`_sortArray_`** Sorts an array from lowest to highest
## baseHelpers.bash
## checks.bash
Commonly used functions in many scripts
Functions for validating common use-cases
- `_execute_` Executes commands with safety and logging options. Respects `DRYRUN` and `VERBOSE` flags.
- `_findBaseDir_` Locates the real directory of the script being run. Similar to GNU readlink -n
- `_checkBinary_` Check if a binary exists in the search PATH
- `_haveFunction_` Tests if a function exists
- `_pauseScript_` Pause a script at any point and continue after user input
- `_progressBar_` Prints a progress bar within a for/while loop
- `_rootAvailable_` Validate we have superuser access as root (via sudo if requested)
- `_runAsRoot_` Run the requested command as root (via sudo if requested)
- `_safeExit_` Cleans up temporary files before exiting a script
- `_seekConfirmation_` Seek user input for yes/no question
- `_setPATH_` Add directories to $PATH so script can find executables
## csv.bash
Functions to write to a CSV file.
- `_makeCSV_` Creates a new CSV file if one does not already exist
- `_writeCSV_` Takes passed arguments and writes them as a comma separated line
- **`_binaryExists_`** Check if a binary exists in the PATH
- **`_functionExists_`** Tests if a function is available in current scope
- **`_isInternetAvailable_`** Checks if Internet connections are possible
- **`_isAlpha_`** Validate that a given variable contains only alphabetic characters
- **`_isAlphaDash_`** Validate that a given variable contains only alpha-numeric characters, as well as dashes and underscores
- **`_isAlphaNum_`** Validate that a given variable contains only alpha-numeric characters
- **`_isDir_`** Validate that a given input points to a valid directory
- **`_isEmail_`** Validates that an input is a valid email address
- **`_isFile_`** Validate that a given input points to a valid file
- **`_isIPv4_`** Validates that an input is a valid IPv4 address
- **`_isIPv6_`** Validates that an input is a valid IPv6 address
- **`_isNum_`** Validate that a given variable contains only numeric characters
- **`_isTerminal_`** Checks if script is run in an interactive terminal
- **`_rootAvailable_`** Validate we have superuser access as root (via sudo if requested)
- **`_varIsEmpty_`** Checks if a given variable is empty or null
- **`_varIsFalse_`** Checks if a given variable is false
- **`_varIsTrue_`** Checks if a given variable is true
## dates.bash
Common utilities for working with dates in BASH scripts.
Functions for working with dates and time.
- `_monthToNumber_` Convert a month name to a number
- `_numberToMonth_` Convert a month number to its name
- `_parseDate_` Takes a string as input and attempts to find a date within it to parse into component parts (day, month, year)
- `_formatDate_` Reformats dates into user specified formats
- **`_convertToUnixTimestamp_`** Converts a date to unix timestamp
- **`_countdown_`** Sleep for a specified amount of time
- **`_dateUnixTimestamp_`** Current time in unix timestamp
- **`_formatDate_`** Reformats dates into user specified formats
- **`_fromSeconds_`** Convert seconds to HH:MM:SS
- **`_monthToNumber_`** Convert a month name to a number
- **`_numberToMonth_`** Convert a month number to its name
- **`_parseDate_`** Takes a string as input and attempts to find a date within it to parse into component parts (day, month, year)
- **`_readableUnixTimestamp_`** Format unix timestamp to human readable format
- **`_toSeconds_`** Converts HH:MM:SS to seconds
## debug.bash
Functions to aid in debugging BASH scripts
- **`_pauseScript_`** Pause a script at any point and continue after user input
- **`_printAnsi_`** Helps debug ansi escape sequence in text by displaying the escape codes
- **`_printArray_`** Prints the content of array as key value pairs for easier debugging
## files.bash
Common utilities for working with files.
Functions for working with files.
- `_listFiles_` Find files in a directory. Use either glob or regex.
- `_backupFile_` Creates a backup of a specified file with .bak extension or optionally to a specified directory.
- `_parseFilename_` Break a filename into its component parts which and place them into prefixed variables for use in your script (dir, basename, extension, path, etc.)
- `_decryptFile_` Decrypts a file with `openssl`
- `_encryptFile_` Encrypts a file with `openssl`
- `_extract_` Extract a compressed file
- `_json2yaml_` Convert JSON to YAML uses python
- `_makeSymlink_` Creates a symlink and backs up a file which may be overwritten by the new symlink. If the exact same symlink already exists, nothing is done.
- `_parseYAML_` Convert a YAML file into BASH variables for use in a shell script
- `_readFile_` Prints each line of a file
- `_sourceFile_` Source a file into a script
- `_uniqueFileName_` Ensure a file to be created has a unique filename to avoid overwriting other files
- `_yaml2json_` Convert a YAML file to JSON with python
- **`_backupFile_`** Creates a backup of a specified file with .bak extension or optionally to a specified directory.
- **`_decryptFile_`** Decrypts a file with `openssl`
- **`_encryptFile_`** Encrypts a file with `openssl`
- **`_extractArchive_`** Extract a compressed file
- **`_fileAbsPath_`** Finds the absolute path to a relative file or directory
- **`_fileBasename_`** Gets the basename of a file from a file name
- **`_fileContains_`** Tests whether a file contains a given pattern
- **`_fileDirectory_`** Finds the directory name from a file path
- **`_fileExtension_`** Gets the extension of a file
- **`_fileName_`** Prints a filename from a path
- **`_json2yaml_`** Convert JSON to YAML uses python
- **`_listFiles_`** Find files in a directory. Use either glob or regex.
- **`_makeSymlink_`** Creates a symlink and backs up a file which may be overwritten by the new symlink. If the exact same symlink already exists, nothing is done.
- **`_parseFilename_`** Break a filename into its component parts which and place them into global variables for use in your script (dir, basename, extension, path, etc.)
- **`_parseYAML_`** Convert a YAML file into BASH variables for use in a shell script
- **`_readFile_`** Prints each line of a file
- **`_sourceFile_`** Source a file into a script
- **`_createUniqueFilename_`** Ensure a file to be created has a unique filename to avoid overwriting other files
- **`_yaml2json_`** Convert a YAML file to JSON with python
## macOS.bash
Functions useful when writing scripts to be run on macOS
- `_haveScriptableFinder_` Determine whether we can script the Finder or not
- `_guiInput_` Ask for user input using a Mac dialog box
- **`_guiInput_`** Ask for user input using a Mac dialog box
- **`_haveScriptableFinder_`** Determine whether we can script the Finder or not
- **`_useGNUUtils_`** Add GNU utilities to PATH to allow consistent use of sed/grep/tar/etc. on MacOS
## numbers.bash
## misc.bash
Helpers to work with numbers
Miscellaneous functions
- `_fromSeconds_` Convert seconds to HH:MM:SS
- `_toSeconds_` Converts HH:MM:SS to seconds
- `_countdown_` Sleep for a specified amount of time
- **`_acquireScriptLock_`** Acquire script lock to prevent running the same script a second time before the first instance exits
- **`_detectLinuxDistro_`** Detects the host computer's distribution of Linux
- **`_detectMacOSVersion_`** Detects the host computer's version of macOS
- **`_detectOS_`** Detect the the host computer's operating system
- **`_execute_`** Executes commands with safety and logging options. Respects `DRYRUN` and `VERBOSE` flags.
- **`_findBaseDir_`** Locates the real directory of the script being run. Similar to GNU readlink -n
- **`_generateUUID_`** Generates a unique UUID
- **`_makeProgressBar_`** Prints a progress bar within a for/while loop
- **`_runAsRoot_`** Run the requested command as root (via sudo if requested)
- **`_seekConfirmation_`** Seek user input for yes/no question
- **`_trapCleanup_`** Cleans up after a trapped error.
## services.bash
Functions to work with external services
- `_haveInternet_` Tests to see if there is an active Internet connection
- `_httpStatus_` Report the HTTP status of a specified URL
- `_pushover_` Sends a notification via Pushover (Requires API keys)
- **`_haveInternet_`** Tests to see if there is an active Internet connection
- **`_httpStatus_`** Report the HTTP status of a specified URL
- **`_pushover_`** Sends a notification via Pushover (Requires API keys)
## testProcessing.bash
## strings.bash
Work with strings in your script
Functions for string manipulation
- `_cleanString_` Cleans a string of text
- `_stopWords_` Removes common stopwords from a string. Requires a sed stopwords file. Customize to your needs.
- `_escape_` Escapes a string by adding `\` before special chars
- `_htmlDecode_` Decode HTML characters with sed. (Requires sed file)
- `_htmlEncode_` Encode HTML characters with sed (Requires sed file)
- `_lower_` Convert a string to lowercase
- `_upper_` Convert a string to uppercase
- `_ltrim_` Removes all leading whitespace (from the left)
- `_regex_` Use regex to validate and parse strings
- `_rtrim_` Removes all leading whitespace (from the right)
- `_trim_` Removes all leading/trailing whitespace
- `_urlEncode_` URL encode a string
- `_urlDecode_` Decode a URL encoded string
- **`_cleanString_`** Cleans a string of text
- **`_decodeHTML_`** Decode HTML characters with sed. (Requires sed file)
- **`_decodeURL_`** Decode a URL encoded string
- **`_encodeHTML_`** Encode HTML characters with sed (Requires sed file)
- **`_encodeURL_`** URL encode a string
- **`_escapeString_`** Escapes a string by adding `\` before special chars
- **`_lower_`** Convert a string to lowercase
- **`_ltrim_`** Removes all leading whitespace (from the left)
- **`_regexCapture_`** Use regex to validate and parse strings
- **`_rtrim_`** Removes all leading whitespace (from the right)
- **`_splitString_`** Split a string based on a given delimeter
- **`_stringContains_`** Tests whether a string matches a substring
- **`_stringRegex_`** Tests whether a string matches a regex pattern
- **`_stripANSI_`** Strips ANSI escape sequences from text
- **`_stripStopwords_`** Removes common stopwords from a string using a list of sed replacements located in an external file.
- **`_trim_`** Removes all leading/trailing whitespace
- **`_upper_`** Convert a string to uppercase
## A Note on Code Reuse
## template_utils.bash
Functions required to allow the script template and alert functions to be used
- **`_makeTempDir_`** Creates a temp directory to house temporary files
- **`_safeExit_`** Cleans up temporary files before exiting a script
- **`_setPATH_`** Add directories to $PATH so script can find executables
# Coding conventions
Where possible, I follow [defensive BASH programming](https://kfirlavi.herokuapp.com/blog/2012/11/14/defensive-bash-programming/) principles.
- Function names use camel case surrounded by underscores: `_nameOfFunction_`
- Local variable names use camel case with a starting underscore: `_localVariable`
- Global variables are in ALL_CAPS with underscores seperating words
- Exceptions to the variable an function naming rules are made for alerting functions and colors to ease my speed of programming. (Breaking years of habits is hard...) I.e. `notice "Some log item: ${blue}blue text${reset}` Where `notice` is a function and `$blue` and `$reset` are global variables but are lowercase.
- Variables are always surrounded by quotes and brackets `"${1}"` (It's verbose, but a safe practice)
- Formatting is provided by [shfmt](https://github.com/mvdan/sh) using 4 spaces for indentation
## A Note on Code Reuse and Prior Art
I compiled these scripting utilities over many years without having an intention to make them public. As a novice programmer, I have Googled, GitHubbed, and StackExchanged a path to solve my own scripting needs. I often lift a function whole-cloth from a GitHub repo don't keep track of its original location. I have done my best within these files to recreate my footsteps and give credit to the original creators of the code when possible. Unfortunately, I fear that I missed as many as I found. My goal in making this repository public is not to take credit for the code written by others. If you recognize something that I didn't credit, please let me know.