Skip to content

Instantly share code, notes, and snippets.

@Winterhuman
Last active January 12, 2026 21:37
Show Gist options
  • Select an option

  • Save Winterhuman/21d7b148db40ff041f397b07a7aafb83 to your computer and use it in GitHub Desktop.

Select an option

Save Winterhuman/21d7b148db40ff041f397b07a7aafb83 to your computer and use it in GitHub Desktop.
A script for finding the smallest lossless encoding of a PNG or GIF input image (that I know of).
#!/usr/bin/env -S unshare --pid --mount-proc --kill-child --map-root-user /bin/sh
# shellcheck shell=sh
# Licensed under the Zero-Clause BSD terms: https://opensource.org/license/0bsd
## Requires: find, awk, oxipng, and gifsicle
## Optional: pngquant, cwebp & gif2webp, gif2apng, and (perl-image-)exiftool
## -C: Fail if redirects try to overwrite an existing file.
## -e: Fail if any command fails (with exceptions).
## -u: Fail if an unset variable tries to be expanded.
## -f: No glob expansion.
set -Ceuf
perr() { printf "\a\033[1;31m‽\033[0;39m %s\033[0;39m\n" "${1-"$(cat)"}" >&2; }
clear_prog() { printf "\033]9;4;0\007" >&2; }
exit_clear() { clear_prog; exit 1; }
osc777() { printf "\a\033]777;notify;Sisyphus;%b\033\\" "$1" >&2; }
nexit() { osc777 "${1-"$(cat)"}"; exit_clear; }
pquit() { arg="${1-"$(cat)"}"; perr "$arg"; nexit "$arg"; }
pstat() { printf "\033[1;34m%20b\033[0;39m%s\033[0;39m\n" "$1" "${2-"$(cat)"}"; }
## `kill` handles background jobs, but `exit` is required for normal processes.
### The second trap handles unexpected signals, where a notification IS desired.
#### Trying to trap SIGTERM leads to a "Segmentation fault" error
trap 'kill "$$"; exit_clear' INT
trap 'kill "$$"; nexit "SIGQUIT or SIGABRT received! Was operating on: $1"' QUIT ABRT
help() {
cat <<"HELP"; exit 0
Usage: sisyphus [OPTION]... SRC [DEST]
Losslessly optimise PNGs & GIFs by all known means.
If DEST is omitted, DEST is set to '{SRC wo/ext}.new.EXT'. Possible output
formats include: PNG, GIF, WebP, and APNG.
Options:
-a, --all-oxi <bool> Optionally accepts a boolean value. Controls whether to
always or never try all OxiPNG variations. When this
option is omitted, a heuristic determines whether any
OxiPNG variations are attempted.
-b, --brute-lines <int> Accepts an integer between 1 and the SRC image's height.
Sets the maximum '--brute-lines' value for the OxiPNG
variations. The default is '16'.
-d, --dry-run Perform all operations except for writing the output.
-f, --force Allow existing destination files to be overwritten.
-m, --max-procs <int> Accepts an integer greater than 0. Limits the number of
simultaneous processes for 'xargs'. The default is '8'.
-n, --no-webp Do not attempt to use CWebP nor GIF2WebP.
-N, --no-apng Do not attempt to use GIF2APNG.
-q, --quiet Do not print to STDOUT or send notifications. This option
implies '--results 0', but this can be overridden after.
-r, --results <arg> Accepts a boolean value or an integer greater than 0.
Controls how the results list is printed, if at all.
If <arg> is an integer, only the top <arg> trials will
be printed. The results are printed to STDERR.
-s, --size <int> Accepts an integer in bytes. Sets the minimum size target.
If the best encoding is greater or equal to the minimum
size, then DEST is not created.
-S, --stdout Write the best output to STDOUT. This option semi-implies
'--quiet' except it does NOT also imply '--results 0'.
-h, --help Display this help message, and then exit.
Booleans:
-a, --all-oxi Disable: '0', 'no', or 'false'.
Enable: '1', 'yes', or 'true'.
-r, --results Hide: 'no', 'false', 'hide', or 'none'.
Show: 'yes', 'true', 'show', or 'all'.
Warning:
HDR SRC images will NOT be losslessly optimised!
HELP
}
# Setup a private TMPFS for this script to use, which is what 'unshare' is for.
## 'nr_inodes' must be >= to 'max number of files + 1'. This is adjusted later
work="/tmp"
mount --types tmpfs --options nosuid,nodev,noexec,size=50%,nr_inodes=16 \
sisyphus "$work"/ || pquit "Failed to overmount '$work/'!"
# Argument parsing
ALL_OXI="" MAX_BLIN="16" DRY="" FORCE="" MAX_PROCS="8"
NO_WEBP="" NO_APNG="" STDOUT="" RESULTS="0" SIZE=""
INTERNAL="" ARG_COUNT="0" SKIP=""
while [ "$ARG_COUNT" -lt "$#" ]; do
if [ -z "$SKIP" ]; then
case "$1" in
-a|--all-oxi)
case "${2:-}" in
0|no|false) ALL_OXI="0"; shift ;;
1|yes|true) ALL_OXI="1"; shift ;;
*) ALL_OXI="1" ;;
esac
;;
-b|--brute-lines)
case "${2:-}" in
[1-9]*) MAX_BLIN="$2"; shift ;;
*) pquit "No positive integer greater than zero was given for '-b|--brute-lines'!" ;;
esac
;;
-d|--dry-run) DRY="1" ;;
-f|--force) FORCE="1" ;;
-m|--max-procs)
case "${2:-}" in
[1-9]*) MAX_PROCS="$2"; shift ;;
*) pquit "No positive integer greater than zero was given for '-m|--max-procs'!" ;;
esac
;;
-n|--no-webp) NO_WEBP="1" ;;
-N|--no-apng) NO_APNG="1" ;;
-q|--quiet|-S|--stdout)
## Only allow writing to STDOUT on FD 8.
### Don't do `exec 8>&1` twice or else FD 8 will
### point to `/dev/null` too
if [ "$(readlink /proc/self/fd/1)" != "/dev/null" ]; then
exec 8>&1
exec 1>/dev/null
fi
## '--results 0' should only be set when
## '--quiet' is used; '--stdout' doesn't do this
case "$1" in
-S|--stdout) STDOUT="1" ;;
-q|--quiet) RESULTS="0" ;;
esac
## Functions have to be redefined to be updated.
### This `perr()` omits `\a`
perr() { printf "\033[1;31m‽\033[0;39m %s\033[0;39m\n" "${1-"$(cat)"}" >&2; }
osc777() { :; }
pquit() { perr "$1"; exit_clear; }
;;
-r|--results)
case "${2:-}" in
no|false|hide|none) shift ;;
yes|true|show|all) RESULTS="ALL"; shift ;;
[0-9]*) RESULTS="$2"; shift ;;
*) pquit "An unknown value was given for '-r|--results'!" ;;
esac
;;
-s|--size)
case "${2:-}" in
[0-9]*) SIZE="$2"; shift ;;
*) pquit "No positive integer was given for '-s|--size'!" ;;
esac
;;
-h|--help) help ;;
--_internal)
## DO NOT USE MANUALLY. For self-execution
INTERNAL="1"
;;
--) SKIP="1" ;;
-*) pquit "'$1' is not a known option!" ;;
*) set -- "$@" "$1"; ARG_COUNT="$(( ARG_COUNT + 1 ))" ;;
esac
else set -- "$@" "$1"; ARG_COUNT="$(( ARG_COUNT + 1 ))"
fi
shift
done
# Validate arguments
## `[ cond ] && cmd` will carry over non-zero exit codes; always use `||`
[ "$#" -le 2 ] || pquit "Too many arguments given!"
## Don't try WebP if `cwebp` & `gif2webp` aren't available
command -v cwebp >/dev/null || NO_WEBP="1"
command -v gif2webp >/dev/null || NO_WEBP="1"
## Don't try APNG if `gif2apng` isn't available
command -v gif2apng >/dev/null || NO_APNG="1"
## Don't try `pngquant` if it's not available
NO_QUANT=""
command -v pngquant >/dev/null || NO_QUANT="1"
## Don't try anything if `find` or `awk` are unavailable
command -v find >/dev/null || pquit "The 'find' command is required!"
command -v awk >/dev/null || pquit "The 'awk' command is required!"
## SRC
src_real="${1:?$(pquit "No source file was specified!")}"
[ -s "$src_real" ] || pquit "'$src_real' is not a non-empty file!"
### Assign the SRC a file descriptor
exec 3<"$src_real"
src="/proc/self/fd/3"
src_real="$(realpath -- "$src_real")"
### Size detection
find_size() {
# Handle non-existent files by giving them arbitrarily huge sizes
if [ -f "$1" ]; then
wc --bytes <"$1"
else
printf "4294967296"
fi
}
src_size="$(find_size "$src")"
### Minimum size target validation
check_number() {
[ "$1" != "" ] || return 0
num="$1" oldnum="$num" option="$2"
## Strip leading zeros and avoid integer overflow
num="$(printf "%s" "$num" | sed "s/^0*//")"
## If `sed` trimmed '0' to nothing, undo that
num="${num:-"0"}"
if ! printf "%d" "$num" >/dev/null 2>&1; then
perr "The value given for '$option' ($oldnum) is not an integer, or is far beyond the integer limit!"
return 1
fi
if [ "$num" -ne "$(printf "%.12s" "$num")" ]; then
perr "The value given for '$option' ($oldnum) is over the integer limit!" \
return 1
fi
printf "%d" "$num"
}
SIZE="$(check_number "$SIZE" "-s|--size")"
### Max processes validation
MAX_PROCS="$(check_number "$MAX_PROCS" "-m|--max-procs")"
### Max `--brute-lines` validation
MAX_BLIN="$(check_number "$MAX_BLIN" "-b|--brute-lines")"
## Results validation
[ "$RESULTS" = "ALL" ] || RESULTS="$(check_number "$RESULTS" "-r|--results")"
### Mimetype detection
mime="$(file --dereference --brief --mime-type -- "$src")"
mode="PNG"
safe="all"
case "$mime" in
"image/png") : ;;
"image/gif") mode="GIF" safe="safe" ;;
*) pquit "The mimetype of '$src_real' is neither PNG nor GIF!" ;;
esac
#### When '$INTERNAL' is set, the input is APNG and must be safely stripped
[ -z "$INTERNAL" ] || safe="safe"
src_height="$(file --dereference --brief -- "$src" |
sed -n "s/.*x \([[:digit:]]\+\).*/\1/p")"
src_height="$(check_number "$src_height" "\$src_height")"
## DEST
dest_exists() {
if [ -n "$FORCE" ] || [ -n "$STDOUT" ]; then return 0; fi
## Allow globbing for just this for-loop
set +f
for similar in "$1"*; do
[ -e "$similar" ] || continue
if [ "${similar%.*}" = "$1" ]; then
perr "'$similar' shares a filename with '$1'!"
return 1
fi
done
set -f
}
dest_path="${2:-}"
if [ -n "$STDOUT" ]; then
[ -z "$dest_path" ] ||
pquit "A destination, '$dest_path', and '--stdout' were both specified!"
else
dest_path="${dest_path%.*}"
dest_path="${dest_path:-${src_real%.*}.new}"
dest_path="$(realpath -- "$dest_path")"
dest_exists "$dest_path" ||
nexit "'$similar' shares a filename with '$dest_path'!"
fi
# Print known information
printf "%s \033[2m(%s)" "$src_real" "$mime" | pstat "Input ← "
printf "%s\033[2m.???" "$dest_path" | pstat "Output template ⇨ "
[ -z "$SIZE" ] || pstat "Size target ↘ " "$SIZE bytes"
# Optimisation
create_png_bases() {
cwebp_base="$work/cwebp.webp"
if [ "$NO_WEBP" != "1" ]; then
## `-z 9` implies `-q 100 -m 6 -lossless`;
## `-z 9 -q 100` disables lossless mode
cwebp -quiet -mt -z 9 -alpha_filter best -o "$cwebp_base" \
-- "$src" ||
perr "Passing '$src_real' through 'cwebp' failed!"
fi &
## Values above the SRC height aren't useful for `--brute-lines`.
## For the baseline, limit it to a sensible value: 8.
baseline_max_blin="$((
src_height > 8
? 8
: src_height
))"
## If `--brute-lines` was set below 8, respect that too
baseline_max_blin="$((
baseline_max_blin > MAX_BLIN
? MAX_BLIN
: baseline_max_blin
))"
oxipng_base="$work/oxipng + zc12 + f9 + level=5 + lines=$baseline_max_blin.png"
{
oxipng --opt max --filters 9 --brute-level=5 \
--brute-lines="$baseline_max_blin" --strip "$safe" \
--alpha --out "$oxipng_base" -- "$src" \
>/dev/null 2>&1 ||
perr "Creating '$oxipng_base' failed!"
} &
if [ "$safe" != "safe" ] && [ -z "$NO_QUANT" ]; then
quant_src="$work/quant.png"
## '$quant_src' won't be created if it can't be quantised
## losslessly (`--quality 100-100`).
### In that case, it'll exit with code 99
pngquant --quality 100-100 --speed 1 --strip \
--output "$quant_src" -- "$src" ||:
## If '$src' was already quantised beforehand, '$quant_src' will
## just be a duplicate that should be removed.
### `unset` works after `||/&&`, unlike `quant_src=""`
[ -f "$quant_src" ] || unset quant_src
if cmp -- "$src" "${quant_src:-}" >/dev/null 2>&1; then
rm -- "$quant_src" ||
perr "Failed to remove '$quant_src'!"
unset quant_src
fi
fi
wait
oxipng_base_size="$(find_size "$oxipng_base")"
cwebp_base_size="$(find_size "$cwebp_base")"
smallest_heuristic="$((
oxipng_base_size < cwebp_base_size
? oxipng_base_size
: cwebp_base_size
))"
## '$smallest_known' gets read by `try_oxi_vars`
smallest_known="$((
smallest_heuristic < src_size
? smallest_heuristic
: src_size
))"
## If '$SIZE' was given, factor it into the smallest-known size value
[ -z "$SIZE" ] ||
smallest_known="$((
smallest_heuristic < SIZE
? smallest_heuristic
: SIZE
))"
}
gen_list() {
# Create the command list `xargs` will parse
# For GIF mode
if [ "$mode" = "GIF" ]; then
gif2webp_dest="$work/gif2webp.webp"
gif2apng_prefix="$work/gif2apng + z"
gifsicle_prefix="$work/gifsicle + o"
nproc="$(nproc)"
# Skip `gif2webp` and or `gif2apng` if requested.
[ -n "$NO_WEBP" ] || cat <<-GIF2WEBP
$src $gif2webp_dest gif2webp -quiet -mt -min_size -m 6 -q 100 -metadata none
GIF2WEBP
[ -n "$NO_APNG" ] || cat <<-GIF2APNG
$src ${gif2apng_prefix}0.apng gif2apng -i20 -z0
$src ${gif2apng_prefix}1.apng gif2apng -i20 -z1
$src ${gif2apng_prefix}2.apng gif2apng -i20 -z2
$src ${gif2apng_prefix}0 + kp.apng gif2apng -i20 -z0 -kp
$src ${gif2apng_prefix}1 + kp.apng gif2apng -i20 -z1 -kp
$src ${gif2apng_prefix}2 + kp.apng gif2apng -i20 -z2 -kp
GIF2APNG
cat <<-GIFSICLE
$src ${gifsicle_prefix}3.gif gifsicle --optimize=3 --optimize=keep-empty --threads="$nproc"
$src ${gifsicle_prefix}2.gif gifsicle --optimize=2 --optimize=keep-empty --threads="$nproc"
$src ${gifsicle_prefix}1.gif gifsicle --optimize=1 --optimize=keep-empty --threads="$nproc"
GIFSICLE
return 0
fi
# For PNG mode
pre="$work/oxipng"
osrc="$src"
## Duplicate the list for '$quant_src' (if set). This is `break`ed later
while :; do
## `--brute-{level,lines}` only applies to the brute filter (`-f 9`).
zc="1"
while [ "$zc" -le 12 ]; do
pzc="$(printf "%-2s" "$zc")"
cat <<-ZC
$osrc $pre + zc$pzc + f0-8.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8
$osrc $pre + zc$pzc + f0-8 + nb.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb
$osrc $pre + zc$pzc + f0-8 + nc.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nc
$osrc $pre + zc$pzc + f0-8 + nb + nc.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --nc
$osrc $pre + zc$pzc + f0-8 + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --ng
$osrc $pre + zc$pzc + f0-8 + nb + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --ng
$osrc $pre + zc$pzc + f0-8 + nc + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nc --ng
$osrc $pre + zc$pzc + f0-8 + nb + nc + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --nc --ng
$osrc $pre + zc$pzc + f0-8 + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --np
$osrc $pre + zc$pzc + f0-8 + nb + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --np
$osrc $pre + zc$pzc + f0-8 + nc + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nc --np
$osrc $pre + zc$pzc + f0-8 + nb + nc + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --nc --np
$osrc $pre + zc$pzc + f0-8 + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --ng --np
$osrc $pre + zc$pzc + f0-8 + nb + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --ng --np
$osrc $pre + zc$pzc + f0-8 + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nc --ng --np
$osrc $pre + zc$pzc + f0-8 + nb + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=0-8 --nb --nc --ng --np
ZC
zc="$(( zc + 1 ))"
done
cat <<-ZOP
$osrc $pre + zop + f0-8.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8
$osrc $pre + zop + f0-8 + nb.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb
$osrc $pre + zop + f0-8 + nc.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nc
$osrc $pre + zop + f0-8 + nb + nc.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --nc
$osrc $pre + zop + f0-8 + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --ng
$osrc $pre + zop + f0-8 + nb + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --ng
$osrc $pre + zop + f0-8 + nc + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nc --ng
$osrc $pre + zop + f0-8 + nb + nc + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --nc --ng
$osrc $pre + zop + f0-8 + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --np
$osrc $pre + zop + f0-8 + nb + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --np
$osrc $pre + zop + f0-8 + nc + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nc --np
$osrc $pre + zop + f0-8 + nb + nc + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --nc --np
$osrc $pre + zop + f0-8 + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --ng --np
$osrc $pre + zop + f0-8 + nb + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --ng --np
$osrc $pre + zop + f0-8 + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nc --ng --np
$osrc $pre + zop + f0-8 + nb + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=0-8 --nb --nc --ng --np
ZOP
## `--brute-lines` only makes sense up to the height of the SRC image
blin_max="$((
src_height > MAX_BLIN
? MAX_BLIN
: src_height
))"
## If '$src_height' is 1, ensure at least 1 line is tried.
blin="2"
if [ "$blin_max" -lt "$blin" ]; then
blin="1"
blin_max="$blin"
fi
while [ "$blin" -le "$blin_max" ]; do
blev="1"
while [ "$blev" -le 12 ]; do
## Ensure numbers like '1' and '12' are the same length
pblin="$(printf "%-2s" "$blin")"
pblev="$(printf "%-2s" "$blev")"
zc="1"
while [ "$zc" -le 12 ]; do
pzc="$(printf "%-2s" "$zc")"
## `${pblin% *}` prevents "... lines=N .png"
cat <<-ZCF9
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=${pblin% *}.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=${pblin% *}
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nc.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + nc.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --ng
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --ng
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nc + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc --ng
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + nc + ng.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc --ng
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nc + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + nc + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --ng --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --ng --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc --ng --np
$osrc $pre + zc$pzc + f9 + level=$pblev + lines=$pblin + nb + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zc=$pzc --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc --ng --np
ZCF9
zc="$(( zc + 1 ))"
done
## Don't duplicate the `--zopfli` lines per `--zc` level
cat <<-ZOPF9
$osrc $pre + zop + f9 + level=$pblev + lines=${pblin% *}.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=${pblin% *}
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nc.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + nc.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --ng
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --ng
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nc + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc --ng
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + nc + ng.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc --ng
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nc + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + nc + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --ng --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --ng --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nc --ng --np
$osrc $pre + zop + f9 + level=$pblev + lines=$pblin + nb + nc + ng + np.png oxipng --opt max --alpha --strip $safe --zopfli --zi=1024 --ziwi=16 --filters=9 --brute-level=$pblev --brute-lines=$pblin --nb --nc --ng --np
ZOPF9
blev="$(( blev + 1 ))"
done
blin="$(( blin + 1 ))"
done
## If '$quant_src' is set, generate the list for each of '$src' and
## '$quant_src' sequentially
if [ "$osrc" = "${quant_src:-}" ] || [ -z "${quant_src:-}" ]; then
break; fi
osrc="$quant_src" pre="$work/quant > oxipng"
done ## This `done` is for the `while :;` statement
}
gen_script() {
cat <<-"SCRIPT"
## `noexec` on `$work` prevents having `#!/bin/sh` here instead
set -Ceuf
## Redirect this STDERR to FD 9. This is reverted outside `xargs`
exec 2>&9
perr() {
arg="${1-"$(cat)"}"
printf "\a\033[1;31m‽\033[0;39m %s\033[0;39m\n" "$arg" >&2
printf "\033]777;notify;Sisyphus;%b\033\\" "$arg"
}
## `exit 255` prevents `xargs` from processing any more line batches (the
## batches from `--max-procs`)
pquit() { perr "${1-"$(cat)"}"; exit 255; }
smallest_known="$1" total_lines="$2" oxipng_base="$3"
## Parse the line
IFS=" " read -r index input output cmd <<-LINE ||:
$4
LINE
## Check if the output (e.g. '$oxipng_base') already exists
[ ! -f "${output:-}" ] || exit 0
## Update the progress bar now. Otherwise, long-running '$cmd's will cause the
## bar to jump backwards on their completion.
### Without `>&2`, and when `--stdout` is used, the updates get redirected too
printf "\033]9;4;1;%d\007" "$(( ( index * 100 ) / total_lines ))" >&2
## Handle each command differently where needed
case "$cmd" in
quant*|oxipng*)
env --split-string "$cmd" --out "$output" -- "$input" \
2>&1 || pquit "'$cmd --out $output -- $input' failed!"
;;
gif2webp*|gifsicle*)
env --split-string "$cmd" -o "$output" -- "$input" \
2>&1 || pquit "'$cmd -o $output -- $input' failed!"
;;
gif2apng*)
## `gif2apng` only accepts relative paths for its arguments:
## https://sourceforge.net/p/gif2apng/discussion/1022150/thread/8ec5e7e288
input_rel="$(realpath --relative-to="$PWD" -- "$input")"
output_rel="$(realpath --relative-to="$PWD" -- "$output")"
env --split-string "$cmd" -- "$input_rel" "$output_rel" \
2>&1 || pquit "'$cmd -- $input_rel $output_rel' failed!"
exiftool -overwrite_original_in_place -all= -- "$output" 2>&1 ||:
;;
*) pquit "Unknown command: $cmd" ;;
esac >/dev/null
## If the output doesn't exist, don't continue
[ -f "$output" ] || exit 0
## Also, don't continue if '$oxipng_base' is unset
[ -n "${oxipng_base:-}" ] || exit 0
discard() {
trial="$(realpath -- "$1")"
baseline="$(realpath -- "$2")"
[ -s "$trial" ] || return 0
[ -s "$baseline" ] ||
perr "'$baseline' does not exist, or has no content, but should!"
if cmp "$1" "$2" >/dev/null; then
## This handles otherwise truncated trials which are
## ordered before '$baseline' in the final list
ln --force --symbolic "$baseline" "$trial" \
2>/dev/null ||
perr "Couldn't replace '$trial' with a symlink to '$baseline'!"
else
trial_size="$(wc --bytes <"$trial")"
[ "$smallest_known" -gt "$trial_size" ] ||
fallocate --punch-hole --length="$trial_size" \
-- "$trial"
fi
}
## If '$quant_src' was the input, the output should be compared to that instead
[ "$output" = "${output#quant > }" ] || oxipng_base="$input"
discard "$output" "$oxipng_base"
SCRIPT
}
optimise() {
if [ "$mode" = "PNG" ]; then
# Create the PNG-mode baselines, and decide on whether
# further variants need to be created
create_png_bases
## If requested (`-a 0`), skip the OxiPNG variants
[ "$ALL_OXI" != "0" ] || return 0
## Skip the OxiPNG variants if:
## 1. It's not been explicitly requested to try them.
## 2. And, if the baseline OxiPNG size is greater than
## 256 bytes.
## 3. And, if the input image is greater in size than
## the smallest known encoding.
if [ "$ALL_OXI" != "1" ] &&
[ "$oxipng_base_size" -gt 256 ] &&
[ "$src_size" -gt "$smallest_heuristic" ]; then
return 0
fi
# Calculate '$total_lines' for use in progress tracking.
## Levels index from 2, so subtract 1
levels="$((
(src_height > MAX_BLIN ? MAX_BLIN : src_height)
- 1
))"
## Also, ensure '$levels' isn't less than 1
levels="$(( levels > 0 ? levels : 1 ))"
zcs="12"
oxis="16"
lines="12"
## The `+ 1`'s account for the Zopfli variants adding an
## extra '$oxis' for brute and non-brute variants each
total_lines="$((
(zcs + 1) * (lines * levels + 1) * oxis
))"
[ -z "${quant_src:-}" ] ||
total_lines="$(( total_lines * 2 ))"
else
smallest_known="$src_size"
## At minimum, the 3 `gifsicle` trials are attempted
total_lines="3"
[ -n "$NO_WEBP" ] || total_lines="$(( total_lines + 1 ))"
[ -n "$NO_APNG" ] || total_lines="$(( total_lines + 6 ))"
fi
# Remount '$work' with a high enough `nr_inodes` limit
## '16' is a random, safe number to ensure non-`xargs` files can exist
mount --options remount,nosuid,nodev,noexec,size=50%,nr_inodes="$(( total_lines + 16 ))" \
"$work"/ || pquit "Failed to remount '$work/'!"
# Create the script `xargs` will execute
tmp_oxi="$work/oxi"
gen_script >"$tmp_oxi"
# Parse & execute each line in the command list
## `9>&2` redirects the script's STDERR to FD 2, while redirecting the
## STDERR of `xargs` to `/dev/null`. `exit 255` causes the STDERR logs.
### `cat --number` indexes the lines for progress tracking
gen_list | cat --number | xargs \
--max-procs "$MAX_PROCS" \
--delimiter "\n" \
--replace="%" \
-- sh "$tmp_oxi" \
"$smallest_known" \
"$total_lines" \
"${oxipng_base:-}" \
"%" \
9>&2 2>/dev/null ||
exit_clear
clear_prog
## The script isn't useful anymore, and `find` will see it, so remove it
rm "$tmp_oxi" || pquit "Failed to remove '$tmp_oxi'!"
# Optimise the `gif2apng` outputs with this script
## ...unless told otherwise
[ -z "$NO_APNG" ] || return 0
## Allow globbing for just these for-loops
set +f
## De-duplicate the APNGs beforehand
for dup in /tmp/gif2apng*; do
[ -f "$dup" ] || continue
[ ! -L "$dup" ] || continue
for dup2 in /tmp/gif2apng*; do
[ "$dup" != "$dup2" ] || continue
! cmp "$dup" "$dup2" >/dev/null 2>&1 ||
ln --force --symbolic "$dup" "$dup2" 2>/dev/null ||
perr "Failed to symlink '$dup2' to '$dup'!"
done
done
for apngsrc in /tmp/gif2apng*; do
## Don't optimise duplicate SRC images; the output is identical
if [ -L "$apngsrc" ]; then
printf "'%s' and '%s' are identical. Skipping." \
"$apngsrc" "$(readlink -- "$apngsrc")" |
pstat "APNG ⇦ "
continue
fi
[ -f "$apngsrc" ] || continue
apngdest="${apngsrc%.*}.oxipng.apng"
## Each `sisyphus` instance has its own '$work' directory;
## therefore, pass the input as a file descriptor
exec 4<"$apngsrc"
apngsrc_real="$apngsrc"
apngsrc="/proc/self/fd/4"
pstat "APNG ← " "Optimising '$apngsrc_real'…"
## `--stdout` implies `--quiet`
"$0" --_internal --max-procs "$MAX_PROCS" --no-webp \
--all-oxi "${ALL_OXI:-"1"}" --stdout -- "$apngsrc" \
>"$apngdest" ||:
## If '$apngdest' is still empty, substitute it with '$apngsrc'
if [ -f "$apngdest" ] && [ ! -s "$apngdest" ]; then
ln --force --symbolic "$apngsrc_real" "$apngdest" \
2>/dev/null ||
perr "Failed to symlink '$apngdest' to '$apngsrc_real'!"
fi
done
set -f
}
optimise
# Output listing & selection
tmp_smallest="$work/smallest"
## 0. Gather all the results, while ignoring any possible empty files that can
## be created by failed APNG optimisations.
## 1. Prefix lines with their length followed by a space, e.g. '20 8 ...'.
## 2. Sort the lines by their size (2n) and then by their length (1V).
## 3. Remove the first, length field (2-), e.g. '8 ...'.
find "$work"/ -maxdepth 1 -follow -type f -size +0c -printf "%s\t%f\n" |
awk '{ printf "%d\t%s\n", length, $0 }' |
sort --field-separator=" " --key="2n" --key="1V" | {
# | Start of the inside of this pipe |
## Read the first line from STDIN, which will be for the smallest result, while
## preserving the lines following it
IFS=" " read -r _ smallest_size smallest
smallest_read="$smallest_size $smallest"
if [ "$RESULTS" != "0" ]; then
pstat "Size order ↯ " ""
## Make sure `cut` & `sed` only filter the results by using `{}`
{
## Print the smallest result
### `0\t` is a placeholder for the length
printf "0\t%s\n" "$smallest_read"
## Print the remaining results
if [ "$RESULTS" = "ALL" ]; then
cat
else
## The smallest result was already printed, so
## print one fewer lines than specified
head --lines="$(( RESULTS - 1 ))"
fi
} |
cut --delimiter=" " --fields=2- |
sed "s/\t/ bytes\t/g ; s/^/\t/g" >&2
## This will only be printed if '--quiet' is omitted; it redirects FD 1
printf "\n"
fi
## Variables set inside pipelines are isolated from the script, so write the
## smallest result to a file.
### Do the write at the end in case `find` tries to include '$tmp_smallest'.
### After the RESULTS-ALL if-statement, it wouldn't matter if it were found
printf "%s\n" "$smallest_read" >"$tmp_smallest"
# | End of the inside of this pipe |
}
## Read '$tmp_smallest' for the best known result
IFS=" " read -r smallest_size smallest <"$tmp_smallest" ||
pquit "Failed to read the smallest result from '$tmp_smallest'!"
smallest="$work/$smallest"
[ -f "$smallest" ] ||
pquit "'$smallest' is the best result, but it doesn't exist or isn't a file!"
## Check for an improvement
[ "$smallest_size" -le "${SIZE:-"$smallest_size"}" ] ||
printf "'%s' is equal to, or larger than, the minimum size target: %d bytes." \
"$smallest" "$SIZE" | pquit
if [ "$smallest_size" -eq "$src_size" ]; then
pstat "\033[32mAlready optimal ✓ " "The input is as small as the best result!"
## '--quiet' redefines `osc777()` to be `true`; this is already handled
osc777 "'$src_real' is already optimal!"
exit 0
fi
[ "$smallest_size" -le "$src_size" ] ||
pquit "'$smallest' is larger than '$src_real'."
## Symlinks also have an apparent size of '0'; exclude them from this check
if [ "$(stat --format "%b" -- "$smallest")" -le 0 ] && [ ! -L "$smallest" ]; then
pquit "'$smallest' was truncated at some point; it cannot be used!"; fi
## Output the best result (unless '--dry-run' was specified)
if [ -n "$STDOUT" ]; then
## '--stdout' implies '--quiet'; therefore:
### FD 8 prints to STDOUT
[ -n "$DRY" ] || cat "$smallest" >&8
### And, the proceeding steps are irrelevant
exit 0
fi
final_dest="$dest_path.${smallest##*.}"
if [ -e "$final_dest" ] && [ -z "$FORCE" ]; then
pquit "'$final_dest' already exists!"; fi
[ -n "$DRY" ] || cp -- "$smallest" "$final_dest"
## Print the final statistics.
### "> ": as in the "quant > ..." prefix of the filename
method="${smallest##*"> "}"
method="${smallest##*"/"}"
method="${method%.*}"
printf "%s \033[2m(%s)" "$final_dest" "$method" | pstat "Final output → "
printf "%d bytes \033[2m↘\033[0;39m %d bytes" "$src_size" "$smallest_size" |
pstat "Size diff ↘ "
osc777 "'$final_dest' is finished! Size diff: $src_size bytes -> $smallest_size bytes ($method)"
@Winterhuman
Copy link
Author

I'm not able to test it since it's an EXE, but, since I think it's still using OptiPNG under the hood, I would guess my script is better for now (OptiPNG always faired worse than OxiPNG from my testing). The script currently tries every valid, unordered permutation of:

[pngquant --quality 100-100 --speed 1 --strip |,] oxipng [--zopfli --zi=255,--zc={0..12}] --n[bcgp] --opt max --strip all --alpha

If, when FileOptimizer switches to OxiPNG, it additonally executes oxipng with some other options beyond these (which I'm not aware of currently), or it implements a palette randomiser itself (which is what pngquant functions as here), it could beat this script.

@TPS
Copy link

TPS commented Feb 26, 2025

Per the current source @ https://sourceforge.net/p/nikkhokkho/code/HEAD/tree/trunk/FileOptimizer/Source/cppMain.cpp#l2087, FO currently uses "apngopt, pngquant, PngOptimizer, TruePNG, pngout, oxipng, pngwolf, Leanify, ect, pingo, advpng, deflopt, defluff, deflopt" (not a typo, runs deflopt 2× based on testing) in the PNG toolchain, w/ configurable options via GUI & INI.

FO reportedly works fine under WINE & emulators. Perhaps you could compare that way?

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

Managed to use Bottles to run FO, and... inconclusive? Here's what I'm noticing:

  1. Sisyphus can beat FO, and vice-versa, so neither is guaranteed to win unfortunately.
  2. From that cppMain.cpp file you linked, I'm noticing that oxipng -Z is always being used, however, Zopfli can actually lose to non-Zopfli compression (and also, there's no option permutation like I'm doing here, which can also help).
  3. pingo is able to beat cwebp sometimes, which is unfortunate since pingo is also a Windows-only executable, so this script is definitely missing out in those cases.

I haven't tested GIFs or APNGs yet, and I also need to update this script with my new APNG brute-forcing pipeline, but from the above results, I'm guessing that's going to be a mixed bag as well.

EDIT: I've updated Sisyphus with the new APNG stuff, along with some other changes.

@TPS
Copy link

TPS commented Feb 26, 2025

Thanks for the detailed comparison! If you've test files you'd be willing to make available that Sisyphus beats FO, that'd be much appreciated. 🙇🏾‍♂️

Pinging @javiergutierrezchamorro (the FO dev) to take note of above feedback.

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

Here's some samples:


decipher-sisyphus

Sisyphus: 1824 bytes (zop.png. I threw these first two samples in to show that FO can beat Sisyphus)

decipher-fo

FO: 1823 bytes


dd-sisyphus

Sisyphus: 479 bytes (quant > zop + nb.png)

dd-fo

FO: 474 bytes


thisline-sisyphus

[FO >] Sisyphus: 413 bytes (zop.png. It's probably --opt max that makes the difference with these zop.png results)

thisline-fo

FO: 414 bytes


disc-sisyphus

Sisyphus: 534 bytes (zop.png)

disc-fo

FO: 537 bytes

disc-fo-sisyphus

FO > Sisyphus: 536 bytes (zop.png)


seat-sisyphus

[FO >] Sisyphus: 246 bytes (quant > zop.png. Regular zop.png matched FO, so pngquant did something here)

seat-fo

FO: 247 bytes


space-sisyphus

Sisyphus: 3396 bytes (zop.png)

space-fo

FO: 3398 bytes

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

I forgot to set the optimisation level to 9 in FO... one moment.

EDIT: Done

@TPS
Copy link

TPS commented Feb 26, 2025

Just to make sure I understand, just the last example beats FO (by 1 byte)? 🤔

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

Yeah, I had some other images going before I realised I forgot the level 9 setting; will edit with some of the other examples when I have time.

EDIT: Okay, so bad news. Pretty much every time Sisyphus yields a smaller result, it's because it uses --zopfli and --opt max, which FO doesn't. Except for that quant > zop.png result, where regular zop.png matched FO, so the extra palette randomisation that pngquant introduces can help on rare occasions.

But yeah, ever since I set the level in FO's settings correctly, I've been struggling to beat it; it's usually a tie or loss save for these samples. Welp.

@TPS
Copy link

TPS commented Feb 27, 2025

1 tip: I think you're pretty used to long processing times b/c of your script, but, if you've the spare processing power, multiple instances of FO work quite well on split file lists to speed up overall processing.

@TPS
Copy link

TPS commented Feb 27, 2025

Also, does running Sisyphus before/after FO-processed images improve anything?

@Winterhuman
Copy link
Author

Winterhuman commented Feb 27, 2025

Found these (I'll edit the previous samples if I find anything for them too):

general-fo

FO: 388 bytes

general-fo-sisyphus

FO > Sisyphus: 387 bytes (zop + nb.png. Regular zop.png matches FO, and every permutation of zop + *.png beats it by a single byte, so it's not just --opt max here)


balloon-fo

FO: 2987 bytes (FO > FO: Same size)

balloon-fo-sisyphus

FO > Sisyphus: 2956 bytes (zop.png. Probably --opt max again)

balloon-fo-sisyphus-fo

FO > Sisyphus > FO: 2955 bytes (???)

@TPS
Copy link

TPS commented Feb 27, 2025

Sometimes running FO multiple times on the same file (Shift+F5,▶️) back-to-back gives better results, too, so that's kind-of expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment