bash

bash script standard file header

#!/usr/bin/env bash
set -o errexit
set -o nounset
set -o pipefail

bash cheat sheet


## Directory exists
[[ -d ${file} ]]

## File exists
[[ -e ${file} ]]

Delete all directories that are empty

find . -empty -type d -delete

Filter only email addresses from file

grep -EiEio '\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b' <filename>

Networking tools

  • ipcalc: calculcate broadcast, network, etc range for a given IP address and netmask
  • prips: print the IP addresses in a given range

Docker


docker build . -t djg:1.0                             # Build an image with the anem and tag djg:1.0
docker run -p 8080:5000 -d djg:1.0                    # Start container detached with port 8000 forwarded to port 5000

docker container ls                                   # List running containers

docker exec -ti my_container sh -c "echo a && echo b" # Run more processes in a container
docker stop my_container                              # Stop container

docker run -it bash                                   # Run bash container interactively and connect to terminal

docker start -ai my_container                         # Rerun a container

docker inspect my_container                           # Inspect my_container
docker volume inspect volume_id                       # Inspect volume_id

docker images                                         # List docker images

docker ps -a -s                                       # Show information about running containers

docker system prune -af --volumes                     # Clean up everything and don't ask
docker rm $(docker ps --filter status=exited -q)      # Remove all stopped containers

docker run -it --rm --platform linux/amd64 alpine     # Run x86-64 alpine image

Elasticsearch

# list indexes
GET /_cat/indices?v

# list disk allocation
GET /_cat/allocation?v

# delete an index
DELETE my-old-index

JSON tooling

A great overview of jq: https://earthly.dev/blog/jq-select/

Download sample data: ```wget -O rki-covid.json ‘https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_COVID19/FeatureServer/0/query?where=1%3D1&outFields=*&outSR=4326&f=json’````

Format document:

jsonlint rki-covid.json
jq '.' rki-covid.json

Query data:

# Extract all fields
jq '.["fields"]' rki-covid.json

# Extract first field
jq '.["fields"][0]' rki-covid.json

# Remove all whitespace
jq -c '.' rki-covid.json

Make JSON greppable

gron rki-covid.json

Images

# Make all jpeg files in a directrory smaller
mogrify -strip -quality 75% *.jpg

bash


# Set VAR to DEFAULT if no command line argument exists
VAR="${1:-"DEFAULT"}"

# Functions and local variables
hello_world(){
  local name="$1"
  echo "Hello world: ${name}"
}

# if then
if [[ "$NAME" = "Dirk" || "$USER_NAME" = "Jose" ]]; then
  ...
fi

mapfile

Go through a file in chunks

NUMBER_OF_LINES=25
while mapfile -t -n $NUMBER_OF_LINES ary && ((${#ary[@]})) ; do
  echo "${ary[@]}"
  # ... do something useful here
done < input_file