bash script standard file header
#!/usr/bin/env bash
set -o errexit # Exit immediately when a command fails.
set -o nounset # Treat unset variables and parameters as an error when performing parameter expansion.
set -o pipefail # the return value of a pipeline is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands in the pipeline exit successfully
bash
# Set VAR to DEFAULT if no command line argument exists
VAR="${1:-"DEFAULT"}"
# Functions and local variables
hello_world(){
local name="$1"
echo "Hello world: ${name}"
}
# if then
if [[ "$NAME" = "Dirk" || "$USER_NAME" = "Jose" ]]; then
...
fi
## Directory exists
[[ -d ${file} ]]
## File exists
[[ -e ${file} ]]
Delete all directories that are empty
find . -empty -type d -delete
Filter only email addresses from file
grep -EiEio '\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b' <filename>
Networking tools
- ipcalc: calculcate broadcast, network, etc range for a given IP address and netmask
- prips: print the IP addresses in a given range
Docker
# Build an image with name and tag djg:1.0
docker build . -t djg:1.0
# Start container detached with port 8000 forwarded to port 5000
docker run -p 8080:5000 -d djg:1.0
# List running containers
docker container ls
# Run more processes in a container
docker exec -ti my_container sh -c "echo a && echo b"
# Stop container
docker stop my_container
# Run bash container interactively and connect to terminal
docker run -it bash
# Rerun a container
docker start -ai my_container
# Inspect my_container
docker inspect my_container
# Inspect volume_id
docker volume inspect volume_id
# List docker images
docker images
# Show information about running containers
docker ps -a -s
# Clean up everything and don't ask
docker system prune -af --volumes
# Remove all stopped containers
docker rm $(docker ps --filter status=exited -q)
# Run x86-64 alpine image
docker run -it --rm --platform linux/amd64 alpine
# Run Python 3.13 image and mount cwd as /code into the container
docker run -it --rm -v .:/code python:3.13 /bin/bash
Elasticsearch
# list indexes
GET /_cat/indices?v
# list disk allocation
GET /_cat/allocation?v
# delete an index
DELETE my-old-index
JSON tooling
A great overview of jq: https://earthly.dev/blog/jq-select/
Download sample data: ```wget -O rki-covid.json ‘https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_COVID19/FeatureServer/0/query?where=1%3D1&outFields=*&outSR=4326&f=json'````
Format document:
jsonlint rki-covid.json
jq '.' rki-covid.json
Query data:
# Extract all fields
jq '.["fields"]' rki-covid.json
# Extract first field
jq '.["fields"][0]' rki-covid.json
# Remove all whitespace
jq -c '.' rki-covid.json
Make JSON greppable
gron rki-covid.json
Images
# Make all jpeg files in a directrory smaller
mogrify -strip -quality 75% *.jpg
mapfile
Go through a file in chunks
NUMBER_OF_LINES=25
while mapfile -t -n $NUMBER_OF_LINES ary && ((${#ary[@]})) ; do
echo "${ary[@]}"
# ... do something useful here
done < input_file
Gitlab
# Rule that is always true
rules:
- when: always
VSCode extensions
Interesting VSCode extensions:
- autoDocstring
- Black Formatter
- Dart
- Data Wrangler
- Dev Containers
- Docker
- Error Lens
- ESLint
- Flutter
- Git History
- Github Copilot
- Github Copilot Chat
- IntelliCode
- isort
- Jupyter
- Jupyyter Cell Tags
- Jupyter Keymap
- Jupyter Notebook Renderers
- Jupyter Slide Show
- LaTex Workshop
- Markdown All in one
- markdownlint
- Package Json Upgrade
- PlantUML
- Pylance
- Pyright
- Python
- Python Debugger
- Rainbow CSV
- Ruff
- String Manipulation
- XML
- YAML