Today, I learned about shell process substitution from Break on Penguin’s latest video. Assume you have two files: $ cat file1 a b c $ cat file2 c b a To see if they are identical when sorted without manually creating your own temporary files, you can: diff <(sort file1) <(sort file2) What is happening here? <() is doing process substitution, which is running the command and then exposing its output as a temporary file. ...
I got bit by this again today, so I’m writing it down so I can reference it later. When running docker logs -f CONTAINER_NAME | grep -v "foo" | jq, it is important to consider grep’s buffers. I am using GNU grep and, by default, it uses block buffering when not connected to a terminal (which is my case, since I piped it to jq). This means jq doesn’t get any input until a large-ish amount of text (4k? 8k?) is sent to grep. Since container I was inspecting logs little, I got zero log lines. ...
This post documents the high level concepts I need to remember about *nix Load Average. Load average can been seen in the output of uptime and top: From uptime: $ uptime 14:40 up 21 days, 21:48, 5 users, load averages: 2.63 2.75 3.45 From top: root@blog-server-2:~# top -n 1 | head -2 top - 15:28:37 up 10 days, 12:41, 1 user, load average: 0.04, 0.03, 0.00 Tasks: 95 total, 1 running, 94 sleeping, 0 stopped, 0 zombie root@blog-server-2:~# There are 3 decimal numbers shown. They present the load average over three different time scales: ...
I recently was having to work with Ansible for several days to deploy changes globally across prod. I was storing my playbook limit in a file like so: [18:45] james_simas@widget ~/Downloads $ cat limit.txt host1.domain.com host2.domain.com host3.domain.com At one point, I needed to use supply the list of hostnames in BASH, but rather than having them be newline separated, I needed them to be separated by commas like this: host1.domain.com,host2.domain.com,host3.domain.com I didn’t want to modify my original files (I still needed them) and this seemed like a problem somebody else has problably solved. I Googled a bit and found an answer using tr. ...
Send 10,000 requests, using 10 sockets at a time: ab -n 10000 -c 10 https://10.10.10.10/ Send requests with a BIG-IP persistence cookie: ab -n 10000 -C BIGipServerMyPool=1677787402.36895.0000 http://10.10.10.10/
Miscellaneous: Run script with verbose output: bash -v Check for syntax errors: bash -n Show xtrace info: bash -x Lines which start with + are executed in the shell Lines which start with ++ are executed in a sub-shell Loop over alphabet with upper and lower case: for l in {{A..Z},{a..z}}; do echo $l ; done Print text in beautiful columns by piping to: column -t Echo a string to stderr: >&2 echo Hello, world. Keyboard shortcuts: ...
Display response code and total time: curl -s -w "%{http_code} - %{time_total}\n" -o /dev/null www.jamessimas.com curl multiple IPs with one command: curl 10.20.20.8[0-9]:80/ Overriding DNS: curl --resolve "www.jamessimas.com:80/10.11.12.13" www.jamessimas.com
Delimited: The delimiter in the below examples is :. Show field 1: cut -f 1 -d : /etc/password Show field 2 and sort results numerically: cut -f 2 -d : /etc/password | sort -n Show all fields after field 1: cut -d: -f 2: Show fields 1 through 2: cut -d : -f1-2 Miscellaneous: Cut 10 from start of line: cut -c -10
Display short response: dig +short www.google.com Display only answer section: dig +noall +answer www.google.com Specify source address for query: dig -b <source address[:#port]> Specify DNS server to query: dig @DNS_SERVER www.google.com Use ends0: dig @DNS_SERVER +subnet=10.11.12.13 1.1.1.1
Copy directory recursively with status bars: rsync -avh --progress -e ssh /source/dir/ user@remotehost:/dest/dir/