
Summary:
This page will be the central hub for all things BASH. I usually post little tidbits that I’ve found that are useful, neat, cleaver or all of the above.
I usually find these snippets by searching for a certain Bash function syntax for a script I may be writing. More often than not I usually come across a new function or method that I want to remember.
This page will be used to keep track of all of those little tips, tricks and scripts that I come across and hopefully also present in a organized manner.
Lets Start with Some Reference Sources:
Here are some useful web resources that you may find useful.
- Beginner’s Guide to the Bash Terminal – YouTube Video Crash Course – I thought this was a very good, to the point and well spoken guide. Anyone that is looking for an introduction to the Linux shell and Bash, this is a good place to start. The video is about a hour & 14 minutes long and for the amount of material covered in that short time is well worth the time you spend watching it.
- Advanced Bash-Scripting Guide – An in-depth exploration of the art of shell scripting
- Bash Guide – Greg’s Wiki – This is a nice site with plenty of information on Bash
- Bash Hackers Wiki – Command Substitution – This site does a good job of explaining command substitution, which is something that I was having a little trouble with
- Bash Parameters Guide
- How to Use Awk and Regular Expressions to Filter Text or String in Files – tecmint.com – I got some ideas from this guide when trying to filter out ‘data-attribute’ variables fro HTML files for the vocab-dl script
-
Awk Using Multiple Field Separators Using Awk – This page has a tip about getting access to Windows Clipboard. Below is an excerpt from the site:
awk using multiple field separators
awk -F “=| “You can use multiple field separators by separating them with | (=or). This may be helpful when you want to split a string by two separators for example. #echo “one=two three” | awk -F “=| ” {‘print $1, $3’} one three1 Alternatives+ Submit Alt
-
I spent a bunch of time yesterday looking for the xsel package in Cygwin- turns out you can use the /dev/clipboard device to do the same thing. Show Sample Output
cat /dev/clipboard; $(somecommand) > /dev/clipboard· 2009-07-10 18:48:21 1
What Others Think
-
-
Store the output of a command into a Array
Excerpt from site:If you just want the numbers at the end of each line:numbers=( $(pdc ... | grep -oP 'okay.+?\K\d+$') )
If you want to store each line into the array
mapfile -t lines < <(pdc ...)
To retrieve the data from the arrays:
for (( i=0; i<${#numbers[@]}; i++ )); do echo ${numbers[i]}; done echo printf "%s\n" "${lines[@]}"
33 22 11 [okay ]: you are currently listening: 33 [okay ]: you are currently listening: 22 [okay ]: you are currently listening: 11
Another curl(1) – Linux man page – Another man page on curl
- Linux Shell Scripting Tutorial This page has a 300+ page slide show that’s more of a book. Requires an account to download though
More links to come…
My Current aliases in ~/.bashrc:
These are the aliases I set up in my ~/.bashrc file. TODO: look into the notify-send command.
# some more ls aliases alias ll='ls -alF' alias la='ls -lAhF' #alias l='ls -CF' alias l='ls -lhF' alias fhist='history | grep ' # Add an "alert" alias for long running commands. Use like so: # sleep 10; alert alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"' alias cls="ls -lha --color=always -F --group-directories-first |awk '{k=0;s=0;for(i=0;i<=8;i++){;k+=((substr(\$1,i+2,1)~/[rwxst]/)*2^(8-i));};j=4;for(i=4;i<=10;i+=3){;s+=((substr(\$1,i,1)~/[stST]/)*j);j/=2;};if(k){;printf(\"%0o%0o \",s,k);};print;}'"
Generating Random Numbers:
shuf -i 1-100 -n 1
Found at: unix.stackexchange - Generate Random Numbers in Specific Range
Don't Forget the Quotes!
I ran into an issue with 'unary operator expected' in one of my if statements. This tip came in handy to understand what I was doing wrong:Source: Bash Script Error Unary Operator Expected
Quotes!
if [ "$1" != -v ]; then
Otherwise, when
$1
is completely empty, your test becomes:[ != -v ]
instead of
[ "" != -v ]
...and
!=
is not a unary operator (that is, one capable of taking only a single argument).General Purpose Tips:
Bash'es Special Variables
I found the special Bsah variable table below at the following site: Bash Scripting Tutorial - Variables - This page contained helpful information on Bash Variable with descriptions and examples. It's worth checking out.
There are a few other variables that the system sets for you to use as well.
- $0 - The name of the Bash script.
- $1 - $9 - The first 9 arguments to the Bash script. (As mentioned above.)
- $# - How many arguments were passed to the Bash script.
- $@ - All the arguments supplied to the Bash script.
- $? - The exit status of the most recently run process.
- $$ - The process ID of the current script.
- $USER - The username of the user running the script.
- $HOSTNAME - The hostname of the machine the script is running on.
- $SECONDS - The number of seconds since the script was started.
- $RANDOM - Returns a different random number each time is it referred to.
- $LINENO - Returns the current line number in the Bash script.
If you type the command env on the command line you will see a listing of other variables which you may also refer to.Using sudo & echo together
The following tips were originally found at: sudo echo Does Not Work Together in Ubuntu - Another Waste of Time Issue - Page has info on using echo with sudo privileges.
Sometimes, you need to deal with a really time-wasting issue. I think it
happens even more often if you work in the IT industry. You know the
resolution must be simple, but all your attempts failed. So you try to
search the web, looking for a plain sentence that will lead you out of
that mess...If you want to write some text to a file that needs a root privileges, you do it this way:
echo "Text I want to write" | sudo tee /path/to/file > /dev/null
or (updated after reading discussion below):
sudo sh -c 'echo "Text I want to write" > /path/to/file'
If you just want to append some text, you do it this way:
echo "Text I want to write" | sudo tee -a /path/to/file > /dev/null
or (updated after reading discussion below):
sudo sh -c 'echo "Text I want to write" >> /path/to/file'
This won't work:
sudo echo "Text I want to write" > /path/to/file
Found the previous reference from the following Google Search: "sudo -s echo"
Why some lines in BASH history become '*'
I use the history command a bunch with Bash, especially with grep. One, my memory ain't what it used to be and two, my younger self wouldn't even be able to rember some of these commands because of the syntax & length.
There's a reason we have these tools available to us.
With the history command however, I noticed that certain commands I entered had asterisk (*) next to them and wondered what that meant.
In an unusual fashion, the man page 'man history' did not contain info about the '*'. The help command 'history --help' did shed some light on the situation.:
Display the history list with line numbers, prefixing each modified
entry with a `*'. An argument of N lists only the last N entries.To understand this a little better, I did a quick search and found this post: Why Some Lines in Bash History Become Asterisk - StackOverflow.com - Which basically states that if you hit the [UP ARROW] to display the previous command, change it in some way, then without hitting [ENTER], press the [DOWN ARROW] to return to the last prompt and run the history command, the history will now show an asterisk (*) in front of the previous command.
I find this helpful when I enter a command in wrong, whether it's a mis-key, syntax error or just plain stupidity, I like to modify the commands I entered incorrectly so I don't won't find my self running them again.
In the past, if I had a long command that I new I ran not too long ago, I'll hit the [UP ARROW] repeatedly until the command appears. The issue I've ran into is that the error rate and the command length are directly correlated. So if I mis-typed a command and left it in the history, the next time I need to use it and scroll back through the history, I often forget, and overlook the fact that I entered the command wrong at one point & wouldn't you know its my luck to always pick that command to run.
When I scroll through the history to run a command to help prevent me from these mis-key errors and end up picking the original command I mis-typed to execute...well, it's dumb and it pisses me off.
What I've been trying to get in the habit of doing is modifying the mis-typed commands so they are the correct syntax, then with out hitting [ENTER], I scroll down to the last command and usually execute something simple list a 'ls' command so the modified command gets saved into the history.
The reason this is necessary is because if you hit [ENTER] after fixing the command syntax, that command would be saved as the most recent line in the history and the line with the error will remain.
I've included a animated gif to show as an example:
TODO: Lookup Ctrl+ key combo for saving partial/edited commands.
Input Redirection Examples:
This section has some examples of input redirection.
Bash Variables:
Comparing Integer Variable Examples
With Bash being a dynamically typed language, explicitly setting variables to be either an integer or string has tripped me up a couple times. Below is a list of some examples found at: stackoverflow - Comparing The Integer Using If Condition in Bash Scripts -The post below is the first time I saw someone use '=' '<' and '>' symbols for comparison operators.
Everywhere else I've seen the usage of '-gt', '-ge', '-lt', '-le', '-eq', '-ne' when comparing integers.
Also, for catching the output of a command and put inside of a variable use:
var1=$(put your command with params here)
Therefore, the first line would be:
output=$(df --output=pcent /beep/data | grep -Eo '[0-9]+') echo "${output}"
In BASH, there can be no spaces between the equal symbol, the name of the variable and the assigned value.
limit=80
Finally, for comparing integers use double parenthesis and variables without '$' to make the comparison, instead double brackets.
if (( output >= limit )); then echo 'output is greater or equal than limit' fi
You can use for comparing:
== Equal to != Not equal > Greater than < Less than >= Greater or equal <= Less or equal
curl:
Some curl usage
The following example is from: askubuntu.com - Bash Script Store curl Output in Variable Then Format Against String in
var="$(curl ipinfo.io/"8.8.8.8" 2>/dev/null)" <<<"$var" awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}'
% var="$(curl ipinfo.io/"8.8.8.8" 2>/dev/null)" % <<<"$var" awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}' Mountain View, California
However unless you want to use
curl
's output multiple times you may just use a pipe:curl ipinfo.io/"8.8.8.8" 2>/dev/null | awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}'
curl ipinfo.io/"8.8.8.8" 2>/dev/null | awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}' Mountain View, California
<<<
is a form of input redirection called "here string"; it redirects the STDIN of a command from the terminal to a string.What happens here is that
$var
is expanded between the double quotes; the STDIN of the AWK command is redirected from the terminal to the expanded string and AWK consequently reads the string as its input file.Bash Shared Function - This page has a Bash script with lots of useful functions. Definitely worth checking out.
More curl examples
I found this command on stack overflow How to Build an If Condition in Shell to Check Whether curl Succeeded:
if [ $(curl -sL -w "%{http_code}" "http://google.com" -o /dev/null) = 200 ]; then echo "Success" else echo "Fail" fi
Here are a couple more user examples:
if [ "$(curl -sL -w '%{http_code}' http://google.com -o /dev/null)" = "200" ]; then echo "Success" else echo "Fail" fi
As Charles suggested, you can further simplify this with
--fail
option, as long as you are looking for a success or failure:if curl -sL --fail http://google.com -o /dev/null; then echo "Success" else echo "Fail" fi
Here is a link to some more curl resources
-End of Page
This is a useful one, thanks. The alternative command is unrelated, but decent too.