Linux and Terminal Basics
The working environment of computational science is mostly the Linux terminal. DFT codes such as SIESTA and VASP run in the shell without a GUI, and everything — connecting to compute servers, watching logs, editing inputs, retrieving results — is done with shell commands. This chapter distills the minimum shell knowledge needed to follow this tutorial, using hands-on examples with the files you will actually handle (input.fdf, siesta.out, OUTCAR, etc.). If you are already comfortable in the terminal, you can just skim the Frequently Used Combinations table at the end and move on.
Navigating Directories
| Command | Role |
|---|---|
pwd | Print the current directory path |
ls | List files (-l detailed, -lh human-readable sizes, -ltr most recently modified last) |
cd path | Change directory (cd .. up one level, cd ~ home, cd - previous location) |
pwd # /home/user/work/ch01-siesta-scf
ls -ltr # files just written by the calculation appear at the bottom of the list
cd ../ch02-bands # move to a sibling directory
ls -ltr is especially useful in a calculation directory — the most recently updated files come last, so you can immediately see what the calculation is writing right now.
File Operations
| Command | Role |
|---|---|
mkdir -p path | Create a directory (-p creates intermediate paths in one go) |
cp source target | Copy (-r for whole directories) |
mv source target | Move or rename |
rm file | Delete (-r for directories) |
mkdir -p scan/c_5.16 # create a calculation case directory
cp input.fdf input.fdf.bak # back up before editing
mv siesta.out siesta.out.old # preserve the previous log before rerunning
Linux has no trash bin. Files deleted with rm cannot be restored, and in particular rm -rf directory wipes the entire tree without confirmation. You can lose days of calculation results with a single command, so it is safer to build the habit of moving to an archive directory instead of deleting, e.g. mv old_run/ _archive/. Before a wildcard delete (rm *.out), first run ls *.out with the same pattern to check the target list.
Inspecting File Contents
| Command | Role |
|---|---|
cat file | Print the whole file — for short files (checking input files) |
less file | Page-by-page viewing — for long logs. /string to search, G to jump to the end, q to quit |
head -n 20 file | First 20 lines |
tail -n 20 file | Last 20 lines |
tail -f file | Follow the end of the file in real time — watching a running calculation log |
cat input.fdf # final check of the input just before submission
tail -f siesta.out # watch SCF iterations appear in real time (exit with Ctrl-C)
tail -f is the most basic way to confirm a calculation is progressing normally. If the output has been stalled for several minutes, something has likely gone wrong.
Searching — grep
grep extracts lines matching a pattern from a file. It is the essential tool for finding the numbers and warnings you want in calculation logs that run to thousands of lines.
grep -i "total energy" siesta.out # search energy lines, case-insensitive (-i)
grep "SCF cycle converged" siesta.out # check that the convergence message exists
grep -in "error" siesta.out # search for errors, showing line numbers (-n)
grep -c "scf:" siesta.out # count SCF iterations (-c)
grep -A 6 "TOTAL-FORCE" OUTCAR # matching line plus 6 lines below (-A) — VASP force block
grep -r "MeshCutoff" scan/ # search a whole directory (-r)
Frequently used options — -i ignore case, -n line numbers, -c count, -A N/-B N include N lines after/before the match, -r recursive, -v only non-matching lines.
Streams and Redirection
Every program has three streams — standard input (stdin), standard output (stdout), and standard error (stderr). The run command in this tutorial has exactly this structure.
siesta < input.fdf > siesta.out
< input.fdf— connects the file to stdin. SIESTA follows the convention of reading its input from stdin.> siesta.out— sends stdout to a file (overwriting existing contents).>>appends instead of overwriting.- With the command above, stderr still goes to the screen. To capture errors into the same file, append
2>&1— meaning "merge stream 2 (stderr) into where stream 1 (stdout) goes."
mpirun -np 4 siesta < input.fdf > siesta.out 2>&1
In MPI runs and batch jobs, library errors and out-of-memory messages often appear only on stderr, so appending 2>&1 is the safe choice.
Pipes
| passes the stdout of the preceding command to the stdin of the following one. You assemble small commands into one-line analyses.
grep "scf:" siesta.out | wc -l # count SCF iteration lines (wc -l = line count)
grep "E0=" OSZICAR | tail -n 1 # only the last ionic-step energy from VASP
grep "Total =" scan/*/siesta.out | sort # sort scan results by case name
wc -l (line count), sort (sorting; -g for numeric order), and uniq (remove duplicates) are the staple pieces of pipes.
sed — One-Line Substitution
sed is a stream editor. When making a copy with one input parameter changed, there is no need to open an editor.
sed 's/300. Ry/400. Ry/' input.fdf > input_400.fdf # create a substituted copy
sed -i.bak 's/300. Ry/400. Ry/' input.fdf # in-place edit, original backed up as .bak
diff input.fdf.bak input.fdf # check only the changed lines
The basic syntax is s/pattern/replacement/; appending g at the end replaces every match within a line. If you modified a file with sed -i, always verify with diff that only the intended lines changed — accidents where the pattern also matches unexpected lines are common.
Shell Variables and for Loops — Lattice Constant Scan
Variables are defined as name=value (no spaces around the equals sign) and referenced as $name. Combined with a for loop, parameter scans can be automated. Suppose you have prepared template.fdf from the input.fdf of Chapter 01, replacing the cell length 5.160000 with the placeholder __C__ (switch the atomic coordinates to Fractional so they scale automatically with the cell).
#!/bin/bash
# scan of cell length c — substitute __C__ in template.fdf with the actual value and run
for c in 5.00 5.08 5.16 5.24 5.32; do
dir=c_${c}
mkdir -p ${dir}
sed "s/__C__/${c}/" template.fdf > ${dir}/input.fdf
cp C.psml ${dir}/
(cd ${dir} && siesta < input.fdf > siesta.out)
echo "c = ${c} done"
done
The parentheses ( ... ) create a subshell, so a cd inside them does not change where the loop is running. When the scan finishes, harvest the results in one line.
grep "Total =" c_*/siesta.out
Archiving and Transfer — tar, scp, rsync
tar czf ch01_results.tar.gz ch01-siesta-scf/ # compress a directory into an archive (c=create, z=gzip, f=filename)
tar xzf ch01_results.tar.gz # extract (x=extract)
scp ch01_results.tar.gz user@server:~/work/ # copy to a remote server
rsync -avz user@server:~/work/ch01/ ./ch01/ # synchronize/retrieve a remote directory
rsync skips files already received and transfers only what changed, so it beats scp for repeatedly retrieving large calculation directories. -avz is archive mode (-a) + list transferred files (-v) + compressed transfer (-z). Whether the path ends with / decides between "the contents of the directory" and "the directory itself," so at first it is safer to check the target list with -n (dry-run).
ssh — Remote Compute Servers
ssh user@server.example.edu # open a remote shell
ssh user@server.example.edu "ls work" # run a single remote command without logging in
Registering host aliases and key settings in ~/.ssh/config lets you type short commands like ssh myserver. Accounts, hosts, and authentication methods vary by institution, so follow your institution's instructions.
Keeping Calculations Alive After Logout — nohup and tmux
A calculation launched plainly in an ssh session dies when the connection drops. There are two remedies.
nohup mpirun -np 4 siesta < input.fdf > siesta.out 2>&1 &
nohup makes the process ignore the hangup signal, and the trailing & sends it to the background. After logging out, reconnect and check progress with tail -f siesta.out.
tmux is a tool that keeps the terminal session itself on the server.
tmux new -s calc # create a session named calc
# (run the calculation inside the session)
# press Ctrl-b then d — detach, leaving the session attached
tmux attach -t calc # return to the session after reconnecting
You can keep several running calculations open and move between them, so tmux is more convenient on compute servers used interactively. On proper clusters, however, jobs are submitted through a batch scheduler instead — covered in Chapter 12.
Execute Permission — chmod +x
A shell script needs execute permission to be run as ./script.
chmod +x scan.sh
./scan.sh
Running without permission yields "Permission denied." You can also invoke the interpreter directly, as in bash scan.sh, but the standard convention is to have both the #!/bin/bash (shebang) on the first line of the script and the execute permission.
Frequently Used Combinations
| Situation | One-liner |
|---|---|
| Watch a running calculation log in real time | tail -f siesta.out |
| How many SCF iterations have run | grep -c "scf:" siesta.out |
| Check SCF convergence | grep "SCF cycle converged" siesta.out |
| SIESTA final energy | grep "Total =" siesta.out |
| VASP last ionic-step energy | grep "E0=" OSZICAR | tail -n 1 |
| Check the VASP force block | grep -A 6 "TOTAL-FORCE" OUTCAR |
| Find errors/warnings in a log | grep -in "error" siesta.out |
| Check recently updated files | ls -ltr |
| Harvest parameter scan results | grep "Total =" c_*/siesta.out |
| Substitute one input parameter (with backup) | sed -i.bak 's/300. Ry/400. Ry/' input.fdf |
| Verify the edit | diff input.fdf.bak input.fdf |
| Compress a results directory | tar czf results.tar.gz ch01-siesta-scf/ |
| Retrieve results from a server | rsync -avz user@server:~/work/ch01/ ./ch01/ |
| Keep a calculation alive after logout | nohup mpirun -np 4 siesta < input.fdf > siesta.out 2>&1 & |
| Check directory sizes | du -sh */ |
Once the commands in this table are second nature, the shell will never be what blocks you in the main tutorial. Next, What Is SIESTA gives an overview of this tutorial's primary code.