Linux tools

Just some reminders for Linux-Tools and commands…

mkfifo - Named Pipes

Instead of redirecting the output of a command to a new one, you can use named pipes instead

$ mkfifo newpipe
$ tail  > newpipe

Behind that simple command lies a whole chapter of named pipes, usage scenarios and things to do wrong. What you can basically can do is communicate between different processes using a pip-file (FIFO). More Info.

col - Filter reverse line feeds from input

Useless? Well, handy to save man pages as plain-text …

$ man less | col -b > less.txt

lsblk

fdisk is out, lsblk is in (when it comes to display the current block devices and their settings):

NAME                     MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                        8:0    0 111,8G  0 disk
├─sda1                     8:1    0   243M  0 part  /boot
├─sda2                     8:2    0     1K  0 part
└─sda5                     8:5    0 111,6G  0 part
  └─sda5_crypt (dm-0)    252:0    0 111,6G  0 crypt
    ├─mint-root (dm-1)   252:1    0 103,8G  0 lvm   /
    └─mint-swap_1 (dm-2) 252:2    0   7,7G  0 lvm   [SWAP]
sdb                        8:16   0   1,4T  0 disk
└─sdb1                     8:17   0   1,4T  0 part  /media/jt/8a9d3a12-7e3a-431f-871f-5e0e21ca09c8
sr0                       11:0    1  1024M  0 rom
loop0                      7:0    0   120M  0 loop
└─truecrypt1 (dm-3)      252:3    0 119,8M  0 dm    /media/truecrypt.system

make

Make is more handy than actually considered by the most admins. You can use it to automatically maintain a couple of files and keep them updated by just running one command. The clue lies within the makefile-file which you create where you define all the dependencies and commands to run.

For example a makefile can look like this:

sampleexec : src1.o src2.o
       gcc -o sampleexec src1.o src2.o

src1.o : src1.c
       gcc -c src1.c

src2.o : src2.c
       gcc -c src2.c

This results in the following procedure done by make for creating the file sampleexec:

  1. Sees that sampleexec depends on the object files src1.o and src2.o.

  2. Looks for the target definition of the two object files.

  3. Sees that src1.o depends on the file src1.c.

  4. Executes the commands given in src1.o’s rule and compiles src1.c to get the object file.

  5. Similarly looks at the target src2.o and compiles the object files.

  6. Prepares sampleexec by combining the two object files.

Exit status

How to get the exit status from a piped Command-shell (where the exit status usually comes from the last command)

In Bash the exit codes are provided in the special array PIPESTATUS.

cmd1-exit code is in $PIPESTATUS[0], cmd2-exit code in $PIPESTATUS[2], so that $? is always the same as $PIPESTATUS: -1.

cmd1 | tail -n 2
if [ ${PIPESTATUS[0]} -ne 0]
then
  echo “cmd1 failed.”
  EXIT_CODE = 1
fi

Tail - Multitail

For controlling more than a single file with tail -f you simply need to specify the additional files

$> tail -f /varlog/messages -f /var/log/syslog

You’ll get an output like this:

==> messages <== Nov 22 00:43:07 laptop kernel: [ 61.487969] ADDRCONF(NETDEV_UP): eth0: link is not ready
==> syslog <== Nov 22 00:43:31 laptop pulseaudio[1773]: ratelimit.c: 1 events suppressed
==> messages <== Nov 22 00:43:31 laptop pulseaudio[1773]: ratelimit.c: 1 events suppressed

==> syslog <==
Nov 22 00:45:44 laptop ntpd[1118]: kernel time sync status change 6001

So that keeps you updated. If you want it in a different way you could go for multitail. That lists like the original tail, but opens a couple of windows to switch between.

mmv

Renaming files can be a pain in the ass. Especially when renaming a couple of files which only have slight error in the naming (according to your idea of proper naming). The tool mmv (available from the repositories) can fix this.

Assuming you’ve got a file called artists_-_title-stupid comment.mp3 and you want to rename it to a simple artist-title.mp3 and do this for all files in a directory:

$ mmv \*_-_\*-\*.mp3 \#1-\#2.mp3

And done.

Keep also an eye on the parameter switches available, that e.g. let make you a copy instead of overwriting the original file.

Mass File Renaming

Find is your buddy here:

# Remove stupid special characters
$ find . -maxdepth 1 -type f | rename 's/ /_/g'
$ find . -maxdepth 1 -type f | rename 's/\(//g'
$ find . -maxdepth 1 -type f | rename 's/\)//g'
$ find . -maxdepth 1 -type f | rename 's/\,//g'
$ find . -maxdepth 1 -type f | rename "s/\'//g"
$ find /tmp/ -depth -name "* *" -execdir rename 's/ /_/g' "{}" \;

Alternatively have a look at detox.

Screen

Yeah, yeah.. common virtual terminal tool.

Moving Terminals to new position

  • change to the window you want to mo*

  • type (for example) &#94;x:number 1

    • &#94;x is the host key (usually &#94;a on most machines)

    • :number (typed literally) is the command

    • 1 (as a digit) the number to move the current screen to

Resizing splittet windows

  • type (for example) &#94;x:resize -4

    • &#94;x is the host key (usually &#94;a on most machines)

    • :resize is the command

    • -4 (as a digit) the number of lines to make the current window smaller.

Logging

You can either make a hard copy or switch on the logging of the terminal window you’re in

Hardcopy

  • Command: hardcopy

    (C-a h, C-a C-h)
    Writes out the current display contents to the file hardcopy.n in the window’s default directory, where n is the number of the current window. This either appends or overwrites the file if it exists, as determined by the hardcopy_append command.

  • Command: hardcopy_append state

    (none)
    If set to on, screen will append to the hardcopy.nr files created by the command hardcopy otherwise, these files are overwritten each time.

  • Command: hardcopydir directory

    (none)
    Defines a directory where hard copy files will be placed. If unset Hardcopys are dumped in screen’s current working directory.

Continuous logging

  • Command: log [state]

    (C-a H)
    Begins/ends logging of the current window to the file screenlog.n in the window’s default directory, where n is the number of the current window. If no parameter is given, the logging state is toggled. The session log is appended to the previous contents of the file if it already exists. The current contents and the contents of the scroll back history are not included in the session log. Default is off.

  • Command: logfile <logfilename>

    (none)
    Defines the filename of the logfile. I prefer to use something like screenlog.%Y%m%d.console_%n. The command also can use the bash environmental variables like e.g. $HOME. But what I can’t do is redirect the output into an encfs encrypted folder.

  • Command: logstamp after <seconds>

    (none)
    Drops a timestamp into the logs after the number of seconds specified.

VI

VI just crashed (well, it was VIM . Bastard.

Recovery

But I learned something new about recovery:

Within vi

:rec:

gets you what you lost. In my case when using mutt I found the file in /tmp.

The VIM Documentation has as well a chapter about recovery you might want to check.

Swapping Character

If you’re like me and you’re typing sometimes to fast, this will swap the current character with the next one:

xp

Upper/Lowercase

Inside vi you can easily convert whole lines or the whole file to uppercase or lowercase letters:

:%s/.*/\L&/g     # All letters to lowercase
:%s/.*/\U&/g     # All letters to uppercase

Apt

# Install application dependencies
$ apt-get build-dep <packagename>

Dirstack, pushd, popd

If you temporarily need to build up a stack of some directories you’re gonna switch between you can use the dirs stack and the tools pushd and popd to access them.

  • dirs: Shows the current stack.

  • pushd: Adds directories to the stack or/and switches between them.

    pushd +2: jump to the second entry and switch to the directory
    pushd .: Add the current directory to the stack
    pushd: Switcht the first two entries of the stack and change to the new top directory.

  • command:popd: Removes the first entry of the stack and changes the folder to it.

CDPATH

The search path for the cd command. This is a colon-separated list of directories in which the shell looks for destination directories specified by the cd command. A sample value is .:~:/usr.

By placing common directories in this variable we can change common directories without navigating there.

(pwd = /home/juser)
cd /usr/local/projects/abc
 vs.
cd abc
 (pwd = /usr/local/projects/abc)

 (pwd = /usr/local/projects/abc/src/db/old/migration/)
cd ../../../..
 vs.
cd abc
 (pwd = /usr/local/projects/abc)

Indeed, one can find many subtle uses for this feature. A reasonable list to start with on a Red Hat Linux system, for example, might be

.:~:~/docs:/mnt:/usr/src/redhat:/usr/src/redhat/RPMS:/usr/src:/usr/lib:/usr/local

Note

An empty entry (:: or a leading or trailing :) in $CDPATH (or in $PATH) is interpreted as the current working directory!

Configure options for a build package

Sometimes you have to recompile an already installed package just to add a single option that is missing in the standard packages that comes from your sources. You might wonder what where the default options it was compiled with.

PATH=$PATH:/build/buildd/asterisk-1.6.2.7/debian/dummyprogs ./configure \
   --host=x86_64-linux-gnu --build=x86_64-linux-gnu \
   --prefix=/usr \
   --mandir=\${prefix}/share/man \
   --infodir=\${prefix}/share/info \
   --with-cap \
   --with-gsm \
   --with-imap=system \
   --with-pwlib=/usr/share/pwlib/include/ \
   --with-h323=/usr/share/openh323/

Timezone adjustements

Occasionally timezones are a bit off and that has quite a negative impact on your logs and other data, where time is a kind of - well: critical information. So basically everywhere.

So when you find yourself in a different timezone than you think it should be…

shell> cat /etc/timezone
Etc/UTV

adjust it by tuning…

$ sudo dpkg-reconfigure tzdata

Don’t forget to restart the cron-daemon. He doesn’t get the new timezone until being restarted.

$ sudo systemctl restart cron

Copying files

Some nice and neat tricks to get files from one machine to another.

Netcat

On the receiver side:

nc -l 12345 > file.tar

On the sender side (three possibilities):

$ nc x.x.x.x 12345 < file.tar
$ cat file.tar > /dev/tcp/x.x.x.x/12345 (without netcat)
$ pv file.tar | nc x.x.x.x 12345 (with status)

Python

python2# python -m SimpleHTTPServer
python3# python -m http.server

It spawns a proper HTTP server (including directory listing) sharing the current folder.