My Vim Cheat Sheet

Today I cleaned up and formatted the Vim cheat sheet that I created a while ago. Hope this can help you understand Vim better. If you have any suggestions, please let me know so I can improve it.

This cheat sheet is inspired by this blog post from Yan. I also made my changes based on my understanding of how Vim works.

Remove old Linux kernels in Ubuntu

If you are a GNU/Linux user, you are very likely to use one of the popular Linux distributions. For example, Ubuntu, Fedora, Debian, Arch Linux and so on. These distributions will provide free updates to your entire operating system, ranging from the Linux kernel, necessary daemons and your day-to-day applications. When you update your system to the latest version, you may or may not know that sometimes your kernel gets updated, too.

In Ubuntu (I believe this also applies to many other distributions), when the package manager updates most other packages installed in your operating system, the old package will be replaced. That is, the package with older version is uninstalled first. So that it does not conflict with the newer version of the package. Then the newer version is installed to finish the update.

However, kernels are updated in a different manner. As the package manager updates your kernel, your old kernel is preserved. This is because that in the old times, using a new version of kernel may cause your system to not boot at all. So many times in the boot loader you may see a long list of all available kernels for you to start the operating system with. And if one does not work, you still has the hope to try another.

Nonetheless, this problem rarely existed now because it is already 2012 and the Linux kernel is very mature. If you are not a hardcore kernel hacker, most if not all times you just need the latest one -- who knows what security hole they fixed two days ago. In this case, you probably want to remove older Linux kernels to free up disk space because each one takes about 100MB when installed. Who doesn't like free disk space?

This gist can be used to remove all unused Linux kernels installed on your Ubuntu operating system. Note that it removes all kernels that are not currently in use. If you are using kernel version 3.2.0-24 and the latest kernel installed on your system is 3.2.0-27, the 3.2.0-24 one will be kept but the latest 3.2.0-27 kernel will be removed. So make sure use this gist only after you have rebooted from the newly installed kernel. In other words, after you installed a new Linux kernel, reboot to use the latest kernel before using this gist.

Gist break down:

• dpkg -l lists all packages known by your dpkg based system.
• awk '{print $2}' filters the output of the previous command so that only the second column is kept, which is the name of the packages.
• grep "linux-\(image\|headers\)-[0-9]" filters the output to only include package names like linux-image-X.. or linux-headers-X.. where X is a numerical character. The result of this is simply all kernel packages known by your Ubuntu system.
• grep -v `uname -r | sed s/-generic//`
• uname -r outputs the release of your currently active kernel. On a Ubuntu distribution it usually is in the form of "X.X.X-X-generic".
• sed s/-generic// replaces "-generic" with an empty string. The result is only in the form "X.X.X-X" as the version of the currently active kernel.
• grep -v `...` filters the list of all kernel packages so that the package names does not contain the currently active kernel version. The result of this is a list of all unused kernel packages.
• xargs sudo apt-get purge -y takes the previous result as command-line arguments. This command removes from your system the packages and configuration files in the unused kernel packages. Note that because of the -y option, it will not give you a chance to confirm (For some reason without the -y option, apt-get will simply abort). Handle with care.

For more information, please refer the man pages for these commands.

Creating a manual network delay on MySQL (or almost any) connections.

While working in TransLoc Inc. I had to stress test a component that writes to a MySQL database in a very frequent manner. Each process of the component will update about 10-20 rows in one table every second, and there will be about fifty of those processes running.

In the production environment, the component does not run on the same server as the MySQL server, so there is a network latency in each query. Thus, the result of stress testing the component on my own machine will not be realistic because it does not take the network latency into account.

To create a manual network delay in the testing, I used the SSH Tunnel to forward my MySQL request to a remote host, and forward it back onto the MySQL server on my own machine using the following command:

ssh  -N -L 23306:127.0.0.1:60000 -R 60000:127.0.0.1:3306

Command break down:

• ssh -N creates a SSH tunnel without executing any command on the remote host. This way, you won't see the usual remote shell command line after you execute the command.
• -L 23306:127.0.0.1:60000 forwards the local connection to port 23306 to the 127.0.0.1 of the remote host port 60000. Which means, when a client tries to connect to your local machine port 23306, the request will have same effect of doing it on the remote host to 127.0.0.1:60000.
• -R 60000:127.0.0.1:3306 forwards a connection to the remote host port 60000 to the port 3306 of 127.0.0.1 of your local machine. it opens the port 60000 on the remote host. When there is a connection to the port, it is forwarded to the 127.0.0.1:3306 of the local machine.

To explain SSH tunnel a little bit more, here's a better example:

Imagine the source machine wants to connect to a port on the target machine. But both machines are in a sub-network and cannot connect to the each other directly. To solve this problem, An SSH tunnel can be created on the local host to the remote host using this command:

ssh 61.21.61.8 -N -L 12345:10.10.10.200:54321

Note that 61.21.61.8 is the public IP address the remote host have. It must be an address that can be seen by the local host. Now if the source machine make a connection request to local host port 12345, the request will be forwarded to the remote host, then to the target machine port 54321. This way, a program listening on port 54321 on target machine can receive the request and accept the connection request. The source machine can just connect to 192.168.1.1:12345 and it will be equivalent to directly connect to 10.10.10.200:54321 on remote host.

For more information, you can always refer to the manual page of ssh.

man ssh

Removing all residual config packages in Ubuntu

In Ubuntu/Debian based operating systems, when a program is uninstalled in Ubuntu Software Center or by using apt-get, the configuration files are not removed if you do not explicitly say so. Usually after several install/uninstalls, there will be a lot of left over configuration files that is never needed anymore.

To remove these configuration files, Synaptic Package Manager have a status filter "Not Installed (residual config)" that can let you easily find these packages. But if you do not have Synaptic installed, a good way to remove all these packages is by using the line below:

$ sudo apt-get remove --purge `dpkg -l | grep '^rc' | awk '{print $2}'`

This line of command is explained:

• dpkg -l will list all packages installed in your system.
• grep '^rc' will search for "rc" in the start of each line, which is the status printed out by dpkg meaning "residual config".
• awk '{print $2}' will print out only the second column of the above output. The second column is the name of the packages that should be purged.
• sudo apt-get remove --purge explicitly removes both the packages themselves and the residual configurations of the packages.

Message Indicator for Empathy in Ubuntu 12.04 not working?

After I upgraded my Ubuntu to 12.04 LTS, the message indicator (or, Messaging Menu) doesn't notify me for incoming messages anymore. The little envelope in the notification area used to turn green when I got a new message from Empathy. But it does not do it anymore, and other nice integration between Empathy and the applet was gone. There was still Notify OSD, though. But it will only display the bubble for 5 seconds. So if I was AFK and someone sent me a message, I was not notified in any obvious way.

This has bothered me for a little while, and finally I found the solution. I have to install a plugin for Empathy, which is called telepathy-indicator. It can simply be installed by the following command:

sudo apt-get install telepathy-indicator

Or, if you are not familiar with linux shell, go into Ubuntu Software Center and search for Empathy, and telepathy-indicator is listed as an Add-on there.

Then what you need to do is just restart your computer (or, log out then log in), and you will be notified whenever a message comes in.

Python multi-line strings

There are several ways to compose a Python string literal in multiple lines if the string is too long. One way is to use the add operator. For example:

str1 = 'this is a very long string.\n' +\
'second line of the string.'

In this first way, it is annoying that you need to type extra characters each line,
i.e. the plus operator and line continuation mark. Another way you can do this is to use the triple-quotes ''' or """:

str2 = '''this is a very long string.
second line of the string.'''

They all yield the same Python string. But this method can look very ugly because it can violate the indentation in the source code.

In Python if you just put two string literals together, with or without spaces in between, your result string will be the concatenated string of all the strings. For example:

str3 = 'this ' 'is ' 'a ' 'string.'

and value of str3 will be:

'this is a string.'

Thus, my preferred way of doing this is to use the implicit line continuation by parentheses, with the implicit string concatenation. For example:

str4 = ('this is a very long string.\n'
        'second line of the string.'
       )

This will give you the same result of the first two examples.

Nice, isn't it?

Update:

There was a PEP 3126 that suggests implicit string concatenation to be removed from Python 3, but it has been rejected. I tested this in Python 3 and it still works. But the document also suggests a good way to do this with the join function:

"\n".join(["this is a very long string.",
"second line of the string."])

Meego and Arch Linux dual boot.

Recently I bought a "notbook" ThinkPad x120e (not arrived yet) which is much more powerful than my two year old netbook Toshiba NB201. So I decided to set up a home server (file server, torrent slave, ipv6 router...) using that Intel Atom based netbook. The idea is to keep me away from high electric bills and noise.

I decided to install MeeGo and Arch Linux on it. This is really strange combination (I know...). But the idea is that the latter is the home server OS, while the former as a minimal backup desktop environment, just in case anything happens and I need to have some easy internet.

MeeGo uses syslinux(extlinux) as its bootloader. It doesn't seem to support chainloading grub, also it cannot directly load a OS from another partition. So the easiest way to dual boot is to use grub in Arch Linux.

1. Install MeeGo (I am using netbook version 1.2). Follow the directions here.

When you are asked to partition your hard drive, You need at least have this following 3 partitions,

a. One ext3 partition mount as /boot, this will be used by Arch Linux later as well, I think 200MB is well enough.

b. One btrfs or ext3 partition mount as / for Meego.

c. A swap partition, of course.

After installed Meego, try boot into MeeGo and find out the name of the linux kernel file. To do this, boot in MeeGo, open a Terminal, type

cd /boot

then type

ls -l

There will be a file name starting with vmlinuz-, with a series of numbers after it (Mine is vmlinuz-2.6.37.2-8.27). This should be your linux kernel file for MeeGo. Write it on a paper for later use.

2. Install Arch Linux (I am using version 2011.8.19). Follow the instructions here.

When you are asked to prepare hard drive, do not use Auto-Prepare. Rather, use "Manually partition Hard Drives". Here you will create a new partition for Arch Linux (or more if needed). Then select "Write" to write the partition table.

Then you will need to manually configure mountpoints. Mount the following partition to Arch Linux,

a. The ext3 partition created in 1a. Also mount it to /boot. Do NOT format it!

b. Format the newly created partition as btrfs or ext3. mount it to / for Arch Linux.

c. The swap partition created in 1c.

Now, the installer will write the partition table.

Later, when you are asked to select bootloader, choose GRUB.

After installation of packages, the installer will ask you to install bootloader for the system. You will be asked to edit the configuration file. After the two Arch Linux items, add the following lines to the configuration file,

# (2) MeeGo
title MeeGo (2.6.37.2-8.27)
root   (hd0,0)
kernel /vmlinuz-2.6.37.2-8.27 root=/dev/sda2 ro quiet

Note that you should change line 3 to whatever partition your mountpoint /boot is on, which is created in step 1a. For example, if it is on /dev/sdb3 it should be root (hd1,2) (Note this one counts from 0).

For line 4, the root= part is the partition you created in step 1b. Thus it should be in a different partition.

Now save, and the installer will quit. Reboot and you will get fresh dual boot of MeeGo and Arch Linux, cheers!