set sqlite3 for django in centos 5.3

I upgraded the python in my centos from 5.4 to 5.5. However, when I try to set up the django to use the sqlite3 database, and it stop working: I cannot get sqlite3 for python 5.5 as the rpm already installed for 5.4. And the one for 5.4 give me some unresolved symbol errors.

I workaround this by download the source from pysqlite. and then run:

python setup.py build_static install 

Now: 

manage.py syncdb

works.

Linux process details

 
Here is the execution ps -ax in my 2.6 64 bit Linux. Here we try to explain what these processes are.

PID TTY      STAT   TIME COMMAND
    1 ?        Ss     0:00 init [5]        
    2 ?        S<     0:00 [migration/0]
    3 ?        SN     0:00 [ksoftirqd/0]
    4 ?        S<     0:00 [watchdog/0]
    5 ?        S<     0:00 [migration/1]
    6 ?        SN     0:00 [ksoftirqd/1]
    7 ?        S<     0:00 [watchdog/1]
    8 ?        S<     0:00 [events/0]
    9 ?        S<     0:00 [events/1]
   10 ?        S<     0:00 [khelper]
   27 ?        S<     0:00 [kthread]
   32 ?        S<     0:00 [kblockd/0]
   33 ?        S<     0:00 [kblockd/1]
   34 ?        S<     0:00 [kacpid]
  132 ?        S<     0:00 [cqueue/0]
  133 ?        S<     0:00 [cqueue/1]
  136 ?        S<     0:00 [khubd]
  138 ?        S<     0:00 [kseriod]
  210 ?        S      0:00 [khungtaskd]
  213 ?        S<     0:10 [kswapd0]
  214 ?        S<     0:00 [aio/0]
  215 ?        S<     0:00 [aio/1]
  364 ?        S<     0:00 [kpsmoused]
  396 ?        S<     0:00 [ata/0]
  397 ?        S<     1:22 [ata/1]
  398 ?        S<     0:00 [ata_aux]
  405 ?        S<     0:00 [scsi_eh_0]
  406 ?        S<     3:52 [scsi_eh_1]
  413 ?        S<     0:00 [kstriped]
  426 ?        S<     0:00 [ksnapd]
  441 ?        S<     0:28 [kjournald]
  474 ?        S<     0:00 [kauditd]
  516 ?        S<s    0:00 /sbin/udevd -d
 1577 ?        S<     0:00 [hd-audio0]
 1786 ?        S<     0:00 [kmpathd/0]
 1787 ?        S<     0:00 [kmpathd/1]
 1788 ?        S<     0:00 [kmpath_handlerd]
 1813 ?        S<     0:00 [kjournald]
 2007 ?        S<     0:00 [kondemand/0]
 2008 ?        S<     0:00 [kondemand/1]
 2025 ?        S<     0:00 [iscsi_eh]
 2069 ?        S<     0:00 [ib_addr]
 2079 ?        S<     0:00 [ib_mcast]
 2080 ?        S<     0:00 [ib_inform]
 2081 ?        S<     0:00 [local_sa]
 2085 ?        S<     0:00 [iw_cm_wq]
 2089 ?        S<     0:00 [ib_cm/0]
 2090 ?        S<     0:00 [ib_cm/1]
 2094 ?        S<     0:00 [rdma_cm]
 2112 ?        Ssl    0:00 brcm_iscsiuio
 2118 ?        Ss     0:00 iscsid
 2119 ?        S<Ls   0:00 iscsid
 2220 ?        Ss     0:00 mcstransd
 2502 ?        Ss     0:02 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient-eth0.leases -pf /var/run/dhclient-eth0.pid eth0
 2553 ?        S<sl   0:00 auditd
 2555 ?        S<sl   0:00 /sbin/audispd
 2567 ?        S      0:02 stardict
 2571 ?        Ss     0:00 /usr/bin/esd -terminate -nobeeps -as 2 -spawnfd 24
 2573 ?        Ss     0:00 /usr/sbin/restorecond
 2584 ?        Ss     0:01 syslogd -m 0
 2587 ?        Ss     0:00 klogd -x
 2623 ?        Ss     0:00 portmap
 2640 ?        S      0:13 /usr/bin/python2.4 /usr/share/meld/meld
 2648 ?        Ss     0:00 rpc.statd
 2674 pts/9    S+     0:01 ssh rh8
 2675 pts/10   S+     0:01 ssh rh8
 2682 ?        S<     0:00 [rpciod/0]
 2683 ?        S<     0:00 [rpciod/1]
 2690 ?        Ss     0:00 rpc.idmapd
 2709 ?        Ssl    0:18 dbus-daemon --system
 2721 ?        Ss     0:00 /usr/sbin/hcid
 2727 ?        Ss     0:00 /usr/sbin/sdpd
 2750 ?        S<     0:00 [krfcommd]
 2792 ?        Ssl    0:01 pcscd
 2808 ?        Ss     0:00 /usr/bin/hidd --server
 2827 ?        Ssl    0:00 automount
 2860 ?        Ss     0:00 /usr/sbin/acpid
 2871 ?        Ss     0:00 ./hpiod
 2876 ?        S      0:00 python ./hpssd.py
 2891 ?        Ss     0:00 /usr/sbin/sshd
 2902 ?        Ss     0:00 cupsd
 2916 ?        SLs    0:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
 2936 ?        Ss     0:00 sendmail: accepting connections
 2945 ?        Ss     0:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue
 2957 ?        Ss     0:00 gpm -m /dev/input/mice -t exps2
 2968 ?        Ss     0:00 crond
 3003 ?        Ss     0:00 xfs -droppriv -daemon

init: created in the system start up which is used to monitor and manager all other processes
migration: ?
ksoftirqd: kernel threads to manage the softirq
watchdog: ?
events: predefine kernel threads to manage work queues
khelper:?
kthread:?
kblockd: special work queue management threads for block device
kacpid:
khubd:
kseriod:
khungtaskd:
kswapd: the kernel thread to mange the memory swap
aio
kpsmoused:
ata:
ata_aux:
scsi_eh_0:
kstriped
ksnapd
kjournald
kauditd
/sbin/udevd -d: event manage daemon to handle device management (udev)
hd-audio0
kmpathd
kmpath_handlerd
kondemand
iscsi_eh
ib_addr
ib_mcast
ib_inform
local_sa
iw_cm_wq
ib_cm
rdma_cm
cqueue:

large file cache use mmap

We had a 3G file and fields position can be identified using hash value. We use the mmap to deal with the cache.
(1) open the file
(2) caculate the position, if the data is not in memory, then call:
mmap2(NULL, 16384, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0x210e8)
which loads the 16K into memory.
(3) for the data in the next request, call the mmap2 again (the mmap can be called multiple times).
(4) When you run out of cache (let's say 300M), find the least used fields, then run
munmap(....)
to cache the fileds out.

Perl provides a module Cache:FastMmap to share memory cross processes.

process virtual memory layout in 32 bit and 64 bit Linux

I did some experiment to compare the process virtual memory layout in 32 bit linux (2.6.9) and 64 bit linux(2.6.18).

This program is used:

const size_t arraySize=0x1000000; //16M
char global[arraySize*2];
int main(int argc, char** argv)
{
        int i=0;
        char local[arraySize];
        char* heap=(char*)malloc(arraySize*3);
        for (i=0; i        for (i=0; i        for (i=0; i        printf("local %lx, global %lx, heap %lx\r\n",local,global, heap);
}

(1) In linux 32 bit, the output is:

local bee40ef0, global 08049800, heap 4000a008

The memory map is:
00734000-00749000 r-xp 00000000 03:03 32066      /lib/ld-2.3.4.so
00749000-0074a000 r--p 00015000 03:03 32066      /lib/ld-2.3.4.so
0074a000-0074b000 rw-p 00016000 03:03 32066      /lib/ld-2.3.4.so
00753000-0075c000 r-xp 00000000 03:03 38761      /lib/libgcc_s-3.4.6-20060404.so.1
0075c000-0075d000 rw-p 00009000 03:03 38761      /lib/libgcc_s-3.4.6-20060404.so.1
00774000-00898000 r-xp 00000000 03:03 32151      /lib/tls/libc-2.3.4.so
00898000-00899000 r--p 00124000 03:03 32151      /lib/tls/libc-2.3.4.so
00899000-0089c000 rw-p 00125000 03:03 32151      /lib/tls/libc-2.3.4.so
0089c000-0089e000 rw-p 0089c000 00:00 0
008a0000-008c1000 r-xp 00000000 03:03 38754      /lib/tls/libm-2.3.4.so
008c1000-008c3000 rw-p 00020000 03:03 38754      /lib/tls/libm-2.3.4.so
00af9000-00bb9000 r-xp 00000000 03:03 195327     /usr/lib/libstdc++.so.6.0.3
00bb9000-00bbe000 rw-p 000bf000 03:03 195327     /usr/lib/libstdc++.so.6.0.3
00bbe000-00bc4000 rw-p 00bbe000 00:00 0
08048000-08049000 r-xp 00000000 00:14 34996225   /mnt/centos/ut/a.out
08049000-0804a000 rw-p 00000000 00:14 34996225   /mnt/centos/ut/a.out
0804a000-0a04a000 rw-p 0804a000 00:00 0
40008000-4300d000 rw-p 40008000 00:00 0
bee40000-c0000000 rw-p bee40000 00:00 0
ffffe000-fffff000 ---p 00000000 00:00 0

The exeutable is start from 0x08048000, and then the data and bss section for global variable (32M):
0804a000-0a04a000 rw-p 0804a000 00:00 0

And then the heap starts from 1/3 of the user space (0x4000000), which is 48M:
40008000-4300d000 rw-p 40008000 00:00 0

The stack begins at 0xc0000000 and grows to low address, which is 18M:
bee40000-c0000000 rw-p bee40000 00:00 0

(2) in Linux 64 bit:

The output is:
local 7fffa88b2e40, global 00600b20, heap 2ab801210010

The memory map is:
00400000-00401000 r-xp 00000000 fd:00 34996225                           /home/bliu/ut/a.out
00600000-00601000 rw-p 00000000 fd:00 34996225                           /home/bliu/ut/a.out
00601000-02601000 rw-p 00601000 00:00 0
314ec00000-314ec1c000 r-xp 00000000 fd:00 11796527                       /lib64/ld-2.5.so
314ee1b000-314ee1c000 r--p 0001b000 fd:00 11796527                       /lib64/ld-2.5.so
314ee1c000-314ee1d000 rw-p 0001c000 fd:00 11796527                       /lib64/ld-2.5.so
314f000000-314f14d000 r-xp 00000000 fd:00 11796545                       /lib64/libc-2.5.so
314f14d000-314f34d000 ---p 0014d000 fd:00 11796545                       /lib64/libc-2.5.so
314f34d000-314f351000 r--p 0014d000 fd:00 11796545                       /lib64/libc-2.5.so
314f351000-314f352000 rw-p 00151000 fd:00 11796545                       /lib64/libc-2.5.so
314f352000-314f357000 rw-p 314f352000 00:00 0
315ae00000-315ae0d000 r-xp 00000000 fd:00 11796784                       /lib64/libgcc_s-4.1.2-20080825.so.1
315ae0d000-315b00d000 ---p 0000d000 fd:00 11796784                       /lib64/libgcc_s-4.1.2-20080825.so.1
315b00d000-315b00e000 rw-p 0000d000 fd:00 11796784                       /lib64/libgcc_s-4.1.2-20080825.so.1
3cb2600000-3cb26e6000 r-xp 00000000 fd:00 18000239                       /usr/lib64/libstdc++.so.6.0.8
3cb26e6000-3cb28e5000 ---p 000e6000 fd:00 18000239                       /usr/lib64/libstdc++.so.6.0.8
3cb28e5000-3cb28eb000 r--p 000e5000 fd:00 18000239                       /usr/lib64/libstdc++.so.6.0.8
3cb28eb000-3cb28ee000 rw-p 000eb000 fd:00 18000239                       /usr/lib64/libstdc++.so.6.0.8
3cb28ee000-3cb2900000 rw-p 3cb28ee000 00:00 0
3ecfe00000-3ecfe82000 r-xp 00000000 fd:00 35454980                       /lib64/libm-2.5.so
3ecfe82000-3ed0081000 ---p 00082000 fd:00 35454980                       /lib64/libm-2.5.so
3ed0081000-3ed0082000 r--p 00081000 fd:00 35454980                       /lib64/libm-2.5.so
3ed0082000-3ed0083000 rw-p 00082000 fd:00 35454980                       /lib64/libm-2.5.so
2ab8011f6000-2ab8011f7000 rw-p 2ab8011f6000 00:00 0
2ab80120d000-2ab804213000 rw-p 2ab80120d000 00:00 0
7fffa88b2000-7fffa98b4000 rw-p 7ffffeffd000 00:00 0                      [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0                  [vdso]


Here the executable is start from 0x00400000, then 32M is allocated for the global bss section:
00601000-02601000 rw-p 00601000 00:00 0

After load the library at low address, the heap is started at 0x2ab80120d000, which is 48M:
2ab80120d000-2ab804213000

The stack is started at 7fffa98b4000, which is 16M:
7fffa88b2000-7fffa98b4000


Then VDSO(Virtual dynamic shared object):
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0                  [vdso]
the vdso mapped to linux-gate.so.1, which is a virtual shared object exposed by kernel. It is used to make virtual system call using sysenter/sysexit, and this is faster than regular interruption.

understand memory usage in Linux

Linux tries to optimize the memory management in different ways, which make it is hard to predicate the real memory usage:
(1) Binaries (including the libraries) are "demanded paged", only part of the library actually used by a process will be loaded from disk.
(2) Two different processes may share the same loaded library.
(3) The writable page will use COW(copy on write). So if you spawn a process, it will not allocate the memory for the sub-process until you try to write that page. Then a private copy of that page will be allocated.

If you run top in Linux:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                     
 3528 xxx      15   0  584m 233m 8752 S  0.7 11.6  32:40.23 gnome-terminal  

Here the virtual set size is 584M (which include all code, data, shared library and swap out pages) and resident set size is 233M (the non-swap physical memory used). Well, we know that also includes some libraries. You can factor them out by print out memory map:
%pmap -d 3528
... ...
000000000d046000  272004 rw--- 000000000d046000 000:00000   [ anon ]
... ....
mapped: 606660K    writeable/private: 284196K    shared: 620K
The biggest one is 272M which is anonymous memory. What is the hell of anonymous memory:

Let's look at smap instead.
% cat /proc/3528/smaps
...
0d046000-1d9e7000 rw-p 0d046000 00:00 0                                  [heap]
Size:            272004 kB
Rss:             229712 kB
Shared_Clean:         0 kB
Shared_Dirty:         0 kB
Private_Clean:     4928 kB
Private_Dirty:   224784 kB
Swap:    40520 kB

Well, the heap is using 270M ... if you are using 2.6.25 kernel, you can use better /proc/$PID/pagemaps as well.

There are a few other things can contribute to the anonymous memory, like thread stack and mmap.

linux file soft link vs. hard link

Hard link uses different inode and cannot cross devices. Soft link store the file name as its data.

So if you have two files hard linked, even if you delete the origianl file, you can still access it using linked file name.

On the other hand, if you have a soft link to another file. If you deleted the origianl file, you can not access it even if the origianl file has some hard link to other files.

One more thing: If the file is opened by other process and you deleted the file, the file will be still kept in the hard driver until you close it.

self pipe trick

Our old red hat Linux kernel does not support pselect(), so we have to use self pipe trick to handle it. In a select loop, we have to monitor both the file descriptors and signal. Whenever one of them happens, it should not block. For example,
while (1)
{
if (shutdownSignal)
     //do shutdown
<===== The shutdown signal may come here
select (socketfd);
}
When no traffic sent through, and the select may block here until somebody send some data.

Or you can use  siglongjmp, which will jump out from the signal handler even if you are inside of the select() call.

summary of the assemble code map in gcc

The gcc function call stack layout looks like this (from high address to low address):
Grand Parent ebp ----> Parameters ----> parent return address ---> parent ebp ---> Local variables

Assume that you will call
m=foo("abcdef")
And it will be disassembled as

a) push 0x422dfsf ; This is the address store "abcdef"
b) call foo; when this is called, the parent instruction address in c) will be on the stack
c) add 0x4 %esp; pop up the parameter pushed into after call
e) mov %eax, 0xfffffffc(%ebp); the return value is in the %eax, and it will be assigned to the local variable

In foo() function
a) push %ebp; save parent ebp
b) move %esp, %ebp; now point to new ebp
c) sub 0x4, %eps; move ebps to allocate the space for local variables.

...
d) move  0xfffffffc(%ebp), %eax; store the function return result into the %eax
e) leave
f) ret; change the eps to point to the parameters at 8(%ebp), set eip (instruction pointer) to parent return address 4(%ebp), set ebp to parent ebp.  

GDB cheat sheet

Here is the cheat sheet for GDB in Linux

• Enable the core file

% ulimit -a
% ulimit -c 500000

Set maximum core file size to 500K.

• List all  proc information and function list

(gdb) info proc
(gdb) info functions

• Disasseeble

(gdb) disass main

• Display and clear breakpoints

(gdb) info breakpoints
(gdb) delete breakpoints 1

• Watching it live

(gdb) break 188
(gdb) commands 1
print variable
continue
end

Note: break at 188 lines and then set a group of commands at break point 1.

• Break when a specific address is accessed. Refer to this article.

(gdb)  watch *((int*)0xABCDEF)

• Define in macro and alias in ~/.gdbinit A good article about it.

As an simple example:
define cls
shell clear
end
document cls
clears screen for gdb
end

Example Macros for STL debugging. 

• Handle Signal

(gdb) i handle
(gdb) i signal
(gdb) handle SIG32 nostop print pass

• Multiple threads

(gdb) b foo.cpp:13 thread 28 if x>lim

• Remote debugging

Server side: gdbserver:9999 -attach pid
Client side: gdb program
(gdb) handle SIGTRAP nonstop noprint pass
(gdb) target remote gdbserverip:9999

• Store history

Convenience variables:
(gdb) set $table=*table_ptr
(gdb) show conv

Checkpoint
(gdb) checkpoint
(gdb) i checkpoint
(gdb) restart checkpoint-id

• See Macros in program

g++ -gdwarf-2 -g3 a.cpp

• Why the program is stopped

(gdb) i prog

• exit a loop

(gdb) until line-no

• print, info, show

print: print value of expression
info: show things about the program being debugged
show: show things about the debugger

rpm strip out debug symbol

When I build the rpm, it always extract out the debug symbols by running:
rpm --eval %__spec_install_post
/usr/lib/rpm/brp-compress
/usr/lib/rpm/brp-strip
/usr/lib/rpm/brp-strip-static-archive
/usr/lib/rpm/brp-strip-comment-note

which is very frustrating. You can disable it by using:

%define __spec_install_port /usr/lib/rpm/brp-compress

%define debug_package %{nil} 

However, I already have a version installed in the production.

I got core file, but cannot get useful backtrace. So this time, 

I have all these debug options enabled:

-ggdb -g3 -O0 

Linux buffer cache

In Linux, when you write a file to a disk, it actually stay at cache buffer for a while (5 seconds by default in /proc/sys/vm/dirty_writeback_centisecs). The sync just issues a command to flush the cache, but when to do it still depends on the O.S. In Redhat 2.6 Kernel, you can use /proc/sys/vm/drop_caches to force the cache wrote back to the disk, but it is a system-wide settings.

If an application, like database, want to know that the file is actually written to the storage, it can use fsync system call. This system call will block until the device reported the operation completed. Another option is to open file using O_DIRECT flag, so the file buffer cache will be bypassed.

Qt3 menu in Centos 5.4

I installed the Qt3 in my Centos box:

sudo yum install qt-designer

The Qt3 designer is automatically added to my Application->Programming menu. When I click it, there is no respond. Maybe the path is wrong? So I modified the desktop file:

sudo vi /usr/share/applications/qt-designer3.desktop
sudo vi /usr/share/applications/qt-assistant3.desktop
sudo vi /usr/share/applications/qt-linguist3.desktop

by adding the path to the Exec. Now it worked like a charm.