SWAT Blog

Convert CRLF to LF line endings recursively

January 20th, 2012 by Atul Sharma

dos2unix utility converts a file from dos to unix format, i.e replaces CRLF line endings to LF line endings.

This utility can be installed on any *nix environment using apt, yum, ports, etc.

To install on Mac:

sudo port -d selfupdate
sudo port install dos2unix

To install on Debian/Ubuntu/Mint:

apt-get install dos2unix

To run this utility recursively in a directory (excluding subdirectories) cd to the directory and then run following command:

find . -name *.* -exec dos2unix {} ;

To run this utility recursively (including subdirectories) cd to the directory and then run following command:

for file in `find . -type f`; do dos2unix $file $file; done

Dissecting Executable and Linking Format (ELF)

April 1st, 2011 by Atul Sharma

ELF: Executable and Linking Format
ELF is a portable object file format defining the composition and organization of the object file. Kernel and binary loader looks at this format to know how to load this file and find various pieces of information like code, initialized data, dependencies on shared libraries etc.

Types of Object files

  1. Relocatable files – holds code and data suitable to link with other object files
  2. Executable files – suitable for execution
  3. Shared object files – holds code and data suitable to link with other relocatable object or shared objects
    Object files are created by the assembler and link editor, object files are binary representations of programs intended to execute directly on a processor. 

    File Format

    Object files participate in program linking (building a program) and program execution (running a program). For convenience and efficiency, the object file format provides parallel views of a file’s contents, reflecting the differing needs of these activities.

    • ELF Header: Holds the roadmap describing the file’s organization
      Sections: Holds code/instructions, data, symbol table, relocation information, etc.
    • Program header table: Holds information to create a process image.Program header table is an array of structures, each describing a segment or other information the system needs to prepare the program for execution. An object file segment contains one or more sections
    • Section Header table: Section is an area in the object file that contains information which is useful for linking: program’s code, program’s data (variables, array, string), relocation information and other. So, in each area, several information is grouped and it has a distinct meaning: code section only hold code, data section only holds initialized or non-initialized data, etc.
    • Segments: An object file segment contains one or more sections. Segements are of variuos types – loadable, dynamic, etc.

    Dissecting an ELF

    We will start with a small C program to dissect ELF.

    #include <stdio.h>
    #include <stdlib.h>
    
    int globalVariableAssigned = 10;
    int globalVariableUnassigned;
    
    int main(int argc, char **argv){
    	int localVariable = 100;
    	printf("ELF test Coden");
    	printf("globalVariableAssigned = %dn", globalVariableAssigned);
    	printf("globalVariableUnassigned = %dn", globalVariableUnassigned);
    	printf("localVariable = %dn", localVariable);
    	return EXIT_SUCCESS;
    }

    Compile the above program

    $ gcc -o elf.o elf.c

    Now we’ll use readelf to look into the object file elf.o in more detail.

    • Looking at the ELF Header
      $ readelf -h elf.o
      • ELF Header: Holds the roadmap describing the file’s organization
        Sections: Holds code/instructions, data, symbol table, relocation information, etc.
      • Program header table: Holds information to create a process image.Program header table is an array of structures, each describing a segment or other information the system needs to prepare the program for execution. An object file segment contains one or more sections
      • Section Header table: Section is an area in the object file that contains information which is useful for linking: program’s code, program’s data (variables, array, string), relocation information and other. So, in each area, several information is grouped and it has a distinct meaning: code section only hold code, data section only holds initialized or non-initialized data, etc.
      • Segments: An object file segment contains one or more sections. Segements are of variuos types – loadable, dynamic, etc.
        • The executable is created for Advanced Micro Devices X86-64 (Machine field)
        • The object is a relocatable object file (Type field)
        • The program has 25 sections
      • Looking at the ELF Header

        $ readelf -h elf.o

        Thue output is as follows on my 64 bit machine:

        ELF Header:
          Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
          Class:                             ELF64
          Data:                              2's complement, little endian
          Version:                           1 (current)
          OS/ABI:                            UNIX - System V
          ABI Version:                       0
          Type:                              REL (Relocatable file)
          Machine:                           Advanced Micro Devices X86-64
          Version:                           0x1
          Entry point address:               0x0
          Start of program headers:          0 (bytes into file)
          Start of section headers:          22296 (bytes into file)
          Flags:                             0x0
          Size of this header:               64 (bytes)
          Size of program headers:           0 (bytes)
          Number of program headers:         0
          Size of section headers:           64 (bytes)
          Number of section headers:         25
          Section header string table index: 22

        Few interesting things that the header highlights:

      • Listing sections
        Below is the stripped out version of the output listing all 25 sections:

        Section Headers:
          [Nr] Name              Type             Flags  Link  Info  Align
          [ 0]                   NULL             	   0	 0     0
          [ 1] .text             PROGBITS         AX       0     0     4
          [ 2] .rela.text        RELA                     23     1     8
          [ 3] .data             PROGBITS         WA       0     0     4
          [ 4] .bss              NOBITS           WA       0     0     4
          [ 5] .debug_abbrev     PROGBITS                  0     0     1
          [ 6] .debug_info       PROGBITS                  0     0     1
          [ 7] .rela.debug_info  RELA                     23     6     8
          [ 8] .debug_line       PROGBITS                  0     0     1
          [ 9] .rela.debug_line  RELA                     23     8     8
          [10] .debug_macinfo    PROGBITS                  0     0     1
          [11] .rodata           PROGBITS         A       0     0     8
          [12] .debug_loc        PROGBITS                  0     0     1
          [13] .debug_pubnames   PROGBITS                  0     0     1
          [14] .rela.debug_pubna RELA                     23    13     8
          [15] .debug_aranges    PROGBITS                  0     0     1
          [16] .rela.debug_arang RELA                     23    15     8
          [17] .debug_str        PROGBITS         MS       0     0     1
          [18] .comment          PROGBITS         MS       0     0     1
          [19] .note.GNU-stack   PROGBITS                  0     0     1
          [20] .eh_frame         PROGBITS          A       0     0     8
          [21] .rela.eh_frame    RELA                     23    20     8
          [22] .shstrtab         STRTAB                    0     0     1
          [23] .symtab           SYMTAB                   24    17     8
          [24] .strtab           STRTAB                    0     0     1

        Most notable are:

        • “.text” which has a “X” or executable flag against it.
          To find out what is contained in this section we can take an object dump of this section

          $ objdump -d -j .text elf.o

          And the output is indeed the machine code of main() function. see partial output below:

          0000000000000000 <main>:
             0:	55                   	push   %rbp
             1:	48 89 e5             	mov    %rsp,%rbp
             4:	48 83 ec 20          	sub    $0x20,%rsp
             8:	89 7d ec             	mov    %edi,-0x14(%rbp)
             b:	48 89 75 e0          	mov    %rsi,-0x20(%rbp)
             f:	c7 45 fc 64 00 00 00 	movl   $0x64,-0x4(%rbp)
            16:	bf 00 00 00 00       	mov    $0x0,%edi
          ...
          ...
            66:	e8 00 00 00 00       	callq  6b <main+0x6b>
            6b:	b8 00 00 00 00       	mov    $0x0,%eax
            70:	c9                   	leaveq
            71:	c3                   	retq   
        • “.data” section hold all the initialized global variables inside the program that do not live inside the stack.
          To find out what is contained in this section we can take an object dump of this section

          $ objdump -d -j .data elf.o

          And output shows one assigned variable, which actually is variable globalVariableAssigned in the test program

          Disassembly of section .data:
          
          0000000000000000 <globalVariableAssigned>:
             0:	0a 00 00 00                                         ....
        • “.bss” – BSS (Block Started by Symbol) is a section where all unitialized variables are mapped. You can find out the content of this section using objdump. I am not going to to save some formatting on the blog. :)

SAMBA PDC SERVER ON UBUNTU 10.04

March 25th, 2011 by Bhagwan Dass

Primary Domain Controller
First, install Samba,

[root@linux10 ~]#apt-get install samba

Configure Samba by editing /etc/samba/smb.conf.
My smb.conf file is shown below. Make changes according to your requirement.

[root@linux10 ~]#vi /etc/samba/smb.conf

#======================= Global Settings =======================

[global]

## Browsing/Identification ###

# Change this to the workgroup/NT-domain name your Samba server will part of
workgroup = TVIN

# server string is the equivalent of the NT Description field
server string = %h server (Samba, Ubuntu)

# Windows Internet Name Serving Support Section:
# WINS Support – Tells the NMBD component of Samba to enable its WINS Server
# wins support = no

# WINS Server – Tells the NMBD components of Samba to be a WINS Client
# Note: Samba can be either a WINS Server, or a WINS Client, but NOT both
; wins server = w.x.y.z

# This will prevent nmbd to search for NetBIOS names through DNS.
dns proxy = no

# What naming service and in what order should we use to resolve host names
# to IP addresses
; name resolve order = lmhosts host wins bcast

#### Networking ####

# The specific set of interfaces / networks to bind to
# This can be either the interface name or an IP address/netmask;
# interface names are normally preferred
; interfaces = 127.0.0.0/8 eth0

# Only bind to the named interfaces and/or networks; you must use the
# ‘interfaces’ option above to use this.
# It is recommended that you enable this feature if your Samba machine is
# not protected by a firewall or is a firewall itself. However, this
# option cannot handle dynamic or non-broadcast interfaces correctly.
; bind interfaces only = yes

#### Debugging/Accounting ####

# This tells Samba to use a separate log file for each machine
# that connects
log file = /var/log/samba/log.%m

# Cap the size of the individual log files (in KiB).
max log size = 1000

# If you want Samba to only log through syslog then set the following
# parameter to ‘yes’.
# syslog only = no

# We want Samba to log a minimum amount of information to syslog. Everything
# should go to /var/log/samba/log.{smbd,nmbd} instead. If you want to log
# through syslog you should set the following parameter to something higher.
syslog = 0

# Do something sensible when Samba crashes: mail the admin a backtrace
panic action = /usr/share/samba/panic-action %d

####### Authentication #######

# “security = user” is always a good idea. This will require a Unix account
# in this server for every user accessing the server. See
# /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html
# in the samba-doc package for details.
security = user

# You may wish to use password encryption. See the section on
# ‘encrypt passwords’ in the smb.conf(5) manpage before enabling.
encrypt passwords = true

# If you are using encrypted passwords, Samba will need to know what
# password database type you are using.
passdb backend = tdbsam

obey pam restrictions = yes

# This boolean parameter controls whether Samba attempts to sync the Unix
# password with the SMB password when the encrypted SMB password in the
# passdb is changed.
unix password sync = yes

# For Unix password sync to work on a Debian GNU/Linux system, the following
# parameters must be set (thanks to Ian Kahan < for
# sending the correct chat script for the passwd program in Debian Sarge).
passwd program = /usr/bin/passwd %u
passwd chat = *Entersnews*spassword:* %nn *Retypesnews*spassword:* %nn *passwordsupdatedssuccessfully* .

# This boolean controls whether PAM will be used for password changes
# when requested by an SMB client instead of the program listed in
# ‘passwd program’. The default is ‘no’.
pam password change = yes

# This option controls how unsuccessful authentication attempts are mapped
# to anonymous connections
map to guest = bad user

########## Domains ###########

# Is this machine able to authenticate users. Both PDC and BDC
# must have this setting enabled. If you are the BDC you must
# change the ‘domain master’ setting to no
#
domain logons = yes
domain master = yes
local master = yes
preferred master = yes
os level = 64

#
# The following setting only takes effect if ‘domain logons’ is set
# It specifies the location of the user’s profile directory
# from the client point of view)
# The following required a [profiles] share to be setup on the
# samba server (see below)
; logon path = \%Nprofiles%U
# Another common choice is storing the profile in the user’s home directory
# (this is Samba’s default)
logon path = \%N%Uprofile

# The following setting only takes effect if ‘domain logons’ is set
# It specifies the location of a user’s home directory (from the client
# point of view)
logon drive = H:
logon home = \%N%U

# The following setting only takes effect if ‘domain logons’ is set
# It specifies the script to run during logon. The script must be stored
# in the [netlogon] share
# NOTE: Must be store in ‘DOS’ file format convention
; logon script = logon.cmd

# This allows Unix users to be created on the domain controller via the SAMR
# RPC pipe. The example command creates a user account with a disabled Unix
# password; please adapt to your needs
; add user script = /usr/sbin/adduser –quiet –disabled-password –gecos “” %u

# This allows machine accounts to be created on the domain controller via the
# SAMR RPC pipe.
# The following assumes a “machines” group exists on the system
add machine script = /usr/sbin/useradd -g machines -c “%u machine account” -d /var/lib/samba -s /bin/false %u

# This allows Unix groups to be created on the domain controller via the SAMR
# RPC pipe.
; add group script = /usr/sbin/addgroup –force-badname %g

########## Printing ##########

# If you want to automatically load your printer list rather
# than setting them up individually then you’ll need this
# load printers = yes

# lpr(ng) printing. You may wish to override the location of the
# printcap file
; printing = bsd
; printcap name = /etc/printcap

# CUPS printing. See also the cupsaddsmb(8) manpage in the
# cupsys-client package.
; printing = cups
; printcap name = cups

############ Misc ############

# Using the following line enables you to customise your configuration
# on a per machine basis. The %m gets replaced with the netbios name
# of the machine that is connecting
; include = /home/samba/etc/smb.conf.%m

# Most people will find that this option gives better performance.
# See smb.conf(5) and /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/speed.html
# for details
# You may want to add the following on a Linux system:
# SO_RCVBUF=8192 SO_SNDBUF=8192
# socket options = TCP_NODELAY

# The following parameter is useful only if you have the linpopup package
# installed. The samba maintainer and the linpopup maintainer are
# working to ease installation and configuration of linpopup and samba.
; message command = /bin/sh -c ‘/usr/bin/linpopup “%f” “%m” %s; rm %s’ &

# Domain Master specifies Samba to be the Domain Master Browser. If this
# machine will be configured as a BDC (a secondary logon server), you
# must set this to ‘no'; otherwise, the default behavior is recommended.
# domain master = auto

# Some defaults for winbind (make sure you’re not using the ranges
# for something else.)
; idmap uid = 10000-20000
; idmap gid = 10000-20000
; template shell = /bin/bash

# The following was the default behaviour in sarge,
# but samba upstream reverted the default because it might induce
# performance issues in large organizations.
# See Debian bug #368251 for some of the consequences of *not*
# having this setting and smb.conf(5) for details.
; winbind enum groups = yes
; winbind enum users = yes

# Setup usershare options to enable non-root users to share folders
# with the net usershare command.

# Maximum number of usershare. 0 (default) means that usershare is disabled.
; usershare max shares = 100

# Allow users who’ve been granted usershare privileges to create
# public shares, not just authenticated ones
usershare allow guests = yes

#======================= Share Definitions =======================

# Un-comment the following (and tweak the other settings below to suit)
# to enable the default home directory shares. This will share each
# user’s home directory as \serverusername
[homes]
comment = Home Directories
browseable = no
writeable = yes

# By default, the home directories are exported read-only. Change the
# next parameter to ‘no’ if you want to be able to write to them.
; read only = yes

# File creation mask is set to 0700 for security reasons. If you want to
# create files with group=rw permissions, set next parameter to 0775.
; create mask = 0700

# Directory creation mask is set to 0700 for security reasons. If you want to
# create dirs. with group=rw permissions, set next parameter to 0775.
; directory mask = 0700

# By default, \serverusername shares can be connected to by anyone
# with access to the samba server. Un-comment the following parameter
# to make sure that only “username” can connect to \serverusername
# This might need tweaking when using external authentication schemes
; valid users = %S

# Un-comment the following and create the netlogon directory for Domain Logons
# (you need to configure Samba to act as a domain controller too.)
[netlogon]
comment = Network Logon Service
path = /home/samba/netlogon
guest ok = yes
read only = yes
share modes = no

# Un-comment the following and create the profiles directory to store
# users profiles (see the “logon path” option above)
# (you need to configure Samba to act as a domain controller too.)
# The path below should be writable by all users so that their
# profile directory may be created the first time they log on
[profiles]
comment = Users profiles
path = /home/samba/profiles
guest ok = no
browseable = no
create mask = 0600
directory mask = 0700

[printers]
comment = All Printers
browseable = no
path = /var/spool/samba
printable = yes
guest ok = no
read only = yes
create mask = 0700

# Windows clients look for this share name as a source of downloadable
# printer drivers
[print$]
comment = Printer Drivers
path = /var/lib/samba/printers
browseable = yes
read only = yes
guest ok = no
# Uncomment to allow remote administration of Windows print drivers.
# You may need to replace ‘lpadmin’ with the name of the group your
# admin users are members of.
# Please note that you also need to set appropriate Unix permissions
# to the drivers directory for these users to have write rights in it
; write list = root, @lpadmin

# A sample share for sharing your CD-ROM with others.
;[cdrom]
; comment = Samba server’s CD-ROM
; read only = yes
; locking = no
; path = /cdrom
; guest ok = yes

# The next two parameters show how to auto-mount a CD-ROM when the
# cdrom share is accesed. For this to work /etc/fstab must contain
# an entry like this:
#
# /dev/scd0 /cdrom iso9660 defaults,noauto,ro,user 0 0
#
# The CD-ROM gets unmounted automatically after the connection to the
#
# If you don’t want to use auto-mounting/unmounting make sure the CD
# is mounted on /cdrom
#
; preexec = /bin/mount /cdrom
; postexec = /bin/umount /cdrom

Save the file.
Then create a group named machines.

[root@linux10 ~]#groupadd -g 201 machines

Create the netlogon directory for Domain Logons.

[root@linux10 ~]#mkdir -m 0775 /home/netlogon

Create the profiles directory to store users profiles.

[root@linux10 ~]#mkdir /home/samba /home/samba/profiles

Set permission on profiles directory.

[root@linux10 ~]#chmod 1757 /home/samba/profiles

Create users, passwords and smbpasswords.

[root@linux10 ~]#useradd -m tactical
[root@linux10 ~]#passwd tactical
[root@linux10 ~]#smbpasswd -a tactical
[root@linux10 ~]#useradd -m swat
[root@linux10 ~]#passwd swat
[root@linux10 ~]#smbpasswd -a swat

Set root password for samba authentication while matchine joininig the domain.

[root@linux10 ~]#smbpasswd -a root

Final Step to Restart Samba Service.

[root@linux10 ~]#service smbd restart
[root@linux10 ~]#service nmbd restart

Samba Primary Domain Controller is ready to accept client requests.
Joining a Windows machine to Samba PDC.
1.Right Click on Mycomputer.
2.Go to Properties select the Computer Name tab.
3.Click Change.
4.Click Domain radio button, enter the Domain name like TVIN, then click ok.
5.At the prompt, enter the user name root and root smbpassword and then click OK.
6.Click OK twice to return to the System Properties dialog box.
7.Click OK, and then click Yes to restart the computer.

Windows xp is ready to login with samba user and make sure you should use smbpassword while user login.

Process Address Space – Code, gvar, BSS, Heap & Stack

March 23rd, 2011 by Atul Sharma

Almost all modern operating systems implement Memory Protection to protect access to a private address allocated to a process by other processes to avoid storage violation. this private space is called Process Address Space.

Process Address Space partitioned into various memory areas or segments based on the functional differences.

Text/Code Segment

This segment, also known as the code segment, holds the executable instructions of a program.

  • execute-only
  • fixed size

Data Segment

The data area contains global and static variables used by the process that are initialized. This segment can be further classified into initialized read-only area and initialized read-write area.

  • Gvar Section
    A global variable that is initialized and stored in the data segment. This section has read/write attributes but cannot be shared among processes running the same program.
  • BSS Section
    This section holds uninitialized data. This data consists of global variables that the system initializes with 0s upon program execution. Another name for this section is the zero-initialized data section.
  • Heap Section
    This is used to grow the linear address space of a process. When a program uses malloc() to obtain dynamic memory, this memory is placed in the heap.The heap area begins at the end of the BSS segment and “grows up” to larger addresses from there. The Heap area is shared by all shared libraries and dynamically loaded modules in a process.

    • read/write
    • variable size
    • dynamic allocation by request

Stack Segment

This contains all the local variables that get allocated. When a function is called, the local variables for that function are pushed onto the stack. As soon as a function ends, the variables associated with the function are popped from the stack. Other information, including return addresses and parameters, is also stored in the stack. The stack is a LIFO structure, typically located in the higher parts of memory. It usually “grows down” with every register, immediate value or stack frame being added to it.

  • read/write, variable size
  • automatic growth/shrinkage

popen() – execute shell command from C/C++

March 23rd, 2011 by Atul Sharma

The popen() function executes a command and pipes the executes command to the calling program, returning a pointer to the stream which can be used by calling program to read/write to the pipe.
Below are the C/C++ snippets to run a simple command, read the stream from pipe and then write to console.

C Implementation

#include <stdio.h>

int main(void) {
	FILE *in;
	extern FILE *popen();
	char buff[512];

	if(!(in = popen("ls -sail", "r"))){
		exit(1);
	}

	while(fgets(buff, sizeof(buff), in)!=NULL){
		printf("%s", buff);
	}
	pclose(in);

}

C++ Implementation

#include <iostream>
#include <stdio.h>

using namespace std;

int main() {
	FILE *in;
	char buff[512];

	if(!(in = popen("ls -sail", "r"))){
		return 1;
	}

	while(fgets(buff, sizeof(buff), in)!=NULL){
		cout << buff;
	}
	pclose(in);

	return 0;
}

popen() is included in SUS version 2. More details can be found at http://pubs.opengroup.org/onlinepubs/007908799/xsh/popen.html

SQL Server Comma Delimited to table

March 1st, 2011 by Shailendra Parate
One of the frequently asked questions in a lot of SQL Server forums is how to handle a comma-delimited value passed as a parameter in a stored procedure.
To better illustrate the question, let’s say you have a website wherein you have a page that lets the users select data items which are sent back to the database for further processing in variety of formats, and of all the formats, comma separated is probably the format which developers struggle with most if they do not know how to consume this structure.

Databases do not understand data arranged horizontally, so how to convert the data from CSV (horizontal) to a table (vertical) ?

Fortunately there are a couple of tricks up database’s sleeve.

1. Dynamic SQL Approach
This method involves creating a SQL statement during execution time to evaluate the where condition. But this approach is as ugly as it sounds nasty. Creating SQL statments on the the fly would results in lot of activity in shared pool. i.e. Compilation of statement, execution plan generation and final execution of query. All of these consume precious server resources and hence should avoided if at all possible.
Due to the obvious pit-falls in this approach I am not even going to attempt to show how-to.
2. Common Table Expression (CTE) based Approach
Comman table expressions (CTE) is probably one of the most powerful programming construct introduced in the SQL Server 2005. It can be used in hierarchical queries, special aggregations and various other problem solving. Various uses of CTE are discussed elsewhere in my blog, which can be found using the search button on the blog homepage.
Below is a CTE based implementation. The good aspect about this approach is that it is completely object independent so the function can be used for any table in any schema of the database as long as you have sufficient permissions.

USE [AdventureWorks]
GO
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N‘[dbo].[ufnCSVToTable]’)
AND type in (N‘FN’, N‘IF’, N‘TF’, N‘FS’, N‘FT’))
DROP FUNCTION [dbo].[ufnCSVToTable]
GO
CREATE FUNCTION [dbo].[ufnCSVToTable](@str VARCHAR(4000), @sep CHAR(1))
RETURNS @table TABLE
( [colx] VARCHAR(4000) NULL )
AS
BEGIN
WITH r0 AS
(
SELECT 1 n,1 m
UNION ALL
SELECT n+1, m+ CASE WHEN SUBSTRING(@str,n,1)=@sep THEN 1 ELSE 0 END
FROM r0 WHERE n<=LEN(@str)
)
INSERT INTO @table
SELECT SUBSTRING(@str,MIN(n), MAX(n)-MIN(n)) colx
FROM r0 GROUP BY m;
RETURN;
END;
GO
–Independent usage
SELECT * FROM [dbo].[ufnCSVToTable](this,is,a,comma,separated,sentence,‘,’)
Output:
colx
this
is
a
comma
separated
sentence


–Usage with table
DECLARE @str VARCHAR(1000); SET @str = ‘1,2’;
SELECT *
FROM Person.Address Ad
INNER JOIN [dbo].[ufnCSVToTable](@str,‘,’) CSV ON Ad.AddressID = CSV.colx

Voila!

I hope you find this function useful.

Linux kernel development and debugging using Eclipse CDT

February 11th, 2011 by Atul Sharma

This blog entry is inspired by Takis blog . This entry can be seen as the revised version of Takis blog entry with consideration to latest Eclipse and Linux Kernel versions.

Prerequisites

The steps below would work on Debian, Ubuntu and Mint Linux. I have used Linux Mint.

You need latest version of Eclipse IDE for C/C++ Developers (I have used Helios). You can download from http://www.eclipse.org/downloads/

For rest of this guide you’d need several packages that you can install using following command


apt-get install git-core kernel-package fakeroot build-essential libncurses5 libncurses5-dev qemu kvm crash kexec-tools makedumpfile kernel-wedge libelf-dev asciidoc binutils-dev debootstrap genext2fs e2fsprogs

STEP 1: Get the Kernel source using git.

Run following command to clone the kernel source into your home folder:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git ~/linux-2.6

STEP 2: Create a build directory

Create build directory in /mnt directory as its the best place to keep any mountable file system.

mkdir -p /mnt/build/linux-2.6

STEP 3: Configure kernel

First run make oldconfig, which will read the defaults from an existing .config and rewrite necessary links and files.

cd ~/linux-2.6
yes "" | make oldconfig

Now make changes to default configure to enable debugging options using menuconfig. You can also use xconfig if you’re using X.

make menuconfig O=/mnt/build/linux-2.6

Select “Kernel hacking” menu


Enable “Compile the kernel with debug info”


Enable “Compile the kernel with frame pointers”

STEP 4: Disable automatic build in Eclipse

Open Eclipse with CDT plugin

Kernel code is huge and we wouldn’t want to enable automatic building after every change.  Go to “Window->Preferences” menu, Select “General->Workspace” and deselect “Build automatically” checkbox

STEP 5: Add kernel project in Eclipse

Create a new project. Go to  File->New->Project then select C Project.

Click Next

Enter project name – I have used linux2.6

Uncheck “Use Default Location” and select the linux source directory (~/linux2.6 in my case)

Select “Empty Project” under “Makefile project” as Project Type

Select Linux GCC as Toolchain

Click Next

Keep default values in Select Configurations window

Click Finish

Now eclipse will import the kernel source into the project and index it. This will take a while. Once complete all progress bars will disappear and the UI will look similar to below:

STEP 6: Configure build properties in Eclipse

Right click on Project and click on “Properties” from context menu.

Select “C/C++ Build” in properties window.

Uncheck “Use default build command”

Set build command as – make  O=/mnt/build/linux-2.6

Set build directory as – /mnt/build/linux-2.6

Now the project is ready to build. Go to the menu-bar select “Project->Build all”.

If all goes well then after some time the Linux kernel build will be completed.

STEP 7: Configure debug settings in Eclipse

Right click on Project. Click on “Debug As ..” >> “Debug Configurations”

Double Click on “C/C++ Attach to Application”

Select vmlinux binary (in my case /mnt/build/linux-2.6/vmlinux) in C/C++ Application field on Main tab

Click on Debugger tab

Select GDB Server in Debugger option under main tab

Click on Connection tab

Select TCP in Connection “Type” field

Enter localhost in “Host name or IP address” field

Enter 1234 in “Port Number” field

STEP 8: Prepare a root filesystem for testing kernel

You’d need a root filesystem in order to boot and test the kernel. I prefer not using filesystem of my host machine. Here is a recipe to build a basic 8GB root file system


cd /mnt
sudo debootstrap sid sid

echo -e "rootnroot" | chroot sid passwd
echo -e "auto loniface lo inet loopbacknauto eth0niface eth0 inet dhcp" > sid/etc/network/interfaces
ln -sf vimrc sid/etc/vimrc.tiny
rm -f sid/etc/udev/rules.d/70-persistent-net.rules
echo atul > sid/etc/hostname
echo cgroup /mnt/cgroup cgroup defaults >> sid/etc/fstab
mkdir -p sid/mnt/cgroup

BLOCKS=$(((1024*$(du -m -s sid | awk '{print $1}')*12)/10))
genext2fs -z -d sid -b $BLOCKS -i 1024 sid.ext3
resize2fs sid.ext3 8G
tune2fs -j -c 0 -i 0 sid.ext3
rm -rf sid

Notice I have setup root password as “root”

STEP 9: Run QEMU to launch the kernel build

Use following command to startup QEMU


qemu -no-kvm -s -S  -kernel /mnt/build/linux-2.6/arch/x86/boot/bzImage -hda /mnt/sid.ext3 -append "root=/dev/sda"

In case you’re using 64 bit host the use following command


qemu-system-x86_64 -no-kvm -s -S  -kernel /mnt/build/linux-2.6/arch/x86/boot/bzImage -hda /mnt/sid.ext3 -append "root=/dev/sda"

At this point you’ll see a blank QEMU terminal window. The “-s” option is a shorthand for -gdb tcp::1234, i.e. open a gdbserver on TCP port
1234. The “-S” option stops the CPU to start at startup. Now QEMU is waiting for kernel to start in debug mode
STEP 10: Start Kernel in Debug mode
From the menu bar in Eclipse click on debug icon
Then click on Linux2.6 Default debug configuration to run from the drop down list.
Launching might take a while, you can see the progress a the bottom right progress bar

Once the application is ready to launch it will prompt you to open the debugging perspective. Click on Yes.

Now the Eclipse perspective will change form development to Debug

Click “Run > Step over” or press the “F6″ key to execute the kernel code line by line and examine what’s happening.
Or click “Run > Resume” or press the “F8″ key to execute the kernel code without breaks.

You can find several articles on CDT at http://www.eclipse.org/cdt/documentation.php

UPDATED:
Here is the updated URL to Taki’s blog – http://issaris.org/blog/2007/12/14/debugging-the-linux-kernel-using-eclipsecdt-and-qemu/

PostgreSQL Databases Backup & Restore with pg_dump command

February 11th, 2011 by Bhagwan Dass

Make sure you should conncete to your server via ssh command line.
Login as super user type the following command.

[root@linux10 ~]#su – postgres

Get list of database(s) to backup.

[postgres@linux10 ~$]#psql -l

List of databases

Name | Owner | Encoding | Collation | Ctype | Access privileges

———–+———-+———-+————-+————-+———————–

postgres | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 |

mine | mine | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 |

template0| postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres

:postgres=CTc/postgres

template1 | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres

: postgres=CTc/postgres

(4 rows)

~

~

(END)

Make a backup using pg_dump, pg_dump is a utility for backing up a PostgreSQL database. It dumps only one database at a time.

Dump a mine database:-

[postgres@linux10 ~$]#pg_dump mine > /opt/mine.out

To restore a mine database:-

[postgres@linux10 ~$]#psql -d mine -f mine.out

Or, If you have new server first create database then apply command:-

[postgres@linux10 ~$]#createdb mine
[postgres@linux10 ~$]#psql mine
[postgres@linux10 ~$]#psql -d mine -f mine.out

Second option is use to pg_dumpall command.It dumps (backs up) each database, and preserves cluster-wide data such as users and groups. You can use it as follows:-

[postgres@linux10 ~$]#pg_dumpall > /opt/all.db.out

To restore backup use the following command:

[postgres@linux10 ~$]#psql -f all.db.out postgres

Subversion Installation on Ubuntu 10.04

February 8th, 2011 by Bhagwan Dass

Make sure you should login as root:–

[root@linux10 ~]#apt-get install apache2
[root@linux10 ~]#apt-get install subversion libapache2-svn

The next command is configure the user access for subversion.

[root@linux10 ~]#htpasswd -c /etc/subversion/passwd username

Where username is a new user. All subsequent users are created by doing the following

[root@linux10 ~]#htpasswd /etc/subversion/passwd username

It is important to note if you have the ‘-c’ command it WILL overwrite the existing user file. So run this command only first time.
Now you need to decide where you want to put your repositories. I suggest you put them all under one root path, for example /srv/svn/yourproject. For each repository you want create a new folder in the svn root folder and then run the following command to turn it into a repository.

[root@linux10 ~]#mkdir /srv/svn/
[root@linux10 ~]#mkdir /srv/svn/yourproject/
[root@linux10 ~]#svnadmin create /srv/svn/yourproject

We are going to use apache to serve the repositories so we need to make the svn root folder and all subsequent child files and folders accessable by apache. This is done by using the following command

[root@linux10 ~]#chown -R www-data:www-data /srv/svn/yourproject

The final part to getting repositories to be accessed from a web address add the following code to your sites apache config file located in /etc/apache2/mods-enabled/dav_svn.conf

[root@linux10 ~]#vi /etc/apache2/mods-enabled/dav_svn.conf

DAV svn
SVNPath /srv/svn/yourproject
AuthType Basic
AuthName “yourproject Repository”
AuthUserFile /etc/subversion/passwd
Require valid-user

The last thing to do is restart apache after saving the file

[root@linux10 ~]#/etc/init.d/apache2 restart

After browsing your project, If you have promte for user name and password that means you have done successfully.
http://ip-address/svn/yourproject/

Show Chinese characters on JSP pages using property files in web application

February 4th, 2011 by Vikash Trivedi

Below is procedure to show Chinese characters on JSP pages using property files in web application.

  1. Create new property file for Chinese language of extension _zh.properties like ABC_zh.properties.
  2. Open the newly created property file in UTF-8 enabled editor like notepad or Edit+.
  3. Put the relevant key values in Chinese language. Like. customer.number=客户号码
  4. After setting all key values in property file, save file as encoding as UTF-8.
  5. Use tool native2ascii from JDK bin directory to convert Chinese characters to Unicode using following command by ensuring the encoding to UTF-8.
  6. Convert native to ASCII using following command:
    $JAVA_JOMEbin>native2ascii.exe -encoding UTF-8 ABC_zh.properties ABC_zh_1.properties
  7. Copy newly created ABC_zh_1.properties to original location of property files and renamed to ABC_zh.properties
  8. Add the following line in starting the weblogic server like
    -Dfile.encoding=UTF-8

    i.e.

  9. Add following java options:
    set JAVA_OPTIONS=%JAVA_OPTIONS% %JAVA_PROPERTIES% -Dwlw.iterativeDev=%iterativeDevFlag% -Dwlw.testConsole=%testConsoleFlag% - Dwlw.logErrorsToConsole=%logErrorsToConsoleFlag% -Dfile.encoding=UTF-8
  10. Add the content type as UTF-8 in displaying JSP pages by adding following line at start of JSP file.
  11. Thats It !!!
  • © 2004-2015 Special Work & Technology Limited