Truth, Computing and Fail

So much for late night coding. Yesterday I missed out on a very important part about setting the hard limits on resources. But you need to have a superuser process to achieve this. Referring to the code of exec.c in the previous post, put in this after line no. 18.

limit.rlim_max =1;

This is actually the hard limit and as the man page says, it acts as a ceiling for the soft limit i.e. rlim_cur. The advantage being that if the process exceeds the limit and yet continues running such as by handling the SIGXCPU signal in the previous example, this time a SIGKILL will be issued which would force it to terminate.

Now compile exec.c as usual and run it as root. You won’t see any output as we set the hard limit equal to the soft limit.

• Soft limit reached, SIGXCPU sent
• Process tries to handle it
• At the same time the hard limit i.e. the ceiling for soft limit is reached too. So a SIGKILL is sent and the process terminates before any output.

References

setrlimit(2) man page

This might seem trivial at first sight but you have no idea how darn happy I’m to see this works . So happy that I decided to blog about it. I already hear trumpets blowing in the distance.

Here is what I was worried about:

Normally, exec() is carried out in a forked process. Setting resource limits on a forked process is straightforward using setrlimit(). But I was apprehensive whether those limits would hold once I exec() another binary image within that forked process, and it wouldn’t be overstating it if I said I was about to front kick my laptop in joy .

I tested by limiting the amount of CPU time(RLIMIT_CPU) that a running process is allocated since this would be the easiest to test IMO. The signal delivered to the process when it exceeds the CPU time limit is SIGXCPU.

Since this is all that is to it, wasting no more words, let’s get our hands dirty.

The program in which a process is forked and exec() is run (exec.c):

#include <stdio.h>
#include <stdlib.h>
#include <sys/resource.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>

int main()
{
    int pid;
    int rv;

    if (!( pid=fork() ))
    {
        struct rlimit limit;
        getrlimit(RLIMIT_CPU, &limit);

        limit.rlim_cur = 1;

        setrlimit(RLIMIT_CPU, &limit);

        execl("./inf","inf",NULL);
    }
    else if(pid)
    {
        wait(&rv);
    }
    else
        printf("Error forking\n");

    return 0;
}

The program whose binary image is specified in exec() (inf.c):

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>

void handler(int signum)
{
    if (signum == SIGXCPU)
        printf("Caught SIGXCPU signal, exec in peace\n");
}

int main()
{
signal(SIGXCPU, handler);
while(1);
return 0;
}

Now compile
$gcc -Wall exec.c -o exec
$gcc -Wall inf.c -o inf

and run
$./exec

You’ll see SIGXCPU being handled which means the resource limit is working!

I’ve spent the past few weeks changing the look and feel of my Vim environment. One fine day I decided to explore some :colorscheme while I was on GVim apart from the classic ‘slate’ that I always use.

Now, now. I can already see purists scoffing at me for using GVim. The main reason for me using GVim is just for the occasional visual treat. While you are on the terminal and working on Vim, you don’t really have much choice of beautifying things within the editor itself. You rather have to make some changes to the terminal profile about the background/font color, font to be used and all that stuff.

Coming back to where I started off, I googled and found a few color schemes. This caught my eye because it claimed to be The last vim color scheme you’ll ever need. Apparently this was supposed to give GVim a TextMate kinda look but applying the colorscheme is only the first step. You need to set the proper fonts and background too. I suggest you checkout the vimrc and gvimrc of the author of the linked post.

After all was done, I didn’t find the final outcome to be that impressive. In addition to GVim being buggy on occasions like no input displayed in the command mode, this time it would get stuck after I’d scroll down or up about 20 lines for like a second. Besides, it wasn’t looking too great. See for yourself.

Now it was time to make some changes to gnome-terminal and see if it suited my tastes. As usual none of set backgorund=dark and colorscheme ir_black showed the necessary changes within Vim. This part is easy. Just make a new terminal profile with the following options:

• White on black for foreground/background
• Font:Monaco with size 12

After this just add the following lines to your .vimrc

set background=dark
colorscheme ir_black


Switch to the new profile, open up Vim and see the difference in the richness of the colors and better font rendering.

Bonus Tip

If you are a python coder, you might want to look into this post by Samuel Huckins on making Vim a complete IDE for python development. I’ve been using the NERDTree and code folding plugins mentioned there and they have been of really great help.

I tried this thing last semester too but I wasn’t too serious about it. I had decided to go for gentoo for obvious benefits that’d support the frequent rebuilding of the kernel. Somewhere down the line gentoo got caught in a cyclic dependency error and I forgot about the whole thing. But I am digressing.

Anyway, I built gentoo from scratch and got things working. This step by step guide is quite good to get started. Note that this is about adding system calls to the kernel, not implementing them.

The guide is a bit old though, and just one thing needs to be changed. Step #16 mentions the use of the _syscallN macro. Don’t use it. From the man page of _syscall

NAME

_syscall – invoking a system call without library support (OBSOLETE)

NOTES

Starting around kernel 2.6.18, the _syscall macros were removed from header files supplied to user space. Use syscall(2) instead.

The _syscall() macros do not produce a prototype. You may have to create one, especially for C++ users.

Instead use a function wrapper like this:

long mycall(int i, int * result)

{

	return syscall(__NR_mycall, i, result);

}

i and result are the arguments I used for my syscall and quite obviously it would vary according to whatever you decide to write.

There are quite a few other guides too on this topic but they are generally old and not updated. So in any case you do need to poke around quite a bit to get things working.

Some really good reading material:

• Kernel command using Linux system calls ( uses the _syscallN macro in examples )
• Playing with the cr0 register. This is a bit advanced for my current knowledge level and I’m in the process of fully understanding how the register works. Try at your own risk.

I started off with BeautifulSoup for a certain project that needs to construct database queries out of XML. I was almost done with this component when I discovered much to my chagrin that BeautifulSoup does not respect the case of the strings in the tag name. For eg some text would yield the name of the tag as ‘foobar’. I had to find some way to replace the code base within a short time.
I quickly went through some of my starred items in Google Reader. One of the interesting finds was using lxml from the IBM developer works pages. One of the examples in that page seemed very similar to using a SAX parser. I wasn’t quite interested at that moment to learn a third party library from scratch, so I decided to go for the xml.sax package already provided by default. A tutorial and reading through a couple of pages from the book ‘Python and XML’ later, I managed to get it right.

Basically, an SAX parser is event driven, and you can define functions that can be described like event handlers in JAVA which act upon those events. These events include encountering the opening or closing of an XML text node, an XML element node etc. The JAVA implementation of SAX defined interfaces for the handlers. SAX for one thing, doesn’t have a formal specification as yet and since Python doesn’t support interfaces, it included four classes in xml.sax.handler. You can read about them in the python docs. From the docs:

Handler implementations should inherit from the base classes provided in the module xml.sax.handler, so that all methods get default implementations.

The one we will be using the most is the class ContentHandler. The handler for our own XML format will inherit it. We override functions implemented in ContentHandler to customize them to our own needs. Some of the important functions are:

startElement(name,attrs): This is called by the parser when it encounters the start of an element. name holds the name of the element and attr holds the attributes.

endElement(name): Called by the parser when it encounters the end of an element.

character(content): Returns chunks of character data when found. The character data may be included in a single chunk or split into multiple chunks. It’s advisable to use flags rather than relying on the former assumption.

Since a SAX parser is a stream parser and doesn’t build an in-memory tree, you can’t backtrack in the tree and nodes don’t have the usual child-parent relationship found in DOM style parsers. Flags are used to track what element the parser is currently in and take appropriate actions. There is a caveat here. If you are working on a tree with a large depth, it can be really frustrating and painful to manage a lot of flags. The one I am working on can have a maximum depth of 5, but it would rarely exceed 4. So it wasn’t a big deal for me to handle that many flags.

The basic outline of a program that makes use of SAX in Python would be like
#####################################################

from xml.sax.handler import ContentHandler
from xml.sax import make_parser

class CustomHandler(ContentHandler):
	def __init__(self):
		#initialize flags and other data structures if needed
	def startElement(self,name,attrs):
		#set flags, get value of attributes etc
	def endElement(self,name):
		#clear flags etc
	def character(self,ch):
		#copy character data to any data structure if needed
ch=CustomHandler()
saxparser=make_parser()
saxparser.setContentHandler(ch)
saxparser.parse(some_file_stream)

########################################################

PS: I am still stuck with Python 2.5.2. I’m trying to keep pace by reading the changes in 2.6. I am counting on Harsh to bail me out when the need arises :p

No big deal. Everyone has to do it once for their UNIX concepts practical work I guess. What irked me was that the teacher and some of the ‘good students’ were hell bent on arguing that non-preemptive process scheduling simulations do not require any kind of data regarding the arrival time of processes in the ready queue. I still don’t understand how that simulates a real life scenario, that is, after the completion of a process, to which other process would the scheduler perform a context switch. Could someone point me out if I am missing something here?

Anyways, this is the code I wrote today during the practical class:

#include<stdio.h>

struct processinfo

{

        int id;

        int burst;

        int arr;

};

typedef struct processinfo process;

/*checks if all the processes have finished execution*/

int check(process* p,int n)

{

        int i=0,flag=0;

        for(i=0;i<n;i++)

        {

                if(p[i].burst!=0)

                {

                        flag=1;

                        break;

                }

        }

        return flag;

}

int main()

{

        int n;

        printf("No. of processes:");

        scanf("%d",&n);

        process p[n];

        int i=0,j=0;

        int wt[n]/*the waiting time array*/,last[n]/*last[i] stores the last executed instant of the process i*/;

        for(i=0;i<n;i++)

        {

        	printf("Proccess id:");

        	scanf("%d",&p[i].id);

                printf("Arrival time:");

                scanf("%d",&p[i].arr);

                printf("Burst time:");

                scanf("%d",&p[i].burst);

                wt[i]=0;

                last[i]=0;

        }

        /*gah, bubble sort*/

       /*anyways we won't be doing any sorting in a real scheduler, I guess*/

        for(i=0;i<n-1;i++)
		{

        	for(j=i+1;j<n;j++)

        	{

                if(p[j-1].arr>p[j].arr)

                {

                 process temp=p[j-1];

                 p[j-1]=p[j];

                 p[j]=temp;

                }

        	}

		}

int timequantum=2/*the time quantum*/,time=0/*running time*/;

while(1)

{

        if(!check(p,n))

                break;

	int k=0;

for(k=0;k<n;k++)

{

        if(p[k].burst!=0)

        {

                if(p[k].burst<timequantum)

                {

                        wt[k]+=time-last[k];

                        time+=p[k].burst;

                        p[k].burst=0;

                }

                else

                {

                        p[k].burst-=timequantum;

                        wt[k]+=time-last[k];

                        time+=timequantum;

                }
                last[k]=time;
        }

}

 }//end of while

for(i=0;i<n;i++)

	printf("\nWaiting time for Process %d:%d",p[i].id,wt[i]);

return 0;

}

It is 3 in the morning. Don’t expect me to blabber much. Windows users, please fuck off at this point. Just give the file a py extension and move it to ~/.xchat2 so that it loads automatically at startup. Use /show to tell everyone in the channel you are playing [insert an emo band name here] and proceed to cut your wrist or let everyone know you are a real emo with the legwarmers and all. OK ENOUGH.

import socket,re,xchat

__module_name__ = "mpd-np"
__module_version__ = "0.1"
__module_description__ = "mpd now playing"

def playing(word, word_eol, userdata):
	mpd=socket.socket ( socket.AF_INET, socket.SOCK_STREAM )
	host='127.0.0.1'
	port=6600
	mpd.connect((host,port))
        #welcome msg could be used to extract mpd version information
	welcome=mpd.recv(1024)
	mpd.send('currentsong\r\n')
        #get all the info about current track being played
	data=mpd.recv(4096)
	#I wont be explaining any REs
	artist=re.findall(r'Artist[:]\s[\S ]+',data)[0].split(':')[1]
	title=re.findall(r'Title[:]\s[\S ]+',data)[0].split(':')[1]
	msg='Now playing:'+artist+'-'+title
	xchat.command('me '+msg)
	data=""
	mpd.close()
	return xchat.EAT_ALL
#hooks into the show xchat command
xchat.hook_command("show",playing)

Reports of idiotic behavior go into the comments.

I had to plot a graph for the result of the simulation models I am running. The output happens to be something like the following for a simple model consisting of just a single HTTP server and another client requesting HTTP data with certain probability distributions that specify the number of page requests in a session and also the number of objects that would be there on a requested page. Actually it uses the dml configuration file included with the SSFNet WWW package in the test directory. This is how it looks if simulated for 3600 seconds:

199.27878718 [ sid 0 start 89.99831406 ] clnt 1 srv 2(0) #pages: 3 #objects: 28 total: 395466B seconds: 109.28047312 SUCCESS
346.0642575 [ sid 1 start 345.81315335 ] clnt 1 srv 2(0) #pages: 1 #objects: 2 total: 18603B seconds: 0.25110415 SUCCESS
1311.24547598 [ sid 2 start 901.76614728 ] clnt 1 srv 2(0) #pages: 2 #objects: 725 total: 5925458B seconds: 409.4793287 SUCCESS
1339.09795493 [ sid 3 start 1338.52968641 ] clnt 1 srv 2(0) #pages: 1 #objects: 2 total: 4977B seconds: 0.56826852 SUCCESS
1421.11860574 [ sid 4 start 1345.75319688 ] clnt 1 srv 2(0) #pages: 3 #objects: 6 total: 25174B seconds: 75.36540886 SUCCESS
1559.72102735 [ sid 5 start 1530.85835094 ] clnt 1 srv 2(0) #pages: 2 #objects: 3 total: 9565B seconds: 28.86267641 SUCCESS
1762.46745924 [ sid 6 start 1591.87139928 ] clnt 1 srv 2(0) #pages: 3 #objects: 3 total: 9898B seconds: 170.59605996 SUCCESS
2020.12789558 [ sid 7 start 1822.41333981 ] clnt 1 srv 2(0) #pages: 6 #objects: 9 total: 61346B seconds: 197.71455577 SUCCESS
2348.85181273 [ sid 8 start 2067.45385462 ] clnt 1 srv 2(0) #pages: 7 #objects: 14 total: 69341B seconds: 281.39795811 SUCCESS
2492.52192579 [ sid 9 start 2492.49153171 ] clnt 1 srv 2(0) #pages: 1 #objects: 1 total: 2606B seconds: 0.03039408 SUCCESS
2590.15405846 [ sid 10 start 2528.44871353 ] clnt 1 srv 2(0) #pages: 3 #objects: 5 total: 36481B seconds: 61.70534493 SUCCESS
2631.19590927 [ sid 11 start 2593.40280719 ] clnt 1 srv 2(0) #pages: 2 #objects: 4 total: 14987B seconds: 37.79310208 SUCCESS
2768.67779732 [ sid 12 start 2708.89820375 ] clnt 1 srv 2(0) #pages: 3 #objects: 3 total: 17770B seconds: 59.77959357 SUCCESS
3037.57726869 [ sid 13 start 2854.45071944 ] clnt 1 srv 2(0) #pages: 5 #objects: 9 total: 33623B seconds: 183.12654925 SUCCESS
3212.22081067 [ sid 14 start 3132.34843616 ] clnt 1 srv 2(0) #pages: 3 #objects: 3 total: 15702B seconds: 79.87237451 SUCCESS

I made a few regexs to parse the required data from this output like the start time, end time, bytes transferred etc etc. But then I’ll save it for the next post where I’ll deal with it at length and talk about what this post is supposed to be about i.e. gnuplot.

gnuplot is really a versatile tool for plotting both 2D graphs and 3D surfaces. It comes with a very small syntax set that is easy to get hang of once you start using it. I first installed it from the repos without checking the version. So even though the current stable version is v4.2.3 , the repos had v4.0 without any patches. I decided to chuck it and rather build from source. It was a real pain. I spent two hours on getting the right configuration to have the X Windows system support installed. Tried quite a lot of flags with –with-x being included every time but still it was configured without the X windows system support. In the mean time I was being helped by a certain user on #gnuplot. I was about to give it up for the day but then we had a discussion about the possibility of X11 library paths being fucked up in ubuntu. So I decided to configure with the following flags:

$./configure --with-x --with-x-includes=/usr/include/X11 --with-x-libraries=/usr/lib/X11 --with-readline=gnu
..and it was done!

There are different kinds of 2D plots in gnuplot, like lines, linespoints, points, steps etc. I was looking for something that would give me something like a bar chart for the time intervals of the different sessions. So I went for the boxes. The tricky part in plotting with boxes is that you can specify the width of each box in a separate column but the box is drawn centred around the x-axis values. This had me confused for some time before I read the help topic on boxes. So I made a minor change in my program that would put the midpoint of the interval for the x-axis values. The graph looks something like this:

Resources:
1. The demo directory in the source code. Minimal online explanation here.
2. Collection of gnuplot tutorials

I made one I think doing a md5 hash of the resulting ciphertext would add an extra layer of security. What do the others think of this idea? Of course, the safe storage and transmission of the XOR key becomes an issue. Check the code and see if you could come up with suggestions to optimize it. I particularly don’t like the exponential order for loop at the very end. The XOR key is stored in the xor-key file in the same directory where the code is run.

import sys

if len(sys.argv)<=1:
	print 'Usage: python basicsalt.py <input file>'
	sys.exit(1)

frandom=open('/dev/urandom','r')  #open the random device
fpickle=open(sys.argv[1],'r')	#open the input file in read-only mode
bytes=1024

key=open('xor-key','w')	#open/create the xor-key file

picklebuf=fpickle.read() #read the input file
fpickle.close()

fpickle=open(sys.argv[1],'w')	#open the input file in write mode
				#the ciphertext is stored in the same file
tempicklebuf=''			#temporary buffer for storing the ciphertext

fileLen=len(picklebuf)
print fileLen

#a function that doesn't exactly 'add' the salt in the classical sense of the term
def addsalt(pb,sb,fileLen):
	global tempicklebuf
	for i in range(fileLen):
		tempicklebuf+=chr(ord(pb[i]) ^ ord(sb[i]))	#just plain XOR

bufsalt=frandom.read(fileLen)
addsalt(picklebuf,bufsalt,fileLen)
key.write(bufsalt)

fpickle.write(tempicklebuf)
frandom.close()
key.close()

Ladies and gentlemen, fasten your seat belts and get ready to be taken through a whirlwind tour through the kingdom of the self proclaimed ‘tech bloggers’ who profess to live, eat and breathe technology. Soon you would come to know they fart too.

Some generalized observations on the sweeping epidemic that is ‘tech blogging’ :

1. Someone starts a damn, fucking blog with a nice theme. Point to be noted. He could later throw around that fact and claim to be a CSS geek.

2. He starts writing vociferously about mobile phones, ‘gadgets’, ‘latest tech news’ (sic), ‘tips and tricks’ (did I forget to add Windows here?), ‘tweaks’ (o yeah, what’s next? tweaking nipples video on youtube?) etc etc

3. Uses the term ‘geek’ atleast once while describing himself.

That is all that is there to it. Uhm..you would say how do I term it as an epidemic? Let me begin with some specific cases in the Indian blogosphere scene. Cases that would make Tim Berners Lee cringe.

——–
Case I:
——–

Ashfame tech blog

What is it about: So this guy is a tech blogger. Heck, he knows he is damn popular. He even tom toms his blog stats in a separate post.

What does his blog offer: Let us allow him to describe it himself, “I blog about blogging, tips and tricks, tutorials, hacking, hardware and reviews. A niche is less than what I blog about.”

Fart factor (on a scale of 10): 9.5

Quick analysis: Looking at the first page, I see posts on Opera download, some obscure tool for creating animated gif, ‘tips’ on using IrfanView (aargh) and write protecting USB drives. No need to take the pain to delve dipper into the guano. Mr. Ashfame, you’d have got a 8 but for your zomg-look-i-am-a-hacker posts. Now Mr. Ashfame has ‘tips’ for becoming a hacker too! No wonder the old farts at AntiOnline are going to lose their jobs soon. In his md5sum post, he has a radically different view of the security of hash functions:

It is extremely unlikely for two non-identical files to have same md5sum as calculated by hashing algorithms (however the theory says something else).

Extremely unlikely, no way. The very nature of the 128-bit md5 hashing function opens it up to collisions as some researchers have already demonstrated and which I am not fully qualified to discuss. Mr. Ashfame, I have a task for you. I have a XOR encrypted C source file. Let me see you break it. This hint should be enough for you as I presume you have extensive knowledge of something even more secure than this i.e. one-way hashing functions.

——–
Case II
——–

Akshay Gandhi’s tips & tricks and blah blah…

What is it about: A blog that would give you tons of info on freeware and…again…*bangs head on a wall* tips & tricks!

What does his blog offer: Again, nothing better than allowing the person himself to demonstrate it to you.

An ALL INCLUSIVE BLOG – Find trivias, graphology, Vista tips, troubleshooting, mobile secrets, mobile code, reviews, freewares, tips, tricks,legal info, law firms, legal view, jokes, interesting facts, etc…

Fart factor (on a scale of 10): 9

Quick analysis: An all inclusive blog. What more could you ask for, eh? But wait, I thought people satisfied those needs by visiting santabanta.com . Some remarkable gems from the first page: Power Defragmenter 2.0.125, Rapid Typing tutor (they still have openings for typists?) and some shit about System Restore in Vista. Seriously dude, if you need ‘tips & tricks’ for working on Vista you better re-evaluate your current technical knowledge base.

———-
Case III
———-

John TP

Do I need to say more?

*SIGH* See, how things get redundant after a short trip to only two blogs? Redundancy, as we have come to know, is frowned upon in the computing world. We have extremely reliable and well edited sources of information for gadgets and cellphone news, new softwares, games and OS releases, major policy upheaval by decision makers in Government and all that on the Wired, Ars Technica, Endgadget, ZDNet blogs (no, I won’t be mentioning TechCrunch here) and a lot many which I don’t visit but have a loyal reader base. The point I’m trying to make here is that don’t fucking post just for the sake of it if you don’t have anything new to add to the already vast source of information available on the same topic. Why? It gets really frustrating for a newbie who would google for something like optimizing his PC and would end up at the countless sites like the ones mentioned above. Total wastage of his bandwidth and time. If an established site has already covered the same thing, DO NOT fucking post the same thing again. (Do I sound like Brad Pitt here?)

BONUS TIP: Stop using Windows and you’d never need to rummage through such shitholes for optimizing applications. There are enough GNU and other open source tools that come bundled with all *nix OSs which would take care of such nifty matters. Case in point: iptables or Zone Alarm?

Get a cue from the blogs of the numerous FOSS developers and also programming stalwarts like Jeff Atwood. Write something that really matters, has some real content. Else, just fucking get off the tube. You are doing no good to us. Just because you have been handed an internet connection, you can’t get away with swinging your fuckin badass boner like it was nobody else’s business (read: posting content with 8+ fart factor).

Signing off, as the lords of code would say:

Talk is cheap. Show me the code.