Truth, Computing and Fail

I started off with BeautifulSoup for a certain project that needs to construct database queries out of XML. I was almost done with this component when I discovered much to my chagrin that BeautifulSoup does not respect the case of the strings in the tag name. For eg some text would yield the name of the tag as 'foobar'. I had to find some way to replace the code base within a short time.
I quickly went through some of my starred items in Google Reader. One of the interesting finds was using lxml from the IBM developer works pages. One of the examples in that page seemed very similar to using a SAX parser. I wasn't quite interested at that moment to learn a third party library from scratch, so I decided to go for the xml.sax package already provided by default. A tutorial and reading through a couple of pages from the book 'Python and XML' later, I managed to get it right.

Basically, an SAX parser is event driven, and you can define functions that can be described like event handlers in JAVA which act upon those events. These events include encountering the opening or closing of an XML text node, an XML element node etc. The JAVA implementation of SAX defined interfaces for the handlers. SAX for one thing, doesn't have a formal specification as yet and since Python doesn't support interfaces, it included four classes in xml.sax.handler. You can read about them in the python docs. From the docs:

Handler implementations should inherit from the base classes provided in the module xml.sax.handler, so that all methods get default implementations.

The one we will be using the most is the class ContentHandler. The handler for our own XML format will inherit it. We override functions implemented in ContentHandler to customize them to our own needs. Some of the important functions are:

startElement(name,attrs): This is called by the parser when it encounters the start of an element. name holds the name of the element and attr holds the attributes.

endElement(name): Called by the parser when it encounters the end of an element.

character(content): Returns chunks of character data when found. The character data may be included in a single chunk or split into multiple chunks. It's advisable to use flags rather than relying on the former assumption.

Since a SAX parser is a stream parser and doesn't build an in-memory tree, you can't backtrack in the tree and nodes don't have the usual child-parent relationship found in DOM style parsers. Flags are used to track what element the parser is currently in and take appropriate actions. There is a caveat here. If you are working on a tree with a large depth, it can be really frustrating and painful to manage a lot of flags. The one I am working on can have a maximum depth of 5, but it would rarely exceed 4. So it wasn't a big deal for me to handle that many flags.

The basic outline of a program that makes use of SAX in Python would be like
#####################################################

########################################################

PS: I am still stuck with Python 2.5.2. I'm trying to keep pace by reading the changes in 2.6. I am counting on Harsh to bail me out when the need arises :p

No big deal. Everyone has to do it once for their UNIX concepts practical work I guess. What irked me was that the teacher and some of the 'good students' were hell bent on arguing that non-preemptive process scheduling simulations do not require any kind of data regarding the arrival time of processes in the ready queue. I still don't understand how that simulates a real life scenario, that is, after the completion of a process, to which other process would the scheduler perform a context switch. Could someone point me out if I am missing something here?

Anyways, this is the code I wrote today during the practical class:

It is 3 in the morning. Don't expect me to blabber much. Windows users, please fuck off at this point. Just give the file a py extension and move it to ~/.xchat2 so that it loads automatically at startup. Use /show to tell everyone in the channel you are playing [insert an emo band name here] and proceed to cut your wrist or let everyone know you are a real emo with the legwarmers and all. OK ENOUGH.

Reports of idiotic behavior go into the comments.

I had to plot a graph for the result of the simulation models I am running. The output happens to be something like the following for a simple model consisting of just a single HTTP server and another client requesting HTTP data with certain probability distributions that specify the number of page requests in a session and also the number of objects that would be there on a requested page. Actually it uses the dml configuration file included with the SSFNet WWW package in the test directory. This is how it looks if simulated for 3600 seconds:

199.27878718 [ sid 0 start 89.99831406 ] clnt 1 srv 2(0) #pages: 3 #objects: 28 total: 395466B seconds: 109.28047312 SUCCESS
346.0642575 [ sid 1 start 345.81315335 ] clnt 1 srv 2(0) #pages: 1 #objects: 2 total: 18603B seconds: 0.25110415 SUCCESS
1311.24547598 [ sid 2 start 901.76614728 ] clnt 1 srv 2(0) #pages: 2 #objects: 725 total: 5925458B seconds: 409.4793287 SUCCESS
1339.09795493 [ sid 3 start 1338.52968641 ] clnt 1 srv 2(0) #pages: 1 #objects: 2 total: 4977B seconds: 0.56826852 SUCCESS
1421.11860574 [ sid 4 start 1345.75319688 ] clnt 1 srv 2(0) #pages: 3 #objects: 6 total: 25174B seconds: 75.36540886 SUCCESS
1559.72102735 [ sid 5 start 1530.85835094 ] clnt 1 srv 2(0) #pages: 2 #objects: 3 total: 9565B seconds: 28.86267641 SUCCESS
1762.46745924 [ sid 6 start 1591.87139928 ] clnt 1 srv 2(0) #pages: 3 #objects: 3 total: 9898B seconds: 170.59605996 SUCCESS
2020.12789558 [ sid 7 start 1822.41333981 ] clnt 1 srv 2(0) #pages: 6 #objects: 9 total: 61346B seconds: 197.71455577 SUCCESS
2348.85181273 [ sid 8 start 2067.45385462 ] clnt 1 srv 2(0) #pages: 7 #objects: 14 total: 69341B seconds: 281.39795811 SUCCESS
2492.52192579 [ sid 9 start 2492.49153171 ] clnt 1 srv 2(0) #pages: 1 #objects: 1 total: 2606B seconds: 0.03039408 SUCCESS
2590.15405846 [ sid 10 start 2528.44871353 ] clnt 1 srv 2(0) #pages: 3 #objects: 5 total: 36481B seconds: 61.70534493 SUCCESS
2631.19590927 [ sid 11 start 2593.40280719 ] clnt 1 srv 2(0) #pages: 2 #objects: 4 total: 14987B seconds: 37.79310208 SUCCESS
2768.67779732 [ sid 12 start 2708.89820375 ] clnt 1 srv 2(0) #pages: 3 #objects: 3 total: 17770B seconds: 59.77959357 SUCCESS
3037.57726869 [ sid 13 start 2854.45071944 ] clnt 1 srv 2(0) #pages: 5 #objects: 9 total: 33623B seconds: 183.12654925 SUCCESS
3212.22081067 [ sid 14 start 3132.34843616 ] clnt 1 srv 2(0) #pages: 3 #objects: 3 total: 15702B seconds: 79.87237451 SUCCESS

I made a few regexs to parse the required data from this output like the start time, end time, bytes transferred etc etc. But then I'll save it for the next post where I'll deal with it at length and talk about what this post is supposed to be about i.e. gnuplot.

gnuplot is really a versatile tool for plotting both 2D graphs and 3D surfaces. It comes with a very small syntax set that is easy to get hang of once you start using it. I first installed it from the repos without checking the version. So even though the current stable version is v4.2.3 , the repos had v4.0 without any patches. I decided to chuck it and rather build from source. It was a real pain. I spent two hours on getting the right configuration to have the X Windows system support installed. Tried quite a lot of flags with --with-x being included every time but still it was configured without the X windows system support. In the mean time I was being helped by a certain user on #gnuplot. I was about to give it up for the day but then we had a discussion about the possibility of X11 library paths being fucked up in ubuntu. So I decided to configure with the following flags:

$./configure --with-x --with-x-includes=/usr/include/X11 --with-x-libraries=/usr/lib/X11 --with-readline=gnu
..and it was done!

There are different kinds of 2D plots in gnuplot, like lines, linespoints, points, steps etc. I was looking for something that would give me something like a bar chart for the time intervals of the different sessions. So I went for the boxes. The tricky part in plotting with boxes is that you can specify the width of each box in a separate column but the box is drawn centred around the x-axis values. This had me confused for some time before I read the help topic on boxes. So I made a minor change in my program that would put the midpoint of the interval for the x-axis values. The graph looks something like this:

Resources:
1. The demo directory in the source code. Minimal online explanation here.
2. Collection of gnuplot tutorials

I made one I think doing a md5 hash of the resulting ciphertext would add an extra layer of security. What do the others think of this idea? Of course, the safe storage and transmission of the XOR key becomes an issue. Check the code and see if you could come up with suggestions to optimize it. I particularly don't like the exponential order for loop at the very end. The XOR key is stored in the xor-key file in the same directory where the code is run.

Ladies and gentlemen, fasten your seat belts and get ready to be taken through a whirlwind tour through the kingdom of the self proclaimed 'tech bloggers' who profess to live, eat and breathe technology. Soon you would come to know they fart too.

Some generalized observations on the sweeping epidemic that is 'tech blogging' :

1. Someone starts a damn, fucking blog with a nice theme. Point to be noted. He could later throw around that fact and claim to be a CSS geek.

2. He starts writing vociferously about mobile phones, 'gadgets', 'latest tech news' (sic), 'tips and tricks' (did I forget to add Windows here?), 'tweaks' (o yeah, what's next? tweaking nipples video on youtube?) etc etc

3. Uses the term 'geek' atleast once while describing himself.

That is all that is there to it. Uhm..you would say how do I term it as an epidemic? Let me begin with some specific cases in the Indian blogosphere scene. Cases that would make Tim Berners Lee cringe.

--------
Case I:
--------

Ashfame tech blog

What is it about: So this guy is a tech blogger. Heck, he knows he is damn popular. He even tom toms his blog stats in a separate post.

What does his blog offer: Let us allow him to describe it himself, "I blog about blogging, tips and tricks, tutorials, hacking, hardware and reviews. A niche is less than what I blog about."

Fart factor (on a scale of 10): 9.5

Quick analysis: Looking at the first page, I see posts on Opera download, some obscure tool for creating animated gif, 'tips' on using IrfanView (aargh) and write protecting USB drives. No need to take the pain to delve dipper into the guano. Mr. Ashfame, you'd have got a 8 but for your zomg-look-i-am-a-hacker posts. Now Mr. Ashfame has 'tips' for becoming a hacker too! No wonder the old farts at AntiOnline are going to lose their jobs soon. In his md5sum post, he has a radically different view of the security of hash functions:

It is extremely unlikely for two non-identical files to have same md5sum as calculated by hashing algorithms (however the theory says something else).

Extremely unlikely, no way. The very nature of the 128-bit md5 hashing function opens it up to collisions as some researchers have already demonstrated and which I am not fully qualified to discuss. Mr. Ashfame, I have a task for you. I have a XOR encrypted C source file. Let me see you break it. This hint should be enough for you as I presume you have extensive knowledge of something even more secure than this i.e. one-way hashing functions.

--------
Case II
--------

Akshay Gandhi's tips & tricks and blah blah...

What is it about: A blog that would give you tons of info on freeware and...again...*bangs head on a wall* tips & tricks!

What does his blog offer: Again, nothing better than allowing the person himself to demonstrate it to you.

An ALL INCLUSIVE BLOG - Find trivias, graphology, Vista tips, troubleshooting, mobile secrets, mobile code, reviews, freewares, tips, tricks,legal info, law firms, legal view, jokes, interesting facts, etc...

Fart factor (on a scale of 10): 9

Quick analysis: An all inclusive blog. What more could you ask for, eh? But wait, I thought people satisfied those needs by visiting santabanta.com . Some remarkable gems from the first page: Power Defragmenter 2.0.125, Rapid Typing tutor (they still have openings for typists?) and some shit about System Restore in Vista. Seriously dude, if you need 'tips & tricks' for working on Vista you better re-evaluate your current technical knowledge base.

----------
Case III
----------

John TP

Do I need to say more?

*SIGH* See, how things get redundant after a short trip to only two blogs? Redundancy, as we have come to know, is frowned upon in the computing world. We have extremely reliable and well edited sources of information for gadgets and cellphone news, new softwares, games and OS releases, major policy upheaval by decision makers in Government and all that on the Wired, Ars Technica, Endgadget, ZDNet blogs (no, I won't be mentioning TechCrunch here) and a lot many which I don't visit but have a loyal reader base. The point I'm trying to make here is that don't fucking post just for the sake of it if you don't have anything new to add to the already vast source of information available on the same topic. Why? It gets really frustrating for a newbie who would google for something like optimizing his PC and would end up at the countless sites like the ones mentioned above. Total wastage of his bandwidth and time. If an established site has already covered the same thing, DO NOT fucking post the same thing again. (Do I sound like Brad Pitt here?)

BONUS TIP: Stop using Windows and you'd never need to rummage through such shitholes for optimizing applications. There are enough GNU and other open source tools that come bundled with all *nix OSs which would take care of such nifty matters. Case in point: iptables or Zone Alarm?

Get a cue from the blogs of the numerous FOSS developers and also programming stalwarts like Jeff Atwood. Write something that really matters, has some real content. Else, just fucking get off the tube. You are doing no good to us. Just because you have been handed an internet connection, you can't get away with swinging your fuckin badass boner like it was nobody else's business (read: posting content with 8+ fart factor).

Signing off, as the lords of code would say:

Talk is cheap. Show me the code.

After relaxing for 5 days and staying away from the internet as much as possible, I'm back to some serious work.

Wrote a client and server process in python where the client sends inorder arithmetic expressions(not parenthesized, as of yet) and the server evaluates them and returns the result to the client. Pretty lame but I think it was good for an hour's effort . And Harsh, if you are thinking it is a threaded server, sorry its not
Server
Client

Coming to the other thing I'm working on, it is simulation of a network model with SSFNet and I won't lie, at the moment I am not able make any head or tail of how to proceed with the work.

If you don't wanna take the trouble of downloading the files and just want to check the code, read on:
Read the rest of this entry »

This is meant for NASM running on a linux machine. First things first: everything in UNIX is a file. A device is a file, a network socket is a file and everything else you can imagine is a file. Each file has a file descriptor attached to it. Some standard file descriptors are 0 for STDIN (standard input), 1 for STDOUT (standard output) and 2 for STDERR (standard error output). Since everything is a file, you can read() and write() on them provided you have the file descriptor.

read(), write() and open() are some of the very important syscalls in linux that would allow you to manipulate any file (that is, provided you have the permissions). There are some 150+ of them if you are really interested to find out. For syscalls, you need to pass the necessary parameters in the ebx, ecx, edx registers and so on, depending upon the number of parameters. You can also call libc functions by declaring them as extern printf for example and pushing the necessary parameters on the stack, the last parameter to be pushed first. To understand what kind of parameters need to be passed for the syscalls, read the developer man pages. Like man 2 write for write(). Use this for any other syscall like read() or open() and also keep the list of syscalls mentioned earlier in this post handy.

Regarding passing arguments to the program, they are stored in the stack as

argc
argv[0]
argv[1]
argv[2]

and so on...
argv[0] is the program name itself, so we can pop and discard it safely.

The following is a small program in NASM that takes a filename as an argument and reads 8kB from the file and displays it on the screen.

section .data
buf times 8 db 0
bufsize db 8192

section .text
global _start

_start:
pop ebx
pop ebx
pop ebx
mov eax,5
mov ecx,2	;the read mode
int 80h

test eax,eax
jns file_read

mov ebx,eax
mov eax,1
int 80h
ret

file_read:
mov ebx,eax	;move the file descriptor
mov eax,3	;3 for read()
mov ecx,buf	;the buffer
mov edx,bufsize	;the buffer size
int 80h

mov edx,eax	;move the number of bytes read from the previous syscall to edx
mov eax,4	;4 for write()
mov ebx,1	;1, the STDOUT file descriptor
int 80h

mov eax,1
mov ebx,0
int 80h		;exit with error code 0
ret

To assemble: nasm -f elf filename.asm
To link: ld -s -o executable-name filename.o

To know more about the read and write modes, the file descriptors and some increased masochistic behavior, start with reading the /usr/include/unistd.h and /usr/include/fcntl.h files. Poke around that directory for some more knowledge.

Follow this link.

DOS is dead. It's 16-bit. It's obsolete. The only people that still write DOS assembly are crazy old hackers that are too attached to their 386s to throw them away.

Couldn't agree more. I see no reason to be working with 16-bit registers on systems where you can afford 32-bit mode. Developing assembly programs for DOS using 8086/8088 code means you are forced to run them in 'real mode', that the processor can address only 1MByte of memory. Also that you will be tearing out your hair keeping track of segments and also that you would never know when a segfault would be staring you in the face. Take into account the fact that unlike linux which would give an error like Segmentation fault(Core dumped) and then come back to its helpful self, if you are working on the Command Prompt you would only get to see something like An illegal instruction occured at and two so called 'options' (close and ignore) which anyways, would lead to the same thing i.e. you having to terminate the process. If it is the real 16-bit MS-DOS, then God save you or to be more precise, the reset button.

I have no idea though why our colleges are so obsessed with totally obsolete standards that are out of sync with the requirements of modern computing.

Some might have started learning assembly language programming in MASM and maybe already under the impression that finally they are 'doing some shit'. Well, I don't want to sound as if I am a know-it-all guy. Even I was under the same impression, and we can't be blamed. Well, when you see a screen all fluorescent green and bright blue showing the registers and segments getting updated as you trace through the program, you can be excused for feeling on top of the world. That is, if you are a Computer Science student and most importantly think like one too!

Well, sorry for the digression. Coming back to the point, there are two major syntaxes of writing assembly language code for the x86 processors: the AT&T syntax and the Intel syntax itself. For the first one, we can use the GNU assembler in linux with the as command (which is a part of the gcc package) and for the latter, another assembler called NASM is used. In this scenario, assemblers like MASM and TASM don't exactly fit in and are neither here nor there.

You would say, why should I give a f to the assembler and its syntax as long as it is churning out the binaries. After all, whatever the syntax it would get translated to the same machine code in the end. Sample this, you are stuck on quite a large assembly code and you need some urgent help. You go to an IRC channel, show the guys out there the code and ask them to help you out. Chances are that none of them would be able to help you out. Why? Simply because MASM and TASM syntaxes aren't exactly standards and so most probably they haven't even cared to learn them.

Now if you are stuck in the lab using MASM and on top of that debugging using Turbo Debugger (yeah, that Turbo shit!), I would advise that at least use the debugger that comes with DOS. I can say this from my personal experiences. TD will often load the code from some other code segment that you had executed previously rather than the one you want to trace right now. It can be really irritating and for the programs which have almost the same functionality like multiplying two 16-bit numbers and multiplying a 32-bit and a 16-bit number, after you have traced through half the program you realise this is not the program you wanted to trace. I am yet to face any such issues with debug.