READIN

Friday, May second, 2008

🦋 Visual Space

This week I have been making an effort to add illustrations to my posts and to my sidebar links. I'm pretty happy with how it's looking and intend to keep doing that -- it seems to make the page a lot more visually engaging. Also thinking of trying to find a graphic that would fit nicely in the upper right-hand corner of the page, where there is a lot of empty space.*

Somewhat related, I've been doing a lot of work on the interface for adding new posts and editing posts, so that I'm able to see how the post will render as I'm adding it. Would like to do something similar for adding/editing the daily "Of interest:" links, but that is going to be a bit more involved.

*Update: Found one! How do you like it?

posted evening of May second, 2008: Respond
➳ More posts about The site

Thursday, May first, 2008

🦋 Speaking of birthdays

I totally forgot to note the passing of another year blogging, which happened last Friday. I have been keeping this journal for 5 years, now! (At roughly 240235 posts/year.*) That is a long time.

I feel like writing some reminiscences of starting this journal, but not right now. Maybe on the weekend I will. (In the meantime, here is an early post about my motivations in starting a blog.)

*Forgot: the id number of the latest post is not quite the same as the number of posts, since there have been a couple of deletions here and there. (It's very front-loaded though: in the first four years the average number of posts is 176, in the fifth year there are 473 posts. I wonder what the sixth year will bring?)

posted morning of May first, 2008: Respond
➳ More posts about Projects

Saturday, March 29th, 2008

🦋 AJAX geekery

Permit me to wax geeky for a moment: last night I added a new feature to the site, which involves dynamic loading of page elements. Fun! About half (a little less) of the size of this page (ie, roughly 20K bytes) is the blogroll, on the right-hand side of the page under the "Where to go from here" heading -- which you might not think to look at it, since most of the data is hidden when the page loads and only shown when you click on the category headings.

This means that whenever a person or a robot downloads the page, 20K of data is sent that is probably not going to be displayed or used. I've been trying for a while to figure out how to only send it to people who are interested in it, i.e. not to robots or to one-time visitors who come here from a Google search for a book their class is reading, which together account for the great majority of page views. Last night I came up with a pretty seamless solution:

I recently implemented a sticky blogroll, using cookies to ensure that once you have clicked a blogroll category header, the category will remain visible when you reload the page. So it's easy to check whether a user is a repeat visitor who has in the past looked at the blogroll -- since the blogroll is stored in a file on the server that gets read at load time, all I had to do was create a truncated version of that file (containing only data for the default visible categories) and include that instead if the relevant cookie was not set.

But then when a new visitor decides s/he wants to look at the blogroll, and clicks on a category header, how is s/he going to get data? Well in the truncated blogroll file, the empty headers are linked to a Javascript function called load_full_links, which uses XMLHttpRequest to download the complete blogroll -- so the first time you click a category you will see about a 1-second delay. But then your cookie is set, so going forward the blogroll is loaded with the page. I think it's a pretty nice bit of design/programming and I'm looking forward to using a similar algorithm for other pieces of functionality.

Potential problems:

• I have tested this with Firefox and Internet Explorer. I'm not sure how it will work with other browsers, and it will definitely not work in a browser where Javascript is disabled. This doesn't seem like a big deal to me but if it causes trouble for you, let me know and I'll try and come up with a solution.
• Search engines will no longer index most of my blogroll. This is, on one hand, good -- I have seen referrals from searches that hit an item on the blogroll, which generally seem like my page is not what the searcher was looking for -- and on the other hand, possibly not ideal -- Google and Technorati both pay a lot of attention to outgoing links.

posted morning of March 29th, 2008: Respond

Tuesday, March 18th, 2008

🦋 stderr and cgi

Handy to know: if you write to stderr from a CGI application (note: when I say "CGI application", I don't actually know what that means -- what I mean is, an application invoked by the http server in response to a GET or POST request and which uses environment variables to get information about the request), the output will go into the http server's log file. At least it will if the http server is Apache.

posted afternoon of March 18th, 2008: Respond
➳ More posts about Programming

Sunday, March 16th, 2008

🦋 Sticky sidebar

Last night and this morning, I added another feature to the site, which you might not notice if you don't use my blogroll often; but it's a neat feature so I'm going to tell you about it. The expandable categories in the blogroll will now remember between invocations of the site, whether they are open or not; so if you click on "Blogs | Politics" and then visit Obsidian Wings, next time you come to READIN, the Blogs | Politics links will be open. A little thing but I had been wanting to do it for a few months now.

Ideally I would like to have the categories be prefaced by a "+" or "-" character to indicate that there is material beneath them; right now you just have to know that something is there "because it looks like a blogroll", which doesn't seem ideal. I think this is within my Javascript programming abilities, look for it to happen sometime in the next month or two.

Also, I improved the formatting of the indentations in the blogroll. I had been doing it with blocks of &nbsp; characters; instead I am now using <span>s with padding-left. This means that long link titles wrap with a hanging indent rather than wrapping to the leftmost column; much prettier.

posted morning of March 16th, 2008: Respond

Friday, February 15th, 2008

🦋 Multiplexing

The fix is in -- the server is using poll instead of select, a new version has been built and delivered to the client, it can handle loads of clients. Here is the long and short of how you do it (without error-checking, which is dull*):

The Old Code

void select_files(int *fds, int nfds)
{
    int i, maxid;
    fd_set rset, wset;
    timeval tval;

    FD_ZERO (&rset);
    FD_ZERO (&wset);
    maxid = 0;
    for (i = 0; i < nfds; ++i) {
        FD_SET (fds[i], &rset);
        FD_SET (fds[i], &wset);
        if (fds[i] > maxid) maxid = fds[i];
    }
    tval.tv_sec = 5;
    tval.tv_usec = 0;

    select (maxid + 1, &rset, &wset, NULL, &tval);
    for (i = 0; i < nfds; ++i) {
        if (FD_ISSET(fds[i], &rset)) 
            read_file(fds[i]);
        if (FD_ISSET(fds[i], &wset)) 
            write_file(fds[i]);
    }
}

The New Code

void poll_files(int *fds, int nfds)
{
    int i;
    pollfd *pfds = (pollfd *) 
              malloc (nfds * sizeof (pollfd));

    for (i = 0; i < nfds; ++i) {
        pfds[i].fd = fds[i];
        pfds[i].events = POLLIN | POLLOUT;
        pfds[i].revents = 0;
    }

    poll (pfds, nfds, 5000);
    for (i = 0; i < nfds; ++i) {
        if (pfds[i].revents & POLLIN) 
            read_file(fds[i]);
        if (pfds[i].revents & POLLOUT) 
            write_file(fds[i]);
    }
}

In order to take advantage of the newly accessible file descriptors above 1024, you will need to add these lines to your /etc/security/limits.conf file:

(username)          soft    nofile          1024
(username)          hard    nofile          4096

I chose 1024 for the soft limit since most apps are not interested in the high number of files, and 4096 for the hard limit because I read on some message boards that performance will degrade above that number. Feel free to choose other values.

You then need to make the following calls from your code (or call ulimit from the script that starts your application):

    struct rlimit nofile;

    if (getrlimit (RLIMIT_NOFILE, &nofile) != 0) {
        fprintf (stderr, "Could not get NOFILE");
        exit (1);
    }
    nofile.rlim_cur = 4096;
    if (setrlimit (RLIMIT_NOFILE, &nofile) != 0) {
        fprintf (stderr, "Could not set NOFILE");
        exit (1);
    }

*If you're interested in the error-checking code, drop me a line -- I just don't feel like typing it out right now.

posted afternoon of February 15th, 2008: Respond

Wednesday, February 13th, 2008

🦋 vim rules!

Here's a neat trick: let's say you forgot to close a code block somewhere in a large C source file, and you can't figure out where, and the compiler is not helping you. Try putting a } character at the very end of the file, placing the cursor on it, and presing the % key -- vim will jump to the most recent { which was not matched by a }. (Note: this will not work if you have unmatched {'s inside conditional compilation blocks, which is generally a bad habit anyway.)

posted afternoon of February 13th, 2008: Respond

🦋 1024

A good thing to keep in mind when you are trying to write a TCP server that will support thousands of simultaneous connections: the default ulimit for maximum number of open files on a Linux system appears to be 1024. A further thing to keep in mind once you figure out how to adjust that upwards: there is a reason the default maximum is 1024!

See, if you use select() to multiplex your I/O (like I do), you will be passing structures of type fd_set around. These structures can only deal with file descriptors less than or equal to 1023. Try and set bit 1024, and you will break your program. But fear not! There is a solution; that solution is to use poll() instead of select(); apparently poll() is the new standard. First I ever heard of it! Switching from one to the other seems like it's not too hard, though I've just now started.

posted afternoon of February 13th, 2008: Respond

Wednesday, January 23rd, 2008

🦋 watch

Cool! I found and fixed a bug today using gdb's watchpoint feature, which I have never tried before. (Not cool: the bug was a careless typo that I should never have introduced.)

posted evening of January 23rd, 2008: Respond

Wednesday, January 9th, 2008

🦋 Video editing question

If I have an AVI file that is 128bit, 29 frame/sec, and I want to copy a clip from it into a new AVI file, what is the best tool to use? I am trying to use AVITrimmer, but the output file doesn't look any good. It still claims to be 29 frame/sec, but the video is wrong. It is not synchronized with the audio any longer.

Hmm... wait, that might be wrong... Yeah -- that was just weirdness from Windows Media Player.

posted evening of January 9th, 2008: Respond

Previous posts about Programming Projects
Archives