|
|
Monday, October 8th, 2007
Like I said below, I don't have much experience with database design. I don't really have any clue how to write a design document. But I want to describe the design I've come up with and see if I can make it sound as good as it appears to me to be. The thinking behind this is as follows: I have a lot of text records ("posts") which I want to classify by subject. I've done this, just like every other blog around, by using keywords -- if I tag a post with "food" say, or "singing", then it will show up when somebody looks at the site filtering for that subject. This is implemented with a simple search through the list of keywords on each post; not particularly fast but that's not a major problem in the context of my low-traffic site. But when I was putting the new software together, I had the idea that it would be great if, when somebody looked at the blog filtering for "food", they would see a little sidebar explaining what I write about when I write about food, and maybe some links to food sites I like etc. And more to the point, when somebody filters for "book:namered" (which is how I've been tagging my reading posts, "book:" and then a short identifier for the title), they would see up top that the posts were about My Name is Red by Orhan Pamuk, links to some outside reviews, links to Amazon and Abebooks, maybe a list of other of Pamuk's books that I have written about. So that is the problem I am trying to solve; and I think my solution is a pretty good one. First, simple keywords, like "food" and "singing". This is pretty easy; I have a table keyword with columns tag and description -- the description is what will be displayed in the sidebar when somebody filters by the tag. And I have a table (which I decided to name categories, for reasons that will soon become apparent) with two columns, postid and keyword -- I can join this table with posts when I want to do a filtering operation. Now what about the complex keywords like "book:namered", which include a class and an instance? Well check it out: every time I add a keyword which has a new class, I can just add a column to the categories table with the class name as the column name. And add a table with that name, which looks the same as the keyword table. And think of simple keywords as a special case of complex keywords, as if they had "keyword:" in front of them. So if somebody requests a filter for "book:namered", I can query from "posts JOIN categories ON posts.id = categories.postid JOIN book ON categories.book = book.tag" where book.tag = "namered". This will work for movies, projects, whatever. But the really cool thing is, I can add whatever columns I want to the book table and write a custom script to display the data associated with the tag "namered" in my sidebar. Consider these three requests:
- SELECT posts.* FROM posts JOIN categories ON posts.id = categories.postid WHERE categories.book = 'namered';
(This query would be represented by the keyword "book:namered".)
- SELECT DISTINCT posts.* FROM posts JOIN categories ON posts.id = categories.postid JOIN book ON categories.book = book.tag;
(This query would be represented by the keyword "book:".)
- SELECT posts.* FROM posts JOIN categories ON posts.id = categories.postid JOIN book ON categories.book = book.tag WHERE book.author = 'pamuk';
(This query would be represented by the keyword "book:author:pamuk".)
The first query will bring back all posts about My Name is Red. The second query will bring back all posts about reading any book. The third query will bring back all posts about reading any book by Orhan Pamuk. And all this is pretty easy to automate! It's all nearly in place! The next step, which will be a bit of effort to keep it elegant but totally within reach, is to create an administrative page for writing scripts to render an informative sidebar based on the column data contained in, say, the "namered" record in books.
posted evening of October 8th, 2007: Respond
➳ More posts about Programming Projects
|  |
|
When I was new to programming, in 1994 or '5 -- when OLE was a pretty freshly minted technology -- one of the projects I was working on was a way to abstract the functionality of some of my company's libraries into a common interface so that a program could load any of the libraries dynamically at runtime, based on a string key. I came up with the stunning realization that the interface could be expressed as a pure virtual C++ base class. All the libraries had to do was to export a function called "Create_x" which would instantiate an object whose class inherited interface x. This seemed to me like an awesome bit of innovation. By funny coincidence, another project I was working on around the same time was converting some of the company's VBX controls to OCX. (I don't think the term "ActiveX" had even been coined yet, but regardless we were not using it.) I wasn't reading the documentation of OLE very closely, relying on Microsoft's compiler to do most of the work for me; so it wasn't until a month or so later that I realized I had just reinvented a subset of OLE, and that I could have used OLE's framework to give my design a little more robustness. But whatever, the feeling that I was doing something new and inventive was payoff enough. So why this now? Well, I've been doing some pretty intensive design work in coming up with the database that supports this blog ("READIN 2.0", I am calling it in my head), and I have come up with a pretty cool idea. It seems innovative to me because it is something I've never heard of anyone doing; but I am not at all schooled in database design. I will write it up later on or tomorrow, and hopefully somebody will write back to me and let me know who invented it and where I can find out more.
posted evening of October 8th, 2007: Respond
➳ More posts about Programming
|  |
Saturday, October 6th, 2007
OK so it's a little corny... I spent last night and some of this morning writing code to administer and display at random different images and quotes at the top of the blog. This is fun, but I think I am doing it mainly for the sake of getting better at writing SQL queries and PHP scripts. The administrative pages are set up pretty nice and clean, I think.
...And guess what I have now!!! -- The ability to delete posts, something I have never been able to do before; and an automated backup script for the whole site, databases and scripts and all. Currently all my data zips up to ¾M.
Ok, so instead of putting up new posts every time I add a feature, I am just going to update this post for a while. (Hopefully I will get out of programmer head sometime and be able to think about anything besides updating the site...*) Just now I wrote a really cool addition to the database which handles categorization of posts with SQL joins instead of dumb text searching. This will eventually, I am thinking, allow me to include lots of interesting (?) information in the sidebar about what category of post is being displayed, which will involve some pretty sophisticated programming.
*This morning I was trying to read Other Colors and I couldn't stop thinking about database tables! How annoying.
posted afternoon of October 6th, 2007: Respond
➳ More posts about Projects
|  |
Friday, October 5th, 2007
So here's what I did, see: There are thousands of links all over the internets pointing to my blog, with the address http://www.readin.com/blog/blog.asp. Well I wanted to write the site in PHP; but what to do about all those old links? As it turns out I just kept the same url and told my http server to send .asp files to php: AddHandler php5-script asp I'm pretty sure the new script is able to handle all the parameters the old script was, and to give back quite similar results for nearly any set of parameters. So hopefully all those old links are going to continue to work.
posted evening of October 5th, 2007: Respond
➳ More posts about php
| |
|
Hi everybody, this is my new blog. I realize it looks largely the same as my old blog, if not indistinguishable. But it's quite different under the interface, and I have got lotsa plans for ways to enhance it and improve your user experience. (Hopefully they will come to fruition sooner than the plans expressed in the last paragraph here.)
posted evening of October 5th, 2007: Respond
|  |
Friday, March 12th, 2004
Yesterday one of my batch processes stopped working. I was a little baffled. The batch downloads some files from an ftp site, then expands them using pkunzip, then sends them to a program for processing. Pkunzip was telling me that I needed version 4.5 or later to expand the files -- never a problem in the past. I thought maybe the vendor had changed zip formats, which struck me as pretty bizarre. Everywhere on the web that I could find pkunzip, it was the same version as the one I was using (2.03g). And then I thought to try opening the files in WinZip. That worked of course; and I was very happy to discover that a command-line add-in is now available, along with a new version of WinZip. So... problem solved! (And into the bargain, wzunzip is way faster than the pkunzip I was using.) But what was the problem? It hit me when I was reading the "What's New" page in the WinZip 9.0 help file: In addition to supporting the original Zip file format, WinZip 9.0 also supports the 64-bit extensions to the Zip file format. The extended format lets you store all the data you need in Zip files of virtually unlimited size. The original Zip file format limited the number of member files in a Zip file to 65,535, and the maximum size of both the Zip file itself and any member file to 4 gigabytes. For all practical purposes, the 64-bit extended format eliminates all these restrictions. Using the extended format, the member file size, Zip file size, and number of member files you can add to a Zip file are limited only by your system's resources. So I checked and yep, the file size of the download is now a hair over 4G!
posted afternoon of March 12th, 2004: Respond
|  |
Thursday, May first, 2003
I've got things pretty well back into shape. The links were all lost so I will have to rebuild them by hand, not too hard to do but a pain anyway.
posted afternoon of May first, 2003: Respond
|  |
|
Well I came up with a cool idea for how the site data should be physically organized -- a script to put that organization into effect -- ran it and bang, it deleted all my posts! So I'm trying to get them back, meanwhile the links are not going to be there, sorry.
posted afternoon of May first, 2003: Respond
| |
Tuesday, April 29th, 2003
So why did I create this journal? The reasons are severalfold. I've been pretty fascinated by the blogging phenomenon since last summer, when I discovered a couple of the sites I've been reading regularly since -- Calpundit, Talking Points Memo, and Body and Soul are perhaps the "big three" for me -- and have been wondering if I could sustain such a steady level of posting and keep it interesting, and how it would sound if I did. I started my first "web log" before I knew that term, back in 1999 with the READIN book diary; but page generation was manual, not automated, and maintaining the site was a hassle, and I never really got far with it. Although, take a look at the page for Faulkner's The Hamlet to get an idea of where I wanted to go with it. Once I found out about ASP it seemed like the perfect fit -- I just had to learn how to code automatically generated journal pages and good things would come of it. Two things I wanted to learn to do formatting-wise; expressing dates and times in human terms, and displaying links in a hierarchical format. All the m/d/yyyy dates and hh:mm:ss times you see on web pages don't do it for me. They are over-determined and difficult to read. I wanted to express recent dates as "Yesterday", "Last Sunday", and posting times as just "morning", "evening", etc. I think I have come up with a pretty coherent way of doing this! And the hierarchical links, well, take a look at the left hand side of this page, I think they are good. Update: Thinking about the two formatting goals above, I realize they are both concerned with limiting the amount of information presented in order to maximize the amound of information communicated. Funny... And the archiving system I have vaguely in mind could be thought of along the same lines too.
posted morning of April 29th, 2003: Respond
| |
Monday, April 28th, 2003
This morning I put in date formatting -- it still needs more testing but all in all seems to work pretty well. Places I want to go in the near future: Archives, subject filter, view by date. Long-term goals: comments!
posted afternoon of April 28th, 2003: Respond
| Previous posts about The site
Archives  | |
|
Drop me a line! or, sign my Guestbook.
•
Check out Ellen's writing at Patch.com.
| |
