In Valid Logic

I’ve been working on finding my way through Rails, since learning Ruby/Rails has been on my list to do for a while now. Naturally, when I decided to dive in, naturally wanted to go out and buy the books that come highly recommended for it. So I went out and bought Agile Web Development with Rails and Programming Rails (aka, the pickaxe book), both by Dave Thomas. They both came highly recommended, and while I’ve only started on Agile Web Development with Rails, I am having a hell of a time wading through the thick propaganda in the book. Seriously, the book creates this aura where before Rails, we were in the dark ages of the internet and it is just this golden light, bringing logic and reason to everything.

The most striking example of this comes when it starts talking about application design and how Rails is Agile and mentions Use Cases:

A use case is simply a statement about how some entity uses a system. Consultants invent these kids of phrases to label things we’ve all known all along–it’s a preversion of business life that fancy words always cost more than plain ones, even though the plain ones are more valuable. (Ch 5, page 55)

Ooo, “use case” goes along with that big bad W word–Waterfall.

Please, I am reading this book for education, not for propaganda. I don’t need to be preached to about the evils of waterfall, the saving grace of agile, and the huge corporate conspiracy that is every language not named Ruby.

Of course, it continues, just a mere 10 pages later, find this when talking about handling deployments and databases.

Many people like using interactive tools to create and maintain schemas. The phpMyAdmin tool, for example, lets you maintain a MySQL database using web forms. At first sight this approach to database maintenance is attractive… However, this convenience comes at a price… It also makes it hard for yo to deploy your application you have to remember to make the same changes to both your development and production databases (and we all known hat if you’re going to fat finger something, it’ll be when you’re editing the production schema). (Ch 6, page 64-65)

Even long before Rails came about, I have never been on a project where changes were made to production databases through a GUI. Even back doing ASP, before .NET and Rails came about, I worked for the government and we always pushed changes through a controlled process. We made changes directly to a dev environment, scripted them out, pushed to stage, tested, then pushed to production. I didn’t even have any type of write access to production. I could only do selects and run stored procedures, couldn’t do any direct table insert/update/delete commands.

Even in Community Server, when Rails was first coming about, we had a database schema versioning system in place. Rails didn’t invent it, pretty much everyone was already doing it, they were just the first ones to add it to the framework.

This book is driving me nuts. I am reading it rolling my eyes as I go. I recognize Rails has done a lot for convention, and that it strives to build in all these things aimed at making the developer’s life easier. But don’t preach. I think I might just shelve Programming Ruby, since it is also by Dave, and instead moving onto something else. I’ve heard The Rails Way is highly recommended, maybe give it a shot.

Humans are never perfect, and since humans write documentation, it isn’t always 100% accurate.

Take the documentation on MSDN regarding Managed Thread Pools, in the “Maximum Number of Thread Pool Threads” section (emphasis mine):

The number of operations that can be queued to the thread pool is limited only by available memory; however, the thread pool limits the number of threads that can be active in the process simultaneously. By default, the limit is 25 worker threads per CPU and 1,000 I/O completion thread.

25 threads per CPU? Not bad, pretty good. Except it is a little bit off. In fact, it is missing a 0. Yes, it isn’t 25 threads per CPU, it is 250 threads per CPU. An error of 10 fold.

This has reared its head a few times with the Community Server add-on Mail Gateway. It makes use of the thread pool to process email messages as they come in and send them to the CS site. One problem was that when there is a backlog of messages, it can hammer the site quite a bit. If you have some issues connecting to the mail server, or configuration issues, etc, and messages build up in the mailbox (ie, couple thousand, or some clients 50k+ messages), then when things get resolved, it tries to process the whole back log, but ends up hitting the site so hard it will fail processing messages and continue to retry them, making it work through the messages very slowly.

When Mail Gateway was written, I had read about the “25 threads per CPU” and figured CS could surely handle a maxed out thread pool no problem… essentially 25 users on a site? That’s not bad. And if they had a dual CPU machine, thats 50… that’s maybe pushing it, but still should work ok.

However, we first starting using the managed thread pool like 2 years ago. Since then, we’ve seen a major change with multi-core machines.

So now, look at the issue of hammer a site with Mail Gateway. 250 threads? All hammering on a site? Ohh yeah, I could see that one causing a problem. But what if the client has a multi-core machine? Say it is a dual quad-core CPU box… you’re looking at 8 CPUs as far as it is concerned. You’re talking 2,000 threads. Think 2,000 messages being submitted concurrently to a site would bring it to its knees? Yeah, I think so.

The 25 vs 250 thing is pretty big, especially when you take into account the “per CPU” bit.

So the solution? Override the number of threads available by specifying it in the application on start up using ThreadPool.SetMaxThreads(). This way, we can also avoid the scaling per CPU issue as well… we don’t want to send more messages because there is another CPU, we want a fixed number. .NET 1.1 (when we originally wrote the code) didn’t have a method available to set the number of threads, but with .NET 2.0, we now can. Expect an updated build of the Mail Gateway service soon with the change. It will now default to only 15 threads, but can be configurable in the mailgateway.config file.

Bookmarking has been extended in several respects with sites such as del.icio.us, which introduces a social aspect into sharing and discovering bookmarks from other users, and then you have your “personal bookmarks”, the ones you find under the “Bookmarks” menu item of your favorite browser.

Traditionally, browser bookmarks have always served two purposes: quick access to frequent sites/pages, saving or archiving previously “found” material.

The access to frequently visited sites is one that has always been useful, and will continue to be. Even with the usefulness of auto-complete functionality when typing an address into the address bar, having a quick links browser bar, or list of sites quickly accessible with a few clicks is sometimes still easier. Perhaps you like getting to the Business section of your local newspaper without having to type it in or click through a few pages to get there. You can easily link right there.

The area where I feel that bookmarks fail me is with the archiving of pages I’d previously found and saved as reference type material. In reality, this is what a lot of my personal bookmarks are, especially the ones I don’t have neatly organized at the top of the list, or in little folders. I might find some nice, in depth blog post I want to read, but don’t have the time to right now. Or find an article on some coding ideology, or a list of network ports common servers use, or some process for working with photos for my photography hobby. These are things that I want to come back to later and keep around as things I might use at some point. There are several flaws with this system though.

First, they are not permanent. These links are often individual pages buried within a site, not the front page or a prominent section. Because of this, they can fall victim to the evolution of the site. Maybe the owner redesigns there site, causing the location to change. Your bookmark is now invalid and you might have to find the new location. Maybe they decide to start fresh, changing blog software of something and not importing old posts. Your content is now gone. Or maybe the sites goes away, domain expires or something, then the page is lost forever as well, and you don’t even have a way to now email the author to see if they have it saved somewhere. I used to have a list of several bookmarks to different techniques for making black and white photos. Unfortunately, when I went back to some of them, the links no longer worked.

Second, they are not searchable. If your bookmarks are searchable, it is likely only searchable by the title or URL. Perhaps you remember reading some stat or article and only remember a phrase from it, or something similar. You have no way to quickly find where that was from unless you visit each of your bookmarks and search the page (assuming the links are still valid).

And thirdly, the organization of bookmarks doesn’t scale. Most people quickly bookmark a link and move on, so when the thought does come to them about organizing them, they now have a nice pile of unorganized stuff. Sometimes, organizing never gets done because of the size of the task keeps growing, so getting it done becomes a dreaded task. Most browsers just have basic folder system for “categorizing” bookmarks, but this often isn’t descriptive enough and can be vague. You might have to develop your own folder structure to nicely describe links.

There are different strategies that try to remedy some of these. For instance, many people pipe their saved links through del.icio.us, since by going that route, you can tag your posts (which is a great way to solve #3), but can fail elsewhere. You can enter a description, but then you can only search that description, not the full-text of the page, so #2 is only partially resolved. And it is of course still victim to #1.

Backpack is extremely useful, but it is primarily plain text. You cannot cut & paste, or easily preserve, the full rich-text feel of the page. Preserving images is an absolute must. For my photography tips I mentioned earlier, photos of the steps would definitely need to be kept.

OneNote is an excellent program, but it too tends to lack in the rich-text department. You can send documents to it, but most of the time it will essentially go through a virtual printer. Print format doesn’t always look as good as the original. Quite often, I hate the printed format of a webpage. It usually isn’t as pleasant to read or as friendly. Also, no Mac version.

Then there are several journal/library type applications for both PC/Mac other than OneNote, but most are focused around note taking and keeping snippets, but it is kind of like none were designed for the specific purpose of archiving a webpage. Those that support rich-text formats would likely handle this in kind of a cut & paste fashion, but then you still tend to lose formatting.

What is really needed is a stronger archiving/library type program/site that allows you to fully preserve what you saw the day you first visited the site. The one application I’ve found that lets me do pretty much everything I’m looking for is DevonThink by Devon Technologies. It has an AppleScript plug in that allows you to send a webpage you are currently viewing to it as a “web archive”, in which it will basically “slurp” the entire page and preserve it within its database. Then you can come back and view it as if you were looking at the site itself. It is rather pricey though.

I am wondering what else is out there though. Anyone else know of something that could achieve this? Perhaps an online application? I love the idea of Backpack, where it is essentially cross platform and available anywhere, but I just haven’t been able to find anything that gives that “slurp” type functionality to preserving a page. Part of it might be copyright issues… IE, one thing to save a page yourself for personal use, though a website might have issues with them saving it for someone else. Not sure, I’m not a lawyer (thank god). Anyone have any suggestions?

With Apple’s announcement today of its iTunes Movie Rentals, I was rather quickly disappointed that it will suffer from the same thing that Xbox Live’s movie rentals have: 24 hour viewing time.

With Xbox Live, when you download the movie, you can watch it anytime within 14 days. However, once you click play and start watching it, you must finish it within 24 hours. With iTunes, they’ve changed it a little where you can watch it within 30 days, however you still must watch it within 24 hours once you start playing it.

I understand the 14 day / 30 day portion… well, at least with iTunes. You might rent a movie and put it on your iPod or something to watch on a trip. That would be useful (assuming you can start the movie after downloading without an internet connection, who knows). With the Xbox, you can’t really transfer it anywhere, so mute point.

Really, what is the logic with this 24 hour business? This is nothing new, since Microsoft was doing it before Apple, just Apple is doing it more to the masses. 24 hours is simply not enough. What if you normally only get to sit down and watch TV/movies at 8pm and for some reason you get interrupted (phone call, kids, falling asleep, whatever), so you don’t get to finish the movie. Well, the next night you are basically toast since if you can’t get back to finish it before 8pm (your normal time), you’re screwed.

The 24-hour limitation completely negates the on-demand convenience. You now have to finish the movie that night. So don’t let your kids wake up. Don’t you dare fall asleep.

Some are calling for a 27 hour rental, others 48-72 hours. I think 48-72 would be nice, though likely don’t need that much. Being able to resume watching the movie at the same time the following night would be enough. 27 hours would be enough. Ideally, I’d vote for 36 hours. It gives you extra flexibility as you could basically start watching it anytime the first night and finish it anytime the following night.

Will change come? Hopefully. Until then, the idea sounds cool, though is definitely a major turn off. Maybe Apple is counting on you not finishing it and having to re-rent it, just like Blockbuster used to do with their “due by noon” crap.

In line with my New Years resolutions, I’ve been working on re-implementing GTD (Getting Things Done) throughout my life. So I’ve been evaluating GTD applications for OS X lately to figure out which one fits my needs the best. Overall, I was looking at three main contenders: iGTD, OmniFocus, and Things. After putting each through the tests since basically the day I brought my Mac Pro home, here is the conclusion I’ve reached.

iGTD: The first one I used, and liked it, though I was kind of disappointed with their UI. The main interface is just a little too busy. Then I checked out the preview information on iGTD2 and was completely turned off. It added a bunch of fluff I didn’t really see useful to my needs, with even more clutter. Based on where the application was headed with iGTD2, decided to steer away.

OmniFocus: OmniFocus is poised to be released soon and is certainly impressive. It moves away from the “spreadsheet table” listing format in iGTD and has a nice, minimal interface. It has a nice way to organize projects and also has “single-action lists”, which are kind of like projects that are never done (I use these for stuff like bugs, support issues, life, home improvement). I was a little disappointed it didn’t really support anything for priorities. iGTD had 5 different levels, and while I didn’t need that many, it would be nice to highlight some as high/medium/low. All it has was the order you put them in or you could flag certain ones. Flagging was a toss up though. I liked using flagging to mark ones I was going to focus on today, then narrow my view for the day to just those, but then I lost the way I was thinking of marking items as important. Important doesn’t always equal now, it just means “do before the others” and “probably don’t want to forget”.

Things: I had almost settled on OmniFocus, but then I heard about Things from a smaller software company that was having a private alpha. I had emailed the company to ask about getting in on it and they graciously added me to the alpha. After using it for a little over a week, I am truly impressed. Instead of focusing on the traditional project/context type structure other GTD applications use, they went with a project/tags concept. First you have projects and areas (areas are equivalent to OmniFocus “single-action lists”, projects that never end), which are just the same as in other applications, but then it adds on a simple and powerful tagging system.

Essentially, for most the tags equate to the context of the task (ie, @Home, @Computer, @Errands), but it can also be used for other things. I will also use the tags to mark some items as high priority or low priority, and it also includes some default tags for signifying effort (ie, show an action is 15mins, 1hour, or longer).

In addition, they also have a set of “focuses”. Items you want to work on today can get flagged as “Today” and show up in a list of items marked accordingly. You then have the Next view, which shows an aggregate list of your items. And then you have Postponed and Someday. Some items you can’t work on now and don’t want to be bothered with them, so you can postpone them a week and they’ll be hidden from your normal view (though can see them again by clicking a button at the bottom of the project view) until it is time to deal with them. Someday items are ones that are basically hidden as your “someday I’ll do this”. These are things that aren’t really things you need to do, but would like to do at some point in the future. IE, want to climb Mount Everest? Mark it as a Someday. Have an idea for a cool program or site to make sometime? There you go.

Overall, Things was the one that was most in line with how I viewed my own GTD system. Think of something while in the middle of coding? Hit Control-Option-Space, up comes the quick entry box. I can quickly enter an item and hit C 1 to tag it as coding and high priority (you can assign shortcuts to tags too!), and I’m done. Each morning, I look through my Next view and mark things I plan on tackling today as Today, then I switch to my Today view and that is where I stay all day. Today is better than using due dates, since if I don’t get something done that day, I don’t want to see red text or get warnings something is overdue. I want to use due dates on things that are actually due by a certain time, like remember to pay gas bill by the 5th. Otherwise, due dates are arbitrary. Laundry isn’t due today, if I don’t get it done today, so what, it’ll be on my list tomorrow still. Additionally, with the tags, I then get to mark items with priorities. I don’t need the priorities so much for sorting and data purposes, but just rather when I look over my list so I can see if one is high or low. Additionally, OmniFocus didn’t seem to have as much on the Postponed/Someday concept other than having things in a @Waiting context.

Things is currently in development, and while they still have a lot of must-have features to add (syncing, recurring tasks, etc), the features and methodology they already have certain make it a must-have for me. They recently released a public preview and would highly recommend giving it a look. I’m glad I did, since I just threw OmniFocus into AppZapper and will be sticking with Things. Also have a nice list of feature requests I’ll be sending off which hopefully get included.

It is that time of the year again. It is the season for mass gym sign ups and diet crazes that last a month and then fizzle out. But sometimes, come up with some truly good goals and I’m able to stick to them. This year, my list of goals for 2008 includes:

• Wake up earlier, and get ready earlier: Especially when Nick was born, I’d started sleeping in later in order to make up for the time lost awake in the middle of the night feeding. Now, he is sleeping through the night, but want to push myself to wake up even earlier than I did before. Previously, I got up at 7am. When Nick was born, it became 8am. Now, I want to go for 6am. Additionally, I want to get myself to get ready for the day earlier (ie, shower, dress, shave, etc). Often, I wake up and go right to the computer and begin work, and am glued there until like 11am or even later when it occurs to me I should shower. The ability to have this lifestyle is nice, but gets in the way when it is noon and you want to run out and get lunch, but look like the living dead. Or, I have a self imposed deadline for going to Starbucks (otherwise it throws my lunch time off), so sometimes it’ll close in on the deadline and I want my mocha/latte, but still have to shower. Plus, when up early, dressed and ready, I feel more productive.
• More productive with my afternoons: Not to say I am not productive in the afternoon, but this week Trish started her new job and her new job is only working mornings. Her old pre-school turned into a daycare after “class”, but this one doesn’t. That means some days I’ll have Trish and Nick at home, which can be distracting. Usually, it is in the form of TV and just other general noises, but minor interruptions can quickly zap productivity. I’d like to find ways to get more done after lunch. Perhaps spending some afternoons working at a Starbucks, or go to Panera for lunch and stay afterwards working, or just unplug and go to the library. Or, if at home, put less “deep-in-thought” things for after lunch, like smaller tasks: minor/medium bug fixes, organizing, emails, follow up on support issues, etc.
• Go All Mac: Sadly, I am becoming addicted. By the end of 2008, I think I’d like to be all Mac. Still have my WHS box, still have my server on Windows (hopefully Windows Server 2008), but likely toss my Vista MCE box (the digital cable conversion may mean greater investment than its worth). This basically leaves a MacBook for Trish and a MacBook Pro for myself. Perhaps I should figure out a savings plan.
• Simplification of life: Sounds kind of vague, but with all the changes in my life in the past year, things keep getting complicated. From a year ago to now, have moved into a house and got a baby to take care of. Responsibilities have taken a huge jump to include yard work, raising a child, planning for not just our future but his, and so on. The sources of items for my to-do list are much bigger. This year, need to re-implement GTD especially in my personal life, and find ways to more efficiently manage my list. Not so much reserving time slots to do things, but finding ways to streamline them. So instead of always borrowing my father-in-law’s hedge trimmer, get my own. Instead of manually paying bills (or queuing them with online bill pay), maybe set them up on auto-pay.
• Lose weight: Yes, the cliqued resolution, but in reality, I do need to. Really, working from home leads to a fairly seditary lifestyle, and after 3 years of it, the affects are undeniable. Losing like 50 pounds would be excellent, though even just 20-25 would make a huge difference.
• Keep the coffee maker clean: I am notorious for leaving used filters in the coffee machine, then they get gross, and then I have to do a big cleaning of the coffee maker before using it again. I love fresh brewed coffee in the morning, and would certainly save me some dough over going to Starbucks. It is just a matter of remembering to take out the used filter when I’m done, and quickly rinsing/washing the pot.

One of the biggest changes to get used to in switching from Windows to a Mac is some of the different keyboard shortcuts. Some things like Command-C instead of Control-C for copy/paste is easy to adjust to, but others like how the home and end keys work are totally different.

On Macs, the home and end keys will go to the begining/end of document, however I still desire a way to easily go to the beginning/end of a line. I use this plenty when posting online, writing emails, and even coding. You can do Command-Left/Right to do the same, however if you are typing in a browser, doing Command-Left will also navigate away from the page, basically losing everything you just typed. Never like that.

Luckily, you can fairly easily change the keys behave like you’re used to Windows. Just create a file in /Users/[username]/Library/KeyBindings called DefaultKeyBindings.dict (create the folder if not found) and change the keys as you like. I copied the first blurb in the “Examples” section of this page. Or, here is another more thorough example.

One thing to note is that the changed key bindings don’t work in Firefox or Camino, so the Control-Left is still there, but it does work in Safari… but then in Safari, seem to run into issues with WYSIWYG editors on many sites (yes, including Community Server and Graffiti). I’m not sure whether or not I love or hate Safari.

PS: This post was written in Safari with proper functioning home/end keys (ahh, heaven). The key to using it is to remember the site is expecting HTML.

Update: The next beta of Graffiti will support WYSIWYG editing in Safari according to this. The current Beta1 release doesn’t support it, though.