Revolution Systems Blog

Some quick updates

It's been a busy and exciting week for us.  Jacob has been at PyCon in Chicago where he is participating in a number of panel discussions and giving quite a few talks as well.   Right now I imagine he's neck deep in code in the Django sprint helping to finish up the upcoming 1.1 release. If you're running a production site built with Django you should absolutely check out the talk he is giving with James Bennett on Real World Django.

While my week has been busy hacking away on several client projects and moving my main work machine to a shiny new MacBook Pro (can't recommend these highly enough), I was interviewed by Daniel Dern of Business Trends Quarterly in his post about scaling and performance titled For Scaling, Brains May Beat Brawn. We talk about how just throwing more money and hardware at a problem is not always the best solution.  Often there are architectural, design, and/or configuration changes that can bring significant cost savings to your project.  Both in terms of the hardware necessary to keep everything flowing, but also in on going system maintenance labor costs.  I'm not talking about pre-optimization evils or complicating things for your admin, often these changes are transparent to day to day operations, but certainly not to your bottom line. For example, just using the proper RAID levels and physical disk configurations for your particular PostgreSQL database can be a huge win in performance.

I also added a tidbit of wisdom in an advice post to budding entrepreneurs called 163 Ways How To Become An Entrepreneur.

 

  

Welcome Jacob Kaplan-Moss

I'm very pleased to announce that Jacob Kaplan-Moss has joined Revolution Systems to head up a new line of services around the ever growing Django web development framework. First up are commercial Django Support Plans, but look for more Django related offerings in the near future.

Jacob has been a good friend of mine since before Django was even released.  It was a pleasure to work with him at our previous day jobs and I'm very excited for the future ahead.  Not only is he obviously an authority on Django, he's an amazing developer and generally an expert on all things tech. Jacob and Adrian are both great examples of how to lead an Open Source project and grow a real community around it.

By offering Django Support packages we hope to help adoption of Django in the business world, which helps grow the community at large.  

ORD Camp a Huge Success

I was luck enough to be invited to attend ORD Camp this last weekend in blisteringly cold Chicago.  ORD Camp is an invite only, FooCamp style unconference targeted at geeks living in the Midwest. Having never attended a FooCamp style event I wasn't sure what to expect.  I can now say if you ever have the opportunity to attend an event like this it is well worth your time.

As you can see from the attendee list it was a very diverse group of people, not just the usual crowd of notable Open Source geeks.  The amount of brain power in that room was simply amazing and I can't remember when I had as much fun.  Some sessions were presentations, others were just focused discussions.  Everything from how words work, brewing beer, life hacking, to what not to do as a startup. 

While I loved the sessions the most fun was getting into random conversations ( some ended up being NSFW after midnight and many beers ) others were more typical.  Spent some time talking with people about PostgreSQL's advantages over MySQL, alternative business models, how a certain entrepreneur might improve the performance of their servers, etc.

It is difficult to determine how important this conference will be to my business in the future, but I can easily say that it has increased my drive, ambition, and overall excitement level.  Passionate people, doing amazing things will do that to you! I can't wait to attend next year.

ResumeBucket.com Launches

I hope everyone had a great holiday this year.  For the past few months I've been working on an online resume site ResumeBucket.com and I need your help taking it for a test drive.  Our goal with this site is to create a site where you can upload your current resume in Word form, build a new resume using our online resume creation tool, or even just type in what you want using using our online text editor. 

The site gives you a unique URL you can give out to friends and prospective employers so they can instantly access an up to date copy of your resume.  There are options for them to also download a copy of your resume in Word or PDF format.  For an example of how your resume can look see our CEO's resume page.

Unlike the other resume services out there, employers are able to search the database without paying any huge fees which will drive more qualified employment leads to your INBOX.

Please take a few minutes to kick the tires.  You can leave feedback here in the comments or E-mail me at frank@revsys.com. Thanks!

Why isn't PostgreSQL using my index?

This is a common question I see from PostgreSQL users.  In fact, someone was just in IRC asking it and it prompted this post.  The exchange usually goes:

Steve: I have this column foo and I have an index on it, but when I do SELECT * FROM table WHERE foo = X, EXPLAIN doesn't use the index.  What am I doing wrong?

90% of the time the answer is unfortunately "Because the query optimizer is smarter than you are". Or maybe it's fortunately depending on how you think about it!

In this user's particular case he was mocking up a database schema and had only one row in the table he was querying against.  People who are more familiar with PostgreSQL will probably roll their eyes at the question, but if you put yourself in the user's shoes I can see how people would be confused by this.  They are thinking, "I put an index on there on purpose, why the hell isn't it working?". 

PostgreSQL's query optimizer is smart, really smart and unless you have evidence otherwise you should trust what it is doing.  In this particular case, the optimizer realizes that if a table has only a few rows that using the index is actually slower than just spinning through the entire table.  Just because PostgreSQL isn't using your index today with a small number of rows, does not mean it won't choose to use it later when you have more data or the query changes. Because he was just mocking up a design he didn't have real world data, which is almost always a bad way to performance tune your system unless you are very familiar with how PostgreSQL behaves. 

Now there are other reasons why it might not be using the index.  If you have lots of data and the query you're running appears that it would benefit from the index, it might be a simple matter of forgetting to run an ANALYZE on the table or not having autovacuum turned on.  Until PostgreSQL re-analyzes your table it doesn't really "know" about that index to take it into account when building the query plan. 

While performance tuning PostgreSQL is much easier and better documented than in days gone by, it can still be very confusing and time consuming for the inexperienced.  If your business needs help tuning their system you might consider my PostgreSQL Tuning Service.

Fret Free -- Introduction to Django and the Django Software Foundation

LinuxPro Magazine just released my latest article, an introduction to Django and some discussion about the newly created Django Software Foundation. Being a life long Perl user, I didn't think I would enjoy Django at all. I have to admit that it is a VERY polished system.  It has great PostgreSQL support, in fact the core developers smartly prefer it over MySQL for their own systems.

You can download a PDF copy of the article at, Fret Free -- Django and the Django Software Foundation.  The print issue will hit the stands in October.  Hope you enjoy it!

Installing Apache2::Request on a 64-bit system

I usually shy away from using 64-bit systems unless there is a clear need for it.  But with more and more hosting companies installing 64-but Linux distrobutions by default I sometimes have to suffer through.

Today I ran into a very small bug in installing Apache2::Request modules via the CPAN shell.  This simply installs the default libapreq2 library with the Perl bindings for you. When attempting to start Apache I received this error:

Can't load '/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/APR/Request/Request.so' for module APR::Request: libapreq2.so.2: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230. at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/APR/Request/Param.pm line 27 Compilation failed in require at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/APR/Request/Param.pm line 27. BEGIN failed--compilation aborted at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/APR/Request/Param.pm line 27. Compilation failed in require at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Apache2/Request.pm line 2. BEGIN failed--compilation aborted at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Apache2/Request.pm line 2.

Basically it is saying it can't load libapreq2.so.2.  Digging around in the system I found it had installed them in /usr/lib instead of the proper /usr/lib64.  The actual shared library was compiled for 64-bit, but was just installed in the wrong location.  A quick symlink like this fixes the isuse:

ln -s /usr/lib/libapreq2.so.2 /usr/lib64/libapreq2.so.2

Hope this helps you get over this small issue.

Django Software Foundation

The Django guys have scored another milestone today with the creation of the Django Software Foundation. Being a huge Open Source advocate I love hearing great news like this.  You can read more about the foundation at:

• Announcing the Django Software Foundation
• New Foundation for Django

On an unrelated note, an article I wrote for LinuxPro Magazine last November about using Perlbal was recently put online.  You can read it at The Juggler -- Let the nimble Perlbal webserver keep your traffic in balance .

Test Driven Development and Getting Things Done

I'm sure someone, somewhere, has already determined this.  It's probably been blogged about before, but I just realized something today.  Test driven development dovetails nicely with David Allen's time management book Getting Things Done.  By creating your tests first, with or without even marking them as TODO tests, builds you a TODO list of sorts for your project.

Other than just being a different way of doing things, I never really took to test driven development for small or personal projects.  Most of my projects are web applications and "unit testing" modules never seemed worthwhile because you weren't easily able to test the database and web interactions.  Well I can easily say that Perl Testing: A Developer's Notebook by Ian Langworth and chromatic has shown me the way.

I had always focused too much on ensuring that 'make test' would be able to be run from anywhere and simply did not think about the fact that I could automate most of the work of building a test environment and if I wasn't able, to bail on those tests so they would be skipped.  For example, you could automate creating your PostgreSQL database, but what if PostgreSQL isn't installed on the server or what about when your user isn't allowed to create databases?  You just bail out of those tests.

I was definitely too concerned with getting 100% of the tests to pass 100% of the time, in all possible scenarios now that I think about it. 

Want to be a better manager?

I've been reading a great book recently titled First, Break All the Rules: What the World's Greatest Managers Do Differently. I highly recommend it for anyone who manages employees, but it makes two great points early on that are especially appropriate for technology managers:

Treat your Employees Differently

You should treat your employees differently.  Each has unique strengths, weaknesses, and differ in the way they learn and you should capitalize on that, not try to homogenize everyone. Sally might be an awesome systems architect, so you don't want to weigh her down pounding out mundane features.  Bob might be the fastest developer and giving him those same mundane features makes sense.  Steve on the other hand might be your more efficient debugger, etc. 

They learn in different ways, some by reading, others by doing, and some need a more traditional classroom environment. Sending a do-er to a lecture class is a waste of everyone's time. 

Don't Let Human Resources Get in the Way of Your Hiring

I've touched on this topic before, but you can't leave the resume collection and filtering process to your HR department.  Even in our more tech savvy age, I still hear horror stories from programmers how they lost out on being interviewed for a job because the HR person didn't know J2EE or J2ME were 'Java', that Ubuntu is a Linux distribution, or that having 6 years of mod_perl experience probably means they know Perl pretty well.

A great technology manager has to hire a person based on several things, one of which is experience with a particular technology or technologies.  But that's only part of it.  Troubleshooting skills, breadth of experience in many technologies, personality and how this new hire will fit into the overall team structure are just as important.  The worst possible case is when the manager is barely or not at all involved in the hiring process.

Now there are exceptions to this.  The best example is my friend Josh Stomel who runs NeoHire. Josh is one of those rare recruiters that actually understands technology and more importantly knows when he doesn't know and has a huge network of geeks like me to ask questions. You can just hand NeoHire a list of qualifications and they will bring you back a group of top notch talent.

So it is possible to have HR deeply involved in the hiring process, it just has to be the right people.