March 11, 2008

Want to be a better manager?

I've been reading a great book recently titled First, Break All the Rules: What the World's Greatest Managers Do Differently. I highly recommend it for anyone who manages employees, but it makes two great points early on that are especially appropriate for technology managers:

Treat your Employees Differently

You should treat your employees differently.  Each has unique strengths, weaknesses, and differ in the way they learn and you should capitalize on that, not try to homogenize everyone. Sally might be an awesome systems architect, so you don't want to weigh her down pounding out mundane features.  Bob might be the fastest developer and giving him those same mundane features makes sense.  Steve on the other hand might be your more efficient debugger, etc. 

They learn in different ways, some by reading, others by doing, and some need a more traditional classroom environment. Sending a do-er to a lecture class is a waste of everyone's time. 

Don't Let Human Resources Get in the Way of Your Hiring

I've touched on this topic before, but you can't leave the resume collection and filtering process to your HR department.  Even in our more tech savvy age, I still hear horror stories from programmers how they lost out on being interviewed for a job because the HR person didn't know J2EE or J2ME were 'Java', that Ubuntu is a Linux distribution, or that having 6 years of mod_perl experience probably means they know Perl pretty well.

A great technology manager has to hire a person based on several things, one of which is experience with a particular technology or technologies.  But that's only part of it.  Troubleshooting skills, breadth of experience in many technologies, personality and how this new hire will fit into the overall team structure are just as important.  The worst possible case is when the manager is barely or not at all involved in the hiring process.

Now there are exceptions to this.  The best example is my friend Josh Stomel who runs NeoHire. Josh is one of those rare recruiters that actually understands technology and more importantly knows when he doesn't know and has a huge network of geeks like me to ask questions. You can just hand NeoHire a list of qualifications and they will bring you back a group of top notch talent.

So it is possible to have HR deeply involved in the hiring process, it just has to be the right people. 

February 29, 2008

Building mongrel on Solaris

I keep running into this same situation, so I figured I would write a post for everyone's benefit.  When trying to build recent mongrels or some other gem dependencies such as fastthread on a Solaris system the automatically generated Makefile is all sorts of messed up. 

This testing was done on two different Joyent Accelerators, your milelage may vary on other Solaris installations.

The steps are pretty easy:
1. Attempt to install the gem via the normal 'gem install mongrel'
2. This will fail, but will download the gem for you.
3. Move into your gems directory.  On these sytems that is /opt/csw/lib/ruby/gems/1.8/gems/
4. Move into the particular gem you are building that is causing you trouble. In this case mongrel-1.1.3
5. Move into the ext/ directory and the subdirectory that can't be built.  You'll see it in the error messages from step 1.
6. Edit the generated Makefile as such:

    Make sure CC is pointing to your C compiler, I had to edit this to reference gcc rather than cc.
    Make sure LD is pointing to a real ld tool.  On this system that meant I changed ld -G to gld -G
    Edit CFLAGS to be CFLAGS= -03 -I/path/to/libs Essentially you just need to remove the -KPIC and all     -x es from the generated file.

7. Run make install
8. Copy the gemspec file to /opt/csw/lib/ruby/gems/1.8/specifications so it will be registered with the gem tools.

Hope this helps.

February 04, 2008

PostgreSQL version 8.3 Released

I just got word that version 8.3 of PostgreSQL has been released.  Along with the usual amount of improvements there are some new features in 8.3 that should be of interest to PostgreSQL admins and developers such as:

  • Integrated TSearch
  • ENUM and UUID data types
  • Faster sorting technique used for LIMIT operations
  • Faster LIKE and ILIKE operations
  • Lazy XID assignment which will make many read only operations much faster

Check out the full list of features at the PostgreSQL site or download it from the download section of their site.

January 24, 2008

EveryBlock.com is now launched

My friend and former co-worker Adrian Holovaty and his team just launched their new project EveryBlock.com. EveryBlock takes the term hyperlocal to a whole new level.  They aggregate tons of public data sources by geo location so you can for example find all of the recent crime around a particular address, neighborhood, zip code, etc.  Or maybe you might be interested in the building code violations of where you live or work?

Right now they have San Francisco, Chicago, and New York up and running, but will be adding more cities as time goes on.  Adrian asked me to help performance tune their PostgreSQL database a couple of months ago and so far things seem to be humming along nicely.

Here are some links to other blogs talking about EveryBlock.com:

Congrats and the best of luck to EveryBlock! I'm sure we'll see even more new and interesting things from this team in the future!

January 18, 2008

Some interesting links

Hope all of my readers have recovered from the holiday season.  Here are a couple links I've come across recently, but neglected to write about during the holidays.

Zed Shaw, author of mongrel which is used by many Ruby on Rails applications, posted an interesting rant about the state of Rails development and the personalities of some of the major players. 

I've never been a huge fan of Rails because I've never been shown a compelling reason to switch away from the more mature Perl equivalents.  Lately I've been working with mongrel and Rails apps more and more as many of my clients have switched to that as their platform of choice.  The more I work with it the less fond of it I am, it really is just immature compared to other platforms out there.  Toward the end he mentions the Rails creator himself has an application that had to be restarted 400 times per day... with some fixes it only has to be restarted 10 times a day.  400 is just insane, but 10 is still very troubling.  If the creator can't make things run better than that it really doesn't speak well for the platform.  Normally I wouldn't link to such a huge rant, but when it comes from a major Rails contributor there has to be at least some truth to it.

My friend Stas Bekman has created a new site where you can make it known how you want a better version of something.  It's called i-want-a-better.com.  I think it's a great way for people to vent their frustrations with products, but I think the really winning idea is that entrepreneurs can peruse the site for new products and untaped markets. Hop on the site and let us all know what's frustrating you!

 


January 17, 2008

Making the software written in any language more readable

There are two very simple ways to improve the readability and maintenance of the software you write. They are so simple they are often ignored in favor of more complicated tools and the various programming methodologies people blather on about.  This comes from our human nature to think our own problems are more special and complicated than they really are and from not following the KISS principle.

So how do you improve your software?

By using better names and being consistent. It really is that simple, which is probably why it is overlooked.  A development manager might score some points with his boss by switching to Agile programming, having some scrums, doubling the amount of developer documentation or maybe even switching to a whole new platform like Ruby on Rails.  But who scores any points by saying, "We're going to do better about naming things appropriately?"

Appropriate Naming

Obviously having a variable named temp_user isn't all that descriptive, but it is an improvement over just temp or simply t. Most, if not all, programmers realize this.  However, you will often see variables named clt when they should really be named client, as if those three extra characters were single handedly going to cause carpal tunnel.  Your variable names should be as descriptive as possible without being absurd about it.  A variable named the_current_user_object_we_are_working_with is obviously overkill. But perhaps current_user_object or even current_user is appropriate.

Naming your functions and methods should also be given the same amount of care. Naming a function fix_client doesn't tell us anything useful, we can only speculate something is wrong with the client or the data and this function does something about it.  normalize_client_name is a much better name when you read that all the function does is properly set the case of the letters in the client's name.

The names you choose for your libraries are also very important.  This is one of the reasons programmers find Perl's CPAN much more useful than other programming languages' library collections. Things are named and categorized (for the most part) sanely.  Need something related to E-mail, check the libraries in the Mail category.  Can you guess what Net::SSH, IO::File::CompressOnClose, and WWW::Google::News do? Yeah I thought you could.

If I stumble upon your use of a library called Util, it doesn't tell me anything I still have to go look at the library to see how it fits in with this code.  If it had been named something like  DB::Util or Client::Utils I would at least know the library is probably related to the database or client.

Consistency

Consistency is another area where you can improve your code base without much effort.  If all of your classes contain an initialization method, they should all be named the same.  Not initialize in some classes and init in others.  Things that should be consistent throughout your code base, not only consistent within a single application. If possible, the source in your department should be consistent with other departments and business units.

Consistency through conventions is one of the main reasons people like web frameworks like Ruby on Rails. I'm not advocating Ruby on Rails necessarily, you can accomplish these same things in any language.

Things that should be consistent:

  • variable, function, and method naming conventions ( underscores or CamelCase, but not both )
  • frequency and layout of comments
  • documentation
  • configuration file syntax
  • on disk layout of source code, binaries, configuration files, etc.
  • installation, upgrades and package management
  • language(s) used for development

The point being the more consistent you are in how you build an application the easier it is to get down to the task at hand.  Be it new development or bug fixing.  Got a configuration file format you like and have already written libraries to parse it? Then use it EVERYWHERE, in every single darn application you build. No one has to learn the new format to start developing, no time is spent discussing which format is better, and another developer in your organization can jump in to fix bugs or add a new feature.

The last development shop I ran we focused on being consistent. 95% of the time we were building web applications that were either used internally by fellow employees or externally by our customers.  If you saw the source to one of our applications, then all of our other applications would seem very familiar. One might be a ticket tracking system and the other an accounting package, but the layout, coding style, and use of common libraries let the developers dive right in and not have to worry so much about how this particular app is written differently than the others.

Consistency in your applications also makes refactoring easier.  If all of your applications use a particular technique, library, etc. in the same way replacing it with a new tool you have fallen in love with is much easier.  If everyone is doing everything even slightly differently, you have to start worrying more and more how the change might impact your code.

I'm definitely a "right tool for the right job" sort of fellow, but mixing and matching tools and techniques for every single application you build is a recipe for disaster. One shop I know (name withheld to protect the guilty) has two developers.  One who works entirely in Perl, the other entirely in Ruby.  Another application I saw was written mostly in Java, but with a smattering of C# and Python around the edges for kicks. None of these three were chosen from a "right tool for the job" perspective, but simply because those were the favorite languages of the specific developers tasked with those sub-systems. These are obviously worst case examples.

These ideas aren't new, I'm positive I am not the first to use them or even write about them.  But I see these two simple rules violated so often by programmers of all experience levels that I felt the need to reiterate them.


October 05, 2007

Log Buffer #65: a Carnival of the Vanities for DBAs

Welcome to the 65th edition of Log Buffer, the weekly survey of database related blogs.

First let's start with some miscellaneous entries that could be of interest to any DBA.  Crazy DBA has an interesting post about how attending conferences helped to grow his professional network, which in turn has made him a better DBA.  And Thomas Kyte has a great post about why it's the data, not the application itself, that matters.  Brian Aker gives us a great link to a Werner Vogels' entry on Dynamo, one of the key technologies used behind the scenes at Amazon.

Oracle users will certainly find these two links of interest.  First off, Frederik Visser shows you how to play with Oracle 11g RAC in VMWare. And Alex Gorbachev has a nice write up about Miracle Open World.

SQL Server DBAs might enjoy the following posts.  If you're thinking about using or upgrading to Idera SQLsafe v4.5, you'll want to check out Sean McCown's post about some of that product's issues. Steve Jones has some thoughts on monitoring and alerting with your SQL Server, but are valid for any database. Need to know when your SQL Server instance was started? Check out Joe Webb's tip on how to find out.  And Mladen Prajdić has some advice on how to notify a client in a long running process with SQL Server.

MySQL users will find this post on accurately measuring how far behind your slave is lagging. Over at the MySQL Performance Blog there is an opportunity to ask questions of Heikki Tuuri, the creator of InnoDB, and Peter has some thoughts on a few serious bugs in the MySQL 5.0 release.  Kaj Arnö has an interesting post on how MySQL GmbH and MySQL AB help birds of a feather to flock together, quite literally and about how they have opened up the call for papers for the 2008 MySQL Users Conference.

Kevin Burton talks about how to avoid swapping insanity with InnoDB. Want a free MySQL Magazine? Lewis Cunningham has found one for us all. Jan Kneschke introduces us to the Wormhole storage engine for MySQL.  Not really sure how useful it is, but it is definitely interesting.

Hubert Lubaczewski has written a great tool to help you determine the optimal layout of tables, indexes, etc. on your various tablespaces for PostgreSQL.  Robert Treat follows up with some additional thoughts to consider

Joshua Drake has announced the speakers and topics for the PostgreSQL Conference Fall 2007, which is October 20th 2007 at Portland State University. Greg Sabino Mullane has a nice explanation of why you can't used prepared queries when using DBD::Pg and pg_bouncer. And to finish our this week's  links, Francisco Figueiredo Jr mentions that PostgreSQL will have a UUID data type in version 8.3.

Enjoy!

September 20, 2007

Safely making changes to remote systems

I stumbled across this post about a great technique for safely making configuration changes to remote systems.  Everyone has done it before, the "Oops I just shutdown sshd" or turned off the wrong network interface. 

September 11, 2007

Under-used CPAN Modules

Perl often suffers from its history.  While it is great that there are so many online tutorials and code samples out there for new Perl developers to learn from, they sometimes miss out on all of the useful new code that has been created since the article was written.  Here are a few CPAN modules that are, in my opinion, under-used by new developers.

These modules are great for several reasons.  The most compelling ones are there is lesss code for you to write/debug than not using these modules, the resuling code is often clearer to your fellow developers, and in many cases the module will out perform your hand hacked code.

List manipulations

There are two list related modules you should familiarize yourself with.  They are List::Util and it's cousin List::MoreUtils.

Need to shuffle the values in an array? Don't reinvent the wheel, use List::Util's shuffle function.

use strict; 
use warnings;

use List::Util qw( shuffle );

my @array = qw( 1 2 3 4 5 6 7 8 9 );
my @shuffled = shuffle( @array );

Need to make a list only contain unique values? You can use the method found in the Perl Cookbook, or you can use the simpler uniq() function from List::MoreUtils like so:

use strict; 
use warnings;

use List::MoreUtils qw( uniq );

my @array = qw( 1 1 2 2 3 3 4 4 5 5 );
my @unique_values = uniq( @array );

Not to mention List::MoreUtils' implementation is done in C, so it is much faster than a Pure Perl implementation.  I once saw this improve the performance of a large web application by 5% because the developers were uniquing several lists for each page view.

Be sure to also check out the other functions in these two modules such as any(), all(), first(), max(), etc. While you will have to install List::MoreUtils from CPAN, List::Util is part of Perl core.

Merging Hashes

While you can use the simple:

my %merged_hash = ( %hash_one, %hash_two ); 

Someitmes you need a bit more power, enter Hash::Merge.  It gives you several options on how to handle conflicting keys, based on a left/right order, or by storage method. Or you can even control whether or not your data is cloned on not.  I find this very useful for merging in command line arguments against a configuration file.

Speaking of configuration options and files....

Repeat after me.  I will not write my own configuration parsing code.  I will not invent my own configuration file format. I will not parse my own command line options without a damn good reason.

Why waste your time on the most boring part of your code? Use one of the already existing modules and configuration file formats.  This saves you time, debugging headaches, and makes the configuration file syntax familiar to your users. 

I strongly suggest looking into Getopt::Long for handling command line arguments. It might take you a bit of time to get used to this module, but once you're over the initial learning curve you'll wonder why you ever bothered doing this by hand in the first place.

If you like Apache style configuration files (who doesn't?), start using Config::General and cut your configuration processing code into a use statement and a couple of lines of code. Or if you happen to prefer .INI style configuration files take a look at Config::Tiny.

August 30, 2007

Technique for improving code over time

I was talking with a friend of mine recently about an all too common problem in software development.  That problem is finding the time and resources to revisit working, but not perfect code.  For whatever your definition of "perfect" may be.

The developers feel this particular bit of code is just plain nasty. Maybe it is written in a confusing manner, poorly documented, or the landscape has changed so much that the entire architecture could use a rethink. The programmers may have thought up a better way, changed their style/best practices, or perhaps have just improved their skills since that code was written. But it works. It accomplishes the business goals.

Because it works, the management/marketing/sales folks are not very interested in making it better.  They would rather the programmers focus their efforts on new features, because they don't feel they can sell code quality. "Maybe next month/year/release we can do that, but right now go knock out some of these TODO list items...", is what you typically hear.

Both sides of this struggle need to realize that they both are correct and both incorrect. The developers are right in saying the code isn't perfect and could be improved.  They are also wrong, because sometimes what you have is "good enough".

The biz folks are correct in their thinking that they can't directly sell the results of the programmer's effort. They can't put it on brochure and it certainly won't get them talked about in FastCompany. But they are also wrong, because they are missing the long term impact the code quality/design will have on their future business. It is difficult for them to understand how changing, for example to a messaging oriented architecture, helps their modularity and scalability. How it might drastically reduce future development time, improve testing, etc, etc, etc.

Arguing this out could take forever. Not to mention you will have to argue each architecture change as you find/want them.

A technique I have used with great success in the past is what I have so cleverly named "Friday Afternoons".  Friday afternoons are probably the least productive part of an employee's week.  They are watching the clock in anticipation of the weekend. It happens to everyone.  Experienced sales people will tell you that it is basically impossible for them to get any real sales on Monday mornings and Friday afternoons.

As a compromise,  I suggest management give the developers free reign to make architecture, design, and code clean up changes on Friday afternoons. You may need to implement rules around this, such as the changes can't impact any deadlines, they can't create any new work for others, can't impact the product in adverse ways from the customer's perspective, all work must be done in a cleanup branch, etc.

Some of you managers out there are probably thinking, but "No way, that's 10% of our development time".  If you feel this way, you're really deluding yourself. No one is 100% productive, and I would estimate no one is even 25% productive on Friday afternoons, yourself included.

So why do this? There are several reasons:

  • Developers can cleanup the code, effectively making their workplace ( aka the code ) more enjoyable. And everyone knows that Great Code comes from Happy Coders.
  • The programmers feel their concerns are being heard. That management cares about about the quality and not just the dollar signs.
  • You eliminate the time drain of the meetings arguing about these issues.
  • The coders are self-motivated in their Friday afternoon sessions. You weren't going to get much out of them anyway, but at least there is a chance that you can reap future benefits from this work.

Maybe one of those architecture changes means that a requested new feature takes 2 days to implement rather than 2 weeks.  Or perhaps the code will run 5% faster or scale 2x as well.  Even if you just end up with better documentation and a slighly improved employee morale this is win.

If you end up using this technique for awhile, please send me an E-mail or leave a comment here.  I would love to see how this idea plays out in different environments.

2005-2007 © Copyright Revolution Systems, LLC. All rights reserved.