Revolution Systems Blog

Want to be a better manager?

I've been reading a great book recently titled First, Break All the Rules: What the World's Greatest Managers Do Differently. I highly recommend it for anyone who manages employees, but it makes two great points early on that are especially appropriate for technology managers:

Treat your Employees Differently

You should treat your employees differently.  Each has unique strengths, weaknesses, and differ in the way they learn and you should capitalize on that, not try to homogenize everyone. Sally might be an awesome systems architect, so you don't want to weigh her down pounding out mundane features.  Bob might be the fastest developer and giving him those same mundane features makes sense.  Steve on the other hand might be your more efficient debugger, etc. 

They learn in different ways, some by reading, others by doing, and some need a more traditional classroom environment. Sending a do-er to a lecture class is a waste of everyone's time. 

Don't Let Human Resources Get in the Way of Your Hiring

I've touched on this topic before, but you can't leave the resume collection and filtering process to your HR department.  Even in our more tech savvy age, I still hear horror stories from programmers how they lost out on being interviewed for a job because the HR person didn't know J2EE or J2ME were 'Java', that Ubuntu is a Linux distribution, or that having 6 years of mod_perl experience probably means they know Perl pretty well.

A great technology manager has to hire a person based on several things, one of which is experience with a particular technology or technologies.  But that's only part of it.  Troubleshooting skills, breadth of experience in many technologies, personality and how this new hire will fit into the overall team structure are just as important.  The worst possible case is when the manager is barely or not at all involved in the hiring process.

Now there are exceptions to this.  The best example is my friend Josh Stomel who runs NeoHire. Josh is one of those rare recruiters that actually understands technology and more importantly knows when he doesn't know and has a huge network of geeks like me to ask questions. You can just hand NeoHire a list of qualifications and they will bring you back a group of top notch talent.

So it is possible to have HR deeply involved in the hiring process, it just has to be the right people. 

Some interesting links

Hope all of my readers have recovered from the holiday season.  Here are a couple links I've come across recently, but neglected to write about during the holidays.

Zed Shaw, author of mongrel which is used by many Ruby on Rails applications, posted an interesting rant about the state of Rails development and the personalities of some of the major players. 

I've never been a huge fan of Rails because I've never been shown a compelling reason to switch away from the more mature Perl equivalents.  Lately I've been working with mongrel and Rails apps more and more as many of my clients have switched to that as their platform of choice.  The more I work with it the less fond of it I am, it really is just immature compared to other platforms out there.  Toward the end he mentions the Rails creator himself has an application that had to be restarted 400 times per day... with some fixes it only has to be restarted 10 times a day.  400 is just insane, but 10 is still very troubling.  If the creator can't make things run better than that it really doesn't speak well for the platform.  Normally I wouldn't link to such a huge rant, but when it comes from a major Rails contributor there has to be at least some truth to it.

My friend Stas Bekman has created a new site where you can make it known how you want a better version of something.  It's called i-want-a-better.com.  I think it's a great way for people to vent their frustrations with products, but I think the really winning idea is that entrepreneurs can peruse the site for new products and untaped markets. Hop on the site and let us all know what's frustrating you!

 

Making the software written in any language more readable

There are two very simple ways to improve the readability and maintenance of the software you write. They are so simple they are often ignored in favor of more complicated tools and the various programming methodologies people blather on about.  This comes from our human nature to think our own problems are more special and complicated than they really are and from not following the KISS principle.

So how do you improve your software?

By using better names and being consistent. It really is that simple, which is probably why it is overlooked.  A development manager might score some points with his boss by switching to Agile programming, having some scrums, doubling the amount of developer documentation or maybe even switching to a whole new platform like Ruby on Rails.  But who scores any points by saying, "We're going to do better about naming things appropriately?"

Appropriate Naming

Obviously having a variable named temp_user isn't all that descriptive, but it is an improvement over just temp or simply t. Most, if not all, programmers realize this.  However, you will often see variables named clt when they should really be named client, as if those three extra characters were single handedly going to cause carpal tunnel.  Your variable names should be as descriptive as possible without being absurd about it.  A variable named the_current_user_object_we_are_working_with is obviously overkill. But perhaps current_user_object or even current_user is appropriate.

Naming your functions and methods should also be given the same amount of care. Naming a function fix_client doesn't tell us anything useful, we can only speculate something is wrong with the client or the data and this function does something about it.  normalize_client_name is a much better name when you read that all the function does is properly set the case of the letters in the client's name.

The names you choose for your libraries are also very important.  This is one of the reasons programmers find Perl's CPAN much more useful than other programming languages' library collections. Things are named and categorized (for the most part) sanely.  Need something related to E-mail, check the libraries in the Mail category.  Can you guess what Net::SSH, IO::File::CompressOnClose, and WWW::Google::News do? Yeah I thought you could.

If I stumble upon your use of a library called Util, it doesn't tell me anything I still have to go look at the library to see how it fits in with this code.  If it had been named something like  DB::Util or Client::Utils I would at least know the library is probably related to the database or client.

Consistency

Consistency is another area where you can improve your code base without much effort.  If all of your classes contain an initialization method, they should all be named the same.  Not initialize in some classes and init in others.  Things that should be consistent throughout your code base, not only consistent within a single application. If possible, the source in your department should be consistent with other departments and business units.

Consistency through conventions is one of the main reasons people like web frameworks like Ruby on Rails. I'm not advocating Ruby on Rails necessarily, you can accomplish these same things in any language.

Things that should be consistent:

• variable, function, and method naming conventions ( underscores or CamelCase, but not both )
• frequency and layout of comments
• documentation
• configuration file syntax
• on disk layout of source code, binaries, configuration files, etc.
• installation, upgrades and package management
• language(s) used for development

The point being the more consistent you are in how you build an application the easier it is to get down to the task at hand.  Be it new development or bug fixing.  Got a configuration file format you like and have already written libraries to parse it? Then use it EVERYWHERE, in every single darn application you build. No one has to learn the new format to start developing, no time is spent discussing which format is better, and another developer in your organization can jump in to fix bugs or add a new feature.

The last development shop I ran we focused on being consistent. 95% of the time we were building web applications that were either used internally by fellow employees or externally by our customers.  If you saw the source to one of our applications, then all of our other applications would seem very familiar. One might be a ticket tracking system and the other an accounting package, but the layout, coding style, and use of common libraries let the developers dive right in and not have to worry so much about how this particular app is written differently than the others.

Consistency in your applications also makes refactoring easier.  If all of your applications use a particular technique, library, etc. in the same way replacing it with a new tool you have fallen in love with is much easier.  If everyone is doing everything even slightly differently, you have to start worrying more and more how the change might impact your code.

I'm definitely a "right tool for the right job" sort of fellow, but mixing and matching tools and techniques for every single application you build is a recipe for disaster. One shop I know (name withheld to protect the guilty) has two developers.  One who works entirely in Perl, the other entirely in Ruby.  Another application I saw was written mostly in Java, but with a smattering of C# and Python around the edges for kicks. None of these three were chosen from a "right tool for the job" perspective, but simply because those were the favorite languages of the specific developers tasked with those sub-systems. These are obviously worst case examples.

These ideas aren't new, I'm positive I am not the first to use them or even write about them.  But I see these two simple rules violated so often by programmers of all experience levels that I felt the need to reiterate them.

Followup to "A Guide to Hiring Programmers"

Please excuse my laziness, but I simply don't have time to respond to each and every person who has E-mailed or left comments on digg, reddit, or the original post itself.  I would like to respond to a few of the larger themes I've seen in the questions/responses:

This applies to more than just programming

I definitely agree that this can be applied to nearly any type of job, not just programming.  A great designer is worth much more than an average one.  And I honestly wasn't trying to single out sales and customer service people.  I do agree that a great sales person or customer service rep is worth more than the average, and should be paid accordingly. And yes every employee is important to the company.

Using customer service as an example, I've worked with the worst where they would literally scream at the customer on the phone to the best. The problem with customer service is that the metrics are against them in that even the best customer service person can only take a few more calls/tickets than the worst.  Just because of the nature of the interactions. It is also very difficult to measure if this person has pleased/retained more customers than another. With programming it is often easier to see how an individuals contributions impact the whole project.

Sales is a different monster, but hopefully this has cleared up what I was trying to express.

Only experts?

Do I think it's reasonable to hire only experts?  Yes, in many situations a company can and should staff themselves with the vast majority being experts. Is it possible for larger companies with larger products? Probably not.  If the problem simply demands 50 developers, it would be difficult to staff that entirely with experts. However, I do believe they would see a boost if they were able to have at least 10-15 of those developers be experts. Instead most companies have 1-3 experts that lead the team of the masses.

If you can't find experts, you should attempt to hire staff that could become experts over time as they gain experience.

How do you become an expert?

Everyone is correct in saying that experts started out as novices, I was certainly a novice.  In many ways I still am.  Being personally interested in martial arts I remember a story of someone, after years of training, finally receiving their black belt in Aikido and being told, "Now you are ready to learn."  I believe this is true of programming, technology, and most professions.  The learning doesn't and shouldn't stop.

So how can you become an expert? I think the best advice I can give is to read up as much as possible on your field.  You don't become an expert simply repeating what you did yesterday for many years until, poof, you're an expert. You need to be learning new idioms, patterns, and tips from your peers.

Too many developers sit in their cubes and pound out code and never look up. You need to be reading up on your profession as much as possible, exploring new languages/tools to determine if you could be doing something easier or better.

An example of what I see far too often happened again recently at OSCON. A professional Python developer did not know Django was the predominant web framework for that language. I'm not a Python user, but even I know this. Maybe it's because I'm friends with the core Django team, but even if that had not been the case I would at least be aware of it and in general what it was from my day to day  tech reading.

The other advice I would give is to read and become involved in an Open Source project.  This improves your code quality and allows you to see how other, presumably senior, developers work. Even if you aren't able to contribute to the project directly, get on the mailing lists and examine how those developers work.

How do you find and hire experts?

I think the biggest mistake managers make is leaving this up to HR.  I've always made sure I received every resume that came in for a position I was hiring for.  HR will often reject a candidate because their resume states they have "Years of J2EE experience", but since it's a Java programming position it goes in the trash.  Perhaps it is time we start hiring "HR Engineers" like we have "Sales Engineers."

The first place I look when hiring programmers is the Open Source community. If they are involved in an Open Source project you can easily see how they work with others on the mailing lists, see examples of their code, etc.

They also tend to be of higher quality because Open Source is a meritocracy. Not to mention the simple act of being involved in a project, for no monetary gain, shows a strong love of their craft.

I think multiple choice tests are a very poor indicator of programming prowess. Too often they have a couple of esoteric or even trick questions that really compare the test writers ability to confuse with the test takers' ability to decipher.  It is much more important for your new hire to know how to find the answer than it is for him to actually have it tucked away in a brain cell. Ability to effectively use Google to search for the answer is much more important than many realize.

If you happen to be one of the people who are looking for an expert Perl programmer I suggest you get in touch with my new friend Uri Guttman, The Perl Hunter, at uri@perloncall.com.  He specializes in finding execellent Perl programmers for companies. Being an accomplished developer himself he easily separates the wheat from the chaff and can find someone who will be a good overall fit for your organization.

Many problems are marketing and management's fault...

This is also very true. Bad management will bring down any team or project, no matter how many experts they have on staff. This isn't even restricted to technology management.

Marketing often over promises what can be delivered and demands it in an unreasonable time frame.  Unfortunately most of the time we blame the developers, because long after the sale all that we see is the code and not the brochure.

My advice to marketing and management is that you bring a problem to your developers and then base your plans on when they believe they can deliver the solution.  Far too often management has already determined time lines and set things in motion before the development team has even been told about the project.  This is backwards.  You don't schedule your building contractors before you have the proper permits or before even speaking with the architect about the project.

Even Microsoft gets this right. They realized it was much better to delay Vista until it is ready than to ship it too early just because they originally said a certain date.

Obviously you can't always just wait around for something to be perfect. There are always restrictions and requirements that are outside of your control.  No one could move January 1st, 2000 out a few more weeks just because their Y2K cleanup wasn't done. But often I see companies attempt to move mountains to hit some arbitrary date when one of the largest consequences of delaying would be everyone had to update their Outlook calendars.

My language bias

I received a bunch of comments on my use of "Perl vs Java" in the example, that simply was what we were talking about at dinner that night. I probably should have used "agile language X vs cumbersome language Y" to keep the flames down to a minimum.

You can write efficient, readable, and maintainable Perl.  I've even had some notable Python programmers say that about code I was in charge of and honestly the code in question wasn't what I would consider the best of the best. I think Python is a great language, but for me personally I haven't been shown any compelling reason to switch.

You can write crappy unreadable code in any language. You can make most any language/framework/toolset scale and perform to your needs.  For every "large app/website/etc" that uses language X I'm sure I can find you a comparable app/website using language Y.  Any performance differences between language X and Y can usually be solved with $100 worth of extra CPU.  What really matters is programmer efficiency.  That is where you save money and reap benefits. I simply don't see how having to write, read, troubleshoot, and maintain 10x the number of lines of code is an efficient use of the programmer's time.

However, I do agree that you should use the right tool for the right job. Java/C++/C# are definitely the right tools in many situations.  I just feel that because everyone has seen a horribly written Perl CGI ( or written one themselves ) they think this is somehow ingrained in the language and because of this Perl simply isn't an option for anything "real."

Perl is a language where the developer must use some self-control rather than having it imposed on him by his tool. Which is why Perl (or many of the more agile languages such as PHP/Python/etc) written by novice programmers is so awful. The knowledge and self-control comes with experience.

The largest problem with any language is the use of poor variable, function, class, and method names.  Using adequately long and descriptive names is probably the single best way to improve code quality and no language out there enforces this. Some enforce a certain style, others force certain methodologies, but this is really only picking at less important aspects of the problem.

Company bias

By comparing Apple vs Microsoft I wasn't really singling out their development staffs.  I'm sure their management, design, and marketing departments are as much to blame for any successes or failures these companies have.

What I was trying to get across was the "It simply works."  I would say the second most common comment I hear from Mac users, after how pretty/well designed they are, is that it "just works."  I don't hear that very often from Microsoft users.

A Guide to Hiring Programmers: The High Cost of Low Quality

I was invited to a wonderful dinner party (I swear it wasn't too spicy Sarah!) with some St. Louis Perl peoples this week while I'm here on business.  At one point we were talking about hiring programmers, specifically Perl programmers.

We agreed on the following:

• Finding good programmers is hard in any language.  And that a good programmer can be as effective as 5-10 average programmers.
• Average pay rates between equivalent programmers are out of sync and are based more on the language used than the skill of the programmer.
• You don't need to hire an expert in language X, you can and should look for expert programmers that are willing to learn language X. An expert can easily cross over from being a novice in any language in a matter of a few weeks.
• You should seriously consider allowing your expert developers to telecommute full-time. Restricting your search to programmers who live in your area or are willing to move limits the talent you can acquire. Arguments regarding "face time", productivity, etc. can easily be nullified when you look at how some of the largest and most successful Open Source projects such as Linux, Apache, and Firefox are developed by individuals rarely living in the same time zone or even country.
• We love Perl and think it's a great language that you graduate to after you have been forced to use less agile languages such as Java, C/C++/C#, etc. Not necessarily a first language you get your feet wet with and then move onto a *cough* "real" language.

Many people in the Perl community have been writing on this topic lately and wanted to share my opinions on the subject, as it is one I have put many hours of thought into. Doing my best to keep this language agnostic as I believe these tips can be applied to any programming language. I will however, use Perl in some examples as it is my preferred language.

Why is it so hard to find good programmers?

The simplest reason is when a company finds a good developer they do more to make sure that person is happy which leads to longer tenures. Better salary, more flexible working conditions, good tools, interesting projects, and better perks can often keep a programmer working for you longer.

Another obvious reason is that experts in any field are small in number, so your possible talent pool is limited. This leads managers and HR departments to settle for average or even below average developers.  I believe this is the single biggest mistake a technology oriented company can make, regarding developers, just short of not using a good version control system.

We're not talking about customer service representatives or sales people here. Just having a body to fill the seat is not, I repeat not, always a win for the company. Sub-standard programmers drag down the efficiency of your other developers with beginner questions, poor comments/documentation, and bad code that someone else will later be forced to spend time fixing.

Companies need to stop thinking about their developers as cogs in the machine. They are more akin to artists, authors, designers, architects, scientists, or CEOs.  Would your HR department rush to find the first person who would be willing to take on the role of Chief Scientist, Art Director, or CEO in your company? Of course not, they would spend the time to do a thorough talent search for just the right candidate, court them, and then compensate them appropriately. They realize that having the wrong person in that seat is much worse than having the seat empty. It is absolutely the same with programming.

Anyone who has been a developer or managed developers can tell you that an expert can accomplish as much as 10 average developers.  However, companies typically pay only a 10-20% premium for an expert over the average programmer. Whether or not their title is Lead, Architect, Development Manager, Guru or whatever nomenclature the company uses. I am not saying that if your average developer is paid $50k/year that you should pony up $500k/year for an expert. The employer/employee relationship never works like that, but what employers don't seem to realize is that in the end paying more saves them more.

Let's look at an example.  One common argument from HR departments is that they "can't find any Perl programmers, but they can't swing a cat without hitting a Java developer".  While this is fairly accurate, they are approaching the problem from the wrong direction.  If you fill your shop with 15 average Java developers, paying an average of $60k per developer you have an approximate labor cost of $900k/year for your development staff.  Not considering any non-salary benefits.

Suppose you instead took the time to find 5 experts, or at least above average, Perl developers at $120k each per year. Here is a partial list of the pros and cons of such a scenario:

Cons:

• You must spend extra time finding, evaluating, and courting these more sought after developers.
• Your company and what the developer may be asked to build may simply not be attractive to this class of developer.  Very few people want to work for a spammer or a small web design firm that caters solely to freelance accountants for example. Smart people find boring things even more boring than the masses.
• When one of them leaves the company, there is the feeling that your company's business objectives are more at risk due to having only 4/5ths of your normal resources. Or that a larger chunk of your corporate knowledge just walked out the door. This is more of a perceived problem than an actual one as good developers are better at writing readable/maintainable code, commenting their work, and writing effective documentation.

Pros:

• Each developer will be more content with their job, due in part to the higher than average salary, but also because his or her co-workers are of a much higher quality which improves anyone's job satisfaction.
• Development would require less overall communication as there are less people to communicate with.  This obviously improves efficiency as anyone who has been on a 20+ person conference call can attest to. Or read the Mythical Man Month if you want a more in-depth analysis of this phenomenon.
• Experts travel in the same social circles.  Having one expert on staff makes it much easier to find other experts in the same field, no matter what field that may be.
• You would save 2/3rds on infrastructure costs.  Things like cubicles, computers, cell phones, free lunches, training costs, travel, office space, air conditioning, electricity, etc, etc. The list is essentially endless.
• Your HR department would have 1/3rd the number of developers that it would need to take care of. Less paper work, less questions, less everything, and less turn over because of the lower number of employees.
• Oh and you'd save $300k/year on your labor costs.  Not to mention non-salary benefits such as stock options, retirement matches, health insurance premiums, perks, etc. You could spend as much as $100k/year on your talent searches and still be $200k/year ahead.  Hell, you could dedicate an entire HR person just to this task.

What is an expert programmer?

Experience is key, but not necessarily in ways you might imagine.  Time in the saddle, with a particular language is not as important as diversity of experience.  Someone who has worked in several disparate industries, a generalist, is often a much better developer than one who has spent years in the same industry.  There are exceptions to this, but in general I have found this to be the case.  Bonus points if your developer was a systems administrator in a former life.

Some of the best developers I know were originally trained as journalists, mathmaticians, linguists, and other professions not normally associated with software development.

Experts use better tools and care deeply about their craft.  They aren't assembling bits on an assembly line, they are crafting a unique product to solve a unique problem.  Experts are lazy, they work smarter rather than harder.  Experts prefer the easiest solution that gets the job done. Experts aren't interested in creating complex solutions simply to have the complexity, that misguided egoism is the territory of more junior developers. They often get it right the first try and almost always on the second one.

Simply put, experts write readable code.  They comment and document it appropriately based on the complexity and criticality of that particular piece of code.

All of this pays huge dividends when the next developer has to pick up where they left off. Especially if the next person isn't an expert.

More reasons you want an expert programmer

Is your business technology oriented?  Perhaps the software you create is even your main product. If nothing else I'm sure we can agree that if the software your developers create is to some degree critical to your business.

I've worked in many different environments, with people of every skill level, and it's very easy to tell whether or not a company has expert developers. Do you often find that the software is down? That it has as many bugs or even just idiosyncrasies that make no sense to the user as it does features?  Do the users find it difficult to use?  Is the problem at hand relatively simple compared to the training or documentation necessary to begin using the software?

If you answered yes to any of those questions you more than likely have average or below average developers.

When you work in an environment with experts things simply work.  They are easier to use and require less initial training. The software is easier to modify.  Requested changes happen more frequently and easily.  Things just flow.  It is the difference between Apple and Microsoft.  It's the difference between the iPod and a 400 disc CD changer with 50+ buttons.

As with many things in life, sometimes you get what you pay for. I'd love to hear your comments and opinions on the subject.

UPDATE: I've written a response to some of the questions and comments I've received on this article in a follow up post A follow up to "A Guide for Hiring Programmers"