Software Development: An Industry of Amateurs

[article]
Summary:
David Bernstein says the software industry is an industry of amateurs. It's a young field, and he doesn’t think it's yet graduated into a true profession. Here, David contrasts the software industry with other, more established fields, and he talks about what software professionals need to do in order for the industry to become accepted and esteemed.

I have called the software industry an industry of amateurs. We are a young field, and I don’t think we’ve yet graduated into a true profession.

True professions have some defining characteristics. They have a common body of knowledge that all professionals understand and adhere to. We do not have this in the software industry. Programmers learn programming languages in school, but they’re rarely taught how to actually create a well-written program. In fact, terms like “well-written” and “good code” are highly subjective because we don't yet have a definitive description for what “good” is in our industry.

This is ironic because the waste in the software industry is some of the greatest expense to our society. We know that there's a lot of waste in our industry, but we have not yet determined how to ferret that out.

Take, by contrast, medicine or the law—or even carpentry. There are defined paths for becoming a professional in these fields. There is a known body of knowledge that all professionals have, including a common terminology as well as industry standards and practices.

You can’t become a lawyer without going to law school. You can’t become a doctor without going to medical school. And in both cases, you're required to go through a long and intensive apprenticeship or residency where you’re working with real professionals on real cases. Even carpentry has an apprenticeship model where you learn on the job from a master carpenter, and there is an external body that certifies candidates in all of these fields.

Software development has none of these things. You can go to school and study computer science or software engineering, but it's not very relevant to what you need to know in the field. Just as many professional software developers do not have a computer science degree as those who do.

Furthermore, there is no apprenticeship model in the software industry. And because schools are at least twenty years behind the times, actually doing the job while being lucky enough to apprentice with someone who’s been doing it for a while is the only way to really learn the state of the art of software development.

True professions have a defined path of entry. You can't just hang a shingle on your wall saying you're a doctor. To be a doctor, you must graduate medical school and be part of a rigorous residency program. Nurses, too, have to undergo intensive training in order to become professionals. Lawyers have to pass the bar exam and then serve an internship with an attorney.

To become a programmer, you simply have to declare yourself a programmer. I know people who call themselves programmers and only write HTML. HTML is not a programming language, but even when you know a programming language or two, it doesn't necessarily make you a professional software developer any more than memorizing some words from a medical textbook makes you a doctor.

Around half of all professional software developers are self-taught, meaning that they did not study the subject in college. In truth, all professional software developers are self-taught, because the skills we need on the job are rarely covered outside the workplace. We learn as we go.

It’s easy to become a mediocre developer. Making a computer do a task is really not that difficult. But just making the computer do a task is a very small part of being a professional software developer. We must write software in a way that communicates our designs in the code so that others who come after us can understand, maintain, and extend the software we write. We also must communicate and collaborate on a team when building software professionally, and I rarely see schools offering those courses to engineering students.

True professions have standards and practices that are defined and enforced by some kind of oversight organization. In the US, for medicine it’s the American Medical Association, and for law it’s the American Bar Association. The software industry does not yet have this. We don't even have any organizations in our industry that are candidates to become this.

Employers think that software development skills have to do with technologies, so they hire programmers with experience in specific languages or frameworks, but these are just tools. It's like hiring a carpenter based on whether they've ever used a nine-pound hammer. If you know how to swing a hammer, then the size of the hammer shouldn’t really matter much.

A good software developer can learn a new technology in a matter of days, but becoming a good software developer can take many years. The key skills have to do with problem-solving, writing maintainable code, and being a good communicator. Software development is a team effort, and being able to collaborate and communicate are critical skills in developing software.

Software development today is one of the easiest professions to get started in. Everyone has a computer. You can download programming languages like Eclipse for Java and many others for free. As long as you have the time and the patience, you can learn a programming language.

Like many professions, software development can be very challenging to master. There are a lot of mediocre programmers out there but very few true masters in software development. We are a young profession and we're still figuring it all out. We've come very far, but we still have a long way to go.

User Comments

8 comments
Ed Weller's picture

Nice article. One of the contributing problems we face is the need for multiple domain expertise - the domain of the application (e.g., flight systems, finance, medical) and software engineering. Rarely do both of these skills come packaged together.

October 18, 2018 - 1:30pm
Keith Collyer's picture

It's amazing how few people understand this. People call themselves software engineers, when software is still a long way from engineering. All engineering disciplines are based on concepts of strength (often many, to be used in different contexts). We have nothing like this in software.

I would take issue with one thing. There are bodies that are candidates for becoming professional bodies like the AMA, the IEEE is one such.

October 19, 2018 - 6:13am
Thuc Nguyen's picture

Picking out characteristics of several well-known professions such as doctor, lawyer, nurse and carpenter and universalizing those characteristics for all other professions is a hasty generalization. For instance, not having a standardized test to pass doesn't mean a profession is not a profession. The fundamental difference between professionals and amateurs simply boils down to: are you getting paid for what you do or not. That means are we valued by the market or not.

If we follow along this article's reasoning, we'll inevitably disqualify true and esteemed professions such as writer, professional athlete, CEO, etc. JK Rowling didn't go through a formal training yet she's still a billionaire writer. Usain Bolt doesn't need a graduate degree hanging on his wall to be a world-class sprinter. And Bill Gates as well as countless other famous CEOs are all self-taught yet they're (or were) running multi-billion companies.

Not having an institutionalized apprenticeship model doesn’t mean there’s no apprenticeship out there. Apprenticeship in software development is just more social and hands-on.

Not having a defined path of entry doesn’t make a profession not a profession. We simply got a diversity of entry points and self-paced paths.

Having a self-taught education doesn’t mean that you’re not educated. It’s just another approach of acquiring knowledge and no approach is universally the best.

Having a low entry barrier is not a curse. It’s actually a blessing. You just need a computer, internet connection and some dedication to start contributing value to society. What not to love?

October 19, 2018 - 7:04am
Keith Collyer's picture

It's difficult to know where to start with this, but I'll try.

First, just because you get paid to do something, it does not mean that you are a professional. The Oxford Dictionary of English says a professional is:

  • a person engaged or qualified in a profession: professionals such as lawyers and surveyors.
  • a person engaged in a specified activity, especially a sport, as a main paid occupation rather than as a pastime: his first season as a professional.
  • a person competent or skilled in a particular activity: she was a real professional on stage.

The definition we are using here is clearly the first. So what is a "Profession". Again, from the ODE:

  • A paid occupation, especially one that involves prolonged training and a formal qualification: his chosen profession of teaching | a barrister by profession.

That is what marks you out as a professional. And the rest of your argument falls apart in that light.

I don't think you understand apprentice either. Let's use the ODE:

  • a person who is learning a trade from a skilled employer, having agreed to work for a fixed period at low wages: [as modifier] an apprentice electrician.

The key point is that the person you learn from is skilled, and usually has qualifications to prove it. In your less formal approach there is no guarantee that either is the case.

Having a low barrier to entry is very much a curse. It's probably why a monstrosity like Slack can consume multiple gigabytes of memory for what is essentially a glorified chat program with a sexy interface.

March 25, 2019 - 11:30am
Clifford Berg's picture

I think I am a good counter-point. I have little formal training in programming (one university course), but I have written four books in the profession - two acclaimed. One was so successful and respected that it was adopted by Sun Microsystems as its own book on "Enterprise Java". I also co-founded a successful IT services startup. Early in my career I built some pretty advanced compilers - and I am entrely self taught in that technology. Today I am a consultant who helps organizations to "do it right".

I think I am a professional.

But I agree with the intentions of the article that, as Alan Kay claims, too many programmers don't learn - they hack. They don't reflect or think about the best approach - they just plod forward. Professionally _trained_ programmes would know about established patterns. The challenge is that patterns change. One must keep up. That is true in any profession.

So I don't know the answer. To institute a certification and apprenticeship barrier might exclude people like me. But I also think we need a higher bar for what is considered a "professional programmer".

July 4, 2019 - 4:52pm
Joe Schofield's picture

Well done sir.  Getting paid is no evidence of a profession; shysters and con artists get paid too.  A profession has as you stated, some BoK, and a code of ethical behaviors.  Most professions are licensed and insured.  Outliers that are successful in lieu of professional accreditation are merely that, outliers.  They aren't the norm and no one should expect resilent software from incapable developers.  I know very capable developers in narrowly-defined skillsets; some across multiple skillsets.  They tend to be atypical.  Anyhow, good discussion by all . . .

October 21, 2018 - 3:11pm
Nicholas Snogren's picture

Who's ever heard of excessively expensive law fees or overpriced medicine? We should apply those same conditions to software!

Let's introduce licensing to software developers so that self taught, self motivated people have to pay someone money in order to get a good job!

Or... we could thank God that software hasn't been affected yet with the diseases prevelant in so-called "mature" professions, and be thankful that a smart person off the street can study hard and land a good job in a great industry.

March 21, 2019 - 1:19pm
Clifford Berg's picture

I think that what is missing in the software industry is that the authors of software are not generally liable for the costs of software problems. Sometimes those who bear the cost of a failure succeed in a law suit, such as in the case of the Toyota acceleration control - the software was examined and discovered to be "spaghetti code" (the words of the analyst). Millions were paid by Toyota.

But all too often, broken software merely results in frustration for users.

If companies experienced financial pain when their software failed, they would quickly tighten up their standards.

July 4, 2019 - 5:00pm

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.