The rate of obsolescence of knowledge in Software Engineering

A physicist loses half the value of their physics knowledge in just four years whereas an English professor would take over 25 years to lose half the value of the knowledge they had at the beginning of their career [1]. These estimates are taken from a paper written in 1982 so software engineers obviously weren’t included. But it begs the question… What would the half-life of the value of a software engineer’s knowledge be? I suspect it’s somewhere between the physicist’s and the English professor’s because a software engineer’s knowledge is a combination of eternal computer science/engineering principles and ephemeral technologies that drift in and out of popularity over time.

Knowledge that doesn’t expire

Software engineers with a traditional computer science background learn things that never expire with age: data structures, algorithms, compilers, distributed systems, etc. But most of us don’t work with these concepts directly. Abstractions and frameworks are built on top of these well studied ideas so we don’t have to get into the nitty-gritty details on the job (at least most of the time). Examples are the C++ standard library which implements optimized sorting for arrays, and Apache Spark which provides fault tolerant cluster computing out of the box.

Knowledge that does expire

The unavoidable ephemeral knowledge one accumulates during their career comes in many forms and this isn’t an exhaustive list:

  1. A vogue framework or programming paradigm that falls out of favor in a couple of years.
  2. Domain knowledge in a rapidly evolving industry/field.
  3. Knowledge of proprietry technology: e.g. internal tools at your company.

Knowledge of this kind can quickly transition from zealous adoption to every company who uses said knowledge trying to sunset everything they’ve built with it.

My Experience

I’m inclined to say I use ephemeral knowledge more than eternal knowledge to perform my job. And then there’s the added pressure in our industry of always having to learn new and useful things.

A theoretical physicist who spends a great deal of time mastering a theory and the mathematical techniques behind it, only to see that theory rendered obsolete by a new and improved theory several years later, is analogous to a software engineer who spends a great deal of time mastering a web development framework, learning its intricacies and gotcha’s, only to see that framework rendered obsolete by a new framework several years later, leaving no more demand for that knowledge in the labor market.

I relate to the physicist more than the English professor. What do you think?

References

[1] McDowell, John M. “Obsolescence of Knowledge and Career Publication Profiles: Some Evidence of Differences among Fields in Costs of Interrupted Careers.” The American Economic Review, vol. 72, no. 4, 1982, pp. 752–768. JSTOR, www.jstor.org/stable/1810015. Accessed 9 Sept. 2020.

Written on September 9, 2020