Academic software engineering – fears for the future

I graduated in the early 1970s and, like many people around that time, did not relish the thought of a corporate career. Universities seemed to me to be more interesting places that had liberal views, accommodated eccentricity, gave people time to think and to pursue their own interests. I therefore liked the idea of working in that environment and didn’t really think much about salary or whether I would earn more or less in industry.

Since then, I’ve worked in a number of universities and also spent time in industry and, to be honest, the universities offered a more amenable environment. Some of my colleagues in industry earned a bit more than me but not life-changing amounts so leaving for a higher salary was never an issue. I have no idea if I would have earned more or less in my lifetime if I had chosen to work in industry – probably less if I had wanted the same level of pension offered by the university scheme, although these are not things you think about in your ’20s.

Now, however, both universities and industry have changed. Universities are, frankly, more boring places that are driven by corporate goals and are much less welcoming to the eccentric scholars who published little but were stimulating conversationalists. Its all about outcomes, line managers and targets now – rather like a corporate environment. Industry too has changed  too – to attract the best people, large companies such as Google offer a stimulating and engaging environment as well as much higher salaries than are paid in universities.

The other major change is even more significant. In the 1970s, there is no doubt that the most exciting work in practical computer science and software engineering was going on in universities and research labs. Working in industry mostly meant programming in FORTRAN or COBOL or doing ‘systems analysis’ -I was never very sure what that meant. Now, the challenging software engineering problems all stem from scale – dealing with vast number of computers, building systems with thousands of distributed components and so on . Universities, unfortunately, simply cannot afford to create such large-scale experimental environments and most of the leading-edge work concerned with scale has moved to companies such as Google and Amazon.

So, we have a situation now where companies have a more relaxed working environment, pay much more and provide more interesting work.  For the best graduates who have an interest in the practical aspects of the subject, this is a no-brainer. Why work in a university when you can be paid a lot more to do more challenging things? Add to that the academic ‘publish or perish’ pressures, a heavy teaching load and university bureaucracy and an academic career doesn’t look that attractive.

The outlook for academic software engineering and other practical aspects of computer science therefore doesn’t look that good. There will always be people interested in the theoretical aspects of the discipline and they will prefer a university environment so we are not looking at the end of computer science as an academic discipline. But if we can’t recruit the most able software engineers, it will become increasingly theoretical and out of touch with what is going on in the world.

I therefore fear for the future of practical software engineering in universities as my generation ages and retires. The debate over whether CS is an engineering discipline or a mathematical science will be settled once and for all – by default, as engineers simply won’t be involved.

15 thoughts on “Academic software engineering – fears for the future

  • December 15, 2012 at 1:00 am
    Permalink

    As a recent grad, this is exactly why I did not even remotely consider academia. Meanwhile, at work, I still get read papers and write novel algorithms, with the added pleasure of actually shipping ground breaking software and seeing my work implemented in the real world – a pleasure that many academics are probably denied.

    Reply
    • December 16, 2012 at 10:39 am
      Permalink

      Abe – I’m pretty sure that I’d be doing the same if I was starting out now.

      Reply
  • December 15, 2012 at 2:15 pm
    Permalink

    This is also why I will be moving to industry on after completing my Ph.D. The academic environments I’ve been exposed to feel very similar to commercial environments, only academia is less well-funded, harder to get things done within, and the feedback loop in terms of seeing work published to the world is much longer.

    Reply
  • December 15, 2012 at 2:25 pm
    Permalink

    If academic software engineering retreats to insular theoretical research, who will teach the next generation of software engineers? Maybe the required knowledge will be replaced by narrow certifications. However, I fear that such professionals will lack the deep knowledge required for creating ground-breaking software and solving tough problems. We must find ways to make universities attractive to the best and brightest people interested in computer science. Given the constraints you described that universities face, we must also find ways to make companies more like universities, performing computer science research that will produce fruits not just in the next quarter but in a decade.

    Reply
    • December 16, 2012 at 10:42 am
      Permalink

      The problem of teaching is what I’m most concerned about – it’s already happening that people are teaching software engineering with no understanding of the complexity of large software systems.

      It’s hard to see how to make universities attractive to the best people when people like Google are offering new PhD grads salaries that are comparable with senior professors.

      Reply
  • December 15, 2012 at 10:58 pm
    Permalink

    I think this happened years ago. Colleagues of mine observe that they find that they hire grads from everywhere else except the UK, because ours are just not trained. I’ve had at least one conversation with a top professor about training the kids in software practice that made no sense to him at all. I’ve also taught 4th years (top department) who have never seen version control.

    Should software engineering be looking at other disciplines where there’s a more fluid relationship with cutting edge practice, such as medicine, law (and maybe even “real” engineering”)?

    Reply
  • December 16, 2012 at 6:43 am
    Permalink

    I am an academician, but in India where the “publish or perish” pressure is not as acute. But the observations about academics becoming more corporate and less stimulating is very true. Interesting to read how things remain roughly the same whether it is the US or India. Only hope that the cycle does not reverse over the next few decades.

    Reply
  • December 17, 2012 at 8:16 am
    Permalink

    Ian,

    As you know, I have recently moved back to a UK university role after more than 20 years working as a software engineer in industry. While broadly I can agree with your sentiments here (as we have discussed many times!) a couple of quick thoughts:

    1. I could not get into a CS department as I was “not sufficiently academic” so am now in a business school. I find this in many ways a better fit for my interests and skills.

    2. Be very careful with the comment that “we have a situation now where companies have a more relaxed working environment, pay much more and provide more interesting work”. In the vast majority of cases you will get one of these three, at best. The last 8 years working at IBM have left many scars…in industry it is more and more about making the quarterly numbers, making payrole at the end of the month, or being pushed out the door.

    3. Having been on both sides of the fence, it is clear to me the real value is the partnership. The software industry needs high quality universities, and it knows it. It is less clear to me that universities understand how and why they need industry partnerships beyond wanting their money and somewhere to place students. The focus needs to be new ways to support this relationship.

    As I said…a long discussion to be had on this topic…and I’m happy to take part over a real or virtual beer!

    Alan.

    Reply
  • December 18, 2012 at 1:21 pm
    Permalink

    I agree with Alan on realising the value of partnership between industry and academia. The situation is worse in Ghana, where the ‘publish or perish’ syndrome is so entrenched — frustrating. Salaries paid by companies also compounds it further with CS grads not going into research, but end up being IT/DB administrators.
    I hope to write a blog post on it soon.

    Reply
  • December 21, 2012 at 6:33 am
    Permalink

    I agree with Alan Brown’s points 2 and 3.

    However, still think Ians’s thoughts are some how very close to reality (sadly) – and I would not think he was generalizing , i would consider his thoughts may apply to those who could be good enough to be able to chose their own paths and chose between a good place in academia or industry (it is suppose academia should recrute from the “best” students shouldn’t it?)

    I have known of some Profs totally who are unaware (on purpose?) of what Ian talks here, and also disappointed because their current star students want to go to Industry instead of following their “academic genealogy” (a concept some academics people are obsessed with!).
    I know ne of those students who was so surprised of all the “non sense according to him” efforts he would have to make to be an established academic having a salary not very promising -according to him- …and yes …he was offered a job in Google of course plus another big company …
    in any case things have changed… and times in Uni are different ..big pressure on the value of the numbers ….publish or perish ….

    Reply
  • December 21, 2012 at 7:19 am
    Permalink

    “it will become increasingly theoretical and out of touch with what is going on in the world”

    Isn’t already like this ? Or can you prove otherwise ?

    Reply
    • December 21, 2012 at 8:57 am
      Permalink

      I think the situation probably differs from country to country. In the UK, I don’t think we have reached this stage yet but the signs are there.

      Reply
  • December 21, 2012 at 9:31 am
    Permalink

    I keep thinking about this post having read it the other day.

    I think my main reaction is that everything you describe is more or less true, but overwhelmingly it’s good news!

    I don’t mean I want academic SE to be in trouble, but I mean that it’s great that our grads can be challenged so much in real world jobs. My recent PhD graduate went to Adobe, and has explicitly thanked me for telling him it could be just as intellectually challenging as academia. And I think Adobe think he’s great too.

    Plus of course it’s a triumph for software engineering and other areas of computer science. For example, in industry you are very unlikely to implement sorting algorithms unless you are seriously interested in sorting algorithms. Why? Because software reuse is mature enough through libraries that it’s extremely unlikely you have a coding environment where outstanding sort implementations are not available.

    Also I agree with Alan’s comment (2). I think we academics forget some of the perks of the job that we probably take for granted. Like coming in to work whenever we want, leaving whenever we want, and nobody caring that we do that – with relatively rare exceptions like lectures. I don’t dispute there are many ways in which academia is less relaxed, but there are good points.

    To put it another way: when I went half time I explained to my parents-in-law that I was doing it because I would feel guilty if I just coasted for a few years and take full pay while doing all the childcare I would do half time, because it would be about 5 years before anybody noticed my output had dropped. My academic father-in-law understood immediately while my non-academic mother-in-law had no concept of how I could get away with it.

    On publish or perish and bean counting in academia, yes that is a real problem. I have been cheered lately by the fact that while REF is a baleful influence overall, the focus on small numbers of papers means that it’s been far more important to show that you have a small number of really good papers. On the other hand, a great scholar who doesn’t get grants is unlikely to become a professor in computing nowadays.

    So to be more nuanced, I think your post is about the problems of success of academic SE & CS. I don’t dispute there are problems for academia, but also just wanted to say it’s great what we (in and out of academia) have achieved to make it such a problem.

    Reply
  • January 14, 2013 at 12:31 am
    Permalink

    I think two things are going to happen, and in fact both have already started:

    (1) Basic CS engineering skills are going to become much more widespread in other disciplines, such that many companies can hire domain experts from other departments and get the software engineering skills they need.

    (2) Companies that hire ordinary software engineers are going to continue to recruit from the CS departments (all the major tech companies have college recruiting teams). Because they need this talent pool, they’re going to work to ensure it remains “up to snuff.” The companies that do this best have better access to that talent pool, which over time lends them an advantage, which means this model should win out over a more isolationist model in which universities and companies go their separate ways.

    Both of these are already happening. If you’re a biochemical/pharmaceutical company and you need a statistician, you might hire a recent Math graduate, but you’re more likely to hire a biochemist who happened to pick up a lot of formal stats methods along the way. In fact, if you’re a software company and need a statistician, you’re going to pull from that same biostats pool because those students tend to have more practical experience. It’s similar for CS. If a company is doing X (where X != CS), often it can find talented software engineers in the X department.

    And of course, many large companies are already partnering with universities (some partner with several).

    Reply
  • May 6, 2013 at 3:41 am
    Permalink

    This article makes no sense.

    Every university is different, and every business is different.

    There is nothing as “academic software engineering” vs. “industrial software engineering” because we cannot generalize so much.

    Now, when speaking of Google, we cannot compare it with an average university. Google is a top company, so let’s compare with a top university.

    Do you seriously believe that job at Google is more interesting than a job at Stanford, MIT, or Harvard?

    Compare top industry with top academia, and you may as well find that industry is always, in many aspects, doing things wrong in favor of quick releases and market pressures. This can be one of main reasons why the “hottest” software engineering position is somewhere in the middle: half-academic, half-industrial.

    If you choose only academia, or only the industry, you are missing out 50% of software engineering, and eventually become dissatisfied because you know you are missing something.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *