a further thought on volkswagen

October 10th, 2015


corporate crap, aka a couple of software engineers

October 9th, 2015

Volkswagen — and their misleading and deceptive behaviour — are all the news at the moment. Diesel engines run in a continuum, with variables like particulates, fuel consumption, power output, and nitrous oxide (NOx) production in play, with a summary that to minimise particulates, fuel consumption, and reasonable power output, the production of nitrous oxides result.

In large diesels, such as those found in trucks, buses, and so on, this is dealt with by injecting a urea solution into the exhaust, where a catalyst neutralises the NOx component of the exhaust. This solution adds weight and complexity, as well as the requirement to “refuel” the urea solution regularly, so it isn’t often found on small diesels.

People though don’t tolerate smoky diesels, particularly in small passenger vehicles. The addition of a diesel particulate filter theoretically addresses some of this, but better to minimise the particulates anyway. The added bonus of minimising particulates is lower fuel consumption, and more usable torque and power. NOx aside, low fuel consumption, low soot, and higher power output is what customers do want.

Which brings the question of how small diesel manufacturers have been passing the extremely strict emissions standards, which care more about NOx than particulates, given that NOx is a main ingredient in photochemical smog. As is now turns out, at least one manufacturer — Volkswagen — has been passing the tests through duplicity: running a “fuel-rich,” probably particulate-heavy, NOx-light engine profile when under test; running a leaner, particulate-light, NOx-heavy engine profile for normal driving.

Give a one-dimensional metric to an engineer, and they’ll find a way to ‘optimise’ that metric. Measure ticket closure rates in a support environment, and tickets will be closed as quickly as possible — probably quicker than would result in happy customers.

However, I don’t for a moment believe that “a couple of software engineers” would have gone to these lengths unbidden; for starters, the test regime for such engine profiles would require significant coordination among many different people within the organisation, all the way up to — at the very least — a program manager. Dyno runs; tests to ensure the software can figure out the difference between a rolling road and a real one; the engine mappings themselves: all of this takes time, effort, expense, and coordination. If it were as simple as “a couple of software engineers,” the development costs associated with new cars wouldn’t be as large as they are in the first place.

Further, why would “a couple of software engineers” be too concerned about passing some environmental tests, unless they had been directed to be concerned about it?

My guess, having seen similar behaviours — even including tossing “a couple of software engineers” under the bus when caught — can be summarised like so:

  1. Diesel engine was developed.
  2. Diesel engine fails internal benchmarks against the relevant environmental standards.
  3. “A couple of software engineers” are tasked with fine-tuning the engine run profile to meet the environmental standards.
  4. Engine now passes benchmarks, but fails to provide the driving performance that would be preferred.
  5. “A couple of software engineers” are now tasked with finding a solution to this problem, with the inference that they still need to be able to pass the environmental standards.

Given this trade-off, the use of two separate engine maps, tuned for each use case — and additional code to determine which one to use — is almost the only logical outcome.

So yes, it probably was “a couple of software engineers” who wrote the code. It almost certainly couldn’t have been done without fairly sophisticated coordination across the whole product team.

operations as an endangered species

August 12th, 2015

Back in 2007, I took a change in role away from UNIX type systems administration toward network engineering focused roles. This was for a multitude of reasons, not least of which was it better matched my interests. A consideration however was that virtualisation was well into changing the world of systems administration, and with that it was clear that the market for system administration generalists was going to become more, rather than less, limited with time. Essentially as virtualisation took hold, it was my prediction that one system administration generalist could likely be expected to support an order of magnitude more applications; while this isn’t a bad thing, it certainly limited the career scope of working in related spaces. It was my guess that working in networking would be a little safer, being that irrespective of what happened to compute, there’d likely always be a need for managed network connectivity (more on that later; suffice to say I no longer believe this either).

Roll on 2015. If anything, this forecast has demonstrated itself to to be underoptimistic; while Amazon’s EC2 started making itself felt in around 2008, I and many others failed to second-guess that shared infrastructure compute would become the centrepiece of businesses’ deployment strategies so soon; I would have suggested at the time that relying on EC2 and its ilk for mainstream business needs was — somehow — inappropriate governance.

Late 2008 saw the global financial crisis took hold. I don’t think it’s an accident that business interest in gaining efficiencies by relying on X-as-a-service offerings really took hold in its wake; managing business services in-house can be expensive, and short of overt outsourcing of all aspects of a business’ IT needs, using as-a-service offerings allows a reasonable compromise between a tailored solution and a low-cost, one-size-fits-all one. The GFC caused many businesses to search for efficiencies, and going to fully virtualised platforms is a logical way forward.

With X-as-a-service has come the disappearance of highly skilled, highly focused specialists: these are now hired by, and work for, the businesses delivering the service. In a sense, the ranks of generalists has also diminished somewhat; the sense that there’s a trend here is unmistakable.

Information Technology in all its forms is now becoming more like a utility, insofar as everyone has to have a basic understanding of how to operate all aspects of it that are relevant to them, and a very very few — specialists — actually touch any of the moving parts. This isn’t a short lived trend, and the average CxO *wants* the trend to continue; they don’t want to hire people into generic IT operations roles. IT operations roles have been derided as janitorial for quite some time; the disappearance of operations generalists is the net result.

Nor is this a bad thing. Information Technology — almost more than any other career path — is defined by disruption; it’s driven disruption in every other industry it’s touched, and with that disruption comes changes in the very nature of IT itself.

Back to networking. In 2007 I was of the belief that networking would be relatively immune to this; after all, no matter where the compute is housed, people need access to it. There’s an underlying assumption here: that the nature of networking wouldn’t change along with compute and storage — an assumption that is, naturally, wrong. Network-as-a-service, simplified and automated to the point where a skilled network generalist is no longer required in the business itself, is clearly in the near term future for many businesses.

None of this is to say that generalist operations people are now irrelevant: there are still times where a generalist is needed. However what is changing is the need for multitudes of such staff, hired for and working within a business which treats IT as a supporting function. Instead, a small number, possibly working for multiple businesses — in the same way that a business might have maintenance staff come in once per week, or an electrician on-call for light-duty ad-hoc work — is a more likely outcome.

customer satisfaction

August 7th, 2015

I recently had an experience involving very poor customer satisfaction. This isn’t at all surprising — less-than-brilliant customer service is increasingly the norm, as good customer service can be rather expensive to deliver with no direct economically measurable benefit — but the nature of the particular interaction got me thinking about the nature of customer satisfaction.

The industry, organisation, and individuals involved aren’t, for the purposes of this story, relevant; the fact that I’m personally familiar with many of the parties, much more so, but only to the extent that it allowed me the ability to take a step back and think through what the problem was.

Read the rest of this entry »


August 4th, 2015

It turns out that Parliamentary submissions aren’t made under oath, and there are no meaningful penalties whatsoever for lying to Parliament.

Just imagine if there were meaningful penalties. Further, imagine if those penalties applied to the political caste, too.

all software sucks

July 22nd, 2015

The only thing that surprises me in this article is that the attack took until mid-2015 to happen:

[…] The attack […] can compromise those Uconnect computers—an optional upgrade feature that doesn’t come standard in the Chrysler vehicles—through their cellular Internet connection to tamper with dashboard functions and track their GPS coordinates.

For 2014 Jeep Cherokees in particular, [the attack extends] to the vehicle’s CAN bus, the network that controls functions like steering, brakes, and transmission.

Attacks like this become a safety critical issue — rather than just an annoyance — due to two factors.

First, all software sucks, with consumers failing to demand the kind of qualities in software that they now routinely expect in hardware (mostly, to be fair, because consumers don’t know to ask — or how to ask).

Worse, most engineering reliability analyses don’t know how to model software reliability, so it gets neatly tuned out of a hardware failure modes analysis — meaning that it’s then easy to make the incorrect assumption that the software can’t fail, let alone take hardware out with it.

Engineers and other technical types might not make that assumption, but they’re not the ones generally making the decisions on what and where to cut the budget. Which brings us to the second factor: businesses are attuned to looking for maximum profit at minimum cost.

That means in turn that obvious safety considerations, such as not having any form of physical link between safety critical systems and online entertainment systems, are foregone because it’s an easy way to cut costs. One set of connectivity is cheaper than two, particularly when the safety critical systems already have sensors and the like which the entertainment systems can then utilise.

on alternative government

July 22nd, 2015

It doesn’t matter which “side” of politics is best represented by this essay; true democracy comes from truly alternate viewpoints being correctly represented in government:

The Overton window — it’s the political theory that, at any point, there are policy ideas within the window that are seen as acceptable. Mainstream. Anything outside that particular window is seen as mad, bad, or dangerous. Not acceptable. Not to be taken seriously.

It’s my opinion that the Overton window has been significantly pushed to the right over the past couple of years. […] There is a sense that what were once overly punitive ideas are now meat and potatoes.

You know the kind of thing — punishing immigrants, demonising them, even. Cutting benefits from those who need them most, dismantling the NHS, shackling the unions — that kind of stuff. The kind of policies that would have previously raised eyebrows — and blood pressure — are now greeted with a nod and a muttered ‘quite right’.


You need brave people. Unselfish people. Ready to stand outside, prepared to be mocked. Passionate, committed and determined. I don’t see that in many of our politicians.

Last night, I heard a lettuce-fuelled Labour leadership hopeful tell a TV reporter that he is re-evaluating the ‘core values’ in the eyes of the electorate, as if by merely saying those words with his mouth, then the answer will turn up in a taxi.

Let me tell you the core values, sunshine. They are to stand up to inequality, punch hard for those who cannot, and REFUSE to be bowed in the face of battle. Save yourself the money you were going to spend on shiny leaflets. You won’t need them. Just stand up, charge forward and bloody fight.

Labor — the Australian one, that is — was and is wrestling with this same issue; I infer it’s a common problem in all “western” democracies at the present. They still haven’t figured it out, and they started struggling with this so-called problem while they were nominally in charge. Now they’re trying to be functionally the same as the current government, which for all intents and purposes means we’ve got a one-party system.

Thing is, I’m fairly sure that the average voter, when not being told what to think by the media establishments, would be quite happy to see a genuinely alternate viewpoint represented by an opposition party with significant size to it.

Without it, we’re heading well down the path toward an entirely less compassionate society.