SEMAT's challenge: carts, horses, and drivers

I have been following SEMAT since its inception, yet hesitated to sign as a public supporter, even though the vision is lofty and I agree that the underlying issue is critical. The list of signatories and supporters is truly impressive: it's filled with people I respect and follow (although some of those I admire most, eg Laurie Williams, are missing). So why have I, still, not yet signed?

Getting a problem well-solved is critically dependent on defining it well. But as I read the position papers, I noticed that many identified part of the "software problem" and then immediately described how their own preferred approach already solves it. To me that was a "bad smell" and seemed like "putting the cart before the horse".

To date, I think one essential piece of a clear problem definition for SEMAT has been lacking. To extend the cart and horse analogy a bit (hopefully not to the breaking point):

• A cart needs one or more horses, certainly. A cart without passengers can certainly go fast if many strong horses can be harnessed together - but then no one but the driver (and the horses) will care where it ends up. 

• A cart with horses but without the right driver still isn't likely to get the passengers of the cart where they need to go.

• And without talking to the passengers, how can any driver know where they need to go?

So even if SEMAT succeeds in building a theoretical harness or methodological framework that gets all of the horses pulling together - a significant challenge in itself - if the practitioners of software industry aren't on board and giving direct guidance to the driver, it won't get the people who matter to their brave new world of better software.

My overall impression of SEMAT is of many strong people (or horses) trying to nudge the cart from behind or pull independently on the reins from the front, steering the cart in the direction and at the speed each believes it needs to go. In such a scenario, going around in circles or tipping the cart seems likely.

So I ask: Who are the passengers, and who will drive? Exactly whose problem are the SEMAT participants trying to solve, and what aspects do those people care about most towards solving it?  i.e.

• Who are SEMAT's customers? 

• Which customers matter most? Not all stakeholders are equally important; they must be prioritized, and the SEMAT participants must collectively agree on the priorities. 

• What do those customers need? The stakeholders of SEMAT, especially the highest-priority customers, must be actively engaged and involved in defining the problem and prioritizing the needs SEMAT tries to solve.

I see two possible customer groups which SEMAT should consider:

1) the world (literally) of industrial software practitioners

2) the even bigger world of people who buy and use software-based products (the actual end customers of group #1)

One might argue that Group #1, the industrial software practitioners, are responsible for knowing what their Group #2 end customers want and need, and for meeting those needs, to stay in business; that it's impractical to include a representative sample of Group #2; or that we all use software products in our lives and therefore any of us can 'represent' Group #2. I don't totally buy that argument, but I could live with it if SEMAT decided that their customer base was Group #1 and not #2. So let's call Group #1 the passengers in the cart.

What I would definitely be unwilling to accept would be if the illustrious participants in SEMAT decide they are, themselves, adequate proxies for today's software practitioners ("we know what they need, we don't need to talk to or involve them"). Such an approach would risk falling prey to one of the most common failings of the software projects they aim to improve: assuming they know what their customers/markets need, and developing or selling a perfectly good solution to the wrong problem.

Even industrial software engineering researchers like me are just proxies for the practitioners in our company; we are not them. Even those of us who have been 'them' in the past - which I think applies to many of the SEMAT participants as well as myself - are not 'them' today, and we should not assume that 'they' still have the same problems and priorities we knew and lived. 

My impression that this is what was happening, perhaps unintentionally, has underpinned my hesitancy in 'signing on' to SEMAT. Someone who truly hears and represents what Group #1 needs today must DRIVE the cart to where the group needs to go. 

Why does this matter?

I believe SEMAT will fizzle and fail if it is not focused strongly on what the software industry (most important customer/stakeholder class) needs - and 'software industry' cannot be represented adequately by consultants, methodologists, academics, or tool-makers. It should mean primarily the people who design, write, test, and manage software products and services that our lives depend on.

Reaching intellectual harmony among the SEMAT signatories is a stretch goal, but possible. Yet even the achievement of a  new unified theory will deliver no lasting practical value if it is not widely adopted and does not make software product deliveries better in  practice. The adoption topic - the difference between theory and practice, and the technology transfer gap between academia and industry, i.e. convincing the passengers to get on board - I will save for another day.

In conclusion, I was glad to see that during the Zurich workshop, the discussion turned towards trying to agree on the problem they were trying to solve (what IS software engineering?) and that a 6th track of work on defining requirements for SEMAT has been started. However, the focus of that 6th track should not be just on figuring out what software engineering is - it should be on figuring out who the passengers (Group #1 practitioners) are, eliciting where they need to go, and deciding who will drive the cart and who will tell the driver where to go. (For now let's ignore that it may also involve deciding that more than one cart, team, route, and destination are needed.) If the cart isn't going where Group #1 needs to go, why should they get on board? And if they aren't on board, the cart trip doesn't really matter - the software products used by everyone in Group #2 will not get any better.

My respectful challenge to the SEMAT participants: rather than hoping to win a battle for the driver's seat, are you willing to go out to talk to your potential passengers in Group #1, or be one of many strong horses of many colors pulling the cart to where the passengers (and driver, on their behalf) need to go?