The DIY Syndrome

When facing a new design objective, we check off all the established tools and flows we know we are going to need. For everything else, we default to an expectation that we will paper over the gaps with scripting, documentation and spreadsheets. And why not? When we don’t know what we will have to deal with, in documentation, scheduling, project communication or requirements tracking, starting with unstructured MS Office documents (Word, Excel, PowerPoint) looks like a no-brainer. There’s nothing new to learn and because these documents are unstructured, you can do whatever you want and adapt easily as needs change.

This works well when supporting the needs of a small team, or even larger teams when expectations around structure and traceability are primarily informal and any compliance documentation required with products can be generated through brute-force reviews. But it doesn’t work so well when you’re trying to track changes across multiple inter-dependent products and you need to keep track of how changes relate to evolving requirements, why changes were made, the discussion that went into accepting (or rejecting changes) and being able to defend all of that to a customer or in a compliance review.

This isn’t about design data management (DDM); of course you need to handle that too. This is about traceability in the decision-making that went into approving changes to the requirements, which ultimately led to the design changes tracked in the DDM. You can store comments in a DDM when you check-in a design change but 99 times out of 100 these are pretty informal, and far from meeting traceability requirements of standards like DO-254.

So you decide to manage requirements changes/tracing in Word or Excel or a combination of the two. Seems reasonable. You turn on review tracking so reviewers can add comments and make changes. Nice and disciplined, traceable, you know who made what comments, what’s not to like? Again, there’s nothing wrong with this approach when what you are doing is entirely for the benefit of a local team, isn’t going to lead to need for automation and doesn’t have to be formally shared with independent and distributed design team or with customers.

When any of these needs arise, DIY (do it yourself) requirements tracing in Office starts to show some fairly serious weaknesses. I’ll use Word to illustrate; Excel is better in some respects, worse in others. We’ll ease into why slowly, starting with revision tracking. With tracking turned on, Word does a good job of tracking changes and comments, requirements by requirement, but it doesn’t show when comments or changes were made, comments are not time-ordered and, without manually-added information, there is no way to link a change or comment to a higher-level requirement or to impact on lower-level functions.

In addition, anyone who has used revision-tracking in Word knows that after a few rounds of review, text littered with edits and comments becomes difficult to read. You have a meeting in which you agree to accept all changes (or a subset), the text is cleaned up but you just lost traceability and, if you remove comments, history on why changes were made. When the document is finally released, all of those edits and comments disappear.

Suppose you decide to restructure the document; maybe a select set of requirements and associated comments should be moved under a different higher-level requirement. Do you do this with change tracking on (which will mess up your document even more) or do you turn tracking off, make the move then hope you remember to turn it back on again? This gets scary when structure changes become common.

Of course you can revision control the doc in a DDM, but how often should you do that? Per individual change (in which case, why are you using Word tracking) or periodically? If periodically, how easily can you compare versions of the document, which may contain figures and other explanatory text for requirements, varying substantially between versions? Methods to compare documents work well up to a point but can be derailed by significant differences. Which makes it very painful to compare how requirements have evolved over time.

Then think about how you might handle hierarchy. Requirements generally aren’t a flat list. There are top-level requirements, sub-requirements under each of these and so on down to leaf-level requirements. You could put all of this in a single document with sections and sub-sections and sub-sub-sections and so on until Word has right shifted so far that whatever text and diagrams you have to enter are squeezed into a narrow column at the right of the page. You might also start to worry about Word reliability on large documents. Word is a great tool, very widely used but I have personal experience that as documents get very large, behavior starts to become unpredictable.

More probably you split the whole thing into multiple documents, but now you have to track interdependencies between these documents. Word doesn’t provide much help there. You might use hyperlinks but how are you going to handle ancestor and descendent connections on a requirement? And you better be careful about file locations – moving files could break links. Maybe put it all in a sharefile location – more complications.

Which brings me to another point; in general, how do you plan to track relationships between requirements in Word? And if you make a change to a requirement, how do you know what other requirements will be impacted? In theory through the document organization, but how many projects do you know of where all requirements are nicely bounded in a tree hierarchy so that any change in a requirement will only impact sub-requirements, and all appropriate sub-requirements (a coverage question)? Conversely, what will be impacted in higher-level requirements if I change this requirement?

There are more issues, but you get the point. Using Word or Excel starts easy but can become very cumbersome very quickly. A common reaction at this point is to launch a project to develop Visual Basic software or something similar to automate away these problems. At that point, you should say “stop – what are we doing?”. I have personal experience in developing apps around Word and Excel. It’s do-able, it takes a lot of time and it takes a lot of maintenance. Is that really the best use of your time, or should you look for an existing commercial solution?

You might check out Aldec Spec-TRACER. It addresses all of these points, it can import existing documents and output spec, compliance and other documents. In the long haul, it will cost you a lot less than your DIY project and it won’t go up in smoke as soon as the one person who understands the doc and customization finds another job. You can watch an Aldec webinar on this topic HERE.