4
Posting this message on behalf of Gopal.
Regards
Rajdeep
________________________
Please permit to ask a stupid question or two or three..
- Is there a software tool that can measure the effectiveness of a technical documentation?
- I mean: Is there a tool that can rank a technical documentation in a scale of say, 1 to 10, with 10 indicating that that particular documentation is the most effective? .
- Well, before answering that, is there a term `effectiveness of a technical documentation’ ?
If you
- laugh at these, it will strengthen your facial muscles.
- frown at these, it may help your temple muscles
- start thinking, it may help your grey cells
- send me an answer, it will delight me.
Win-win situation, all the way!
Best,
Gopal
I am always delighted when you poke a question or two to me; it is a learning curve for me. Anyway, to answer your question, did you get a chance to attend one of my many presentations on ‘Usability Testing of Documentation’? if not, then let me tell you that I started taking interest on usability due to a purpose of identifying a process of measuring the effectiveness of technical documentation. Usability in any form is basically understanding if your product is easy to use or not.
There are no software tool available in the market according to my knowledge that can assess the effectiveness of a tech doc. How will you measure something that cannot be quantified? Is content quantifiable? No! it is its quality of writing that tags a content good or bad. In user documentation, we rely a lot on customer’ feedback and hardly it comes.
So, one of the ways to assess the effectiveness of a tech doc is to develop a knowledge based portal, make docs available on the portal and ask your internal customers to rate it-1, 2, 3 and so on. The other thing is develop an excel sheet, collating data from the tech support team such as which are the topics that customers regularly ask, are there new queries randomly asked and if something new always. Next, you go back to your document and identify if the information is missing or not. If yes then your document is an ineffective one.
Yes, there are terms, which are also called metrics in tech doc “effectiveness’ , ‘reliability’, ‘memorability’ and so on, but there are no available tools to assess content, which is not quantifiable. You can only convince your management to purchase a tool when you have data of the content. Unfortunately, technical writers have an editing team in every company to do this.
Thanks for asking and listening to me sir!.
Regards
Rajdeep
Hi Gopal,
Your question made me laugh first, then made me frown, then made me think, and finally I am answering your question. So your question was definitely effective!
Anything that is done with a purpose can be more or less effective. So can technical documentation. The question of effectiveness of technical documentation is highly pertinent, and it should guide us in writing documents. I’m afraid, however, there are no tools to measure the effectiveness of a document. The factors are too many and too specific to the context to be measurable by a tool.
Can we measure the effectiveness of a technical document in any other way? I believe we can, but you have to devise the effectiveness metrics (similar to quality metrics but different) that would suit your particular situation. The following questions can help you in this regard:
1) What are the objectives of your document?
We must ask this question to ourselves before beginning to write a document.
2) How can you measure the extent to which your document meets its objectives?
Maybe usability tests can help. The guidance of an experienced writer, peer review, or client review (if possible) will also be useful.
3) How can you know whether the document actually met its objectives?
This is relevant only after your document is delivered to its target audience. You can ask your audience to rate your document on various parameters.
Regards,
Bibhu
From the knowledge which I gained by reading and googling, I believe, that usability testing is a way to measure the effectiveness of the user documentation.
You conduct a test with two groups of potential users. To one group, you give your product to install and use for a period without documentation. To the other group, you give the product with documentation. Once you have done this, you ask each participant:
(a) What value does the document hold?
(b) Was it effective in reducing the cognitive friction when working on the product?
Yes, there is no tool to measure the effectiveness. But, the rating of the end user can perhaps be considered.
Hi Gopal,
To be frank, your question directly took me to “Send me an answer” mode because I’ve been googling since morning on the same lines but slightly different from your need.
As we have a common question on quality metrics for technical documentation, I found some of these in web and guess it should help both of us:
(Source: http://www.stc-carolina.org/newsletter/tiki-index.php?page=Documentation+Quality+Metrics)
Rate the document based on the following parameters:
- Targeted to the correct audience
- Accurate
- Complete
- Well written
- Easy to navigate
- Properly indexed
- Appropriately illustrated
- Practical
- Relevant
- Consistent
(Source: http://www.stcuk.org/newsletters/Nov2006/QualityAssessment.htm)
Quality Statistics for Content
Various techniques can be employed by technical communicators to measure the quality of written material using metrics and statistics that focus on the structure and language used. Some proprietary word processors provide a measure of readability based on one or more recognized tests, such as:
Flesch Reading Ease. This computes readability based on the average number of syllables per word and the average number of words per sentence. Scores range from 0 to 100. Standard writing averages approximately 60-70. The higher the score, the greater the number of people who can readily understand the document.
Flesch-Kincaid Grade Level. This computes readability based on the average number of syllables per word and the average number of words per sentence. In this case, the score indicates a U.S. grade-school level. For example, a score of 8.0 means that an eighth-grade student would understand the document. Standard writing approximates 7.0 to 8.0.
Coleman-Liau and Bormuth Grade Levels. These are similar to the Flesch-Kincaid test in that they measure readability by grade-school level, but instead use word length (in characters) and sentence length (in words) to determine the value of the level.
Use of Passive Voice. It is generally considered more effective to write in an active rather than a passive voice. Therefore, a measure of the percentage of passive sentences is often used as a quality metric. Active voice emphasizes the agent of the action and is therefore more direct. For example, ‘Press the Play button’ is written in active voice, while ‘The Play button can be pressed’ is written in passive voice.
————x————–
Thats all I’ve in hand to share with you. Incase i come across something better, i’ll put it across here and u too share with us if you’ve found anything useful on these lines.