Sunday, September 21, 2014

Open Letter to the Testing Community (re: ISO 29119 conflict)

Although folks have been talking about ISO29119 and its possible impact on the testing community for a while now, it wasn't until this past August at CAST 2014 that the issue really took off. James Christie, one of the speakers at the conference, gave a presentation regarding his opposition to the standard. This action is what I consider the catalyst for the opposition that has followed since then. But why the opposition? And why now? These are but a couple of questions I will try to answer during this writing. 

Who are the opponents? 
The main opponents of 29119 seemed to have gathered around the newly formed ISST whose mission, as stated on their website, is to "Advocate for the adoption of context driven testing". I have not yet found any opponent to 29119 that were not a member of the Context-Driven school of testing. And if there are any, they have not been vocal as of yet. 

Why are they opposing it? 
In order to understand the opposition, one has to do a little bit of research and go back in history (not that long just a few years). I have observed that there always has been conflict and friction going on between the ISTQB and the CDT folks. And since the creators of the petition feel that there is a link between the ISTQB and ISO29119 they must oppose it. Evidence of this as well as this opposition has been brewing on many social circles including LinkedIn and Twitter just to name a few, just do a search on any of these mediums and you'll find plenty of lively debates on the subject. In all of the conversations I follow on social media and read on blog posts from the many members of the CDT the common theme is basically the same; we must oppose the standard because we cannot standardize testing. They basically just oppose it but do not offer an alternative. This leads me to believe that they are opposing it because they must! Picture here in the USA you are a member of the Republican party and they propose a bill. The Democrats will review this bill and decide they are against it and request that all members of their party vote against this bill. But wait, what about all of the members of both political parties individual opinion about this bill? Well, that doesn’t count as much as the greater goal: to advance the parties political agendas. 

Why now? 
Quite frankly the reason this petition to #stop29119 happened is because the time was right. There was a captive audience (at CAST), a presentation was given (regarding 29119 opposition), the response was observed and the opportunity taken. And the petition was born. Granted this is an over-simplified timeline, however, based on the transcripts of roundtable discussions I have read its all based on facts. 

What about us free-thinking testers that welcome input from all schools? 
I for one choose not join any particular school of thought. I am a free-thinking tester that believes that testing has always been a human function and based on context. In fact everything we do, as humans, is based on context! I also welcome any structure or process offered by an organization or individual (standard or not), why? Because if we truly believe that testing is based on context, then obviously one must keep an open mind to _ALL_ approaches, processes and methods. Further, I think that opposing a specific approach, process or method limits our ability as a tester and does not help our customer in the long run. This is why I have abstained from signing the #stop29119 petition and urge all folks in our profession to do your own research before signing. Ask yourselves how will this standard change the way I test? My answer is, if you believe testing is a human function and based on context, it will not. 

Conclusion 
As free-thinking testers it is our responsibility to take charge of the direction we want our profession to head towards. It is my view that keeping an open mind and including everyone is the only way to stop this conflict and advance our profession. Everything has its place; standardization and certification as well as no standardization or certification. As context driven testers we should know when to choose each one. All this petition to #stop29119 is doing is polarizing testing and the members of our community. This goes against my values and my mission so I choose to abstain from signing the petition.

30 comments:

  1. Hi, Freddy...

    You ask, "What about us free-thinking testers that welcome input from all schools?"

    I'll put this as clearly as I can: you are welcome to do that under a context-driven model. However, "context-driven" doesn't mean uncritical; nor does it mean that you must adopt every part of every context in which you might find yourself. It means that you must consider context *first*, and deal with it according to your best lights. This is the opposite of what a standard is intended to do; a standard demands that you consider *the standard* first, and that the standard is paramount. This seems in opposition to what you are advocating here: freedom of thought.

    ReplyDelete
    Replies
    1. Greetings Michael,

      Thanks for the thoughtful reply. And yes I agree with you that context-driven doesn't mean uncritical nor that we must adopt every part of every context. I get that.

      What I am saying is that the folks at ISO are not saying this either. I can see how you can come to this conclusion being that "the standard", if a company chooses to implement a project with it, must be followed. But in contrast the same would be true if a company chooses to implement a project using _ANY_ other process.

      The way that I test widget A will not change whether I am following a standard or not. Testing is testing in my view.

      Delete
  2. Hello again...

    You claim that the opponents of 29119 say "we must oppose the standard because we cannot standardize testing". That's true, but that is only one reason why I oppose the standard. (I believe you're aware of this, because you've read my blog---or if you haven't, you've commented on it without reading it, which would greatly surprise me.)

    ReplyDelete
    Replies
    1. Hi again,

      yes I did read your blog post regarding the FAQ. In fact I've read a lot of your articles. You may find this hard to believe but I do agree with a lot of what you have to say about testing in general. We just happen to disagree on opposing the standard.

      I want to be clear that I neither support nor oppose the standard. Because all I have read (as most of us have) is an outline. I cannot make a decision based on just that and I cannot fathom how anyone else can, absent the standard.

      The general theme in folks opposing the standard is that it is young and not ready for standardization. That is what I am trying to convey by that quote. All other reasons that have been cited are either speculation or opinions (which I respect both), however, not enough no sell me on opposing it. Because I either don't share the opinion or the speculation.

      Delete
  3. I apologize for commenting in this disjointed fashion, but bugs in Blogger's mobile implementation make editing the comment very difficult indeed---a source of consistent frustration on your blog and on others. Usually I just give up.

    ReplyDelete
    Replies
    1. I apologize for bugs in the Blogger platform. I will look into this. Glad you didn't give up.

      Delete
  4. One more for the road: you say that we oppose the standard, but we do not offer an alternative. Once again, we DO offer an alternative: EVERYTHING ELSE. Your comments above are answered here: http://www.developsense.com/blog/2014/09/frequently-asked-questions-about-the-29119-controversy/

    ReplyDelete
  5. Am I a member of the Context Driven School? I have never claimed to be. I joined ISST after my talk in New York. I am a member of AST too, but I joined that after my proposal for my CAST talk was accepted. I am a member of these organisations because they are aligned with my own views on issues I was already campaigning on. Your claim that we are opposing ISO 29119 to toe the party line is way off the mark.

    ReplyDelete
    Replies
    1. Hi James,

      I know I am generalizing, my apologies for that, I am not saying you specifically are a party liner. The reason for my "political party analogy" is because as I have been following discussions on social media regarding this conflict I have, on numerous occasions, witnessed folks inciting other folks to go sign the petition and be heard a la political party style.

      Now please don't get me wrong, I am not saying there is something wrong with this approach but, lets call a spade a spade. The CDT folks have been actively recruiting for this stop campaign, they have even openly admitted this plus I have witnessed it myself on social media; again, there is nothing wrong with this. I am just calling it like I see it.

      I admire the fact that you have brought this issue to the forefront. I just don't subscribe to campaigning to stop something that I have not even reviewed myself. Plus I truly don't believe that standard or not by testing (the way I actually test) will be affected by it.

      Delete
    2. The standard is wrong in principle. I've made that argument in many blogs and articles. Others have argued the same. The content of the standard is a secondary issue.

      Delete
    3. If you had to pick one blog or article for folks to read (out of the many) that describes how you think the standard is wrong in principle which one would you recommend I and others read?

      Delete
    4. I wrote this in 2009 to argue against the idea of testing standards. I hadn't heard of ISO 29119 when I wrote it. I still stand by what I wrote, though perhaps I was too generous about the content of IEEE829.
      http://clarotesting.com/page20.htm

      Delete
    5. Great minds think alike. I had already picked this one and started reading it. Mainly because it was written in 09. More later as I am juggling reading it and work.

      Delete
    6. Hi James,

      Thanks for pointing me to that post. It was indeed a lot of information to digest. I also took the liberty and read (as well as re-read) a few of the other posts you had linked within there. I understand a bit better your stance on this issue now. As well as your rationale for opposing the standard.

      And yes I can see that you have been opposed to a standard for software testing process for a while. I also see that your motivation and heart is in the right place (not that it should matter to you what I think) but I did want to tell you anyways.

      I wanted to thank you very much for your perspective (based on your many years of experience). It has been a learning experience.

      FV

      Delete
    7. Thanks. It was a long article and there are plenty of other strong arguments against standards. However, it does make clear where I'm coming from and explains why I was opposed to ISO 29119 in principle

      Delete
  6. Some thoughts on your post:

    - You claim that people opposing the standard do not offer alternatives. Let me give much more extreme proposition: Let's standardize art! If you oppose this idea, you can give arguments why it's not a good idea, but there is no alternative scheme you can offer other than let's not standarize art as it is not a good idea.

    - You speculate that maybe people are signing just because they are urged to do so. I would advise that you wouldn't do that because it's quite insulting thing to say and can easily make people less responsive to your ideas. Maybe you have seen worse, but the couple advertisements for the petition I have seen have all said: "Please sign if you agree."

    - To me it seems that "free-thinking" in your vocabulary means "not taking a stance". It's like saying we always have to test a product in all possible ways and methods because we have to keep all options open. I believe that you should try to say that, given these constraints and possibiliies, in this particular situation, some actions are better and more suitable than others. Notice that many opponents of ISO 29119 are not against standardization of things in itself. It's just that in this situation it seems very hard or impossible to write a good standard for software testing and having a poor one is worse than not having any.

    - Finally, you say that "keeping an open mind and including everyone is the only way to stop this conflict and advance our profession." You seem to say it like discussing this is waste of time and doesn't give anything to the community. In my opinion it gives something as it makes (at least hopefully) people to clarify their thoughts because they have to put them on paper. Additionally, if we stop the conflict, which way it should end. Should there be a standard in the end?

    ReplyDelete
    Replies
    1. Hello Esko,

      Thanks for your thoughts, I do appreciate them. Here are some thoughts for you regarding your comment:

      "Let's standardize art!" -- Its ok if you feel that testing is an art; you are certainly free to think that. I choose not to. I am not an artist, in fact I do draw from time to time but its really not that good.

      "You speculate that maybe people are signing just because they are urged to do so." -- No, not speculating. I am obviously not basing my comment on "the couple of ads you have seen for the petition" I am basing them on what I have seen. And in fact people are being urged to do so. This is not called speculation, this is called to deduce "arrive at (a fact or a conclusion) by reasoning; draw as a logical conclusion." You too can come to this conclusion by only following the discussion on twitter with hashtag stop29119. There you will find lots of folks being urged to sign. It is not my intention for anyone to be insulted (since I am not insulting anyone) why do you have to go there? You seem to say that you like discussing and debating but then get insulted when an opposing point of view is expressed? Please do not take offense I am just stating my opinion.

      "To me it seems that "free-thinking" in your vocabulary means "not taking a stance" -- Well to me "free-thinking" means arrive at a decision when you feel comfortable that you have reviewed all the evidence. To me "free-thinking" means take the stance of not choosing a side if you want to. Like I did. My stance is not to choose a side. If for no other reason that most folks haven't even read the standard yet (I'm one of them).

      "It's just that in this situation it seems very hard or impossible to write a good standard for software testing" -- I will ask you this, how will this standard affect or change the way that you test? If you truly believe, as I do, that testing is a human function how will a standard change the way you actually test? My answer is that it wouldn't. Testing is testing. Right?

      "keeping an open mind and including everyone is the only way to stop this conflict and advance our profession." You seem to say it like discussing this is waste of time and doesn't give anything to the community." -- no I am all for discussing and debating, however, this is not the same as objecting and rejecting it for reasons like for example "My methodology had no fair shot at selection".

      "In my opinion it gives something as it makes (at least hopefully) people to clarify their thoughts because they have to put them on paper." -- I do agree with you there. I wouldn't have put my thoughts on this electronic paper if that hadn't happened.

      "Additionally, if we stop the conflict, which way it should end. Should there be a standard in the end?" -- The conflict should stop and a debate should begin. Should there be a standard in the end? well that is the million dollar question. What do you think a standard should look like? or do you think there shouldn't be one?

      Again thanks for your thoughts.

      FV

      Delete
    2. The "Let's standarize art" didn't say that testing was art but was an allegory. It was meant to say that "Let's not standardize testing" is an alternative.

      You still can't know why people have signed the petition if they haven't themselves said: "I signed the petition because I was asked to." This is why I call it speculation. I reviewed all the tweets with hastag stop29119 and most of them that urged people to sign it also had a link to a page that explained what the petition is about.

      You can insult people even if you don't mean to or if you are just stating your opinion. I reserve right to feel insulted if you do so. I find it insulting that you imply I (and others) have signed the petition just because I was told to. It's the same as saying I'm not thinking for myself. This is why I suggested you wouldn't say so. Maybe you didn't mean to say that?

      This standard may affect how well I can spend the time I have for testing a product if it requires me to do things that don't make sense. This way it may affect my testing.

      What is your definition for conflict and debate here? Aren't we having debate about the conflict? In your opinion, what should we debate?

      I signed the petition so my view on the matter is rather clear. I wanted to get you more involved and get your opinion. Even if you haven't read the standard, you have read the criticism. What is your opinion on the following claim?

      "Significant disagreement and sustained opposition exists amongst professional testers as to the validity of existing testing standards, and that there is no consensus as to their content."

      Delete
    3. Hi Esko,

      "I reserve right to feel insulted if you do so." -- Yes you do.

      "I find it insulting that you imply I (and others) have signed the petition just because I was told to." -- Yes you have the right to get insulted even when no one is insulting you. I cannot take that away from you.

      "It's the same as saying I'm not thinking for myself. This is why I suggested you wouldn't say so. Maybe you didn't mean to say that?" -- in fact I didn't say that. You said that. What I did do was use a political party analogy which I found fit my observations.

      "This standard may affect how well I can spend the time I have for testing" -- can you tell me exactly how the standard will do that?

      "What is your definition for conflict and debate here? Aren't we having debate about the conflict? In your opinion, what should we debate?" -- what?

      "I wanted to get you more involved and get your opinion. Even if you haven't read the standard, you have read the criticism. What is your opinion on the following claim?" -- It is my view that without reading the standard I cannot objectively have a view.

      I understand you oppose the standard. I get it. I'm not trying to convince you otherwise. The purpose of my letter is to let everyone else know (if they don't already know) that, as free-thinking testers we should abstain from taking action until we have all the facts.

      FV

      Delete
  7. Ok, let's try another way. Could you please try to answer as well as you can to these questions. "I don't know" is of course valid answer.

    Do you agree that just doing what others tell you is the same as not thinking to yourself?

    You said that we should end the conflict and start a debate. I don't understand this statement. Could you clarify what you mean by it?

    Do you agree that there are people that oppose the existing testing standards and their content?

    ReplyDelete
    Replies
    1. "Do you agree that just doing what others tell you is the same as not thinking to yourself?" -- No I do not agree. These are not mutually exclusive.

      "You said that we should end the conflict and start a debate. I don't understand this statement. Could you clarify what you mean by it?" -- No I cannot. I think I've made it as clear as I can. If you can then please do so. Keeping in mind that you'll be using your own definitions not mine.

      "Do you agree that there are people that oppose the existing testing standards and their content?" -- yes

      Delete
    2. Hmm. I can see how you could think to yourself and still do what others tell you to do. But I can't see how people can sign _because_ they are told to sign (as in some cases in politics) and still claim to think for themselves.

      In politics, the behaviour of voting according to party is an agreement inside the party. There is no such agreement (or authoritarian ruler) in testing communities.

      --

      In my opinion there is a conflict because some people find ISO29119 desirable and others seem it harmful. There is a debate on what we should do. The only way to end the conflict is to have the debate and reach understanding and agreement. This won't happen in near future.

      --

      Do you agree that the people opposing the testing standards are professionals?

      Delete
    3. "There is no such agreement (or authoritarian ruler) in testing communities." -- Of course there is an agreement. The agreement was reached at CAST2014 after the James Christie presentation. Check it out here: http://www.youtube.com/watch?v=A721ltyVw3o you have to skip to the end during the open season. This was the catalyst for the agreement, then came the round table discussions which solidified the agreement, found here (in three parts):
      http://bit.ly/1rlBWWZ - Part 1
      http://bit.ly/1mqE9QG - Part 2
      http://bit.ly/1ognjQ4 - Part 3
      So, while there is no authoritarian rule there is "an agreement inside the party" hence my analogy.

      "In my opinion there is a conflict because some people find ISO29119 desirable and others seem it harmful." -- I concur. I'll add that there is yet another group of people that do not feel threatened by standards, like me (and there are others).

      "Do you agree that the people opposing the testing standards are professionals?" -- I'm afraid I cannot possible know the answer to this. Unless you give me specific names and I have had the opportunity to evaluate these folks, then and only then will I be able to answer this question objectively.

      FV

      Delete
    4. Thank you for the links.

      In a political party the possible agreement is made prior to individual matters that party members are supposed to vote uniamously. Possibly there is also an agreement on which kind kind of matters you can make an expection and make an individual voting decision.

      The agreement to make this petition is not the same thing. Signing is not required from anyone or by anybody. There is no sanctions on not signing (e..g expelling from party). The agreement made in the conference was that something needs to be done and starting a petition would be a good idea.

      ----

      Unfortunately I can't yet provide you a list of names that signed the petition. Hopefully we see that list at some point. However, would you agree that people writing in the blogs on this list (*) are professionals?

      * http://www.huibschoots.nl/wordpress/?page_id=1771

      Do you recognize some of the people you have seen opposing the standard in different social media are professional testers?

      Delete
    5. You're welcome.

      "There is no sanctions on not signing (e..g expelling from party)." -- There is no expulsions from political parties either. At least here in the US we (individuals) declare our affiliation to the party, not the other way around.

      "The agreement made in the conference was that something needs to be done and starting a petition would be a good idea." -- That is what we have been told. I believe these agreements (i.e. the call to action) was pre-planned prior to the James Christie presentation. This ties in with your comment: "In a political party the possible agreement is made prior to individual matters that party members are supposed to vote uniamously." -- hence my political party analogy.

      Please clarify why you keep asking me if I think these people are professionals. Can you please provide me some context? In the absence of context I would stand by last answer. And add, why does it matter what I think about their professionalism? (again context)

      FV

      Delete
    6. Well, I wasn't in the conference and at least 3/4 of the other signers were not in the conference either. They have still signed the petition after it was made public.

      If there is a petition, of course people are encouraged to sign it. However, with the political party analogy you are assuming that people associating themselves with context-based testing community not agreeing with this particular action are still signing it. I don't believe this to be the case as it's quite against the idea of judging yourself what is sensible.

      I keep asking about people opposing the standard because if you agree that there is such people and that they are professionals and that they have spoken against the testing standards for some time, it's only a question of contents of one ISO guide if you could agree with the following:

      "It is our view that significant disagreement and sustained opposition exists amongst professional testers as to the validity of these standards, and that there is no consensus (per definition 1.7 of ISO/IEC Guide 2:2004) as to their content."

      Delete
    7. i was wrong when saying "expulsion from party". What I meant to say was "explusion from parliamentary group" which is something that at least in Finland and some other countries may happen as discipline action. The members of course stay in the parliament, but either has to join another group or be on their own.

      Delete
  8. Another message that I wanted to separate from my questions.

    You quoted me saying:
    "This standard may affect how well I can spend the time I have for testing"

    Then you asked:
    "can you tell me exactly how the standard will do that?"

    The text that you quoted already had the answer to your question in the same sentence!
    "This standard may affect how well I can spend the time I have for testing a product if it requires me to do things that don't make sense."

    ReplyDelete
  9. If the standard was published in Amazon as a book people can buy, we will see how many people waste money for such useless book. There are tons of great books about testing today, we don't need read standard, but read books, and follow great leaders.

    ReplyDelete
    Replies
    1. Anonymous its not about "reading the standard", its about using it, depending on context of course, to develop software.

      Delete

Creative Commons License
VGP-Miami Web and Mobile Automation Blog by Alfred Vega is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.