Examples of Mental Mistakes Made by Systems Engineers While Creating Tradeoff Studies

James Bohlman, A. Terry Bahill

Abstract


Problem statement: Humans often make poor decisions. To help them make better decisions, engineers are taught to create tradeoff studies. However, these engineers are usually unaware of mental mistakes that they make while creating their tradeoff studies. We need to increase awareness of a dozen specific mental mistakes that engineers commonly make while creating tradeoff studies.

Aims of the research: To prove that engineers actually do make mental mistakes while creating tradeoff studies. To identify which mental mistakes can be detected in tradeoff study documentation.

Methodology: Over the past two decades, teams of students and practicing engineers in Bahill’s Systems Engineering courses wrote the system design documents for an assigned system. On average, each of these document sets took 100 man-hours to create and comprised 75 pages. We used 110 of these projects, two dozen government contractor tradeoff studies and three publicly accessible tradeoff studies. We scoured these document sets looking for examples of 28 specific mental mistakes that might affect a tradeoff study. We found instances of a dozen of these mental mistakes.

Results: Often evidence of some of these mistakes cannot be found in the final documentation. To find evidence for such mistakes, the experimenters would have had to be a part of the data collection and decision making process. That is why, in this paper, we present only 12 of the original 28 mental mistakes. We found hundreds of examples of such mistakes. We provide suggestions to help people avoid making these mental mistakes while doing tradeoff studies.

Conclusions: This paper shows evidence of a dozen common mental mistakes that are continually being repeated by engineers while creating tradeoff studies. When engineers are taught about these mistakes, they can minimize their occurrence in the future.


Full Text:

PDF

References


REFERENCES

M. Abdellaaoui, Parameter-free elicitation of utility and probability weighting functions, Management Science, 46(11) (2000) 1497-1512.

A. T. Author2 and F. F. Dean, Discovering system requirements, Chapter 4 in A. P. Sage and W. B. Rouse (Eds), Handbook of Systems Engineering and Management, Second edition, John Wiley & Sons, 205-266, 2009.

A. T. Author2, and W. J. Karnavas, Risk analysis of a pinewood derby: A case study, Systems Engineering, 3(3) (2000), 143-155.

A. T. Author2, Decision Making and Tradeoff Studies, slides 123-164, http://www.sie.arizona.edu/sysengr/slides/tradeoffStudy.ppt, last accessed October 2008.

H. Bleichrodt and J. L. Pinto, A parameter-free elicitation of the probability weighting function in medical decision analysis, Management Science, 46(11) (2000) 1485-1496.

W. L. Chapman, A. T. Author2 and A. W. Wymore, Engineering modeling and design, CRC Press Inc., Boca Raton, FL, 1992.

M. B. Chrissis, M. Konrad, and S. Shrum, CMMI: Guidelines for Process Integration and Product Improvement, Pearson Education Inc., Boston, 2003.

CMMI for Development, ver. 1.2, Software Engineering Institute, Pittsburgh, PA, 2006, http://www.sei.cmu.edu/cmmi/, retrieved February 2009.

J. Daniels and A. T. Author2, Hybrid process combines traditional requirements and use cases, Systems Engineering, 7(4) (2004) 303-319.

J. Daniels, P. W. Werner and A. T. Author2, Quantitative methods for tradeoff analyses, Systems Engineering 4(3) (2001), 190-212.

B. R. Forer, The fallacy of personal validation: A classroom demonstration of gullibility, Journal of Abnormal Psychology 44 (1949), 118-121.

I. F. Hooks, and K. A. Farry, Customer-centered products: Creating successful products through smart requirements management, AMACOM, New York, 2001.

E. Hull, K. Jackson, and J. Dick, Requirements engineering, Springer, London, 2005.

INCOSE, INCOSE Systems Engineering Handbook v2a, 2004. Retrieved March 2005 http://www.incose.org/ProductsPubs/incosestore.aspx

D. Kahneman, A perspective on judgment and choice: Mapping bounded rationality, American Psychologist 58(9) (2003), 697-720.

D. Kahneman, Maps of bounded rationality: a perspective on intuitive judgment and choice, Nobel Prize Lecture, December 8, 2002, this site also has a video of the speech, http://nobelprize.org/nobel_prizes/economics/laureates/2002/kahneman-lecture.html

D. Kahneman, and A. Tversky, Prospect theory: An analysis of decision under risk, Econometrica 46(2) (1979), 171-185.

W. J. Karnavas, P. J. Sanchez and A. T. Author2, Sensitivity analyses of continuous and discrete systems in the time and frequency domains, IEEE Transactions on Systems, Man and Cybernetics SMC-23 (1993), 488-501.

R. L. Keeney, Value-focused thinking: A path to creative decision making, Harvard University Press, Cambridge, MA, 1992.

J. L. Marquard and S. M. Robinson, Reducing perceptual and cognitive challenges in making decisions with models, in Decision Modeling and Behavior in Complex and Uncertain Environments, Eds. T. Kugler, J. C. Smith, T. Connolly and Y. J. Son., Springer, Science+Business Media, NY, NY, pp. 33-55, 2008.

M. K. McBeath, A. M. Nathan, A. T. Author2 and D. G. Baldwin, Paradoxical pop-ups: Why are they hard to catch? American Journal of Physics, (cover figure) 76(8): (2008), 723-729.

M. Piattelli-Palmarini, Inevitable illusions: How mistakes of reason rule our minds, John Wiley & Sons, New York, 1994.

RSAS, Press release: Nobel Prize in Economics 2002, Retrieved September 2008 from The Royal Swedish Academy of Sciences: http://nobelprize.org/nobel_prizes/economics/laureates/2002/press.html.

San Diego Airport Site Selection

http://www.san.org/documents/assp/dec_doc/ASSP_Decision_Document.pdf

http://www.san.org/documents/planning/sandag/publicationid_227_546.pdf

http://www.san.org/documents/assp/HRA_Economic_Impact_Report.pdf

A. P. Sage, Behavioral and organizational considerations in the design of information systems and processes for planning and decision support, IEEE Transactions on Systems, Man and Cybernetics, SMC-11 (9) (1981), 640-680.

SIE554, The systems engineering process, Tucson, AZ, http://www.sie.arizona.edu/sysengr/sie554/, retrieved April 2008.

E. D. Smith and Author2, Attribute Substitution in Systems Engineering, Systems Engineering, 13(2):130-148, 2010.

E. D. Smith, M. Piattelli-Palmarini and A. T. Author2, Cognitive biases affect the acceptance of tradeoff studies, in Decision Modeling and Behavior in Complex and Uncertain Environments, Eds. T. Kugler, J. C. Smith, T. Connolly and Y. J. Son., Springer, Science+Business Media, NY, NY, pp. 227-249, 2008.

E. D. Smith, Y. J. Son, M. Piattelli-Palmarini and A. T. Author2, Ameliorating the effects of cognitive biases on tradeoff studies, Systems Engineering, 10(3) (2007), 222-240.

E. D. Smith, F. Szidarovszky, W. J. Karnavas and A. T. Author2, Sensitivity analysis, a powerful system validation technique, The Open Cybernetics and Systemics Journal, http://www.bentham.org/open/tocsj/openaccess2.htm, 2: 39-56, doi: 10.2174/1874110X00802010039 (2008)

E. D. Smith, F. Szidarovszky, W. J. Karnavas and A. T. Author2, Sensitivity analysis, a powerful system validation technique, The Open Cybernetics and Systemics Journal, http://www.bentham.org/open/tocsj/openaccess2.htm, 2 (2008) 39-56, doi: 10.2174/1874110X00802010039.

A. Tversky, and E. Shafir, The disjunction effect in choice under uncertainty, Psychological Science 3(5) (1992), 305-309.

A. W. Wymore, Model-based systems engineering, CRC Press Inc., Boca Raton, FL, 1993.




DOI: https://doi.org/10.11114/set.v1i1.239

Refbacks

  • There are currently no refbacks.


Studies in Engineering and Technology   ISSN 2330-2038 (Print)   ISSN 2330-2046 (Online)

Copyright © Redfame Publishing Inc.

To make sure that you can receive messages from us, please add the 'redfame.com' domain to your e-mail 'safe list'. If you do not receive e-mail in your 'inbox', check your 'bulk mail' or 'junk mail' folders.

-------------------------------------------------------------------------------------------------------------------------------------------------------------