You are invited to participate in a research study about the quality of various software debugging aids and the helpfulness of suggested code repairs. We are trying to learn how the quality of automatic repairs compares to the quality of human generated repairs. The only requirements for participating in this study are prior programming experience, Python reading experience (even if limited), and being at least 18 years old. We recommend that you take the survey in Google Chrome.
If you agree to be part of the research study, you will be asked to first answer a few survey questions to gauge your Python programming experience and your general programming experience. Then you will be presented with a series of Python 3 programs and associated artifacts and annotations (e.g., suggested repairs, program inputs, comments, etc.). You will be asked for both short response and scale-based questions about each program and associated tasks. For instance, you may be asked to describe the cause of a programming bug or assess the helpfulness of a given repair. Finally, there will be a post-survey where you will be asked for any additional comments and to explain your reasoning for some of your answers.
This study may help researchers evaluate existing software repair and debugging tools more effectively. It will also help us learn how to build more effective tools in the future, including helping us design tools that better support future programmers, both novices and experts alike.
This study will take time to complete (~30 minutes), and some of the code debugging questions may be challenging.
As compensation for participating in this study, you can elect to be entered into a drawing for one of two $50 Amazon gift cards. Contact information will be collected from a second survey linked at the end of this survey. Note: The rewards of this study are contingent upon successful completion of the entire study. We do not consider responses that are random or largely left blank to be a successful completion.
Participating in this study is completely voluntary. Even if you decide to participate now, you may change your mind and stop at any time. You may choose not to answer any survey question for any reason.
We will protect the confidentiality of your research records by never storing any identifiable information (e.g. IP addresses) and not collecting your contact information (should you elect to enter the drawing) on the same survey as your other responses. Information collected in this project may be shared with other researchers, but we will not share any information that could identify you.
If you have any questions about this study, please contact the investigators:
This study uses both cookies and JavaScript to process your responses. Please be sure that JavaScript and cookies are enabled in your browser. To ensure full conformance with our code, please also disable any browser extensions (such as ad-blockers), which might attempt to rewrite webpages. We recommend restarting the survey in Google Chrome.
By clicking "Begin Survey", you certify that you consent to participate in this study. You may retract consent at any time before completing the survey by closing this browser window.
Note: The University of Michigan Institutional Review Board Health Sciences and Behavioral Sciences has determined that this study is exempt from IRB oversight (HUM00214155). Principal Investigator: Madeline Endres, PhD Student, University of Michigan