This study investigates the quality of automatic type-error repairs for student OCaml programs. We are trying to learn how the quality of our repairs compares to the quality of previous work. Such repairs will be used later in a more complete context to give valuable feedback to novice OCaml programmers.
OCaml reading experience (even if very minimal) is required.
We will not ask you for your name and will not record any identifying information, and you may stop at any time. Data obtained in this study will only be used to compare the effectiveness of our prototype tools.
Most people complete this study in less than 45 minutes (~30 minutes is standard), but there is no time limit.
If you have any questions about this study, you may contact the investigators:
First, you will be asked to take a Pre-Survey so we can collect information on your programming and OCaml experience.
Next, you will be shown a series of 10 stimuli. Each stimulus contains a buggy OCaml program and two repairs for that program. Along with the OCaml programs, you will see a short description of the programmer's intent for the buggy function in the stimulus, and the type-error message that the OCaml interpreter gives for the buggy program.
For each stimulus, you should use the original buggy program code and the suggested repairs to try and figure out which repair makes the original program correct (having in mind the intent of that program from the description box) and thus, is a better quality repair.
Repairs can be incomplete, meaning they have "holes" in them. Holes are denoted by [[..x..]]. x can be "fun", "var", etc. meaning that the hole should be completed with some function or variable respectively, or a type, suggesting the type the hole should have. [[...]] means the hole can be any possible expression.
After completing the Pre-Survey, a mock-up stimulus will be shown along with more detailed instructions.
This study uses both cookies and JavaScript to process your responses. Please be sure that JavaScript and cookies are enabled in your browser. To ensure full conformance with our code, please also disable any browser extensions (such as ad-blockers), which might attempt to rewrite webpages. We have experienced minor issues with line numbers in Firefox in the past; if you encounter such issues we recommend restarting the survey in Google Chrome.
By clicking "Begin Survey", you certify that you consent to participate in this study. You may retract consent at any time before completing the survey by closing this browser window.
Note: This study has been ruled IRB exempt by the University of Michigan IRB under case number HUM00158717.