Answer Tests
Description
Answer tests are how swirl determines whether a user has answered
a question correctly or not. Each question has one or more answer
tests associated with it, all of which must be satisfied in order for
a user's response to be considered correct. As the instructor, you
can specify any combination of our predefined answer tests or create your
own custom answer tests to suit your specific needs. This document will
explain your options.
Details
For each question that you author as part of a swirl lesson, you
must specify exactly one correct answer. This is separate and
distinct from the answer tests. This does not have to be
the only correct answer, but it must answer the question correctly.
If a user skips your question, this is the answer that will be
entered on his or her behalf.
If you're using the swirlify
authoring tool, the correct answer will
be automatically translated into the appropriate answer test for most
question types. Questions that require the user to enter a valid
command at the R prompt (which we'll refer to as command questions)
are the only exception. Since there are often many possible ways to
answer a command question, you must determine how you'd like swirl to
assess the correctness of a user's response. This is where answer
tests come in.
You can specify any number of answer tests. If you use more than one, you
must separate them with semicolons. If you do not specify any answer tests
for a command question, then the default test will be used. The default
test is omnitest(correctExpr='<correct_answer_here>'), which will
simply check that the user's expression matches the expression that you
provided as a correct answer.
In many cases, the default answer test will provide sufficient vetting of
a user's response to a command question. While it is somewhat restrictive
in the sense that it requires an exact match of expressions (ignoring
whitespace), it is liberating to the course author for two reasons.
-
It allows for fast prototyping of content. As you're developing
content, you may find that determining how to test for correctness
distracts you from the message you're trying to communicate.
-
You don't have to worry about what happens if the user enters
an incorrect response, but is allowed to proceed because of an oversight
in the answer tests. Since swirl sessions are continuous, accepting
an incorrect answer early in a lesson can cause problems later on. By
using the default answer test, you eliminate this burden by requiring an
exact match of expressions and hence not allowing the user to advance
until you are certain they've entered the correct response.
It's important to keep in mind that as your content matures, you can always
go back and make your answer testing strategy more elaborate. The main
benefit of using tests other than the default is that the user will not be
required to enter an expression exactly the way you've specified it. He or
she will have more freedom in terms of how they respond to a question, as
long as they satisfy the conditions that you see as being most important.
Predefined Answer Tests
Each of the predefined answer tests listed below has
its own help file, where you'll find more detailed explanations and
examples.
any_of_exprs: Test that the user's expression matches any of several possible expressions.
calculates_same_value: Test that the user's expression evaluates to a certain value.
expr_creates_var: Test that a new variable has been created.
expr_identical_to: Test that the user has entered a particular expression.
expr_is_a: Test that the expression itself is of a specific class.
expr_uses_func: Test that a particular function has been used.
func_of_newvar_equals: Test the result of a computation such as mean(newVar) applied to a specific (user-named) variable created in a previous question.
omnitest: Test for a correct expression, a correct value, or both.
val_has_length: Test that the value of the expression has a particular length.
val_matches: Test that the user's expression matches a regular expression (regex).
var_is_a: Test that the value of the expression is of a specific class.
Custom Answer Tests
Occasionally, you may want to test something that is outside the scope of
our predefined answer tests. Fortunately, this is very easy to do. If you
are using the swirlify authoring tool, then a file called
customTests.R (case-sensitive) is automatically created in the lesson
directory. If it's not there already, you can create the file manually.
In this file, you can write your own answer tests. These answer tests are
then available to you just the same as any of the standard tests. However,
the scope of a custom answer test is limited to the lesson within which
you've defined it.
Each custom answer test is simply an R function that follows a few
basic rules:
-
Give the function a distinct name that will help you remember what
is does (e.g. creates_matrix_with_n_rows).
-
The first line of the function body is
e <- get("e", parent.frame()), which gives you access to the
environment e. Any important information, such as the expression
typed by the user, will be available to you through e.
-
Access the expression entered by the user with e$expr and
the value of the expression with e$val.
Note that e$expr comes in
the form of an unevaluated R expression.
-
The function returns TRUE if the test is passed and
FALSE otherwise. You should be careful that no other
value could be returned (e.g. NA, NULL, etc.)
See Also
Other AnswerTests:
any_of_exprs(),
calculates_same_value(),
expr_creates_var(),
expr_identical_to(),
expr_is_a(),
expr_uses_func(),
func_of_newvar_equals(),
omnitest(),
val_has_length(),
val_matches(),
var_is_a()