silverback turns the files you download from the Gorilla Experiment Builder into three tidy tibbles: one for task responses, one for questionnaire answers and one for participants. Eye tracking and mouse tracking recordings can be read too. A typical session looks like this:
library(silverback)
g <- read_gorilla("data_exp_123456-v3.zip")
gorilla_checks(g)
trials <- gorilla_trials(g)
answers <- gorilla_wide(g)
people <- gorilla_participants(g)
write_gorilla(g, "tidy")The rest of this page goes through each step with a small invented experiment that comes with the package.
On the Data tab of your experiment, generate the data for the versions you need and download it. Gorilla offers two layouts, one file per node of the experiment tree or combined files with all tasks in one file and all questionnaires in another. Either works, as CSV or as Excel files. Questionnaires can be downloaded in long or short format.
Excel files give the same tables as CSV files. Some Excel downloads
store numbers as numbers, however, so a code such as 007
may already have lost its leading zeros in the file. If your
spreadsheets or answers contain such codes, download CSV files.
Unzip the download or leave it as it is. read_gorilla()
accepts the ZIP file, the folder or single files. Keep one layout and
one format per folder: otherwise the same rows would be read twice, and
read_gorilla() warns about it.
The example experiment has a lexical decision task made with Task Builder 2, a rhyme judgement task made with Task Builder 1, a consent form made with Questionnaire Builder 1 and a background questionnaire made with Questionnaire Builder 2. One branch of the experiment was never reached, so one of the files is empty.
g <- read_gorilla(gorilla_example())
#> Skipped 1 file without data (nodes no participant reached): data_exp_123456-v3_questionnaire-zz1e.csv
g
#> <gorilla_data> experiment 123456, version 3
#> 6 participants
#> complete: 5, live: 1
#> Tasks:
#> Lexical decision (Task Builder 2): 6 participants
#> Rhyme judgement (Task Builder 1): 5 participants
#> Questionnaires:
#> Background: 7 items, 6 participants
#> Consent: 2 items, 6 participants
#> 1 empty file skippedThe printout lists the tasks and questionnaires that were found and how many participants did each. Empty files are skipped with a message.
read_gorilla() returns a list of four tables.
g$files has one row per file and says which builder each file comes from. g$tasks
keeps every row of every task file, g$questionnaires holds
one row per questionnaire answer and g$participants one row
per participant. You will rarely need the first two directly.
g$files
#> # A tibble: 5 × 8
#> file experiment version node kind builder task rows
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <int>
#> 1 data_exp_123456-v3_questio… 123456 3 ques… ques… questi… Back… 66
#> 2 data_exp_123456-v3_questio… 123456 3 ques… ques… questi… Cons… 36
#> 3 data_exp_123456-v3_questio… 123456 3 ques… ques… questi… <NA> 0
#> 4 data_exp_123456-v3_task-ld… 123456 3 task… task task_b… Lexi… 104
#> 5 data_exp_123456-v3_task-rh… 123456 3 task… task task_b… Rhym… 90Column names are converted to snake case, so
Participant Private ID becomes
participant_private_id. Columns that come from your
spreadsheets start with spreadsheet_, and values saved to
the store start with store_. Task
Builder 1 exports give spreadsheet columns no prefix at all, and
silverback adds it so that both builders look the same.
Before the analysis it is worth looking at what came in.
summary() gives one line per task and questionnaire:
summary(g)
#> 6 participants (complete: 5, live: 1)
#>
#> Tasks
#> task_name builder participants responses_min responses_median
#> Lexical decision Task Builder 2 6 7 16
#> Rhyme judgement Task Builder 1 5 8 8
#> responses_max correct rt_median fast slow minutes
#> 16 0.943 673.1 0 0 0.4
#> 8 0.950 910.1 0 0 0.4
#>
#> Questionnaires
#> task_name participants items unanswered
#> Background 6 7 0
#> Consent 6 2 0For tasks, the line shows how many responses each participant gave, the share Gorilla marked as correct, the median reaction time and how long people spent on the task. That share only means something if scoring was set up in Gorilla. For questionnaires, it shows how many items were left unanswered.
gorilla_checks() goes further and lists findings about
single participants:
gorilla_checks(g)
#> # A tibble: 2 × 4
#> check participant_private_id task_name detail
#> <chr> <chr> <chr> <chr>
#> 1 status 9100222 <NA> status is live
#> 2 missing_task 9100222 Rhyme judgement not done, although 2 of 3…In the example one participant stopped during the lexical decision task. The checks report the status and the missing rhyme judgement task. On real data they also find people who took part twice under the same public ID, who pressed the same key almost every time, who often answered faster than 200 ms, or who used a different type of device from most participants. Randomiser groups of very different sizes are reported too. You can change the thresholds with the function’s arguments. Nothing is removed, so you decide what to exclude.
Task files hold far more rows than responses. Instructions, fixation
crosses, continue buttons and audio events all get rows of their own.
gorilla_trials() keeps only the rows with a response:
ld <- gorilla_trials(g, task = "Lexical decision")
ld
#> # A tibble: 87 × 19
#> participant_private_id experiment_version tree_node_key repeat_key task_name
#> <chr> <int> <chr> <chr> <chr>
#> 1 9100037 3 task-ld2k <NA> Lexical de…
#> 2 9100037 3 task-ld2k <NA> Lexical de…
#> 3 9100037 3 task-ld2k <NA> Lexical de…
#> 4 9100037 3 task-ld2k <NA> Lexical de…
#> 5 9100037 3 task-ld2k <NA> Lexical de…
#> 6 9100037 3 task-ld2k <NA> Lexical de…
#> # ℹ 81 more rows
#> # ℹ 14 more variables: task_version <int>, spreadsheet <chr>, trial <dbl>,
#> # display <chr>, screen <chr>, component <chr>, object <chr>, response <chr>,
#> # rt <dbl>, correct <lgl>, time <dttm>, spreadsheet_word <chr>,
#> # spreadsheet_lexicality <chr>, spreadsheet_answer <chr>The first columns are the same for every task, whichever builder it was made with. The table below lists them.
| Column | Content |
|---|---|
participant_private_id |
Gorilla’s ID for the participant |
experiment_version, tree_node_key,
repeat_key |
where in the experiment the response was given |
task_name, task_version |
the task and its version |
spreadsheet, trial |
the spreadsheet and trial number |
display, screen |
the display and screen (Task Builder 1 has no displays) |
component, object |
what recorded the response (zone type and zone name in Task Builder 1) |
response, rt, correct |
the response, the reaction time in ms and whether it was correct |
time |
when the response was given, in UTC |
Next come the timing columns that Gorilla
recorded for the responses, such as onset_time and
response_duration in Task Builder 2 or
reaction_onset in Task Builder 1. Columns without values
are left out. The invented example has no timing data, so these columns
are missing below. The spreadsheet and store columns of the task follow,
so the conditions are ready to use:
ld |>
filter(correct) |>
summarise(rt = median(rt), .by = spreadsheet_lexicality)
#> # A tibble: 2 × 2
#> spreadsheet_lexicality rt
#> <chr> <dbl>
#> 1 nonword 723.
#> 2 word 650.Some components can record a response without a value: a Click
Response whose objects have no response set up in Gorilla records only
which object was clicked. gorilla_trials() then gives a
message, and the answer is in object.
gorilla_checks() uses the object for these components when
it looks for identical responses.
Leave out task to get the responses of all tasks in one
table. If you need the rows around each response, such as the moment an
audio file started, use all_rows = TRUE. Every column of
the export, including the metadata left out here, remains in
g$tasks.
gorilla_wide() puts each questionnaire item in its own
column:
answers <- gorilla_wide(g)
answers
#> # A tibble: 6 × 14
#> participant_private_id age gender gender_quantised languages_english
#> <chr> <int> <chr> <dbl> <int>
#> 1 9100037 23 Female 1 1
#> 2 9100074 34 Female 1 1
#> 3 9100111 22 Female 1 1
#> 4 9100148 26 Male 2 1
#> 5 9100185 29 Female 1 0
#> 6 9100222 34 Male 2 0
#> # ℹ 9 more variables: languages_german <int>, languages_italian <int>,
#> # languages_portuguese <int>, handedness <chr>, handedness_quantised <dbl>,
#> # consent_data <chr>, consent_data_quantised <dbl>, consent_withdraw <chr>,
#> # consent_withdraw_quantised <dbl>Dropdowns, multiple choice questions and Likert scales also have a
quantised value, the position of the chosen
option, which goes into a second column ending in
_quantised. Checkboxes give one column per option, such as
languages_german. Questionnaires from both builders and in
both download formats end up with the same column names.
Questions are named after their object in the Questionnaire Builder.
If several objects in one questionnaire share a name, for example
because they kept Gorilla’s default name Rating Scale, they
are still different questions, and each gets its object ID added:
rating_scale_object_29, rating_scale_object_33
and so on.
Sometimes a participant answers an item twice. This happens after a
repeated node, or when someone goes back a page and submits it again.
The first answer stays in the item’s column and later answers go into
columns ending in __2, __3 and so on.
gorilla_participants() collects what Gorilla records
about each person:
people <- gorilla_participants(g)
people
#> # A tibble: 6 × 18
#> participant_private_id participant_public_id experiment_id
#> <chr> <chr> <int>
#> 1 9100037 c805ceab3f4eb9172ecd0b95 123456
#> 2 9100074 1d1c4b74f2b229c3711cbcc8 123456
#> 3 9100111 b48c18f6af1d44ac3fe078e2 123456
#> 4 9100148 cce8638538eef07b5c3ce7a5 123456
#> 5 9100185 7f50b2600354023e074434c0 123456
#> 6 9100222 9b2d42c0a5c131039b4134ea 123456
#> # ℹ 15 more variables: participant_device_type <chr>, participant_device <chr>,
#> # participant_os <chr>, participant_browser <chr>,
#> # participant_monitor_size <chr>, participant_viewport_size <chr>,
#> # local_timezone <int>, experiment_version <int>, participant_status <chr>,
#> # start <dttm>, end <dttm>, tasks <int>, randomiser_ab12 <chr>,
#> # store_score <int>, minutes <dbl>Each row has the participant’s status, device, operating system and
browser. It also has the times of the first and last event with the
minutes in between, the group that each randomiser or branch node assigned, and the last value
of each store field. Columns that are empty for everybody are dropped.
status = "complete" keeps only participants who
finished.
The three tables share participant_private_id, so they
can be joined with left_join(). Here the background
questionnaire and the randomiser group are added to each response:
ld |>
left_join(select(answers, participant_private_id, age, gender), by = "participant_private_id") |>
left_join(select(people, participant_private_id, randomiser_ab12), by = "participant_private_id") |>
select(participant_private_id, spreadsheet_word, rt, age, gender, randomiser_ab12)
#> # A tibble: 87 × 6
#> participant_private_id spreadsheet_word rt age gender randomiser_ab12
#> <chr> <chr> <dbl> <int> <chr> <chr>
#> 1 9100037 mukket 628. 23 Female List A
#> 2 9100037 venisk 733. 23 Female List A
#> 3 9100037 mirror 840. 23 Female List A
#> 4 9100037 drosk 1110. 23 Female List A
#> 5 9100037 basket 470. 23 Female List A
#> 6 9100037 pencil 650. 23 Female List A
#> # ℹ 81 more rowsAll three can go straight into dplyr or ggplot2.
The Eye Tracking (Webgazer) and Mouse Tracking components of Task
Builder 2 do not put their recordings in the main export. Gorilla saves
one file per trial instead, named like
123456-3-9100037-task-ab12-50000037-collection-10-2.csv.
read_eyetracking() and read_mousetracking()
read them from the same folder or ZIP file:
eye <- read_eyetracking("data", x = g)
eye$gaze # one row per gaze sample
eye$zones # the zones of each screen
eye$calibration # one row per calibration point and attempt
mouse <- read_mousetracking("data", x = g)
mouse$mouse # one row per mouse samplePositions are on a scale from 0 to 1 across the task area, with the
origin at the bottom left. Each sample is assigned to the zone it falls
in, and where zones overlap the smallest one wins;
zones = "Picture" limits the assignment to zones with that
name. The recordings do not say which task they belong to, so pass the
main export as x to add task_name and the
screen name. The samples can then be joined to
gorilla_trials() by participant_private_id,
tree_node_key, display, trial and
screen.
If you recruited through a platform such as Prolific, the public IDs are the platform’s IDs and can identify
people. blind_ids() replaces them, the external session IDs
and the completion codes with codes such as P001,
consistently across all tables:
blind_ids(g) |>
gorilla_participants() |>
select(participant_private_id, participant_public_id)
#> # A tibble: 6 × 2
#> participant_private_id participant_public_id
#> <chr> <chr>
#> 1 9100037 P001
#> 2 9100074 P002
#> 3 9100111 P003
#> 4 9100148 P004
#> 5 9100185 P005
#> 6 9100222 P006Gorilla’s private IDs stay, because they mean nothing outside Gorilla. Free-text answers can still hold personal information such as email addresses, so check the questionnaires before you share them.
To continue the analysis in other software, or to share the data,
write_gorilla() writes the tables to a folder of CSV files:
out <- file.path(tempdir(), "tidy")
files <- write_gorilla(blind_ids(g), out)
basename(files)
#> [1] "participants.csv" "trials.csv"
#> [3] "questionnaires.csv" "questionnaires_long.csv"The folder has participants.csv,
trials.csv, questionnaires.csv with one row
per participant and questionnaires_long.csv with one row
per answer. Missing values are empty cells and times are in UTC. With
by_task = TRUE, each task gets its own trials file, which
keeps only the spreadsheet columns of that task. Existing files are only
replaced with overwrite = TRUE.
silverback has been tested on real exports from many different
experiments, but Gorilla changes its exports from time to time, and a
new format or an unusual experiment may give results that are not what
you expect. Before you analyse or publish the data, compare the tables
with your raw files, for example the number of participants and of
responses in each task, and look at the output of summary()
and gorilla_checks().
If a file is read wrongly or not at all, please open an issue, if possible with a small example file. Remove personal data such as Prolific IDs or email addresses before you share it.