This project integrates a process tracing methodology (variously called an “information board,” “decision board,” and other terms) with Qualtrics. I couldn’t find an acceptable way to create one within Qualtrics itself, so I created this project.
Overview
I used a simplified version of the JavaScript for Customized Qualtrics Questions post I wrote a while back. In essence, a respondent begins with part 1 of the questionnaire on Qualtrics. The respondent is then passed automatically to part 2 on an off-Qualtrics server, where the process tracing presentations and measures take place. The respondent is then transferred back to Qualtrics to complete the questionnaire in part 3. Here, the off-Qualtrics process tracing data are sent to the Qualtrics “part 3” study, where they are available for download from Qualtrics.
The basic functionality of the process-tracing portion of the method involves a “main” page that gives the basics of the stimulus object. The “main” page then provides (in the form of a menu) a variety of optional information pages that give further details about the stimulus object. The participant may view any/all/none of information pages, and may view any of them more than once. After viewing each optional information page, the participant is returned to the “main” page. A simple example can be found here. When the participant chooses the “I’m finished evaluating the stimulus object” menu item, control is returned to Qualtrics, along with a record of the information that was considered by the participant. The record shows which information pages the participant chose to view, the order in which the pages were viewed, and how long each page was viewed (in milliseconds).
Warning
Using this project is not for the faint of heart. This project requires some basic familiarity with HTML, Javascript, and hosting your own web pages. In this description, I will assume the reader is familiar with the basics of my “JavaScript for Customized Qualtrics Questions” post. Furthermore, other things need my attention right now, so the description provided here is minimal.
The Files
Six files are available here.
• “start.htm” initializes the process. This file should be directly called by the ‘Part 1’ Qualtrics study. In addition to obvious substantive edits, this file should be edited in the following ways:
next_screen_name = "main.htm";
Change the filename to point to the “main” page to which the participant will return after viewing each information page.
• “main.htm” is the main information page that the participant will be returned to after viewing each information page. It contains the menu from which the participant will choose the various information pages. In addition to obvious substantive edits, this file should be edited in the following ways:
location.replace('end.htm');
Change the filename to point to the “end” page, which is described infra.
location.replace('page1.htm');
Change the filename to point to the “page 1” page.
The same for page 2 and page 3. More pages can be added by simply copying that code within “main.htm” and adding an appropriate “pageX.htm” file.
• “page1.htm” is information page #1. In addition to obvious substantive edits, this file should be edited in the following ways:
var info_board_record = info_board_record + '###page1@@@' + elapsed_time;
Change the “page1” to be the indicator in the process data that will signifiy this page was viewed.
location.replace('main.htm');
Change the filename to point to the “main” page.
• page2.htm, page3.htm, etc. will need to be edited just as page1.htm
• “end.htm” does the wrap-up for the process. This file does not display any information to the participant, so no substantive edits are needed. This file should be edited in the following way:
var Qualtrics_SID = "SV_a36OXiqiA5SMdlr";
Change to point to the Qualtrics SID of the “Part 3” Qualtrics q’re.
• In the “Part 1” Qualtrics study, the “Survey Options” should be edited:
Redirect to a full URL >>
[full URL]/start.htm?Part_1_ResponseID=${e://Field/ResponseID}
• In the “Part 3” Qualtrics study, the “Survey Flow” should be edited:
Set Embedded Data >> Type variable name right into what appears to be a dropdown menu.
Part_1_ResponseID >> Value will be set from Panel or URL
This imports the Qualtrics ResponseID from Part 1
info_board_record >> Value will be set from Panel or URL
This imports the “info_board_record,” which contains all the data created by the process tracing method.
The Data Structure
The variable “info_board_record” found in Qualtrics “Part 3” takes the form …
###<page_identifier>@@@<elapsed_time>
… and this repeats for each page viewed by the participant.
For example …
###main@@@8000###page1@@@1500###main@@@1000
… signifies that the participant viewed the main page for 8,000 milliseconds (i.e., 8 seconds), chose to view page 1 for 1.5 seconds, and then viewed the main page again for 1 second before choosing “I’m finished.”
When analyzing the information board record, bear in mind a few issues:
(1) For each participant, the main page will be the first page viewed. For most participants, the main page will be the last page viewed (because each time an information page is visited the participant will be returned to the main page). If a participant only visits the main page and selects none of the optional information pages; such a participant will have only the main page entry in the record.
(2) Visiting the information pages is optional, can be done in any order, and can be repeated. The information board record will reflect all these visits, in order. Therefore, other than the “###<page_identifier>@@@ <elapsed_time>” structure, this variable is highly unstructured. Brace yourself for a challenging analysis.
I hope these files are helpful to you in your adventures, but I cannot warranty them in any way, and any risk in using these files is your own. Obviously, you should carefully pretest any web-based questionnaires before you field them. You can freely use the files I have provided here; they are licensed under a Creative Commons License (Attribution 3.0).
– Eric DeRosia