Over the past month I have been developing my first memory experiment to collect data online using Qualtrics.
I was hoping to learn how to write my own code and collect data online without having to use/pay for another set of software. However, I got some advice from a friend about how long it would take me – realistically – to get an experiment up and running that way and decided that I just didn’t have the time available to dedicate to that right now. My grant funding runs out in a few months and I need to have collected this data by then!
I currently use SurveyMonkey to collect data in questionnaire-type format, but I needed something that can present one stimulus at a time, for a set time, and record response times online.
So, I decided to try out Qualtrics research suite and I am really liking it. This first experiment has taken a while to perfect, but that is always the way. I have found the same with lab-based programmes such as EPrime – the first experiment always takes the longest to create; after that you can work from that experiment as a base and just adapt each of the pages/screens/events/stimuli to work in your new experiment.
Good things about Qualtrics:
- It has a user-friendly interface and it is easy and quick to add different kinds of questions (e.g., multiple choice, text descriptions)…
- Or to change a question from one type to another if you change your mind about something.
- The branching facility is also pretty simple to use – so if you want participants to see one set of blocks if they respond to a question in a particular way, or if you want participants to be randomly assigned to 1 of 3 conditions, for example….
- However, if you want to do both these things at once it got a little more tricky, but really it only took me about 30 minutes to figure it out!
- The ‘survey flow’ view is a really good feature to help with this, especially if you have participants in some conditions actually viewing some of the same blocks (e.g., if the encoding phases are different across conditions but the test phases are the same).
- In ‘survey flow’ you can see a list of each of the blocks that make up each of the conditions, even if some blocks are duplicated across conditions.
There are a few small things that annoy be about Qualtrics though:
- I can’t seem to find any decent instructions on the Qualtrics site for some of the things I want to do. I may not be looking hard enough, but the instructions I have found seem to be explaining things at a very superficial level.
- There are some niggly little things – like when I’ve made a piece of text bold, and it looks bold in the editing screen, but then when previewing the experiment it is not bold – arrrghhh, so annoying!
- There are limits to what you can change, for example, for some parts of my experiment I would like the “next” button to be in centre of the screen, or I would like the experiment to move on to show the next screen without there being a “next” button visible (you can set it to advance after a certain time limit, but not remove the “next” button) – I guess some of the things I want to do are beyond what you can change without knowing code – a reason to learn to programme myself! Update (14/06/13) – I have now found the code and instructions regarding removing the “next” button, they could have made this much easier, but at least it can do it 🙂
- You can have a list (e.g., of stimuli) that are going to appear in a particular ‘field’ on a page (and the page will be repeated until all the stimuli have been shown) but you can’t copy and paste the list in from another programme (e.g., Excel) you have to physically type out each of the words.
- And finally, Qualtrics is expensive! – at the moment I’m developing this first experiment using a free student account, but with that, although you get unlimited surveys – which sounds excellent, you only get 250 respondents – ACROSS surveys, so really the fact that you get unlimited surveys is a big con.
But, on the whole, I am happy with the ease of learning and using Qualtrics, and I am excited about getting my first experiment up and collecting data soon… Just have to get the ethics application approved first 🙂