When surveying college-age students, Amazon and iTunes e-gift cards are frequently offered as incentive for participation [1]. While I’ve frequently heard that students prefer iTunes, the administrative burden of sending iTunes gift cards is high. The iTunes store limits each account to $100 dollars in gift card purchases per month, so if your compensation needs go over $100, you have to schlep to the store, buy gift cards, and put them in the mail. Amazon, on the other hand, offers an effortless interface for sending gift cards and does not appear to have an unreasonable monetary restriction. So if you choose the ease of Amazon over the shiny iTunes brand, do you lose anything?
Recently, I ran a survey of first-year students at UNC that tested preferences toward compensation. The survey offered a dual-tier lottery compensation: Participants were entered to win an iPod touch or their choice of three gift certificates (See [2] for more on dual-tier incentives). The three gift card choices were iTunes, Amazon, or a popular on-campus cafe, in the amount of ten dollars. Response to the survey was good, by email-solicitations standards, at 31% (n~1200). Males were slightly underrepresented, as is commonly the case.
So, what gift cards did my students prefer? Clearly, the students preferred gift cards to iTunes (n=442) and Amazon (n=442) over the local cafe (n=131). And we don’t really need any significance tests to see that the difference between iTunes and Amazon is a wash (p=.8406).
When conducting surveys, we’re not always interested in a large homogeneous population. Sometimes we’re interested in sub-populations, such as certain genders, ages, or ethnicities. Breaking the perferences out by gender, visual inspection indicates that female students prefer iTunes over Amazon, while male students prefer Amazon over iTunes. Since neither population comes close to preferring the local cafe, I will focus on the difference between iTunes and Amazon for the rest of the analysis (i.e. drop the people who prefer the Local Cafe).
Of the students that selected Amazon or iTunes, we see that 53% of female students prefer iTunes, 47% Amazon. Of males, 58% prefer Amazon, 42% iTunes. The Chi-square test indicates a relationship between gender and preference (p=.001), and within-gender Chi-square goodness of fit tests indicate that while the female student preference difference is insignificant (.0922), the male preference towards Amazon is significant (p=.0064).
To test some higher order interactions, I employed a logistic regression model to test the effects of gender and a few other covariates. First, since much of my sample is from NC, I tested to see if NC residency might contribute towards a preference. In this model, gender remained significant, but NC residence was not significant (p=.828). Next, looked to see if GPA might be a factor in preference. Gender remained significant, and GPA’s p-value was low (p=.081), but not close to significance (directionality was higher GPA’s towards Amazon).
In the last two models, I looked at ethnicity and age. In the ethnicity model, gender is significant, and only one ethnicity is significant. Compared to other ethnicities, students who self-report as Asian demonstrate a preference towards Amazon (OR=.158, p=.000). With age, gender again remained significant, but 19 year old students (compared to 18 year old students) seem to prefer iTunes (OR 1.49, p=.004). Notably, a gender by age interaction was not significant, however.
To briefly review, it seems that among my population, the anecdotal preference towards iTunes is just that: anecdotal. This is good news for me, because it is much more complicated to process iTunes gift cards than Amazon gift cards. Some final notes: This is not really a proper experiment – such an experiment would use completely randomized solicitation. Also, the presence of the third category (Local Cafe) is potentially troubling if being a fan of a Local Cafe also correlates to, say, being an iTunes fan or an Amazon fan. Caveat emptor, blog post, not peer reviewed, etc.
1. I don’t have a citation for this, but I do monitor to a number of email lists that frequently offer research solicitations. YMMV.
2. See Li, Kaiwen (2006). Student Preference for Survey Incentive. UC Davis Student Affairs Research & Information Tech Report.
Finally, I promise that Amazon has not compensated me in any way, say, by sending me a bunch of gift certificates or a Nikon 12-24mm DX lens or anything like that.
Interesting stuff. But the real question – the one that’s very hard to answer – is if the incentives positively affected the survey. Did they increase the response rate? If so, did that increase have a negative impact on the quality of the data (i.e. did the respondents influenced by the incentives provide accurate responses or did they just fill out the survey quickly and inaccurately to get the incentive)? So there are not only issues related to non-response bias but also data quality. And what about the effect of pre-survey incentives vs. post-survey incentives? So many interesting questions! (Which is why I’m writing a dissertation focusing on survey response.)
I think there was some internal discussion at my research shop about trying to make available Amazon gift codes as survey incentives. I don’t know what happened to that idea but one thing you mention – the relative ease with which Amazon gift codes can be purchased and distributed – was a very important consideration for us.
Kevin, you’re absolutely right – this design doesn’t let us get at these questions. Bob Groves has good work in this area.
Personally, my gut is that I’m thrilled to get 30% off an email solicitation. That’s up near RDD and mail solicits, which email almost always under-performs. Though the question is – was it the incentive or the TDM tactics?
It was shocking to me how difficult it was (administratively) to do iTunes cards. I can’t imagine that survey researchers are a big part of the iTunes gift card budget, but I’m never going to go through that hassle again when compared with the ease of Amazon.
Fred, Thanks for reporting on this. With the informal research I do as an experience designer, I’ve always wondered what types of incentives encourage a greater response.
I have an anecdotal suspicion that there’s a relationship between the location (Amazon, iTunes, etc) and the amount of the incentive that’s manifested in the number of products that can be purchased.
Consider that $15 from iTunes can purchase 15 items, whereas $15 from Amazon might only provide one (or part of a) book. Considering this, I have to wonder if the incentive cost is fixed at $15, if the iTunes or Amazon card will be more effective. From anecdotal observation, I believe this might be true, but I want to test it at some point.
To your comment, I’ve faced all sorts of issues with purchasing iTunes cards. I use them for small amounts, but I try to avoid them and save the administrative trouble.