Online Research 2000-2004 at FESSEL-GfK
aus: GfK mediaReview 1/2004
Quantitative Research via the Web
Quantitative market research using the Internet as a carrier medium is
going into its fifth year at FESSEL-GfK. Not counting a number of pilot
questionnaires and projects offered but not ordered, around 200
research projects were conducted since June 2000. The software
used for "Computer Assisted Web Interviews (CAWI)" has been the
"Internet Research Manager (IRM)", developed as proprietary software
by Domestic Data/Vienna together with FESSEL-GfK online research.
The IRM is a menu-based multi-language Internet research suite which
can be operated without knowledge of HTML by any junior research
assistant – that is without help by IT personnel. Under the licence
negotiated with Domestic Data, the programme can be – and is being –
used by any number of staff/computers at FESSEL-GfK and its
subsidiaries in Central and Eastern Europe. At present, IRM works in
14 languages – not only to translate the wording of questions but also to
work entirely in one's native language with the desk interface. The IRM
software can handle all forms of questions possible on the Web including
multi-media stimuli (image, sound, video etc.). Amongst other features,
the IRM includes a recoding module for processing open questions. It
enables customized online reporting and has a number of possibilities for
recording paradata – such as average question and interview duration,
break-off statistics etc. Field reports give a precise picture of the
functioning of a study on an hourly basis. Data can be processed by
Quantum, SPSS or directly online and exported into MS Office
programmes.
Quantitative online research is being conducted by FESSEL-GfK along
four distinct methodological lines:
Pool surveys:
Here the sample is drawn from an address database recruited offline
and, therefore, representative of the Austrian Internet user population
(sometimes, but incorrectly, called "access panel"). In the case of
FESSEL-GfK the address pool comprises about 25 000 persons of
whom ca 15 000 are active at the moment. As an incentive, 8-20 points
are being allotted per completed questionnaire.
For 100 points respondents receive a shopping voucher worth 15 Euros .
Pop-up Surveys:
Every nth visitor of a Website is confronted with a pop-up window inviting
him/her to participate in a professional market research project. Thus
Website operators can find out about the demographic structure of their
audience and their attitudes towards the site. Sometimes, pop-ups are
also used for other subject matter. By a special method, respondents are
informed that the pop-up window is not a commercial advertising. Double
entry is prevented by a cookie.
Surveys based on customer-provided addresses:
Employee studies, customer surveys and membership projects are being
conducted on the basis of address material held or provided by the
client. A practical problem that may arise here is poor quality of
addresses. If the client does not trust the institute and organizes the
mailings himself, difficulties in timely reminders etc. may occur. In
borderline cases anonymous market research tends to become CRM.
Link Surveys:
In special cases, research may be based on persons who are asked to
follow a link on the Web or displayed on a PC - e.g. in a hotel lobby.
Such surveys can record demographics and attitudes of visitors and
guests of museums, fairs etc.
FESSEL-GfK has not created a special task force for online research but
follows the policy to let each department decide for themselves if they
want to use CAWI or not. The author is acting as a consultant.
Using the built-in performance statistics of IRM, it is possible to describe
some of the major characteristics of the research work carried out by the
various departments at FESSEL-GfK. One observation was quite
surprising: the studies conducted over the last few years varied in their
respective parameters much more than we opurselves would have
expected of "typical" Internet research projects:
Few and many interviews
The median ("typical") online project was based on n = 211 complete
interviews – the result of a median gross sample of N = 505 people. The
median response rate was 49 percent (min 12%, max 100%). Sample
sizes varied considerably: from n = 10 complete interviews (in a Pop-up
Test on a very low-traffic Website) to n = 6 091 complete interviews (in a
transnational employee study for an Austrian bank).
Short and long interviews
The median duration of online interviews was 9.4 minutes - with a
minimum of 1.3 minutes ("how much do you spend on your mobile
phone?") and a maximum of 46.3 minutes ("use of various cosmetics").
Few and many questions
Considering that a respondent may not have to answer all questions due
to filters, the mean number of questions per project was 34, with a
minimum of 2 ("mobile phone" again) and a maximum of 194 questions
("cosmetics" again). The most frequent number of questions per project
(mode) was 20.
Short and long field periods
Although there were some studies carried out within 1-2 days, the most
frequent field period was 4 days. This is explained by the fact that
relatively many poster pre-tests were conducted. In such studies with
short field time it is our policy to include the weekend in order to reach
also "light surfers" and/or people who do most of their surfing at home.
Median duration of field work was 14 days with a maximum of 102 days
(in a business-to-business study).
The year 2003
Like everybody else we too suffered from the effects of economic
slowdown and the backlash following the Internet hype of the years
preceding 2002. Nevertheless, FESSEL-GfK carried out 60 quantitative
online research projects last year. Besides the employee study and the
cosmetics survey quoted above, major projects included the Austrian
Internet Radar (AIR - 2 waves at n = 5 000) and the study Austrian Youth
Online (3 waves at n = 1 000). The AIR is the most important source of
data on Internet usage in Austria. By means of the "logo recall method"
120 leading Internet sites are being polled online and their audience size
and composition determined. The method and main results of this
survey, carried out for ORF, the Austrian Public Radio and TV Network,
can be studied at http://mediaresearch.orf.at/internet.htm. Besides the
logfile statistics, published at http://www.oewa.at, the AIR is the only
reliable basis for online advertizing in Austria, as long as there is no
Internet audience size measurement panel.
Another interesting and successful online project is the so-called Poster
Online Pretest ("POP"). This series of studies on billboards is carried out
for a major Austrian outdoor advertizing firm. At regular intervals up to a
dozen posters are judged by ca. 200 respondents. Details of the method
including the questionnaire can be studied at our client's Website:
http://www.aussenwerbung.at
Under the title "Online Express" colleagues in the consumer goods
section and in other departments have started to sell a "genuine" Internet
research product: a general advertising copy test online for pretesting
concepts, ads, logos, wrappings, TV or radio spots, etc. on the basis of
n = 200 and at an attractive price of Euro 1 000.- per tested item. It
seems however, that the Austrian market is still rather reluctant to make
use of such opportunities.
Finally, it should be mentioned that several Pop up-Tests were
conducted in order to show the efficiency of Websites operated by
commercial firms or state agencies. Return of investment is best proved
by such surveys.
Qualitative Research about and via the Web
The main instrument to check usability and efficiency of a Website is the
Qualitative Webtest. On the institute premises a special room has been
reserved for in-depth interviews with respondents who sit in front of a
desktop PC and occupy themselves with a specific Website.
In this way, e.g. one of the main offers of Austrian e-government was
investigated. On the basis of n = 30, a thorough analysis of practical use
of this guide through government agencies could be made. A Pop up-
Test to quantify results is to follow.
While studying Websites in group discussions or by personal "in front of
the screen" interviews has become commonplace with FESSEL-GfK, the
Online Focus Group has not yet entered practical use. In spite of the fact
that a customized programme for easy use has been developed together
with the Regensburg based software firm SPiN AG
( http://www.spingroups.de ), the Austrian market is said to be still
reluctant to accept Web-based discussion groups. Maybe the colleagues
in Prague and Ljubljana, who have also tested our software, will be
leading the way. At any rate, further virtual focus group experiments will
be conducted in 2004 together with the psychology department at
FESSEL-GfK.
Into the future
From the statistical evaluation of around 60 recent and 200 overall online
studies carried out by FESSEL-GfK and its subsidiaries, a number of
conclusions can be made:
1. Online research (O.R.) in order to be introduced on a broad level
needs strong back-up by management. Workshops to train staff and
inform clients are a must. At least one person per company must work
fulltime in this field.
2. O.R. software like the IRM that can be understood and handled by
practically everybody in the institute is a good basis for take-up of the
method. In many cases the media research department will take the lead
and spread the knowledge.
3. O.R. needs imaginative leadership. The younger generation of
researchers who are completely familiar with the possibilities of the
digital age are not yet in management positions. But they can contribute
new and fresh ideas – provided such ideas are welcome.
4. Commercial calculation of cost has proved that O.R. - if conducted
professionally – today is not a lot cheaper than conventional research.
At least in the initial years the cost of software, of some new hardware
and of necessary expertise and training must be carried as an additional
load on prices. In the long run O.R. will however prove its cost efficiency
vis á vis face-to-face and phone interviews - not counting its many
methodological advantages.
5. O.R. has the capacity both to supplement and replace conventional
surveys. Take a project to test 52 packages of hair cosmetics on a
nationwide quantitative basis – what can be cheaper and faster than
O.R.? In many other applications – e.g. in social research with an
emphasis on the total population – O.R. is not yet practical. In some
instances – e.g. with respondent e-mail addresses provided by the client,
O.R. cannot be beaten.
6. The basic philosophy of O.R. according to theory is "quick and
straight": short and easy-to-answer questionnaires administered within a
few days with results delivered online for an attractive price. In practice,
such products have been the exception. As long as there is no forced
marketing for "typical" online services, conventional studies transferred to
the Web seem to prevail. There is no objection against methodological
"change-over" from conventional to online, but it would be desirable to
design and sell more "true" online products too.
7. The decision whether to install a specific individual or working group
responsible for O.R. - or leave it entirely up to the respective
departments to decide about using and marketing O.R. - is not easy.
In the beginning, it seems to me from my own practice, it would be
advisable to have both. Later on all branches in the company will be
willing and able to go for O.R. without any central support.
8. One of the big advantages of O.R. is its trans-national potential.
The built-in translation module of the IRM and its multi-language
interface makes it possible for all CEE subsidiaries to work with it in their
respective national language. Although "all business is local", the
opening of the Central and Eastern European markets in an enlarged
European Union will bring about many chances for cross-national, multi-
language research projects. This should be born in mind regardless of
the somewhat slower development of Internet use in the new CEE
democracies.
9. While in the beginning, that is a few years ago, exchange of opinion
on O.R. methodology and commercial practice – in personal or digital
form - was rather frequent, lately there has been little interchange of
information between GfK online researchers. With economic growth and
market chances recovering it would be desirable to re-establish such
contacts.
10. There is hardly any field in which change is so noticeable and so fast
as in Information Technology. O.R. is a typical product of the digital age.
In order to stay abreast of the time it is advisable to invest a lot of
energy (and some money) in this fascinating new method. FESSEL-GfK
- as could be shown - has decided to do so.
http://www.gfk.at/EN/product/onlineresearch/default.asp
Peter Diem
onlineforschung@eunet.at
|