Research Methods: Surveys and Questionnaires
Thursday 25 November 2010
Conclusion
Monday 22 November 2010
Reliability and validity of qualitative and quantitative data in surveys
This section will focus on the definition and the methods of evaluating reliability and validity in survey research.
Reliability and validity is a major issue when it comes to research, indeed failure to assure the validity and/or reliability of the findings may cause the research to be questioned even worse rejected as invalid.
Reliability refers to consistency and/or repeatability of the measurement; in other words, consistency can relate here to the questionnaires being clear and well define in order to not confuse the respondents and repeatability here means that if searchers have findings from a group they should be able to repeat the survey and get exactly the same results.
Consistency:
There are several ways to measure survey research consistency;
Qualitative research:
Consistency may be measured by using triangulation; this will be done using multi sources of information for example by taking different information from three different investigator or just by combining more than one methods of data collection in the same research; interview and survey and so on. This will actually fructify and increase consistency of the research and therefore reliability as well. The advantages from this approach are that is time consuming, expensive and request lots of effort.
Quantitative research:
Consistency could be measure by using one or more following strategies:
- Inter-Rater or Inter-Observer Reliability
- Test-Retest Reliability
- Parallel-Forms Reliability
- Internal Consistency Reliability
This will also result the findings to be reliable.
Validity refers to the degree to which the measurement procedure actually measures the concept that it is intended to measure.
Validity research has several ways to be obtained:
- Face validity
- Content validity
- Predictive validity
- Concurrent validity
The picture below shows us the relativity between reliability and validity; it gives us a well descriptive view of the relationship between reliability and validity.
As the picture shows, there are four possibilities, first possibility the measure can be reliable but not valid, second one the measure may be valid but not reliable, the third one is the worst case where the measure are neither reliable and not valid as well however the last measure is a perfect case where we have both reliability and validity.
To conclude from the picture, we noticed that reliability and validity have related ideas however one can work without the other one, or both at the same time.
References:
William M.K. Trochim. (2006). Types of Reliability. Available: http://www.socialresearchmethods.net/kb/reltypes.php . Last accessed 18th Nov 2010.
Adri Labuschagne. (2003). Qualitative Research - Airy Fairy or Fundamental?. Available: http://www.nova.edu/ssss/QR/QR8-1/labuschagne.html . Last accessed 18th nov 2010.
Celia Taylor, Graham R. Gibbs and Ann Lewins. (2005). Quality of qualitative analysis. Available: http://onlineqda.hud.ac.uk/Intro_QDA/qualitative_analysis.php . Last accessed 18th nov 2010.
William M.K. Trochim. (2006). Reliability & Validity. Available: http://www.socialresearchmethods.net/kb/relandval.php . Last accessed 18th Nov 2010.
Thursday 18 November 2010
Surveys
Surveys like all other data collection have its advantages and disadvantages. The major noticeable advantages of surveys are time saver as surveys allow to collect a large amount of data in short time, they are less expensive than most of the other type of data collection and they allowed to collect data on wide range of things however surveys are not perfect they also have disadvantages for example accuracy, the response given may not reflect the reality therefore not accurate, and there is no way to know if the participants are reliable.
In the next two sections we will have a closer look on how surveys can use qualitative data (e.g. ask open-ended questions) or quantitative data (e.g. use forced-choice questions) measures however the type of survey to be carried out depends on the target population and the subject under investigation.
Researchers must be precise about their questionnaires as the quality of the data from quantitative research is directly dependent on that. The objectives of quantitative data are to quantify data and generalize results from a sample to the population of interest and also to measure the incidence of various views and opinions in a chosen sample.
The advantages of qualitative data, its methods usually provide quantifiable, reliable data and generalisable to the larger number of population of interest. They can also be represented visually in tables, graphs, charts or histograms.
However there are some weaknesses as well among other things closed question which can be answered with “yes or no”, a single word or a short phrase.
The quantitative data research can be used to determine the scale of satisfied customers for example by disturbing a questionnaire sample to customers (questionnaires are usually easy and quick to answer) and analyse the answers and scale them with 4 or 5 point scales from Very Satisfied, Satisfied, (Neither), Dissatisfied, Very Dissatisfied of course there is more behind that to really determine the satisfaction of customers.
Usually after a quantitative data research searchers often use qualitative research to explore further the findings from quantitative data which are “yes or no” or just a word.
Are you happy with your current ASDA local store?
2/Satisfied
3/ Dissatisfied
4/Very dissatisfied
Yes
No
Interviewers questions are generally vague, they ask the participants open-ended questions which request thinking and reflection thus they are able to express themselves and with their own words and idea, give their opinions, their feelings and perceptions on a particularly topic which might be sensible accordingly their answers as well. For that reason interviewers must be cautious about the ethical and confidentiality issues because open-ended questions involve personal and honest responses. The interviewers must keep the participants’ identity confidential and protected.
Qualitative data can be gained questioning customers, citizens or students (look questions below), or by immersion in a culture (ethnography) in this case the interviewers will probably have to deal with the ethical issues. In quantitative data, the interviewers/ researchers become the instrument of data collection that may have consequences and vary the results depending on who is conducting the research, as consequence the result may be considered as invalid therefore to valid as accurate, the interviewers or researchers must then compare his findings information with similar information from other surveys.
What did you fail on your course?
How do you keep focused on your course?
What are the good things and the less good things about your course?
References:
Kendra Cherry. (2010). What Is a Survey?. Available: http://psychology.about.com/od/researchmethods/f/survey.htm. Last accessed 11/11/2010.
Ira Kerns. (2003). Quantitative & Qualitative Research. Available: http://www.guidestarco.com/Qualitative-and-Quantitative-Survey-Research.HTM. Last accessed 11/11/2010.
Nedra Kline Weinreich. (2006). Integrating Quantitative and Qualitative Methods in Social Marketing Research. Available: http://www.social-marketing.com/research.html. Last accessed 11/11/2010.
unknown. (2010). Qualitative research. Available: http://en.wikipedia.org/wiki/Qualitative_research. Last accessed 11/11/2010.
Wednesday 17 November 2010
Environment of surveys
The following list describes the methods of surveys including their use, advantages and drawbacks.
E-mail survey: (Commonly used in all areas)
Advantages:
- Fast results
- Easy to modify
- Data sets are created in real time
- Inexpensive in most cases
- Large sample size
Disadvantages:
- Honesty of responses can be an issue
- If not password protected, easy to manipulate by completing multiple times.
Advantages:
- Higher response rate
- Good for large sampling frames
- Higher validity of answer
Disadvantages:
- More time consuming
- Might be expensive (international surveys)
- Might be annoying to the subjects to be contacted on phone
Online survey: (used by all areas)
Advantages:
- Very fast results
- Easy to modify
- Used in large scale of industries
- Cheap and very efficient
- Easy to target certain interest groups (i.e Facebook)
- Easy to process the data
- Large sample size
Disadvantages:
- Difficult to validate the gathered data
- Subject might not give complete/ honest answers
- Not suitable for people who do not use computers
Face to face interviews:
Advantages:
- Face to face communication- more honest answers
- Subject are more likely to give more accurate and detailed answers (qualitative data)
Disadvantages:
- Some people do not like to talk about personal information (prefer to write)
- Time consuming
- Expensive
- Smaller sample size
Organisations and scientific institutions have to consider these aspects before they decide about the method of surveys and they also have to find the balance between cheap and quality research.
Conducting the surveys still has ambiguities that need to be handled or solved. Would there be a more efficient and cheaper method? How can technology help to develop surveys?
At this stage the only limit is the technology and the knowledge of subjects about technology…
References:
Statistical survey
http://en.wikipedia.org/wiki/Statistical_survey#Telephone
Advantages and disadvantages of Questionnaires
Thursday 11 November 2010
Questionnaires: Obtaining Quantitative/Qualitative data
Tuesday 9 November 2010
Introduction
It would be advisable before any researcher to think about the way questionnaires and surveys are designed before starting to conduct a research project. Robson (2002) suggested that objectives of the study can be achieved if questionnaires are well defined and designed, short and less than 12 words; for example if the respondent does not understand the question, the answer can be different from the one they would have given if they had understood the questionnaire. We agree with Robson’s views about the design of questionnaires because getting the respondents to understand the questions is crucial for obtaining the right information related to the study.
In this blog we are going to be focusing on areas such as questionnaires, quantitative and qualitative data, environment of surveys and questionnaires, etc.