Published on March 12, 2016
1. RESEARCH METHODS IN INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY www.humanikaconsulting.com
2. Goals of Science • Description • Prediction • Explanation
3. Research process - summarized as 5–step sequence Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research The Empirical Research Cycle
4. STATEMENT OF THE PROBLEM Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research Theory - Inductive method - Deductive method
5. STATEMENT OF THE PROBLEM Statement of the problem Statement: It is difficult for individuals in dual-career families to experience WF balance. Research Question: How can individuals in dual- career families experience WF balance? Hypothesis: Dual-career individuals who have family and organizational support are more likely to experience WF balance compared to dual-career individuals with no family and organizational support. Hypothesis Example:
6. Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research RESEARCH DESIGN Plan of Study - Internal & External Validity - Naturalness of Setting - Degree of Control Primary Research Methods - Laboratory Experiment - Quasi Experiment - Questionnaire - Observation - Qualitative Secondary Research
7. Plan of Study: Internal Validity The extent to which we can infer that a relationship between two variables is causal or that absence of a relationship implies absence of cause. The extent to which observed relationship obtained from research design/study is real or artifactual.
8. Plan of Study: External Validity The extent to which the findings from a research study are relevant to individuals and settings beyond those specifically examined in the study. The extent to which observed relationship obtained from research design/study are “generalizable”.
9. or Plan of Study: Naturalness of Research Setting - "artificiality" - contrived and artificial - controlled -"naturalness" - typically employs a real–life setting Lab Field
10. Plan of Study: Degree of Control • Confounding and extraneous variables • Manipulation—this is reflective of a high degree of control • Research designs that permit manipulation are technically referred to as "experiments"
11. Primary Research Laboratory (experimental) Quasi-experimental Questionnaire Observation Qualitative There are 5 categories of types of Primary Research:
12. Primary Research: Experimental Research Experiment • Investigator manipulates a variable under carefully controlled conditions and observes whether changes occur in a second variable • Used to detect cause-and-effect relationships Conditions that make a true experiment • Manipulation of independent variables • Random assignment into experimental conditions (experimental conditions & control)
13. Primary Research: Experimental and Control Groups Experimental group •Subjects who receive some special treatment in regard to the independent variable Control group •Subjects who do not receive the special treatment given to the experimental group LOGIC: If the 2 groups are identical except for the variation created by the manipulation of IV, then any differences between groups must be due to manipulation of the IV
14. Sample ExperimentalControl Measure DV Example of Experimental Design
15. Research Methods Experiment Study conducted in a contrived environment • Benefits: – Provides more safety – Cause and effect relationships • Manipulate I.V. (e.g., leadership style) • Measure D.V. (e.g., task performance) • Control extraneous variables (e.g., experience) • Disadvantages: – Time consuming Quasi-Experiment – not randomized or unable to manipulate IV (e.g., gender)
16. • Participants must be and are selected for different conditions from pre–existing groups • Levels of the IV are/may be selected from pre–existing values and not created through manipulation by the researcher • Unlike true experimental designs where participants are randomly assigned to experimental and control groups, with quasi–experimental designs they are NOT • Quasi–experiments DO NOT permit the researcher to control the assignment of participants to conditions or groups Primary Research Field Experiments: Quasi-Experiments
17. Quasi-Experimental Example Greenberg: Employee Theft and Underpayment Inequity
18. Greenberg: Employee Theft and Underpayment Inequity • Theft is a mechanism for redressing states of inequity • Adequate explanations can lessen feelings of inequity • This is “dose-responsive”: magnitude of the expressed inequity, rate of theft Pay deduction Expressed inequity (Employment theft)
19. Plant 1 Plant 2 Plant 3 Control No cut in pay Condition 1 Inadequate explanation Condition 2 Adequate explanation DV Employee theft Greenberg: Employee Theft and Underpayment Inequity
20. Greenberg: Employee Theft and Underpayment Inequity Time 1 End Measurement Plant A 64 55 1. Actuarial data on employee theft 2. Self-reported measures Plant B 53 30 Plant C Control 66 58 •Randomly selected treatment for A and B, C as control •Assumed/proved homogeneity among subjects in different plants •Same characteristics among those who dropped out. •Treatment was received the same by all workers in a plant.
21. 0 1 2 3 4 5 6 7 8 9 B efore During A fter Time Period R elative to Pay C ut MeanPercentageof EmploymentTheft Inadequate explanation Adequate explanation C ontrol Greenberg: Employee Theft and Underpayment Inequity
22. Primary Research: Naturalistic Observation Careful, usually prolonged, observation of behavior without intervening directly with the subjects • No manipulation by researcher • No random assignment Often referred to as ex post facto designs
23. Research Methods Naturalistic Observation Observe overt behaviors over time – Systematic sampling at various times – Representative sample • Benefits: – Use to generate hypotheses • Disadvantages: – Experimenter bias – Obtrusiveness – Frequency of behavior occurring
24. Primary Research: Survey Research Measurement and assessment of opinions, attitudes, and other descriptive phenomenon usually by means of questionnaires and sampling methods • Popular method of research for I/O psychologists • Limitations include return rate • Web-based survey
25. Research Methods Questionnaire/Survey Self-report to obtain data on attitudes/behaviors conducted by phone, mail, interviews, electronically • Benefits: – Can collect a large quantity of data • Disadvantages: – Accuracy of reporting – Representativeness of sample – Return rate
26. Primary Research: Qualitative A class of research methods in which the investigator takes an active role in interacting with the subjects he or she wishes to study • Interview/focus group • Ethnography: a research method that utilizes field observation to study a society’s culture. • Emic versus Etic - Emic: an approach to researching phenomena that emphasizes knowledge derived from the participants’ understanding of their own culture. - Etic: An approach to researching phenomena that emphasizes knowledge derived from the perspective of an objective investigator in understanding a culture.
27. Primary Research: Summary Laboratory (experimental) Quasi-experimental Questionnaire Observation Qualitative Issues: Obtaining access to samples Common method bias Choosing the correct design to answer the research question.
28. Secondary Research Methods Meta-analysis – statistical procedure designed to combine the results of many individual, independently conducted empirical studies into a single result or outcome Differences in studies could be due to statistical artifacts. Issues: - File draw effect - Subjective nature of research A class of research methods that examines existing information from research
29. Measurement of Variables Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research Types of Measurement Level of Measurement Characteristic
30. • Independent/dependent • Predictor/criterion • Continuous/discrete • Qualitative/quantitative Measurement of Variables: Types of Variables Variable: Some property of an object, phenomenon, or event whose measurement can take on two or more values
31. Measurement of Variables: Types of Variables In a study of the effects of different types of legal arguments on jurors’ perceptions of the guilt or innocence of a defendant, subjects were randomly assigned to hear an argument which related to their daily experiences or to an argument of a more abstract and idealistic nature. After listening to one of these legal arguments, subjects were asked to rate the guilt or innocence of the defendant on a twelve-point scale. What is the DV and what is the IV?
32. Measurement of Variables: Levels of Measurement A scale is a measuring device used to assess a person's score or status on a variable The four basic types of scales are: Nominal scales Ordinal scales Interval scales Ratio scales
33. Measurement of Variables: Levels of Measurement Nominal Scale: 1=Single 2=Married Ordinal Scale Not Satisfied Satisfied Very Satisfied 1 2 3 Interval Scale Degrees Fahrenheit 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 Ratio Scale Weight in pounds 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
34. Good test or measurement system should be: - reliable - valid - objective - standardized Measurement of Variables: Characteristics of Good Measurement
35. STATISTICAL ANALYSES OF DATA Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research Purpose Distributions and Their Shape Measures of Central Tendency Measures of Variability Correlation
36. Statistical tests are procedures that are used to: - Describe data - Analyze relationships between variables (i.e., make inferences) Statistical Analysis: Purpose
37. Research Steps : Statistical Analysis Descriptive vs. Inferential Statistics • Descriptive stats merely describe data – Frequency – Central tendency – Variability • Inferential stats used to test hypotheses – T-Test – Analysis of variance – Correlation – Regression – Non-parametrics
38. Data Analysis Central Tendency 1. Mean – average: X = ∑X / N Mean = 72 / 8 = 9 2. Median – middle score (when placed in order) - use when outliers exaggerate the mean Median = 8.5 3. Mode – most often occurring score Mode = 6 _ example scores = 5, 6, 6, 8, 9, 10, 11, 17 * In a normal distribution, Mean = Median = Mode
39. Data Analysis Variability • Range - distance between highest and lowest score – (Range = High score – Low score) – Range = 17 – 5 = 12 • Standard Deviation – average distance from the mean – S= Σ(x – x)2 / n – 1 S = (5-9) 2 + (6-9) 2 + (6-9) 2 + (8-9) 2 + (9-9) 2 + (10-9) 2 + (11-9) 2 + (17-9) 2 / 7 S = 3.85
40. 0 2 4 6 8 10 12 14 16 18 20 Frequency 65- 74 75- 84 85- 94 95- 104 105- 114 115- 124 125- 134 135- 144 145- 154 155- 164 IQ Scores 0 5 10 15 20 25 30 Frequency 250- 259 260- 269 270- 279 280- 289 290- 299 300- 309 310- 319 320- 329 330- 339 340- 349 Weight (lbs) of NFL Lineman 0 2 4 6 8 10 12 14 16 18 20 Frequency 65- 66 67- 68 69- 70 71- 72 73- 74 75- 76 77- 78 79- 80 81- 82 82- 83 84- 85 Professional Golf Scores Positively Skewed Distribution Negatively Skewed Distribution Normal or Bell-shaped Distribution Data Analysis Skewed Frequency Distributions
41. Data Analysis Correlation Correlation ( r ) – Degree of relationship between two variables – Used for prediction – Cannot be used to infer causation – Range from –1 to +1 – Negative r – as one variable increases the other decreases – Positive r – as one variable increases so does the other – Zero r – no relationship between the two variables r A B C A 1.0 B .40 1.0 C .20 .09 1.0
42. Data Analysis Correlation Positive Correlation Negative Correlation 0 5 10 15 20 60 80 100 120 YearsofPractice Golf Scores 0 0,5 1 1,5 2 2,5 3 3,5 4 600 800 1000 1200 1400 1600 CollegeGPA GRE Scores ** * * * * * * * * * ** * * * * * * * * * * * ** * * * * * * * * * * ** * * * * * * * * * ** * * * * * * * * * * ** * * * * * * * * * * ** * * *** * ** * * * * *** ** * * ** * * * * *** ** * ** * * * * *** ** ** * * * * *** ** * ** * * * * *** ** * ** * * * * *** ** * * *** *
43. Correlation Examples • IQ scores of identical twins: r = +.86 • Phases of the moon & # acts of violence: r = .00 • Economic conditions & # lynchings: r = -.43 • Amount of ice cream sold & # drownings: r = +.60 • Price of rum in Cuba & priests salaries in New England: r = +.38 • Number of cigarettes smoked per day & incidence of lung cancer: r = ???
44. Statistical Analysis: Correlation coefficients examples Bond, F. W., Bunce, D. (2003). The Role of Acceptance and Job Control in Mental Health, Job Satisfaction, and Work Performance. Journal of Applied Psychology, 88, 1057-1067.
45. Statistical Analysis: Correlation coefficients examples Barling, J., Kelloway, K. E., Iverson, R. D. (2003). High-quality work, job satisfaction, and occupational injuries. Journal of Applied Psychology, 88, 276-283.
46. Statistical Methods Regression Regression Variables (used for prediction) Yi = ß0 + ß1Xi1 + ß2Xi2 (Y = a + b1X1) • Predictor Variable (X) – measure used to predict an outcome (similar to independent variable) – Example: selection test scores, years of experience, education level • Criterion Variable (Y) – outcome to be predicted – Example: work performance, turnover, sales, absenteeism, promotion, etc. • Example: AFOQT scores as predictors of pilot training performance
47. Conclusions Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research • Theoretical and applied implications • Limitations • Generalizability • Size and representativeness of sample • Research method & protocol • Suggestions for future research
48. Statistical Pitfalls: Bias • Representative Sampling – Selecting a sample that parallels the population – Might use covariates to account for differences • Statistical Assumptions – ANOVA assumes a normal distribution and independence • Lack of normality is only minor problem, but may want to identify distribution shape and why • Observations may not be independent, may need to aggregate (e.g., class instead of student)
49. Statistical Pitfalls: Errors in Methodology • Statistical Power – probability of detecting a true difference of a particular size – Type I error – falsely reject null hypothesis when a true difference does not exist – Type II error – fail to reject null hypothesis when a true difference does exist – Power affected by • Sample size • Effect size (e.g., Cohen’s D) • Type I error rate selected (alpha) • Variability of sample – (F ratio = var between group / var within group)
50. Statistical Pitfalls: Errors in Methodology • Multiple Comparisons – if you compare enough variables, will find a relationship by chance alone – Bonferroni correction – family-wise adjustment (alpha = .05 / #comparisons) – Replicate – Cross-validate (holdout sample) • Measurement Errors – Reliability: Consistency of Measure – Validity: Measures what it was designed to measure
51. Statistical Pitfalls: Problems with Interpretation • Confusion over significance – P value does not reflect effect size – could have a small effect, but a lot of power • Precision vs. Accuracy – More decimals not necessarily more accurate • Causality – Correlations are not causal, but ANOVA may not be either
52. Statistical Pitfalls: Problems with Interpretation • Graphs – May not provide accurate portrayal of data 81 81,5 82 82,5 83 83,5 84 Group A Group B Score 0 20 40 60 80 100 Group A Group B Score
53. Research Critical Thinking Always think critically about the research you read – Who were the participants in the study? – How strong of a relationship was found? – Was it causal or correlational? – Was it a field study or laboratory study? – How was data collected and analyzed? – Do you agree with the conclusions based on the analyses provided?
54. Ethics in Research: What is Ethical Research? Participant Cost Gains to Field • Do not always know effects ahead of time • Ethical guidelines change over time
55. Ethics in Research: What is Ethical Research? Ethically based research is concerned about the welfare of the research participant, maintaining honesty in conducting and reporting scientific research, giving appropriate credit for ideas and effort and considering how knowledge gained through research should be used. There are no clear “right” or “wrong’ answers. Treating research participants ethically matters not only for the welfare of the individuals themselves but also for the continued effectiveness of behavioral science as a scientific discipline
56. Ethics in Research: Protecting Participants Type of Threats - Past research: e.g., Milgram studies - Participants may be told they failed an IQ or social skills test - Participant may learn something negative about themselves (tendency to stereotype others or they make unwise decisions) - Participants may perform behavior they are later embarrassed about The Potential for Lasting Impact
57. Ethics in Research: Providing Freedom of Choice Conducting research outside the laboratory - Participant may not know research is happening - Institutions Securing Informed Consent Weighing informed consent versus the research goals
58. Ethics in Research: Power Differentials Avoiding Abuses of Power Respecting Participants’ Privacy - anonymous vs. confidential
59. Ethics in Research: Describing Research Deception: occurs whenever research participants are not completely and fully informed about the nature of the research project before participating in it. - Active vs. Passive - When Deception is necessary - Simulation studies - Consequences - Debriefing
60. Ethics in Research: Ensuring Research is Ethical Department of Health and Human services has developed regulations for the protection of both animal and human research participants. • require all universities set up institutional review board (IRB) to determine whether proposed research meets regulations • researchers submit a written application to IRB requesting permission to conduct research • researchers have to describe potential risks and benefits. Researcher's Own Ethics
61. Research in Summary Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research Each stage should be conducted in an ethical and scientific manner
62. Research in Industry • Distinguishing Features – Arise from organizational problems – Use of results – Motives – Science-practice divide
63. Learning and Giving for Better Indonesia
Industrial and organizational psychology (also known as I–O psychology, occupational psychology, work psychology, WO psychology, IWO psychology and ...
Handbook of Research Methods in Industrial and Organizational Psychology (Blackwell Handbooks of Research Methods in Psychology) eBook: ...
Handbook of Research Methods in Industrial and Organizational Psychology is a comprehensive and contemporary treatment of research philosophies, approaches ...
To cover an extensive topic such as research methods in organizational psychology is most definitely an ambitious goal. Nowadays more than ever researchers ...
Handbook of Research Methods in Industrial and Organizational Psychology is a comprehensive and contemporary treatment of research ...
Industrial and Organizational Psychology is ... Knowledge required for supervision in Industrial/Organizational psychology ... Research Methods
List of Contributors x. Preface xii. Acknowledgments xiii. Part I Foundations 1. 1 History of Research Methods in Industrial and Organizational Psychology ...
"Handbook of Research Methods in Industrial and Organizational Psychology" is a comprehensive and contemporary treatment of research philosophies ...
Spector, P. (2001). Research methods in industrial and organizational psychology: data collection and data analysis with special consideration to ...
Handbook of Research Methods in Industrial and Organizational Psychology (Blackwell Handbooks of Research Methods in Psychology) - Kindle edition by Steven ...