Johnson and Reynolds Chapter 4: Building

Blocks of Social Scientific Research:

Hypotheses, Concepts, and Variables.

Note: this chapter repeats some themes from

before but has several fundamental concepts.

Characteristics of Scientific Research

I Intended to make descriptive and explan-

tory inferences on the basis of empirical

information.

I Uses explicit, codified, and public methods

to generate and analyze data (broadly de-

fined), where reliability is assessed by oth-

ers.

I Conclusions are uncertain and this uncer-

tainty is described probabilistically.

I Einstein: “. . . as far as our propositions are

certain, they do not say anything about re-

ality, and as far as they say anything about

reality, they are not certain.”

I Research adheres to a set of established

rules for inference on which validity rests.

Some Observations

I Science is a social enterprise.

I Simplification and summarization are nec-

essary.

I We have methods which differ from proce-

dures.

I The process of classifying specific events is

inference, although the traditional (statis-

tics oriented) definition of inference is to

draw conclusions about populations given

only sample data.

Four Stages of the Research Process

1. The Research Question

I Asks what is important in the real world.

I Answering this question will make a con-

tribution to some body of knowledge.

I Should be “important” in some way to

other political scientists.

2. The Theory

I Expressed as a hypothesis.

I Must be falsifiable.

I Should generate as many observable im-

plications as possible. This is called

generalizability.

I Simple theories have higher prior prob-

abilities.

3. Data

I A gathering process: observing, record-

ing, reporting.

I The more, the better.

I We want to maximize the validity of our

measurements as much as possible.

I reliability and replicability are important.

4. Analysis

I The area that we usually spend the most

amount of time on.

I The core material of this course: dif-

ferent ways that political scientists do

analysis.

I Without some analysis, there is no schol-

arly contribution.

Various Commandments

. Use observable implications to connect the-

ory and data.

. Maximize leverage: explain as much as pos-

sible with as little as possible.

. Be aware of the ecological fallacy.

. Always report uncertainty.

. Be a skeptic: consider rival hypotheses as

often as possible.

Discussion Item #1: Replication

. Replicability means making your data pub-

lic so that others can run the same analysis

that you have and verify your results.

. Oddly enough replicability, which a corner-

stone of many scientific fields, has had a

hard time permeating political science.

. Why?

. Do you see advantages/disadvantages?

Discussion Item #2: Testable Hypotheses?

. As Education increases, individuals are more

likely to participate in the political process.

. Lobbyists buy access not votes.

. Egyptian clerics influence political values.

. (INSERT NAME) will not vote in the next

election because he is poor.

. City manager run governments are less po-

litical.

. Term limits will make Congress more demo-

cratic.

Discussion Item #3: The Logic of Testing

. Karl Popper: “You can’t prove a nega-

tive because you may have simply done the

wrong test.”

. The file drawer problem:

. Suppose there exist no social patterns

to find.

. So if you had a test that 95% accurate

in finding relationships it would would

find relationships in 5% of your tests of

hypotheses.

. Journals generally only publish positive

findings.

. So everything published would be wrong

and everything we threw in our file draw-

ers would be right.

Some Terms

. Variables:

• independent (explanatory) variable

• dependent (outcome) variable

• antecedent variable

• intervening variable

. Relationships: how a variable relates to an-

other, can be positive or negative.

. Unit of Analysis: the atomic actor specified

by the hypothesis (i.e. members of the

social system under study).

. Tautology: circular reasoning such that causal-

ity cannot be claimed. Example from the

book: “The less support there is for a

country’s political institutions, the more ten-

uous the stability of that country’s political

system.”

. Cross-Level Analysis: making claims at dif-

fering levels of aggregation. Also called

“ecological inference.” Very difficult.

♦ Ecological Inference: the process of us-

ing aggregate (ecological) data to infer dis-

crete individual-level relationships of inter-

est when individual-level data are unavail-

able.

♦ One of the oldest methodological problems

in the social sciences in general.

♦ History of the Ecological Inference Prob-

lem:

1. The oldest recognized quantitative prob-

lem in political science: William Og-

burn and Inez Goltra, in the very first

multivariate statistical analysis of poli-

tics in a political science journal (PSQ

1919) made ecological inferences and

recognized the problem. The big is-

sue: are the newly enfranchised women

going to take over the American politi-

cal system? They regressed votes in an

Oregon referendum on the percent of

women in each precinct in a local elec-

tion where women were allowed to par-

ticipate before 1919. But they worried:

It is also theoretically possible to

gerrymander the precincts in such

a way that there may be a negative

correlation even though men and women

each distribute their votes 50 to 50

on a given measure...

2. Robinson’s (1950) article was the most

influential, causing:

(a) several literatures to retrench, includ-

ing studies of local and regional poli-

tics through aggregate electoral statis-

tics.

(b) a new methodological literature to emerge

devoted to solving the problem.

(c) gave rise to survey research as the

dominant data collection methodology

in several fields, notably political sci-

ence and sociology.

♦ Voting in the Weimer Republic: starting

in the 1930 election in Germany a previ-

ously obscure party, the National Socialist

German Worker’s Party, began amassing

electoral support and would eventually con-

trol the government. What type of person

voted for this party? Were they predom-

inantly from the lower middle class dra-

matically affected by failed economic poli-

cies? Were they from particular religious

groups or geographic regions? Since most

records were destroyed during the war, sur-

vey research is impractical and unreliable in

this setting, and only aggregate data ex-

ists, then it is necessary to make ecological

inferences.

♦ Epidemiology and Public Policy: the Ap-

palachian region of the United States suf-

fers from dramatically higher rates of cer-

tain cancers. While much data can be ob-

tained by the medical records of living pa-

tients and some historical data can be ob-

tained for patients treated in major medi-

cal facilities over the latter part of the last

century, a great many cases were poorly

diagnosed and produce no useful evidence

about type of cancer and contributing causes.

In this case ecological inferences can be

made about covariates.

♦ Education: Some students attend elite

private schools through voucher and assis-

tance programs, whereas others can attend

due to their parents resources. Do the for-

mer students perform as well as the lat-

ter? Since individual student performance

records are protected by a host of privacy

laws, only ecological inference can answer

this question.

♦ Marketing: It is often the case that firms

know the sales figures and the demograph-

ics for various regions but cannot specif-

ically determine which sub-groups of the

population prefer their products. Ecologi-

cal inference can indicate brand preference

in relatively fine granulation.

Motivating “Case”

♦ 1965 Voting Rights Act: along with its

extensions (1970, 1975, 1982) prohibits elec-

toral discrimination based on race, color,

or language. Evidence of discrimination

requires an indication that individuals are

hampered or prevented from voting.

♦ As amended in 1982, the Act also forbids

any practice that is shown to have a dis-

parate effect on minority voting strength:

in aggregating voting from the precinct level

to the district level or higher, the votes of

minorities are sufficiently diluted as to deny

effective representation.

♦ In Thornburg v. Gingles 1986 the U.S.

Supreme Court ruled that discriminatory

intent is not necessary, that it is sufficient

for the plaintiff to show that absent elec-

toral representation, three conditions are

sufficient:

1. there exists a geographical district where

the minority is a majority.

2. the minority group is politically cohe-

sive.

3. the majority votes essentially as a uni-

fied racial voting block.

Motivating “Case” (continued)

♦ June 4, 1990 the U.S. District Court of

the Central District of California filed an

opinion in Yolanda Garza et al. v. County

of Los Angeles, Los Angeles Board of Su-

pervisers et al. in favor of the plaintiffs.

♦ Background: The Los Angeles County Board

of Supervisors consists of 5 members elected

to 4 year terms in non-partisan elections

by district. An Hispanic had never been

elected to the Board. In the ruling the

Judge ruled that the plaintiffs had provided

adequate statistical evidence through eco-

logical inference that the Board of Super-

visors had engaged in periodic redistricting

that diluted Hispanic voting power.

♦ The defining issue: compact precincts of

minority voters were shown to be aggre-

gated together with larger numbers of non-

minority precincts in all of the 5 districts.

The Problem

District Level:Race of Voting Decision

Person Democrat Republican Abstainblack ? ? ? 55,054white ? ? ? 25,706

19,896 10,936 49,928 80,760The 1990 Election to the Ohio State House, District 42.

Precinct Level:Race of Voting Decision

Person Democrat Republican Abstainblack ? ? ? 221white ? ? ? 484

130 92 483 705Precinct 1 of 131 in District 42.