Lies, damn lies, and statistics

I surf the web, as the expression goes, quite a bit. Not that long ago [Oct. 4, 2007, actually], Amitai Etzioni posted an interesting and significant item in The Huffington Post , and it seemed worth sharing with you now.
“There are three types of lies – lies, damn lies, and statistics.” A quote from Mark Twain? Perhaps. You could look it up. In any event, the saying is relevant, and is perhaps as good as Etzioni’s own headline, “Small lies, big lies, and the Israel lobby.” Of course what he is talking about is as much the “anti-Israel lobby” as the Israel lobby. He’s talking about the use – that is, the misuse, the abuse – of information in the service of those who are attacking the bogeyman of the “Israel lobby.” He’s talking about Mearsheimer and Walt.
Amitai’s personal history is worth knowing, by the way. Makes me want to read more.
According to the Wikipedia entry on him, “[h]aving fled to Palestine from Nazi Germany in the 1930s, Etzioni studied with Martin Buber at the Hebrew University in Jerusalem. In 1958 he received his PhD in sociology from the University of California, Berkeley, where he completed his degree in the record time of 18 months. He was a professor of sociology at Columbia University for twenty years, serving as chair of the department for part of his time there. He joined the Brookings Institution as a guest scholar in 1978 and then went on to serve as Senior Advisor to the White House on domestic affairs from 1979-1980. In 1980 he was named the first University Professor at The George Washington University, where he currently serves as the director of the Institute for Communitarian Policy Studies.” {Never heard of Communitarianism? See here.] Last introductory thought: after reading the item below, take a look at Etzioni’s own blog, especially his two follow-ups to the many comments he’s received to the item below.
— Arieh Lebowitz

There are quite a few who have taken for granted the veracity of claims that the Israel lobby is all-powerful on the grounds that a new book making this case has been written by two highly regarded scholars; John J. Mearsheimer and Stephen Walt of the University of Chicago and Harvard’s Kennedy School of Government, respectively. In fact, the quantitative data they cite amount to (at best) a very thin reed on which to hang such a mighty claim. I will donate my house to anyone who can find a half respectable social science publication that would publish what these two present as evidence.
The authors write:

In 1997, Fortune Magazine asked members of Congress and their staffs to list the most powerful lobbies in Washington. AIPAC was ranked second behind the American Association of Retired People, but ahead of the AFL-CIO and the National Rifle Association. A National Journal study in March of 2005 reached a similar conclusion, placing AIPAC in second place (tied with AARP) in the Washington ‘muscle rankings’.

In fact, the Fortune survey was not made of Congress members and their staffs, but of 2,165 “Washington insiders” (chosen by two panels whose membership has not been disclosed), a group that includes an unknown number of congressional members and staffers, among an unknown number of others. More importantly, in both surveys roughly six out of every seven persons asked, i.e., most of those asked, did not respond . The authors’ claim that members of Congress and their staffs ranked the Israel lobby higher than many others is based on 15% of those who were surveyed. No respectable social scientist (and many unrespectable ones) would dare to suggest that they have a sense of what any given group holds on the basis of the responses from such a small minority.
Moreover, social science has numerous procedures to correct for such a deficit of responses. One can return to the same group and elicit more answers, draw another sample, or study the differences between those who did and did not respond–and adjust the conclusions accordingly. None of these methods were employed here.
The number of people who responded is so small that an additional vote or two, or a change of mind by one or two respondents, would have significantly altered the results of the survey. The total number of the National Journal responses — which did survey only law makers — is 73. (Congress, the last time I checked, had 535 members and at least 17,000 staff members). The National Federation of Independent Business was ranked first and the National Rifle Association second–with nine and eight votes, respectively! In third place, ranked as the most powerful by seven members, was the US Chamber of Commerce. The AARP and AIPAC were each given the nod by five members. The oil companies and the arms manufacturers were not on the list of those to be ranked. I wonder if any student at GWU could get away with a term paper that held that such small numbers support a generalization about any given population or the ranking of a set of groups.
Some will say that all of this is nothing other than typical social science hair splitting. But, these data go to the heart of the matter. Is the Israel lobby just one among a whole slew of lobbies, each pulling Washington its own way? Is it one of the more effective ones? Or can it trump all the others? What the data show is surprisingly little. The book stands much more on accusatory anecdotes than, as the authors’ claim, on evidence.