Informational Lobbying: Theory and Effectiveness

post by Matt_Lerner (mattlerner) · 2020-07-30T22:02:15.200Z · score: 44 (15 votes) · EA · GW · 10 comments

Contents

    Conclusions
    Introduction
      A caveat
    My process
      Focus
      Methodology
    Background and Terminology
      Why informational lobbying?
      How does lobbying work?
      How can lobbyists influence policy?
    Theoretical background
      Exchange and persuasion
      Money, access, and information
      Theoretical models of lobbyist behavior
    How lobbying is studied empirically
      Data sources
      Research designs
    Previous Literature Reviews
      Smith (1995)
      Potters and Sloof (1996)
      Burstein and Linton (2002)
      Lowery (2013)
      De Figueiredo and Richter (2013)
    Studies that were instrumental in my conclusions
      Caldeira and Wright (1998)
      De Figueiredo and Silverman (2006)
      Baumgartner et al (2009)
      Montes-Rojas (2013)
      Goldstein and You (2017)
      Ludema, Mayda and Mishra (2018)
    Findings
      Does lobbying work?
      When does lobbying work?
    Case Studies
    "Effective lobbying"
      How would this work?
        1) Identify a set of high-impact policies that meet the following criteria:
        2) Identify the policies least likely to face organized opposition (e.g. from entrenched and well-resourced interests inside or outside the government, or from the public)
        3) Identify what avenues of lobbying are most appropriate for enabling the policy goal
        4) Identify the proximate course of action that would be required to increase the likelihood of each such policy change in the near term:
        5) Fund research necessary to maximize the effectiveness of lobbying for each policy
        6) For each policy in the portfolio, fund lobbying and advocacy informed by the research produced in (4).
        7) Measure impact
      Does effective lobbying already exist?
      Some ideas
      Cost-effectiveness analysis
  Appendix
    How does federal legislation work in the U.S.?
    Code for future in-depth analyses
    Bibliography, etc.
None
10 comments

Note: A version of this document was reviewed by Hauke Hillebrandt, David Manheim, Samuel Hilton, Aaron Gertler, and Tara Law, who I thank profusely for their useful and encouraging feedback.

At the end of this document, I make a proposal for high-leverage lobbying for altruistic ends, conduct a preliminary cost-effectiveness analysis, and offer some rough suggestions for policies that might be worth advocating in Congress. Several reviewers said this was their favorite part of this piece, so you may want to read it first.

Conclusions

My confidence in each conclusion leads each bullet point, in parentheses.

The quality of the evidence in this domain is considerably worse than in other areas of inquiry in the social sciences. In general, the question I tried to ask myself when assessing these potential conclusions was this: "How likely would I have been to see this evidence, in aggregate, if this statement was false?"

Introduction

A few months ago, I started thinking about the Tullock paradox: "Why is there so little money in politics?" That is, given the extraordinarily large rewards available to an interest that captures the cooperation of government, why is outright corruption so rare, and why is industry's expenditure on political influence relatively small? Tullock's observation was made mostly with regard to campaign contributions, but it applies just as well to lobbying: the market for new vehicles in the U.S. was worth nearly half a trillion dollars in 2019, but the entire automotive industry spent only $70 million on lobbying -- 0.01% of the value of the market.

Tullock's observation suggests that, for groups that want something from government, lobbying is potentially an extraordinarily long lever with which to move policy. The question motivating this review is whether that lever is also available for altruistic ends.

This review has three purposes:

1) To serve as a reference regarding the effectiveness of informational lobbying for groups or individuals considering funding advocacy or influence campaigns.

2) To provide a starting point for others who may be interested in conducting further research themselves.

3) To try to answer to my satisfaction the question of whether lobbying works before we try to uncover the most effective lobbying tactics and techniques.

A caveat

It's clear that lobbying has a negative connotation. Stories about the pernicious impacts of lobbying by cigarette companies, oil companies, and professional organizations are common, and it seems clear that the influence of moneyed interests can hurt the public. But lobbying is just a tool: informational lobbying, as I'll discuss further down, is no more or less than talking to government. Although the research I review here is focused on professional lobbyists, lobbying in general can be done by anyone, from K Street professionals to high school students. I don't take a moral stance on whether lobbying in general is good or bad, or whether it should be restricted in some way.

My process

Focus

The empirical section of this review is intentionally narrow. I focus on lobbying directed at legislative bodies at the national level (e.g. the Senate and House) in the United States. Aside from my being American and therefore more familiar with the American system, there are a few reasons for this narrowness:

Crucially, I focus on informational lobbying: when lobbyists communicate directly with policymakers to shift policy. Policymakers include members of Congress and their staffs, members of the executive branch, bureaucrats, and regulators. I expressly do not investigate the empirical impact of political campaign contributions or grassroots advocacy here. However, the topics are closely tied together, as I describe at the beginning of the "Theoretical Background" section.

Methodology

This is a narrative review. I have tried to synthesize what I have learned about the current state of academic knowledge on informational lobbying at the level of the U.S. Congress, communicate my read on the evidence, and highlight the main points of uncertainty.

I took a snowball approach and started with two origin points: (1) a handful of relevant Google Scholar queries (e.g. "empirical lobbying effectiveness", "interest group influence in politics"), and (2) the empirical studies described in the literature reviews I summarize below. Starting from these lists, I followed both backward- and forward-citation chains until I was satisfied I had encountered most of the main themes in the literature and had read most of the empirical work relevant to the review. Not all major studies are included, largely as a function of my own resource constraints. John Mark Hansen's 280-page Gaining Access: Congress and the Farm Lobby, 1919-1981 (1991), for example, does not get the attention it deserves here.

Readers may note that the theoretical section is heavy on the political science literature and comparatively light on relevant subfields in economics, such as public choice theory. The primary reason for this is that the literature in public economics, while theoretically instructive, is somewhat orthogonal to my main concern here, which is this: How effective, as a means of influencing policy, is the provision of information directly to legislators, and what circumstances influence a lobbyist's chances of realizing the policy that s/he advocates?

Background and Terminology

Why informational lobbying?

I use the definition provided by De Figueiredo and Richter (2013): Lobbying is "the transfer of information in private meetings and venues between interest groups and politicians, their staffs, and agents."

Lobbying can be done by the interested parties themselves, but the term "lobbyist" generally refers to individuals and firms that attempt to influence government on behalf of private clients, such as corporations or nonprofits. It is also conventional to distinguish between "inside" lobbying, which consists of direct communication with policymakers, and "outside" lobbying, in which organize interests attempt to shift public opinion in hopes of affecting legislative decisions. This review concerns only inside lobbying.

Lobbying and political campaign contributions, as to PACs, are both types of corporate political activity (CPA), along with in-kind gifts, participation in Chambers of Commerce, legal action, grassroots influence, and other similar activities. It's necessary for me to specify my focus on informational lobbying because lobbying is often conflated with other kinds of CPA, particularly PAC contributions, in both the academic literature and the mainstream press, as in this FT article by the economist Tim Harford.

However, not only are these avenues of influence structurally and legally very different, but spending on lobbying in a given year is typically an order of magnitude larger than spending on campaign contributions. Some researchers (e.g. Lux, Crook and Woehr 2010) view the spending differential between lobbying and PACs as an indication of the greater relative importance of lobbying.

My focus on informational lobbying as opposed to other avenues of influence is pragmatic. Evidence about the efficacy of informational activities is valuable and portable. I see two possible takeaways from this research:

How does lobbying work?

In 2019, reported lobbying expenditures totaled around $3.5 billion. This money, spent largely by industry, pays the salaries of registered lobbyists; it doesn't go directly to members of Congress or their staff. Lobbyists conduct research on relevant topics, including the political constituencies that may support or oppose a given policy, and set meetings with bureaucrats and politicians, which they use to provide information that they hope will sway votes or policy positions. Many firms keep lobbyists on retainer in order to protect the status quo, and change their political influence strategies only when some new threat arises. This means that the effect of lobbying can be hidden: the absence of new legislation can also be a goal of lobbying. Unlike campaign contributions, lobbying expenditures are not limited by law; nevertheless, most firms do not lobby—although they are often members of trade associations and other groups that do engage in political advocacy. There is some evidence that lobbyists sometimes help legislators write legislation (Hall and Deardorff 2006).

In the U.S., lobbyists are required by The Lobbying Disclosure Act of 1995 to register with the government and to file regular reports on their activities and on their spending. A lobbyist is anyone who makes more than one communication to a covered official while on contract to a particular client. Covered officials include not just members of the Senate and House but their employees, employees of Congressional committees, etc. - regardless of their job level or function.

The Executive branch is also covered by the Lobbying Disclosure Act, but is less extensive: only the President, the Vice President, Schedule C political appointees (such as a special counsel), and a handful of other top-level positions are covered by the Act.

How can lobbyists influence policy?

Theoretical background

I don't claim to include all relevant theories in what follows, but I aim to make some useful distinctions and synthesize some of the main themes in the literature. I do not include the sizable literature on the drivers of policy change in general; for that I refer the reader to compendia such as Theories of the Policy Process (Sabatier and Weible 2018).

Exchange and persuasion

Theoretical work about political influence has tended to fall into two broad (though not exclusive) categories. In general, there are "exchange theories" and "theories of persuasion" (Hall & Deardorff 2006).

In exchange models, campaign contributions or in-kind gifts are exchanged for Congressional votes. This is, to a first approximation, the model assumed in most lay conversations about "money in politics." There is mixed evidence regarding this phenomenon, also known as vote buying. For a good (if now somewhat dated) lit review on the topic see Ansolabehere, de Figueiredo, and Snyder (2003).

In persuasion models, information is at the center of the relationship between legislators and lobbyists. Lobbyists, in this view, can use information to either (1) genuinely persuade legislators or (2) convince self-interested legislators that their chances of electoral success would be enhanced by taking the advocated position. Lobbyists have a comparative advantage in their ability to expend significant resources collecting and presenting such information, e.g. polls of relevant constituencies.

Money, access, and information

As Wright (1990) notes, many studies on the impact of contributions on Congressional voting fail to make a distinction between contributions and informational lobbying, which makes it impossible to separate the impact of lobbying from the impact of the contributions themselves. This puts a finer point on an important way that the exchange and persuasion models come together: money may not buy votes, but it's possible that money buys access.

Legislators have limited time, and if they prioritize contributors when arranging meetings and phone calls, then these big spenders will have more opportunities to influence policy. This is the view advanced by Wright, as well as by Austen-Smith (1995), and supported by some empirical evidence (Tripathi, Ansolabehere and Snyder 2000 and Langbein 1986). Contributions may also serve a signaling function to legislators with reference to possible future lobbying, demonstrating that interest groups have both the resources and the intent to provide necessary information.

Theoretical models of lobbyist behavior

Pluralism and the communications approach

In the mid-20th century, a prominent strain in political theory was the pluralist model, which conceived of policy as being primarily a result of competition between diverse interest groups (Denzau and Munger 1986). To pluralist thinkers like Truman (1951) and Latham (1952), competition between interest groups was a source of stability and a fundamental component of democracy. Archetypal pluralist works (e.g. Dahl 1956) held that political outcomes are the result of struggle between interest groups, who compete within specific issues or policy areas and tend to be influential primarily within those domains, rather than dominant in society in general. In the pluralist view, power is dispersed across society; policy is a result of mediation and healthy discussion between a large number of competing interests. The influence of interest groups is viewed as a benign representation of the will of the people, weighted by its salience to a specific subset of citizens (Lowery and Gray 2004). The pluralism hypothesis implies that interest groups primarily serve the purpose of raising legislators’ consciousness of the preferences of existing constituencies; they cannot make legislators do things that are against their own electoral interests.

Pluralism fell from favor under the weight of empirical research finding (a) that American voters were largely ignorant and apathetic about politics and (b) that organized interests seemed to work less like "pressure groups" and more like "service offices" for members of Congress already agreeing with them. At the time, lobbyists spent considerable resources communicating with already-friendly legislators (Bauer, Poole and Dexter 1963). Several other works of the same period and much later came to similar conclusions, holding broadly that lobbyists are at the beck and call of legislators, who have a great deal of agency in how they allocate their time (Ellis 1992).

In his book The Washington Lobbyist (1963), Lester Milbrath presented interest group politics as a game of communication, rather than of influence. The communications approach provided a compelling explanation for the tendency of groups to lobby already-agreeing legislators: targeting opposing legislators constituted an attempt to pass a message through a closed channel. Milbrath, along with Bauer et al., is largely responsible for the modern conception of lobbying as being largely about information. (McGrath 2017).

The supply price for policy

Denzau and Munger (1986), building partly on earlier ideas developed by Stigler (1971), offered a formal model predicting that legislators are most likely to respond to "organized group pressures" when constituents are uninformed or indifferent. In their model, the amount that interest groups must offer a legislator for his services (e.g. his vote) depends on the legislator's productivity and the preferences of voters in his district. The price of policy depends on a legislator's comparative advantage in providing votes. Comparative advantage is determined by the hostility of voters at home to a given policy and the productive capacity of the individual legislator (e.g. seniority, committee assignment, competence). The primary conclusion of this influential paper is that "contributions can have some influence on policies about which voters are divided, ignorant, or indifferent."

Counteractive lobbying

Austin-Smith & Wright (1994) proposed a theory of counteractive lobbying in response to the by-then mainstream awareness that interest groups often lobbied legislators predisposed to agree with them. Counteractive lobbying occurs when groups lobby in order to offset the pressure applied by opposing interests. These authors propose that lobbying does, in fact, influence the positions taken by legislators, but that lobbying by friendly interests takes place in order to prevent allied legislators from changing their positions.

Legislative subsidy

Hall & Deardorff (2006) have proposed a legislative subsidy model, in which lobbying operates with respect to budget, not to preferences. Lobbyists offer a "gift" to like-minded legislators, providing assistance in the form of legislative intelligence, information, connections to other groups and supporters, structured arguments, and other types of free intellectual labor. This more recent model, like the counteractive lobbying theory, addresses the demonstrated tendency of lobbyists to spend time and effort lobbying those who already agree with them. The subsidy model and other models are not mutually exclusive; lobbyists are free to select from a "menu" of available strategies. This model is (for example) consistent with the Denzau & Munger model described above: lobbying under the subsidy model can be seen as an in-kind contribution to legislators, so the supply price of policy is coterminous with lobbying labor purchased. Hall & Deardorff nonetheless argue that "preference-centered" lobbying (in which lobbyists seek to change legislators' preferences) is relatively rare.

Interpretation

Smith (1984) observed the lobbying activities of the National Education Association in the spring and summer of 1977 and proposed a model in which advocates—including not just lobbyists but also fellow members of Congress, staffers and others— influence members' interpretations of the consequences of a policy. Smith proposes a two-stage model of policy formulation. In the first stage, members interpret the policy; in the second, they attempt to discern how their interpreted policy consequences impact their own policy goals. Advocates can shift policy by demonstrating to legislators how a given policy could advance their own goals. However, since basic human constraints (e.g. bounded rationality) cause interpretations to be unstable, advocates' influence on policy is also itself unstable.

Political accountability

Smith (1995) singles out Arnold (1990) as having a rare comprehensive theory in which policy outcomes are driven by (1) whether legislators perceive constituents as attentive or inattentive to the issue at hand, (2) how salient the issue is to constituents, and (3) how capable constituents are of holding legislators accountable. Arnold's theory holds that pressure succeeds only when an issue is viewed as important to attentive citizens and not to inattentive citizens, and when it has champions in Congress who will do everything "procedurally necessary" to ensure that legislators are held accountable by constituents.

How lobbying is studied empirically

Research about political influence has tended to focus overwhelmingly on campaign contributions, to the neglect of informational lobbying. Several of the papers I reviewed (e.g. Evans 1996, Bombardini and Trebbi 2019, Caldeira and Wright 1998) suggest that the disproportionate attention afforded to campaign contributions is mostly a function of greater data availability; at least one (Milyo, Primo and Groseclose 2000) suggests that this tendency in the literature is a case of "looking under the lamppost": data about campaign contributions has historically been easier to come by.

Data sources

While information about federal political contributions has been available since 1974, data about federal lobbying expenditures has only been available since 1999 as a condition of the Lobbying Disclosure Act. The Honest Leadership and Open Government Act of 2007 made electronic filing mandatory, and data since 2015 is now available from the Senate. Senate data is formatted as XML; more helpfully structured data is available from the Center for Responsive Politics (CRP). A considerable portion of recent research on this topic uses data from CRP.

A significant chunk of the research I consider below focuses on the relationship between lobbying expenditure and firms' financial outcomes; these studies often merge CRP data with business information from Compustat. Some sources also use lobbyists.info (now washingtonrepresentatives.com) for biographical information about lobbyists and about their relationships.

Some enterprising researchers (e.g. Evans 1996) have pursued much more labor-intensive data collection operations, with significant amounts of hand-coding and manual review. Baumgartner et al 2009, one of the most impressive recent works on this topic, followed a random sample of issues over a four-year period, collecting information via large numbers of interviews with lobbyists and policymakers and hand-coding their responses.

Research designs

To my knowledge, there has never been a controlled experiment about informational lobbying— this kind of thing would be tricky to pull off, though it's probably not out of the question. In general, work attempting to assess political influence through communications channels takes one or more of the following approaches:

Previous Literature Reviews

Smith (1995)

Smith identifies a body of work that concludes, from formal models of the lobbyist-legislator relationship, that a primary persuasive virtue of lobbyists is their ability to affect decision-making by reducing legislators' uncertainty about bill consequences. Smith finds little empirical support for these models, with the exception of Austin-Smith and Wright's (1992) theory of counteractive lobbying, for which the authors find empirical support in Austin-Smith and Wright (1994).

Smith identifies eight studies from the preceding decade that assess the impact of lobbying on roll call votes, and finds that the research presents a "mixed picture" of the possible effects of lobbying. Smith finds that most empirical studies suffer from measurement and specificity errors, and draws no firm conclusions about the effectiveness of lobbying in general.

Potters and Sloof (1996)

This review explicitly focuses on quantitative data around "how and when" interest groups influence public policy, but does not distinguish between different types of interest group influence (informational lobbying is among those types studied). The authors make a useful distinction between two types of variables in empirical interest group research: (1) the behavior of individual policymakers and (2) policy outcomes. Potters and Sloof find that empirical research on interest group influence outside of campaign contributions is rare. A key observation of this paper is that single-equation models are often used to estimate interest group influence, but that in reality causality flows in both directions: politicians' positions influence group behavior, as well as the reverse.

Potters and Sloof conclude that there is evidence for the effectiveness of lobbying, particularly when bills have "a narrow focus and low public visibility." In addition, they find that interest groups have greater success influencing policy when politicians are under low democratic pressure (e.g. low intra-party competition) and less success when there is a well-informed electorate.

Burstein and Linton (2002)

Burstein and Linton analyze every article on policy change in the top three sociology and top three political science journals. Their unit of analysis is the effect of a predictor on a dependent variable, e.g. between a measure of either the resources or activities of an organization and a policy outcome. They find 53 articles containing a total of 230 effects. They find that statistical significance is a more common standard in this literature than substantive significance, and reviewed findings using a scale that takes both standards into account. Burstein and Linton find that the impact of interest groups is statistically significant roughly half the time, and substantively important (according to the authors' own assessment) in about a fifth of cases.

Burstein and Linton conclude that the hypothesis that interest groups influence policy significantly is "not as well supported by the data as we might expect." However, they offer three potential sources of bias in their estimate.

First, they suggest that their estimate of the frequency with which political organizations influence policy may be too high relative to its actual effect—they note that few studies include variables for public opinion, and suggest that the inclusion of such a variable might cause other apparently significant relationships to disappear.

Second, they suggest that their estimate may be too low: they note that theory predicts that it is the way in which resources are deployed, not the size of the resources themselves, that influences policy; nonetheless, most studies rely on data about resources, regardless of deployment. This thesis is supported by their data: measures of resources alone impacted policy 45% of the time, but resources and activities together impacted policy 65% of the time.

Finally, they suggest that the range of extant lobbying research is very narrow— it covers only a small number of issues and countries, and may simply not be representative enough to draw meaningful general conclusions.

Lowery (2013)

Lowery addresses himself to the findings of three previous literature reviews—Smith (1995), Baumgartner and Leech (1998) and Burstein and Linton (2002)—which held that evidence about the effectiveness of lobbying is inconclusive or inconsistent. Lowery starts from the premise that null findings are common—assuming that, due to the file drawer effect, significant findings are even rarer than they at first appear—and offers some explanations for why that might be so.

Lowery suggests several possible explanations for null findings:

De Figueiredo and Richter (2013)

De Figueiredo and Richter list "empirical regularities" in the study of lobbying. The uncontroversial topline summary is that lobbying is pervasive, that lobbying increases when issues are more salient or when stakes are high for an organized interest, and that both expertise and connections seem to matter for lobbyist effectiveness, but the magnitude of these effects is unclear.

They also list three main challenges in the empirical study of lobbying:

In general, the authors find that lobbying "can be valuable in some contexts" but that its success is heavily dependent on timing, targets, and tactics.

Studies that were instrumental in my conclusions

You can read summaries and key findings for 25 of the studies I reviewed in a Google sheet here. What follows are brief summaries of the studies that I found most useful and convincing.

Caldeira and Wright (1998)

Caldeira and Wright model individual Senate votes in the Bork, Souter, and Thomas Supreme Court nominations as a function of lobbying. They use an instrumental variables approach in which votes are modeled (in the second stage) as a function of partisan affiliation, ideological orientation, constituency support for the nominee, and a lobbying index constructed (in the first stage) as a function of party, ideology, constituency, campaign contribution, and organizational strength (e.g. the number of interest groups lobbying for and against the nominee) that contribute independently to the value of the lobbying index. They find a significant relationship between lobbying and Bork vote; the authors find counterfactually that if the opposition lobbying effort had been 10% smaller, the opposition to Bork would have decreased by two votes. In the case of the Thomas nomination, the authors find that "a 25% reduction in opposition lobbying effort would have garnered an additional 33 votes for Thomas."

De Figueiredo and Silverman (2006)

De Figueiredo and Silverman model academic earmarks for universities against lobbying expenditure by those institutions in an instrumental variables setting. They find a marginal $5 return to an additional dollar of lobbying expenditure. However, this relationship only holds for universities with representation on Congressional appropriation committees.

Baumgartner et al (2009)

This book, Lobbying and Policy Change: Who Wins, Who Loses, and WHY is a contemporary classic. Baumgartner et al take a random sample of issues and conduct a correlational analysis, following the issues over several years and collecting data primarily via interviews with lobbyists. Though this process has its weaknesses, the authors present a number of conclusions that I view as broadly convincing correlational evidence when viewed in light of the full scope of related research.

Key findings:

Montes-Rojas (2013)

Montes-Rojas models foreign aid disbursed by the U.S. as a function of lobbying and finds that a 1% increase in lobbying expenditures in a given year is associated with a 0.04% increase in foreign aid the following year, and up to 0.075% in the long run. I want to note that this effect is huge, given the differences in magnitude between lobbying expenditure and foreign aid. Foreign lobbying expenditures total about half a billion dollars, but the foreign aid budget is around $40 billion. Montes-Rojas uses an IV strategy in which lagged values of the parameter of interest are used as instrumental variables; this is not as crazy as it sounds, and I recommend you read the paper, which leans heavily on the work of David Roodman.

Goldstein and You (2017)

In a pretty novel approach, these authors examine the impact of lobbying expenditure by U.S. cities with populations over 25,000 on disbursements via earmarks and the American Recovery and Reinvestment Act. They also use a novel instrument: the existence of a direct flight between a given city and Washington, D.C, which the authors argue is a source of exogenous variation in lobbying expenditure. They present several convincing robustness checks on this instrument in their Appendix, but an unaddressed area of uncertainty is whether the existence of a direct flight can cause legislators to be more familiar with a city and therefore more likely to disburse funds to it. If so, this could compromise the instrument.

Goldstein and You find that a 10% increase in lobbying expenditures is associated with increases in earmarks and ARRA grants of 10% and 4%, respectively. As in the case of Montes-Rojas (2013), this framing can obscure the magnitude of the impact. Cities spent about $20k on lobbying on average, and received an average of around $1m and $130m in earmarks and ARRA funds respectively, so the coefficients suggest marginal returns of 50x and 2600x, respectively.

Ludema, Mayda and Mishra (2018)

Ludema, Mayda, and Mishra examine tariff suspension bills, of which 1400 were introduced in Congress between 1999 and 2006. They model passage of these bills as a function of lobbying expenditure for the Congresses spanning that time period and use a set of instruments that they argue (convincingly) are not readily available to legislators and are therefore unlikely to influence tariff suspensions. They find that increases in effective lobbying expenditure (expenditures relative to a minimum baseline) in opposition to suspensions are significantly associated with suspension, expenditure in support is likewise associated with no suspension (with a smaller effect size); likewise number of opponents.

Findings

Does lobbying work?

A very heavily qualified yes. I think that there are enough good-quality quantitative studies, with enough convincing theoretical backing, to conclude that informational lobbying can effectively shift policy. There are a number of reasonable concerns with the state of the research: much of the existing work is not causal in nature, and I found almost no published papers on this topic with null results, suggesting that a bit of the file drawer effect may be at play here. For this reason, it's not advisable to attempt to extract any quantitative estimate of the effectiveness of lobbying from the published work.

Two separate factors give me some confidence in this conclusion. The first is the relatively high quality of many of the quasi-experimental studies and the evidence these higher quality studies represent in aggregate. The second, perhaps more debatable, factor is my sense that some of the repeating patterns in correlational and observational work on this topic are best explained by a causal story rather than omitted variables. These patterns include the observed association between the net gap in lobbying (supporting minus opposing) and policy success, consistently higher coefficients on negative lobbying relative to positive lobbying, and the broad agreement on conditions for lobbying success that I discuss below.

When does lobbying work?

Previous literature reviews have identified a number of stylized facts about the determinants of lobbying success that seem to have a strong empirical basis. Please note that the scope of my review centered on legislative lobbying at the national level, and that the following areas of agreement should be considered to be restricted to the legislative domain.

The first area of broad agreement is that lobbying success seems to be more likely on issues of low salience to the electorate, issues with low public visibility, or issues about which the public is largely ignorant. This is a theoretical prediction made by Arnold (1990) and Denzau and Munger (1986) and observed empirically by e.g. Evans (1986). Literature reviews conducted by de Figueiredo and Richter (2013) and Potters and Sloof (1996) consider it well-established.

The second area of broad agreement is that, all other things being equal, lobbying against policies seems to be more effective than lobbying for policies (see e.g. McKay 2011, Caldeira and Wright 1998, Ludema, Mayda and Mishra 2018); in other words, lobbyists for the status quo are likelier to succeed.

The third area of agreement is that having high-level allies, either in committee leadership, the executive branch, or among Congressional staffers, is consistently associated with policy success (see e.g. Baumgartner et al 2009, de Figueiredo and Silverman 2006, Evans 1996).

Some other areas for which I found some empirical support but no broad consensus are as follows:

Case Studies

I think it's worth listing a few instances when minor interventions have significantly impacted policy on a large scale. I'm not picky about whether these examples would have been quantifiable by any of the methods reviewed here; I just want to give an idea of the settings in which lobbying (external or internal) can influence policy (in the bureaucracy or the legislature) and some sense of the magnitude of its potential effects. These cases demonstrate instances in which policy was meaningfully impacted simply by communication, even though they do not all resemble a typical picture of lobbying.

"Effective lobbying"

As observed by many of the papers referenced here, most issues face no lobbying. If the counteractive lobbying theory is true, then the simple act of introducing lobbying on one of these issues can stimulate opposition; if the opposition is from an opponent willing to "outbid" on policy, then they are likely to win, and lobbying can fail, or fail to be cost-effective. Yet in some cases there simply is no organized opposition, and a relatively small investment can meaningfully alter wording, put a policy on the agenda, or cause a bill to become law.

I want to propose something along the lines of "effective lobbying": a rigorous approach to institutional-level change, starting with the legislature, that would take a portfolio approach to policy advocacy.

Although my review has focused on lobbying Congress, any approach to effective lobbying would have to operate with an eye to all possible venues: state and local governments, regulatory agencies, and the executive branch.

How would this work?

1) Identify a set of high-impact policies that meet the following criteria:

2) Identify the policies least likely to face organized opposition (e.g. from entrenched and well-resourced interests inside or outside the government, or from the public)

3) Identify what avenues of lobbying are most appropriate for enabling the policy goal

4) Identify the proximate course of action that would be required to increase the likelihood of each such policy change in the near term:

5) Fund research necessary to maximize the effectiveness of lobbying for each policy

6) For each policy in the portfolio, fund lobbying and advocacy informed by the research produced in (4).

7) Measure impact

The bulk of the work here needs to happen in (1) and in (4). What would make this approach "effective" is the rigorous restriction of the policy portfolio to only potentially very high-value policies, and the deployment of a systematic approach to tracking and facilitating policy success.

Does effective lobbying already exist?

Traditional lobbyists are hired by clients, either for- or non-profit, to advocate for specific policies. Once hired, their success or failure within the specified domain is only partially under their control. As the research detailed here has found, policy success or failure is determined mostly exogenously, and lobbying success is heavily context-dependent. The approach I've suggested is focused on finding the overlap between policies that have high social value and those that face favorable contexts for lobbying success. So the first difference is in issue selection: traditional lobbyists advocate as they're hired to do, whether or not they're likely to be successful, but favorable context is a necessary condition (though not a sufficient one) for effective lobbying.

There's still the question of the degree to which lobbyists working today actively seek to maximize their efficiency, and the degree to which they make use of the mountain of data that is presently available to assist in this end. To conclusively answer these questions would require a whole separate literature review, but I did review some existing studies that paint a broad picture. Hojnacki and Kimball (1998) find evidence that interest groups take into account the degree of support they believe they have in a legislator's district when deciding whether to lobby her and that interest groups do target influential committee members. Victor (2007) reviews the literature and finds that (a) interest group perceptions of legislator knowledge about a topic impact their choice of whether to lobby directly or indirectly, that (b) interest groups consider how the public views an issue before starting to lobby, that (c) lobbyists expend more resources when there is less consensus on a topic, and that (d) groups lobby more when they perceive that there is a procedurally high likelihood of success.

So there is evidence that lobbyists are aware of the factors that seem to enhance success. The degree to which they take a systematic approach is unclear; several of the papers I read for this review hinted at a belief in considerable heterogeneity in effectiveness across lobbyists. If this heterogeneity is significant, then there is probably a lot of value to be gained by identifying, engaging, and facilitating the work of only the most skilled lobbyists.

Foundations and social change organizations already engage in this type of activity, funding advocacy across a wide range of topics. This is very close to what I am proposing; the only difference is that I am suggesting a simultaneously more diversified and more targeted approach. Rather than plowing millions into advocacy groups that fight for a broad roster of changes on a small number of hotly contested issue areas (e.g. climate change), my proposal is to invest smaller amounts on targeted advocacy for specific, high-expectation policies across a wide range of possible focus areas.

Some ideas

Just to give a flavor of some of the potentially high-impact (in expectation) policies that one might want to lobby for. Please note that some of these are likely to face significant opposition, so they're not ideal candidates for effective lobbying, but they're not ipso facto bad candidates (the expected value of avoiding nuclear war, for example, is very high).

Cost-effectiveness analysis

I conducted a limited cost-effectiveness analysis via Monte Carlo simulation by "backing out" distribution parameters from (in my view, conservative) 95% confidence intervals. Here are my assumptions:

This results in the following setup:

The punchline here is that the cost per DALY of this approach lies with 95% confidence between $-157,632 and $27,974,720, with a median value of $879, and with 70% confidence between $0 and $247,117. There's also a 20% chance that the cost per DALY lies in the "sweet spot" of between $0 and $50. So the median value is semi-promising, but there's a lot of uncertainty and there's a 10% chance effective lobbyists will pay as much as $100,000 to make people's lives one year shorter. However, it's worth noting again that I tried to be both very conservative and very uncertain here, and that further rationalization of this setup could result in more precise (and perhaps more optimistic) results.

Sensitivity

Sensitivity analysis for lobbying CE

In addition to the basic cost-effectiveness calculation outlined above, I present in this graph four variations with different parametrizations. The initial calculation is the red distribution (labeled "Original"). In Variations 1 through 4, I adjust only one parameter relative to the original.

The key takeaway of this analysis is that there's an important tradeoff in the way I've set this up: lower median values (and lower, narrow ranges) for positive costs per DALY come at the price of a higher probability of a negative cost per DALY. The interpretation of a negative cost per DALY would be, as I mentioned above, that substantial expenditures are made lobbying for a policy that ends up having a significant negative impact.

This is obviously a function of the binomial term in the setup described above. I think this term is defensible: for any policy that could plausibly affect large numbers of people in a significant way, there's a small but nontrivial possibility of severe, negative, unintended consequences.

One of the counterintuitive consequences of this setup is that reducing the likelihood of harm increases the price per DALY: that's because reducing harm doesn't stop us from spending: we get positive value that otherwise would be missed, but we still end up spending a lot of money on policies at the low end of the impact distribution. So (with this setup, at least) we can't get a better cost distribution just by reducing harm.

So there's two main preconditions to getting a good cost/DALY in lobbying-based policy change. The first is to advocate policies, as I indicated above, that have some probability of producing very large-scale benefits; the second is (obviously) to spend less (see Variation 3).

I think this suggests a couple things. First, the bundle of lobbied policies should include as many genuinely longtermist policies, or policies addressing global catastrophic risks, in order to shift as much mass as possible onto the right side of the impact distribution with as little risk of catastrophic error as possible. Second, if you accept the premise that accidental error in the form of a "sign reversal" is a plausible model, then reducing the error rate becomes proportionally more important as policies that affect larger numbers of people are advocated. One possible avenue here is to preference policies that enhance decision-making in the long term, that improve the functioning of institutions, over policies that encourage near-term changes to government decisions on specific topics.

Appendix

How does federal legislation work in the U.S.?

Lobbyists interested in advancing a policy proposal must first find a bill sponsor. After a bill is introduced, it is considered by the standing committee that has jurisdiction over the bill, which depends on its content. Most bills fail at this stage. Committee chairmen have enormous power, and (as things currently stand) they can decide unilaterally whether or not to bring a bill to a subcommittee, which would hold a public hearing on the bill's content. On some occasions, these public hearings are held in front of the entire committee.

After a public hearing, the bill moves up to the committee level, where it is subject to "markup": amendments, rewrites, and votes that can make or break the bill. The Congressional leadership then decides whether or not to schedule a floor vote on the bill. At this point, the processes in the Senate and House diverge.

In the Senate, the Senate Majority Leader can immediately schedule a vote—or refuse to schedule one. Once on the floor, amendments can be offered indefinitely until debate is cut off by a cloture vote (sixty senators).

In the House, if a majority of the committee approves the bill, it is sent to the House Rules Committee, which receives amendments from members and determines which amendments will be considered on the floor. For any particular bill, the Rules committee can institute an "open rule," which allows consideration of all amendments, a "closed rule," which allows none, or a rule defining specifically which amendments can be considered. On the floor, House members must abide by the rules set by the Rules Committee.

Differences between House and Senate versions of the same bill are reconciled, and then final, identical versions of the bill are approved or rejected by floor votes in each chamber.

Lobbyists can attempt to intervene at any stage during this process: they can attempt to modify the wording of a measure, to encourage votes at the subcommittee, committee, or floor level, to persuade committee or Congressional leadership to bring a bill to a vote, or to influence the way in which House and Senate versions of the same measure are reconciled.

One of the primary ways in which corporate lobbyists can impact their clients' earnings is via earmarks—specific appropriations within a bill that define precisely where and how some segment of the appropriated funds are to be spent: this is what many empirical researchers are considering when they attempt to determine the effects of lobbying expenditure on firm financial performance.

For some classes of issue, significant policy changes are possible via lobbying at other levels and branches of government. Policies of federal agencies, for example, can have a significant impact: think here of rulemaking by the FDA, EPA, and other regulatory agencies. Considerable lobbying takes place at the state level as well, and can indirectly influence federal policy: in major states such as California and New York, state regulations can be a blueprint for federal ones.

Code for future in-depth analyses

As part of my research, and to support the derivation of parameters for the above cost-effectiveness model, I went through the lobbying disclosure data in detail and explored a bit. I discovered that it's hard to get your hands on it in a useful format, so I wrote some code to convert it from XML to tabular, as well as to produce some hopefully useful summary analysis like those below. That code is available publicly on my GitHub. It is clean enough, but is mostly meant to serve as a starting place for others who want to pick up where I have left off here.

The three summary graphics below are extracted from lobbying disclosures for 2018. Each filing contains a number of issue-specific reports, some of which contain references to specific bills and some of which do not. In order to get thes plots below plots, I took reported amounts from quarterly lobbying reports and allocated them across the bills and issues extracted from the text of each report.

Summary statistics on lobbying data

Bibliography, etc.

My bibliography is on Google Sheets, along with a tab providing short summaries, research designs, and key results of about 25 of the empirical studies that I reviewed.

10 comments

Comments sorted by top scores.

comment by jackva · 2020-08-04T21:18:36.246Z · score: 6 (3 votes) · EA(p) · GW(p)

Very impressive and interesting piece, thanks for this! I am a colleague of smclare at Founders Pledge and work a lot on modeling policy advocacy across charities. Would be great to have a chat.

It's a great and very useful summary of the literature, hugely valuable.

That being said, I am less convinced by the approach taken in the cost-effectiveness model which seems to somewhat contradict the prior analysis which stresses (a) contextuality and (b) strategic choice. As far as I can tell, the modeling tries to be entirely general (contradicting a)) and assumes (b) independence between variables that are very likely related (such as spending and success probability). Please let me know if I completely misunderstand what you are doing!

My sense from doing this kind of work in charity evaluation is that we would want to move towards a "suite" of models for different situations -- e.g. (i) pushing genuinely new ideas, (ii) opportunistically exploiting policy windows (such as stimulus season), (iii) pushing high-impact low probability policies, (iv) averting policy rollbacks etc. and that the cost effectiveness for these kinds of things will be very different and essentially unrelated to a generic estimate such as the one featured at the end of your piece.

To me it seems that this kind of classifying into the sub-models will be the only way to realistically bound parameters of interest and also that the dynamics underlying are sufficiently different that they should probably have their own models.

Some minor points/comments: Something I found missing is a bit are standard explanations for why there is so little money in politics, namely (1)because dominant firms in districts have lots of indirect power via employees voting so that they do not need to spend money. (2) It also seems that the report somewhat under-emphasizes the idea of lobbying equilibria where marginal increases by one side would be quickly countered, which would make it look like additional money could be effective when in effect it is not.

I also think that the conclusion which, I believe, mostly draws from Baumgaertner " (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents." is quite surprising and I would be curious to find out why you think that / in how far you trust that conclusion.

comment by Matt_Lerner (mattlerner) · 2020-08-08T02:59:51.920Z · score: 1 (1 votes) · EA(p) · GW(p)

Hello and thank you for your response!

Your criticism of the cost-effectiveness model is fair. Thematically, I guess it does contradict the spirit of my prior analysis in that it avoids the concerns of strategic choice. I was actively trying to be as general as possible, and actively trying to err on the side of greater uncertainty by not including any assumptions about correlatedness, though it occurs to me now that making such an assumption (e.g. a correlation between expenditure and likelihood of success) would actually have increased the variance of the final estimate, which would have been more in line with my goals. When I have time, I may comment here with an updated CEA.

I also agree that the only useful way to do this analysis is, as you've described, with a suite of models for different scenarios. I don't have a defense for not having done this beyond my own capacity constraints, though I hope it's more useful to have included the flawed model than not to have one at all (what do you think?).

I also think that the conclusion which, I believe, mostly draws from Baumgaertner " (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents." is quite surprising and I would be curious to find out why you think that / in how far you trust that conclusion.


Thanks for this, in particular. I think your surprise stems from a lack of clarity on my part. The reason I have high confidence in this conclusion is that it's a much weaker claim than it might seem. It does stem primarily from Baumgartner et al and from Burstein and Linton (2002). The claim here is that resource-rich groups are no more or less likely to get what they want--holding all else equal, including absolute expenditure and the spending differential between groups and their opponents.

There are three types of claim that are closely related:
1) Groups that spend more relative to their opposition on a given policy are likelier to win
2) Groups that spend more in absolute terms are likelier to win
3) Groups that have more money to spend are likelier to win

So I found fairly consistent evidence for (1), some evidence for (2), and no real evidence for (3). It's not obvious to me that (3) should be the case irrespective of (1): why would resource-rich groups succeed in lobbying if they deploy those resources poorly? It seems like the success of resource-rich groups is dependent upon (1), and that (3) should not be true when in isolation, unmediated by (1). Although Baumgartner et al conduct an observational study, the size of their (to me, convincingly representative) sample to me suggests that if such an effect exists, it should be observable as a correlation in their analysis. The association they observe is pretty small.

I have to say, though, that in writing this comment, my confidence in this conclusion has eased up a bit, so I'm curious to hear your response. I also think that since Baumgartner et al do find a small effect, I probably overstate the case here.

Baumgartner et al offer a theoretical take on this: "...organizations rarely lobby alone. Citizen groups, like others, typically participate in policy debates alongside other actors of many types who share the same goals. For every citizen group opposing an action by a given industrial group, for example, there may also be an ally coming from a competing industry with which the group can join forces" (p.12). So it's important to recognize that the finding here is about individual parties, not "sides" or coalitions advocating a given policy.

Finally, I'm curious to hear your take on the two potential money-in-politics explanations you mentioned. I've never found (1) particularly convincing—it's not clear to me that firms and their employees have the same interests, or that (if they do) the marginal value of regulatory capture isn't still high. But I agree that I underemphasized (2) and think it would be useful to have in this thread the "inside view" on lobbying equilibria from someone who works in the field.

comment by smclare · 2020-07-31T15:49:07.393Z · score: 6 (3 votes) · EA(p) · GW(p)

Wow, this is really fantastic work! Thank you for the effort you put into this. Overall I think this paints a more optimistic picture of lobbying than I would have expected, which I find encouraging.

To follow up on a couple specific points:

(1) Just in terms of my own project planning, do you have an estimate of how long you spent on this? If you had another 40 hours, what uncertainties would you seek to reduce?

(2) Your discussion of Bumgartner et al. (2009) is super interesting. You write "Policy change happens over a long time frame." I wonder if you could expand on this briefly. Do you mean that it takes a lot of lobbying over years before a policy change happens, or do you mean that meaningful policy change happens through incremental policy changes over time?

(3) Your finding that lobbying which protects the status quo is much more likely to be effective seems particularly actionable. I mean, once put into words it seems obvious, but it's a point I hadn't thought about before. I notice, though, that your list of ideas seems to consist of positive changes rather than status quo protection. I wonder if it would be worth brainstorming a list of good status quo issues that might be under threat. Protecting these would be less exciting than big changes, but for exactly the reasons you outline here more likely to work!

(4) I'm interested in thinking a bit more about uncertainty about policy implementation. This is something that we're currently grappling with in our models of policy change where I work (Founders Pledge). On the one hand, the Tullock Paradox suggests that we should expect lobbying to be extremely difficult (otherwise everyone would do a lot more of it). On the other hand, we've noticed that very good policy advocates seem to quite regularly affect meaningful policy changes (for example, it seems like the Clean Air Task Force regularly succeeds in their work).

In your model you write that "the change in probability of policy implementation lies with 95% confidence between 0 and 5%, and is distributed normally." I'm not sure about this, but I imagine the distribution of "chance of affecting policy success" over all the possible policies we could work on is much flatter than this. Or perhaps it's bimodal: there are some issues on which it is near impossible to make progress and some issues where we could definitely get policies implemented if we spent a certain amount of money in the right way.

Perhaps we want to start with a low prior chance of policy success, and then update way up or down based on which policy we're working on. Do you think we'd be able to identify highly-likely policies in practice?

(5) I found this post super helpful, but overall I think I'm still quite puzzled by the Tullock Paradox. If anything I'm more confused now, given that this post made me update in favour of policy advocacy. I think perhaps something that's missing here is a discussion of incentives within the civil service or bureaucracy. A policy proposal like taking ICBMs off hair-trigger alert just seems so obvious, so good, and so easy that I think there must be some illegible institutional factors within the decision-making structure stopping it from happening. I don't blame you for excluding this issue considering the size of this post and the amount of research you've already done, but it seems worth flagging!

Thanks again for a great post! I'm really excited about more work in this vein.

comment by Matt_Lerner (mattlerner) · 2020-07-31T19:27:54.309Z · score: 4 (2 votes) · EA(p) · GW(p)

Thanks for your response!

(1) I spent something like 100 hours on this over the course of several months. I think I could have cut this by something like 30-40% if I'd been a little bit more attentive to the scope of the research. I decided on the scope (assessing the effectiveness of national-level legislative lobbying in the U.S.) at the beginning of the project, but I repeatedly wound up off track, pursuing lines of research outside of what I'd decided to focus on. I also spent a good chunk of time on the GitHub repo with the setup for analyzing lobbying data, which wasn't directly related to the lit review but which I felt served the goal of presenting this as a foundation for further research.

If I had 40 more hours, I'd intentionally pursue an expanded scope. In particular, I'd want to fully review the research on lobbying of (a) regulatory agencies and (b) state and local governments. I explicitly excluded studies along those lines, some of which were very interesting.

(2) Thanks for asking for clarification on this. Baumgartner et al mean that it takes a long time for policy change to be observed on any given issue. After starting to pursue a policy goal, lobbyists are more likely to see success after four years than after two.

Baumgartner et al include a chapter that is mostly critical of the incrementalist idea of policy change, which they trace to Charles Lindblom's 1959 article The Science of "Muddling Through". Incrementalism is tied to Herbert Simon's idea of "bounded rationality." Broadly, the incrementalist idea is that policymakers face a broad universe of possible policy options, and in order to reduce the landscape to a manageable set, they choose from only the most available options, e.g. those closest to the status quo: "incremental" changes.

Frank Baumgartner, with Bryan Jones, is now well-known for their theory of "punctuated equilibrium." This is a partial alternative to incrementalism which uses the analogy of friction to understand policy change. Basically: the pressure builds on an issue over a period of time, during which no change occurs. After the pressure is overwhelming, policy shifts in a major way.

I say that punctuated equilibrium is a "partial" alternative because Baumgartner and Jones actually collected data that seems to demonstrate that policy change follows a steeply peeked, fat-tailed distribution. Their overall takeaway is that very small changes are overwhelmingly common, but moderate changes are relatively uncommon, and very large changes are surprisingly common. To come back to your question, Baumgartner et al might say that although most policy change is incremental—like year-to-year changes in agency budgets—meaningful policy change happens in a big way, all of a sudden.

(3) I agree with you. I think some of my suggested policies are not likely to be those most effectively advocated for, and I included them just to give a flavor of the types of things we might care about lobbying for. Coming up with more practicable ideas is, I think, a much bigger, much longer-term project.

I also think that although lobbying for the status quo is more effective all other things being equal, it may not be the best use of EA resources to focus exclusively on that side of things. That's because (per the counteractive lobbying theory) on many issues there is are latent interests that will arise to lobby against harmful proposals. It's hard to identify beforehand which proposals will stimulate this opposition, so there's a lot of prior uncertainty as to whether funding opposition to policy change is marginally useful in expectation.

(4) There are a lot of takes on the Tullock paradox, but I'll present two broad possible explanations.

  • Explanation A: Lobbying is basically ineffective, and the reason we don't see more lobbying is that most organizations recognize its ineffectiveness.
  • Explanation B: Lobbying is highly effective, and the reason we don't see more lobbying is that relatively small expenditures can exert enormous amounts of leverage.

Given the evidence here, I'm starting to be a lot more inclined toward Explanation B. I think it's demonstrably not the case, as you have noted with respect to the Clean Air Task Force, that organizations that lobby are wasting their money. For both altruistic and self-interested interest groups, the rewards to be captured are very large, and they make it worth the risk of wasting money. Alexander, Scholz, and Mazza (2009), for example, find a 22,000% return on investment.

If Explanation B holds, then the question is really just why the market for policy isn't efficient. Why hasn't the price of lobbying been bid up to the value of the rewards to be captured? I think it seems likely that this is down to multiple layers of information asymmetry (between legislators and their staffs, between these staffers and lobbyists, between lobbyists and their clients, etc.), which create multiple layers of uncertainty and drive the expected value of lobbying down from the standpoint of those in a position to purchase it.

I agree with you that a normal distribution is probably not the best choice to model the expected incremental change in probability. I felt like, given my CI for this figure and my sense that values closer to 0% and values closer to 5% were each less likely than values in the middle of that range, this served my purposes here - but please take my code and modify as you see fit!

Perhaps we want to start with a low prior chance of policy success, and then update way up or down based on which policy we're working on. Do you think we'd be able to identify highly-likely policies in practice?

I don't know. I think it's worth investigating. It seems like, given an already-existing basket of policies we'd be interested in advocating for, we can make lobbying more cost-effective just by allocating more resources to (e.g.) issues that are less salient to the public.

I have a sense that lobbyists, do, in fact, do something like what you're describing, and that this is part of the resolution to the Tullock paradox. Money spent on lobbying is not spent all at once: lobbyists can make an effort, check their results, report to their clients, and identify whether or not they're likely to meet with success in continued expenditure. If lobbying expenditure on a given topic seems unlikely to make a difference, then it can just stop. I wasn't able to find anything on how this process actually works, so the next step in this research is to actually talk to some lobbyists.

(5)

I think perhaps something that's missing here is a discussion of incentives within the civil service or bureaucracy

I agree with this too. I'd love for an EA with a public choice background to tackle this topic. I didn't consider it as part of my scope, but I do want to note something:

A policy proposal like taking ICBMs off hair-trigger alert just seems so obvious, so good, and so easy that I think there must be some illegible institutional factors within the decision-making structure stopping it from happening.

I think this is probably true in many if not most cases of yet-to-be-implemented policy changes that are obvious, good, and easy. It is probably true in this case. But I want to warn against concluding that, because some obvious, good, and easy policy change has not been implemented, that means that there is some illegible institutional factor that is stopping it from happening. It could just be that no one has been pushing for it. In EA terms, it's an important and tractable policy change that's neglected by the policy community. Given what I know about the policy community, it's not at all difficult for me to imagine that such policies exist.

comment by smclare · 2020-08-04T09:40:16.392Z · score: 1 (1 votes) · EA(p) · GW(p)

Thanks for this! Something that came to my mind as I was reading this was that it might be time for an update of CEA's list of good policy ideas that won't happen (yet) [EA · GW].

You wrote that "It seems like, given an already-existing basket of policies we'd be interested in advocating for, we can make lobbying more cost-effective just by allocating more resources to (e.g.) issues that are less salient to the public." This made me think it might be useful be to make a list of EA-relevant policy ideas and start organizing them into a Charity Entrepreneurship-style spreadsheet. Something I'll keep musing on!

I'm also curious about what motivated you to take on this project, and what you're planning to work on next?

comment by Matt_Lerner (mattlerner) · 2020-08-08T04:00:06.705Z · score: 1 (1 votes) · EA(p) · GW(p)

I like this spreadsheet idea and think I may kick it off (if you haven't already done so!)

I took the project on because I got interested in this topic, went looking for this, couldn't find it, and decided to make it so that it might be useful to others. I wasn't feeling very useful in my day job, so it was easy to stay motivated to spend time on this for a while. I tend to be most interested in generalizable or flexible approaches to improving welfare across different domains, and this seemed like it might be one of those.

Some areas I'm thinking about exploring. These are pretty rough thoughts:

  • Some more exploration of strategies for ameliorating child abuse in light of the well-known ACES Study. GiveWell and RandomEA [EA · GW] have both explored Nurse-Family Partnerships. This problem is just so huge in terms of people affected (and in terms of second-order effects) that I think it's worth exploring a lot more. I'm particularly interested in focusing on child sexual abuse in particular.
  • Aggregating potentially cost-effective avenues to improve institutional performance. I'm curious about thinking at a higher level of abstraction than institutional decision-making. It seems worthwhile to put together the existing cross-disciplinary evidence on the question: what steps outside of those explicitly focusing on rationality and decision-making can companies/nonprofits/government agencies take to increase the probability that they make good decisions? A good example of one such step is in the apparent evidence that intellectually diverse teams make better decisions.
  • Long-term cost-effectiveness of stress reduction for pregnant women (with potential effects of infant mortality, maternal health, and long-term outcomes like brain development and violence).
  • Review of recent innovations that seem to like they might have potential for expediting scientific progress (like grant lotteries)
comment by lukeprog · 2020-08-03T12:10:25.052Z · score: 3 (2 votes) · EA(p) · GW(p)

Thanks for this!

FWIW, I'd love to see a follow-up review on lobbying Executive Branch agencies. They're less powerful than Congress, but often more influenceable as well, and can sometimes be the most relevant target of lobbying if you're aiming for a very specific goal (that is too "in the weeds" to be addressed directly in legislation). I found Godwin et al. (2012) helpful here, but I haven't read much else. Interestingly, Godwin et al. find that some of the conclusions from Baumgartner et al. (2009) about Congressional lobbying don't hold for agency lobbying.

comment by Jamie_Harris · 2020-08-09T17:10:49.818Z · score: 1 (1 votes) · EA(p) · GW(p)

Thank you for this review! Ever since I did a literature review of mostly political science literature on "Is the US Supreme Court a Driver of Social Change or Driven by it?" I've been interested in comparable reviews of legislative tactics.

I especially appreciate the summary of the empirical research. Some of those findings seem really impressive to me and that has increased my optimism about lobbying. I look forward to reading through your spreadsheet with summaries of other papers.

(80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents.

I was very surprised by this claim... it was part of what spurred me to read the piece immediately rather than save it to my reading list. But I don't recall seeing evidence for the claim in the post. What has encouraged this high confidence in this claim (which seems very counterintuitive to me)? Apologies if I just missed it.

Yet in some cases there simply is no organized opposition, and a relatively small investment can meaningfully alter wording, put a policy on the agenda, or cause a bill to become law.

The suggestion for the spending strategy in the "Effective lobbying" section seems to rest on this claim. But again, I don't recall seeing much empirical support in the review; does this rest on the theoretical discussions that you summarise? Actually, Baumgartner et al. provide some contrary evidence to this hypothesis:

"The table [table 3.1, page 58] shows that a surprisingly large number of issues (seventeen cases) consist of a single side attempting to achieve a goal to which no one objects or in response to which no one bothers to mobilize. Ironically, the lack of countermobilization is a good predictor of failure. Many of these reflect efforts to put an issue on the agenda, but these efforts are either too early in the process for anyone yet to have reacted or they are clearly not moving so others have not gotten involved in the issue."

They discuss this a little more on the following pages. They note that 17 cases had only one side -- 15 of these were 1 opposing the status quo, none defending it. Unfortunately, from a quick look back, I can't see the proportion of these 15 cases that did result in a policy change.

comment by Matt_Lerner (mattlerner) · 2020-08-09T18:06:44.756Z · score: 1 (1 votes) · EA(p) · GW(p)

Thanks for your response!

With respect to your first point, I'm considering striking this conclusion upon reflection - see my discussion with @jackva elsewhere in this thread. In any case, my confidence level here is certainly too high given the evidence, and I really appreciate your close attention to this.

With respect to your second point, I don't mean to imply that the lack of organized opposition is the only thing that justifies lobbying expenditure, and think my wording is sloppy here as well. I used "lack of an organized opposition" to refer broadly to oppositions that are simply doing less of the (ostensibly) effective things — lower "organizational strength" as in Caldeira and Wright (1998), number of groups, as in Wright (1990), or simply lower relative expenditure, as in Ludema, Mayda, and Mishra (2018).

The evidence in Baumgartner et al that you reference about the apparent association between lack of countermobilization and success is also related to @jackva's concern about my underemphasis on potential lobbying equilibria here. On the one hand, I think this is clearly evidence in favor of the hypothesis that there is some efficiency in the market for lobbying- perhaps most lobbyists have a good idea of which efforts succeed, and don't bother to countermobilize against less sophisticated opposition. On the other hand, lobbying is a sequential game, and, since the base rate for policy enactment is so low to start with, it makes sense that opposition wouldn't appear until there's a more significant threat.

EDIT: I've actually struck the first bit, with a note. I wanted to add one more thing, which is that I don't know how much you've adjusted your prior on lobbying, but I wouldn't say this has made me "optimistic" about lobbying. The core thing I've come away with is that lobbying for policy change is extraordinarily unlikely to succeed, but that marginal changes to increase the probability of success are (1) plausible, based on the research and (2) potentially cost-effective, based on the high value of some policies.

comment by brb243 · 2020-07-31T17:39:46.817Z · score: 1 (1 votes) · EA(p) · GW(p)

Hello. I studied lobbying in DC during my MA in International Trade and Economic Diplomacy. I also tried to estimate the cost-effectiveness of lobbying per QALY - and came up with $280 per quality life (or about $4.7/QALY) (p. 3) or the first draft [EA · GW] on the EA Forum.

I think that EA Brazil and some others are working on starting an EA lobbying group. I dm'd you regarding this.