Bill prognosis has gotten an upgrade. A few weeks ago I wrote about a new addition to GovTrack, the bill prognosis, and because it proved to be useful I expanded on it to provide a more detailed and numerical assessment of the future of each bill. For an example, check out H.R. 4323, which we’re currently listing as having a 2% chance of being enacted.
If you missed the first post, here’s an overview: Only a small fraction of the bills and resolutions introduced will ever be voted on. How do you know which bills to pay attention to? We can’t predict the future, but we can highlight factors that favor a bill’s progress and use statistics from past years to assess the likelihood that a bill will be enacted.
In the first version of bill prognosis, we listed whether the bill’s sponsor is the chair of the committee considering the bill and whether any cosponsors of the bill are on the committee. Now, we use factors like these and more to compute an actual probability that the bill will be enacted (or for resolutions that they will be passed).
For H.R. 4323: Consumer Mortgage Choice Act, we are currently listing a 2% chance of the bill being enacted and we show that the factors used in the computation are:
- 3-5 cosponsors serve on a committee to which the bill has been referred.
- There is at least one cosponsor from the majority party and one cosponsor outside of the majority party.
- The sponsor is a member of the majority party.
I unfortunately confused things a bit by running a separate experiment recently. We were listing a probability that each bill would be enacted and were asking whether you thought the number was too low or too high. That wasn’t based on the bill prognosis — I’ll write about where that number was coming from another time. But I saw that GovTrack users were finding it useful, and so I knew enhancing the bill prognosis with a rigorously computed probability would be a good idea.
For the data wonks out there, the new prognosis is based on a logistic regression model. The model predicts a bill’s success based on the following binary factors:
- the title of the bill (such as if it is a bill to name a post office)
- whether the sponsor is a member of the majority party (in the House or Senate as appropriate)
- whether the sponsor is the chair, ranking member (most senior minority party member), or a majority-party member of a committee that the bill has been referred to
- if any cosponsor is the chair or ranking member of a committee the bill has been referred to
- if there are 3-5 cosponsors of the bill serving on a committee the bill has been referred to
- if the bill has a cosponsor from both parties
- if the bill’s sponsor is in the majority party and at least 1/3rd of the cosponsors are from the minority party
- if the bill was a reintroduction of a bill from the previous Congress (same sponsor and title, ignoring any year found in the title) and, separately, if the previous bill had been reported by committee (suggested by Tom Lee and Daniel Schuman shortly after this post was first published)
- if the bill’s sponsor or cosponsors have a high leadership score based on GovTrack’s analysis of bill sponsorship (based on a suggestion from Mackenzie Morgan shortly after this post was first published)
- and if any of these factors are true of a companion (identical) bill introduced in the other house of Congress
Success is for bills if they are enacted and for resolutions if they successfully reach the end of their life cycle (simple resolutions passed, concurrent resolutions passed by both chambers, joint resolutions enacted). A separate model was constructed for each of the eight bill types (H.R., S., S.Res., etc.). Additional models were created for bills that were at least reported by committee, and the prognosis for such bills is based on those separate models. The models were trained on bills and resolutions from 2009-2010 (the previous session of Congress).
Here are some results of the model. Of Senate bills in 2009-2010, only 2.8% were enacted. The regression coefficients for the model for Senate bills are listed below, in order from most indicative of a successful bill to least indicative. Also listed with each is the percentage of bills with that attribute that were enacted, to compare against the 2.8%.
- 2.7 — the bill’s title starts with “A bill to designate the” (usually naming a post office) (24% enacted)
- 2.1 — a companion bill has a cosponsor that is the ranking member of a relevant committee (22% enacted)
- 2.0 — a companion bill’s sponsor is the chair of a relevant committee (29% enacted)
- 1.7 — the bill’s title starts with “A bill to authorize” (10% enacted)
- 1.0 — a cosponsor is the ranking member of a relevant committee (13% enacted)
- 0.94 — a cosponsor is the chair of a relevant committee (9.4% enacted)
- 0.76 — there are cosponsors from both parties (5.2% enacted)
- 0.63 — the sponsor is from the majority party and at least 1/3rd of the cosponsors are from the minority party (6.8% enacted)
- 0.47 — the bill was a re-introduction of a bill that was reported by committee in the previous session of Congress (7.6% enacted)
- 0.37 — the sponsor is the chair of a committee to which the bill was referred (7.7% enacted)
- 0.27 — a companion bill has a cosponsor from the majority party and a cosponsor from the minority party (6.3% enacted)
- 0.12 — a companion bill is a reintroduction of a bill that was reported by committee in a previous session of Congress (11% enacted)
- 0.080 — a companion bill has 6 or more cosponsors on a relevant committee (4.9% enacted)
- 0.013 — a cosponsor in the majority party has a high leadership score (7.4% enacted)
- -0.032 — a companion bill’s sponsor is in the majority party (5.4% enacted)
- -0.13 — a companion bill’s cosponsor is the chair of a relevant committee (8% enacted)
- -0.14 — 3-5 cosponsors are members of a relevant committee (4.7% enacted)
- -0.85 — the sponsor is in the minority party (1.2% enacted)
- -1.5 — the bill was a re-introduction of a bill from the previous session of Congress that had no major action (<1% enacted)
- -34 — the bill’s title starts with “A bill to extend the temporary suspension of duty” (0% enacted)
The regression coefficients are not easily interpretable on their own, except that higher numbers mean that they more importantly indicate they help a bill get enacted. The largest negative numbers indicate those bills pretty much never get enacted. Small negative numbers don’t necessarily mean that the factor hurts a bill’s chances
(Before using the regression model I tested that each factor taken independently was statistically significant, but in a way that in retrospect was not a particularly good way to do it. My intuition is that the regression model factors are probably all statistically significant anyway. Based on that initial test of significance I excluded some of these factors from some of the models. For instance, no factors were statistically significant for the model for Senate bills reported by committee, in part because the sample of 523 bills is so much smaller. No regression model is used in that case, and the overall probability of 21% for all such bills is used as the prognosis for all bills in this category. Note how much larger 21% is than the overall success rate for all Senate bills, 2.8%.)
UPDATES: After folks suggested new factors to consider, I re-created the models and the regression coefficients above were updated.