Analyzing Stock Market Activity of US Senators with Python¶
In 2012, a law called ” Stop Trading on Congressional Knowledge (STOCK) Act of 2012″ was passed, which prohibits the use of non-public information for private profit, including insider trading by members of Congress and other government employees. This law however did not completely ban stock, bond, or commodity trading by members of Congress, it simply banned insider trading (which prior to the act, was technically legal for members of Congress, but illegal for the general population in accordance to the Securities Exchange Act of 1934 and Rule 10b-5.)
The STOCK Act also provided guidelines for sale disclosures and requires every Member of Congress to publicly file and disclose any financial transaction of stocks, bond, commodities futures, and other securities within 45 days on their websites, rather than once a year as was required previously.
Due to these disclosures, we have access to data revealing stock trading activity by Members of Congress. Let’s take a dive into some of this data with Python!
Don’t wait, download now and transform your career!Your FREE Guide to Become a Data Scientist
Data Sources¶
There are actually many different ways of acquiring this data, you could go straight to the source here:
https://efdsearch.senate.gov/search/home/
Where you can search for the disclosures for specific Members of Congress. These disclosures can be difficult to navigate, however, the SEC also has their own page for Senate Stock Disclosures here: https://sec.report/Senate-Stock-Disclosures
The SEC also has a page we can scrape for the data. There are also API services like https://www.quiverquant.com/ that create easy to use APIs with robust calls.
Let’s explore a few different ways of grabbing the data:
Scraping the Data from SEC Disclosures¶
import pandas as pd
import requests
url = 'https://sec.report/Senate-Stock-Disclosures'
header = {
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36",
"X-Requested-With": "XMLHttpRequest"
}
r = requests.get(url, headers=header)
dfs = pd.read_html(r.text)
dfs[0]
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
Filed Date ⇩ Transaction Date | Issuer | ReporterOwnership | |
---|---|---|---|
0 | 2022-09-16 2022-08-10 | Parker-Hannifin Corp 701094AQ7 Rate/Coupon: 3…. | Susan M Collins [Collins, Susan M.] |
1 | Purchase 7c29d4eb-ec9a-4a28-b1ba-7067fe97afc3 | $15,001 – $50,000 | — Spouse |
2 | 2022-09-15 2022-08-01 | iShares Core S&P 500 ETF [IVV] | Thomas H Tuberville [Tuberville, Tommy] |
3 | Sale (Full) 7718a62c-639e-4cf9-bcb8-00eb421ac444 | $15,001 – $50,000 | — Self |
4 | 2022-09-15 2022-08-01 | iShares MSCI EAFE Min Vol Factor ETF [EFAV] | Thomas H Tuberville [Tuberville, Tommy] |
… | … | … | … |
527 | Purchase 0f1deb54-a350-4646-9365-899394573269 | $1,001 – $15,000 | — Joint |
528 | 2022-04-08 2022-03-30 | Alphabet Inc. – Class C Capital Stock [GOOG] | Thomas H Tuberville [Tuberville, Tommy] |
529 | Purchase 0f1deb54-a350-4646-9365-899394573269 | $1,001 – $15,000 | — Joint |
530 | 2022-04-08 2022-03-30 | AbbVie Inc. Common Stock [ABBV] | Thomas H Tuberville [Tuberville, Tommy] |
531 | Purchase 0f1deb54-a350-4646-9365-899394573269 | $1,001 – $15,000 | — Joint |
532 rows × 3 columns
From here we could dive deeper into an individual trade or transaction, or filter out by the Reporter or Ownership. Alternatively we could grab the data directly from the Senate Disclosure site: https://efdsearch.senate.gov/search/
We could then look up the latest disclosures by a Senator, for example: https://efdsearch.senate.gov/search/view/ptr/7718a62c-639e-4cf9-bcb8-00eb421ac444/
Data from Senate Disclosures¶
A very quick and easy way to read a single disclosure form is to simply locate the table you are interested in and copy the table with Ctrl+C and simply use Pandas to quickly read in the table as a DataFrame. Let’s explore an example for Tommy Tuberville, who at the time of this writing has the most recent disclosure available, which he released on Sept 15th,.2022. Remember this is just a single quarterly disclosure, you can explore the website for his previous disclosures or other Senator disclosures.
# Copy the Table here with Ctrl+C
url = "https://efdsearch.senate.gov/search/view/ptr/7718a62c-639e-4cf9-bcb8-00eb421ac444/"
# Then just read it with Pandas
df = pd.read_clipboard()
df.head()
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
# | Transaction Date | Owner | Ticker | Asset Name | Asset Type | Type | Amount | Comment | |
---|---|---|---|---|---|---|---|---|---|
0 | 37 | 08/01/2022 | Self | IVV | iShares Core S&P 500 ETF | Stock | Sale (Full) | $15,001 – $50,000 | — |
1 | 36 | 08/01/2022 | Self | EFAV | iShares MSCI EAFE Min Vol Factor ETF | Stock | Sale (Full) | $15,001 – $50,000 | — |
2 | 35 | 08/01/2022 | Self | XLY | SPDR Select Sector Fund – Consumer Discretionary | Stock | Sale (Full) | $15,001 – $50,000 | — |
3 | 34 | 08/01/2022 | Self | VTRS | Viatris Inc. – Common Stock | Stock | Sale (Full) | $1,001 – $15,000 | — |
4 | 33 | 08/01/2022 | Self | ABNFX | The Bond Fd of America Class F-2 Shares | Stock | Purchase | $15,001 – $50,000 | — |
Exploring a Data Disclosure¶
Let’s take a closer look at this disclosure form to get an understanding of Senator Tuberville’s activity from last quarter:
len(df)
53
A total of 53 transactions, but some will be sales and other would be purchases. Let’s get a range of possible sales and purchases. This means we need to clean up the data a little bit from some formatting issues with the copy/paste, since the Amount column isn’t float yet.
df.columns
Index(['#', 'Transaction Date', 'Owner', 'Ticker', 'Asset Name', 'Asset Type', 'Type', 'Amount', 'Comment'], dtype='object')
df['Amount']
0 $15,001 - $50,000 1 $15,001 - $50,000 2 $15,001 - $50,000 3 $1,001 - $15,000 4 $15,001 - $50,000 5 $15,001 - $50,000 6 $15,001 - $50,000 7 $15,001 - $50,000 8 $15,001 - $50,000 9 $15,001 - $50,000 10 $1,001 - $15,000 11 $15,001 - $50,000 12 $1,001 - $15,000 13 $50,001 - $100,000 14 $15,001 - $50,000 15 $50,001 - $100,000 16 $50,001 - $100,000 17 NaN 18 NaN 19 NaN 20 NaN 21 NaN 22 NaN 23 NaN 24 NaN 25 NaN 26 NaN 27 $100,001 - $250,000 28 $1,001 - $15,000 29 $100,001 - $250,000 30 NaN 31 NaN 32 NaN 33 NaN 34 NaN 35 NaN 36 NaN 37 NaN 38 NaN 39 NaN 40 $1,001 - $15,000 41 $1,001 - $15,000 42 $1,001 - $15,000 43 $1,001 - $15,000 44 $1,001 - $15,000 45 $1,001 - $15,000 46 $1,001 - $15,000 47 $1,001 - $15,000 48 $1,001 - $15,000 49 $1,001 - $15,000 50 $1,001 - $15,000 51 $1,001 - $15,000 52 $1,001 - $15,000 Name: Amount, dtype: object
We’ll create two new columns, Lower Bound and Upper Bound to reflect the range possible, since Senators don’t need to disclose exact dollar amounts, but instead just a range of values.
df['Amount'] = df['Amount'].str.replace('$','').str.replace(',','').str.split(' - ')
x = df['Amount'][17]
x
nan
def lower_bound(amount):
if type(amount) != type([]):
return amount
else:
return int(amount[0])
def upper_bound(amount):
if type(amount) != type([]):
return amount
else:
return int(amount[1])
df['Upper Bound'] = df['Amount'].apply(upper_bound)
df['Lower Bound'] = df['Amount'].apply(lower_bound)
df.head()
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
# | Transaction Date | Owner | Ticker | Asset Name | Asset Type | Type | Amount | Comment | Upper Bound | Lower Bound | |
---|---|---|---|---|---|---|---|---|---|---|---|
0 | 37 | 08/01/2022 | Self | IVV | iShares Core S&P 500 ETF | Stock | Sale (Full) | [15001, 50000] | — | 50000.0 | 15001.0 |
1 | 36 | 08/01/2022 | Self | EFAV | iShares MSCI EAFE Min Vol Factor ETF | Stock | Sale (Full) | [15001, 50000] | — | 50000.0 | 15001.0 |
2 | 35 | 08/01/2022 | Self | XLY | SPDR Select Sector Fund – Consumer Discretionary | Stock | Sale (Full) | [15001, 50000] | — | 50000.0 | 15001.0 |
3 | 34 | 08/01/2022 | Self | VTRS | Viatris Inc. – Common Stock | Stock | Sale (Full) | [1001, 15000] | — | 15000.0 | 1001.0 |
4 | 33 | 08/01/2022 | Self | ABNFX | The Bond Fd of America Class F-2 Shares | Stock | Purchase | [15001, 50000] | — | 50000.0 | 15001.0 |
Exploring Ranges of Purchases and Sales¶
Let’s see the minimum dollar amount the Senator has purchased or sold for himself.
senator_purchases = df[(df['Owner']=='Self') & (df['Type'] =='Purchase')]
senator_purchases['Lower Bound'].sum()
67011.0
This means at a minimum the Senator spent $67,000 on different security purchases, notice that the data also included options trading. What about the Upper Bound?
senator_purchases['Upper Bound'].sum()
305000.0
This means at a maximum the Senator spent over $300,000 on different security purchases. What about sales?
df['Type'].unique()
array(['Sale (Full)', 'Purchase', nan, 'Sale (Partial)'], dtype=object)
Notice there are full sales and partial sales!
# We will only look at completely closed positions
senator_sales = df[(df['Owner']=='Self')]
senator_sales = df[(df['Type']=='Sale (Full)') | (df['Type']=='Sale (Partial)')]
senator_sales['Lower Bound'].sum()
249017.0
senator_sales['Upper Bound'].sum()
785000.0
This means the Senator sold somewhere between approximately $249,000-785,000 worth of securities last quarter. If you want to continue to explore stock trades from Senators, you can review their yearly disclosures, such as the one presented here: https://efdsearch.senate.gov/search/view/annual/0f9f2654-4739-4bfc-af23-703182470220/