Problem: I have a dataframe with various column headers that have names with variations of the multiple strings: 'Fee_code','zip_code', etc. and also some others with: 'street_address','violation_street address', etc.
Expected Outcome: A list with all the column headers that match the keywords: Fee, address, code, name, and possibly others based on the specific file that I'll work on. Note that I DO want to keep the 'agency name' column header.
Solution: I came up with this function to list all of the strings listed above - and some more-:
def drop_cols(df):
list1= list(df.filter(like='nam', axis=1))
list1.remove('agency_name')
list2= list(df.filter(like='add', axis=1))
list3= list(df.filter(like='fee', axis=1))
list4 = list(df.filter(like='code', axis=1))
list5 = list(df.filter(like='status', axis=1))
entry= list1+list2+list3+list4+list5
return entry
Challenge: This code works, but it's bulky and I'm wondering if there are better ways to achieve the same
Sample of column headers: 'ticket_id', 'agency_name', 'inspector_name', 'violator_name', 'violation_street_number', 'violation_street_name', 'violation_zip_code', 'mailing_address_str_number', 'mailing_address_str_name', 'city', 'state', 'zip_code', 'non_us_str_code', 'country', 'ticket_issued_date', 'hearing_date', 'violation_code', 'violation_description', 'disposition', 'fine_amount', 'admin_fee', 'state_fee', 'late_fee', 'discount_amount', 'clean_up_cost', 'judgment_amount', 'payment_amount', 'balance_due', 'payment_date', 'payment_status', 'collection_status', 'grafitti_status', 'compliance_detail', 'compliance']
One way you could go about it:
#create search collection of relevant terms
search='|'.join(['fee','address','code','name'])
#use the filter method in pandas with the regex option
#then drop the 'agency_name' column
#d is the dataframe
d.filter(regex=search,axis=1).drop('agency_name',axis=1)