Pandas dataframe Grouping and Counting with validations in Python

NewNY1990 :

I am currently making an analysis to perform the following:

1. I need to calculate whether 4 entries per year exists for 'No. People' for 2018 and 2019. Same dates one should be excluded (does not matter which one)

It should look like the following:

Year    Gender  No. People 
18      Men         11
        Woman        8
        Not Applied  3
19      Men         14
        Woman        5
        Not Applied  0

The No. People column shows the count of No. People.

2. Check per Gender whether the last 10 months in a 10-day period more than 6 entries in No. People exists.

Result could look like:

Period                   Gender      Entries
01/23/2019 - 01/15/2019  Men         6
N/A                      Woman       N/A
N/A                      Not Applied N/A

3. Check whether there are 11 measures for No. People over the last 3 month

Period                   Gender      Entries
12/20/2018 - 01/23/2019  Men         26
12/20/2018 - 01/23/2019  Woman       13
12/20/2018 - 12/26/2018  Not Applied N/A

Somehow it look complictaed and thats why I struggle with the code.

I started to use the following code:

import pandas as pd
path = 'path'
filename = 'excel.xls'
final_path = path + '/' + filename
ws_name = 'Sheet1'

df.groupby(df['Date'].dt.year)['No. People'].agg(['count'])

but somhow I am struggeling with the results or errors.

The data looks like the following which is in Excel:

Date    Gender  No. People
12/20/18    Men 4
12/21/18    Men 9
12/22/18    Men 3
12/23/18    Men 9
12/24/18    Men 6
12/25/18    Men 1
12/26/18    Men 3
12/27/18    Men 8
12/28/18    Men 3
12/29/18    Men 5
12/30/18    Men 8
12/31/18    Men 
01/01/19    Men 
01/02/19    Men 
01/03/19    Men 
01/04/19    Men 9
01/05/19    Men 7
01/06/19    Men 5
01/07/19    Men 1
01/08/19    Men 8
01/09/19    Men 5
01/10/19    Men 6
01/11/19    Men 9
01/12/19    Men 7
01/13/19    Men 
01/14/19    Men 
01/15/19    Men 
01/16/19    Men 
01/17/19    Men 
01/18/19    Men 
01/19/19    Men 6
01/20/19    Men 5
01/21/19    Men 2
01/22/19    Men 5
01/23/19    Men 1
12/20/18    Women   6
12/21/18    Women   6
12/22/18    Women   2
12/23/18    Women   2
12/24/18    Women   2
12/25/18    Women   
12/26/18    Women   
12/27/18    Women   
12/28/18    Women   1
12/29/18    Women   1
12/30/18    Women   4
12/31/18    Women   
01/01/19    Women   
01/02/19    Women   
01/03/19    Women   
01/04/19    Women   
01/05/19    Women   
01/06/19    Women   
01/07/19    Women   
01/08/19    Women   
01/09/19    Women   
01/10/19    Women   
01/11/19    Women   
01/12/19    Women   
01/13/19    Women   
01/14/19    Women   
01/15/19    Women   
01/16/19    Women   
01/17/19    Women   
01/18/19    Women   
01/19/19    Women   4
01/20/19    Women   6
01/21/19    Women   8
01/22/19    Women   9
01/23/19    Women   4
12/20/18    Not Applied 6
12/21/18    Not Applied 2
12/22/18    Not Applied 3
12/23/18    Not Applied 
12/24/18    Not Applied 
12/25/18    Not Applied 
12/26/18    Not Applied

effy :

For the first, it is good just add grouping by gender too

df['Date'] = pd.to_datetime(df['Date'])
df.groupby([df['Date'].dt.year, 'Gender'])['No. People'].agg(['count'])

For second to group it by periods of 10 days you can use pandas Grouper class

df.sort_values(by=['Date'], ascending=False, inplace=True)
from_date = df.iloc[0]['Date'] - pd.DateOffset(months=10)
last_10_months = df[df.Date >= from_date]
count_people = last_10_months.groupby([pd.Grouper(key='Date', freq='10D'), 'Gender']).count()
count_people[count_people['No. People'] > 6]

same for third with the month

df.sort_values(by=['Date'], ascending=False, inplace=True)
from_date = df.iloc[0]['Date'] - pd.DateOffset(months=3)
last_3_months = df[df.Date >= from_date]
df.groupby(['Gender']).count()
count_people[count_people['No. People'] > 11]