6.1 Fit a logistic regression model on training data and assess against testing data.
a) Fit a logistic regression model on training data.
# Notice we are using new data sets that need to be read into the environment
train = pd.read_csv('/Users/tips_train.csv')
test = pd.read_csv('/Users/tips_test.csv')
train["fifteen"] = 0.15 * train["total_bill"]
train["greater15"] = np.where(train["tip"] > train["fifteen"], 1, 0)
test["fifteen"] = 0.15 * test["total_bill"]
test["greater15"] = np.where(test["tip"] > test["fifteen"], 1, 0)
logreg = sm.formula.glm("greater15 ~ total_bill",
family=sm.families.Binomial(),
data=train).fit()
print(logreg.summary())
b) Assess the model against the testing data.
# Prediction on testing data
predictions = logreg.predict(test)
predY = np.where(predictions < 0.5, 0, 1)
# If the prediction probability is less than 0.5, classify this as a 0
# and otherwise classify as a 1. This isn't the best method -- a better
# method would be randomly assigning a 0 or 1 when a probability of 0.5
# occurrs, but this insures that results are consistent
# Determine how many were correctly classified
Results = np.where(predY == test["greater15"], "Correct", "Wrong")
print(pd.crosstab(index=Results, columns="count"))
A logistic regression model can be implemented using sklearn, howeverstatsmodels.api provides a helpful summary about the model, so it is preferable for this example.
6.2 Fit a linear regression model on training data and assess against testing data.
a) Fit a linear regression model on training data.
# Notice we are using new data sets that need to be read into the environment
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
# Fit a linear regression model
linreg = LinearRegression()
linreg.fit(train.drop(["Target"], axis = 1), train["Target"])
print(linreg.coef_)
print(linreg.intercept_)
b) Assess the model against the testing data.
# Prediction on testing data
prediction = pd.DataFrame()
prediction["predY"] = linreg.predict(test.drop(["Target"], axis = 1))
# Determine mean squared error
prediction["sq_diff"] = (prediction["predY"] - test["Target"])**2
print(np.mean(prediction["sq_diff"]))
6.3 Fit a decision tree model on training data and assess against testing data.
a) Fit a decision tree classification model.
i) Fit a decision tree classification model on training data and determine variable importance.
# Notice we are using new data sets that need to be read into the environment
train = pd.read_csv('/Users/breastcancer_train.csv')
test = pd.read_csv('/Users/breastcancer_test.csv')
from sklearn.tree import DecisionTreeClassifier
# random_state is used to specify a seed for a random integer so that the
# results are reproducible
treeMod = DecisionTreeClassifier(random_state=29)
treeMod.fit(train.drop(["Target"], axis = 1), train["Target"])
# Determine variable importance
var_import = treeMod.feature_importances_
var_import = pd.DataFrame(var_import)
var_import = var_import.rename(columns = {0:'Importance'})
var_import = var_import.sort_values(by="Importance", kind = "mergesort",
ascending = False)
print(var_import.head())
ii) Assess the model against the testing data.
# Prediction on testing data
predY = treeMod.predict(test.drop(["Target"], axis = 1))
# Determine how many were correctly classified
Results = np.where(test["Target"] == predY, "Correct", "Wrong")
print(pd.crosstab(index=Results, columns="count"))
b) Fit a decision tree regression model.
i) Fit a decision tree regression model on training data and determine variable importance.
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
from sklearn.tree import DecisionTreeRegressor
treeMod = DecisionTreeRegressor(random_state=29)
treeMod.fit(train.drop(["Target"], axis = 1), train["Target"])
# Determine variable importance
var_import = treeMod.feature_importances_
var_import = pd.DataFrame(var_import)
var_import = var_import.rename(columns = {0:'Importance'})
var_import = var_import.sort_values(by="Importance", kind = "mergesort",
ascending = False)
print(var_import.head())
ii) Assess the model against the testing data.
# Prediction on testing data
prediction = pd.DataFrame()
prediction["predY"] = treeMod.predict(test.drop(["Target"], axis = 1))
# Determine mean squared error
prediction["sq_diff"] = (prediction["predY"] - test["Target"])**2
print(np.mean(prediction["sq_diff"]))
6.4 Fit a random forest model on training data and assess against testing data.
a) Fit a random forest classification model.
i) Fit a random forest classification model on training data and determine variable importance.
train = pd.read_csv('/Users/breastcancer_train.csv')
test = pd.read_csv('/Users/breastcancer_test.csv')
from sklearn.ensemble import RandomForestClassifier
rfMod = RandomForestClassifier(random_state=29)
rfMod.fit(train.drop(["Target"], axis = 1), train["Target"])
# Determine variable importance
var_import = rfMod.feature_importances_
var_import = pd.DataFrame(var_import)
var_import = var_import.rename(columns = {0:'Importance'})
var_import = var_import.sort_values(by="Importance", kind = "mergesort",
ascending = False)
print(var_import.head())
ii) Assess the model against the testing data.
# Prediction on testing data
predY = rfMod.predict(test.drop(["Target"], axis = 1))
# Determine how many were correctly classified
Results = np.where(test["Target"] == predY, "Correct", "Wrong")
print(pd.crosstab(index=Results, columns="count"))
b) Fit a random forest regression model.
i) Fit a random forest regression model on training data and determine variable importance.
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
from sklearn.ensemble import RandomForestRegressor
rfMod = RandomForestRegressor(random_state=29)
rfMod.fit(train.drop(["Target"], axis = 1), train["Target"])
# Determine variable importance
var_import = rfMod.feature_importances_
var_import = pd.DataFrame(var_import)
var_import = var_import.rename(columns = {0:'Importance'})
var_import = var_import.sort_values(by="Importance", kind = "mergesort",
ascending = False)
print(var_import.head())
ii) Assess the model against the testing data.
# Prediction on testing data
prediction = pd.DataFrame()
prediction["predY"] = rfMod.predict(test.drop(["Target"], axis = 1))
# Determine mean squared error
prediction["sq_diff"] = (test["Target"] - prediction["predY"])**2
print(prediction["sq_diff"].mean())
6.5 Fit a gradient boosting model on training data and assess against testing data.
a) Fit a gradient boosting classification model.
i) Fit a gradient boosting classification model on training data and determine variable importance.
train = pd.read_csv('/Users/breastcancer_train.csv')
test = pd.read_csv('/Users/breastcancer_test.csv')
from sklearn.ensemble import GradientBoostingClassifier
# n_estimators = total number of trees to fit which is analogous to the
# number of iterations
# learning_rate = shrinkage or step-size reduction, where a lower
# learning rate requires more iterations
gbMod = GradientBoostingClassifier(random_state = 29, learning_rate = .01,
n_estimators = 2500)
gbMod.fit(train.drop(["Target"], axis = 1), train["Target"])
# Determine variable importance
var_import = gbMod.feature_importances_
var_import = pd.DataFrame(var_import)
var_import = var_import.rename(columns = {0:'Importance'})
var_import = var_import.sort_values(by="Importance", kind = "mergesort",
ascending = False)
print(var_import.head())
ii) Assess the model against the testing data.
# Prediction on testing data
predY = gbMod.predict(test.drop(["Target"], axis = 1))
# Determine how many were correctly classified
Results = np.where(test["Target"] == predY, "Correct", "Wrong")
print(pd.crosstab(index=Results, columns="count"))
b) Fit a gradient boosting regression model.
i) Fit a gradient boosting regression model on training data and determine variable importance.
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
from sklearn.ensemble import GradientBoostingRegressor
gbMod = GradientBoostingRegressor(random_state = 29, learning_rate = .01,
n_estimators = 2500)
gbMod.fit(train.drop(["Target"], axis = 1), train["Target"])
# Determine variable importance
var_import = gbMod.feature_importances_
var_import = pd.DataFrame(var_import)
var_import = var_import.rename(columns = {0:'Importance'})
var_import = var_import.sort_values(by="Importance", kind = "mergesort",
ascending = False)
print(var_import.head())
ii) Assess the model against the testing data.
# Prediction on testing data
prediction = pd.DataFrame()
prediction["predY"] = gbMod.predict(test.drop(["Target"], axis = 1))
# Determine mean squared error
prediction["sq_diff"] = (test["Target"] - prediction["predY"])**2
6.6 Fit an extreme gradient boosting model on training data and assess against testing data.
a) Fit an extreme gradient boosting classification model on training data and assess against testing data.
i) Fit an extreme gradient boosting classification model on training data.
train = pd.read_csv('/Users/breastcancer_train.csv')
test = pd.read_csv('/Users/breastcancer_test.csv')
from xgboost import XGBClassifier
# Fit XGBClassifier model on training data
xgbMod = XGBClassifier(seed = 29, learning_rate = 0.01,
n_estimators = 2500)
xgbMod.fit(train.drop(["Target"], axis = 1), train["Target"])
ii) Assess the model against the testing data.
# Prediction on testing data
predY = xgbMod.predict(test.drop(["Target"], axis = 1))
# Determine how many were correctly classified
Results = np.where(test["Target"] == predY, "Correct", "Wrong")
print(pd.crosstab(index=Results, columns="count"))
b) Fit an extreme gradient boosting regression model on training data and assess against testing data.
i) Fit an extreme gradient boosting regression model on training data.
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
from xgboost import XGBRegressor
# Fit XGBRegressor model on training data
xgbMod = XGBRegressor(seed = 29, learning_rate = 0.01,
n_estimators = 2500)
xgbMod.fit(train.drop(["Target"], axis = 1), train["Target"])
ii) Assess the model against the testing data.
# Prediction on testing data
prediction = pd.DataFrame()
prediction["predY"] = xgbMod.predict(test.drop(["Target"], axis = 1))
# Determine mean squared error
prediction["sq_diff"] = (test["Target"] - prediction["predY"])**2
print(prediction["sq_diff"].mean())
6.7 Fit a support vector model on training data and assess against testing data.
a) Fit a support vector classification model.
i) Fit a support vector classification model on training data.
Note: In implementation scaling should be used.
train = pd.read_csv('/Users/breastcancer_train.csv')
test = pd.read_csv('/Users/breastcancer_test.csv')
# Fit a support vector classification model
from sklearn.svm import SVC
svMod = SVC(random_state = 29, kernel = 'linear')
svMod.fit(train.drop(["Target"], axis = 1), train["Target"])
ii) Assess the model against the testing data.
# Prediction on testing data
prediction = svMod.predict(test.drop(["Target"], axis = 1))
# Determine how many were correctly classified
Results = np.where(test["Target"] == prediction, "Correct", "Wrong")
print(pd.crosstab(index=Results, columns="count"))
b) Fit a support vector regression model.
i) Fit a support vector regression model on training data.
Note: In implementation scaling should be used.
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
# Fit a support vector regression model
from sklearn.svm import SVR
svMod = SVR()
svMod.fit(train.drop(["Target"], axis = 1), train["Target"])
ii) Assess the model against the testing data.
# Prediction on testing data
prediction = pd.DataFrame()
prediction["predY"] = svMod.predict(test.drop(["Target"], axis = 1))
# Determine mean squared error
prediction["sq_diff"] = (test["Target"] - prediction["predY"])**2
print(prediction["sq_diff"].mean())
6.8 Fit a neural network model on training data and assess against testing data.
a) Fit a neural network classification model.
i) Fit a neural network classification model on training data.
# Notice we are using new data sets
train = pd.read_csv('/Users/digits_train.csv')
test = pd.read_csv('/Users/digits_test.csv')
# Fit a neural network classification model on training data
from sklearn.neural_network import MLPClassifier
nnMod = MLPClassifier(max_iter = 200, hidden_layer_sizes=(100,),
random_state = 29)
nnMod.fit(train.drop(["Target"], axis = 1), train["Target"])
ii) Assess the model against the testing data.
# Prediction on testing data
predY = nnMod.predict(test.drop(["Target"], axis = 1))
# Determine how many were correctly classified
from sklearn.metrics import confusion_matrix
print(confusion_matrix(test["Target"], predY))
b) Fit a neural network regression model.
i) Fit a neural network regression model on training data.
train = pd.read_csv('/Users/boston_train.csv')
test = pd.read_csv('/Users/boston_test.csv')
# Scale input data
from sklearn.preprocessing import StandardScaler
train_features = train.drop(["Target"], axis = 1)
scaler = StandardScaler().fit(np.array(train_features))
train_scaled = scaler.transform(np.array(train_features))
train_scaled = pd.DataFrame(train_scaled)
test_features = test.drop(["Target"], axis = 1)
scaler = StandardScaler().fit(np.array(test_features))
test_scaled = scaler.transform(np.array(test_features))
test_scaled = pd.DataFrame(test_scaled)
# Fit neural network regression model, dividing target by 50 for scaling
from sklearn.neural_network import MLPRegressor
nnMod = MLPRegressor(max_iter = 250, random_state = 29, solver = 'lbfgs')
nnMod = nnMod.fit(train_scaled, train["Target"] / 50)
ii) Assess the model against testing data.
# Prediction on testing data, remembering to multiply by 50
prediction = pd.DataFrame()
prediction["predY"] = nnMod.predict(test_scaled)*50
# Determine mean squared error
prediction["sq_diff"] = (test["Target"] - prediction["predY"])**2
print(prediction["sq_diff"].mean())