統合アルゴリズム、特徴選択、タイタニック号の乗組員が救助されたケース

注釈

統合されたアルゴリズムのカテゴリ：

• バギング：訓練された複数の分類器を平均する各カテゴリは、例えば、同時に訓練することができるので、関係ありません、ランダムフォレスト
• 昇圧：最初の訓練パラメータに配置された分類器の結果、重みを加算することにより各分類、例えばアダブースト、Xgboost
• スタッキング：基本的なアルゴリズムを入力として、複数のトレーニングの結果を予測します

タイタニック号の乗組員が救出します

データインポート：

import pandas as pd
titanic = pds.read_csv("titanic_train.csv")
# 缺失值处理：用中位数代替
titanic["Age"] = titanic["Age"].fillna(titanic["Age"].median())
# 字符值处理：映射成数字
titanic.loc[titanic["Sex"] == "male", "Sex"] = 0
titanic.loc[titanic["Sex"] == "female", "Sex"] = 1
# 缺失值处理：用出现频率最高的值代替
titanic["Embarked"] = titanic["Embarked"].fillna('S')
titanic.loc[titanic["Embarked"] == "S", "Embarked"] = 0
titanic.loc[titanic["Embarked"] == "C", "Embarked"] = 1
titanic.loc[titanic["Embarked"] == "Q", "Embarked"] = 2

線形回帰モデル：

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import KFold
import numpy as np

predictors = ["Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked"]


alg = LinearRegression()
kf = KFold(n_splits=3)

predictions = []
for train, test in kf.split(titanic):
    train_predictors = (titanic[predictors].iloc[train,:])
    train_target = titanic["Survived"].iloc[train]
    alg.fit(train_predictors, train_target)
    test_predictions = alg.predict(titanic[predictors].iloc[test,:])
    predictions.append(test_predictions)

predictions = np.concatenate(predictions, axis=0)
predictions[predictions > .5] = 1
predictions[predictions <=.5] = 0
accuracy = sum(predictions == titanic["Survived"]) / len(predictions)
print (accuracy)
# 0.7833894500561167

ロジスティック回帰モデル：

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
alg = LogisticRegression(max_iter = 3000)
scores = cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=3)
print(scores.mean())
# 0.7946127946127947

ランダムフォレストモデル：

from sklearn.ensemble import RandomForestClassifier
alg = RandomForestClassifier(n_estimators=10, min_samples_split=2, min_samples_leaf=1)
scores = cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=3)
print(scores.mean())
# 0.7934904601571269

# 改变参数值
alg = RandomForestClassifier(random_state=1, n_estimators=100, min_samples_split=4, min_samples_leaf=2)
scores = cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=3)
print(scores.mean())
# 0.8148148148148148

名前から取られた次元の特徴「タイトル」を、追加します。

import re
def get_title(name):
    title_search = re.search(' ([A-Za-z]+)\.', name)
    if title_search:
        return title_search.group(1)
    return ""

titles = titanic["Name"].apply(get_title)
title_mapping = {"Mr": 1, "Miss": 2, "Mrs": 3, "Master": 4, "Dr": 5, "Rev": 6, "Major": 7, "Col": 7, "Mlle": 8, "Mme": 8, "Don": 9, "Lady": 10, "Countess": 10, "Jonkheer": 10, "Sir": 9, "Capt": 7, "Ms": 2}
for k,v in title_mapping.items():
    titles[titles == k] = v

titanic["Title"] = titles

各機能の重要性の検出：

from sklearn.feature_selection import SelectKBest, f_classif
import matplotlib.pyplot as plt

all_predictors = ["Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked","Title"]

selector = SelectKBest(f_classif, k=5)
selector.fit(titanic[all_predictors], titanic["Survived"])
scores = -np.log10(selector.pvalues_)

plt.bar(range(len(predictors)), scores)
plt.xticks(range(len(predictors)), predictors, rotation='vertical')
plt.show()

4つのスタッキングアルゴリズムの最も重要な機能の選択：

from sklearn.ensemble import GradientBoostingClassifier
predictors = ["Pclass", "Sex", "Fare", "Title"]

algorithms = [
    [GradientBoostingClassifier(n_estimators=25, max_depth=3), predictors],
    [LogisticRegression(max_iter = 3000), predictors]
]

kf = KFold(n_splits=3)

predictions = []
for train, test in kf.split(titanic):
    train_target = titanic["Survived"].iloc[train]
    full_test_predictions = []
    for alg, predictors in algorithms:
        alg.fit(titanic[predictors].iloc[train,:], train_target)
        test_predictions = alg.predict_proba(titanic[predictors].iloc[test,:].astype(float))[:,1]
        full_test_predictions.append(test_predictions)
    test_predictions = (full_test_predictions[0] + full_test_predictions[1]) / 2
    test_predictions[test_predictions <= .5] = 0
    test_predictions[test_predictions > .5] = 1
    predictions.append(test_predictions)


predictions = np.concatenate(predictions, axis=0)
accuracy = sum(predictions == titanic["Survived"]) / len(predictions)
print(accuracy)
# 0.8181818181818182