离群点异常检测及可视化分析工具pyod测试

找到了一个对Outlier Detection (Anomaly Detection) 异常值检测(异常检测)的比较好的工具(https://github.com/yzhao062/Pyod),该工具集成了多个算法。

具体包括的算法如下:

Model 1 Angle-based Outlier Detector (ABOD)
Model 2 Cluster-based Local Outlier Factor (CBLOF)
Model 3 Feature Bagging
Model 4 Histogram-base Outlier Detection (HBOS)
Model 5 Isolation Forest
Model 6 K Nearest Neighbors (KNN)
Model 7 Average KNN
Model 8 Median KNN
Model 9 Local Outlier Factor (LOF)
Model 10 Minimum Covariance Determinant (MCD)
Model 11 One-class SVM (OCSVM)
Model 12 Principal Component Analysis (PCA)

这些算法主要都是无监督的方式来实现的异常离群点值检测的方法。

该库提供了这些算法的多种测试例子,如下为ABOD算法的测试结果。

同时也提供了对所有算法的比较:

其核心代码如下:

扫描二维码关注公众号,回复: 2434601 查看本文章
for i, (clf_name, clf) in enumerate(classifiers.items()):
        print()
        print(i + 1, 'fitting', clf_name)
        # fit the data and tag outliers
        clf.fit(X)
        scores_pred = clf.decision_function(X) * -1
        y_pred = clf.predict(X)
        threshold = stats.scoreatpercentile(scores_pred,
                                            100 * outliers_fraction)
        n_errors = (y_pred != ground_truth).sum()
        # plot the levels lines and the points

        Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()]) * -1
        Z = Z.reshape(xx.shape)
        subplot = plt.subplot(3, 4, i + 1)
        subplot.contourf(xx, yy, Z, levels=np.linspace(Z.min(), threshold, 7),
                         cmap=plt.cm.Blues_r)
        a = subplot.contour(xx, yy, Z, levels=[threshold],
                            linewidths=2, colors='red')
        subplot.contourf(xx, yy, Z, levels=[threshold, Z.max()],
                         colors='orange')
        b = subplot.scatter(X[:-n_outliers, 0], X[:-n_outliers, 1], c='white',
                            s=20, edgecolor='k')
        c = subplot.scatter(X[-n_outliers:, 0], X[-n_outliers:, 1], c='black',
                            s=20, edgecolor='k')
        subplot.axis('tight')
        subplot.legend(
            [a.collections[0], b, c],
            ['learned decision function', 'true inliers', 'true outliers'],
            prop=matplotlib.font_manager.FontProperties(size=10),
            loc='lower right')
        subplot.set_xlabel("%d. %s (errors: %d)" % (i + 1, clf_name, n_errors))
        subplot.set_xlim((-7, 7))
        subplot.set_ylim((-7, 7))

运行比较结果如下,可以看出用等值区域法填充的可视化结果可以较好地展现各种不同算法的比较。

猜你喜欢

转载自blog.csdn.net/sparkexpert/article/details/81195418
今日推荐