各位学员好，前天有个《python信用评分卡建模（附代码）》课程学员向我提问。她在银行工作，银行对金融风控模型验证和压力测试非常重视。目前不清楚如果对风控模型压力测试。
这个问题很专业，很少有学生提到这问题。目前国内对风控模型的验证并不重视，只有银行和少数持牌照消费金融公司有模型验证团队。有的模型验证团队并不正规，难以保持工作独立性。
国内银行参考了美国美联储SR 11-7文件Guidance on Model Risk Management，翻译为中文为模型风险管理指南。
美联储2011年4月发布模型风险管理的相关法规——SR 11-7号文《模型风险管理指南》，该文明确了模型风险管理框架，包括模型风险管理定义、模型实施、模型验证、以及模型风险管理的政策制度等内容。SR 11-7号文重点在模型验证环节。
回到正题，模型压力和风险测试很难，当客群变化后，模型稳定性下降，即PSI指数上升，模型是需要重新迭代的。当发生疫情后，如果失业率快速上升，客户违约风险也会快速上升。这不是子模型能解决的问题，需要建立宏观经济模型来提前预测失业率等指标，然后收紧策略，缩小放款范围。
至于高风险客户，模型还是有办法的。我推荐逻辑回归评分卡模型建模，评分卡的woe分箱中，可以把缺失值或异常值(高风险值)单独分箱。因此评分卡模型对异常值和缺失值有很高容忍度。感兴趣朋友可以收藏和关注我的课程《python信用评分卡建模（附代码）》。该课程还有模型验证的常用方法，包括模型区分能力，排序能力，稳定性验证方法。
不幸的是，国内诸多金融公司在建模中存在结构性问题。例如一个申请单进入审批，策略会先评估该客户信用风险，如果没问题才进入模型判断。模型几乎是垫底的。这意味着诸多变量的异常值已经被提前过滤了。这种模型是非常脆弱的。因此我还是建议大家把异常值，高风险值加入模型训练，提高模型抗压能力。毕竟评分卡模型可以对异常值，高风险值单独分箱处理。
大家不要把模型想的很复杂，你可以把模型想成一个人。这个人在平时训练时经历过大风大浪，才有抗压风险。如果这个人经历太顺利，可能遇到一个挫折就不知所措。
对于压力测试，我还想谈谈大家对异常值认真的误区。
很多建模人员遇到异常值时，直接删除异常值，然后建模。对于压力测试，这是不妥的。很多高风险值就是异常值。当欺诈客户行骗时，就会触发某些变量产生异常值。如果我们轻易删除异常值，就不会捕捉到这些欺诈客户，而且模型区分能力可能降低。《python信用评分卡建模（附代码）》的give me some credit案例章节就有去掉异常值后，模型区分能力下降实验测试。
模型风险管理还有个难点就是人祸。经营者在面对巨额利润时，会刻意忽略风险。这也是造成风控部门和业务部门长期对立原因。业务部门想赚更多钱，让风控部门放松策略。风控部门想降低风险，收紧策略。而且业务部门和风控部门考核KPI指标是相反的。
之前发布过文章《捷信金融（Home Credit N.V.）兴衰如梦，留下宝贵机器学习建模数据》。捷信作为国内最早消费金融公司，拥有大批风控专家，其风控策略和风控模型是非常强悍的。随着捷信线下迅速扩张，造成风控跟不上和故意对风险视而不见，最后捷信老大哥跌下神坛。
电影《大空头》给观众解释了美国华尔街金融危机爆发前，诸多数据分析师已经察觉风险。但美国政客，金融从业人员甚至风险监管人员为了高额利润对其视而不见，造成后来全球巨大经融危机，甚至到今天问题都没解决。大家可以去看看这部经典电影。
最后我把SR 11-7：模型风险管理指南（Guidance on Model Risk Management）中英文放入文章，供学员参考。
致各联邦储备银行负责监督的官员和适当的监督检查人员
主题：
模型风险管理指南
美联储和货币监理署 (OCC)正在发布随附_的模型风险管理监管指南_，供银行组织和监管机构在评估组织对模型风险的管理时使用。本指南应酌情适用于受美联储监管的所有银行组织，同时考虑每个组织的规模、性质和复杂性，以及模型使用的范围和复杂程度（定义和讨论如下）。
模型风险管理
银行组织应注意基于不正确或滥用模型的决策可能产生的不利后果（包括财务损失），并应通过积极的模型风险管理来解决这些后果。此SR 信的附件更详细地描述了有效模型风险管理框架的关键方面，包括稳健的模型开发、实施和使用；有效的验证；健全的治理、政策和控制。
美联储和 OCC 之前发布的出版物已经讨论了模型的使用，特别关注模型验证。1基于过去几年的监管和行业经验，本文件扩展了现有指南——最重要的是扩大范围以包括模型风险管理的其他关键方面。
就本文档而言，术语_模型_是指应用统计、经济、金融或数学理论、技术和假设将输入数据处理为定量估计的定量方法、系统或方法。符合此定义的模型可用于分析业务战略、为业务决策提供信息、识别和衡量风险、评估风险敞口、工具或头寸、进行压力测试、评估资本充足性、管理客户资产、衡量内部限制的合规性、维护正式银行的控制设备，或满足财务或监管报告要求并发布公开披露信息。_模型_的定义还包括定量方法，其输入部分或全部是定性的或基于专家判断，前提是输出是定量的。2
模型的使用总是会带来模型风险，即基于不正确或滥用模型输出和报告的决策可能产生不利后果。模型风险可能导致财务损失、糟糕的业务和战略决策，或损害银行组织的声誉。模型风险的发生主要有两个原因： (1)模型可能存在根本性错误并在与其设计目标和预期业务用途相悖时产生不准确的输出；(2) 一个模型可能被错误或不恰当地使用，或者可能对其局限性和假设存在误解。模型风险随着模型复杂性的增加、输入和假设的更高不确定性、更广泛的使用范围和更大的潜在影响而增加。银行组织应该从单个模型和整体模型中管理模型风险。
贯穿整个指南的指导原则是管理模型风险涉及模型的“有效挑战”：由客观、知情的各方进行批判性分析，可以识别模型限制并产生适当的变化。有效的挑战取决于激励、能力和影响力的结合。
与其他风险一样，重要性是模型风险管理中的一个重要考虑因素。如果在某些银行，模型的使用不那么普遍并且对其财务状况的影响较小，那么这些银行可能不需要如此复杂的模型风险管理方法来满足监管预期。但是，如果模型和模型输出对业务决策（包括与风险管理和资本和流动性规划相关的决策）产生重大影响，并且模型失效会对银行的财务状况产生特别有害的影响，银行的模型风险管理框架应该更加广泛和严谨。
模型开发、实施和使用
模型开发很大程度上依赖于开发人员的经验和判断，模型风险管理应包括与模型用户的情况和目标以及银行组织政策相一致的规范的模型开发和实施过程。完善的开发过程包括：明确的目的声明，以确保模型的开发符合其预期用途；模型背后的合理设计、理论和逻辑；稳健的模型方法和处理组件；严格评估数据质量和相关性；和适当的文件。模型开发的一个组成部分是测试，其中评估模型的各个组件及其整体功能，以显示模型按预期执行；证明它是准确、稳健和稳定的；并评估其局限性和假设。重要的是，组织应确保其模型更具判断性和定性的方面的开发也是合理的。
所有模型都有一定程度的不确定性和不准确性，因为它们根据定义是对现实的不完美表示。有效模型开发、实施和使用的一个重要结果是银行组织对这种不确定性的理解和解释。对模型不确定性的考虑可以包括对模型输出应用有充分支持的、判断性的、“保守的”调整，对模型输出的重视程度较低，或确保仅在有其他模型或方法补充时才使用模型。3
模型验证
模型验证是一组过程和活动，旨在验证模型是否按预期执行，符合其设计目标和业务用途。有效的验证有助于确保模型是合理的，识别潜在的限制和假设并评估它们可能产生的影响。所有模型组件——输入、处理、输出和报告——都应经过验证；这同样适用于内部开发的模型以及从供应商或顾问处购买或开发的模型。
验证涉及在一定程度上独立于模型开发和使用。通常，验证由不负责模型开发或使用的人员完成，并且与模型是否被确定为有效无关。实际上，一些验证工作可能由模型开发人员和用户最有效地完成；然而，此类验证工作必须接受独立方的严格审查，该独立方应开展额外活动以确保适当验证。总体而言，验证过程的质量由客观、知识渊博的各方的严格审查以及为解决这些各方确定的问题所采取的行动来表明。
在模型投入使用后，验证活动应持续进行，以跟踪已知模型限制并识别任何新限制。在经济和金融状况良好的时期，验证是一项重要的检查，此时风险和潜在损失的估计可能变得过于乐观，手头的数据可能无法完全反映压力更大的状况。银行组织应该对每个模型进行定期审查——至少每年一次，但如果有必要，可以更频繁地审查，以确定它是否按预期工作以及现有的验证活动是否足够。全面验证的关键要素包括：
-
概念合理性评估。该要素涉及评估模型设计和构建的质量，以及审查支持模型所用方法和所选变量的文档和经验证据。验证的这一步骤应确保在模型设计和构建中做出的判断是知情的、经过仔细考虑的，并与已发表的研究和良好的行业实践相一致。
-
持续监测。验证中的这一步是为了确认模型得到了适当的实施，并且正在按预期使用和执行。必须评估产品、风险敞口、活动、客户或市场条件的变化是否需要调整、重新开发或更换模型，并验证模型超出其原始范围的任何扩展是否有效。在此步骤中可以使用基准测试将给定模型的输入和输出与备选方案的估计值进行比较。
-
结果分析。此步骤涉及将模型输出与相应的实际结果进行比较。回溯测试是结果分析的一种形式，它涉及在模型开发中未使用的样本时间段内以与模型的预测范围或性能窗口相匹配的频率将实际结果与模型预测进行比较。
验证过程的三个核心要素的结果可能会揭示模型开发中的重大错误或不准确之处或始终超出银行组织预定可接受阈值的结果。在这种情况下，需要进行模型调整、重新校准或重新开发。有时，由于缺乏数据或价格可观察性等各种原因，银行组织可能无法使用关键模型验证工具。在这种情况下，在考虑模型使用的适当性时，更应注意模型的局限性，并在使用模型进行决策时充分告知高级管理层这些局限性。一般来说，高级管理层应确保根据已识别的模型限制采取适当的缓解措施，
治理、政策和控制
对模型风险管理框架开发和维护强有力的治理对其有效性至关重要。强有力的治理通过定义相关风险管理活动的政策、实施这些政策的程序、资源分配以及测试政策和程序是否按规定执行的机制，为风险管理职能提供明确的支持和结构。强有力的治理还包括足够详细的模型开发和验证文档，以使不熟悉模型的各方能够了解模型的运行方式及其局限性和关键假设。
董事会和高级管理层在建立全组织范围的模型风险管理方法时，在最高级别提供模型风险治理。董事会成员应确保模型风险水平在他们的容忍范围内。银行组织的内部审计职能应评估模型风险管理框架的整体有效性，包括框架处理单个模型和总体模型风险的能力。每当银行组织使用外部资源进行模型风险管理时，组织应在明确书面和商定的工作范围中指定要开展的活动，并且这些活动应按照本指南进行。此外，组织应维护已实施的模型清单以供使用，
所有银行组织都应确保其内部政策和程序与本指南中包含的风险管理原则和监管预期相一致。
联系人
有关本指南的问题，请致电(202) 452-2904联系高级金融监管分析师David Palmer ；Dwight Smith，资本与监管政策高级监管金融分析师，电话：(202) 452-2773；或助理主任Anna Lee Hewko，电话(202) 530-6260。此外，可以通过委员会的公共网站发送问题。4
Patrick M. Parkinson 银行 监管
司司长_签字_
附件：
模型风险管理指南(PDF)
交叉参考：
- SR 09-1，“市场风险规则在银行控股公司和国有银行中的应用”
笔记：
-
例如，OCC 在OCC 2000-16 （2000 年 5 月 30 日）、其他公告和_主计长手册_的某些主题小册子中提供了关于模型风险的指导，重点是模型验证。美联储发布了SR 信 09-01， “市场风险规则在银行控股公司和国有银行中的应用”，其中强调了与模型风险管理相关的各种概念，包括验证和审查标准、模型验证文档和回-测试。美联储的_交易和资本市场活动手册_也讨论了验证和模型风险管理。此外，高级方法基于风险的资本规则（12 CFR 3， 附录 C； 12 CFR 208， 附录 F；和12 CFR 225， 附录 G)包含针对主题银行组织的明确验证要求。
-
虽然不在本指南的范围内，但银行组织使用的更多定性方法（即那些未根据本指南定义为模型的方法）也应接受严格的控制过程。
-
如果模型用于生成包含在公共财务报表中的金额，则对模型不确定性的任何调整都必须符合公认的会计原则。
-
见http://www.federalreserve.gov/feedback.cfm。
TO THE OFFICER IN CHARGE OF SUPERVISION AND APPROPRIATE SUPERVISORY AND EXAMINATION STAFF AT EACH FEDERAL RESERVE BANK
SUBJECT:
Guidance on Model Risk Management
The Federal Reserve and Office of the Comptroller of the Currency (OCC) are issuing the attached Supervisory Guidance on Model Risk Management, which is intended for use by banking organizations and supervisors as they assess organizations’ management of model risk. This guidance should be applied as appropriate to all banking organizations supervised by the Federal Reserve, taking into account each organization’s size, nature, and complexity, as well as the extent and sophistication of its use of models (as defined and discussed below).
Model Risk Management
Banking organizations should be attentive to the possible adverse consequences (including financial loss) of decisions based on models that are incorrect or misused, and should address those consequences through active model risk management. The attachment to this SR letter describes in more detail the key aspects of an effective model risk management framework, including robust model development, implementation, and use; effective validation; and sound governance, policies, and controls.
Previous publications issued by the Federal Reserve and OCC have addressed the use of models, with particular focus on model validation.1 Based on supervisory and industry experience over the past several years, this document expands upon existing guidance—most importantly by broadening the scope to include other key aspects of model risk management.
For the purposes of this document, the term model refers to a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates. Models meeting this definition might be used for analyzing business strategies, informing business decisions, identifying and measuring risks, valuing exposures, instruments or positions, conducting stress testing, assessing adequacy of capital, managing client assets, measuring compliance with internal limits, maintaining the formal control apparatus of the bank, or meeting financial or regulatory reporting requirements and issuing public disclosures. The definition of model also covers quantitative approaches whose inputs are partially or wholly qualitative or based on expert judgment, provided that the output is quantitative in nature.2
The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. Model risk can lead to financial loss, poor business and strategic decision-making, or damage to a banking organization’s reputation. Model risk occurs primarily for two reasons: (1) a model may have fundamental errors and produce inaccurate outputs when viewed against its design objective and intended business uses; (2) a model may be used incorrectly or inappropriately or there may be a misunderstanding about its limitations and assumptions. Model risk increases with greater model complexity, higher uncertainty about inputs and assumptions, broader extent of use, and larger potential impact. Banking organizations should manage model risk both from individual models and in the aggregate.
A guiding principle throughout the guidance is that managing model risk involves “effective challenge” of models: critical analysis by objective, informed parties that can identify model limitations and produce appropriate changes. Effective challenge depends on a combination of incentives, competence, and influence.
As is generally the case with other risks, materiality is an important consideration in model risk management. If at some banks the use of models is less pervasive and has less impact on their financial condition, then those banks may not need as complex an approach to model risk management in order to meet supervisory expectations. However, where models and model output have a material impact on business decisions, including decisions related to risk management and capital and liquidity planning, and where model failure would have a particularly harmful impact on a bank’s financial condition, a bank’s model risk management framework should be more extensive and rigorous.
Model Development, Implementation, and Use
Model development relies heavily on the experience and judgment of developers, and model risk management should include disciplined model development and implementation processes that are consistent with the situation and goals of the model user and with the banking organization’s policy. A sound development process includes: a clear statement of purpose to ensure that the model is developed in line with its intended use; sound design, theory, and logic underlying the model; robust model methodologies and processing components; rigorous assessment of data quality and relevance; and appropriate documentation. An integral part of model development is testing, in which the various components of a model and its overall functioning are evaluated to show the model is performing as intended; to demonstrate that it is accurate, robust, and stable; and to evaluate its limitations and assumptions. Importantly, organizations should ensure that the development of the more judgmental and qualitative aspects of their models is also sound.
All models have some degree of uncertainty and inaccuracy because they are by definition imperfect representations of reality. An important outcome of effective model development, implementation, and use is a banking organization’s demonstrated understanding of and accounting for such uncertainty. Accounting for model uncertainty can include applying well-supported, judgmental, “conservative” adjustments to model output, placing less emphasis on a model’s output, or ensuring that a model is only used when supplemented by other models or approaches.3
Model Validation
Model validation is the set of processes and activities intended to verify that models are performing as expected, in line with their design objectives and business uses. Effective validation helps to ensure that models are sound, identifying potential limitations and assumptions and assessing their possible impact. All model components—inputs, processing, outputs, and reports—should be subject to validation; this applies equally to models developed in-house and to those purchased from or developed by vendors or consultants.
Validation involves a degree of independence from model development and use. Generally, validation is done by staff who are not responsible for model development or use and do not have a stake in whether a model is determined to be valid. As a practical matter, some validation work may be most effectively done by model developers and users; it is essential, however, that such validation work be subject to critical review by an independent party, who should conduct additional activities to ensure proper validation. Overall, the quality of the validation process is indicated by critical review by objective, knowledgeable parties and the actions taken to address issues identified by those parties.
Validation activities should continue on an ongoing basis after a model goes into use to track known model limitations and to identify any new ones. Validation is an important check during periods of benign economic and financial conditions, when estimates of risk and potential loss can become overly optimistic and the data at hand may not fully reflect more stressed conditions. Banking organizations should conduct a periodic review—at least annually but more frequently if warranted—of each model to determine whether it is working as intended and if the existing validation activities are sufficient. Key elements of comprehensive validation include:
Evaluation of Conceptual Soundness. This element involves assessing the quality of the model design and construction, as well as review of documentation and empirical evidence supporting the methods used and variables selected for the model. This step in validation should ensure that judgment exercised in model design and construction is well informed, carefully considered, and consistent with published research and with sound industry practice.
Ongoing Monitoring. This step in validation is done to confirm that the model is appropriately implemented and is being used and performing as intended. It is essential to evaluate whether changes in products, exposures, activities, clients, or market conditions necessitate adjustment, redevelopment, or replacement of the model and to verify that any extension of the model beyond its original scope is valid. Benchmarking can be used in this step to compare a given model’s inputs and outputs to estimates from alternatives.
Outcomes Analysis. This step involves comparing model outputs to corresponding actual outcomes. Back-testing is one form of outcomes analysis that involves the comparison of actual outcomes with model forecasts during a sample time period not used in model development at a frequency that matches the model’s forecast horizon or performance window.
The results of the three core elements of the validation process may reveal significant errors or inaccuracies in model development or outcomes that consistently fall outside the banking organization’s predetermined thresholds of acceptability. In such cases, model adjustment, recalibration, or redevelopment is warranted. At times, banking organizations may have a limited ability to use key model validation tools for various reasons, such as lack of data or of price observability. In those cases, even more attention should be paid to the model’s limitations when considering the appropriateness of model usage, and senior management should be fully informed of those limitations when using the models for decision-making. Generally, senior management should ensure that appropriate mitigating steps are taken in light of identified model limitations, which can include adjustments to model output, restrictions on model use, reliance on other models or approaches, or other compensating controls
Governance, Policies, and Controls
Developing and maintaining strong governance over the model risk management framework is fundamentally important to its effectiveness. Strong governance provides explicit support and structure to risk management functions through policies defining relevant risk management activities, procedures that implement those policies, allocation of resources, and mechanisms for testing that policies and procedures are being carried out as specified. Strong governance also includes documentation of model development and validation that is sufficiently detailed to allow parties unfamiliar with a model to understand how the model operates, as well as its limitations and key assumptions.
Model risk governance is provided at the highest level by the board of directors and senior management when they establish an organization-wide approach to model risk management. Board members should ensure that the level of model risk is within their tolerance. A banking organization’s internal audit function should assess the overall effectiveness of the model risk management framework, including the framework’s ability to address both types of model risk for individual models and in the aggregate. Whenever a banking organization uses external resources for model risk management, the organization should specify the activities to be conducted in a clearly written and agreed-upon scope of work, and those activities should be conducted in accordance with this guidance. Also, organizations should maintain an inventory of models implemented for use, under development for implementation, or recently retired.
All banking organizations should ensure that their internal policies and procedures are consistent with the risk management principles and supervisory expectations contained in this guidance.
Contacts
For questions regarding this guidance, please contact David Palmer, Senior Supervisory Financial Analyst, Risk, at (202) 452-2904; Dwight Smith, Senior Supervisory Financial Analyst, Capital & Regulatory Policy, at (202) 452-2773; or Anna Lee Hewko, Assistant Director, at (202) 530-6260. In addition, questions may be sent via the Board’s public website.4
signed by
Patrick M. Parkinson
Director
Division of Banking
Supervision and Regulation
Attachments:
Model Risk Management Guidance (PDF)
Cross References:
SR 09-1, “Application of the Market Risk Rule in Bank Holding Companies and State Member Banks”
Notes:
For instance, the OCC provided guidance on model risk, focusing on model validation, in OCC 2000-16 (May 30, 2000), other bulletins, and certain subject matter booklets of the Comptroller’s Handbook. The Federal Reserve issued SR Letter 09-01, “Application of the Market Risk Rule in Bank Holding Companies and State Member Banks,” which highlights various concepts pertinent to model risk management, including standards for validation and review, model validation documentation, and back-testing. The Federal Reserve’s Trading and Capital-Markets Activities Manual also discusses validation and model risk management. In addition, the advanced-approaches risk-based capital rules (12 CFR 3, Appendix C; 12 CFR 208, Appendix F; and 12 CFR 225, Appendix G) contain explicit validation requirements for subject banking organizations. Return to text
While outside the scope of this guidance, more qualitative approaches used by banking organizations—i.e., those not defined as models according to this guidance—should also be subject to a rigorous control process. Return to text
To the extent that models are used to generate amounts included in public financial statements, any adjustments for model uncertainty must comply with generally accepted accounting principles. Return to text
See http://www.federalreserve.gov/feedback.cfm. Return to text
如果有论文涉及风控模型案例复现需求同学，可以联系作者，我们提供专业指导和快速解决问题，节省你们大量调研时间。
论文复现-金融风控模型案例
版权声明：文章来自公众号(python风控模型),未经许可，不得抄袭。遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。