Big Data for Credit Risk Analysis: Efficient Machine Learning Models Using PySpark

EasyChair Preprint 8693

Abstract

Recently, Big Data has become an increasingly important source to support traditional credit scoring. Personal credit evaluation based on machine learning approaches focuses on the application data of clients in open banking and new banking platforms with challenges about Big Data quality and model risk. This paper represents a PySpark code for computationally efficient use of statistical learning and machine learning algorithms for the application scenario of personal credit evaluation with a performance comparison of models including logistic regression, decision tree, random forest, neural network, and support vector machine. The findings of this study reveal that the logistic regression methodology represents a more reasonable coefficient of determination and a lower false-negative rate than other models. Additionally, it is computationally less expensive and more comprehensible. Finally, the paper highlights the steps, perils, and benefits of using Big Data and machine learning algorithms in credit scoring.

Keyphrases: Big Data, Finance, credit score, machine learning, risk management

@booklet{EasyChair:8693,
  author    = {Afshin Ashofteh},
  title     = {Big Data for Credit Risk Analysis: Efficient Machine Learning Models Using PySpark},
  howpublished = {EasyChair Preprint 8693},
  year      = {EasyChair, 2022}}