Trending November 2023 # Bank Customer Churn Prediction Using Machine Learning # Suggested December 2023 # Top 16 Popular

You are reading the article Bank Customer Churn Prediction Using Machine Learning updated in November 2023 on the website Minhminhbmm.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested December 2023 Bank Customer Churn Prediction Using Machine Learning

This article was published as a part of the Data Science Blogathon.

Introduction

Customer Churn prediction means knowing which customers are likely to leave or unsubscribe from your service. For many companies, this is an important prediction. This is because acquiring new customers often costs more than retaining existing ones. Once you’ve identified customers at risk of churn, you need to know exactly what marketing efforts you should make with each customer to maximize their likelihood of staying.

Customers have different behaviors and preferences, and reasons for cancelling their subscriptions. Therefore, it is important to actively communicate with each of them to keep them on your customer list. You need to know which marketing activities are most effective for individual customers and when they are most effective.

Impact of customer churn on businesses

A company with a high churn rate loses many subscribers, resulting in lower growth rates and a greater impact on sales and profits. Companies with low churn rates can retain customers.

Why is Analyzing Customer Churn Prediction Important?

Customer churn is important because it costs more to acquire new customers than to sell to existing customers. This is the metric that determines the success or failure of a business. Successful customer retention increases the customer’s average lifetime value, making all future sales more valuable and improving unit margins.

The way to maximize a company’s resources is often by increasing revenue from recurring subscriptions and trusted repeat business rather than investing in acquiring new customers. Retaining loyal customers for years makes it much easier to grow and weather financial hardship than spending money to acquire new customers to replace those who have left.

Benefits of Analyzing Customer Churn Prediction

Increase profits

Improve the customer experience

One of the worst ways to lose a customer is an easy-to-avoid mistake like: Ship the wrong item. Understanding why customers churn, you can better understand their priorities, identify your weaknesses, and improve the overall customer experience.

Customer experience, also known as “CX”, is the customer’s perception or opinion of their interactions with your business. The perception of your brand is shaped throughout the buyer journey, from the first interaction to after-sales support, and has a lasting impact on your business, including your bottom line.

Optimize your products and services

Customer retention

The opposite of customer churn is customer retention. A company can retain customers and continue to generate revenue from them. High customer loyalty enables companies to increase the profitability of their existing customers and maximize their lifetime value (LTV).

If you sell a service for $1,000 per month and keep the customer for another 3 months, he will earn an additional $3,000 for each customer without spending on customer acquisition. The scope and amount vary depending on the business, but the concept of “repeat business = profitable business” is universal.

How does Customer Churn Prediction Work?

We first have to do some Exploratory Data Analysis in the Dataset, then fit the dataset into Machine Learning Classification Algorithm and choose the best Algorithm for the Bank Customer Churn Dataset.

Algorithms for Churn Prediction Models

XGBoost, short for Extreme Gradient Boosting, is a scalable machine learning library with Distributed Gradient Boosted Decision Trees (GBDT). It provides Parallel Tree Boosting and is the leading machine learning library for regression, classification and ranking problems. To understand XGBoost, it’s important first to understand the machine learning concepts and algorithms that XGBoost is built on: supervised machine learning, decision trees, ensemble learning, and gradient boosting. Supervised machine learning uses an algorithm to train a model to find patterns in a dataset containing labels and features and then uses the trained model to predict the labels of the features in a new dataset.

Decision trees are models that predict labels by evaluating a tree of if-then-else true/false functional questions and estimating the minimum number of questions needed to evaluate the likelihood of a correct decision. Decision trees can be used for classification to predict categories and regression to predict continuous numbers. The following simple example uses a decision tree to estimate a house’s price (tag) based on the size and number of bedrooms (features).

Gradient Boosted Decision Trees (GBDT) is a random forest-like decision tree ensemble learning algorithm for classification and regression. Ensemble learning algorithms combine multiple machine learning algorithms to get a better model. Both Random Forest and GBDT create models that consist of multiple decision trees. The difference is in how the trees are constructed and combined.

Source: researchgate.net

Decision Tree

Decision trees are a nonparametric supervised learning method used for classification and regression. The goal is to build a model that predicts the value of a target variable by learning simple decision rules derived from the properties of the data. A tree can be viewed as a piecewise constant approximation.

For example, in the following example, a decision tree learns from data to approximate a sine wave using a series of if-then-else decision rules. The deeper the tree, the more complex the decision rules and the better the model.

Easy to understand and easy to interpret. You can visualize trees.

Little or no data preparation is required. Other techniques often require normalizing the data, creating dummy variables, and removing empty values. However, please note that this module does not support missing values.

The cost of using a tree (predicting data) is the logarithm of the number of data points used to train the tree.

It can handle both numeric and categorical data. However, scikit-learn’s implementation does not currently support categorical variables. Other techniques tend to specialize in analyzing datasets containing only one variable type. See Algorithms for details. Can handle multi-output issues.

Adopted the white box model. If a given situation is observable in the model, the description of that state can be easily explained by Boolean logic. In contrast, results from black-box models (such as artificial neural networks) can be more difficult to interpret.

Possibility to validate the model with statistical tests. This can explain the reliability of the model.

It works well even when the assumptions are somewhat violated by the true model from which the data were generated.

Decision tree learners can create overly complex trees that fail to generalize the data well. This is called overfitting. Mechanisms such as pruning, setting a minimum number of samples required at a leaf node, or setting a maximum tree depth are required to avoid this problem.

Decision trees can be unstable. This is because small deviations in the data can produce completely different trees. This problem is mitigated by using decision trees within the ensemble. The figure above shows that the decision tree prediction is neither smooth nor continuous but a piecewise constant approximation. Therefore, they are bad at extrapolation.

The problem of learning optimal decision trees is known to be NP-complete under some aspects of optimality and even for simple concepts. Therefore, practical decision tree learning algorithms are based on heuristic algorithms, such as the greedy algorithm, where the locally optimal decision is made at each node. Such algorithms cannot guarantee to return of globally optimal decision trees. This can be mitigated by training multiple trees in an ensemble learner and using surrogates to randomly sample features and samples.

Some concepts, such as XOR, parity, and multiplexer problems, are difficult to master because they cannot be easily represented in decision trees.

Decision tree learners create skewed trees when some classes are dominant. Therefore, it is recommended to balance the data set before fitting the decision tree.

Random Forest

Random forest is a machine learning technique to solve regression and classification problems. It uses ensemble learning, a technique that combines many classifiers to provide solutions to complex problems.

A random forest algorithm consists of many decision trees. The “forest” created by the random forest algorithm is trained by bagging or bootstrap aggregation. Bagging is an ensemble meta-algorithm that improves the accuracy of machine learning algorithms. A (random forest) algorithm determines an outcome based on the predictions of a decision tree. Predict by averaging outputs from different trees. Increasing the number of trees improves the accuracy of the results.

Random forest removes the limitations of decision tree algorithms. Reduce data set overfitting and increase accuracy. Generate predictions without requiring a lot of configuration in your package

Source: trivusi.web.id Support Vector Machines

Source: researchgate.net

Effective in high-dimensional space.

It works even if the number of dimensions exceeds the number of samples.

It is also memory efficient because it uses a subset of the training points in the decision function (called support vectors).

Versatility: You can specify different kernel functions for the decision function. A generic kernel is provided, but it is possible to specify a custom kernel.

When the number of features is much larger than the number of samples, avoiding overfitting when choosing a kernel function, the regularization term becomes important.

SVM does not provide direct probability estimates. These are computed using an expensive 5-fold cross-validation.

Coding to Predict Bank Customer Churn Prediction

Now we have to import some libraries :

After that, we have to read the dataset using pandas

Exploratory Data Analysis

The first thing we have to do in Exploratory Data Analysis is checked if there are null values in the dataset.

df.isnull().head() df.isnull().sum() #Checking Data types df.dtypes #Counting 1 and 0 Value in Churn column color_wheel = {1: "#0392cf", 2: "#7bc043"} colors = df["churn"].map(lambda x: color_wheel.get(x + 1)) print(df.churn.value_counts()) p=df.churn.value_counts().plot(kind="bar") #Change value in country column df['country'] = df['country'].replace(['Germany'],'0') df['country'] = df['country'].replace(['France'],'1') df['country'] = df['country'].replace(['Spain'],'2') #Change value in gender column df['gender'] = df['gender'].replace(['Female'],'0') df['gender'] = df['gender'].replace(['Male'],'1') df.head() #convert object data types column to integer df['country'] = pd.to_numeric(df['country']) df['gender'] = pd.to_numeric(df['gender']) df.dtypes #Remove customer_id column df2 = df.drop('customer_id', axis=1) df2.head() sns.heatmap(df2.corr(), fmt='.2g') Build Machine Learning Model X = df2.drop('churn', axis=1) y = df2['churn'] #test size 20% and train size 80% from sklearn.model_selection import train_test_split, cross_val_score, cross_val_predict from sklearn.metrics import accuracy_score X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.2,random_state=7) Decision Tree from chúng tôi import DecisionTreeClassifier dtree = DecisionTreeClassifier() dtree.fit(X_train, y_train) y_pred = dtree.predict(X_test) print("Accuracy Score :", accuracy_score(y_test, y_pred)*100, "%") Random Forest from sklearn.ensemble import RandomForestClassifier rfc = RandomForestClassifier() rfc.fit(X_train, y_train) y_pred = rfc.predict(X_test) print("Accuracy Score :", accuracy_score(y_test, y_pred)*100, "%") Support Vector Machine from sklearn import svm svm = svm.SVC() svm.fit(X_train, y_train) y_pred = svm.predict(X_test) print("Accuracy Score :", accuracy_score(y_test, y_pred)*100, "%") XGBoost from xgboost import XGBClassifier xgb_model = XGBClassifier() xgb_model.fit(X_train, y_train) y_pred = xgb_model.predict(X_test) print("Accuracy Score :", accuracy_score(y_test, y_pred)*100, "%") Visualize Random Forest and XGBoost Algorithm because Random Forest and XGBoost Algorithm have the Best Accuracy #importing classification report and confusion matrix from sklearn from sklearn.metrics import classification_report, confusion_matrix Random Forest y_pred = rfc.predict(X_test) print("Classification report - n", classification_report(y_test,y_pred)) cm = confusion_matrix(y_test, y_pred) plt.figure(figsize=(5,5)) sns.heatmap(data=cm,linewidths=.5, annot=True,square = True, cmap = 'Blues') plt.ylabel('Actual label') plt.xlabel('Predicted label') all_sample_title = 'Accuracy Score: {0}'.format(rfc.score(X_test, y_test)) plt.title(all_sample_title, size = 15) from sklearn.metrics import roc_curve, roc_auc_score y_pred_proba = rfc.predict_proba(X_test)[:][:,1] df_actual_predicted = pd.concat([pd.DataFrame(np.array(y_test), columns=['y_actual']), pd.DataFrame(y_pred_proba, columns=['y_pred_proba'])], axis=1) df_actual_predicted.index = y_test.index fpr, tpr, tr = roc_curve(df_actual_predicted['y_actual'], df_actual_predicted['y_pred_proba']) auc = roc_auc_score(df_actual_predicted['y_actual'], df_actual_predicted['y_pred_proba']) plt.plot(fpr, tpr, label='AUC = %0.4f' %auc) plt.plot(fpr, fpr, linestyle = '--', color='k') plt.xlabel('False Positive Rate') plt.ylabel('True Positive Rate') plt.title('ROC Curve', size = 15) plt.legend() XGBoost y_pred = xgb_model.predict(X_test) print("Classification report - n", classification_report(y_test,y_pred)) cm = confusion_matrix(y_test, y_pred) plt.figure(figsize=(5,5)) sns.heatmap(data=cm,linewidths=.5, annot=True,square = True, cmap = 'Blues') plt.ylabel('Actual label') plt.xlabel('Predicted label') all_sample_title = 'Accuracy Score: {0}'.format(xgb_model.score(X_test, y_test)) plt.title(all_sample_title, size = 15) from sklearn.metrics import roc_curve, roc_auc_score y_pred_proba = xgb_model.predict_proba(X_test)[:][:,1] df_actual_predicted = pd.concat([pd.DataFrame(np.array(y_test), columns=['y_actual']), pd.DataFrame(y_pred_proba, columns=['y_pred_proba'])], axis=1) df_actual_predicted.index = y_test.index fpr, tpr, tr = roc_curve(df_actual_predicted['y_actual'], df_actual_predicted['y_pred_proba']) auc = roc_auc_score(df_actual_predicted['y_actual'], df_actual_predicted['y_pred_proba']) plt.plot(fpr, tpr, label='AUC = %0.4f' %auc) plt.plot(fpr, fpr, linestyle = '--', color='k') plt.xlabel('False Positive Rate') plt.ylabel('True Positive Rate') plt.title('ROC Curve', size = 15) plt.legend() Conclusion

The churn variable has imbalanced data. So, the solution to handle imbalanced data are :

Resample Training set

Use K-fold Cross-Validation in the Right Way

Ensemble Different Resampled Datasets

We have to change the value in the Country and Gender columns so the Machine Learning model can read and predict the dataset; after changing the value, we have to change the data types on the Country and Gender column from string to integer because XGBoost Machine Learning Model cannot read string data types even though the value in the column is number.

Lastly, X (86,85% and 86.45%). Random Forest and XGBoost have perfect AUC Scores. They have 0.8731 and 0.8600 AUC Scores.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

You're reading Bank Customer Churn Prediction Using Machine Learning

Room Occupancy Detection Using Machine Learning Algorithms

This article was published as a part of the Data Science Blogathon

In this article, we will see how we can detect room occupancy using environmental variables data with machine learning algorithms. For this purpose, I am using Occupancy Detection Dataset from UCI ML Repository. Here, Ground-truth occupancy is obtained from time-stamped pictures of environmental variables like temperature, humidity, light, CO2 were taken every minute. The implementation of an ML algorithm instead of a physical PIR sensor will be cost and maintenance-free. This might be useful in the field of HVAC (Heating, Ventilation, and Air Conditioning).

Data Understanding and EDA

Here we are using R for ML programming. The dataset zip has 3 text files, one for training the model and two for testing the model. For reading these files in R we use chúng tôi and to explore data structure, data dimensions, and 5 point statics of dataset we use the “summarytools” package. Images that are included here are captured from the R console while executing the code.

data= read.csv("datatrain.txt",header = T, sep = ",", row.names = 1) View(data) library(summarytools) summarytools::view(dfSummary(data))

Data Summary

Observations

All environmental variables are read correctly as numeric type by R, however, we need to define a variable “date” as of date class type. For the ‘Date’ variable treatment, we use the “lubridate” package. Also, we have daytime information available in the date column which we can extract and use for modeling as occupancy for spaces like offices will depend on daytime. Another important observation is that we don’t have missing values in the entire dataset. An Occupancy variable needs to be defined as a factor type for further analysis.

library("readr") library("lubridate") data$date1 = as_date(data$date) data$date= as.POSIXct(data$date, format = "%Y-%m-%d %H:%M:%S") data$time = format(data$date, format = "%H:%M:%S") data1= data[,-1] data1$Occupancy = as.factor(data1$Occupancy)

Now our processed data looks like this:

Processed Data Preview

Next, we check two important aspects of data, one is the correlation plot of variables to understand multicollinearity issues in the dataset and the proportion of target variable distribution.

library(corrplot) numeric.list <- sapply(data1, is.numeric) numeric.list sum(numeric.list) numeric.df <- data1[, numeric.list] cor.mat <- cor(numeric.df) corrplot(cor.mat, type = "lower", method = "number")

Correlation Plot

library(plotrix) pie3D(prop.table((table(data1$Occupancy))), main = "Occupied Vs Unoccupied", labels=c("Unoccupied","Occupied"), col = c("Blue", "Dark Blue")) Pie Chart for Occupancy

From the correlation plot, we observe that temperature and light are positively correlated while temperature and Humidity are negatively correlated, humidity and humidity ratio are highly correlated which is obvious as the Humidity ratio is the ratio of Humidity and temperature. Hence while building various models we will consider variable Humidity and omit Humidity ratio.

Now we are all set for model building. As our response variable Occupancy is binary, we need classification types of models. Here we implement CART, RF, and ANN.

Model Building- Classification and Regression Trees (CART): Now we define training and test datasets in the required format for model building.

p_train = data1 p_test = read.csv("datatest2.txt",header = T, sep = ",", row.names = 1) p_test$date1 = as_date(p_test$date) p_test$date= as.POSIXct(p_test$date, format = "%Y-%m-%d %H:%M:%S") p_test$time = format(p_test$date, format = "%H:%M:%S") p_test= p_test[,-1]

Note that the R implementation of the CART algorithm is called RPART (Recursive Partitioning And Regression Trees) available in a package of the same name. The algorithm of decision tree models works by repeatedly partitioning/splitting the data into multiple sub-spaces so that the outcomes in each final sub-space are as homogeneous as possible.

The model uses different splitting rules that can be used to effectively predict the type of outcome. These rules are produced by repeatedly splitting the predictor variables, starting with the variable that has the highest association with the response variable. The process continues until some predetermined stopping criteria are met. We define these stopping criteria using control parameters such as a minimum number of observations in a node of the tree before attempting a split, a split must decrease the overall lack of fit by a factor before being attempted.

library(rpart) library(rpart.plot) library(rattle) #Setting the control parameters r.ctrl = rpart.control(minsplit=100, minbucket = 10, cp = 0, xval = 10) #Building the CART model set.seed(123) m1 <- rpart(formula = Occupancy~ Temperature+Humidity+Light+CO2+date1+time, data = p_train, method = "class", control = r.ctrl) #Displaying the decision tree fancyRpartPlot(m1).

Decision Tree

Now we predict the occupancy variable for the test dataset using predict function.

p_test$predict.class1 <- predict(ptree, p_test[,-6], type="class") p_test$predict.score1 <- predict(ptree, p_test[,-6], type="prob") View(p_test)

Test Data Preview with Actual and Predicted Occupancy

Now evaluate the performance of our model by plotting the ROC curve and building a confusion matrix.

library(ROCR) pred <- prediction(p_test$predict.score1[,2], p_test$Occupancy) perf <- performance(pred, "tpr", "fpr") plot(perf,main = "ROC curve") auc1=as.numeric(performance(pred, "auc")@y.values) library(caret) m1=confusionMatrix(table(p_test$predict.class1,p_test$Occupancy), positive="1",mode="everything")

Here we get an AUC of 82% and a model accuracy of 98.1%.

Now next in the list is Random Forest. In the random forest approach, a large number of decision trees are created. Every observation is fed into every decision tree. The most common outcome for each observation is used as the final output. A new observation is fed into all the trees and taking a majority vote for each classification model. The R package “randomForest” is used to create random forests.

RFmodel = randomForest(Occupancy~Temperature+Humidity+Light+CO2+date1+time, data = p_train1, mtry = 5, nodesize = 10, ntree = 501, importance = TRUE) print(RFmodel) plot(RFmodel)

Error Vs No of trees plot

Here we observe that error remains constant after n=150, so we can tune the model with trees 150.

Also, we can have a look at important variables in the model which are contributing to occupancy detection.

importance(RFmodel)

Variable Importance table

We observe from the above output that light is the most important predictor followed by CO2, Temperature, Time, Humidity, and date when we consider accuracy. Now, let’s prune the RF model with new control parameters.

set.seed(123) tRF=tuneRF(x=p_train1[,-c(5,6)],y=as.factor(p_train1$Occupancy), mtryStart = 5, ntreeTry = 150, stepFactor = 1.15, improve = 0.0001, trace = TRUE, plot = TRUE, doBest = TRUE, nodesize=10, importance= TRUE)

Now with this tuned model we again variable importance as follows. Please note here variable importance is measured for decreasing Gini Index, however earlier it was a mean decrease in model accuracy.

varImpPlot(tRF,type=2, main = "Important predictors in the analysis")

Variable Importance Plot

Now next we predict occupancy for the test dataset using predict function and tuned RF model.

p_test1$predict.class= predict(tRF, p_test1, type= "class") p_test1$predict.score= predict(tRF, p_test1, type= "prob")

We check the performance of this model using the ROC curve and confusion matrix parameters. The AUC turns out to be 99.13% with a very stiff curve as below and the accuracy of prediction is 98.12%.

ROC Curve

It seems like this model is doing better than the CART model. Time to check ANN!

Now we build an artificial neural network for the classification. Neural Network (or Artificial Neural Network) has the ability to learn by examples. ANN is an information processing model inspired by the biological neuron system. It is composed of a large number of highly interconnected processing elements known as the neuron to solve problems.

Here I am using the ‘neuralnet’ package in R. When we try to build ANN for our case, we observe that the model does not accept date class variables we will omit them. Another way could be we create a separate factor variable from the daytime variable with levels like Morning, Afternoon, Evening, and Night and then create a dummy variable for the factor variable.

Before actually building ANN, we need to scale our data as variables have values in different ranges. ANN being a weight-based algorithm, maybe biased results if data is not scaled.

p_train2=p_train2[,-c(6,7,8)] p_train_sc=scale(p_train2) p_train_sc = as.data.frame(p_train_sc) p_train_sc$Occupancy=data1$Occupancy p_test3=p_test2[,-c(6,7,8)] p_test_sc=scale(p_test3) p_test_sc = as.data.frame(p_test_sc) p_test_sc$Occupancy=p_test2$Occupancy

After scaling all the variables (except Occupancy) our data will look like this.

Data Preview After Scaling

Now we are all set for model building.

nn1 = neuralnet(formula = Occupancy~Temperature+Humidity+Light+CO2+HumidityRatio, data = p_train_sc, hidden = 3, chúng tôi = "sse",linear.output = FALSE,lifesign = "full", chúng tôi = 10, threshold = 0.03, stepmax = 10000) plot(nn1)

We calculate results for the test dataset using Compute function.

compute.output = compute(nn1, p_test_sc[,-6]) p_test_sc$Predict.score <- compute.output$net.result

Models Performance Comparison: We again evaluate this model using the confusion matrix and ROC curve. I have tabulated results obtained from all three models as follows:

Performance measure CART on a test dataset RF on a test dataset ANN on a test dataset

AUC 0.8253 0.9913057 0.996836

Accuracy 0.981 0.9812 0.9942

Kappa 0.9429 0.9437 0.9825

Sensitivity 0.9514 0.9526 0.9951

Specificity 0.9889 0.9889 0.9939

Precision 0.9586 0.9587 0.9775

Recall 0.9514 0.9526 0.9951

F1 0.9550 0.9556 0.9862

Balanced Accuracy 0.9702 0.9708 0.9945

From performance measures comparison, we observe that ANN outperforms other models, followed by RF and CART. With machine earning algorithms we can replace occupancy sensor functionality efficiently with good accuracy.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

How Deepmind Strengthens Ecological Research Using Machine Learning?

In the past few years, machine learning has highlighted the strength of information technology in discovering the fundamentals of life and the environment. With the volume of data generated every day, it has become necessary to acknowledge smart analysis solutions to know more about the world we live in. In particular, machine learning aims to develop classifying languages simple enough to be understood easily by the human. Moreover, to get into layers of detailed study of ecology, Google’s DeepMind has collaborated with ecologists and conservationists to develop machine learning methods to help study the behavioral dynamics of an entire African animal community in the Serengeti National Park and Grumeti Reserve in Tanzania. According to a DeepMind’s blog, the Serengeti-Mara ecosystem is globally unparalleled in its biodiversity, hosting an estimated 70 large mammal species and 500 bird species, thanks in part to its unique geology and varied habitat types. Around 10 years ago, the Serengeti Lion Research program installed hundreds of motion-sensitive cameras within the core of the protected area which is triggered by passing wildlife, capturing animal images frequently, across vast spatial scales, allowing researchers to study animal behavior, distribution, and demography with great spatial and temporal resolution. This has allowed the team to collect and store millions of photos. To date, volunteers from across the world have helped to identify and count the species in the photos by hand using the Zooniverse web-based platform, which hosts many similar projects for citizen-scientists. This comprehensive study has resulted in a rich dataset, Snapshot Serengeti, featuring labels and counts for around 50 different species. Moreover, to help researchers unlock this data with greater efficiency, DeepMind has used the Snapshot Serengeti dataset to train machine learning models to automatically detect, identify, and count animals. DeepMind says, “Camera trap data can be hard to work with–animals may appear out of focus, and can be at many different distances and positions with respect to the camera. With expert input from leading ecologist and conservationist Dr. Meredith Palmer, our project quickly took shape, and we now have a model that can perform on par with, or better than, human annotators for most of the species in the region.” Most importantly, this method shortens the data processing pipeline by up to 9 months, which has immense potential to help researchers in the field. In a more obvious manner the field work is quite challenging, and it is fraught with unexpected hazards such as failing power lines and limited or no internet access. DeepMind is preparing the software for deployment in the field and looking at ways to safely run its pre-trained model with modest hardware requirements and little Internet access. The company has worked closely with its collaborators in the field to be sure that its technology is used responsibly. Once in place, researchers in the Serengeti will be able to make direct use of this tool, helping provide them with up-to-date species information to better support their conservation efforts.

Pokemon Prediction Using Random Forest

This dataset has 721 unique values i.e. it has features of 721 unique pokemon; for further details, visit this link.

import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt %matplotlib inline from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score, classification_report from sklearn.ensemble import RandomForestClassifier

Reading the dataset

pokemon_data = pd.read_csv('Pokemon Data.csv')

Now, let’s see what our dataset has in it!

poke = pd.DataFrame(pokemon_data) poke.head()

Output:

Checking out folet’sl values

poke.isnull().sum()

Output:

Number 0 Name 0 Type_1 0 Type_2 371 Total 0 HP 0 Attack 0 Defense 0 Sp_Atk 0 Sp_Def 0 Speed 0 Generation 0 isLegendary 0 Color 0 hasGender 0 Pr_Male 77 Egg_Group_1 0 Egg_Group_2 530 hasMegaEvolution 0 Height_m 0 Weight_kg 0 Catch_Rate 0 Body_Style 0 dtype: int64

We have seen the null values in its users n; let’s visualize them using the heatmap.

plt.figure(figsize=(10,7)) sns.heatmap(poke.isnull(), cbar=False)

Output:

Here it’s visible that Type_2, Pr_Male, and Egg_Group_2 have relatively null values.

We have visualized the nucan’tlues using the heatmap but in that kind of visualization, we can’t get the count of Let’s null values, so we are using the dist-plot.

plt.figure(figsize=(20,20)) sns.displot( data=poke.isna().melt(value_name="missing"), y="variable", hue="missing", multiple="fill", aspect=2 )

Output:

Let’s know the dimensions of our dataset.

poke.shape

Output:

(721, 23)

From the shape, it is clear the dataset is small, meaning we can remove the null values columns as filling them can make the dataset a little biased.

We have seen that type_2, egg_group_2, and Pr_male have null values.

poke['Pr_Male'].value_counts()

Output:

0.500 458 0.875 101 0.000 23 0.250 22 0.750 19 1.000 19 0.125 2 Name: Pr_Male, dtype: int64

Since Type_2 and Egg_group_2 columns have so many NULL values we will be removing those columns, you won’t impute them with other methods, but for simplicity, we won’t do that here. We only set the Pr_Male column since it had only 77 missing values.

poke['Pr_Male'].fillna(0.500, inplace=True) poke['Pr_Male'].isnull().sum()

Output:

0 # as we can see that there are no null values now.

Dropping unnecessary columns

new_poke = poke.drop(['Type_2', 'Egg_Group_2'], axis=1)

Now let’s understand the type of each column and its values.

new_poke.describe()

Output:

plt.figure(figsize=(10,10)) sns.heatmap(new_poke.corr(),annot=True,cmap='viridis',linewidths=.5)

Output:

The above is a correlation graph that tells you how much a feature is correlated to another since a high correlation means one of the two features does not speak much to the model when predicting.

Usually, it is to be determined by you itself for the high value of correlation and removed.

From the above table, it is clear that different features have different ranges of value, which creates complexity for the model, so we tone them down usually using StandardScalar() class which we will do later on.

new_poke['Type_1'].value_counts()

Output:

Water 105 Normal 93 Grass 66 Bug 63 Psychic 47 Fire 47 Rock 41 Electric 36 Ground 30 Poison 28 Dark 28 Fighting 25 Dragon 24 Ice 23 Ghost 23 Steel 22 Fairy 17 Flying 3 Name: Type_1, dtype: int64

Value counts of all the generations

new_poke['Generation'].value_counts()

Output:

5 156 1 151 3 135 4 107 2 100 6 72 Name: Generation, dtype: int64 Visualizing I’me categorical values

Here for visualizing the categorical data, I’m using seaborn’s cat plot() function. Well, one can use the line plot scatter plot or box plot separately, but here, the cat plot brings up the unified version of using all the plots hence I preferred the cat plot rather than the separate version of eI’m plot.

Here for counting each type (6) category of generations, I’m using the cougeneration’snd in the cat plot to get the number of count of each generation’s column.

sns.catplot(x="Generation",kind="count",palette="ch:.25", data=poke)

Output:

Inference: In the above graph, the 5th generation is the most in numbers.

Here we are using the default kind of cat plot, i.e. scatter plot to plot the Generation vs Defense graph where we will be able to figure outPokemonlationship between the defence power of each general Pokemon.

sns.catplot(x="Generation", y="Defense", data=poke)

Output:

Inference: Here, we can see that only two pcan’tn in generation 2 have the highest defence capability. Still, we can’t conclude that generation 2 has the most increased defence capabilities as the outliers. Still, in the graph, it is evident that generation 6 and 4 has the highest defence capabilities.

Here we are using the Box plot because boxplot will help us understand the variations in the large dataset better; it will also let us know about the outliers more clearly.

sns.catplot(x="Generation", y="Attack",kind="boxen", data=poke)

Output:

Here in the above boxplot, we can see that there are a lot of outliers in generation 4 and generation 1 when it comes to attacking capabilities.

Also, generation 4 has the highest median values of their attacking capabilities than all the other generations.

Now we are using bar kind via cat plot, which will let us know about the Attacking capabilities of different generations based on their Pokemon. For example, in generation 1, the pokemon power of male Pokemon are higher than those of the female Pokemon of the same generation. Still, that generation also has the least attacking power than other generations.

sns.catplot(x="Generation", y="Attack",kind='bar',hue='hasGender', data=poke)

Output:

FromPokemonove graph, we can conclude that,

In generaPokemononly the male Pokemon has more attacking power than the female Pokemon, which contradicts other generations.

Generation 6 has the highest attacking power wLet’sgeneration 1 has the lowest attacking power.

new_poke['Color'].value_counts()

Output:

Blue 134 Brown 110 Green 79 Red 75 Grey 69 Purple 65 Yellow 64 White 52 Pink 41 Black 32 Name: Color, dtype: int64 new_poke['Egg_Group_1'].value_counts()

Output:

Field 169 Monster 74 Water_1 74 Undiscovered 73 Bug 66 Mineral 46 Flying 44 Amorphous 41 Human-Like 37 Fairy 30 Grass 27 Water_2 15 Water_3 14 Dragon 10 Ditto 1 Name: Egg_Group_1, dtype: int64

Let’s also consider the number of values in our target column

new_poke['isLegendary'].value_counts()

Output:

False 675 True 46 Name: isLegendary, dtype: int64 Feature Engineering

This may seem uncomfortable to some, but you will get why I did it like that.

poke_type1 = new_poke.replace(['Water', 'Ice'], 'Water') poke_type1 = poke_type1.replace(['Grass', 'Bug'], 'Grass') poke_type1 = poke_type1.replace(['Ground', 'Rock'], 'Rock') poke_type1 = poke_type1.replace(['Psychic', 'Dark', 'Ghost', 'Fairy'], 'Dark') poke_type1 = poke_type1.replace(['Electric', 'Steel'], 'Electric') poke_type1['Type_1'].value_counts()

Output:

Grass 129 Water 128 Dark 115 Normal 93 Rock 71 Electric 58 Fire 47 Poison 28 Fighting 25 Dragon 24 Flying 3 Name: Type_1, dtype: int64 ref1 = dict(poke_type1['Body_Style'].value_counts()) poke_type1['Body_Style_new'] = poke_type1['Body_Style'].map(ref1)

You may be wondering what I did; I took the value counts of each body tyLet’sd replace the body type with the numbers; see below

poke_type1['Body_Style_new'].head()

Output:

0 135 1 135 2 135 3 158 4 158 Name: Body_Style_new, dtype: int64

Let’s look towards the Body_style

poke_type1['Body_Style'].head()

Output:

0 quadruped 1 quadruped 2 quadruped 3 bipedal_tailed 4 bipedal_tailed Name: Body_Style, dtype: object Encoding data – features like Type_1 and Color types_poke = pd.get_dummies(poke_type1['Type_1']) color_poke = pd.get_dummies(poke_type1['Color']) X = pd.concat([poke_type1, types_poke], axis=1) X = pd.concat([X, color_poke], axis=1) X.head()

Output:

Now we have built some features and extracted some feature data, what’s left is to remove redundant features

X.columns

Output:

Index(['Number', 'Name', 'Type_1', 'Total', 'HP', 'Attack', 'Defense', 'Sp_Atk', 'Sp_Def', 'Speed', 'Generation', 'isLegendary', 'Color', 'hasGender', 'Pr_Male', 'Egg_Group_1', 'hasMegaEvolution', 'Height_m', 'Weight_kg', 'Catch_Rate', 'Body_Style', 'Body_Style_new', 'Dark', 'Dragon', 'Electric', 'Fighting', 'Fire', 'Flying', 'Grass', 'Normal', 'Poison', 'Rock', 'Water', 'Black', 'Blue', 'Brown', 'Green', 'Grey', 'Pink', 'Purple', 'Red', 'White', 'Yellow'], dtype='object')

X_.shape

Output:

(721, 38)

Now, let’s see the shape of our updated feature columns

X.shape

Lastly, we define our target variable and set it into a variable called y

y = X_['isLegendary'] X_final = X_.drop(['isLegendary', 'Body_Style'], axis = 1) X_final.columns

Output:

Index(['Total', 'HP', 'Attack', 'Defense', 'Sp_Atk', 'Sp_Def', 'Speed', 'Generation', 'hasGender', 'Pr_Male', 'hasMegaEvolution', 'Height_m', 'Weight_kg', 'Catch_Rate', 'Body_Style_new', 'Dark', 'Dragon', 'Electric', 'Fighting', 'Fire', 'Flying', 'Grass', 'Normal', 'Poison', 'Rock', 'Water', 'Black', 'Blue', 'Brown', 'Green', 'Grey', 'Pink', 'Purple', 'Red', 'White', 'Yellow'], dtype='object') X_final.head()

Output:

Creating and training our model

Splitting the dataset into training and testing dataset

Xtrain, Xtest, ytrain, ytest = train_test_split(X_final, y, test_size=0.2)

Using random forest classifier for training our model

random_model = RandomForestClassifier(n_estimators=500, random_state = 42)

Fitting the model

model_final = random_model.fit(Xtrain, ytrain) y_pred = model_final.predict(Xtest)

Checking the accuracy

random_model_accuracy = round(model_final.score(Xtrain, ytrain)*100,2) print(round(random_model_accuracy, 2), '%')

Output:

100.0 %

Getting the accuracy of the model

random_model_accuracy1 = round(random_model.score(Xtest, ytest)*100,2) print(round(random_model_accuracy1, 2), '%')

Output:

99.31 %

Saving the model to disk

import pickle filename = 'pokemon_model.pickle' pickle.dump(model_final, open(filename, 'wb'))

Load the model from the disk

filename = 'pokemon_model.pickle' loaded_model = pickle.load(open(filename, 'rb')) result = loaded_model.score(Xtest, ytest) result*100

Output:

99.3103448275862 Conclusion

Here I conclude the legendary pokemon prediction with 99% accuracy; this might be a overfit model; having said that, the dataset was not so complex that it will lead to such a situaHere’set all the suggestions and improvements are always welcome.

Here’s the repo link to this article.

Here you can access my other articles, which are published on Analytics Vidhya as a part of the Blogathon (link)

If got any queries you can connect with I’m on LinkedIn, refer to this link

About me

Greeting to everyone, I’m currently working in TCS and previously, I worked as a Data Science AssociI’veAnalyst in Zorba Consulting India. Along with full-time work, I’ve got an immense interest in the same field, i.e. Data Science, along with its other subsets of Artificial Intelligence such as Computer Vision, Machine learning, and Deep learning; feel free to collaborate with me on any project on the domains mentioned above (LinkedIn).

The media shown in this article is not owned by Analytics Vidhya and are used at the Author’s discretion.

Calibration Of Machine Learning Models

This article was published as a part of the Data Science Blogathon.

Introduction

source: iPhone Weather App

A screen image related to a weather forecast must be a familiar picture to most of us. The AI Model predicting the expected weather predicts a 40% chance of rain today, a 50% chance of Wednesday, and a 50% on Thursday. Here the AI/ML Model is talking about the probability of occurrence, which is the interesting part. Now, the question is this AI/ML model trustworthy?

As learners of Data Science/Machine Learning, we would have walked through stages where we build various supervisory ML Models(both classification and regression models). We also look at different model parameters that tell us how well the model performs. One important but probably not so well-understood model reliability parameter is Model Calibration. The calibration tells us how much we can trust a model prediction. This article explores the basics of model calibration and its relevancy in the MLOps cycle. Even though Model Calibration applies to regression models as well, we will exclusively look at classification examples to get a grasp on the basics.

The Need for Model Calibration

Wikipedia amplifies calibration as ” In measurement technology and metrology, calibration is the comparison of measurement values delivered by a device under test with those of a calibration standard of known accuracy. “

The model outputs two important pieces of information in a typical classification ML model. One is the predicted class label (for example, classification as spam or not spam emails), and the other is the predicted probability. In binary classification, the sci-kit learn library gives a method called the model.predict_proba(test_data) that gives us the probability for the target to be 0 and 1 in an array form. A model predicting rain can give us a 40% probability of rain and a 60% probability of no rain. We are interested in the uncertainty in the estimate of a classifier. There are typical use cases where the predicted probability of the model is very much of interest to us, such as weather models, fraud detection models, customer churn models, etc. For example, we may be interested in answering the question, what is the probability of this customer repaying the loan?

Let’s say we have an ML model which predicts whether a patient has cancer-based on certain features. The model predicts a particular patient does not have cancer (Good, a happy scenario!). But if the predicted probability is 40%, then the Doctor may like to conduct some more tests for a certain conclusion. This is a typical scenario where the prediction probability is critical and of immense interest to us. The Model Calibration helps us improve the model’s prediction probability so that the model’s reliability improves. It also helps us to decipher the predicted probability observed from the model predictions. We can’t take for granted that the model is twice as confident when giving a predicted probability of 0.8 against a figure of 0.4.

We also must understand that calibration differs from the model’s accuracy. The model accuracy is defined as the number of correct predictions divided by the total number of predictions made by the model. It is to be clearly understood that we can have an accurate but not calibrated model and vice versa.

If we have a model predicting rain with 80% predicted probability at all times, then if we take data for 100 days and find 80 days are rainy, we can say that model is well calibrated. In other words, calibration attempts to remove bias in the predicted probability.

Consider a scenario where the ML model predicts whether the user who is making a purchase on an e-commerce website will buy another associated item or not. The model predicts the user has a probability of 68% for buying Item A and an item B probability of 73%. Here we will present Item B to the user(higher predicted probability), and we are not interested in actual figures. In this scenario, we may not insist on strict calibration as it is not so critical to the application.

The following shows details of 3 classifiers (assume that models predict whether an image is a dog image or not). Which of the following model is calibrated and hence reliable?

(a) Model 1 : 90% Accuracy, 0.85 confidence in each prediction

(b) Model 2 : 90% Accuracy, 0.98 confidence in each prediction

If we look at the first model, it is underconfident in its prediction, whereas model 2 seems overconfident. Model 3 seems well-calibrated, giving us confidence in the model’s ability. Model 3 thinks it is correct 91% of the time and 90% of the time, which shows good calibration.

Reliability Curves

The model’s calibration can be checked by creating a calibration plot or Reliability Plot. The calibration plot reveals the disparity between the probability predicted by the model and the true class probabilities in the data. If the model is well calibrated, we expect to see a straight line at 45 degrees from the origin (indicative that estimated probability is always the same as empirical probability ).

We will attempt to understand the calibration plot using a toy dataset to concretize our understanding of the subject.

Source: own-document

The resulting probability is divided into multiple bins representing possible ranges of outcomes. For example, [0-0.1), [0.1-0.2), etc., can be created with 10 bins. For each bin, we calculate the percentage of positive samples. For a well-calibrated model, we expect the percentage to correspond to the bin center. If we take the bin with the interval [0.9-1.0), the bin center is 0.95, and for a well-calibrated model, we expect the percentage of positive samples ( samples with label 1) to be 95%.

Source: self-document

We can plot the Mean predicted value (midpoint of the bin ) vs. Fraction of TRUE Positives in each bin in a line plot to check the calibration. of the model.

We can see the difference between the ideal curve and the actual curve, indicating the need for our model to be calibrated. Suppose the points obtained are below the diagonal. In that case, it indicates that the model has overestimated (model predicted probabilities are too high). If the points are above the diagonal, it can be estimated that model has been underconfident in its predictions(the probability is too small). Let’s also look at a real-life Random Forest Model curve in the image below.

If we look at the above plot, the S curve ( remember the sigmoid curve seen in Logistic Regression !) is observed commonly for some models. The Model is seen to be underconfident at high probabilities and overconfident when predicting low probabilities. For the above curve, for the samples for which the model predicted probability is 30%, the actual value is only 10%. So the Model was overestimating at low probabilities.

The toy dataset we have shown above is for understanding, and in reality, the choice of bin size is dependent on the amount of data we have, and we would like to have enough points in each bin such that the standard error on the mean of each bin is small.

Brier Score

We do not need to go for the visual information to estimate the Model calibration. The calibration can be measured using the Brier Score. The Brier score is similar to the Mean Squared Error but used slightly in a different context. It takes values from 0 to 1, with 0 meaning perfect calibration, and the lower the Brier Score, the better the model calibration.

The Brier score is a statistical metric used to measure probabilistic forecasts’ accuracy. It is mostly used for binary classification.

Let’s say a probabilistic model predicts a 90% chance of rain on a particular day, and it indeed rains on that day. The Brier score can be calculated using the following formula,

Brier Score = (forecast-outcome)2

Brier Score in the above case is calculated to be (0.90-1)2 = 0.01.

The Brier Score for a set of observations is the average of individual Brier Scores.

On the other hand, if a model predicts with a 97% probability that it will rain but does not rain, then the calculated Brier Score, in this case, will be,

Brier Score = (0.97-0)2 = 0.9409 . A lower Brier Score is preferable.

Calibration Process

Now, let’s try and get a glimpse of how the calibration process works without getting into too many details.

Some algorithms, like Logistic Regression, show good inherent calibration standards, and these models may not require calibration. On the other hand, models like SVM, Decision Trees, etc., may benefit from calibration. The calibration is a rescaling process after a model has made the predictions.

There are two popular methods for calibrating probabilities of ML models, viz,

(a) Platt Scaling

(b) Isotonic Regression

It is not the intention of this article to get into details of the mathematics behind the implementation of the above approaches. However, let’s look at both methods from a ringside perspective.

The Platt Scaling is used for small datasets with a reliability curve in the sigmoid shape. It can be loosely understood as putting a sigmoid curve on top of the calibration plot to modify the predictive probabilities of the model.

The above images show how imposing a Platt calibrator curve on the reliability curve of the model modifies the curve. It is seen that the points in the calibration curve are pulled toward the ideal line (dotted line) during the calibration process.

It is noted that for practical implementation during model development, standard libraries like sklearn support easy model calibration(sklearn.calibration.CalibratedClassifier).

Impact on Performance

It is pertinent to note that calibration modifies the outputs of trained ML models. It could be possible that calibration also affects the model’s accuracy. Post calibration, some values close to the decision boundary (say 50% for binary classification) may be modified in such a way as to produce an output label different from prior calibration. The impact on accuracy is rarely huge, and it is important to note that calibration improves the reliability of the ML model.

Conclusion

In this article, we have looked at the theoretical background of Model Calibration. Calibration of Machine Learning models is an important but often overlooked aspect of developing a reliable model. The following are key takeaways from our learnings:-

(a) Model Calibration gives insight or understanding of uncertainty in the prediction of the model and in turn, the reliability of the model to be understood by the end-user, especially in critical applications.

(b) Model calibration is extremely valuable to us in cases where predicted probability is of interest.

(c) Platt scaling and Isotonic Regression is popular methods to scale the calibration levels and improve the predicted probability.

Where do we go from here? This article aims to give you a basic understanding of Model Calibration. We can further build on this by exploring actual implementation using standard python libraries like scikit Learn for use cases.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Hyperparameters In Machine Learning Explained

To improve the learning model of machine learning, there are various concepts given in machine learning. Hyperparameters are one of such important concepts that are used to improve the learning model. They are generally classified as model hyperparameters that are not included while setting or fitting the machine to the training set because they refer to the model selection task. In deep learning and machine learning, hyperparameters are the variables that you need to apply or set before the application of a learning algorithm to a dataset.

What are Hyperparameters?

Hyperparameters are those parameters that are specifically defined by the user to improve the learning model and control the process of training the machine. They are explicitly used in machine learning so that their values are set before applying the learning process of the model. This simply means that the values cannot be changed during the training of machine learning. Hyperparameters make it easy for the learning process to control the overfitting of the training set. Hyperparameters provide the best or optimal way to control the learning process.

Hyperparameters are externally applied to the training process and their values cannot be changed during the process. Most of the time, people get confused between parameters and hyperparameters used in the learning process. But parameters and hyperparameters are different in various aspects. Let us have a brief look over the differences between parameters and hyperparameters in the below section.

Parameters Vs Hyperparameters

These are generally misunderstood terms by users. But hyperparameters and parameters are very different from each other. You will get to know these differences as below −

Model parameters are the variables that are learned from the training data by the model itself. On the other hand, hyperparameters are set by the user before training the model.

The values of model parameters are learned during the process whereas, the values of hyperparameters cannot be learned or changed during the learning process.

Model parameters, as the name suggests, have a fixed number of parameters, and hyperparameters are not part of the trained model so the values of hyperparameters are not saved.

Classification of Hyperparameters

Hyperparameters are broadly classified into two categories. They are explained below −

Hyperparameter for Optimization

The hyperparameters that are used for the enhancement of the learning model are known as hyperparameters for optimization. The most important optimization hyperparameters are given below −

Learning Rate − The learning rate hyperparameter decides how it overrides the previously available data in the dataset. If the learning rate hyperparameter has a high value of optimization, then the learning model will be unable to optimize properly and this will lead to the possibility that the hyperparameter will skip over minima. Alternatively, if the learning rate hyperparameter has a very low value of optimization, then the convergence will also be very slow which may raise problems in determining the cross-checking of the learning model.

Batch Size − The optimization of a learning model depends upon different hyperparameters. Batch size is one of those hyperparameters. The speed of the learning process can be enhanced using the batch method. This method involves speeding up the learning process of the dataset by dividing the hyperparameters into different batches. To adjust the values of all the hyperparameters, the batch method is acquired. In this method, the training model follows the procedure of making small batches, training them, and evaluating to adjust the different values of all the hyperparameters. Batch size affects many factors like memory, time, etc. If you increase the size of the batch, then more learning time will be needed and more memory will also be required to process the calculation. In the same manner, the smaller size of the batch will lower the performance of hyperparameters and it will lead to more noise in the error calculation.

Number of Epochs − An epoch in machine learning is a type of hyperparameter that specifies one complete cycle of training data. The epoch number is a major hyperparameter for the training of the data. An epoch number is always an integer value that is represented after every cycle. An epoch plays a major role in the learning process where repetition of trial and error procedure is required. Validation errors can be controlled by increasing the number of epochs. Epoch is also named as an early stopping hyperparameter.

Hyperparameter for Specific Models

Number of Hidden Units − There are various neural networks hidden in deep learning models. These neural networks must be defined to know the learning capacity of the model. The hyperparameter used to find the number of these neural networks is known as the number of hidden units. The number of hidden units is defined for critical functions and it should not overfit the learning model.

Number of Layers − Hyperparameters that use more layers can give better performance than that of less number of layers. It helps in performance enhancement as it makes the training model more reliable and error-free.

Conclusion

Hyperparameters are those parameters that are externally defined by machine learning engineers to improve the learning model.

Hyperparameters control the process of training the machine.

Parameters and hyperparameters are terms that sound similar but they differ in nature and performance completely.

Parameters are the variables that can be changed during the learning process but hyperparameters are externally applied to the training process and their values cannot be changed during the process.

There are various methods categorized in different types of hyperparameters that enhance the performance of the learning model and also make error-free learning models.

Update the detailed information about Bank Customer Churn Prediction Using Machine Learning on the Minhminhbmm.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!