You are reading the article Analyzing Data Made Effortless Using Chatgpt updated in November 2023 on the website Minhminhbmm.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested December 2023 Analyzing Data Made Effortless Using Chatgpt
IntroductionTo learn more about the development of generative models with hands-on experience, join us at the ‘Natural Language Processing using Generative Models’ Workshop at the DataHack Summit 2023. Attending DataHack Summit 2023 will be a game-changer for you. The workshops are designed to deliver immense value, empowering you with practical skills and real-world knowledge. With hands-on experience, you’ll gain the confidence to tackle data challenges head-on. Don’t miss out on this invaluable opportunity to enhance your expertise, connect with industry leaders, and unlock new career opportunities.
Why Prompts are Critical in ChatGPT?I realized that prompts are very critical in order to make use of ChatGPT to its full potential. Even Though ChatGPT is capable of performing any task, in order to make use of it to its full extent, we need to provide the right and detailed prompts. Without the exact prompts, you will not be able to get the desired results.
I am running the experiment to see if ChatGPT can really make sense out of the dataset. I know that ChatGPT can provide me with the code snippets of certain tasks.
For example, given a prompt “help me with the code snippet to check for outliers”. ChatGPT provided me with a code snippet to check and identify the outliers. But can a ChatGPT help me answer the questions such as determining the columns that contain outliers in the dataset? or what is the correlation coefficient between the target variable and features?
In order to answer these questions, ChatGPT has to analyze the specific columns in the dataset and do the math to come up with the answer.
Fingers crossed!
But it’s really interesting to see if ChatGPT can do the math and provide me with the exact answers to the questions. Let’s see!
Exploratory Data Analysis (EDA) Using ChatGPTLet’s try some of the prompts, EDA using ChatGPT:
Prompt 1:I want you to act as a data scientist and analyze the dataset. Provide me with the exact and definitive answer for each question. Do not provide me with the code snippets for the questions. The dataset is provided below. Consider the given dataset for analysis. The first row of the dataset contains the header.
Prompt 2:6,0,3,”Moran, Mr. James”,male,,0,0,330877,8.4583,,Q
Prompt 3:How many rows and columns are present in the dataset?
Prompt 4:List down the numerical and categorical columns
Prompt 5:Check for NANs present in the dataset? If yes, print no. of nans in each column.
Prompt 6:Are there any outliers in the dataset?
Prompt 7:Name the columns that contain the outliers. Provide me with the exact answer.
Prompt 8:What are the significant factors that affect the survival rate?
Prompt 9:Determine the columns that follow the skewed distribution and name them.
Prompt 10:Generate meaningful insights about the dataset.
Such cool stuff 🙂 As you can see here, ChatGPT provided me with a summary of valuable insights and also the important factors that might have affected the survival rate.
ConclusionImpressive! ChatGPT is able to generate meaningful insights in no time. My experiment is successful. And ChatGPT lived up to my expectations.
To learn more about the development of generative models with hands-on experience, join us at the ‘Natural Language Processing using Generative Models’ Workshop at the DataHack Summit 2023. Attending DataHack Summit 2023 will be a game-changer for you. The workshops are designed to deliver immense value, empowering you with practical skills and real-world knowledge. With hands-on experience, you’ll gain the confidence to tackle data challenges head-on. Don’t miss out on this invaluable opportunity to enhance your expertise, connect with industry leaders, and unlock new career opportunities. Register now for the DataHack Summit 2023!
Related
You're reading Analyzing Data Made Effortless Using Chatgpt
Chatgpt Data Breach Confirmed: A Bug Revealed Chatgpt Personal And Billing Data
On Friday, OpenAI confirmed the ChatGPT Data Breach caused a bug in the open-source library. Following the breach, ChatGPT was immediately taken down by OpenAI to resolve the issue, with maintainers of Redis patching the flaw which caused the revelation of the user’s information.
Users reported they were able to see another user’s conversation history titles. While later it was reported about 1.2% of ChatGPT Plus users’ payment-related information was leaked such as First and Last names, email, payment addresses, and last 4 digits of their credit card number (Full credit along with the card’s expiry date).
Key Points:
A bug was discovered in Redis-py which is (a Redis client open-source library) which causes a data breach in OpenAI’s chat service ChatGPT.
1.2% of subscribers’ personal information was leaked during this breach including their first and last name, email, payment address, last four digits of their credit card number, and their expiry date. (Only the last 4-digit were exposed during the breach)
ChatGPT was taken down by OpenAI to resolve the issue, the AI chat service reached out to Redis maintainers with a patch to solve this issue.
What caused the ChatGPT Data Breach?
The cause for ChatGPT’s Data Breach was a “Bug” which was discovered in the Redis client open-source library, Redis-py. It was reported by OpenAI, they reached out to Redis maintainers along with a patch to fix this issue.
A Redis-py library serves as a Python interface. The Developers in ChatGPT utilize Redis in the system to cache user data in their server. This helps avoid the constant need to check the database of chatbots for every request.
The bug resulted in leaking the user’s personal information such as the subscriber’s name, payment address, email, and last four digits of paid subscriber’s credit card number along with its expiry date. Following the breach, ChatGPT immediately took the server down, rendering its services out to resolve the issue.
Even after the restoration of ChatGPT was completed, users’ chat histories were kept hidden for hours to perform a post-mortem, cease the exposing of data, and take suitable action against the same.
OpenAI stated in the post-mortem, “Upon deeper investigation, it was discovered the same bug might have generated the unintentional visibility of paid subscriber’s payment-related data. About 1.2% of ChatGPT Plus subscribers’ (ChatGPT Plus is a premium version of ChatGPT that provides GPT-4 features and responses) data were released who were active during the particular nine-hour window.”
Before ChatGPT was turned offline following the bug issue and subscriber’s data leak on Monday 20th March, a few users conveyed they were able to witness another active user’s personal details such as their first and last name, payment address, email, and last 4-digit of their credit card number (full credit card number was not exposed in the information leak) and expiry date.
OpenAI stated a very low number of ChatGPT Plus subscribers’ data were leaked, and specific actions were taken place to resolve this issue such as:
A subscription confirmation mail was sent to all the paid users whose data were leaked on 20th March, at 1 a.m. and 10 a.m. Users can then confirm their subscription by simply tapping on “My Account” on ChatGPT Plus and then navigating to “Manage my subscription” from 1 a.m. to 10 a.m. on Pacific time. In addition, OpenAI claimed they contacted all the affected ChatGPT Plus subscribers whose payment details were leaked due to a bug to ensure the safety of their subscribers.
OpenAI’s response to the information leak
OpenAI’s CEO Sam Altman, took to his Twitter on 23rd March, addressing the issue, saying “We encountered a significant problem in ChatGPT due to a bug in the open source library, for which a solution now has been cast and we have just completed validating the issue.”
“A small number of subscribers were able to see the titles of other users’ ChatGPT conversation history during this case. We feel terrible about this.”
Sam Alman also said via his Twitter account, “Unfortunately, users will be unable to see their ChatGPT chat history from Monday 1 am PDT until 10 am Monday PDT” We will follow up with a technical postmortem.
Using Data For Successful Multi
How to use data to optimise your multi-channel marketing
Multi-channel marketing provides businesses with the opportunity to engage with consumers across a variety of different fronts, tailoring messages for specific groups while maintaining a consistent message and brand. But it’s not simply a matter of sending your message blindly out into the ether – to achieve true success over multiple channels you need to make effective use of the data at your disposal.
This article demystifies this process, helping you understand what this data is, where you need to find it and what you need to do with it. As with all marketing, a little bit of considered thought at the start of the campaign makes a real difference in the result.
Tracking your dataThis may seem obvious, but the actual obtaining of your data is the most important place to start and tracking a campaign is really the only way to determine whether your marketing efforts are having a positive effect on your business’s bottom line.
In the digital world, track conversion metrics broken down by target audience and geography. When offline, data should be tracked at the lowest level possible to ensure clarity and simplicity. Something else to consider is the manner in which this data will be stored. Marketing produces a high volume of data, and it’s important to ensure you will have an intuitive system for tracking and managing this information. It may even be an idea to outsource this aspect of the process, for simplicities sake.
Analyse your dataNow that you have your data, it’s important to understand what it’s telling you. Consider a consumer’s interaction with your brand as a path, from discovering the initial message to ultimately making a purchase/interacting with your service.
Discovering and acknowledging the channels different groups are using to interact with your brand is a good way to understand improvements you can make on a broader level, as well as some quick victories that can streamline processes.
A thorough data analysis is also a good way to gain a thorough understanding of the consumers who are interacting with your brand. Find out who the high value consumers are and determine ways in which you can enhance engagement. Also, consider the devices they are using in their interactions. You’re as good as your data in marketing, and a thorough analysis will ensure you get more bang for your buck.
Develop a strategyNow that you’ve analysed your data, it’s time to decide how you’re going to respond to it. You’ve discovered the channels your consumers are responding to and the groups of consumers are of highest value, so now it’s time to maximise this and develop a message that will achieve results for your business.
There are a few things to consider in this process. For instance, it’s important to make sure the message that passes through to your customers across multiple channels is a consistent, effective one. It’s also important to make sure the consumer’s journey through different channels is as seamless as possible. An online clothes retailer may have people browsing on mobile devices during the day but only making the purchase when they get home, so keep this in mind at all times.
Respond through preferred channelsNow you’ve analysed your data and formulated an effective strategy, it’s time to bring it all together. Multi-channel marketing allows you to engage different groups of consumers with tailor-made messages, but as mentioned before, it’s important to ensure these messages are consistent with the overall identity of your brand.
Test, test, testThe most important part here is tracking your results, responding and testing. Look for different aspects of your campaign to test and make sure you integrate them into your planning. Think about different conversation metric variables and see how you can tinker with them to achieve different results. As with any marketing, it’s not likely you’re going to find the thing that works best with your first effort, so be flexible and willing to incorporate new ideas into your campaign. The world moves at a fast pace these days and if you’re not willing to keep up, it will be to the detriment of your multi-channel marketing campaign. Testing and a degree of flexibility in your approach allows you to keep track of what is and isn’t working and stay ahead of the curve.
Multi-channel marketing is one of the most effective ways to engage with consumers in 2023. But it’s important to do it correctly. Track your data, analyse your data and develop a strategy that allows you to respond effectively in the appropriate channels. And once you’ve done this test, test and test some more! A proactive approach can achieve serious results for your business, allowing you to maintain a consistent message across multiple platforms and maximise the yield from your consumers.
At the end of the day, multi-channel marketing is about getting as much bang for your buck as possible. An acute awareness of what your data is telling you and how to respond will help your business grow and separate you from the rest of the pack.
Safeguarding Against Data Breaches Using Pam Solutions
Data breaches have unfortunately become a regular occurrence. High-profile instances of data theft involving companies like T-Mobile, the US Transportation Security Administration (TSA), Twitter, and others have been recently documented.
Often, breached companies are hesitant to disclose the specifics of such events, commonly attributing them to hacker attacks. However, evidence suggests that a significant number of these data breaches actually stem from privileged accounts within the victimized enterprises’ information systems. These accounts are highly sought after by cybercriminals.
Accounts with extensive system rights can access thousands of users’ confidential data, business information, and IT system configurations. Once criminals breach the security perimeter, they can remain undetected for months, waiting for the best moment to execute their attack.
The Security Risks Associated with Privileged UsersIn the first scenario, cyber attackers manage to gain access to a privileged accounts. Essentially, this grants them a “master key” to the organization, making a targeted attack a matter of when not if. Whether used directly by hackers or sold on the dark web, the “username-password” pair becomes a serious corporate security breach once compromised. The only hope for preventing an attack is detecting the data leak early.
In the second scenario, the privileged users themselves – intentionally or unwittingly – become the perpetrator. There are many well-documented instances of this. The results of an everyday employee morphing into a cybercriminal can range from deliberate disruption of corporate systems to outright data theft. This is reminiscent of the numerous data leaks various services have experienced in the past.
Privilege Access Management SystemsTo overcome the problems of controlling privileged accounts, there are specialized tools called PAM systems (Privilege Access Management). They allow you to prevent massive leaks of data, as well as control the use of passwords in an organization, allowing you to save on reputational costs. PAM systems solve four important tasks:
User privilege management. Extended rights are granted only to those users who have good reasons for this. In addition, access is not given to all resources but only to those that users really need to fulfill their work duties, and the validity period of privileges is strictly limited in time.
Monitoring the actions of privileged users. The system records user sessions and stores the data for further review. Advanced PAM solutions keep a log of work sessions and can recognize text (OCR function).
Password management. The system stores passwords in encrypted form, updates them, and does not give users access to secret information. Sometimes third-party password management solutions are used for this.
Support for pass-through authentication. It allows users not to enter a password to access each corporate service but “log in” only once with the help of Single Sign-On (SSO).
Who Needs RAM Systems?PAM systems are applicable to organizations of all types, as every organization has privileged users. Not only individual users hold accounts with special rights, but also entities like business partners, contractors, companies that manage information systems, and third-party systems that interact with corporate systems without human intervention. Nearly every business stores data needing special protection, such as employees’ personal data and customers’ personal records. The applicability of PAM systems is virtually unlimited, regardless of the company’s size.
Implementing a PAM systemInstallation and configuration of PAM systems are generally straightforward. They do not require intricate integrations and ensure compatibility with various systems. Typically, PAM is installed “over” other enterprise information systems and becomes a sort of “gateway” for all user access.
The implementation method varies depending on an organization’s competencies and its approach to IT infrastructure development. Some organizations utilize contractors’ services, maintaining an in-house team of administrators to oversee system operations. In contrast, others possess sufficient competence to implement a PAM system independently. This typically includes deploying server components, organizing log and data storage, and installing agents on protected infrastructure nodes.
Scraping Data Using Octoparse For Product Assessment
In today’s data-driven world, it is crucial to have access to reliable and relevant data for informed decision-making. Often, data from external sources is obtained through processes like pulling or pushing from data providers and subsequently stored in a data lake. This marks the beginning of a data preparation journey where various techniques are applied to clean, transform, and apply business rules to the data. Ultimately, this prepared data serves as the foundation for Business Intelligence (BI) or AI applications, tailored to meet individual business requirements. Join me as we dive into the world of data scraping with Octoparse and discover its potential in enhancing data-driven insights.
This article was published as a part of the Data Science Blogathon.
Web Scrapping and AnalyticsYes! In some cases, we have e to grab the data from an external source using Web Scraping techniques and do all data torturing on top of the data to find the insight of the data with techniques.
Same time we do not forget to use to find the relationship and correlation between features and expand the other opportunities to explore further by applying mathematics, statistics, and visualisation techniques, on top of selecting and using machine learning algorithms and finding the prediction/classification/clustering to improve the business opportunities and prospects, this is a tremendous journey.
Focusing on excellent data collection from the right resource is the critical success of a data platform project. I hope you know that. In this article, let’s try to understand the process of gaining data using scraping techniques – zero code.
Before getting into this, I will try to understand a few things better.
Data ProvidersAs I mentioned earlier, the Data Sources for DS/DA could be in from any data source. Here, our focus is on Web-Scraping processes.
What is Web-Scraping and Why?Web-Scraping is the process of extracting data in diverse volumes in a specific format from a website(s) in the form of slice and dice for Data Analytics and Data Science standpoint and file formats depending on the business requirements. It would .csv, JSON, .xlsx,.xml, etc.. Sometimes we can store the data directly into the database.
Why Web-Scraping?Web-Scraping is critical to the process; it allows quick and economical extraction of data from different sources, followed by diverse data processing techniques to gather the insights directed to understand the business better and keep track of the brand and reputation of a company to align with legal limits.
Web Scraping Process RequestVsResponseThe first step is to request the target website(s) for the specific contents of a particular URL, which returns the data in a specific format mentioned in the programming language (or) script.
Parsing&ExtractionAs we know, Parsing is usually applied to programming languages (Java..Net, Python, etc.). It is the structured process of taking the code in the form of text and producing a structured output in understandable ways.
Data-DownloadingThe last part of scrapping is where you can download and save the data in CSV, JSON format or a database. We can use this file as input for Data Analytics and Data Science perspective.
There are multiple Web Scraping tools/software available in the market, and let’s look at a few of them.
In the market, many Web-Scraping tools are available, and let’s review a few of them.
ProWebScraper Features
Completely effortless exercise
It can be used by anyone can who knows how to browse
It can scrape Texts, Table data, Links, Images, Numbers and Key-Value Pairs.
It can scrape multiple pages.
It can be scheduled based on the demand (Hourly, Daily, Weekly, etc.)
Highly scalable, it can run multiple scrapers simultaneously and thousands of pages.
Let’s focus on Octoparse,
The web-Data Extraction tool, Octoparse, stands out from other devices in the market. You can extract the required data without coding, scrape data with modern visual design, and automatically scrapes the data from the website(s) along with the SaaS Web-Data platform feature.
Octoparse provides ready-to-use Scraping templates for different purposes, including Amazon, eBay, Twitter, Instagram, Facebook, BestBuy and many more. It lets us tailor the scraper according to our requirements specific.
Compared with other tools available in the market, it is beneficial at the organisational level with massive Web- Scraping demands. We can use this for multiple industries like e-commerce, travel, investment, social, crypto-currency, marketing, real estate etc.
Features
Both categories could find it easy to use this to extract information from websites.
ZERO code experience is fantastic.
Indeed, it makes life easier and faster to get data from websites without code and with simple configurations.
It can scrape the data from Text, Table, Web-Links, Listing-pages and images.
It can download the data in CSV and Excel formats from multiple pages.
It can be scheduled based on the demand (Hourly, Daily, Weekly, etc.)
Excellent API integration feature, which delivers the data automatically to our systems.
Now time to Scrape eBay product information using Octoparse.
Getting product information from eBay, Let’s open the eBay and select/search for a product, and copy the URL
In a few steps, we were able to complete the entire process.
Open the target webpage
Creating a workflow
Scrapping the content from the specified web pages.
Customizing and validating the data using review future
Extract the data using workflow
Scheduling
Open Target WebpageLet’s login Octoparse, paste the URL and hit the start button; Octoparse starts auto-detect and pulls the details for you in a separate window.
Creating Workflow and New-TaskWait until the search reaches 100% so that you will get data for your needs.
During the detection, Octoparse will select the critical elements for your convenience and save our time.
Note: To remove the cookies, please turn off the browser tag.
Scrapping the Content from the Identified Web-pageOnce we confirm the detection, the Workflow template is ready for configurations and data preview at the bottom. There you can configure the column as convenient (Copy, Delete, Customize the column, etc.,)
Customizing and Validating the Data using Review FutureYou can add your custom field(s) in the Data preview window, import and export the data, and remove duplicates.
Extract the Data using WorkflowOn the Workflow window, based on your hit on each navigation, we could move around the web browser. – Go to the web page, Scroll Page, Loop Item, Extract Data, and you can add new steps.
We can configure time out, file format in JSON or NOT, Before and After the action is performed, and how often the action should perform. After the required configurations have been done, we could act and extract the data.
Save Configuration, and Run the Workflow Schedule-taskYou can run it on your device or in the cloud.
Data Extraction – Process starts Data ready to Export Chose the Data Format for Further Usage Saving the Extracted Data Extracted Data is Ready in the Specified-formatData is ready for further usage either in Data Analytics and Data Science
What is Next! Yes, no doubt about that, have to load in Jupiter notebook and start using the EDA process extensively.
Conclusion
Importance of Data Source
Data Science Lifecycle
What is Web Scrapping and Why
The process involved in Web Scrapping
Top Web Scraping tools and their overview
Octoparse Use case – Data Extraction from eBay
Data Extraction using Octoparse – detailed steps (Zero Code)
I have enjoyed this web-scraping tool and am impressed with its features; you can try and want it to extract free data for your Data Science & Analytics practice projects perspective.
Frequently Asked QuestionsRelated
Make Amazing Data Science Projects Using Pyscript.js
This article was published as a part of the Data Science Blogathon.
Introduction on PyScript.jsWhat is PyScript.js?
It is a front-end framework that enables the use of Python in the browser. It is developed using Emscripten, Pyodide, WASM, and other modern web technologies.
Using Python in the browser does not mean that it can replace Javascript. But it provides more convenience and flexibility to the Python Developers, especially Machine Learning Engineers.
What PyScript Offers?
5. It provides flexibility to the developers. Now they can quickly build their python programs with the existing UI components such as buttons and containers.
This tutorial shows you how we can create our machine learning model with a web GUI using PyScript.
We will use the famous Car Evaluation Dataset to predict the car’s condition based on the six categorical features. We will discuss the dataset later but first, start with setting up the chúng tôi library.
Setting Up PyScript.jsThis section will set up our HTML Template and include the chúng tôi library.
We will use VSCode here, but you can choose any IDE.
1. Create a directory named as PyscriptTut.
$ mkdir PyscriptTut $ cd PyscriptTut2. Creating an HTML Template
Create an HTML template inside it named index.html
Inside this template, place the starter HTML code
Bootstrap CDN is used for Styling the Web Page
PyScript Installation
We will not install the library on our machine, we will directly import the library from the PyScript website.
Important Note:
You have to use a Local Server to run the HTML Code. Otherwise, you may face issues in importing several libraries in a python environment.
If you are using VSCode, then you can use its Live Server Extension.
Or you can also create a python server writing the below command in the terminal
Sample Code
You can try this sample code to check whether PyScript is successfully imported or not.
print(“Welcome to puscript tutorial”) for i in range(1, 10): print(i)
This is a simple program that prints the number from 1 to 9 using a for-loop.
If everything goes fine, the output looks like that
Hurray 🎉, our PyScript library is installed successfully in our template.
Creating GUIThis section will create a web GUI to use our machine learning model for training and testing.
As mentioned above, we will use Bootstrap Library for creating custom styling. I have also used inline CSS in some places.
1. Add Google Fonts CDN
2. Some CSS Configuration
Add the below code to your template. It will enable smooth scrolling on our web page and apply the above font.
* { margin: 0; padding: 0; } html { scroll-behavior: smooth; } body { font-family: ‘Montserrat’, sans-serif; }
3. Adding Bootstrap Navbar Component
<button type=”button” data-toggle=”collapse” data-target=”#navbarSupportedContent”
4. Adding Heading Content
We will create a small landing page with some texts and images.
The Source of the image used in this component can be found here.
5. Component to Train the Model
In this component, we will create some radio buttons and input texts, so that users can select which classifier they want to train and by how many tests split.
<input type=”radio” name=”modelSelection” Random Forest <input type=”radio” name=”modelSelection” Logistic Regression <input type=”radio” name=”modelSelection” MLP Classifier <input type=”radio” name=”modelSelection” Gradient Boosting
6. Component for Alert Messages
This component is used for alerts and success messages.
7. Component for checking the Training Results
In this, we can see the Accuracy and Weighted F1 Score of the selected model after training.
8. Component for selecting Car Parameters
We can select the six parameters to check the performance of the car.
The Submit will remain disabled until you train the model.
9. Component to Output the Result
This component displays the predicted value.
10. Footer (Optional)
This is the footer for our web page
Our GUI is now created, ✌
Small NoteFrom now onwards, we will train our machine learning model. We need to add libraries in the Python Environment
– pandas – scikit-learn – numpy
Importing LibrariesFirstly we will import all the necessary libraries
import pandas as pd import pickle from sklearn.model_selection import train_test_split, GridSearchCV from sklearn.linear_model import LogisticRegression from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier from sklearn.neural_network import MLPClassifier from sklearn.metrics import accuracy_score, f1_score from chúng tôi import open_url import numpy as np Dataset PreprocessingAs discussed earlier, we will use Car Evaluation Dataset from UCI ML Repository.
You can download the dataset from that link.
This dataset contains six categorical features, which are Buying Price, Maintenance Price, No. of Doors, No. of Persons, Luggage Capacity, and Safety Qualifications
6. Safety – low, mid, high
The Output is classified into four classes namely, unacc, acc, good, vgood
4. vgood – Very Good
Function to Upsample the Dataset
def upSampling(data): from sklearn.utils import resample # Majority Class Dataframe df_majority = data[(data['score']==0)] samples_in_majority = data[data.score == 0].shape[0] # Minority Class Dataframe of all the three labels df_minority_1 = data[(data['score']==1)] df_minority_2 = data[(data['score']==2)] df_minority_3 = data[(data['score']==3)] # upsample minority classes df_minority_upsampled_1 = resample(df_minority_1, replace=True, n_samples= samples_in_majority, random_state=42) df_minority_upsampled_2 = resample(df_minority_2, replace=True, n_samples= samples_in_majority, random_state=42) df_minority_upsampled_3 = resample(df_minority_3, replace=True, n_samples= samples_in_majority, random_state=42) # Combine majority class with upsampled minority classes df_upsampled = pd.concat([df_minority_upsampled_1, df_minority_upsampled_2, df_minority_upsampled_3, df_majority]) return df_upsampledFunction to read input data and return processed data.
def datasetPreProcessing(): # Reading the content of CSV file. data = pd.read_csv(csv_url_content) pyscript.write("headingText", "Pre-Processing the Dataset...") # This is used to send messages to the HTML DOM. # Removing all the null values data.isna().sum() # Removing all the duplicates data.drop_duplicates() coloumns = ['buying', 'maint', 'doors', 'people', 'luggaage', 'safety', 'score'] # Converting Categorical Data into Numerical Data data['buying'] = data['buying'].replace('low', 0) data['buying'] = data['buying'].replace('med', 1) data['buying'] = data['buying'].replace('high', 2) data['buying'] = data['buying'].replace('vhigh', 3) data['maint'] = data['maint'].replace('low', 0) data['maint'] = data['maint'].replace('med', 1) data['maint'] = data['maint'].replace('high', 2) data['maint'] = data['maint'].replace('vhigh', 3) data['doors'] = data['doors'].replace('2', 0) data['doors'] = data['doors'].replace('3', 1) data['doors'] = data['doors'].replace('4', 2) data['doors'] = data['doors'].replace('5more', 3) data['people'] = data['people'].replace('2', 0) data['people'] = data['people'].replace('4', 1) data['people'] = data['people'].replace('more', 2) data['luggaage'] = data['luggaage'].replace('small', 0) data['luggaage'] = data['luggaage'].replace('med', 1) data['luggaage'] = data['luggaage'].replace('big', 2) data['safety'] = data['safety'].replace('low', 0) data['safety'] = data['safety'].replace('med', 1) data['safety'] = data['safety'].replace('high', 2) data['score'] = data['score'].replace('unacc', 0) data['score'] = data['score'].replace('acc', 1) data['score'] = data['score'].replace('good', 2) data['score'] = data['score'].replace('vgood', 3) upsampled_data = upSampling(data) return upsampled_dataLet’s understand these above functions in more detail:
1. Firstly, we have read the CSV File using the Pandas library.
2. You may be confused by this line py script.write(“headingText”, “Pre-Processing the Dataset…”).
This code updates the messages component in the HTML DOM that we have created above.
You can write any message in any HTML Tag
3. Then, we have removed the null values and the duplicates. But luckily, this dataset does not contain any null values.
4. Further, we have converted all the categorical data into numerical data.
5. Finally, we have performed upsampling of the dataset.
You can observe that the number of samples in one particular class is far more than in the other classes. Our model will be biased towards a specific class because it has very little data to train on other classes.
So we have to increase the number of samples in other classes. It is also called Upsampling.
I have created a separate function named upSampling that will upsample the data.
Now we have an equal number of samples for all the classes.
Training the ModelFunction to check which machine learning model is selected by the user for training.
def model_selection(): selectedModel = document.querySelector('input[name="modelSelection"]:checked').value; if selectedModel == "rf": document.getElementById("selectedModelContentBox").innerText = "Random Forest Classifier"; return RandomForestClassifier(n_estimators=100) elif selectedModel == "lr": document.getElementById("selectedModelContentBox").innerText = "Logistic Regression"; return LogisticRegression() elif selectedModel == "gb": document.getElementById("selectedModelContentBox").innerText = "Gradient Boosting Classifier"; return GradientBoostingClassifier(n_estimators=100, learning_rate=1.0, max_depth=1, random_state=0) else: document.getElementById("selectedModelContentBox").innerText = "MLP Classifier"; return MLPClassifier()Function to train the model on the chosen classifier.
def classifier(model, X_train, X_test, y_train, y_test): clf = model clf.fit(X_train, y_train) y_pred = clf.predict(X_test) y_score = clf.fit(X_train, y_train) acc_score = accuracy_score(y_test, y_pred) f1Score = f1_score(y_test, y_pred, average='weighted') return acc_score, model, f1Score def trainModel(e=None): global trained_model processed_data = datasetPreProcessing() # Take the Test Split as an input by the user test_split = float(document.getElementById("test_split").value) # If the test split is greater than 1 or less than 0 then we will throw an error. pyscript.write("headingText", "Choose Test Split between 0 to 1") return document.getElementById("testSplitContentBox").innerText = test_split; X = processed_data[['buying', 'maint', 'doors', 'people', 'luggaage', 'safety']] y = processed_data['score'] # Splitting the Dataset into training and testing. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=test_split, random_state=42) # Below function can return the classification model choosen by the user model = model_selection() pyscript.write("headingText", "Model Training Started...") acc_score, trained_model, f1Score = classifier(model, X_train, X_test, y_train, y_test) pyscript.write("headingText", "Model Training Completed.") # Writing the value of accuracy and f1-score to the DOM document.getElementById("accuracyContentBox").innerText = f"{round(acc_score*100, 2)}%"; document.getElementById("f1ContentBox").innerText = f"{round(f1Score*100, 2)}%"; # Below code is to enable the Model Training Button when the Model is successfully trained. document.getElementById("submitBtn").classList.remove("disabled"); document.getElementById("submitBtn").disabled = False; document.getElementById("trainModelBtn").classList.remove("disabled"); document.getElementById("trainModelBtn").disabled = False; if e: e.preventDefault() return False Testing the ModelIn this section, we will test our model on the six parameters that we have discussed above.
Below is the function to test the model.
def testModel(e=None): buying_price = int(document.getElementById("buying_price").value) maintanence_price = int(document.getElementById("maintanence_price").value) doors = int(document.getElementById("doors").value) persons = int(document.getElementById("persons").value) luggage = int(document.getElementById("luggage").value) safety = int(document.getElementById("safety").value) arr = np.array([buying_price, maintanence_price, doors, persons, luggage, safety]).astype('float32') arr = np.expand_dims(arr, axis=0) result = trained_model.predict(arr) condition = "" if result[0] == 0: condition = "Unaccepted" elif result[0] == 1: condition = "Accepted" elif result[0] == 2: condition = "Good" else: condition = "Very Good" pyscript.write("resultText", f"Predicted Value: {condition}") if e: e.preventDefault() return FalseFirstly, we will take the input from the user and the feed that input to the model for prediction. Then finally, we output the results.
Our machine learning model is now trained.
Conclusion
Deployed Version – Link
Before PyScript, we don’t have any proper tool to use Python on the client-side. Frameworks such as Django or Flask can mainly use Python on the backend. In recent years, Python has grown its population immensely. It has been used in Machine Learning, Artificial Intelligence, Robotics, etc.
In this article, we have trained and tested a machine learning model completely in HTML language. You can increase the model’s accuracy by tuning some hyperparameters or searching for the best parameters using Grid Search CV or Randomized Search CV.
The main focus of this article is to use the chúng tôi library, not to achieve a high accuracy classification model.
4. Finally, we have written the code to test the model based on the user’s input.
Do check my other articles also.
Thanks for reading, 😊
The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.
Related
Update the detailed information about Analyzing Data Made Effortless Using Chatgpt on the Minhminhbmm.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!