About Lesson
Dataset:
It is given by Kaggle from UCI Machine Learning Repository, in one of its challenge. It is a dataset of Breast Cancer patients with Malignant and Benign tumor.
Logistic Regression is used to predict whether the given patient is having Malignant or Benign tumor based on the attributes in the given dataset.
Code : Loading Libraries
1
# performing linear algebra
2
import numpy as np
3
4
# data processing
5
import pandas as pd
6
7
# visualisation
8
import matplotlib.pyplot as plt
Code : Loading dataset
1
data = pd.read_csv("..\breast-cancer-wisconsin-data\data.csv")
2
3
print (data.head)
Output :
Code : Loading dataset
1
data.info()
Output :
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 569 entries, 0 to 568
Data columns (total 33 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 id 569 non-null int64
1 diagnosis 569 non-null object
2 radius_mean 569 non-null float64
3 texture_mean 569 non-null float64
4 perimeter_mean 569 non-null float64
5 area_mean 569 non-null float64
6 smoothness_mean 569 non-null float64
7 compactness_mean 569 non-null float64
8 concavity_mean 569 non-null float64
9 concave points_mean 569 non-null float64
10 symmetry_mean 569 non-null float64
11 fractal_dimension_mean 569 non-null float64
12 radius_se 569 non-null float64
13 texture_se 569 non-null float64
14 perimeter_se 569 non-null float64
15 area_se 569 non-null float64
16 smoothness_se 569 non-null float64
17 compactness_se 569 non-null float64
18 concavity_se 569 non-null float64
19 concave points_se 569 non-null float64
20 symmetry_se 569 non-null float64
21 fractal_dimension_se 569 non-null float64
22 radius_worst 569 non-null float64
23 texture_worst 569 non-null float64
24 perimeter_worst 569 non-null float64
25 area_worst 569 non-null float64
26 smoothness_worst 569 non-null float64
27 compactness_worst 569 non-null float64
28 concavity_worst 569 non-null float64
29 concave points_worst 569 non-null float64
30 symmetry_worst 569 non-null float64
31 fractal_dimension_worst 569 non-null float64
32 Unnamed: 32 0 non-null float64
dtypes: float64(31), int64(1), object(1)
memory usage: 146.8+ KB
Code: We are dropping columns – ‘id’ and ‘Unnamed: 32’ as they have no role in prediction
1
data.drop(['Unnamed: 32', 'id'], axis = 1)
2
data.diagnosis = [1 if each == "M" else 0 for each in data.diagnosis]
Code : Input and Output data
1
y = data.diagnosis.values
2
x_data = data.drop(['diagnosis'], axis = 1)
Code : Normalisation
1
x = (x_data - np.min(x_data))/(np.max(x_data) - np.min(x_data))
2
3
# This code is modified by Susobhan Akhuli
Code : Splitting data for training and testing.
1
from sklearn.model_selection import train_test_split
2
x_train, x_test, y_train, y_test = train_test_split(
3
x, y, test_size = 0.15, random_state = 42)
4
5
x_train = x_train.T
6
x_test = x_test.T
7
y_train = y_train.T
8
y_test = y_test.T
9
10
print("x train: ", x_train.shape)
11
print("x test: ", x_test.shape)
12
print("y train: ", y_train.shape)
13
print("y test: ", y_test.shape)
Output :
x train: (32, 483)
x test: (32, 86)
y train: (483,)
y test: (86,)
Code : Weight and bias
1
def initialize_weights_and_bias(dimension):
2
w = np.full((dimension, 1), 0.01)
3
b = 0.0
4
return w, b
Code : Sigmoid Function – calculating z value.
1
# z = np.dot(w.T, x_train)+b
2
def sigmoid(z):
3
y_head = 1/(1 + np.exp(-z))
4
return y_head
Code : Forward-Backward Propagation
1
def forward_backward_propagation(w, b, x_train, y_train):
2
z = np.dot(w.T, x_train) + b
3
y_head = sigmoid(z)
4
loss = - y_train * np.log(y_head) - (1 - y_train) * np.log(1 - y_head)
5
# x_train.shape[1] is for scaling
6
cost = (np.sum(loss)) / x_train.shape[1]
7
8
# backward propagation
9
derivative_weight = (np.dot(x_train, (
10
(y_head - y_train).T))) / x_train.shape[1]
11
derivative_bias = np.sum(
12
y_head-y_train) / x_train.shape[1]
13
gradients = {"derivative_weight": derivative_weight,
14
"derivative_bias": derivative_bias}
15
return cost, gradients
Code : Updating Parameters
1
def update(w, b, x_train, y_train, learning_rate, number_of_iterarion):
2
cost_list = []
3
cost_list2 = []
4
index = []
5
6
# updating(learning) parameters is number_of_iterarion times
7
for i in range(number_of_iterarion):
8
# make forward and backward propagation and find cost and gradients
9
cost, gradients = forward_backward_propagation(w, b, x_train, y_train)
10
cost_list.append(cost)
11
12
# lets update
13
w = w - learning_rate * gradients["derivative_weight"]
14
b = b - learning_rate * gradients["derivative_bias"]
15
if i % 10 == 0:
16
cost_list2.append(cost)
17
index.append(i)
18
print ("Cost after iteration % i: % f" %(i, cost))
19
20
# update(learn) parameters weights and bias
21
parameters = {"weight": w, "bias": b}
22
plt.plot(index, cost_list2)
23
plt.xticks(index, rotation ='vertical')
24
plt.xlabel("Number of Iterarion")
25
plt.ylabel("Cost")
26
plt.show()
27
return parameters, gradients, cost_list
Code : Predictions
1
def predict(w, b, x_test):
2
# x_test is a input for forward propagation
3
z = sigmoid(np.dot(w.T, x_test)+b)
4
Y_prediction = np.zeros((1, x_test.shape[1]))
5
6
# if z is bigger than 0.5, our prediction is sign one (y_head = 1),
7
# if z is smaller than 0.5, our prediction is sign zero (y_head = 0),
8
for i in range(z.shape[1]):
9
if z[0, i]<= 0.5:
10
Y_prediction[0, i] = 0
11
else:
12
Y_prediction[0, i] = 1
13
14
return Y_prediction
Code : Logistic Regression
1
def logistic_regression(x_train, y_train, x_test, y_test,
2
learning_rate, num_iterations):
3
4
dimension = x_train.shape[0]
5
w, b = initialize_weights_and_bias(dimension)
6
7
parameters, gradients, cost_list = update(
8
w, b, x_train, y_train, learning_rate, num_iterations)
9
10
y_prediction_test = predict(
11
parameters["weight"], parameters["bias"], x_test)
12
y_prediction_train = predict(
13
parameters["weight"], parameters["bias"], x_train)
14
15
# train / test Errors
16
print("train accuracy: {} %".format(
17
100 - np.mean(np.abs(y_prediction_train - y_train)) * 100))
18
print("test accuracy: {} %".format(
19
100 - np.mean(np.abs(y_prediction_test - y_test)) * 100))
20
21
logistic_regression(x_train, y_train, x_test,
22
y_test, learning_rate = 1, num_iterations = 100)
Output :
Cost after iteration 0: 0.692836
Cost after iteration 10: 0.498576
Cost after iteration 20: 0.404996
Cost after iteration 30: 0.350059
Cost after iteration 40: 0.313747
Cost after iteration 50: 0.287767
Cost after iteration 60: 0.268114
Cost after iteration 70: 0.252627
Cost after iteration 80: 0.240036
Cost after iteration 90: 0.229543
Cost after iteration 100: 0.220624
Cost after iteration 110: 0.212920
Cost after iteration 120: 0.206175
Cost after iteration 130: 0.200201
Cost after iteration 140: 0.194860
Output :
train accuracy: 37.267080745341616 %
test accuracy: 37.2093023255814 %
Code : Checking results with linear_model.LogisticRegression
1
from sklearn.impute import SimpleImputer
2
from sklearn import linear_model
3
4
# Create an imputer to replace NaN with the mean of the column
5
imputer = SimpleImputer(strategy='mean')
6
7
# Fit the imputer on the training data and transform both training and test data
8
x_train = imputer.fit_transform(x_train.T).T
9
x_test = imputer.transform(x_test.T).T
10
11
logreg = linear_model.LogisticRegression(random_state = 42, max_iter = 150)
12
print("test accuracy: {} ".format(
13
logreg.fit(x_train.T, y_train.T).score(x_test.T, y_test.T)))
14
print("train accuracy: {} ".format(
15
logreg.fit(x_train.T, y_train.T).score(x_train.T, y_train.T)))
16
17
# This code is modified by Susobhan Akhuli
Output :
test accuracy: 0.627906976744186
train accuracy: 0.6273291925465838
Get the complete notebook and dataset link here:
Notebook link : click here.
Dataset link : click here