Futures information:<\/strong><\/p>\n I work daily with Python 3.6+ using a few packages to simplify everyday tasks in data science.<\/p>\n Below are the most important ones.<\/p>\n All requirements are in Ermlab repository as a requirements.txt file.<\/p>\n First of all, we need to import our data using Pandas module.<\/p>\n Before making anything like feature selection, feature extraction and classification, firstly we start with basic data analysis. Let’s look at the features of data.<\/p>\n Now, We need to drop unused columns such as id (not used for classification), Unnamed: 32 (with NaN values) and diagnosis (this is our label). The next step is to convert strings (M, B) to integers (0, 1) using map(),\u00a0 define our features and labels.<\/p>\n First plot: number of malignant and begin cancer.<\/p>\n Picture 1. Count of Benign and Malignant cancer<\/p>\n We have 357 benign and 212 malignant samples of cancer.<\/p>\n Split our data into train and test set and normalize them.<\/p>\n Principal Component Analysis (PCA) is by far the most popular dimensionality reduction algorithm.<\/p>\n Another very useful piece of information is the Explained Variance Ratio<\/strong><\/a> of each principal component. It indicates the proportion of the dataset’s variance.<\/p>\n <\/p>\n Picture 2. Variance ratio of PCA without Std<\/p>\n As you can see in Picture 2., only one variable is necessary without data normalization. But to learn more, let’s make data standardization\u00a0presented in Picture 3.<\/p>\n Picture 3. Variance ratio of PCA with Std<\/p>\n As you can see in Picture 3., only six variables are necessary without data standardization to reach 95% of the variance.<\/p>\n In this section, we compare the classification results of several popular classifiers and neural networks with different architecture.<\/p>\n Support Vector Machines (SVM)<\/strong><\/a><\/p>\n SVM accuracy = 98,83%<\/p>\n K-Nearest Neighbours (K-NN)<\/strong><\/a><\/p>\n K-NN accuracy: 96,74%<\/p>\n Decision Tree<\/a><\/strong><\/p>\n simple visualization of Decision Tree:<\/p>\n Picture 4. Visualization of Decision Tree<\/p>\n <\/p>\n <\/p>\n Decision Tree accuracy: 96,24%<\/p>\n Random Forest<\/a><\/strong><\/p>\n Random Forest accuracy = 95,9%<\/p>\n\n
Python packages<\/h2>\n
\n
Data processing<\/h2>\n
# Load data\r\ndata = pd.read_csv('Data\/data.csv', delimiter=',', header=0)<\/pre>\n# Head method show first 5 rows of data\r\nprint(data.head())<\/pre>\n
id diagnosis ... fractal_dimension_worst Unnamed: 32\r\n0 842302 M ... 0.11890 NaN\r\n1 842517 M ... 0.08902 NaN\r\n2 84300903 M ... 0.08758 NaN\r\n3 84348301 M ... 0.17300 NaN\r\n4 84358402 M ... 0.07678 NaN<\/pre>\n
# Drop unused columns\r\ncolumns = ['Unnamed: 32', 'id', 'diagnosis']\r\n\r\n# Convert strings -> integers\r\nd = {'M': 0, 'B': 1}\r\n\r\n# Define features and labels\r\ny = data['diagnosis'].map(d)\r\nX = data.drop(columns, axis=1)<\/pre>\n# Plot number of M - malignant and B - benign cancer\r\n\r\nax = sns.countplot(y, label=\"Count\", palette=\"muted\")\r\nB, M = y.value_counts()\r\nplt.savefig('count.png')\r\nprint('Number of benign cancer: ', B)\r\nprint('Number of malignant cancer: ', M)<\/pre>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/count.png\")
<\/p>\n# Split dataset into training (80%) and test (20%) set\r\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\r\n\r\n# Normalize data\r\nX_train_N = (X_train-X_train.mean())\/(X_train.max()-X_train.min())\r\nX_test_N = (X_test-X_train.mean())\/(X_test.max()-X_test.min())<\/pre>\n
Dimensionality Reduction<\/h2>\n
![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/pcavariancewithoutstd.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/pcavariancewithstd.png\")
<\/p>\nClassification<\/h2>\n
svc = svm.SVC(kernel='linear', C=1)\r\n\r\n# Pipeline\r\nmodel = Pipeline([\r\n ('reduce_dim', pca),\r\n ('svc', svc)\r\n])\r\n\r\n# Fit\r\nmodel.fit(X_train_N, y_train)\r\nsvm_score = cross_val_score(model, X, y, cv=10, scoring='accuracy')<\/pre>\ndef KnearestNeighbors():\r\n \"\"\"\r\n Function for compute accuracy using K-NN algorithm\r\n :return: k-NN score\r\n \"\"\"\r\n for i in range(1, 5):\r\n knn = KNeighborsClassifier(n_neighbors=i)\r\n knnp = Pipeline([\r\n ('reduce_dim', pca),\r\n ('knn', knn)\r\n ])\r\n k_score = cross_val_score(knnp, X, y, cv=10, scoring=\"accuracy\")<\/pre>\ntrees = tree.DecisionTreeClassifier()\r\ntreeclf = trees.fit(X_train_N, y_train)\r\ntreep = Pipeline([\r\n ('reduce_dim', pca),\r\n ('trees', trees)\r\n ])\r\nscore_trees = cross_val_score(treep, X, y, cv=10)<\/pre>\nfeature_names = X.columns.values\r\n\r\ndef plot_decision_tree1(a,b):\r\n \"\"\"\r\n Function for plot decision tree\r\n :param a: decision tree classifier\r\n :param b: feature names\r\n :return: graph\r\n \"\"\"\r\n dot_data = tree.export_graphviz(a, out_file='Plots\/tree.dot',\r\n feature_names=b,\r\n class_names=['Malignant','Benign'],\r\n filled=False, rounded=True,\r\n special_characters=False)\r\n graph = graphviz.Source(dot_data)\r\n return graph<\/pre>\n
![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/graphviz.png\")
<\/p>\nrf = RandomForestClassifier()\r\nrfp = Pipeline([\r\n ('reduce_dim', pca),\r\n ('rf', rf)\r\n])\r\nscore_rf = cross_val_score(rfp, X, y, cv=10)<\/pre>\n
![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/10\/cf.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/confusionmatrix.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/10\/precision.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/10\/recall.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/10\/f1.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/roccurve.png\")
<\/p>\n![](\"https:\/\/ermlab.com\/wp-content\/uploads\/2018\/09\/corrmap.png\")
<\/p>\n