An Example of kNN Using Sklearn

---

kNN is a simple, supervised machine learning (ML) algorithm that can be used for classification or regression tasks—and is also frequently used in missing value imputation. It is based on the idea that the observations closest to a given data point are the most “similar” observations in a data set, and we can therefore classify unforeseen points based on the values of the closest existing points. By choosing K, the user can select the number of nearby observations to use in the algorithm. This slide will show you how to implement the kNN algorithm for classification, and show how different values of K affect the results.

A kNN Example Using Sklearn
K is the number of nearest neighbors to use. For classification, a majority vote is used to determined which class a new observation should fall into. Larger values of K are often more robust to outliers and produce more stable decision boundaries than very small values (K=3 would be better than K=1, which might produce undesirable results). Below is the Python source code for kNN method. The code does not work for the Web since Python is not a client-side language. To see a working example, check the W3Schools.

All data points	A new data point classified	Another new point classified
The Python Source Code for the Rightmost Figure (K=5)
view source print? 01 # Three lines to make Python compiler able to draw: 02 import sys 03 import matplotlib 04 matplotlib.use( 'Agg' ) 05   06 import matplotlib.pyplot as plt 07 from sklearn.neighbors import KNeighborsClassifier 08   09 x       = [  4,  5, 10,  4,  3, 11, 14,  8, 10, 12 ] 10 y       = [ 21, 19, 24, 17, 16, 25, 24, 22, 21, 21 ] 11 classes = [  0,  0,  1,  0,  0,  1,  1,  0,  1,  1 ] 12   13 data    = list( zip( x, y ) ) 14 knn     = KNeighborsClassifier( n_neighbors=5 ) 15 # Fit a kNN model on the model using 5 nearest neighbors. 16 knn.fit( data, classes ) 17   18 new_x   = 8 19 new_y   = 21 20 new_point = [( new_x, new_y )] 21 # Predict the class of new, unforeseen data points. 22 prediction = knn.predict( new_point ) 23   24 plt.scatter( x + [new_x], y + [new_y], c=classes + [prediction[0]] ) 25 plt.text( x=new_x-1.7, y=new_y-0.7, s=f"new point, class: {prediction[0]}" ) 26 plt.show( ) 27   28 # Two lines to make Python compiler able to draw: 29 plt.savefig( sys.stdout.buffer ) 30 sys.stdout.flush( )