Abstract on data classification methods

- Naive Bayes classifier . The formula for calculating the probability of classifying an observation as one or another class:

#imports
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB

#model fit
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=0)
gnb = GaussianNB()
y_pred = gnb.fit(X_train, y_train).predict(X_test)

#result
print("Number of mislabeled points out of a total %d points : %d"
            % (X_test.shape[0], (y_test != y_pred).sum())) 

- Method of nearest neighbors . Classifies each observation according to the degree of similarity to other observations. The algorithm is nonparametric (there are no restrictions on the data, for example, the distribution function) and uses lazy training (pre-trained models are not used, all available data are used during classification).

#imports
from sklearn.neighbors import KNeighborsClassifier

#model fit
X = [[0], [1], [2], [3]]
y = [0, 0, 1, 1]
neigh = KNeighborsClassifier(n_neighbors=3)
neigh.fit(X, y)

#result
print(neigh.predict([[1.1]]))
print(neigh.predict_proba([[0.9]]))
 

- Support Vector Method (SVM) . Each data object is represented as a vector (point) in p-dimensional space. The task is to separate the points with a hyperplane. That is, is it possible to find such a hyperplane so that the distance from it to the nearest point is maximum. There can be many sought-after hyperplanes; therefore, it is believed that maximizing the gap between classes contributes to a more confident classification.

— . , , . . — , , , , . .

#imports
from sklearn import svm

#model fit
X = [[0, 0], [1, 1]]
y = [0, 1]
clf = svm.SVC()
clf.fit(X, y)

#result
clf.predict([[2., 2.]])
 

- Decision trees . Dividing data into subsamples according to a certain condition in the form of a tree structure. Mathematically, the division into classes occurs until all the conditions that determine the class as precisely as possible are found, i.e., when there are no representatives of another class in each class. In practice, a limited number of characteristics and layers are used, and there are always two branches.

#imports
from sklearn.datasets import load_iris
from sklearn import tree

#model fit
X, y = load_iris(return_X_y=True)
clf = tree.DecisionTreeClassifier()
clf = clf.fit(X, y)

#result
tree.plot_tree(clf.fit(iris.data, iris.target)) 

- / . . — . (random patching) . oob-.

: , , , , , . , . — , . , ( 100 000), — .

#imports
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification

#model fit
X, y = make_classification(n_samples=1000, n_features=4,
                           n_informative=2, n_redundant=0,
                           random_state=0, shuffle=False)
                           
clf = RandomForestClassifier(max_depth=2, random_state=0)
clf.fit(X, y)

#result
print(clf.feature_importances_)
print(clf.predict([[0, 0, 0, 0]]))

- . ( hinge loss function). .

Strengths : high accuracy of classification and forecasting, suitable for multi-class classification. Weaknesses - sensitivity to model parameters.

#imports
from sklearn.linear_model import SGDClassifier

#model fit
X = [[0., 0.], [1., 1.]]
y = [0, 1]
clf = SGDClassifier(loss="hinge", penalty="l2", max_iter=5)
clf.fit(X, y)

#result
clf.predict([[2., 2.]])
clf.coef_
clf.intercept_

- . . , . , , .

: , , , . — .

#imports
import numpy as np
import matplotlib.pyplot as plt
from sklearn import ensemble
from sklearn import datasets
from sklearn.utils import shuffle
from sklearn.metrics import mean_squared_error

#model fit
boston = datasets.load_boston()
X, y = shuffle(boston.data, boston.target, random_state=13)
X = X.astype(np.float32)
offset = int(X.shape[0] * 0.9)
X_train, y_train = X[:offset], y[:offset]
X_test, y_test = X[offset:], y[offset:]
params = {'n_estimators': 500, 'max_depth': 4, 'min_samples_split': 2,
          'learning_rate': 0.01, 'loss': 'ls'}
clf = ensemble.GradientBoostingRegressor(**params)
clf.fit(X_train, y_train)

#result
mse = mean_squared_error(y_test, clf.predict(X_test))
print("MSE: %.4f" % mse)

- /logit. 0 1, (log likelihood). — Y w.

: , . — , .

#imports
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression

#model fit
X, y = load_iris(return_X_y=True)
clf = LogisticRegression(random_state=0).fit(X, y)
clf.predict(X[:2, :])

#result
clf.predict_proba(X[:2, :])
clf.score(X, y) 

-Probit. , , , .

#imports
import statsmodels

#model fit
result_3 = statsmodels.discrete.
    discrete_model.Probit(labf_part, ind_var_probit )

#result
print(result_3.summary()) 

-Tobit. , .

#imports
from sklearn.datasets import make_regression
import matplotlib.pyplot as plt
import pandas as pd
from tobit import *

#model fit
tr = TobitModel()

#result
tr = tr.fit(x, y, cens, verbose=False)
tr.coef_

- , , . , , , . .