Machine learning enables computers to mimic human behaviour by teaching them past and future data. This article identifies several unique machine learning methods, including the Dominance-based rough set approach, Expectation–maximization algorithm, and Out-of-bag error.

Dominance-based rough set approach

Greco, Matarazzo, and Sowiski developed the dominance-based rough set approach (DRSA). It extends the rough set theory for multicriteria decision analysis (MCDA). Compared to classical rough sets, the main difference is that a dominance relation has replaced the indiscernibility relation. This method makes it possible to deal with inconsistencies when considering criteria and preference-ordered decision classes.

Multicriteria classification (sorting) is as follows: given a set of objects evaluated by criteria (attributes with preference-order domains), assign these objects to some pre-defined and preference-ordered decision classes. Due to preference order, an object's class shouldn't change if its evaluation of the criteria improves. Sorting is very similar to classification, but in classification, objects by their regular attributes and the decision classes don't have to be in the order of preference. The problem of multicriteria classification is also called the ordinal classification problem with monotonicity constraints. 

Expectation–maximization algorithm

In statistics, an expectation-maximization (EM) algorithm is an iterative way to find (local) maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models that depend on unobserved latent variables. The EM iteration alternates between an expectation (E) step that creates a function for the expected log-likelihood based on the current estimate of the parameters and a maximization (M) step that finds parameters that maximize the expected log-likelihood found in the E step. Then, in the next step, E, these parameter estimates are used to figure out how the latent variables spread.

In a classic paper from 1977, Arthur Dempster, Nan Laird, and Donald Rubin wrote about and named the EM algorithm. They said the method had been "suggested many times in special cases" by authors before them. When we can't solve the equations directly, the EM algorithm finds the (local) most likely parameters of a statistical model. Most of the time, these models have unknown parameters, known data observations, and latent variables. As a result, some data points are missing, or the model can be simpler. 

Out-of-bag error

Out-of-bag (OOB) error is a way to measure the prediction error of machine learning models using bootstrap aggregation (bagging). OOB error is the average prediction error for each training sample xi, using only the trees whose bootstrap samples did not include xi.

With bootstrap aggregating, you can get an "out-of-bag" estimate of how well your predictions are getting by testing them on observations. After bootstrap aggregation, One set, the bootstrap sample, is the data chosen to be "in the bag" by sampling with replacement. The out-of-bag set is all the information researchers didn't pick during the sampling process. When this process is over, like when making a random forest, many bootstrap samples and OOB sets. 

Image source: Unsplash