supervised classification Archives

What is Ensemble Learning in Machine Learning

June 29, 2023

Ensemble learning typically refers to methods that generate several models which are combined to make a prediction, either in classification or regression problems. This approach has been the object of a significant amount of research in recent years and good results have been reported. This section introduced the basics of the ensemble learning of classification.

Overview

Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem. In contrast to common machine learning approaches which try to learn one hypothesis from training data, ensemble methods try to construct a set of hypotheses and combine them to use. An ensemble contains several learners which are usually called base learners. The generalization ability of an ensemble is usually much stronger than that of base learners. Ensemble learning is appealing because it can boost weak learners which are slightly better than random guesses to strong learners which can make very accurate predictions. So, “base learners” are also referred to as “weak learners”. It is noteworthy, however, that although most theoretical analyses work on weak learners, base learners used in practice are not necessarily weak since using not-so-weak base learners often results in better performance. Ensemble learning is the process by which multiple models, such as classifiers or experts, are strategically generated and combined to solve a particular computational intelligence problem. Ensemble learning is primarily used to improve the (classification, prediction, function approximation, etc.) performance of a model, or reduce the likelihood of an unfortunate selection of a poor one.

The Ensemble Framework

A typical ensemble method for classification tasks contains the following building blocks:

The training set— a labeled dataset used for ensemble training. The training set can be described in a variety of languages. Most frequently, the instances are described as attribute-value vectors. We use the notation A to denote the input attributes containing n attributes: and y to represent the class variable or the target attribute.
Base Inducer— the inducer is an induction algorithm that obtains a training set and forms a classifier representing the generalized relationship between the input and target attributes. Let me represent an inducer. We use the notation M = I(S) for representing a classifier M which was induced by inducer I on a training set S.
Diversity Generator— this component is responsible for generating the diverse classifiers.
Combiner— the combiner is responsible for combining the classifications of the various classifiers.

Figure 1: An Ensemble Learning Classification

It is useful to distinguish between dependent frameworks and independent frameworks for building ensembles. In a dependent framework, the output of a classifier is used in the construction of the next classifier. Thus it is possible to take advantage of knowledge generated in previous iterations to guide the learning in the next iterations. Alternatively, each classifier is built independently and their outputs are combined in some fashion.

Figure 2: Ensemble Learning Model

A Classifier ensemble is a set of learning machines whose decisions are combined to improve the performance of the pattern recognition system. Much of the efforts in classifier combination research focus on improving the accuracy of difficult problems, and managing weaknesses and strengths of each model to give the best possible decision taking into account all ensembles The use of a combination of multiple classifiers was demonstrated to be effective, under some conditions, for several pattern recognition applications.

References

[1] Zhou, Zhi-Hua, "Ensemble learning", Encyclopedia of biometrics (2015): pp. 411-416.

[2] Rokach, Lior, "Ensemble-based classifiers", Artificial Intelligence Review 33.1 (2010): pp. 1-39.

[3] Ren, Ye, Le Zhang, and Ponnuthurai N. Suganthan, "Ensemble classification and regression-recent developments, applications and future directions." IEEE Computational Intelligence Magazine 11.1 (2016): pp. 41-53.

What is Image Classification

May 10, 2023

No Comments

The classification includes a broad range of decision-theoretic approaches to identifying images (or parts thereof). All classification algorithms are based on the assumption that the image in question depicts one or more features (e.g., geometric details in the case of a manufacturing classification system, or spectral regions in the point of remote sensing, as shown in the examples below) and that each of these features belongs to one of several distinct and exclusive classes. The classes may be specified a priori by an analyst (as in supervised classification) or automatically clustered (i.e. as in unsupervised classification) into sets of prototype classes, where the analyst merely specifies the number of desired categories. (Classification and segmentation have closely related objectives, as the former is another form of component labeling that can result in the segmentation of various features in a scene.)

Definition

Image classification is the process of assigning land cover classes to pixels. It refers to the task of extracting information classes from a multiband raster image. The resulting raster from image classification can be used to create thematic maps. Depending on the interaction between the analyst and the computer during classification, there are two types of classification: supervised and unsupervised. Image classification plays an important role in environmental and socioeconomic applications. In order to improve classification accuracy, scientists have laid paths in developing advanced classification techniques.

Figure 1 Image classification

Image classification is perhaps the most important part of digital image analysis. It is very nice to have a "pretty picture" or an image, showing a magnitude of colors illustrating various features of the underlying terrain, but it is quite useless unless to know what the colors mean.

How does It work?

Image classification analyzes the numerical properties of various image features and organizes data into categories. Classification algorithms typically employ two phases of processing: training and testing. In the initial training phase, characteristic properties of typical image features are isolated, and, based on these, a unique description of each classification category, i.e. training class, is created. In the subsequent testing phase, these feature-space partitions are used to classify image features. The description of training classes is an extremely important component of the classification process. In supervised classification, statistical processes (i.e. based on an a priori knowledge of probability distribution functions) or distribution-free processes can be used to extract class descriptors. The unsupervised classification relies on clustering algorithms to automatically segment the training data into prototype classes. In either case, the motivating criteria for constructing training classes are that they are:

Independent, e.a change in the description of one training class should not change the value of another,
Discriminatory, e.different image features should have significantly different descriptions, and
Reliable, all image features within a training group should share the common definitive descriptions of that group.

A convenient way of building a parametric description of this sort is via a feature vector, where n is the number of attributes, which describe each image feature and training class. This representation allows us to consider each image feature as occupying a point, and each training class as occupying a sub-space (i.e. a representative point surrounded by some spread, or deviation), within the n-dimensional classification space. Viewed as such, the classification problem is that of determining to which sub-space class each feature vector belongs.

Image Classification Technique

Two major categories of image classification techniques include unsupervised (calculated by the software) and supervised (human-guided) classification:

Unsupervised Classification

Unsupervised classification is where the outcomes (groupings of pixels with common characteristics) are based on the software analysis of an image without the user providing sample classes. The computer uses techniques to determine which pixels are related and groups them into classes. The user can specify which algorism the software will use and the desired number of output classes but otherwise does not aid in the classification process. However, the user must have knowledge of the area being classified when the groupings of pixels with common characteristics produced by the computer have to be related to actual features on the ground (such as wetlands, developed areas, coniferous forests, etc.)

Supervised Classification

Supervised classification is based on the idea that a user can select sample pixels in an image that are representative of specific classes and then direct the image processing software to use these training sites as references for the classification of all other pixels in the image. Training sites (also known as testing sets or input classes) are selected based on the knowledge of the user. The user also sets the bounds for how similar other pixels must be to group them together. These bounds are often set based on the spectral characteristics of the training area, plus or minus a certain increment (often based on "brightness" or strength of reflection in specific spectral bands). The user also designates the number of classes that the image is classified into. Many analysts use a combination of supervised and unsupervised classification processes to develop final output analysis and classified maps.

References

[1] “Image Classification”, Tutorial, available online at: http://www.microimages.com/documentation/Tutorials/classify.pdf

[2] What is image classification? Available online at: http://desktop.arcgis.com/en/arcmap/latest/extensions/spatial-analyst/image-classification/what-is-image-classification-.htm

[3] S. V. S. Prasad , T. Satya Savithri and Iyyanki V. Murali Krishna, “Techniques in Image Classification; A Survey”, Global Journal of Researches in Engineering: F Electrical and Electronics Engineering, Volume 15, Issue 6, Version 1.0, Year 2015.