In-depth learning (also known as deep structured learning or hierarchical learning ) is part of a broader family of machine learning methods based on the representation of learning data, with task-specific algorithms. Learning can be supervised, semi-supervised or unattended.
In-depth learning architecture such as deep neural networks, deep tissue beliefs and recurring neural networks have been applied to areas including computer vision, voice recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design and board game programs. , where they have produced results that are comparable to and in some cases superior to human experts.
In-depth learning models are vaguely inspired by information processing and communication patterns in the biological nervous system but have differences from the structural and functional properties of the biological brain, which make them incompatible with neuroscience evidence.
Video Deep learning
Definitions
Deep learning is a class of machine learning algorithms that:
- uses a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each layer consecutively uses the output of the previous layer as input.
- learning in supervision (eg, classification) and/or unattended behavior (eg, pattern analysis).
- studying different levels of representation corresponding to different levels of abstraction; level form the concept hierarchy.
Maps Deep learning
Overview
Most modern in-depth learning models are based on artificial neural networks, although they can also include propositional formulas or linearly arranged latent variables in deep generative models such as nodes in Deep Belief Networks and Deep Boltzmann Machines.
In in-depth learning, each level of learning to transform data input into a slightly more abstract and composite representation. In image recognition applications, raw inputs can be pixel matrices; the first representation layer can blur the pixels and encode the ends; the second layer can arrange and encode the edge settings; the third layer can encode the nose and eyes; and the fourth layer can recognize that the image contains the face. Importantly, an in-depth learning process can learn which features are optimally placed at the own level. (Of course, this does not completely eliminate the need for hand-setting; for example, varying the number of layers and the size of layers can give different levels of abstraction.)
"Deep" in "in-depth learning" refers to the number of layers through which data is transformed. More precisely, the in-depth learning system has substantial depth in credit management (CAP = depth credit path). CAP is the transformation chain from input to output. CAP describes a potential causal relationship between input and output. For a feedforward neural network, the depth of the CAP is that of the network and the number of hidden layers plus one (as the output layer is also the parameter). For recurring neural networks, where signals can propagate through layers more than once, the depth of the CAP is potentially unlimited. Unequivocally universally depth threshold divides shallow learning from in-depth learning, but most researchers agree that in-depth learning involves depth of CAP & gt; 2. CAP depth 2 has proven to be a universal approximator in the sense that it can mimic any function. Beyond that more layers do not increase the functionality of the network approximator. Additional layers help in learning the features.
In-depth learning architecture is often built with greedy layer-by-layer methods. In-depth learning helps to decipher this abstraction and choose which features improve performance.
For supervised learning tasks, the in-depth learning method avoids the engineering of features, translating data into representations between compact like main components, and obtaining a layered structure that removes redundancy in representation.
In-depth learning algorithms can be applied to unattended learning tasks. This is an important benefit because unlabeled data is more than labeled data. Examples of inner structures that can be trained in an unsupervised manner are nervous history compressors and deep trust networks.
Interpretation
Neural networks are generally interpreted in terms of universal approach theorems or probabilistic inferences.
The universal approach theorem involves the neural feedforward network capacity with a hidden layer of finite size to approach continuous function. In 1989, the first evidence was published by George Cybenko for the sigmoid activation function and generalized for the multi-layer feed-forward architecture in 1991 by Kurt Hornik.
Probabilistic interpretation comes from the field of machine learning. It features inference, as well as the concept of training and testing optimization, related to adjustments and generalizations, respectively. More specifically, probabilistic interpretations consider nonlinear activation as a cumulative distribution function. Probabilistic interpretation leads to the introduction of dropouts as regularizers in neural networks. Probabilistic interpretations were introduced by researchers including Hopfield, Widrow and Narendra and popularized in surveys such as those by Bishop.
History
The term Deep Learning was introduced to the machine learning community by Rina Dechter in 1986, and to Neural Networks by Igor Aizenberg and colleagues in 2000, in the context of Boolean threshold neurons.
The first commonly watched, in-feedforward, multilayer perceptrons published Algorithm were published by Alexey Ivakhnenko and Lapa in 1965. A 1971 paper describes a deep network of 8 layers trained by group data handling algorithm methods.
Other profound work-learning architectures, especially those built for computer vision, began with Neocognitron introduced by Kunihiko Fukushima in 1980. In 1989, Yann LeCun et al. applied a standard backpropagation algorithm, which has existed as the reverse mode of automatic differentiation since 1970, to a deep neural network with the goal of recognizing ZIP handwriting codes in the mail. When the algorithm works, training takes 3 days.
In 1991 the system was used to recognize hand-written 2-D numbers, while recognizing 3-D objects was done by matching 2-D images to the artificial 3-D object model. Weng et al. suggested that the human brain does not use 3-D monolithic object models and in 1992 they published Cresceptron, a method for introducing 3-D objects in a messy scene. Cresceptron is a cascade layer similar to Neocognitron. But while Neocognitron requires human programmers to combine these features, Cresceptron learns a number of open features in each unattended layer, where each feature is represented by the convolution kernel. Cresceptron divides every object learned from a messy scene through re-analysis through the network. The max merge, now often adopted by deep neural networks (eg the ImageNet test), was first used in Cresceptron to reduce position resolution by a factor (2x2) to 1 through cascade for better generalization.
In 1994, AndrÃÆ'Â © de Carvalho, along with Mike Fairhurst and David Bisset, published experimental results from a multi-layer boolean neural network, also known as a weightless nerve network, consisting of 3-layer self-organizing extraction module extraction network alone (SOFT) followed by a multi-layer class of nerve classification (GSN), which is trained independently. Each layer in the feature extraction module features feature extracted with the growing complexity of the previous layer.
In 1995, Brendan Frey pointed out that it is possible to train (more than two days) a network containing six fully connected layers and several hundred hidden units using a wake-up algorithm, developed in conjunction with Peter Dayan and Hinton. Many factors contributed to the slow speed, including the disappearance gradient problem that was analyzed in 1991 by Sepp Hochreiter.
A simpler model that uses task-specific features such as Gabor filters and support vector machines (SVM) was a popular choice in the 1990s and 2000s, due to the cost of ANN computing and a lack of understanding of how the brain transfers its biological network.
Shallow and deep learning (eg, repetitive webs) ANN has been explored for years. These methods never outperform the non-uniform internal-handcrafting Gaussian mixed model/Hidden Markov model (GMM-HMM) technology based on the generative model of discriminatory trained speech. The main difficulties have been analyzed, including gradient decreases and weak temporal structural correlations in neural prediction models. The additional difficulty is the lack of training data and limited computing power.
Most voice recognition researchers moved from the neural net to pursue generative modeling. Exceptions occurred at SRI International in the late 1990s. Funded by the US government's NSA and DARPA, SRI studied neural networks in speech and speech recognition. Heck speaker recognition team achieved the first significant success with neural networks in speech processing at the National Institute of Standards and evaluation of Introduction to Technology Speakers 1998. While SRI experienced success with neural networks in the introduction of speakers, they were unsuccessful in demonstrating similar success in voice recognition. The principle of elevating the "crude" feature of first hand-made optimization was successfully explored in the in-depth autoencoder architecture on a "standard" spectrogram or linear bank-filter feature in the late 1990s, demonstrating its superiority over the Mel-Cepstral feature that contains the fixed transformation stages of the spectrograph. The raw features of speech, waveform, then produce excellent large-scale results.
Many aspects of speech recognition were taken over by in-depth learning methods called Long short term memory (LSTM), recurrent neural networks published by Hochreiter and Schmidhuber in 1997. LSTM RNNs avoided gradient disappearance problems and could learn "Very Deep Learning" tasks that required memory of events occurring thousands of steps of discrete time before, which is important to speak. In 2003, LSTM began to be competitive with the introduction of traditional speech on specific tasks. It is then combined with a temporary connectionist classification (CTC) in the LSTM RNNs stack. In 2015, Google's speech recognition reported a dramatic jump in performance by 49% through CTC-trained LSTM, provided through Google Voice Search.
In 2006, publications by Geoff Hinton, Ruslan Salakhutdinov, Osindero and Tea showed how the multilevel feedforward nerves can be effectively trained previously one layer at a time, treating each layer alternately as unrestricted limited Boltzmann engines, then fine only. set it using supervised backpropagation. This paper refers to learning for a deep confidence web.
In-depth learning is part of a state-of-the-art system in various disciplines, especially computer vision and automatic voice recognition (ASR). The results of commonly used evaluation sets such as TIMIT (ASR) and MNIST (image classification), as well as various large vocabulary recognition tasks continue to be improved. Convolutional neural networks (CNNs) were replaced for ASR by CTC for LSTM. but more successful in computer vision.
The impact of in-depth learning in the industry began in the early 2000s, when CNN processed about 10% to 20% of all checks written in the US, according to Yann LeCun. The deep learning industry application for the introduction of large-scale speech begins around 2010.
The NIPS 2009 Workshop on Deep Learning for Speech Recognition is motivated by the limitations of deep generative talk-and-speech models, and the possibility of delivering more capable hardware and large-scale data so that deep neural nets can be practical. It is believed that the DNN pre-training training using the generative model of deep faith nets (DBN) will overcome the major difficulties of the neural net. However, it was found that replacing pre-training with large amounts of training data for direct backpropagation when using DNN is large, depending on the context of the output layer results in a dramatically lower error rate than the latest Gaussian mixed (GMM)/Hidden Markov Model (HMM) and also from a more sophisticated generative model-based system. The nature of the recognition errors generated by the two types of systems is very different, offering technical insight into how to integrate in-depth learning into the highly efficient run-time speech randomization system used by all major voice recognition systems. Analysis around 2009-2010, contrasting GMM (and other generative speech models) vs. the DNN model, encouraging early industry investment in learning for speech recognition, which ultimately led to widespread and dominant use in the industry. The analysis was performed with comparable performance (less than 1.5% in error rate) between discriminatory DNNs and generative models.
In 2010, the researchers extended the in-depth study of TIMIT to the introduction of large vocabulary utterances, by adopting a large output layer from DNN based on the state of HMM depending on the context constructed by the decision tree.
Progress in hardware allows renewed interest. In 2009, Nvidia was involved in the so-called "big bang" in-depth learning, "because the neural network in learning is trained with Nvidia's graphics processing unit (GPU)." That year, Google Brain used Nvidia GPU to create a reliable DNN. While there, Ng determines that the GPU can increase the speed of the inner-learning system about 100 times. Specifically, GPU is perfect for mathematical/vector mathematics involved in machine learning. GPU accelerates training algorithms in order of magnitude, reducing the walking time from week to day. Hardware optimization and custom algorithms can be used for efficient processing.
In-depth learning revolution
In 2012, a team led by Dahl won the "Merck Molecular Challenge Activity" using a multi-tasked neural network to predict the biomolecular targets of one drug. In 2014, the Hochreiter group used in-depth learning to detect off-target and toxic effects of environmental chemicals in nutrients, household products and medicines and won the "Tox21 Data Challenge" from NIH, FDA and NCATS.
Significant additional impacts in the introduction of images or objects are felt from 2011 to 2012. Although CNN trained by backpropagation has been around for decades, and the implementation of NN GPU for years, including CNN, the rapid implementation of CNN with the maximum incorporation of GPUs in style from Ciresan and colleagues needed to advance in computer vision. In 2011, this approach achieved for the first time a super human performance in a visual pattern recognition contest. Also in 2011, he won the ICDAR Chinese handwritten contest, and in May 2012, he won the ISBI image segmentation contest. Until 2011, CNN played no major role in computer vision conferences, but in June 2012, a paper by Ciresan et al. at a leading CVPR conference demonstrating how max-pooling CNNs on GPUs can dramatically improve many vision reference records. In October 2012, a similar system by Krizhevsky et al. won a large-scale ImageNet competition with a significant margin over superficial machine learning methods. In November 2012, the Ciresan et al. Also won an ICPR contest on a major medical image analysis for cancer detection, and in the following year also the MICCAI Grand Challenge on the same topic. In 2013 and 2014, the error rate at ImageNet's tasks uses deeper learning is diminishing, following a similar trend in the introduction of large-scale speech. The Wolfram Image Identification Project publishes these improvements.
Image classification is then extended to more challenging tasks to produce descriptions for images, often as a combination of CNN and LSTM.
Some researchers think that the October 2012 ImageNet victory is the beginning of a "profound learning revolution" that has transformed the AI ​​industry.
Artificial neural network
Artificial neural network ( ANNs ) or the connectionist system is a computational system that is inspired by the biological neural networks that make up the animal's brain. Such a system learns (progressively improves their ability) to perform tasks by considering instances, generally without any specific programming tasks. For example, in image recognition, they may learn to identify images containing cats by analyzing samples of images that have been labeled manually as "cat" or "no cat" and use analytic results to identify cats in other images. They have found most use in apps that are difficult to express with traditional computer algorithms using rule-based programming.
ANN is based on a collection of connected units called artificial neurons, (analogous to axons in the biological brain). Any connection (synapse) between neurons can transmit signals to other neurons. The receiver neuron (postsynaptic) can process signal (s) and then the downstream neuron signal connected to it. Neurons may have a state, generally represented by a real number, usually between 0 and 1. Neurons and synapses may also have varying weights as a result of learning, which can increase or decrease the strength of signals sent downstream.
Typically, neurons are arranged in layers. Different layers can perform different types of transformations on their inputs. The trip signal from the first (input), to the last layer (output), maybe after traversing the layer several times.
The original purpose of the neural network approach is to solve problems in the same way as the human brain. Over time, attention is focused on matching certain mental abilities, leading to biological deviations such as backpropagation, or forwarding information in reverse direction and adjusting the network to reflect that information.
Neural networks have been used on various tasks, including computer vision, speech recognition, machine translation, social network filtering, board games and video games and medical diagnosis.
By 2017, neural networks typically have several thousand to several million units and millions of connections. Although this number is some order of magnitude less than the number of neurons in the human brain, this network can perform many tasks on a level outside of humans (eg, recognizing faces, playing "Go").
Neural networks in
A deep neural network (DNN) is a neural network (ANN) with several hidden layers between the input and output layers. DNN can model a complex non-linear relationship. The DNN architecture produces a compositional model in which objects are expressed as layered primitive compositions. The additional layer allows the feature composition of the lower layer, potentially modeling complex data with fewer units than the same superficial network.
The deep architecture includes many variants of some basic approach. Each architecture has found success in a particular domain. It is not always possible to compare the performance of some architectures, unless they have been evaluated on the same data set.
DNN is usually a feedforward network in which data flows from the input layer to the output layer without rolling back.
Recurrent neural networks (RNNs), where data can flow in all directions, are used for applications such as language modeling. Longer-term memory is very effective for this use.
Convolutional deep neural networks (CNNs) are used in computer vision. CNN has also been applied to acoustic modeling for automatic voice recognition (ASR).
Challenges
Like the ANN, many problems can arise with the DNN being trained unluckyly. Two common problems are overfitting and timing.
DNN rentan overfitting karena lapisan abstraksi yang ditambahkan, yang memungkinkan mereka untuk memodelkan dependensi langka dalam data pelatihan. Metode regularisasi seperti pemangkasan unit Ivakhnenko atau pembusukan berat ( -regularization) atau sparsity ( -regularization) dapat diterapkan selama pelatihan untuk memerangi overfitting. Alternatif lain, pengaturan regularisasi secara acak menghilangkan unit dari lapisan tersembunyi selama pelatihan. Ini membantu untuk mengecualikan dependensi langka. Akhirnya, data dapat ditambah melalui metode seperti cropping dan rotating sehingga set pelatihan yang lebih kecil dapat ditingkatkan ukurannya untuk mengurangi kemungkinan overfitting.
DNN should consider many training parameters, such as size (number of layers and number of units per layer), learning level, and initial weights. Sweeping the parameter space for optimal parameters may not be feasible because of the cost of time and computing resources. Various tricks, such as grouping (counting gradients in multiple instances of training at once rather than individual instances) speed up calculations. Large processing capabilities of many-core architectures (eg, GPU or Intel Xeon Phi) have resulted in significant speed improvements in training, due to the suitability of the processing architecture for matrix and vector calculations.
Alternatively, engineers may look for other types of neural networks with easier and convergent training algorithms. CMAC (cerebellar model articulation controller) is one type of neural network. Does not require learning level or random initial weight for CMAC. The training process can be guaranteed to meet in one step with a new batch of data, and the computational complexity of the training algorithm is linear with respect to the number of neurons involved.
Decision tree in
An acyclic graph is directed in decision rules generated for classification and regression tasks with statistical techniques based on statistical-based statistics - decision flow. It builds graphs with strong connectivity by the same fusion nodes statistically, reducing in such a way the width of the predictive model and increasing the regression classification/precision based on the amount of data represented in the leaf node. Decision streams divide and aggregate data multiple times with various features. Since the data partition only becomes a statistically different group, it reduces overfitting and reduces complexity at each level of the predictive model. Leaf merging and model width reduction enforce a very deep graph generation, which can consist of hundreds of levels.
Apps
Automatic greeting recognition
Recognition of large-scale automated speech is the first and most successful in in-depth learning. LSTM RNN can learn the "Deeply Learning" task that involves multi-second intervals containing speech events separated by thousands of discrete time steps, where one time step corresponds to about 10 ms. LSTMs with forgotten gates compete with traditional speech recognition on certain tasks.
Early success in voice recognition is based on small-scale recognition tasks based on TIMIT. The data set contains 630 speakers from the eight main dialects of American English, where each speaker reads 10 sentences. Its small size allows many configurations to be tried. More importantly, TIMIT's task involves the introduction of the phone-sequence, which, unlike word order recognition, allows the weak language of the mobile phone model. This allows the strength of the acoustic modeling aspects of voice recognition easier to analyze. The error rates listed below, including the initial results and measured as the percentage of phone error rate (PER), have been summarized since 1991.
DNN's debut for speaker recognition in the late 1990s and introduction of speeches around 2009-2011 and LSTM around 2003-2007, accelerating progress in eight key areas:
- Upgraded/outgoing and DNC training and accelerated decoding
- Discriminative training in sequence
- Process features by deep model with a strong understanding of the underlying mechanism
- DNN adaptation and related inner model
- Dual multi task and transfer learning by DNN and related inner model
- CNN and how to design it to utilize speech domain knowledge as well as possible
- RNN and rich LSTM variations
- Other in-depth model types include a tensor-based model and an in-depth generative/discriminative model.
All major commercial speech recognition systems (e.g., Microsoft Cortana, Xbox, Skype Translator, Amazon Alexa, Google Now, Apple Series, Baidu and iFlyTek voice search, and various Nuance speech products, etc.). Based on deep learning.
Image recognition
A common evaluation set for image classification is the MNIST database data set. MNIST consists of handwritten digits and includes 60,000 training samples and 10,000 test samples. Like TIMIT, its small size lets users test multiple configurations. A full list of results in this set is available.
The introduction of image-based learning in has become a "super man", producing more accurate results than human contestants. It first happened in 2011.
Vehicles who studied with in-depth training now interpret the 360 ​​Â ° camera view. Another example is the Genetic Analysis of Facial Dysmorphology (FDNA) used to analyze cases of human malformations linked to large databases of genetic syndromes.
Visual arts processing
Closely related to the advances made in image recognition is the increased application of in-depth learning techniques to a variety of visual art tasks. DNN has proven themselves capable of, for example, a) identifying periods of given painting styles, b) "capturing" certain painting styles and applying them in a fun visual way to arbitrary photos, and c) resulting in beatings. imagery based on random visual input fields.
Natural language processing
Neural networks have been used to apply language models since the early 2000s. LSTM helps improve machine translation and language modeling.
The other main techniques in this field are negative sampling and word embedding. Word embedding, like word2vec , can be considered as a representational layer in an in-depth learning architecture that transforms the word atom into a word position representation relative to another word in the dataset; position is represented as a point in the vector space. Using the word embedding as the RNN input layer allows the network to parse sentences and phrases using effective composition vector grammar. A grammar of the composition vector can be regarded as a probabilistic free grammar context (PCFG) implemented by RNN. Recursive automatic encoding makers built on embedded words can rate sentence similarities and paraphrase detection. The neural architecture provides the best results for constituent decomposition, sentiment analysis, information retrieval, oral language comprehension, machine translation, contextual entity relations, introduction of writing style, Text classification and others.
Google Translate (GT) uses a large long-term long-term memory network. GNMT uses an example-based machine translation method in which the system "learns from millions of examples." It translates "whole sentences at once, not snippets." Google Translate supports more than a hundred languages.The network encodes "semantic sentences, not just memorizing phrase-to-phrase translation. "GT uses English as an intermediary between most languages, language pairs.
Drug discovery and toxicology
Most drug candidates fail to win regulatory approval. This failure is due to inadequate efficacy (on-target effects), undesirable interactions (off-target effects), or unexpected toxic effects. Research has explored the use of in-depth learning to predict biomolecular targets, off-target and toxic effects of environmental chemicals in nutrients, household products and medicines.
AtomNet is an in-depth learning system for rational structure-based drug design. AtomNet is used to predict new candidate biomolecules for target diseases such as Ebola virus and multiple sclerosis.
Customer relationship management
Deepening proficiency learning has been used to estimate the value of possible direct marketing actions, defined in terms of the RFM variables. Estimated value function proved to have a natural interpretation as the age value of the customer.
Recommended system
The recommendation system has used in-depth learning to extract meaningful features for latent factor models for content-based music recommendations. In-depth Multiview learning has been applied to learn about user preferences from multiple domains. This model uses a collaborative and content-based hybrid approach and improves recommendations in many tasks.
Bioinformatics
ANN autoencoder is used in bioinformatics, to predict gene ontology annotations and gene-function relationships.
In medical informatics, in-depth learning is used to predict sleep quality based on data from wearable devices and prediction of health complications from electronic medical record data. In-depth study also shows efficacy in health care.
Mobile advertising
Finding the right mobile audience for mobile ads is always challenging, as many data points need to be considered and assimilated before a target segment can be created and used in ad serving by any ad server. In-depth learning has been used to interpret large dimensionally large ad datasets. Many data points are collected during the internet request/serving/click ad cycle. This information can form the basis of machine learning to improve ad selection.
Image restoration
In-depth learning has been successfully applied to inverse problems such as denoising, super resolution, and inpainting. These applications include learning methods such as "Shrinkage Fields for Effective Image Restoration" that train on image datasets, and Deep Image Prior, which trains images that require recovery.
Relationship with cognitive development and human brain
Deep learning is closely related to the theoretical class of brain development (in particular, the development of the neocortex) posed by cognitive neuroscientists in the early 1990s. Developmental theories used in computational models, making them the forerunner of the deep learning system. This developmental model shared the property that the various dynamics of the proposed learning in the brain (eg, waves of nerve growth factors) supporting self-organization is somewhat analogous to the neural networks used in the in-depth learning model. Like neocortex, neural networks use layered filter hierarchies in which each layer considers information from the previous layer (or operating environment), and then passes its output (and possibly original input), to another layer. This process generates a self-regulated transducer stack, which is well tuned into its operating environment. A 1995 description states, "... the infant's brain seems to regulate itself under the influence of waves called trophic factors... different regions of the brain become connected in sequence, with one layer of tissue falling before the other and so on until the entire brain mature. "
Various approaches have been used to investigate the sensible model of learning from a neurobiological perspective. On the one hand, several variants of the backpropagation algorithm have been proposed to improve the realism of the process. Other researchers argue that unsupervised forms of learning, such as those based on hierarchical generative models and deep trust networks, may be closer to biological reality. In this case, the generative neural network model has been associated with neurobiological evidence of sampling-based processing in the cerebral cortex.
Although a systematic comparison between the organization of the human brain and neuronal encoding in deep tissues has not been established, several analogies have been reported. For example, the calculations performed by the inner learning unit may be similar to the actual neurons and nerve populations. Similarly, the representations developed by deep-learning models are similar to those measured in primate visual systems in both single units and at the population level.
Commercial activity
Many organizations use in-depth learning for specific applications. Facebook AI laboratories perform tasks such as marking automatically uploaded images with the names of the people in them.
DeepMind technology from Google developed a system capable of learning how to play Atari video games using only pixels as input data. By 2015 they are demonstrating their AlphaGo system, which learns Go games are good enough to defeat professional Go players. Google Translate uses LSTM to translate between over 100 languages.
In 2015, Blippar is demonstrating mobile augmented reality applications that use in-depth learning to recognize objects in real time.
Criticisms and comments
Deep learning has attracted both criticism and commentary, in some cases from outside the field of computer science.
Theory
The main criticism concerns the lack of theory around methods. The most common in-architecture learning is implemented using well-understood gradient descent. However, the theory around other algorithms, such as contrast differences, is less clear. (eg Is it convergent? If so, how fast? Is that close?) The inner learning method is often seen as a black box, with most confirmations being done empirically, not theoretically.
Others show that in-depth learning should be seen as a step towards realizing a strong AI, not as a comprehensive solution. Despite the power of deep learning methods, they still lack the many functions needed to realize this goal completely. Research psychologist Gary Marcus notes:
"Realistically, in-depth learning is only part of a bigger challenge for building intelligent machines.These techniques lack the way to represent causal relationships (...) have no clear way to make logical conclusions, and they are also far from integrating abstract knowledge, such as information about what objects, for what, and how they are typically used.The most powerful AI systems, such as Watson (...) use techniques such as in-depth learning as only one element in a very complicated ensemble of techniques, ranging from Bayesian inference statistical techniques to deductive reasoning. "
As an alternative to this emphasis on deep learning boundaries, one writer speculates that it is possible to train a pile of vision machine to perform the sophisticated task of distinguishing between "old master" and amateur drawing, and hypothesizes that such sensitivity might represent the basics empathy non-trivial machine. The same authors propose that this would be in line with anthropology, which identifies attention with aesthetics as a key element of behavioral modernity.
In a further reference to the idea that artistic sensitivity may be inherent in the relatively low level of the cognitive hierarchy, a series of published graphical representations of the internal state of neural networks in (20-30 layers) attempt to see in basically random data the images on which they train shows visual appeal: original research notices received over 1,000 comments, and were the subject of what was at the time the most frequently accessed article on The Guardian's website. Error
Some in-depth learning architectures display problem behaviors, such as grouping unrecognizable images with confidence as belonging to familiar categories of known images and very small misclassifications of properly classified images. Goertzel hypothesizes that this behavior is caused by limitations in their internal representation and that this limitation will inhibit integration into the heterogeneous multi-component AGI architecture. These problems may be overcome by an in-depth learning architecture that internally forms a homologous state against the grammatical decomposition of the observed entities and events. Studying the grammar (visual or language) of the training data would be equivalent to limiting the system for logical reasoning that operates on concepts in terms of rules of grammatical production and is a basic objective of human language acquisition and AI.
Cyberthreat
As in-depth learning moves from lab to world, research and experience show that artificial neural networks are vulnerable to hacks and fraud. By identifying the pattern that this system uses to work, an attacker can modify the input to ANN such that the ANN finds matches not recognized by human observers. For example, an attacker can make subtle changes to the image so that ANN finds a match even if the image is not visible to humans like a search target. Such manipulation is called "hostile attack." In 2016 researchers used one ANN for doctors' imagery in trial and error mode, identified another focal point and thus produced a deceptive image of it. The modified image looks no different from the human eye. Another group showed that the printed image was later photographed successfully fooled the image classification system. One defense is the reverse image search, where a fake image may be sent to a site like TinEye which can then find other examples of it. Improvements are searches using only parts of the image, to identify images of snippets that might have been captured .
Another group suggests that certain psychedelic spectacles may deceive the facial recognition system into thinking that ordinary people are celebrities, potentially allowing one person to imitate another's identity. In 2017, researchers add stickers to stop signs and cause ANN to misrepresent them.
However, ANN can be further trained to detect fraudulent attempts, potentially causing attackers and defenders into arms races similar to the kind that already define the malware defense industry. ANN has been trained to defeat ANN's anti-malware software by repeatedly attacking defenses with malware that is constantly being converted by genetic algorithms to deceive anti-malware while retaining its ability to damage targets.
Another group demonstrates that certain sounds can make the Google Now voice commands system open certain web addresses that will download malware.
In "data poisoning", false data continues to be smuggled into the machine learning system training device to prevent it from reaching the mastery.
See also
- Artificial intelligence app
- Comparison of learning software in
- Compressed sensing
- Echo state network
- List of artificial intelligence projects
- Liquid state machine
- List of datasets for machine learning studies
- Computing reservoir
- rarely coding
References
External links
Source of the article : Wikipedia
0 Comments