LSTM Research Papers - Academia.edu (original) (raw)

The latency of current mobile devices' touchscreens is around 100ms and has widely been explored. Latency down to 2ms is noticeable, and latency as low as 25ms reduces users' performance. Previous work reduced touch latency by extrapolating a finger's movement using an ensemble of shallow neural networks and showed that predicting 33ms into the future increases users' performance. Unfortunately, this prediction has a high error. Predicting beyond 33ms did not increase participants' performance, and the error affected the subjective assessment. We use more recent machine learning techniques to reduce the prediction error. We train LSTM networks and multilayer perceptrons using a large data set and regularization. We show that linear extrapolation causes an 116.7% higher error and the previously proposed ensembles of shallow networks cause a 26.7% higher error compared to the LSTM networks. The trained models, the data used for testing, and the source code is available on GitHub.

- by
- •
- Machine Learning, Prediction, LAG, Multilayer Perceptron

In this digital world, artificial intelligence has provided solutions to many problems, likewise to encounter problems related to digital images and operations related to the extensive set of images. We should learn how to analyze an image, and for that, we need feature extraction of the content of that image. Image description methods involve natural language processing and concepts of computer vision. The purpose of this work is to provide an efficient and accurate image description of an unknown image by using deep learning methods. We propose a novel generative robust model that trains a Deep Neural Network to learn about image features after extracting information about the content of images, for that we used the novel combination of CNN and LSTM. We trained our model on MSCOCO dataset, which provides set of annotations for a particular image, and after the model is fully automated, we tested it by providing raw images. And also several experiments are performed to check effici...

- by Shivam Gupta
- •
- Computer Science, Computer Vision, Image Processing, Natural Language Processing

The COVID-19 virus, exactly like in numerous other diseases, can be contaminated from person to person by inhalation. In order to prevent the spread of this virus, which led to a pandemic around the world, a series of rules have been set by governments that people must follow. The obligation to use face masks, especially in public spaces, is one of these rules. Objective: The aim of this study is to determine whether people are wearing the face mask correctly by using deep learning methods. Methods: A dataset consisting of 2000 images was created. In the dataset, images of a person from three different angles were collected in four classes, which are "masked", "non-masked", "masked but nose open", and "masked but under the chin". Using this data, new models are proposed by transferring the learning through AlexNet and VGG16, which are the Convolutional Neural network architectures. Classification layers of these models were removed and, Long-Short Term Memory and Bi-directional Long-Short Term Memory architectures were added instead. Result and conclusions: Although there are four different classes to determine whether the face masks are used correctly, in the six models proposed, high success rates have been achieved. Among all models, the TrVGG16 + BiLSTM model has achieved the highest classification accuracy with 95.67%. Significance: The study has proven that it can take advantage of the proposed models in conjunction with transfer learning to ensure the proper and effective use of the face mask, considering the benefit of society.

- by ilkay CINAR
- •
- Masks & Faces, LSTM, Convolutional Neural Networks, BiLSTM

The accurate forecast of wind speed is critical in the integration of renewable energy within the main electrical grid and an important factor for power electrical grid stability, scheduling, and planning. In this paper, we present the deep learning algorithms, Long Short-Term Memory (LSTM), and bidirectional LSTM algorithms (Bi-LSTM) using different configurations and different activation functions to evaluate the experiments and predict the provisional trend of wind speed. We used both models to predict the wind speed over Gabal Elzayt Wind Farm in Egypt. The used data-set belongs to NASA's monthly MERRA-2 wind speed datasets. The LSTM network using the “SoftSign” function as a state activation function and “Sigmoid” as a gate activation function showed better performance and the lowest RMSE error over other experiments. The trained model after validation is utilized to predict the provisional trend of wind speed for the time-frame 2020-2022 for the wind farm. LSTM and Bi-LSTM showed effectiveness to apply for the long-term wind prediction field.

- by Muhammed (Mohamed) ELTAHAN
- •
- Wind Speed, LSTM, Reanalysis, MERRA

Analyzing natural language-based Customer Satisfaction (CS) is a tedious process. This issue is practically true if one is to manually categorize large datasets. Fortunately, the advent of supervised machine learning techniques has paved the way toward the design of efficient categorization systems used for CS. This paper presents the feasibility of designing a text categorization model using two popular and robust algorithms – the Support Vector Machine (SVM) and Long Short-Term Memory (LSTM) Neural Network, in order to automatically categorize complaints, suggestions, feedbacks, and commendations. The study found that, in terms of training accuracy, SVM has best rating of 98.63% while LSTM has best rating of 99.32%. Such results mean that both SVM and LSTM algorithms are at par with each other in terms of training accuracy, but SVM is significantly faster than LSTM by approximately 35.47s. The training performance results of both algorithms are attributed on the limitations of the dataset size, high-dimensionality of both English and Tagalog languages, and applicability of the feature engineering techniques used. Interestingly, based on the results of actual implementation, both algorithms are found to be 100% effective in accurately predicting the correct CS categories. Hence, the extent of preference between the two algorithms boils down on the available dataset and the skill in optimizing these algorithms through feature engineering techniques and in implementing them toward actual text categorization applications.

Hearing a species in a tropical rainforest is much easier than seeing them. If someone is in the forest, he might not be able to look around and see every type of bird and frog that are there but they can be heard. A forest ranger might know what to do in these situations and he/she might be an expert in recognizing the different type of insects and dangerous species that are out there in the forest but if a common person travels to a rain forest for an adventure, he might not even know how to recognize these species, let alone taking suitable action against them. In this work, a model is built that can take audio signal as input, perform intelligent signal processing for extracting features and patterns, and output which type of species is present in the audio signal. The model works end to end and can work on raw input and a pipeline is also created to perform all the preprocessing steps on the raw input. In this work, different types of neural network architectures based on Long Short Term Memory (LSTM) and Convolution Neural Network (CNN) are tested. Both are showing reliable performance, CNN shows an accuracy of 95.62% and Log Loss of 0.21 while LSTM shows an accuracy of 93.12% and Log Loss of 0.17. Based on these results, it is shown that CNN performs better than LSTM in terms of accuracy while LSTM performs better than CNN in terms of Log Loss. Further, both of these models are combined to achieve high accuracy and low Log Loss. A combination of both LSTM and CNN shows an accuracy of 97.12% and a Log Loss of 0.16.

- by Shakeel Ahmed, PhD and +1
- •
- Spectrogram, LSTM, Multi Class Classification, Audio Classification

Document classification is a fundamental task for many applications, including document annotation, document understanding, and knowledge discovery. This is especially true in STEM fields where the growth rate of scientific publications is exponential, and where the need for document processing and understanding is essential to technological advancement. Classifying a new publication into a specific domain based on the content of the document is an expensive process in terms of cost and time. Therefore, there is a high demand for a reliable document classification system. In this paper, we focus on classification of mathematics documents, which consist of English text and mathematics formulas and symbols. The paper addresses two key questions. The first question is whether math-document classification performance is impacted by math expressions and symbols, either alone or in conjunction with the text contents of documents. Our investigations show that Text-Only embedding produces better classification results. The second question we address is the optimization of a deep learning (DL) model, the LSTM combined with one dimension CNN, for math document classification. We examine the model with several input representations, key design parameters and decision choices, and choices of the best input representation for math documents classification.

The purpose of this study is to come up with a most accurate model for predicting the Solar photovoltaic (PV) power generation and Solar irradiance. For this study, the data is collected from Faculty of Engineering, University of Jaffa solar measuring station. In this paper, deep learning based univariate long short-term memory (LSTM) approach is introduced to predict the Solar irradiance. A univariate LSTM and auto-regressive integrated moving average (ARIMA) based time series approaches are compared. Both models are evaluated using root mean-square error (RMSE). This study suggests that univariate LSTM approach performs well over ARIMA approach.

- by Ahilan Kanagasundaram
- •
- Deep Learning, LSTM, solar forecasting, Forecasting using LSTM

- by Abonia Sojasingarayar
- •
- Computer Science, Chatbot, Natural Language Processing(NLP), LSTM

There has been a strong growth in the global demand for energy over the past decade, especially in relation to renewable energy. The accelerating use of energy by household appliances, rapidly rising sales of air conditioners especially in emerging economies and the ever increasing need for heating in households has put a strain on energy demands across the world. As such, it has become more pertinent for energy producers to be able to accurately predict demand to avoid either over or under supply of electricity to households. In this paper, Apache Spark is used to process London smart meter data collected over a period of a year. The processed time-series data is then used to train a LSTM model for energy demand forecasting. When compared to using a single machine, the Spark cluster performed the same data processing task at a much shorter time, supporting the idea that its parallel architecture is more suitable for processing large datasets.

- by IAEME Publication
- •
- Cloud Computing, Data Engineering, ARIMA, LSTM
- by Nehal Mohamed Ali
- •
- Computer Science, Visualization, Data Mining, Data Warehousing

Captioning an image is a concept of producing a succinct content description for an input image in single sentence considering all the objects in an image in the form of description. It can be done using deep learning architectures with the help of CNN (Convolution Neural Network) and RNN (Recurrent Neural Network). A particular kind of RNN called long short-term memory (LSTM) is used. The image from the dataset is taken as the input and accordingly caption is produced as an output from the given set in the form of text. It has numerous applications in various fields namely Image Indexing, Application Recommendation, Social media etc. It also aids the visually impaired and short sightedness people by automatically decoding the image and describing it in the form of text in a large format.

- by IJRASET Publication
- •
- Deep Learning, CNN effect, LSTM, RNN

- by Sonu Pratap Singh Gurjar and +1
- •
- Computer Vision, Image Processing, Natural Language Processing, Deep Learning

The forecasting consists of taking historical data as inputs then using them to predict future observations, thus determining future trends. Demand prediction is a crucial component in the supply chain's process that allows each member to enhance its performance and its profit. Nevertheless, because of demand uncertainty supply chains usually suffer from many problems such as the bullwhip effect. As a solution to those logistics issues, this paper presents a comparative analysis of four time series demand forecasting models; namely, the autoregressive integrated moving Average (ARIMA) a statistical model, the multi-layer perceptron (MLP) a feedforward neural network, the long short-term memory model (LSTM) a recurrent neural network and the convolutional neural network (CNN or ConvNet) a deep learning model. The experimentations are carried out using a real-life dataset provided by a supermarket in Morocco. The results clearly show that the convolutional neural network gives slightly better forecasting results than the Long short-term memory network.

Any sequential learning task relies on the idea of connecting previous time-stamp information to the immediate present time-stamp task to predict the future. The underlying challenge is to understand the hidden patterns in the sequence by means of analyzing short-and long-term dependencies and temporal differences. Recurrent Neural Networks (RNNs) and their variants, such as Long Short-Term Memory (LSTM) are widely used in problem domains like speech recognition, Natural Language Processing (NLP), fault prediction, and language translation modeling over the past few years. Higher accuracy demands complex LSTM network models which lead to high computational cost, area overhead, and excessive power consumption. Reversible logic circuit synthesis, in the context of ideally Zero heat dissipation, has emerged as a new research paradigm for low power circuit designs. In this paper, we have proposed a novel design of LSTM architecture using reversible logic gates. To the best of our knowledge, the proposed approach is the first attempt to implement a complete feedforward LSTM circuit using only reversible logic gates. The hardware implementation of the proposed method is presented using VHDL and Altera Arria10 GX FPGA. The comparative analysis demonstrates that the proposed approach has achieved an approximately 17% reduction in overall power dissipation compared to traditional networks. The proposed approach also has better scalability than the classical design approach.

- by Kasem Khalil and +1
- •
- Machine Learning, Neural Networks, Artificial Neural Networks, Deep Learning

Intelligent Conversational Agent development using Artificial Intelligence or Machine Learning technique is an interesting problem in the field of Natural Language Processing. In many research projects, they are using Artificial Intelligence, Machine Learning algorithms and Natural Language Processing techniques for developing conversation/dialogue agent. In the past, methods for constructing chatbot architectures have relied on handwritten rules and templates or simple statistical methods. With the rise of deep learning, these models were quickly replaced by end-to-end trainable neural networks around 2015. More specifically, the recurrent encoder-decoder model [Cho et al., 2014] dominates the task of conversational modeling. This architecture was adapted from the neural machine translation domain, where it performs extremely well. Since then a multitude of variations[Serbanetal.,2016]and features were presented that augment the quality of the conversation that chatbots are capable of[Richard.,2017].Among current chatbots, many are developed using rule-based techniques, simple machine learning algorithms or retrieval based techniques which do not generate good results. In this paper, I have developed a Seq2Seq AI Chatbot using modern-day techniques. For developing Seq2Seq AI Chatbot, We have implemented encoder-decoder attention mechanism architecture. This encoder-decoder is using Recurrent Neural Network with LSTM (Long-Short-Term-Memory) cells. These conversation agents are predominately used by businesses, government organizations and non-profit organizations. They are frequently deployed by financial organizations like bank, credit card companies, businesses like online retail stores and start-ups.

- by Abonia Sojasingarayar
- •
- Chatbot, Natural Language Processing(NLP), LSTM, seq2seq

Generating accurate and timely internal and external audit reports may seem difficult for some auditors due to limited time or expertise in matching the correct clauses of the standard with the textual statement of findings. To overcome this gap, this paper presents the design of text classification models using support vector machine (SVM) and long short-term memory (LSTM) neural network in order to automatically classify audit findings and standard requirements according to text patterns. Specifically, the study explored the optimization of datasets, holdout percentage and vocabulary of learned words called NumWords, then analyzed their capability to predict training accuracy and timeliness performance of the proposed text classification models. The study found that SVM (96.74%) and LSTM (97.54%) were at par with each other in terms of the best training accuracy, although SVM (67.96±17.93 seconds [s]) was found to be significantly faster than LSTM (136.67±96.42 s) in any dataset size. The study proposed optimization formulas that highlight dataset and holdout as predictors of accuracy, while dataset and NumWords as predictors of timeliness. In terms of actual implementation, both classification models were able to accurately classify 20 out of 20 sample audit findings at 1 and 3 s, respectively. Hence, the extent of choosing between the two algorithms depend on the datasets size, learned words, holdout percentage, and workstation speed. This paper is part of a series, which explores the use of Artificial Intelligence (AI) techniques in optimizing the performance of QMS in the context of a state university.

- by Ralph Sherwin Corpuz
- •
- Optimization, Text Classification, Support Vector Machines (SVMs), LSTM

Our study explores offensive and hate speech detection for the Arabic language, as previous studies are minimal. Based on two-class, three-class, and six-class Arabic-Twitter datasets, we develop single and ensemble CNN and BiLSTM classifiers that we train with non-contextual (Fasttext-SkipGram) and contextual (Multilingual Bert and AraBert) word-embedding models. For each hate/offensive classification task, we conduct a battery of experiments to evaluate the performance of single and ensemble classifiers on testing datasets. The average-based ensemble approach was found to be the best performing, as it returned F-scores of 91%, 84%, and 80% for two-class, three-class and six-class prediction tasks, respectively. We also perform an error analysis of the best ensemble model for each task.

- by Samira Sadaoui
- •
- Deep Learning, LSTM, Deep learning Methods, Word Embedding

Climate change has affected the weather forecast on a regular basis compared to reality. Meanwhile, weather forecast
plays an important role in daily life and especially it affects developed countries in agricultural fields around the world. When
we apply information technology software, we can assess the general weather condition of a given city, and with the help of
recent modern scientific methods for more accurate analysis and prediction of weather based on those collected weather data
for a period of a week earlier or longer for future weather forecasts.

- by IJRASET Publication
- •
- Neural Network, Numerical Weather Prediction, Recurrent Neural Network, LSTM

Classifying unstructured text data written in natural languages is a cumbersome task, and this is even worse in cases of vast datasets with multiple languages. In this paper, the author explored the utilization of Long Short-Term Neural Network (LSTM) in designing a classification model that can learn text patterns and classify English and Tagalog-based complaints, feedbacks and commendations of customers in the context of a state university in the Philippines. Results shown that the LSTM has its best training accuracy of 91.67% and elapsed time of 34s when it is tuned with 50 word embedding size and 50 hidden units. The study found that the lesser the number of hidden units in the network correlates to a higher classification accuracy and faster training time, but word embedding size has no correlation to the classification performance. Furthermore, results of actual testing proven that the proposed text classification model was able to predict 19 out of 20 test data correctly, hence, 95% classification accuracy. This means that the method conducted was effective in realizing the primary outcome of the study. This paper is part of a series of studies that employs machine and deep learning techniques toward the improvement of data analytics in a Quality Management System (QMS).

- by Evangelia Oikonomou
- •
- Artificial Intelligence, Image Processing, Machine Learning, Data Mining

Intelligent Models for predicting diseases whether building a model to help the doctor or even preventing its spread in an area globally, is increasing day by day. Here we present a noble approach to predict the disease prone area using the power of Text Analysis and Machine Learning. Epidemic Search model using the power of the social network data analysis and then using this data to provide a probability score of the spread and to analyse the areas whether going to suffer from any epidemic spread-out, is the main focus of this work. We have tried to analyse and showcase how the model with different kinds of pre-processing and algorithms predict the output. We have used the combination of words-n grams, word embeddings and TF-IDF with different data mining and deep learning algorithms like SVM, Naïve Bayes and RNN-LSTM. Naïve Bayes with TF-IDF performed better in comparison to others.

Green Roofs (GRs) are increasing in popularity due to their ability to manage roof runoff while providing a number of additional ecosystem services. Improvement of hydrological models for the simulation of GRs will aid design of individual roofs as well as city scale planning that relies on the predicted impacts of widespread GR implementation. Machine learning (ML) has exploded in popularity in recent years, however there are no studies focusing on the use of ML in hydrological simulation of GRs. We focus on two types of ML-based model: long short-term memory (LSTM) and gated recurrent unit (GRU), in modelling GRs hydrological performance, with sequence input andsingle output (SISO), and synced sequence input and output (SSIO) architectures. Results of this paper indicate that both LSTM and GRU are useful tools for GR modelling. As the time window length (memory length, time step length of input data) increases, SISO appears to have a higher overall forecast accuracy. SSIO delivers the best overall performance, when the SSIO is close to, or even exceeds, the maximum window size.

- by K.W. Chau
- •
- Machine Learning, Hydrology, Applications of Machine Learning, Hydrologic Modelling

This paper tackle the problem of query expansion in Arabic based on ontologies and user data. In this context, a way to extend the search request in order to be able to enrich the request in Arabic is proposed; in order to later integrate it into a system for information retrieval. This system analyzes the search request and then passes it to the ontologies to obtain appropriate concepts, so that the result will be in the form of an expanded request. To ensure results are in line with the requirements and characteristics of each user, personalization based on user data is employed. For evaluation, deep learning models are resorted such us Long short-term memory (LSTM), Gated recurrent neural network (GRU), Bi directionel- Long short-term memory (Bi-LSTM) and Bi-gated recurrent neural network (Bi-GRU). Our results are comparable to best state of the art methods

- by IAEME Publication
- •
- Extension, LSTM, GRU, Bi-LSTM

The Efficient Market Hypothesis states that markets are self correcting and that predicting them accurately every time is impossible. In the age where machine learning and natural computing algorithms have various applications, we test the prediction efficacy of LSTMs on intraday time-series data for forecasting price of index FTSE 100 (UK). In our study, we have created 2 Deep Learning architectures with LSTM and used Support Vector Machine Regression and Multivariate Linear Regression to benchmark the results and draw comparison. We start with raw prices in dataset and later incorporate some technical indicators as well. A bit of feature engineering is also attempted using k-means clustering. We have carefully designed our experiments to reach an optimal level of accuracy where the algorithm 'learns' and not merely 'memorizes' the dataset. Hence tradeoffs have been made to sacrifice RMSE values in order to have a better goodness of fit.

- by Aman Thakur
- •
- Forecasting, Deep Learning, LSTM, Timeseries Forecasting
- by Adnan Ul-Hasan
- •
- OCR, Mocr, LSTM

Stock index prices predicting is a tough task and, because of various reasons relating to many technological and non-tech reasons, share price knowledge is an extremely difficult, unpredictable and dynamic environment. In parallel to deep learning techniques, a variety of academic experiments from different disciplines to resolve this topic and machine learning techniques are one of the many technologies used. Many machine learning techniques in this field were able to produce acceptable outcomes while it was used in this type of predictions. This paper studies stock market price prediction using LSTM model which is applied on Stock index prices historical data along with indications analysis which will be used to achieve more accurate results. In this study, data sets of historical prices of common stock of Agilent Technology, and American Airlines Group Common Stock were gathered to achieve this objective, and several tests were carried out using LSTM, the findings were evaluated using RMSE and RMSPE values that guarantee better performance for the LSTM method used.

- by IAEME Publication
- •
- Prediction, Stock Market, LSTM, RNN

Crimes are common social problems that can even affect the quality of life, even the economic growth of a country. Big Data Analytics (BDA) is used for analyzing and identifying different crime patterns, their relations, and the trends within a large amount of crime data. Here, BDA is applied to criminal data in which, data analysis is conducted for the purpose of visualization. Big data analytics and visualization techniques were utilized to analyze crime big data within the different parts of India. Here, we have taken all the states of Indian for analysis, visualization and prediction. The series of operations performed are data collection, data pre-processing, visualization and trends prediction, in which LSTM model is used. The data includes different cases of crimes with in different years and the crimes such as crime against women and children in which, kidnap, murder, rape. The predictive results show that the LSTM perform better than neural network models. Hence, the generated outcomes will benefit for police and law enforcement organizations to clearly understand crime issues and that will help them to track activities, predict the similar incidents, and optimize the decision making process.

- by IJRASET Publication
- •
- Visualization, Prediction, LSTM, Crime Data
- by Houcine Feriani
- •
- Actuarial Science, Statistics, Time series analysis, Predictive Analytics

With the advent of Big Data, nowadays in many applications databases containing large quantities of similar time series are available. Forecasting time series in these domains with traditional univariate forecasting procedures leaves great potentials for producing accurate forecasts untapped. Recurrent neural networks (RNNs), and in particular Long Short-Term Memory (LSTM) networks, have proven recently that they are able to outperform state-of-the-art univariate time series forecasting methods in this context when trained across all available time series. However, if the time series database is heterogeneous, accuracy may degenerate, so that on the way towards fully automatic forecasting methods in this space, a notion of similarity between the time series needs to be built into the methods. To this end, we present a prediction model that can be used with different types of RNN models on subgroups of similar time series, which are identified by time series clustering techniques. We...

- by Slawek Smyl
- •
- Computer Science, Time Series, Forecasting, Mathematical Sciences

The swift progress in the study field of human-computer interaction (HCI) causes to increase in the interest in systems for Speech emotion recognition (SER). The speech Emotion Recognition System is the system that can identify the emotional states of human beings from their voice. There are well works in Speech Emotion Recognition for different language but few researches have implemented for Arabic SER systems and that because of the shortage of available Arabic speech emotion databases. The most commonly considered languages for SER is English and other European and Asian languages. Several machine learning-based classifiers that have been used by researchers to distinguish emotional classes: SVMs, RFs, and the KNN algorithm, hidden Markov models (HMMs), MLPs and deep learning. In this paper we propose ASERS-LSTM model for Arabic Speech Emotion Recognition based on LSTM model. We extracted five features from the speech: Mel-Frequency Cepstral Coefficients (MFCC) features, chromagram, Melscaled spectrogram, spectral contrast and tonal centroid features (tonnetz). We evaluated our model using Arabic speech dataset named Basic Arabic Expressive Speech corpus (BAES-DB). In addition of that we also construct a DNN for classify the Emotion and compare the accuracy between LSTM and DNN model. For DNN the accuracy is 93.34% and for LSTM is 96.81%.

Due to the enormous amount of data and opinions being produced, shared and transferred everyday across the internet and other media, Sentiment analysis has become vital for developing opinion mining systems. This paper introduces a developed classification sentiment analysis using deep learning networks and introduces comparative results of different deep learning networks. Multilayer Perceptron (MLP) was developed as a baseline for other networks results. Long short-term memory (LSTM) recurrent neural network, Convolutional Neural Network (CNN) in addition to a hybrid model of LSTM and CNN were developed and applied on IMDB dataset consists of 50K movies reviews files. Dataset was divided to 50% positive reviews and 50% negative reviews. The data was initially pre-processed using Word2Vec and word embedding was applied accordingly. The results have shown that, the hybrid CNN_LSTM model have outperformed the MLP and singular CNN and LSTM networks. CNN_LSTM have reported the accuracy of 89.2% while CNN has given accuracy of 87.7%, while MLP and LSTM have reported accuracy of 86.74% and 86.64 respectively. Moreover, the results have elaborated that the proposed deep learning models have also outperformed SVM, Naïve Bayes and RNTN that were published in other works using English datasets.

When processing arguments in online user interactive discourse, it is often necessary to determine their bases of support. In this paper, we describe a supervised approach, based on deep neural networks, for classifying the claims made in online arguments. We conduct experiments using convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) on two claim data sets compiled from online user comments. Using different types of distributional word embeddings, but without incorporating any rich, expensive set of features, we achieve a significant improvement over the state of the art for one data set (which categorizes arguments as factual vs.\ emotional), and performance comparable to the state of the art on the other data set (which categorizes claims according to their verifiability). Our approach has the advantages of using a generalized, simple, and effective methodology that works for claim categorization on different data sets and tasks.

Crude oil and petroleum products are among the critical inputs of industrial production and have an essential role in logistics and transportation. Hence, sudden increases and decreases in oil prices cause particular problems in global economies and thus, they have a direct or indirect effect on economies. Furthermore, due to crises in developing economies, trade disputes between major economies, and the dynamic nature of the oil price effect on demand and supply for oil and petroleum products, and time to time volatility in the oil price are very severe. The uncertainty in oil prices can leave both consumers and producers with heavy potential losses. Due to this rapid variability, predicting oil prices has global importance. In this study, to increase the accuracy and stability, the Long-Short Term Memory (LSTM) and Facebook's Prophet (FBPr) were applied to foresee future tendencies in Brent oil prices considering their previous prices. Comparing the two models made using the 32-year data set between June 1988 and June 2020 weekly for oil prices, and the model with the best fit was determined. The dataset was split into two sets: training and test sets-the twenty-five years are used for the training set and the seven years are used to validate forecasting accuracy. The coefficient of determination (R 2) for the LSTM and FBPr models was found as 0.92, 0.89 in the training stage, and 0.89, 0.62 in the testing stage, respectively. According to the results obtained, the LSTM model has superior results to predict the trend of oil prices.

- by Erdemalp Özden
- •
- Artificial Intelligence, Machine Learning, Deep Learning, Makine öğrenmesi

The main reason behind the spread of fake news is because of many fake and hyperpartisan sites present on the Internet. These fake sites try to manipulate the truth which creates misunderstanding in society. Therefore, it is important to detect fake news and try to make people aware of the truth. This paper gives an insight into how to detect fake news using Machine Learning and Deep Learning Techniques. On observing our data, we have categorized our data into five attributes namely Title, Text, Subject, Date, and Labels. In order to develop an efficient fake news detection system, the feature along with its degree of impact on the system must be taken into consideration. This paper attempts at providing a detailed analysis of detecting fake news using various models such as LSTM, ANN, Naïve Bayes, SVM, Logistic Regression, XGBoost, and Bert.

- by IAEME Publication
- •
- Machine Learning, NLP, Annales school, Deep Learning

Countless learning tasks require dealing with sequential data. Image captioning, speech synthesis, and music generation all require that a model produce outputs that are sequences. In other domains, such as time series prediction, video analysis, and musical information retrieval, a model must learn from inputs that are sequences. Interactive tasks, such as translat- ing natural language, engaging in dialogue, and controlling a robot, often demand both capabilities. Recurrent neural networks (RNNs) are connec- tionist models that capture the dynamics of sequences via cycles in the network of nodes. Unlike standard feedforward neural networks, recurrent networks retain a state that can represent information from an arbitrarily long context window. Although recurrent neural networks have tradition- ally been difficult to train, and often contain millions of parameters, recent advances in network architectures, optimization techniques, and paral- lel computation have enabled successful large-scale learning with them. In recent years, systems based on long short-term memory (LSTM) and bidirectional (BRNN) architectures have demonstrated ground-breaking performance on tasks as varied as image captioning, language translation, and handwriting recognition. In this survey, we review and synthesize the research that over the past three decades first yielded and then made practical these powerful learning models. When appropriate, we reconcile conflicting notation and nomenclature. Our goal is to provide a self- contained explication of the state of the art together with a historical perspective and references to primary research.

Although the applications of deep neural networks (deep learning) have allowed investors to estimate the direction of the movement of financial assets, there are not still many results on these applications for the so-called " alternative currencies " such as Bitcoin (BTC), which has had phenomenal growth during 2020. That is why we consider pertinent to study and explore investment applications of various typical, and not so typical, statistical models of data science for the problem of estimating daily direction future of BTC. In particular, the penultimate model shown is an LSTM neural network whose topology is different from classical networks and makes them especially useful for working with data that have a short-and long-term time dependence, such as financial time series. We have used in this project the Python language and real historical data from the BTC. The originality of the work is due not only to the application of various statistical models but also to the incorporation of metrics called " on-chain " linked purely and exclusively to the block chain, and which are gaining more and more relevance.

- by Agustin Muñoz González
- •
- Statistical Modeling, Data Science, Python Programming, Bitcoin

- by IAEME Publication
- •
- Prediction, Stock Market, LSTM, RNN

Financial markets are fascinating if you can predict them. Also, the traders acting on financial markets produce a vast amount of information to analyse the consequences of investing according to the current market trends. Stock Market prediction is the technique to determine whether stock value will go up or down as it plays an active role in the financial gain of nation's economic status. The foundation factor for each investor is to gain maximum profits on their investments. If the company's profits go up, you own some of those profits and if they go down, you lose profits with them. We propose a framework using Long Short Term Memory machine learning algorithm and adaptive stock technical indicators for efficient forecasting by using various parameters obtained from the historical data set considered for a particular company. This algorithm works on historical data retrieved from Yahoo Finance. For prediction of share price using Long Short Term Memory, there are two modules, one is training session and other is predicting price based on previously trained data. The results will attempt to predict whether a stock price in the future will be higher or lower than it is on a given day to increase transparency among investors in the market. The proposed model also attempts to use sentiment analysis of financial news and opinions fetched from social media platform like Twitter as it acts as another influencing factor in governing stock trends.

- by IJRASET Publication
- •
- Machine Learning, LSTM, RNN, long short-term memory (LSTM)

This paper focuses on liquidity modelling to explore the world of limit order book markets. By trying to predict the bid-ask spread across a month of active trading days, we aim to compare the accuracy of different algorithms by the model Root Mean Square Error of said algorithms. We first implement two different regression algorithms, namely Linear Regression and Autoregressive Integrated Moving Average (ARIMA) model. After examining the faults and assumptions behind the regression algorithms, we then apply a recurrent neural network based language model, specifically the Long Term Short Term Memory (LSTM). Empirical findings are provided to support the rationale of each proposed algorithm.

- by Jaime Isabel Koh Naden
- •
- Machine Learning, Statistical Modeling, Liquidity management, ARIMA

In recurrent neural networks such as the Long Short-Term Memory (LSTM), the sigmoid and hyperbolic tangent functions are commonly used as activation functions in the network units. Other activation functions developed for the neural networks are not thoroughly analyzed in LSTMs. While many researchers have adopted LSTM networks for classification tasks, no comprehensive study is available on the choice of activation functions for the gates in these networks. In this paper, we compare 23 different kinds of activation functions in a basic LSTM network with a single hidden layer. Performance of different activation functions and different number of LSTM blocks in the hidden layer are analyzed for classification of records in the IMDB, Movie Review, and MNIST data sets. The quantitative results on all data sets demonstrate that the least average error is achieved with the Elliott activation function and its modifications. Specifically, this family of functions exhibit better results than the sigmoid activation function which is popular in LSTM networks.

- by Amir Farzad and +1
- •
- Machine Learning, Deep Learning, LSTM, Activation Functions

Many emerging social sites, famous forums, review sites, and many bloggers generate huge amount of data in the form of user sentimental reviews, emotions, opinions, arguments, viewpoints etc. about different social events, products, brands, and politics, movies etc. Sentiments expressed by the users has great effect on readers, political images, online vendors. So the data present in scattered and unstructured manner needs to be managed properly and in this context sentiment analysis has got attention at very large level. Sentiment analysis can be defined as organization of the text which is used to understand the mindsets or feelings expressed in the form of different manners such as negative, positive, neutral, not satisfactory etc. This paper explains the implementation and accuracy of sentiment analysis using Tensor flow and python with any kind of text data. It works on embedding, LSTM and Sigmoid layers and finds the accuracy of data in iterative manner for better result.

- by GJESR Journal
- •
- Sentiment Analysis, Deep Learning, Sigmoid, LSTM

Cheating during exams is a problem in the field of education. Cheating during exams undermine the efforts to evaluate the student's proficiency and growth. We propose a real-time cheating detection system using video feed that allows the ability to monitor students during written exams for any illegal behaviors and gestures, such as giving codes, looking at friends, using a cheat sheet, talking and exchanging papers between students. The gestures recognized during the runtime of the video from sequences of actions performed by the subjects which are then used to generate textual descriptions based on the detected cheating gestures. These textual descriptions help the process of documenting activities that transpired during the exams for later use. Our proposed system comprises two primary subsystems, a gesture recognition model based on 3DCNN and XGBoost and a language generation model based on an LSTM network. The gesture recognition model achieves recognition of the cheating gestures with 81.11% accuracy and Kappa statistic 0.760. The language generation model achieves 95.3 % word accuracy and average edit distance 1.076 on single subject description sentences, and 96.6% word accuracy and average edit distance 3.305 on interaction description sentences. The system runs at 32.54 fps on a mid-range laptop.

- by Mohamad Ivan Fanany
- •
- Computer Vision, Cheating and Plagiarism, Machine Learning, Video Processing

Recurrent Neural Networks (RNNs) have become a popular method for learning sequences of data. It is sometimes tough to parallelize all RNN computations on conventional hardware due to its recurrent nature. One challenge of RNN is to find the optimal structure for RNN because of computing complex hidden units exist. This paper presents a new approach to Long Short-Term Memory (LSTM) that aims to reduce the cost of the computation unit. The proposed Economic LSTM (ELSTM) is designed using a few hardware units to perform its functionality. ELSTM has fewer units compared to the existing LSTM versions which makes it very attractive in processing speed and hardware design cost. The proposed approach is tested using three datasets and compared with the other methods. The simulation results show the proposed method has comparable accuracy with the other methods. At the hardware level, the proposed method is implemented on Altera FPGA.

- by Kasem Khalil and +1
- •
- Machine Learning, Pattern Recognition, Face Recognition, Recognition

Activity detection is becoming an integral part of many mobile applications. Therefore, the algorithms for this purpose should be lightweight to operate on mobile or other wearable device, but accurate at the same time. In this paper, we develop a new lightweight algorithm for activity detection based on Long Short Term Memory networks, which is able to learn features from raw accelerometer data, completely bypassing the process of generating hand-crafted features. We evaluate our algorithm on data collected in controlled setting, as well as on data collected under field conditions, and we show that our algorithm is robust and performs almost equally good for both scenarios, while outperforming other approaches from the literature.

In the field of sentiment classification, opinions or sentiments of the people are analyzed. Sentiment analysis systems are being applied in social platforms and in almost every business because the opinions or sentiments are the reflection of the beliefs, choices and activities of the people. With these systems it is possible to make decisions for businesses to political agendas. In recent times a huge number of people share their opinions across the Internet using Bengali. In this paper a new way of sentiment classification of Bengali text using Recurrent Neural Network(RNN) is presented. Using deep recurrent neural network with BiLSTM, the accuracy 85.67% is achieved.

As part of the shared task of GermEval 2018 we developed a system that is able to detect offensive speech in German tweets. To increase the size of the existing training set we made an application for gathering trending tweets in Germany. This application also assists in manual annotation of those tweets. The main part of the training data consists of the set provided by the organizers of the shared task. We implement three different models. The first one follows the n-gram approach. The second model utilizes word vectors to create word clusters which contributes to a new array of features. Our last model is a composition of a recurrent and a convolutional neural network. We evaluate our approaches by splitting the given data into train, validation and test sets. The final evaluation is done by the organizers of the task who compare our predicted results with the unpublished ground truth.

- by Jelena Mitrović and +1
- •
- Machine Learning, German Language, Neural Networks, LSTM

The next major feature of the age of conversational services is chatbots in the new era of technology. A Chatbot
framework is a software program that uses natural language to communi- cate with users. Chatbots is a virtual entity that can
effectively explore the use of digital textual competencies with any hu- man being. Recently, their growth as a medium of
conversa- tion between people and computers has taken a great step for- ward. The aim of the chatbot framework for machine
learn- ing and artificial intelligence is to simulate a human conver-sation, maybe through text or speech. Natural Language Processing understands one or more human languages through chatbot software. To simulate informal chat communication, the
chatbot structure combines a language model and compu- tational algorithms covering enormous natural language pro- cessing
techniques. This paper discusses other applications that may be useful for chatbots, such as a computer conversa- tion system,
virtual agent, dialogue system, retrieval of infor- mation, industry, telecommunications, banking, health, cus- tomer call centers,
and e-commerce. It also offers an overview of cloud-based chatbots technology along with the program- ming of chatbots and
programming problems in the chatbot’s present and future periods.

- by IJRASET Publication
- •
- NLP, Machine Intelligence, Deep Learning, Chatbots