Search results for: finite time convergence

Authors: Z. Bouchama, N. Essounbouli, A. Hamzaoui, M. N. Harmas

Abstract:

A new robust finite time synergetic controller is presented based on recently developed synergetic control methodology and a terminal attractor technique. A Fast Terminal Synergetic Control (FTSC) is proposed for controlling DC-DC buck converter. Unlike Synergetic Control (SC) and sliding mode control, the proposed control scheme has the characteristics of finite time convergence and chattering free phenomena. Simulation of stabilization and reference tracking for buck converter systems illustrates the approach effectiveness while stability is assured in the Lyapunov sense and converse Lyapunov results involving scalar differential inequalities are given for finite-time stability.

Keywords: dc-dc buck converter, synergetic control, finite time convergence, terminal synergetic control, fast terminal synergetic control, Lyapunov

Procedia PDF Downloads 426

Authors: Reza Mollapourasl, Majid Haghi

Abstract:

In this study, we develop local meshfree methods known as radial basis function-generated finite difference (RBF-FD) method and Hermite finite difference (RBF-HFD) method to design stencil weights and spatial discretization for Helmholtz equation. The convergence and stability of schemes are investigated numerically in three dimensions with irregular shaped domain. These localized meshless methods incorporate the advantages of the RBF method, finite difference and Hermite finite difference methods to handle the ill-conditioning issue that often destroys the convergence rate of global RBF methods. Moreover, numerical illustrations show that the proposed localized RBF type methods are efficient and applicable for problems with complex geometries. The convergence and accuracy of both schemes are compared by solving a test problem.

Keywords: radial basis functions, Hermite finite difference, Helmholtz equation, stability

Procedia PDF Downloads 63

Authors: H. Abderrezek, M. N. Harmas

Abstract:

DC-DC converters are widely used as reliable power source for many industrial and military applications, computers and electronic devices. Several control methods were developed for DC-DC converters control mostly with asymptotic convergence. Synergetic control (SC) is a proven robust control approach and will be used here in a so-called terminal scheme to achieve finite time convergence. Lyapunov synthesis is adopted to assure controlled system stability. Furthermore particle swarm optimization (PSO) algorithm, based on an integral time absolute of error (ITAE) criterion will be used to optimize controller parameters. Simulation of terminal synergetic control of a DC-DC converter is carried out for different operating conditions and results are compared to classic synergetic control performance, that which demonstrate the effectiveness and feasibility of the proposed control method.

Keywords: DC-DC converter, PSO, finite time, terminal, synergetic control

Procedia PDF Downloads 473

Authors: Sifeddine Abderrahmani, Sonia Bouafia

Abstract:

The finite element method is the most practical tool for the analysis of structures, whatever the geometrical shape and behavior. It is extensively used in many high-tech industries, such as civil or military engineering, for the modeling of bridges, motor bodies, fuselages, and airplane wings. Additionally, experience demonstrates that engineers like modeling their structures using the most basic finite elements. Numerous models of finite elements may be utilized in the numerical analysis depending on the interpolation field that is selected, and it is generally known that convergence to the proper value will occur considerably more quickly with a good displacement pattern than with a poor pattern, saving computation time. The method for creating finite elements using the strain approach (S.B.A.) is presented in this presentation. When the results are compared with those provided by equivalent displacement-based elements, having the same total number of degrees of freedom, an excellent convergence can be obtained through some application and validation tests using recently developed membrane elements, plate bending elements, and flat shell elements. The effectiveness and performance of the strain-based finite elements in modeling structures are proven by the findings for deflections and stresses.

Keywords: finite elements, plate bending, strain approach, displacement formulation, shell element

Procedia PDF Downloads 64

Authors: Raphael Zanella

Abstract:

This works deals with the finite element approximation of axisymmetric problems. The weak formulation of the heat equation under the axisymmetry assumption is established for continuous finite elements. The weak formulation is implemented in a C++ solver with implicit march-in-time. The code is verified by space and time convergence tests using a manufactured solution. The solving of an example problem with an axisymmetric formulation is compared to that with a full-3D formulation. Both formulations lead to the same result, but the code based on the axisymmetric formulation is much faster due to the lower number of degrees of freedom. This confirms the correctness of our approach and the interest in using an axisymmetric formulation when it is possible.

Keywords: axisymmetric problem, continuous finite elements, heat equation, weak formulation

Procedia PDF Downloads 156

Authors: Kwounghee Choi

Abstract:

Technological convergence between the various sectors is expected to have a very large impact on future industrial and economy. This study attempts to do empirical approach between specific technologies’ classification. For technological convergence classification, it is necessary to set the target technology to be analyzed. This study selected target technology from national research and development plan. At first we found a source technology for analysis. Depending on the weight of source technology, NT-based, BT-based, IT-based, ET-based, CS-based convergence types were classified. This study aims to empirically show the concept of convergence technology and convergence types. If we use the source technology to classify convergence type, it will be useful to make practical strategies of convergence technology.

Keywords: technology convergence, source technology, convergence type, R&D strategy, technology classification

Procedia PDF Downloads 442

Authors: Swarn Singh, Suruchi Singh

Abstract:

In this paper, we present a fourth order two level implicit finite difference scheme for 1D Pennes bio-heat equation. Unconditional stability and convergence of the proposed scheme is discussed. Numerical results are obtained to demonstrate the efficiency of the scheme. In this paper we present a fourth order two level implicit finite difference scheme for 1D Pennes bio-heat equation. Unconditional stability and convergence of the proposed scheme is discussed. Numerical results are obtained to demonstrate the efficiency of the scheme.

Keywords: convergence, finite difference scheme, Pennes bio-heat equation, stability

Procedia PDF Downloads 435

Authors: Pyung Soo Kim, Eung Hyuk Lee, Mun Suck Jang

Abstract:

A packet loss probability (PLP) estimation filter with finite memory structure is proposed to estimate the packet rate mean and variance of the input traffic process in real-time while removing undesired system and measurement noises. The proposed PLP estimation filter is developed under a weighted least square criterion using only the finite traffic measurements on the most recent window. The proposed PLP estimation filter is shown to have several inherent properties such as unbiasedness, deadbeat, robustness. A guideline for choosing appropriate window length is described since it can affect significantly the estimation performance. Using computer simulations, the proposed PLP estimation filter is shown to be superior to the Kalman filter for the temporarily uncertain system. One possible explanation for this is that the proposed PLP estimation filter can have greater convergence time of a filtered estimate as the window length M decreases.

Keywords: packet loss probability estimation, finite memory filter, infinite memory filter, Kalman filter

Procedia PDF Downloads 634

Authors: Sandeep Kaur Chouhan, Jatinderdeep Kaur, S. S. Bhatia

Abstract:

The existence of sine and cosine series as a Fourier series, their L1-convergence seems to be one of the difficult question in theory of convergence of trigonometric series in L1-metric norm. In the literature so far available, various authors have studied the L1-convergence of cosine and sine trigonometric series with special coefficients. In this paper, we present a modified cosine and sine sums and criterion for L1-convergence of these modified sums is obtained. Also, a necessary and sufficient condition for the L1-convergence of the cosine and sine series is deduced as corollaries.

Keywords: conjugate Dirichlet kernel, Dirichlet kernel, L1-convergence, modified sums

Procedia PDF Downloads 320

Authors: A. B. Bolkhir, A. Elshafie, T. K. Yousif

Abstract:

This paper highlights the methods of error estimation in finite element analysis (FEA) results. It indicates that the modeling error could be eliminated by performing finite element analysis with successively finer meshes or by extrapolating response predictions from an orderly sequence of relatively low degree of freedom analysis results. In addition, the paper eliminates the round-off error by running the code at a higher precision. The paper provides application in finite element analysis results. It draws a conclusion based on results of application of methods of error estimation.

Keywords: finite element analysis (FEA), discretization error, round-off error, mesh refinement, richardson extrapolation, monotonic convergence

Procedia PDF Downloads 450

Authors: Ahmad Alhawarat, Mustafa Mamat, Mohd Rivaie, Ismail Mohd

Abstract:

Conjugate gradient method has been enormously used to solve large scale unconstrained optimization problems due to the number of iteration, memory, CPU time, and convergence property, in this paper we find a new class of nonlinear conjugate gradient coefficient with global convergence properties proved by exact line search. The numerical results for our new βK give a good result when it compared with well-known formulas.

Keywords: conjugate gradient method, conjugate gradient coefficient, global convergence

Procedia PDF Downloads 410

Authors: Amir T. Payandeh Najafabadi

Abstract:

This article considers the problem of evaluating infinite-time (or finite-time) ruin probability under a given compound Poisson surplus process by approximating the claim size distribution by a finite mixture exponential, say Hyperexponential, distribution. It restates the infinite-time (or finite-time) ruin probability as a solvable ordinary differential equation (or a partial differential equation). Application of our findings has been given through a simulation study.

Keywords: ruin probability, compound poisson processes, mixture exponential (hyperexponential) distribution, heavy-tailed distributions

Procedia PDF Downloads 308

Authors: Rishikesh Yadav, Ramakanta Meher, Vishnu Narayan Mishra

Abstract:

The main aim of this article is to investigate the statistical convergence of the summation of integral type operators and to obtain the weighted statistical convergence. The rate of statistical convergence by means of modulus of continuity and function belonging to the Lipschitz class are also studied. We discuss the convergence of the defined operators by graphical representation and put a better rate of convergence than the Szasz-Mirakjan-Kantorovich operators. In the last section, we extend said operators into bivariate operators to study about the rate of convergence in sense of modulus of continuity and by means of Lipschitz class by using function of two variables.

Keywords: The Szasz-Mirakjan-Kantorovich operators, statistical convergence, modulus of continuity, Peeters K-functional, weighted modulus of continuity

Procedia PDF Downloads 166

Authors: Raoul Ouambo Tobou, Alexis Kuitche, Marcel Edoun

Abstract:

The FWM (Finite Window Method) is a new numerical meshfree technique for solving problems defined either in terms of PDEs (Partial Differential Equation) or by a set of conservation/equilibrium laws. The principle behind the FWM is that in such problem each element of the concerned domain is interacting with its neighbors and will always try to adapt to keep in equilibrium with respect to those neighbors. This leads to a very simple and robust problem solving scheme, well suited for transfer problems. In this work, we have applied the FWM to an unsteady scalar convection-diffusion equation. Despite its simplicity, it is well known that convection-diffusion problems can be challenging to be solved numerically, especially when convection is highly dominant. This has led researchers to set the scalar convection-diffusion equation as a benchmark one used to analyze and derive the required conditions or artifacts needed to numerically solve problems where convection and diffusion occur simultaneously. We have shown here that the standard FWM can be used to solve convection-diffusion equations in a robust manner as no adjustments (Upwinding or Artificial Diffusion addition) were required to obtain good results even for high Peclet numbers and coarse space and time steps. A comparison was performed between the FWM scheme and both a first order implicit Finite Volume Scheme (Upwind scheme) and a third order implicit Finite Volume Scheme (QUICK Scheme). The results of the comparison was that for equal space and time grid spacing, the FWM yields a much better precision than the used Finite Volume schemes, all having similar computational cost and conditioning number.

Keywords: Finite Window Method, Convection-Diffusion, Numerical Technique, Convergence

Procedia PDF Downloads 302

Authors: Amjad Naveed

Abstract:

This study analysed the role of structural change in the process of labour productivity convergence at country and regional levels. Many forms of structural changes occurred within the European Union (EU) countries i.e. variation in sectoral employment share, changes in demand for products, variations in trade patterns and advancement in technology which may have an influence on the process of convergence. Earlier studies on convergence have neglected the role of structural changes which can have resulted in different conclusion on the nature of convergence. The contribution of this study is to examine the role of structural change in testing labour productivity convergence at various levels. For the empirical purpose, the data of 19 EU countries, 259 regions and 6 industries is used for the period of 1991-2009. The results indicate that convergence varies across regional and country levels for different industries when considered the role of structural change.

Keywords: labor produvitivty, convergence, structural change, panel unit root

Procedia PDF Downloads 246

Authors: R. Haoui

Abstract:

The purpose of this work is to simulate the flow at the exit of Vulcan 1 engine of European launcher Ariane 5. The geometry of the propellant nozzle is already determined using the characteristics method. The pressure in the outlet section of the nozzle is less than atmospheric pressure on the ground, causing the existence of oblique and normal shock waves at the exit. During the rise of the launcher, the atmospheric pressure decreases and the shock wave disappears. The code allows the capture of shock wave at exit of nozzle. The numerical technique uses the Flux Vector Splitting method of Van Leer to ensure convergence and avoid the calculation instabilities. The Courant, Friedrichs and Lewy coefficient (CFL) and mesh size level are selected to ensure the numerical convergence. The nonlinear partial derivative equations system which governs this flow is solved by an explicit unsteady numerical scheme by the finite volume method. The accuracy of the solution depends on the size of the mesh and also the step of time used in the discretized equations. We have chosen in this study the mesh that gives us a stationary solution with good accuracy.

Keywords: finite volume, lunchers, nozzles, shock wave

Procedia PDF Downloads 260

Authors: Andreas Aceranti, Guido Bighiani, Francesca Crotto, Marco Colorato, Stefania Zaghi, Marino Zanetti, Simonetta Vernocchi

Abstract:

The management of eye parameters such as convergence, accommodation, and miosis is very complex; in fact, both the neurovegetative system and the complex Oculocephalgiria system come into play. We have found the effectiveness of the "highvelocity low amplitude" technique directed on C7-T1 (where the cilio-spinal nucleus of the budge is located) in improving the convergence parameter through the measurement of the point of maximum convergence. With this research, we set out to investigate whether the improvement obtained through the High Velocity Low Amplitude maneuver lasts over time, carrying out a pre-manipulation measurement, one immediately after manipulation and one month after manipulation. We took a population of 30 subjects with both refractive and non-refractive problems. Of the 30 patients tested, 27 gave a positive result after the High Velocity Low Amplitude maneuver, giving an improvement in the point of maximum convergence. After a month, we retested all 27 subjects: some further improved the result, others kept, and three subjects slightly lost the gain obtained. None of the re-tested patients returned to the point of maximum convergence starting pre-manipulation. This result opens the door to a multidisciplinary approach between ophthalmologists and osteopaths with the aim of addressing oculomotricity and convergence deficits that increasingly afflict our society due to the massive use of devices and for the conduct of life in closed and restricted environments.

Keywords: point of maximum convergence, HVLA, improvement in PPC, convergence

Procedia PDF Downloads 40

Authors: Raphael Zanella, Todd A. Oliver, Karl W. Schulz

Abstract:

This work deals with the finite element approximation of axisymmetric compressible flows with swirl velocity. We are interested in problems where the flow, while weakly dependent on the azimuthal coordinate, may have a strong azimuthal velocity component. We describe the approximation of the compressible Navier-Stokes equations with H1-conformal spaces of axisymmetric functions. The weak formulation is implemented in a C++ solver with explicit time marching. The code is first verified with a convergence test on a manufactured solution. The verification is completed by comparing the numerical and analytical solutions in a Poiseuille flow case and a Taylor-Couette flow case. The code is finally applied to the problem of a swirling subsonic air flow in a plasma torch geometry.

Keywords: axisymmetric problem, compressible Navier-Stokes equations, continuous finite elements, swirling flow

Procedia PDF Downloads 144

Authors: F. Berna Benli, Özgür Keskin

Abstract:

Lately, many mathematicians have been studied the statistical convergence of a sequence of fuzzy numbers. We know that Lambda-statistically convergence is a kind of convergence between ordinary convergence and statistical convergence. In this paper, we will introduce the new kind of convergence such as λ-levelwise statistical convergence. Then, we will define the concept of the λ-levelwise statistical cluster and limit points of a sequence of fuzzy numbers. Also, we will discuss the relations between the sets of λ-levelwise statistical cluster points and λ-levelwise statistical limit points of sequences of fuzzy numbers. This work has been extended in this paper, where some relations have been considered such that when lambda-statistical limit inferior and lambda-statistical limit superior for lambda-statistically convergent sequences of fuzzy numbers are equal. Furthermore, lambda-statistical boundedness condition for different sequences of fuzzy numbers has been studied.

Keywords: fuzzy number, λ-levelwise statistical cluster points, λ-levelwise statistical convergence, λ-levelwise statistical limit points, λ-statistical cluster points, λ-statistical convergence, λ-statistical limit points

Procedia PDF Downloads 432

Authors: Neha Bhardwaj

Abstract:

In this paper, we consider positive linear operators and study the Voronovskaya type result of the operator then obtain an error estimate in terms of the higher order modulus of continuity of the function being approximated and its A-statistical convergence. Also, we compute the corresponding rate of A-statistical convergence for the linear positive operators.

Keywords: Poisson distribution, Voronovskaya, modulus of continuity, a-statistical convergence

Procedia PDF Downloads 293

Authors: Maryam Moosavi, Hadi Khatibzadeh

Abstract:

The main purpose of this paper is to study the proximal point algorithm for quasi-nonexpansive mappings in Hadamard spaces. △-convergence and strong convergence of cyclic resolvents for a finite family of quasi-nonexpansive mappings one to a fixed point of the mappings are established

Keywords: Fixed point, Hadamard space, Proximal point algorithm, Quasi-nonexpansive sequence of mappings, Resolvent

Procedia PDF Downloads 52

Authors: Khyati Sharma, A. Satyanarayana Reddy

Abstract:

The topic of how many subgroups exist within a certain finite group naturally arises in the study of finite groups. Over the years, different researchers have investigated this issue from a variety of angles. The significant contributions of the key mathematicians over the time have been summarized in this article. To this end, we classify finite groups into three categories viz. (a) Groups for which the number of subgroups is less than |G|, (b) equals to |G|, and finally, (c) greater than |G|. Because every element of a finite group generates a cyclic subgroup, counting cyclic subgroups is the most important task in this endeavor. A brief survey on the number of cyclic subgroups of finite groups is also conducted by us. Furthermore, we also covered certain arithmetic relations between the order of a finite group |G| and the number of its distinct cyclic subgroups |C(G)|. In order to provide pertinent context and possibly reveal new novel areas of potential research within the field of research on finite groups, we finally pose and solicit a few open questions.

Keywords: abstract algebra, cyclic subgroup, finite group, subgroup

Procedia PDF Downloads 83

Authors: Berkay Buluş, Aytekin İşman, Kübra Yüzüncüyıl

Abstract:

Developments in media and communication technologies have changed the way we use media. The importance of convergence culture has been increasing day by day within the framework of these developments. With new media, it is possible to say that social networks are the most powerful platforms that are integrated to this digitalization process. Although social networks seem like the place that people can socialize, they can also be utilized as places of production. On the other hand, audience has become users within the framework of transformation from national to global broadcasting. User generated contents make conventional media and new media collide. In this study, these communication platforms will be examined not as platforms that replace one another but mediums that unify each other. In the light of this information, information that is produced by users regarding new media platforms and all new media use practices are called convergence culture. In other words, convergence culture means intersections of conventional and new media. In this study, examples of convergence culture will be analyzed in detail.

Keywords: new media, convergence culture, convergence, use of new media, user generated content

Procedia PDF Downloads 230

Authors: Vishnu Pratap Singh Kirar

Abstract:

Artificial Neural Network (ANN) can be trained using backpropagation (BP). It is the most widely used algorithm for supervised learning with multi-layered feed-forward networks. Efficient learning by the BP algorithm is required for many practical applications. The BP algorithm calculates the weight changes of artificial neural networks, and a common approach is to use a two-term algorithm consisting of a learning rate (LR) and a momentum factor (MF). The major drawbacks of the two-term BP learning algorithm are the problems of local minima and slow convergence speeds, which limit the scope for real-time applications. Recently the addition of an extra term, called a proportional factor (PF), to the two-term BP algorithm was proposed. The third increases the speed of the BP algorithm. However, the PF term also reduces the convergence of the BP algorithm, and criteria for evaluating convergence are required to facilitate the application of the three terms BP algorithm. Although these two seem to be closely related, as described later, we summarize various improvements to overcome the drawbacks. Here we compare the different methods of convergence of the new three-term BP algorithm.

Keywords: neural network, backpropagation, local minima, fast convergence rate

Procedia PDF Downloads 464

Authors: Temami Oussama, Bessais Lakhdar, Hamadi Djamal, Abderrahmani Sifeddine

Abstract:

The analysis of complex structures encourages the engineer to make simplifying assumptions, sometimes attempting the analysis of the whole structure as complex as it is, and it can be done using the finite element method (FEM). In the modeling of complex structures by finite elements, various elements can be used: beam element, membrane element, solid element, plates and shells elements. These elements formulated according to the classical formulation and do not generally share the same nodal degrees of freedom, which complicates the development of a compatible model. The compatibility of the elements with each other is often a difficult problem for modeling complicated structure. This compatibility is necessary to ensure the convergence. To overcome this problem, we have proposed finite elements with a rotational degree of freedom. The study used is based on the strain approach formulation with 2D and 3D formulation with different degrees of freedom at each node. For the comparison and confrontation of results; the finite elements available in ABAQUS/Standard are used.

Keywords: compatibility requirement, complex structures, finite elements, modeling, strain approach

Procedia PDF Downloads 405

Authors: Vishal Jaunky, Jonas Grafström

Abstract:

The development of the European Union’s (EU) single economic market and rapid technological change has resulted in major structural changes in EU’s member states economies. The general liberalization process that the countries has undergone together has convinced the governments of the member states of need to upgrade their economic and training systems in order to be able to face the economic globalization. Several signs of economic convergence have been found but less is known about the knowledge production. This paper addresses the convergence pattern of technological innovation in 13 European Union (EU) states over the time period 1990-2011 by means of parametric and non-parametric techniques. Parametric approaches revolve around the neoclassical convergence theories. This paper reveals divergence of both the β and σ types. Further, we found evidence of stochastic divergence and non-parametric convergence approach such as distribution dynamics shows a tendency towards divergence. This result is supported with the occurrence of γ-divergence. The policies of the EU to reduce technological gap among its member states seem to be missing its target, something that can have negative long run consequences for the market.

Keywords: convergence, patents, panel data, European union

Procedia PDF Downloads 250

Authors: Sonia Sonia

Abstract:

This paper presents a family of iterative scheme for solving nonlinear systems of equations which have wide application in sciences and engineering. The proposed iterative family is based upon some parameters which generates many different iterative schemes. This family is completely derivative free and uses first of divided difference operator. Moreover some numerical experiments are performed and compared with existing methods. Analysis of convergence shows that the presented family has fourth-order of convergence. The dynamical behaviour of proposed family and local convergence have also been discussed. The numerical performance and convergence region comparison demonstrates that proposed family is efficient.

Keywords: convergence, divided difference operator, nonlinear system, Newton's method

Procedia PDF Downloads 195

Authors: Bharti Gupta, V. K. Kukreja

Abstract:

A comprehensive numerical study is presented for the solution of time-dependent advection diffusion problems by using cubic B-spline collocation method. The linear combination of cubic B-spline basis, taken as approximating function, is evaluated using the zeros of shifted Chebyshev polynomials as collocation points in each element to obtain the best approximation. A comparison, on the basis of efficiency and accuracy, with the previous techniques is made which confirms the superiority of the proposed method. An asymptotic convergence analysis of technique is also discussed, and the method is found to be of order two. The theoretical analysis is supported with suitable examples to show second order convergence of technique. Different numerical examples are simulated using MATLAB in which the 3-D graphical presentation has taken at different time steps as well as different domain of interest.

Keywords: cubic B-spline basis, spectral norms, shifted Chebyshev polynomials, collocation points, error estimates

Procedia PDF Downloads 189

Authors: Juhee Lee, Yongjun Lee

Abstract:

In designing a low-energy-consuming buildings, the heat transfer through a large glass or wall becomes critical. Multiple layers of the window glasses and walls are employed for the high insulation. The gravity driven air flow between window glasses or wall layers is a natural heat convection phenomenon being a key of the heat transfer. For the first step of the natural heat transfer analysis, in this study the development and application of a finite volume method for the numerical computation of viscous incompressible flows is presented. It will become a part of the natural convection analysis with high-order scheme, multi-grid method, and dual-time step in the future. A finite volume method based on a fully-implicit second-order is used to discretize and solve the fluid flow on unstructured grids composed of arbitrary-shaped cells. The integrations of the governing equation are discretised in the finite volume manner using a collocated arrangement of variables. The convergence of the SIMPLE segregated algorithm for the solution of the coupled nonlinear algebraic equations is accelerated by using a sparse matrix solver such as BiCGSTAB. The method used in the present study is verified by applying it to some flows for which either the numerical solution is known or the solution can be obtained using another numerical technique available in the other researches. The accuracy of the method is assessed through the grid refinement.

Keywords: finite volume method, fluid flow, laminar flow, unstructured grid

Procedia PDF Downloads 254

Authors: Bastien Batardière, Joon Kwon

Abstract:

For finite-sum optimization, variance-reduced gradient methods (VR) compute at each iteration the gradient of a single function (or of a mini-batch), and yet achieve faster convergence than SGD thanks to a carefully crafted lower-variance stochastic gradient estimator that reuses past gradients. Another important line of research of the past decade in continuous optimization is the adaptive algorithms such as AdaGrad, that dynamically adjust the (possibly coordinate-wise) learning rate to past gradients and thereby adapt to the geometry of the objective function. Variants such as RMSprop and Adam demonstrate outstanding practical performance that have contributed to the success of deep learning. In this work, we present AdaLVR, which combines the AdaGrad algorithm with loopless variance-reduced gradient estimators such as SAGA or L-SVRG that benefits from a straightforward construction and a streamlined analysis. We assess that AdaLVR inherits both good convergence properties from VR methods and the adaptive nature of AdaGrad: in the case of L-smooth convex functions we establish a gradient complexity of O(n + (L + √ nL)/ε) without prior knowledge of L. Numerical experiments demonstrate the superiority of AdaLVR over state-of-the-art methods. Moreover, we empirically show that the RMSprop and Adam algorithm combined with variance-reduced gradients estimators achieve even faster convergence.

Keywords: convex optimization, variance reduction, adaptive algorithms, loopless

Procedia PDF Downloads 18