Software Related to Funded Research of F.L. Lewis


This Software was developed under Research Sponsored by:

National Science Foundation

Army Research Office, Army DEVCOM

Office of Naval Research


This software is not supported.  If you have trouble making it work, please figure it out yourself.  The methods are explained in the papers.


Microgrid Cooperative Control

Bidram, A.; Davoudi, A.; Lewis, F.L., “A Multiobjective Distributed Control Framework for Islanded AC Microgrids,” IEEE Trans. Industrial Informatics, Volume 10, no. 3, Pages: 1785 1798, 2014.

Link to software


Ling-ling Fan, V. Nasirian, H. Modares, F.L. Lewis, Y.D. Song, and A. Davoudi, “Game-theoretic Control of Active Loads in DC Microgrids,” IEEE Trans. Energy Conversion, vol. 31, no. 3, pp. 882-895, 2016.

V. Nasirian, H. Modares, F. Lewis, and A. Davoudi, "Active loads of a microgrid as players in a differential game," Proc. IEEE 7th International Symposium on Resilient Control Systems, Paper ID: ID-000388, Aug. 2015

Link to software


Graphical Games

K.G. Vamvoudakis, F.L. Lewis, and G.R. Hudas, “Multi-Agent Differential Graphical Games: online adaptive learning solution for synchronization with optimality,” Automatica, vol. 48, no. 8, pp. 1598-1611, Aug. 2012.

Link to software


M. Abouheaf, K. Vamvoudakis, S. Haesaert, F. Lewis, and R. Babuska, “Multi-Agent Discrete-Time Graphical Games and Reinforcement Learning Solutions,” Automatica, Vol. 50(12), pp. 3038-3053, 2014.

Link to software


M. Abouheaf, F. Lewis, M. Mahmoud, and D. Mikulski, “Discrete-Time Dynamic Graphical Games: Model-Free Reinforcement Learning Solution,” Control Theory and Technology, vol. 13(1), pp. 333-347, 2015.

Link to software


Output Feedback Design Using Integral Reinforcement Learning

[1]            L. Zhu, H. Modares, Gan Oon Peen, F.L. Lewis, and Baozeng Yue, “Adaptive Suboptimal Output-Feedback Control for Linear Systems Using Integral Reinforcement Learning,” IEEE Trans. Control Systems Technology, vol. 23, no. 1, pp. 264-273, Jan. 2015.

Link to software


Approx. Dynamic Programming for Discrete-Time Systems

Link to software for basic discrete-time ADP-



AD HDP (Q learning)



[1]    A. Al-Tamimi, M. Abu-Khalaf, and F.L. Lewis, “Adaptive Critic Designs for Discrete-Time Zero-Sum Games with Application to H-Infinity Control,” IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, no. 1, pp. 240-247, Feb. 2007.

[2]    A. Al-Tamimi, F.L. Lewis, and M. Abu-Khalaf, “Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control,” Automatica, vol. 43, pp. 473-481, 2007.

Link to software for nonlinear discrete-time ADP


[1]            B. Kiumarsi, F.L. Lewis, H. Modares, and M.B. Naghibi-Sistani, “Reinforcement Q-Learning for Optimal Tracking Control of Linear Discrete-time Systems with Unknown Dynamics,” Automatica, vol. 50, pp. 1167-1175, 2014.

Link to software for Tracking Control Using Q Learning


Approx. Dynamic Programming for Continuous-Time Systems

[1]    D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, “Adaptive optimal control for continuous-time linear systems based on policy iteration,” Automatica, vol. 45, pp. 477-484, 2009.

[2]    D. Vrabie and F.L. Lewis, “Neural network approach to continuous-time direct adaptive optimal control for partially-unknown nonlinear systems,” Neural Networks, to appear, 2009.

Link to Vrabie software for continuous-time ADP


H. Modares and F.L. Lewis, “Linear Quadratic Tracking Control of Partially-Unknown Continuous-time Systems using Reinforcement Learning,” IEEE Trans. Automatic Control, vol. 59, no. 11, pp. 3051-3068, Nov. 2014.

Link to Modares LQT software for continuous-time ADP


H. Modares, F.L. Lewis, and Z.-P. Jiang, “H-infinity Tracking Control of Completely-unknown Continuous-time Systems via Off-policy Reinforcement Learning ,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 10, pp. 2550-2562, Oct. 2015.

Link to above software


H. Modares, F.L. Lewis, and M.B. Naghibi Sistani, “Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems,” Automatica, vol. 50, pp. 193-202, 2014.

Link to above software



Approx. Dynamic Programming Using Output Feedback

F.L. Lewis and K.G. Vamvoudakis, “Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data,” IEEE Trans. Systems, Man, And Cybernetics- Part B: vol. 41, no. 1, pp. 14-25, Feb. 2011.

Link to software for ADP using OPFB


Synchronous Policy Iteration for Continuous-Time Systems:  Online Learning of Optimal Ctrl. and Game Saddle Point

[1]            K. Vamvoudakis, D. Vrabie, and F. Lewis, “Online policy iteration based algorithms to solve the continuous-time infinite horizon optimal control problem, “Proc. IEEE Symp. ADPRL, pp. 36-41, Nashville, Mar. 2009.

Link to software for synch PI opt control


[2]            K.G. Vamvoudakis and F. L. Lewis, “Online Actor Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” Proc. Int. Joint Conf. on Neural Networks, pp. 3180-3187, Atlanta, June 2009.

Link to software for synch PI: Online Game solution


Neural Network Adaptive Control

[1]            F.L. Lewis, K. Liu, and A. Yesildirek, “Neural net robot controller with guaranteed tracking performance,” IEEE Trans. Neural Networks, vol. 6, no. 3, pp. 703-715, 1995.

[2]            F.L. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural net robot controller with guaranteed tracking performance, IEEE Trans. Neural Networks, vol. 7, no. 2, pp. 388-399, Mar. 1996.

Link to software for neural network adaptive control


Off-Line Design of Optimal Control Systems Using NN Value Function Approximation

[1]     M. Abu-Khalaf, J. Huang, and F.L. Lewis, Nonlinear H2/H-Infinity Constrained Feedback Control: A Practical Design Approach Using Neural Networks, Springer-Verlag, Berlin, 2006.

[2]    M. Abu-Khalaf and F.L. Lewis, “Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach,” Automatica, vol. 41, pp. 779-791, 2005.

Link to software for VFA Offline Optimal Design to solve nonlinear HJB equation


[3]    M. Abu-Khalaf, F.L. Lewis, and J. Huang, “Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems,” IEEE Trans. Neural Networks, vol. 19, no. 7, pp. 1243-1252, July 2008.

Link to software for VFA Offline Optimal Game Design to solve nonlinear HJI equation


Aircraft/Rotorcraft Control

[1]    J. Gadewadikar, Frank L. Lewis, L. Xie, V. Kucera and M. Abu-Khalaf, “Parameterization of all stabilizing H static state-feedback gains: Application to output-feedback design,” Automaticavol. 43, no. 9pp. 1597-1604, September 2007.

[2]    J. Gadewadikar; F.L. Lewis; K. Subbarao; B. M. Chen, “Structured H-Infinity Command and Control-Loop Design for Unmanned Helicopters,” J. Guidance, Control, and Dynamics, vol.31, no.4 , pp. 1093-1102, 2008.

Link to software for H-infinity ac/rotorcraft controller design


[3]    A. Das, F.L. Lewis, and K. Subbarao, “Backstepping approach for controlling a quadrotor using Lagrangian form dynamics,” J. Intelligent & Robotic Systems, vol. 56, no. 1-2, pp. 127-152, Sept. 2009.

Link to software for quadrotor controller design


Discrete Event Systems and Supervisory Rule-Based Control:  Matrix Model
for WSN and Manufacturing Systems

[1]    D. Tacconi and F.L. Lewis, “A new matrix model for discrete event systems:  application to simulation,” IEEE Control Systems Magazine, pp. 62-71, Oct. 1997.

[2]    J. Mireles and F.L. Lewis, “Intelligent Material Handling:  Development and implementation of a matrix-based discrete-event controller,” IEEE Trans. Industrial Electronics, vol. 48, no. 6, pp. 1087-1097, Dec. 2001.

[3]    V. Giordano, P.Ballal, F.L. Lewis, B. Turchiano, J.B. Zhang, “Supervisory control of mobile sensor networks: matrix formulation, simulation and implementation,” IEEE Trans. Systems, Man, Cybernetics- Part B, vol. 36, no. 4, pp. 806-819, Aug. 2006.

[4]    P. Ballal and F.L. Lewis, “Deadlock free dynamic resource assignment in multi-robot systems with multiple missions: Application in Wireless Sensor Networks”, J. Control Theory and Applications, 2009, to appear.

Link to software for Discrete-Event controller simulation based on matrix model