**National Science
Foundation**

**Army Research Office,
Army DEVCOM**

**Office of Naval
Research**

**This software is not supported. If you have trouble making it work, please
figure it out yourself. The methods are
explained in the papers.**

Bidram, A.; Davoudi, A.; Lewis, F.L., “A Multiobjective
Distributed Control Framework for Islanded AC Microgrids**,” **IEEE Trans. Industrial Informatics, Volume 10, no. 3, Pages: 1785 – 1798, 2014.

Ling-ling Fan, V. Nasirian, H. Modares, F.L. Lewis, Y.D. Song, and A. Davoudi, “Game-theoretic Control of Active Loads in DC Microgrids,” IEEE Trans. Energy Conversion, vol. 31, no. 3, pp. 882-895, 2016.

V. Nasirian, H. Modares, F. Lewis, and A. Davoudi, "Active loads of a microgrid as players in a differential game," Proc. IEEE 7th International Symposium on Resilient Control Systems, Paper ID: ID-000388, Aug. 2015

K.G. Vamvoudakis, F.L. Lewis, and G.R. Hudas, “Multi-Agent Differential Graphical Games: online adaptive learning solution for synchronization with optimality,” Automatica, vol. 48, no. 8, pp. 1598-1611, Aug. 2012.

M. Abouheaf, K. Vamvoudakis, S. Haesaert,
F. Lewis, and R. Babuska, “Multi-Agent Discrete-Time Graphical Games and
Reinforcement Learning Solutions,” *Automatica*, Vol. 50(12), pp.
3038-3053, 2014.

M. Abouheaf, F. Lewis, M. Mahmoud, and D. Mikulski, “Discrete-Time
Dynamic Graphical Games: Model-Free Reinforcement Learning Solution,” *Control
Theory and Technology*, vol. 13(1), pp. 333-347, 2015.

[1]
L. Zhu, H. Modares, Gan Oon Peen, F.L. Lewis, and Baozeng Yue, “Adaptive
Suboptimal Output-Feedback Control for Linear Systems Using Integral
Reinforcement Learning,” IEEE Trans. Control Systems Technology, vol.
23, no. 1, pp. 264-273, Jan. 2015.

Link to software for basic discrete-time ADP-

HDP

DHP

AD HDP (Q learning)

AD DHP

[1] A. Al-Tamimi, M. Abu-Khalaf, and F.L. Lewis, “Adaptive Critic Designs for Discrete-Time Zero-Sum Games with Application to H-Infinity Control,” IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, no. 1, pp. 240-247, Feb. 2007.

[2]
A. Al-Tamimi, F.L. Lewis, and M. Abu-Khalaf,
“Model-free Q-learning designs for linear discrete-time zero-

Link to software for nonlinear discrete-time ADP

[1] B. Kiumarsi, F.L. Lewis, H. Modares, and M.B. Naghibi-Sistani, “Reinforcement Q-Learning for Optimal Tracking Control of Linear Discrete-time Systems with Unknown Dynamics,” Automatica, vol. 50, pp. 1167-1175, 2014.

Link to software for Tracking Control Using Q Learning

[1] D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, “Adaptive optimal control for continuous-time linear systems based on policy iteration,” Automatica, vol. 45, pp. 477-484, 2009.

[2]
D. Vrabie and F.L. Lewis

Link to Vrabie software for continuous-time ADP

H. Modares and F.L. Lewis, “Linear Quadratic Tracking Control of Partially-Unknown Continuous-time Systems using Reinforcement Learning,” IEEE Trans. Automatic Control, vol. 59, no. 11, pp. 3051-3068, Nov. 2014.

Link to Modares LQT software for continuous-time ADP

H. Modares, F.L. Lewis, and Z.-P. Jiang, “H-infinity Tracking Control of Completely-unknown Continuous-time Systems via Off-policy Reinforcement Learning ,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 10, pp. 2550-2562, Oct. 2015.

H. Modares,
F.L. Lewis, and M.B. Naghibi Sistani, “Integral
reinforcement learning and experience replay for adaptive optimal control of
partially-unknown constrained-input continuous-time systems,” Automatica, vol.
50, pp. 193-202, 2014.

F.L. Lewis and K.G. Vamvoudakis, “Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data,” IEEE Trans. Systems, Man, And Cybernetics- Part B: vol. 41, no. 1, pp. 14-25, Feb. 2011.

Link to software for ADP using OPFB

[1]
K. Vamvoudakis, D. Vrabie, and F. Lewis, “Online policy
iteration based algorithms to solve the continuous-time infinite horizon
optimal control problem, “Proc. IEEE Symp. ADPRL, pp.
36-41,

Link to software for synch PI opt control

[2] K.G. Vamvoudakis and F. L. Lewis, “Online Actor Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” Proc. Int. Joint Conf. on Neural Networks, pp. 3180-3187, Atlanta, June 2009.

Link to software for synch PI: Online Game solution

[1] F.L. Lewis, K. Liu, and A. Yesildirek, “Neural net robot controller with guaranteed tracking performance,” IEEE Trans. Neural Networks, vol. 6, no. 3, pp. 703-715, 1995.

[2] F.L. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural net robot controller with guaranteed tracking performance, IEEE Trans. Neural Networks, vol. 7, no. 2, pp. 388-399, Mar. 1996.

Link to software for neural network adaptive control

[1] M.
Abu-Khalaf, J. Huang, and F.L. Lewis,* **Nonlinear
H2/H-Infinity Constrained Feedback Control: **A Practical Design Approach Using Neural Networks**,* Springer-Verlag, Berlin, 2006.

[2] M. Abu-Khalaf and F.L. Lewis, “Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach,” Automatica, vol. 41, pp. 779-791, 2005.

Link to software for VFA Offline Optimal Design to solve nonlinear HJB equation

[3] M. Abu-Khalaf, F.L. Lewis, and J. Huang, “Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems,” IEEE Trans. Neural Networks, vol. 19, no. 7, pp. 1243-1252, July 2008.

Link to software for VFA Offline Optimal Game Design to solve nonlinear HJI equation

[1]
J. Gadewadikar, Frank L. Lewis,
L. Xie, V. Kucera and M. Abu-Khalaf, “Parameterization of all
stabilizing H_{∞}
static state-feedback gains: Application to output-feedback design,” *Automatica*, vol. 43, no. 9, pp. 1597-1604, September 2007.

[2] J. Gadewadikar; F.L. Lewis; K. Subbarao; B. M. Chen, “Structured H-Infinity Command and Control-Loop Design for Unmanned Helicopters,” J. Guidance, Control, and Dynamics, vol.31, no.4 , pp. 1093-1102, 2008.

Link to software for H-infinity ac/rotorcraft controller design

[3] A. Das, F.L. Lewis, and K. Subbarao, “Backstepping approach for controlling a quadrotor using Lagrangian form dynamics,” J. Intelligent & Robotic Systems, vol. 56, no. 1-2, pp. 127-152, Sept. 2009.

Link to software for quadrotor controller design

for WSN and Manufacturing Systems

[1] D. Tacconi and F.L. Lewis, “A new matrix model for discrete event systems: application to simulation,” IEEE Control Systems Magazine, pp. 62-71, Oct. 1997.

[2] J. Mireles and F.L. Lewis, “Intelligent Material Handling: Development and implementation of a matrix-based discrete-event controller,” IEEE Trans. Industrial Electronics, vol. 48, no. 6, pp. 1087-1097, Dec. 2001.

[3] V. Giordano, P.Ballal, F.L. Lewis, B. Turchiano, J.B. Zhang, “Supervisory control of mobile sensor networks: matrix formulation, simulation and implementation,” IEEE Trans. Systems, Man, Cybernetics- Part B, vol. 36, no. 4, pp. 806-819, Aug. 2006.

[4] P. Ballal and F.L. Lewis, “Deadlock free dynamic resource assignment in multi-robot systems with multiple missions: Application in Wireless Sensor Networks”, J. Control Theory and Applications, 2009, to appear.

Link to software for Discrete-Event controller simulation based on matrix model