Dynamic Modelling of Engineering Systems: A Review of Fitness-for-purpose Issues in Simulation Models and Data (original) (raw)

[1] Beisbart, C. and Saam, N.J. (Eds.), Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives, Springer Nature, Cham, Switzerland 2019. doi:10.1007/978-3-319-70766-2 [2] Zeigler, B.P., Muzy, A. and Kofman, E., Theory of Modeling and Simulation: Discrete Event and Iterative System Computational Foundations, 3rd Edition. Academic Press, San Diego, CA, USA, 2018. doi: 10.1016/C2016-0-03987-6. [3] Gelfert, A., Assessing the credibility of conceptual models. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. In C. Beisbart, N.J. Saam (Eds.) pp. 249-269, Springer Nature, Cham, Switzerland 2019.. [4] Close, C., Frederick, D.K. and Newell, J.C., Modeling and Analysis of Dynamic Systems, 3rd Edition. Wiley, New York, USA, 2001. ISBN 978-0-471-39442-6. [5] Bonvin, D., Georgakis, C., Pantelides et al., Linking models and experiments. Ind. Eng. Chem. Res. 55, 2016. doi: 10.1021/acs.iecr.5b04801. [6] Murray-Smith, D.J., Testing and Validation of Computer Simulation Models, Principles, Methods and Applications, Springer, Cham, Switzerland, 2015. doi: 10.1007/978-3-319- 15099-4. [7] Zenor, J.J., Murray-Smith, D.J., McGookin, E.W. et al., Development of a multi-rate simulation model of an unmanned underwater vehicle for real-time applications. Simulation Notes Europe, 19(3-4), 47-54, 2009. doi: 10.11128/sne.19.tn.09951. [8] Matlab/Simulink® modelling and simulation software. Mathworks Inc.. https://www.mathworks.com. 2022. [9] Modelica® and the Modelica Standard Library, The Modelica Association. https://www.modelica.org. 2022. [10] Scilab and Xcos software, ESI Group. https://www.scilab.org. 2022. [11] Caro, J.J., Briggs, A.H., Siebert, U. et al.,. Modelling good research practices - overview: a report of the ISPOR-SMDM Modelling Good Research Practices Task Force–1. Med. Decis. Making, 32(5), 667-677, 2012. doi: 10.1177/0272989X12454577. [12] Roberts, M., Russell, L.B., Paltiel, A.D. et al., Conceptualizing a model: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2. Med. Decis. Making, 32(5), 678-689, 2012. doi: 10.1177/0272989X12454941. [13] Chatfield, C., The Analysis of Time Series: an Introduction, 6th Edition, Chapman & Hall/ CRC, Boca Raton, FL, USA, 2003. doi: 10.4324/9780203491. [14] Otnes, R.K. and Enochsen, L., Applied Time Series Analysis, Vol.1 Basic Techniques, Wiley, New York, USA, 1978. ISBN 0-471-24235-2. [15] Ljung, L., System Identification: Theory for the User, 2nd Edition, Prentice Hall, Upper Saddle River, NJ, USA, 1999. ISBN 978-0-13-656695-3. [16] Walter, E. and Pronzato, L., Identification of Parametric Models from Experimental Data, Springer, Berlin/Heidelberg, Germany, 1997. ISBN 978-1-84-996996-3. [17] Bryce, G.W., Murray-Smith, D.J. Foord, T.R. et al., Hybrid simulation of water turbine governors. In R.E. Crosbie and J.L.Hay, (eds.), Simulation Councils Proceedings Series, 6(1), pp. 35-44, Simulations Councils, La Jolla, CA, USA.1976. ISSN 0037-5519. [18] Close, C., Frederick, D.K. Electromechanical systems, In: Modeling and Analysis of Dynamic Systems, C. Close and D.K. Frederick, pp. 388-424, Wiley, New York, USA, 1995. ISBN 978-0-471-39442-6. [19] Close, C., Frederick, D.K. Transfer function analysis, In: Modeling and Analysis of Dynamic Systems, C. Close and D.K. Frederick, pp. 300-354, Wiley, New York, 1995. ISBN 978-0-471-39442-6. [20] Beck, J.V. and Arnold, K.J., Parameter Estimation in Engineering and Science, Wiley, New York, USA, 1977. ISBN 978-0-471-031182. [21] Unbehauen, H. and Rao, G.P. Identification of Continuous Systems, Elsevier, Amsterdam, Netherlands, 1987. ISBN 978-0-447-03163. [22] Sὃderstrὃm, T. and Stoica, P., System Identification, Prentice Hall, New York, USA, 1989. ISBN 978-0-138-812362. [23] Raol, J.R., Girija, G. and Singh, J., Modelling and Parameter Estimation of Dynamic Systems, IET Control Engineering Series No. 65, IET, London, UK, 2004. ISBN 978-0-86341- 363-6. [24] Müller, T.G., Timmer, J., Parameter identification techniques for partial differential equations. Int. J. Bifurc. Chaos, 14(6), 2053-2060, 2004. doi: 10.1142/S0218127404010424. [25] Goldberg, D.E., Genetic Algorithms in Search, Optimization and Machine Learning, Addison Wesley, Boston, MA, USA, 1989. ISBN 978-0201157673. [26] Gray, G.J., Li, Y., Murray-Smith, D.J. et al., Structural system identification using Genetic Programming and a block diagram oriented simulation tool. Electron. Lett., 32(16), 1422, 1996. doi: 10.1049/el:19960888. [27] Tan, K.C., Li, Y. , Murray-Smith, D. et al.., System identification and linearisation using genetic algorithms with simulated annealing. In: Genetic Algorithms in Engineering Systems: Innovations and Applications (GALESIA 95), 1st IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems, pp. 164-169, IET, London, UK, 1995. doi: 10.1049/cp:19951043. [28] Otnes, R.K., Enochsen, L. Transfer functions and coherence function, In Time Series Analysis, Vol.1 Basic Techniques, R.K. Otnes and L. Enochsen, Chapter 9, Wiley, New York, 1978. ISBN 0-471-24235-2. [29] Bellman, R. , Åstrὃm, K.J., On structural identifiability. Math. Biosc., 7, 329-339, 1970. doi: 10.1016/0025-5564(70)90132-X [30] Brown, F., Godfrey, K.R., Problems of determinacy in compartmental modelling with application to bilirubin kinetics. Math. Biosc., 40, 205-224, 1978. doi: 10.1016/0025- 5564(78)90085-8 [31] Tomović, R., Sensitivity Analysis of Dynamic Systems, McGraw-Hill, New York, USA, 1963. [32] Frank, P.M., An Introduction to System Sensitivity Theory, Academic, London, 1978. ISBN 0-12-265650-4. [33] Goodwin, G.C. and Payne, R.L., Dynamic System Identification: Experiment Design and Data Analysis, Academic, New York, 1977. ISBN 978-0122897504. [34] Bache, R.A., Gray, W.M., Murray-Smith, D.J. Time domain system identification applied to non-invasive estimation of cardio-pulmonary quantities. IEE Proc. 128 Pt. D, 56-64, 1983. doi: 10.1049/ip-d:19810010. [35] Barriero, X.R. and Villaverde, A.F., Benchmarking tools for a priori identifiability analysis, Bioinformatics, 39, 2, btad065,2023. doi: 10.1093/bioinformatics/btad065. [36] Wanika, L., Egan, J.R., Swaminathan, N. et al., Structural and practical identifiability in bioengineering: a beginners guide. J. Biol. Eng. 18, 20, 2024. doi: 10.1186/s13036-024- 00410-x. [37] Bradley, R., Padfield, G.D., Murray-Smith, D.J. Validation of helicopter mathematical models. Trans. Inst. Meas. Control, 12, 186-196,1990. doi: 10.1177/014233129001200405. [38] Briggs, A.H., Weinstein, M.C., Fenwick, E.A.L. et al., Model parameter estimation and uncertainty: A report of the ISPOR-SMDM Modelling Good Research Practices Task Force–1. Med. Dec. Making, 32(5), 722–735, 2012. doi: 10.1177/0272989X12458348. [39] Villaverde, A., Pathirana, D., Froehlich, F. et al., A protocol for dynamic model calibration. Brief. Bioinform. 23, bbab387, 2021. doi: 10.1093/bib/bbab387 [40] Raue, A., Kreutz, C., Maiwald, T. et al., Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics, 25, 1923-1929, 2009. doi:10.1093/bioinformatics/btp358. [41] Gabor, A., Villaverde, A.F. and Banga, J.R., Parameter identifiability analysis and visualization in large-scale kinetic models of biosystems. BMC Systems Biology, 11, 54, 2017. doi: 10.1186/s12918-017-0428-y. [42] Wieland, F.G. Hauber, A., Rosenblatt, M. et al., On structural and practical identifiability. Curr. Opin. Syst. Biol., 25, 60–69, 2021. doi: 10.1016/j.coisb.2021.03.005. [43] Zi, Z., Sensitivity analysis approaches applied to systems biology models. IET Systems Biology, 5, 6, 336-346, 2011. doi: 10.1049/iet-syb.2011.0015. [44] D’Azzo, J.J., Houpis, C.H. and Sheldon, S.N., Linear Control System Analysis and Design with Matlab, 5th Edition, Marcel Dekker, New York, USA, 2003. ISBN 978-0824740382. [45] Ogata, K., Modern Control Engineering, 5th Edition, Prentice Hall, Upper Saddle River, NJ, USA, 2009. ISBN 978-0136156734. [46] Jagacinski, R.J. and Flach, J.M. Control Theory for Humans, Quantitative Approaches to Modeling Performance, CRC Press, Boca Raton, FL, USA, 2009. ISBN 0-8058-2292-5. [47] Schwenzer, M., Ay, M., Bergs, T. et al., Review on model predictive control: an engineering perspective. Int. J. Adv. Manuf. Technol., 117, 1327-1349, 2021. doi:10.1007/s00170-021-07682-3. [48] Willard, J., Jia,X, Xu, S. et al. Integrating physics-based modeling with machine learning: a survey. arXiv.049919v4, 2020. doi: 10.1145/1122445.1122456. [49] Selvarajan, S., Tappe, A. and Heiduk, C., Parameter identification concept for process models combining systems theory and deep learning. Eng. Proc. 19(1), 27, 2022. doi:10.3390/ECP2022-12686. [50] Fink, O., Wang, Q, Svensen, M. et al., Potential challenges and future directions for deep learning in prognostics and health management applications. Eng. App. Art. Intell., 92, 103678, 2020. doi: 10.3929/ethz-b-000414034. [51] Chao, M., Lilley, D.S., Mathé, P. et al., Calibration and uncertainty quantification of gas turbine performance models, Proc. ASME Turbine Technical Conference and Exposition (Turbo Expo 2015), Vol. 7A. Paper No: GT2015-42392, V07AT29A001, 12 pp., ASME, 2015. doi: 10.1115/GT2015-42392. [52] Chao, M.A., Kulkarni, C. Goebel, K. et al., Fusing physics-based and deep learning models for prognostics. Reliab. Eng. Syst. Saf., 217, 107961, 2022. doi: 10.1016/j.ress.2021.107961. [53] Raissi, M., Perdikaris, P. and Karniadakis,G ., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707, 2019. doi: 10.1016/j.jcp.2018.10.045. [54] Yazdani, A., Lu, L., Raissi, M. et al., Systems biology informed deep learning for inferring parameters and hidden dynamics. PLOS Comput. Biol., 16, 11, e1007575, 2020. doi: 10.1371/journal.pcbi.1007575. [55] Lee, K. and Parish, E.J., Parameterized neural ODEs: applications to computational physics problems. Proc. Roy. Soc. A (Math., Phys, Eng Sci.), 477, 20210162, 2021. doi: 10.1098/rspa.2021.0162. [56] Owoyele, O. and Pal, P., ChemNODE: A neural ordinary differential equations framework for efficient chemical kinetic solvers. Energy and A.I., 7, 10118, 2022. doi: 10.1016/j.egyai.2021.100118. [57] Khalid, M.M., Schenkendorf, R., Parameter identification in manufacturing systems using physics informed neural networks. In A. Wagner, K. Alexopoulos, S. Makris (eds.) . Advances in Artificial Intelligence in Manufacturing, ESAIM 2023. . pp. 51-60. Lecture Notes in Mechanical Engineering, Springer, Cham, 2023. doi: 10.1007/978-3-031-57496- 3_6. [58] Godfrey, K.R., Perturbation Signals for System Identification, Prentice Hall, Hemel Hempstead, UK, 1993. ISBN 978-0-13-656414-0. [59] van den Bos, A., Periodic test signals - properties and use, In Perturbation Signals for System Identification, K.R. Godfrey, (Ed.), Prentice Hall, Hemel Hempstead, UK, pp 161- 175, 1993. ISBN 978-0-13-656414-0. [60] Hunter, W.G., Hill, W.J. and Henson, H.L., Designing experiments for precise estimation of some of the constants of a mechanistic model. Can. J. Chem. Eng. 47, 76, 1969. [61] Federov, V.V., Theory of Optimal Experiments. Academic, New York, 1972. ISBN 978- 0122507502. [62] Schenkendorf, R., Xie, X., Rehbein, M. et al., The impact of global sensitivities and design measures in model-based optimal experimental design. Processes 6(4), 27, 2018. doi: 10.3390/pr6040027. [63] Sharma, N. and Liu, Y.A. A hybrid science-guided machine learning approach for modeling chemical processes: A review. Amer. Inst. Chem. Eng. J., 68(5), e17609, 2022. doi: 10.1002/aic.17609. [64] Abt, V., Barz, T., Cruz, N. et al., Model-based tools for optimal experiments in bioprocess engineering. Curr. Opin. Chem. Eng., 22, 244–252, 2018. doi:10.1016/j.coche.2018.11.007. [65] Krausch, N., Barz, T., Kemmer, A. et al., Monte Carlo Simulations for the Analysis of Non-linear Parameter Confidence Intervals in Optimal Experimental Design. Front. Bioeng. Biotechnol., 7, 122, 2019. doi: 10.3389/fbioe.2019.00122 [66] Nimmegeers, P., Bhonsale, S., Telen, D. et al.. Optimal experiment design under parametric uncertainty: A comparison of a sensitivities based approach versus a polynomial chaos based stochastic approach. Chem. Eng. Sci., 221, 115651, 2020. doi:10.1016/j.ces.2020.115651. [67] Toppe, A.A., Selvarajan, S., Heiduk, C. et al., PINN-based design of experiment concept for process model parameter identification, Comp. Aided Chem. Eng., 52, 837- 842, 2023. doi: 10.1016/8978-0-443-15274-0.50134-7. [68] Hess R.A., Gao C. and Wang S.H., A generalized technique for inverse simulation applied to aircraft maneuvers. AIAA J. Guid. Contr. Dyn. 14 (5): 920-926, 1991. doi: 10.2514/3.20732 [69] Thomson D.G. and Bradley R., The principles and practical application of helicopter inverse simulation. Sim. Pract. Theory. 6 (1): 47-70, 1998. doi:10.1016/S0928- 4869(97)00012-8 [70] Lu, L., Murray-Smith, D.J. and Thomson, D.G., Issues of numerical accuracy and stability in inverse simulation, Sim. Mod. Pract Theory. 16(9), 1350-1364, 2008. doi:10.1016/j.simpat.2008.07.003. [71] Buchholz, J.J., von Grünhagen, W., Inversion Impossible?. Bremen, Germany: University of Applied Sciences, Technical Report, 2004 [72] Murray-Smith, D.J., Feedback methods for inverse simulation of dynamic models, Math. Comput. Model. Dyn. Syst. 17, 515, 2011. doi: 10.1080/13873954.2011.584323. [73] Thomson, D. and Bradley, R., Inverse simulation as a tool for flight dynamics research- principles and applications. Prog. Aerosp. Sci., 42, 174, 2006. doi: 10.1016/j.paerosci.2006.07.002 [74] Murray-Smith, D. J., Inverse simulation and analysis of underwater vehicle dynamics using feedback principles. Math. Comput. Model. Dyn. Syst., 20(1), 45-65, 2014. doi: 10.1080/13873954.2013.805146. [75] Murray-Smith, D.J. and McGookin, E.W. , A case study involving continuous system methods of inverse simulation for an unmanned aerial vehicle application. Proc. Inst. Mech. Eng. Part G: J. Aerosp. Eng., 229(14), 2700-2717, 2015. doi: 10.1177/0954410015586842. [76] Murray-Smith, D.J., Development of an inverse simulation method for the analysis of train performance. Proc. Inst. Mech. Eng.. Pt. F: J. Rail Rapid Transit, 232(5), 1295-1308, 2018. doi: 10.1177/0954409717720349. [77] Zhou, W., Wang, H., Thomson, D. et al., Failure effect analysis and reconfiguration of thrusters based on inverse simulation of manually controlled RVD. Aerospace Science and Technology, 77, pp. 485-498, 2018. doi: 10.1016/j.ast.2018.03.019. [78] Shilliday, S.R.K.; McGookin, E.W. and Thomson, D.G., A comparison of forward and inverse simulation for fault detection on a rover, Presented at 8th Intl. Conf. on Control, Decision and Information Technologies (CoDIT), Istanbul, Turkey, 17-20 May 2022, Published (IEEE) 2022 8th Intl. Conf. on Control, Decision and Information Technologies (CoDIT) 1, 1432-1437, 2022. doi: 10.1109/CoDIT55151.2022.9804106. [79] Shilliday, S.R.K.; McGookin, E.W. and Thomson, D.G., Rover inverse simulation using a segmented model approach, IFAC PapersOnline, 56, 2, 6820-6825, 2023. doi: 10.1016/j.ifacol.2023.10.395. [80] Razavi, S., Jakeman, A., Saltelli, A.et al., The future of sensitivity analysis: an essential discipline for systems modeling and policy support. Environ. Model. Softw. 137, 104954, 2020. doi:10.1016/j.envsoft.2020.104954. [81] Kit, E., Software Testing in the Real World, Addison Wesley, Harlow, U.K., 1995. ISBN 0-201-87756-2. [82] Rider, W. J. The foundations of verification in modelling and simulation. In Computer Simulation Validation, Fundamental Concepts, methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 271-294, Springer Nature, Cham, Switzerland 2019. doi: 10.1007/978-3-319-70766-2_11 [83] Hayhurst, K.L., Veerhusen, D.S., Chilenski, J.L. et al., A practical tutorial on modified condition/decision coverage. NASA/TM—2001-210876, Hampton, Virginia, USA, National Aeronautics and Space Administration, Langley Research Center, 2001. [84] Kuhn, D.R., Craigen, D. and Saaltink, M., Practical application of formal methods in Modeling and Simulation. Presented at 2003 Summer Computer Simulation Conference SCSC 2003, July 20-24, Montreal, Canada, Paper S200, Society for Modeling and Simulation International, San Diego, CA, USA, 2003. [85] Trojet, W., Safety design for simulation models based on formal methods. Int. J. Comput. App., 180(8), 1-5, 2017. doi:10.5120/ijca2017916058. [86] Heitmeyer, C.L. Formal methods for specifying , validating and verifying requirements. J. Univers. Comput. Sci. 135(5), 607-618, 2007. doi:10.3217/jucs-013-05-0607. [87] Hughes, J.G. and Hase, T.P.A. Measurements and their Uncertainties: A Practical Guide to Modern Error Analysis, Oxford Univ. Press, Oxford, U.K., 2010. ISBN 978-0-19-956632-7. [88] Tischler, M.B. and Remple, R.K., Aircraft and Rotorcraft System Identification: Engineering Methods and Flight Test Examples, American Institute of Aeronautics and Astronautics, Reston, VA, USA, 2006. [89] Murray-Smith, D.J. Measures of quality for model validation. In Testing and Validation of Computer Simulation Models, Principles, Methods and Applications, D.J. Murray-Smith, Chapter 3, pp. 35-48, Springer, Cham, Switzerland, 2015. doi: 10.1007/978-3-319-15099-4_3. [90] Heylen, W. and Lammens, S., FRAC: A consistent way of comparing frequency response functions. In Proc. of Conf. on Ident. in Eng. Syst., Swansea, 1996. University of Wales, Swansea, M.J. Friswell, J.E. Mottershead (Eds.), pp 48-57, 1996. [91] Huynh, D.P.B., Knezevic, D.J. and Patera, A.T. Certified reduced basis model characterization: a frequentist uncertainty framework. Comput. Meth. Appl. Mech. Eng., 201-204, 13-24, 2012. doi: 10.1016/j.cma.2011.09.011. [92] Robinson, A.P., Testing simulation models with frequentist statistics. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 465-496, Springer Nature, Cham, Switzerland 2019. doi: 10.1007/978-3-319-70766-2_19. [93] The Mitre Corporation, Verification and validation of simulation models. In: Mitre Systems Engineering Guide, The Mitre Corp., Bedford. 2014. [94] Pace, D.K. Modeling and simulation verification and validation challenges, Johns Hopkins APL Technical Digest, 25, 163, 2004. [56] Brade, D. A Generalized Process for the Verification and Validation of Models and Simulation Results. Dissertation submitted for award of degree Dr rer. nat. Fakultāt für Informatik, Universitāt der Bundeswehr, München, Germany, 2003. [96] Murray-Smith, D.J. The use of experimental data in simulation model validation. In Computer Simulation Validation, Fundamental Concepts, methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 357-382, Springer Nature, Cham, Switzerland 2019. doi: 10.1007/978-3-319-70766-2_15. [97] Roy, C.J. Errors and uncertainties: their sources and treatment. In Computer Simulation Validation, Fundamental Concepts, methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 119-141, Springer Nature, Cham, Switzerland 2019. doi: 10.1007/978-3-319-70766-2_5. [98] Knudsen, M., Experimental modeling of dynamic systems: an educational approach, IEEE Trans. Education, 49, 29-38, 2006. doi:10.1109/TE.2005.853071 [99] Butterfield, M.H. and Thomas, P.J., Methods of quantitative validation for dynamic system models-part 1: theory, Trans. Inst. Meas. Control, 8(4), 182-200, 1986. doi: 10.1177/014233128600800403. [100] Butterfield, M.H. and Thomas, P.J., Quantitative validation of dynamic models for use in fast reactor safety assessments. In Proc. Int. Conf. on Science and Technology for Fast Nuclear Reactor Safety, Guernsey (UK), 12-16 May 1986. British Nuclear Energy Society, London, U.K., pp. 145-152, 1987. [101] Butterfield, M.H. and Thomas, P.J., Methods of quantitative validation for dynamic - part 2: applications, Trans. Inst. Meas. Control, 8(4), 201-219, 1986. doi: 10.1177/014233128600800404. [102] Butterfield, M.H., A method of quantitative validation based on model distortion, Trans. Inst. Meas. Control, 12(4), 167-173, 1990. doi: 10.1177/014233129001200402. [103]Thomas, P.J., Simulation of Industrial Processes for Control Engineers, ButterworthHeinemann, Oxford, UK, 1999. ISBN 0 7506 4161 4. [104] Cameron, R.G., Model validation by the distortion method: linear state space systems control theory and applications, IEE Proc. D, 139, 296-300, 1998. ISSN 1350-2379. [105] Pranja, S. Barrier certificates for nonlinear model validation. Automatica, 42, 1, 117- 126, 2006. doi: 10.1016/j.automatica.2006.08.007. [106] Anderson, J. and Papachristodoulu, A., On validation and invalidation of biological models. BMC Bioinformatics, 10, 132, 2009. doi:10.1186/1471-2105-10-132. [107] Murray-Smith, D.J. Face validation. In Testing and Validation of Computer Simulation Models: Principles, Methods and Applications, D.J. Murray-Smith, Section 7.3, pp 95-101, Springer, Cham, Switzerland. 2015. doi: 10.1007/978-3-319-15099-4_7. [108] Bryce, G.W., Agnew, P.W., Foord, T.R. et al., On-site investigation of electrohydraulic governors for water turbines. Proc. Inst. Elect. Eng. 124(2), 147-153, 1977. ISSN: 0020- 3270. [109] Winning D. J., Marshall A. G., Findlay D. G. E. et al., Controller testing facility on 32.5 MW water turbine, Proc. Inst. Elect. Eng., Pt. C, 127(6), 357-359, 1980. ISSN: 0020- 3270 [110] Eddy, D.M., Hollingsworth, W., Caro, J.J. et al., Model transparency and validation: A report of the ISPOR-SMDM modelling good research practices task force- 7. Med. Decis. Making, 32(5), 733-743, 2012. doi: 10.1177/0272989X12454579. [111] Marshall, D.A., Burgos-Liz, L., IJzerman, M.J. et al., Applying Dynamic Simulation Modeling Methods in Health Care Delivery Research—The SIMULATE Checklist. Report of the ISPOR Simulation Modeling Emerging Good Practices Task Force, Value in Health, 18, 5-16, 2015. doi: 10.1016/j.jval.2014.12.001 [112] Marshall, D.A. Burgos-Liz, L., IJzerman, M.J et al., Selecting a Dynamic Simulation Modeling Method for Health Care Delivery Research—Part 2: Report of the ISPOR Dynamic Simulation Modeling Emerging Good Practices Task Force, Value in Health, 18, 147 – 160, 2015. doi: 10.1016/j.jval.2015.01.006. [113] Murray-Smith, D.J., Management issues within simulation model development and testing. In D.J. Murray-Smith, Testing and Validation of Computer Simulation Models: Principles, Methods and Applications, Chapter 8, pp 113-124, 2015. doi: 10.1007/978-3- 319-15099-4_8. [114] F. Liu and M. Yang, The management of simulation validation. In Computer Simulation Validation, Fundamental Concepts, methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 587-606, Springer Nature, Cham, Switzerland, 2019. doi: 10.1007/978-3-319-70766-2_24. [115] Murray-Smith, D.J. Modelling and Simulation of Integrated Systems in Engineering: Issues of Methodology, Quality, Testing and Application, Woodhead Pub., Cambridge, U.K., 2012. ISBN 978-0-85709-078-2. [116] Murray-Smith, D.J., Issues of fitness for purpose in train simulation models: a review, Simulation Notes Europe, 32(4), 177-186, 2022. doi: 10.11128/sne.32.on.10621. [117] Hu, J. and Gu, H. Survey of flight control techniques for large-scale helicopters. Int J Aerospace Eng, 2017 (1), 1-14, 2017. doi: 10.1155/2017/5309403. [118] Hamel, P.G., Aerospace vehicle modelling requirements for high bandwidth flight control. In Aerospace Vehicle Dynamics and Control, M.V. Cook and M.J. Rycroft, (Eds.), , Clarendon, Oxford, 1-31, 1994. ISBN 0-19-853473-6. [119] Enenakpogbe, E., Whidborne, J.F and Lu, L. Control of an eVTOL using nonlinear dynamic inversion. In 13th UK Automatic Control Council (UKACC) International Conference (Control 2022), Plymouth 20-22 April 2022. doi:10.1109/Control55989.2022.9781449 [120] White, M.D., Padfield, G.D., Lu, L.. et al., Review of flight simulation fidelity requirements to help reduce ‘rotorcraft loss of control in flight’ accident rates, CEAS Aeronautical J. 12(4), 701-772, 2021. doi: 10.1007/s13272-021-00542-6. [121] Anonymous, Rotorcraft System Identification. AGARD Advisory Report 280 (AGARDAR-280), AGARD, Neuilly-sur-Seine. 1991. [122] Anonymous, Rotorcraft System Identification. AGARD Lecture Series 178 (AGARD-LS178), AGARD, Neuilly-sur-Seine. 1991. [123] Cruz, R.V. and Góes, L.C.S., Results of short-period helicopter system identification using output error and hybrid search-gradient optimization algorithm. Math. Probl. Eng., 2010, Article ID 231594, 17pp, 2010. doi: 10.1155/2010/231594. [124] Padfield, G.D., Helicopter Flight Dynamics, 2nd Edition, American Inst. of Aeronautics and Astronautics, 2007. ISBN 978-1563479205. [125] Ren, B., Ge, S.S., Chen, C. et al., Modeling, Control and Coordination of Helicopter Systems, Springer, New York, 2012. ISBN 978-1-4614-1562-6. [126] Leith, D.J. and Murray-Smith, D.J., Experience with multi-step test inputs for helicopter parameter identification. Vertica, 13(3), 403-412, 1989. [127] Bradley, R., Padfield, G.D. and Murray-Smith, D. J., Thomson, D.G., Validation of helicopter mathematical models. Trans. Inst. Meas. Control, 12, 186-196, 1990. doi: 10.1177/014233129001200405. [128] Murray-Smith, D.J., Case study: Model validation and experiment design for helicopter simulation model development and applications. In D. J. Murray-Smith, Testing and Validation of Computer Simulation Models: Principles, Methods and Applications, Chapter 10, pp 147-173, Springer, Cham, Switzerland. 2015. doi: 10.1007/978-3-319-15099-4_10. [129] Berger, T., Tobias, E.L., Tischler, M.B. et al., Advances and modern applications of frequency-domain aircraft and rotorcraft system Identification. J. Aircraft, 60, 5, 1331- 1353, 2023. doi: 10.2514/1.C0372775. [130] Kuppusamy, A. and Yoon, S.J., Experimental flight testing and system identification of Eurocopter AS365N2 Dauphin helicopter using CIFER. Int. J. Appl. Eng. Res., 12(24), 14287-14298, 2017. ISSN 0973-4562. [131] Black, C.G. and Murray-Smith, D.J., A frequency-domain system identification approach to helicopter flight mechanics model validation. Vertica, 13(3), 343-368, 1989. [132] Geluardi, S., Nieuwenhuizen, F.M., Pollini, L. et al., Frequency domain system identification of a light helicopter in hover, Annual Forum Proceedings – AHS International, 3, 1721, 2014. [133] Bradley, R., Black, C.G. and Murray-Smith, D.J., System identification strategies for models incorporating induced flow. Vertica, 13(3) 281-293, 1989. [134] Lu, L., Padfield, G.D., White, M.D. et al., Fidelity enhancement of a rotorcraft simulation model through system identification. Aeronautical J. 115(1170), 453-470, 2011. doi:10.1017/S0001924000006102 [135] Agarwal, D., Lu, L., Padfield, G.D. et al., The use of augmented rotor inflow to predict rotorcraft responses in hover and low-speed manoeuvres. Aeronautical J., 126(1301), 1168-1186, 2022. doi: 10.1017/aer.2021.123. [136] Cameron, N., White, M.D., Padfield, G.D. et al., Rotorcraft modelling renovation for improved fidelity, Presented at Vertical Flight Society, 75th Annual Forum, 13-16 May 2019, Philadelphia, PA, USA. 2019. [137] Agarwal, D, Lu, L., Padfield, G.D. et al., Rotorcraft simulation fidelity for low-speed maneuvring using ‘additive’ system identification, Presented: 45th European Rotorcraft Forum, Warsaw, Poland, 17-20 September 2019, Paper 0144, 2019. [138] Lu, L., Agarwal, D., Padfield, G.D. et al.. A new heuristic approach to rotorcraft system identification. J. Am. Helicopter Soc., 68, 2, 42-58, 2023. doi: 10.4050/JAHS.68.022005. [139] Tischler, M. and White, M. Rotorcraft Flight Simulation Model Fidelity Improvement and Assessment – Final report of NATO STO AVT-296 Research Task Group (Report) (TRAVT-296_UU), 2021. [140] White, M.D., Perfect, P., Padfield, G.D. et al., Acceptance testing and commissioning of a flight simulator for rotorcraft simulation fidelity research. Proc. Inst. Mech. Eng., Part G, J. Aerosp. Eng., 227(4), 663-686, 2012. doi: 10.1177/0954410012439816. [141] van’t Hoff, S., Lu, L., Padfield, G.D. et al. Preliminary guidelines for a requirements-based approach to certification by simulation for rotorcraft. In 48th European Rotorcraft Forum (ERF48 2022), Winterthur, 6-8 September 2022. Paper No.58, 17 pp.. https://www.rotorcraft-forum.eu/ [142] Thomson, D.G., Taylor, C.D., Talbot, N. et al., An investigation of piloting strategies for engine failures during takeoff from offshore platforms. Aeronautical Journal, 99(981), 15-25, 1995. doi: 10.1017/S0001924000096640. [143] Kaletka, J., Instrumentation and data processing. In Rotorcraft System Identification. AGARD Lecture Series 178 (AGARD-LS-178), pp. 3.1 – 3.18, AGARD, Neuilly-sur-Seine. 1991. [144] Kaletka, J., BO 105 identification results. In Rotorcraft System Identification. AGARD Lecture Series 178 (AGARD-LS-178), pp. 9.1 – 9.50, AGARD, Neuilly-sur-Seine. 1991. [145] Breeman, J.H., Flight test data quality evaluation. In Rotorcraft System Identification. AGARD Lecture Series 178 (AGARD-LS-178), pp. 4.1 – 4.8, AGARD, Neuilly�sur-Seine. 1991. [146] Murray-Smith, D.J., Design of experiments. In Rotorcraft System Identification. AGARD Lecture Series 178 (AGARD-LS-178), pp. 2.1 – 2.8, AGARD, Neuilly-sur-Seine. 1991. [147] Abbeel, P., Ganapathi, V. and Ng, A.Y., Learning vehicular dynamics with application to modeling helicopters, In NIPS'05: Proceedings of the 18th International Conference on Neural Information Processing Systems, Y. Weiss, B. Schölkopf and J. C. Platt, (Eds.) pp. 1- 8, MIT Press, Cambridge, MA, USA, 2005. [148] Coates, A.,, Abbeel, P. and Ng, A.Y., Autonomous helicopter flight using reinforcement learning. In Encyclopedia of Machine Learning and Data Mining, C. Sammut and G.I. Webb, (Eds.), 11 pp., Springer, Boston MA, U.S.A., 2017. doi: 10.1007/978-1-4899-7687-1_16 [149] Terblanche, M., Uren, K.R. and van Schoor, G, Flight controller and low-cost test environment for a simulated helicopter, IFAC Proceedings Volumes, 47(3), 7266-7271, 2014. doi: 10.3182/20140824-6-ZA-1003.02194. [150] Punjani, A. , Machine Learning for Helicopter Dynamics Models, U.C. Berkeley, Master of Science Dissertation in Computer Science, 2014. Technical Report No. UCB/EECS�2014-219. http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-219.html [151] Helder, B., Reinforcement Learning for Helicopter Flight Control, Master of Science Thesis, Delft University of Technology 2020. http://resolver.tudelft.nl/uuid:7e99f2d4-89b3-43c4-9b7d-8b04b3c2dc84 [152] Grauer, J.A., Conroy J.K., Hubbard, J.E. et al., System identification of a miniature helicopter, J. Aircraft, 46(4), 1260, 2008. doi: 10.2514/1.40561. [153] Khizer, A.N., Hussain, I. and Kumar, W., Time domain system identification of longitudinal dynamics of single rotor model helicopter using SIDPAC, Mehran Univ. Res. J. Eng. Technol. 34(1), 75-81, 2015. ISSN: 0254-7821 [154] Walker, J. and Tischler, M.B., Simulation model update and optimized control design of a subscale flybarless helicopter. Presented: 78th Annual Forum & Technology Display, Vertical Flight Society, Ft. Worth, Texas, May 10-12 2022, Vertical Flight Society, 2022. [155] Siebert, J., Staudinger, M., van Meerveld, H.J. Validation and over-parameterization – experiences from hydrological modeling. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 811-834, Springer Nature, Cham, Switzerland 2019. https://doi.org/10.1007/978-3-319-70766-2\_33\. [156] Beven, K., Lane, S. Invalidation of models and fitness-for-purpose: a rejectionist approach. In Computer Simulation Validation, Fundamental Concepts, methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 145-172, Springer Nature, Cham, Switzerland 2019. doi: 10.1007/978-3-319-70766-2_6 [157] Min, M., Challenges to simulation validation in the social sciences. A critical rationalist perspective. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 857-880, Springer Nature, Cham, Switzerland, 2019. doi: 10.1007/978-3-319-70766-2_35. [158] Fagiolo, G., Guerini, M., Lamperti, F., Moneta, A., Roventini, A., Validation of agent-based models in economics and finance. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 763-790, Springer Nature, Cham, Switzerland, 2019. doi: 10.1007/978-3-319-70766-2_31. [159] Rood, R.B, Validation of climate models – an essential practice. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 737-762, Springer Nature, Cham, Switzerland, 2019. doi: 10.1007/978-3-319-70766-2_30. [160] Calder, A.C., Townsley, Astrophysical validation. In Computer Simulation Validation, Fundamental Concepts, Methodological Frameworks and Philosophical Perspectives. C. Beisbart, N.J. Saam (Eds.) pp. 685-710, Springer Nature, Cham, Switzerland, 2019. doi: 10.1007/978-3-319-70766-2_28. [161] Bradley, W.; Kim, J.; Kilwein, Z.; Blakely, L.; Eydenberg, M.; Jalvin, J.; Laird, C.; Boukouvala, F. Perspectives on the integration between first-principles and data-driven modeling. Comput. Chem. Eng., 166, 107898, 2022. doi: 10.1016/j.compchemeng. 2022.107898.