Validating EM Data-REQUIRING QUANTITATIVE ACCURACY STATEMENTS IN EM DATA.pdf (original) (raw)

Model Accuracy/Uncertainty-A METHOD TO ESTIMATE THE ACCURACY OF NUMERICAL MODELS

Determinin the accuracy of the results produced b a numerical model remains an on oing a n j importan! problem in computational eLtromagnetics (CEM) to whjch pegaps. the most desirable and convincin approach is to perform a quantitative com mson between experimental data an$ the model output. Such a comparison has &e disadvantage that it embodies two different kinds of errors these bein the physical modelin error (PME) and the numerical modeling error (N h) Althou the P d c a n normally expected to be the larger of the, two and also the more &ficult to evaluate, uantitative knowledge of the NME itself is essential, estimation of which is the yopic of this paper. Two ap roaches are available to estimate the NME, one being the intemal checks that can ! e done within a s E i f i c model or code e.g convergence tests, boundary conditions, ower conservation, etc.), and the otLer 'the extemal checks that involve comparing &e results of two or more different models. The a proach lscussed here is another kind of intemal check but different from the otgers in the sense that it seeks a "noise" level in the mode(output as a measure of the model accuracy. KINDS OF DATA VALIDATION There are hvo fundamental ways to estimate the accuracy of electromagnetic observables. One is the compare the results in uestion with independent data, the expectation k i n that the true values are probabyy as close to these hvo sets of data as they are to ea& other, something that can be described as an external validation or check. This is a more convincing test when the independent data are experimental and come from the actual problem of interest, as the close agreement of two numerical results doesn't necessarily prove that they are correct and also doesn't shed any light on the PME. The other is to perform some test on the numerical results themselves, either to evaluate the conformity of the results relative to Maxwell's equations or their numerical consistency as an intemal validation or check of the NME. In the context of CEM, an iGtemal check that derives from.Maxwell's equations is most satisfactory, but perha s the most commonly used internal numencal check is a convergence test. Unfortunately, the convergence of a numerical model is not always assured to occur, nor is convergence to a correct answer guaranteed either and it can be very expensive and even impractical for large problems. A different kind of internal check using Maxwell's equations is discussed here. It is pased on the obseryation that the vanous field, quantities derived from a defining microscopic descn tion, i.e., Mqwell's equabons, must subsequently e h b i t a known, reduced-orier, macrosco I C behavior. The specific behavior of interest for our purposes is the resonancesike structure of an electromagnetic frequency response, which is yell-described by a rational function [I]. The basic idea is, to appl a low-order rational funchon or fitting model (FM) to a sam led generatin m d l (GM, based on a first-prindples formulation) to b;: tested. %ere these Ghf samples known exactly, then some minimum number would in principle nnit the conhnuous transfer function T to be exactly reconstructed across the Eequenc range that the span if the (F8 exactly re resents tfie GM behavior. In facc however, the J M samples and the FM are onb approximahons, and were no more samples avalable the FM-based T F could possess no better accuracy than these samples. However, by using additional GM samples, the additional information

CEM Validation-VERIFICATION AND VALIDATION OF COMPUTATIONAL ELECTROMAGNETICS SOFTWARE

One of the most time-consuming tasks associated with developing and using computer models in electromagnetics is that of verifying software performance and validating the model results. Even now, relatively few available modeling packages offer the user substantial on-line assistance concerning verification and validation. This paper discusses the kinds of errors that most commonly occur in modeling, the need for quantitative error measures, and various validation tests such as convergence behavior and bounary-condition checks. Use of model-based parameter estimation to develop error estimates or to control uncertainty in an observable is demonstrate several examples. The article concludes by recommending that the Computational Electro-Magnetics community adopt a policy of requiring some minimal standards concerning the accuracy of numerical results accepted for journal articles and meeting presentations.

MBPE-USING MODEL-BASED PARAMETER ESTIMATION TO ESTIMATE THE ACCURACY OF NUMERICAL MODELS

Determining the accuracy of the results produced by a numerical model remains an ongoing and important problem in computational electromagnetics, one to which there are relatively few options. Perhaps the most desirable and convincing approach is to perform a quantitative comparison between experimental data and the model output. Such a comparison, though undeniably useful, has the disadvantage that it embodies two different kinds of errors, these being the physical modeling error (PME) and the numerical modeling error (NME). The former arises from any differences that exist between the actual physical problem and its numerical representation, and the latter occurs because the numerical results actually obtained represent only an approximate solution to that numerical representation. Although the PME can normally be expected to be the larger of the two and also the more difficult to evaluate, quantitative knowledge of the NME itself is essential, estimation of which is the topic of this paper. There are two approaches now used to estimate the NME, one being the internal checks that can be done within a specific model or code (e.g., convergence tests, boundary conditions, power conservation, etc.), and the other the external checks that involve comparing the results of two or more different models. The approach discussed here is another kind of internal check, but different from the others in the sense that it seeks a " noise " level in the model output as a measure of the model accuracy. The basic idea is to apply a low-order, physically motivated, fitting model (FM) to a sampled, generating model (GM, based on a first-principles formulation) to be tested. Were these GM samples known exactly , then some minimum number would in principle permit the continuous transfer function (TF) to be exactly reconstructed across the frequency range that they span if the FM exactly represents the GM behavior. In fact, however, the GM samples and the FM are only approximations, and were no more samples available the FM-based TF could possess no better accuracy either. However, by using more samples from the GM, the additional information they provide can be used to estimate the accuracy of the TF derived from the FM and of the GM samples themselves. Furthermore, these extra GM samples might also be used to improve the accuracy of the FM estimate. INTRODUCTION However obtained, determining the quantitative accuracy of the numerical results developed for a particular electromagnetic problem is essential if such results are to be reliable. This is a problem of data validation. Another related, but broader, problem is that of establishing the accuracy that may be expected to be obtained from a given computer model when it is employed following accepted guidelines for its use. This is a problem of code validation. The latter problem is intrinsically open-ended and one in which one or just a few code developers might be involved while the former is more specific and can involve relatively many more users of that code. Attention in the following is addressed to the problem of data validation .

CEM SOFTWARE: CHARACTERIZATION, COMPARISON, AND VALIDATION OF ELECTROMAGNETIC MODELING SOFTWARE.pdf

The continuously increasing number of electromagnetic computer models (codes) and applications thereof is one result of a rapidly expanding computing resource base of exponentially growing capability. While the growing use of computers in electromagnetics attests to the value of computer modeling for solving problems of practical interest, the proliferation of codes and results being produced increases the need for their validation with respect to both electromagnetic formulation and software implementation. But validation is perhaps the most difficult step in code development, especially for those models intended for general-purpose application where they may be used in unpredictable or inappropriate ways. A procedure or protocol for validating codes both internally, where necessary but not always sufficient checks of a valid computation can be made, and externally, where independent results are used for this purpose, is needed. Ways of comparing differing computer models with respect to their efficiency and utility to make more relevant intercode comparisons and thereby provide a basis for code selection by users having particular problems to model are also needed. These issues are discussed in this article and some proposals are presented for characterizing, comparing, and validating EM modeling codes in ways most relevant to the end user.

Validation, verification and calibration in applied computational electromagnetics

2010

Model validation, data verification, and code calibration (VV&C) in applied computational electromagnetics is discussed. The step by step VV&C procedure is given systematically through canonical scenarios and examples. Propagation over flat-Earth with linearly decreasing vertical refractivity profile, having an analytical exact solution, is taken into account as the real-life problem. The parabolic wave equation (PWE) is considered as the mathematical model. MatLab-based numerical simulators for both the split step Fourier and finite element implementations of the PWE are developed. The simulators are calibrated against analytical exact and high frequency asymptotic solutions. Problems related to the generation of reference data during accurate numerical computations are presented.

Assessing the numerical accuracy of the impedance method

Bioelectromagnetics, 2007

The impedance method has been used extensively to calculate induced electric fields and currents in tissue as a result of applied electromagnetic fields. However, there has previously been no known method for an a priori assessment of the numerical accuracy of the results found by this method. Here, we present a method which permits an a priori assessment of the numerical accuracy of the impedance method applied to physiologically meaningful problems in bioengineering. The assessment method relies on estimating the condition number associated with the impedance matrix for problems with varying shapes, sizes, conductivities, anisotropies, and implementation strategies. Equations have been provided which predict the number of significant figures lost due to poor matrix conditioning as a function of these variables. The results show that, for problems of moderate size and uncomplicated geometry, applied fields should be measured or calculated accurately to at least five or six significant figures. As resolutions are increased and material properties are more widely divergent even more significant figures are needed. The equations provided here should ensure that solutions found from the impedance method are calculated accurately.

Guideline for Numerical Electromagnetic Analysis Method

Ownership of a CIGRE publication, whether in paper form or on electronic support only infers right of use for personal purposes. Are prohibited, except if explicitly agreed by CIGRE, total or partial reproduction of the publication for use other than personal and transfer to a third party; hence circulation on any intranet or other company network is forbidden". Disclaimer notice "CIGRE gives no warranty or assurance about the contents of this publication, nor does it accept any responsibility, as to the accuracy or exhaustiveness of the information. All implied warranties and conditions are excluded to the maximum extent permitted by law".