Simulation without validation is
little more than art.
Image courtesy of SOLIDWORKS.
Without validation, the colorful plots of your simulation will be little more than art. Even if your model manages to converge, without validation there is no way to determine it has merged to a correct value. Setting up simulation models is getting easier, but make a bad assumption and you will find it is also easy to get bad results.
This is why validation is so important. It gives you a sanity check and could expose an inaccurate simulation result. Here are a few methods engineers can use to ensure their simulations obey the laws of physics.
Build a Lab and Track a Field Variable
Lab testing results and simulation
results compared for a hyperelastic material.
Image courtesy of Noumenon Multiphysics.
“We are always looking for new ways to calibrate our results,” notes Stuart Brown, managing partner of consulting firm Veryst Engineering.
For example, he said, “Veryst can validate the final simulation by testing the physics with a field variable or scalar quantity (like strain) we can track through experimentation. These methods can convince us and our clients that the answers are correct.”
As a result, Brown explained, Veryst and many other simulation consultants have started to build in-house laboratory capabilities. These labs have the added benefit that they can also be used to collect data, like material properties, just as easily as they can be used to validate simulations.
However, lab testing can get a little messy and complicated depending on what is being tested. This is particularly true for biological simulations. After all, you can’t just start testing on humans.
Kerim Genc, technical sales manager at Simpleware, suggested that to validate interactions between mechanical designs and biological models, such as with medical devices and safety equipment, you might need to call the butcher before you call Fisher Scientific. He said, “We were working with a large medical device company, and to validate their simulations they studied a pig. The pig they scanned for their model in the software was the same pig they performed their experiments on in real life.”
In this case, using pigs was actually kosher—with respect to validation. Since pigs are anatomically similar to people, validating the simulation with swine was a good way to ensure the mechanical design can save a human’s life.
However, validating the whole system isn’t always an easy, cheap or fast approach. Take Fatima Alleyne, research general engineer at the U.S. Department of Agriculture. She works on simulating agricultural dryers in rural and third world settings.
She said, “We take our design, embed it into a simulation, gather the data and test it in the field. Once we conduct a full testing, which can take days, we compare the results to the simulation. Based on that, we then determine if the simulations are feasible.”
Additionally, waiting to test your simulation on the final design isn’t a great idea. At this point in the design cycle it will be hard and expensive to make changes. As a result, verifying the material properties for a part of your design near the end of the development cycle is asking for trouble.
Simplify Your Model and Experiments
Image courtesy of SOLIDWORKS.
To avoid waiting too long to start validating your simulation, you might want to start with smaller experiments and simpler simulation models. It is also useful to use simpler models to represent subcomponents when modeling large systems and assemblies. These simpler subassemblies can then be independently validated. For example, engineers might model a design with beam or plate elements first before moving to 3D elements.
“Ultimately, validation of simulation is done with empirical testing; however, in cases where empirical testing is cost prohibitive or impossible, our approach is to conduct relevant material testing and verify the constitutive model(s) against the material testing data,” said Oren Lever, principal engineer for Gas Technology Institute.
Lever explained that a certain amount of trust can be put into the finite method software because it is tested every day by engineers who obtain accurate results when they input the correct models, input values and input geometry mesh resolutions. He suggests it is often enough to simulate simpler models and verify the results to available data like material testing. Once the building blocks of the simulation are all validated, the idea of the whole will be valid, as well.
Kyle Koppenhoefer, principal at consulting firm AltaSim Technologies, LLC, agreed. He said, “Validation can be a very challenging thing. For one thing, we know the software companies do a great deal of validation, so we are confident with what they are providing us. So we need to validate and verify the techniques we use the software for. So we will often look for an analytic solution or experimental data to compare against when dealing with a specific class of problems.”
Look for Data and Other Computational Methods
Image courtesy of SOLIDWORKS.Though physical testing of your simulation is typically the most concrete method to validate your simulation, there are other tricks up an engineer’s sleeve. For instance, you can try to use another method to solve the problem and compare your results.
Udayan Kanade, CEO of Oneirix Labs, says that though his consulting firm has its own lab, there are still other ways to tackle the validation problem.
Kanade said, “Finite element analysis is one way to do it, but spectral methods and other computations, like series summation, can be used to do the same thing. Often, many of these methods are applicable and separate enough so that if the values match you can better believe what you did was right.”
Kanade has even more alternative solutions to using an in-house lab, such as:
• Comparing analytical solutions or simplified analytical models with FEA models
• Use another solver or use an in-house solver (if you have one) and compare results
• Compare the simulation with reference data from customers, literature or third parties
Sometimes, You Have to Go with Your Gut.
Image courtesy of SOLIDWORKS.It’s not a good idea to blindly go with your gut. However, Jeffrey Crompton, principal at AltaSim Technologies, explains that there is a point where if you have enough experience, information and understanding of the system, then your gut knows best.
“It cascades down,” said Crompton. “In some cases you have an exact solution for the problem … [or] you might find an analytical solution for specific problems. But after that you are looking at experimental data. Finally, it just comes down to gut feel. The results of the model can be accurate in so far as how the model is set up and you just have to say, ‘That feels right. I understand what is going on.’”
As previously discussed, if you have been using some sort of validation in your model throughout its creation, then at a point this validation will become natural to the engineer who will start to inherently understand the system. At that point, you just know.
But what about situations where it is impossible to do any sort of validation? This can become tricky. At this point, Crompton says, “Statistics and back of the napkin calculations can be helpful in cases where it’s more of a gut feel to validate your simulation. In these cases, you use whatever you can. You might be more reliant on extrapolations and statistical approaches, but you still need to have some level of confidence moving forward or there is no point going forward with the simulation at all.”
What methods do you use to validate your simulations? Is there anything we missed? Comment below.
About the Author
Shawn Wasserman (@ShawnWasserman) is the Internet of Things (IoT) and Simulation Editor at ENGINEERING.com. He is passionate about ensuring engineers make the right decisions when using computer-aided engineering (CAE) software and IoT development tools. Shawn has a Masters in Bio-Engineering from the University of Guelph and a BASc in Chemical Engineering from the University of Waterloo.