The number of test cases in their regression suite.
Last year it was 200. This year it’s 400. Next year someone will propose getting it to 800.
The suite is growing. The confidence is growing with it.
And yet — production incidents keep happening.
Not despite the growing test suite. Sometimes because of it.
The Regression Trap Nobody Talks About
Here’s what actually happens inside most D365 regression suites over time:
Test cases get added after every incident. “That broke last Wave — add a test for it.”
Test cases get added after every go-live. “UAT found this — make sure it’s in regression.”
Test cases get added when new features go live. “Microsoft added this — we should test it.”
Nobody removes tests. Nobody questions whether tests are still relevant. Nobody asks which tests are actually catching defects — and which are just running.
The result is a regression suite that is wide but not deep. Large but not intelligent. Growing but not learning.
What “Smart” Actually Means in D365 Testing
A smart regression suite doesn’t mean AI. It doesn’t mean more automation. It doesn’t mean a bigger library.
It means three things — and most teams are missing all three:
1. Risk-based prioritisation Not every test case deserves equal attention before every Wave release.
A smart suite knows that your custom GL mapping carries more risk than a standard vendor lookup screen. It runs the high-risk tests first, deepest, every single time — and treats low-risk standard flows differently.
Most suites treat every test case the same. That’s not testing intelligently. That’s testing exhaustively — and exhaustion is expensive.
2. Customisation awareness A standard D365 regression suite tests Microsoft’s standard workflows.
Your environment is not standard.
Custom approval hierarchies. Modified financial dimensions. Bespoke integrations. Logic built specifically for your business operations.
A smart suite maps every customisation to the test cases that cover it — and flags immediately when a Wave update touches the underlying architecture those customisations sit on.
Most suites don’t know your customisations exist. They test around them. Not through them.
3. Outcome tracking Do you know which test cases in your suite have caught a real defect in the last 12 months?
Which ones have never failed — not because the feature is stable, but because the test isn’t deep enough to find what’s wrong?
A growing suite adds tests. A smart suite asks: is this test actually protecting us?
The Number That Matters More Than Test Case Count
It isn’t how many tests you have.
It’s defect escape rate — the percentage of defects that make it past your regression suite and into production.
A suite of 200 well-designed, risk-prioritised, customisation-aware test cases will outperform a suite of 2,000 generic test cases on this metric every single time.
Because production doesn’t care how many tests you ran. Production only cares what you missed.
What 20 Years of Enterprise QA Actually Teaches You
When you’ve been testing ERPs — across hundreds of implementations, one pattern becomes impossible to ignore:
The teams with the fewest production incidents don’t have the largest test suites.
They have the most deliberate ones.
They know which modules carry the most risk in their specific environment. They know which customizations are fragile. They know which Wave releases historically touch their most sensitive workflows.
That knowledge doesn’t live in a test case. It lives in the people running the suite — and the framework they’ve built around it.
500+ pre-built D365 test assets is not a number we lead with because it sounds impressive.
We lead with it because every single one of those assets was built from a real D365 environment, a real defect pattern, a real production risk — refined across 20 years of implementations.
That’s the difference between a library and a regression suite that learns.
Three Questions to Ask About Your D365 Regression Suite Today
① When did you last remove a test case? If the answer is never — your suite is an archive, not a safety net.
② Which of your test cases cover your customisations specifically? Not standard workflows. Your modifications. Your integrations. Your business logic.
③ What is your defect escape rate from the last three Wave releases? If you don’t know — you don’t know whether your regression suite is working.
These aren’t trick questions. They’re the questions a QA partner should be able to answer about your environment within 48 hours of assessment.
What a Smarter D365 Regression Approach Looks Like
At Crestech, our regression framework for D365 F&O is built on one principle:
Every test case must earn its place.
It covers a known risk — or it doesn’t run. It maps to a customisation or critical workflow — or it gets reviewed. It has caught or could catch a real defect — or it gets redesigned.
Our functional QA specialists don’t just execute test cases. They build the intelligence layer that tells you which ones matter — and when.
Combined with 500+ pre-built assets covering Finance, Billing, Supply Chain and core D365 workflows — your team gets both speed and depth.
Not just a bigger suite. A smarter one.
Find Out If Your D365 Regression Suite Is Working For You
We’re offering a complimentary D365 Regression Health Check for a limited number of organisations this quarter.
In 48 hours you’ll know:
✅ Whether your suite covers your highest-risk customisations
✅ Your estimated defect escape rate based on suite coverage
✅ The 3 highest-risk gaps ahead of your next Wave release
No obligation. No sales pitch. Just an honest assessment.
👉 [Book Your Free Regression Health Check]
Found this useful? Share it with your D365 implementation partner, QA lead, or anyone heading into a Wave 2 regression cycle. This is the conversation the industry needs to be having.
#Dynamics365 #D365FO #MicrosoftDynamics #ERPTesting #QualityAssurance #RegressionTesting #D365Community #WaveRelease #TestAutomation #SoftwareTesting #CIO #CFO #DigitalTransformation #ERPGoLive #D365Testing
