This has been a fun blog series, and one that I can gleefully say we actually followed through and finished. Let’s recap:

…and the last step to get to BI4 is (in my best David Letterman voice)…

#1 Test, test, and more test.

Don’t tell the grammar police.

Anyone that tells you that you can be running on BI4 in a week is nuts. I think we’ve given you nine reasons before this post to help demonstrate you can and should migrate wisely. With all of those things out of the way, I’m going to impart why I think testing is my #1.

Of all of our Top 10 list points, we want you to have the most stable SAP BusinessObjects landscape possible. But without adequate (or more than adequate) testing, you cannot demonstrably assure stability and accuracy of BI in your new landscape. I’d like to walk through some of the testing scenarios. Before I do, realize this is NOT a check list. This is a list to vet your own landscape against. You need to run the tests that make sense in your landscape.

System Testing

SAP BusinessObjects BI4 is installed. Now what? As the person that just dropped the installer, clicked setup, and configured the landscape, you should really be able to demonstrate you are sure it is working correctly. So what does System Testing involve? Well, it depends. But, the things I tend to think of.

Are all of my configured data sources for my BI working correctly?
Are all of my SAP BusinessObjects required sources like Auditor, Monitoring, etc. working correctly?
Are all of my required authentication methods working correctly, like SAP Authentication, Windows AD, LDAP, etc.?
In a cluster, are all shared file systems interacting correctly from each node in the cluster?
In a cluster, are all of my web tiers able to speak to each cluster node correctly?
In a cluster, does this box fail over correctly?

The system itself varies by implementation. You must define a system test plan based upon the architectural components. But, it’s important to define the tests, document them, and run through them in each landscape built.

Regression Testing

Well gee, Eric, what should I test here? I’m going with the easy answer that I like to help customers answer this question with Sherlock®. It’s concrete. But in your own landscape, if you don’t have it, you need to figure out a representative sample of reports that you can compare side by side in your existing landscape compared directly to your BI4 landscape. This should include reports, dashboards, Information Spaces, etc. You must compare cell by cell, row by row, and validate that formulas match. What happens when they don’t? I wrote about it recently (See related article, Using EV Technologies Sherlock® to Identify Calculation Changes During Upgrade). Either way, you must determine the root cause of the change, the extent of the change, and how you will address it during your migration.

You may be thinking, “Eric, that sounds expensive.” This is intern fodder, plain and simple. We’ve got them. So should you. Go with it.

User Acceptance Testing

Somewhere you have users you can actually trust. No, really. You have to. They understand the tools enough, love it or hate it, to validate them. After they are trained on BI4 (see related article, Training Your Users for SAP BusinessObjects BI4), we really need to lean on them to open their reports in BI4, whether in public folders or favorites, refresh them, use them, click around, and ensure they are willing to sign off for the masses. Make them sign in blood (or other applicable form of ink) that they are good to go to make BI4 go for launch. We want users to be happy. Shake your head yes with me now.

Load Testing

You’ve designed and implemented a system, but how do you know it will support all N-thousand users you are about to bring to it? Load testing. I’m hoping for your sake that your organization already has a load testing suite. Any organization that does ideally has experts that know how to write scripts to invoke those load tests. Thinking through some testing scenarios:

How many concurrent Web Intelligence users refreshing several different reports of varying sizes and complexities can your system ramp up to?
How many schedules from Web Intelligence and Crystal Reports can your system sustain before it starts to go sideways?
How many dashboards can be open concurrently and refreshing while those Web Intelligence users are wrecking your system?

I’d stress these are the stress tests in which you do want to break your system. The point to take home is to understand what that breaking point is. If too low, you need to revisit your requirements and design. If really high, go celebrate that you did it right….or recheck your tests because you missed something.

Is this out of your ballpark? Fret not! Contact us. We can help. Greg Myers is a super nerd about this stuff too and gave a great webinar on capacity planning to avoid epic failure, so we’ve got you covered.

Other Testing That Could Matter

Look around your team and ask yourselves, “Has anybody built any SDK apps?” Oops. Those need to be tweaked and retested. I don’t think there is any scenario in which SDK apps aren’t going to require modification. Changing code = testing code. Make sure the app dev teams doing that work are doing their due diligence and you don’t get caught holding the bag if they didn’t. The buck stops with you.

Conclusion

I don’t think there is any way to wrap up this post and this series than to say, “Remember…”

Footnote: Thanks to D for the meme that will last forever.

10 Steps to SAP BusinessObjects BI4 – Testing