There has been much discussion in the twitterverse, in blogs, on podcasts, and elsewhere I’m sure, about the place of SAP BW. Those discussions have really morphed over the last few months from the flurry of tweets over the perceived death of SAP BW at the hands of its little sister SAP HANA (now hopefully put to rest…or is it???), to the value for non-SAP ERP customers. In one of this year’s episodes of the Diversified Semantic Layer, one question we tried to tackle that and I’m just not sure we got to was: what is the value of SAP BW to SAP BusinessObjects customers? While we had a really great discussion, I think there is still more left there to uncover.
One of the things Steve Lucas drew us to, and later Ethan Jewett brought us back to, was the rift between BW-types and BOBJ-types in terms of utilizing their respective tools as a semantic layer. I think to tackle that rift, you have to take one step farther back. This is going to be an interesting topic I hope. Ethan and I have shot a few emails back and forth since that podcast and are going to each broach this subject from our own points of view in separate, yet connected blog posts. If you haven’t noticed, I am a BOBJ-type only because my number of years as a BOBJ-type dwarf those working with BW. I hold nothing against BW-types. We were just brought up a little differently if we didn’t already have BW in our organization. Will we bring it back to the podcast? Perhaps. But for now, read on.
The value proposition that has come out more recently is that SAP BW makes for a great, out of the box data warehouse. If you are an SAP BusinessObjects customer today not using SAP BW, you have either a) developed a ground up warehouse on something like Oracle, Teradata, etc., or b) have purchased some other 3rd party warehouse solution specific to your vertical. Neither of those solutions are wrong. But champions of the SAP BW platform have suggested that SAP BW is a candidate for this type of solution as well. Remember, I’m still back at the data architecture side of my random train of thought.
Knowing that, I don’t know many SAP BusinessObjects developers that also wear the hat of being a data architect. Yes yes I know you are out there. There are many of you. But I think the majority of organizations follow team models of data architecture, ETL, and business intelligence. Teams may flex and bend around those three key personality types, but it is just what you see predominantly. That said, knowing that when we talk about why SAP BusinessObjects customers should care about SAP BW, shops are going to fall into a few camps:
- I already have a warehouse, but hey, thanks.
- Warehouse? What warehouse? SAP BusinessObjects is used to mash up data from all over an enterprise.
- I’ve had SAP BusinessObjects and SAP BW singing along in harmony for years. It’s rough, but it works. I hope BI4 will make it better some day.
- I’ve had SAP BusinessObjects and SAP BW for years but never mashed them up. I <3 BEx 4 eva.
- What is SAP BusinessObjects?
I’ll be the first to admit that I’m not the sharpest knife in the drawer when it comes to SAP BW with only a few years under my belt. But I’ve actually worked with teams that run the gambit there on numbers 1 through 5. I’ve been on the “Oracle or nothing” project. I’ve also been a part of a project with a Fortune 50 company that literally had both SAP BW and SAP BusinessObjects for years and had never put the peanut butter and jelly together. But I do dig the effort of the two teams (BW-types and BOBJ-types) trying to figure it out and work together.
So with the stage set, why does the rift exist still? BOBJ-types are used to ultimate control. You have a table or view you wanted added? Suuuure. I’ll just put that in the universe and export, then you can drag/drop/slice/dice and have a sexy report. But that control has become a little disrupted.
I’ve been on a few projects now, integrating with SAP BW, where the Basis team, or some other BW dev team controls their semantic layer. What? They create their own views of their data and pass it to BOBJ. I can’t just scrape up a column from the underlying cube any time I want in these efforts. I have to explicitly work with that team to expose the data I want through cubes made available for reporting. Is that much different from abstracting a database table in a view? Well, yes, but in concept, no. A DBA has to update a view and expose a piece of data for me to build my universe against it. But I think the big difference is that SAP BW-types are all-too-familiar with exposing that data through a BEx query as that semantic layer for consumption by users, and part of the value of the universe is lost.
I have read thoughts of a BW-type that the new semantic layer in SAP BusinessObjects has no value to SAP BW customers. I’ve also met customers that have both SAP BW and Oracle or other data sources and welcome it with open arms in the right use cases and as a part of a broader semantic layer strategy.
So where do you fall? What do you do differently if you are being thrust suddenly into a world in which you have more tools to choose from. First, there is some essential reading. Ingo Hilgefort wrote a great presentation on selecting the right tool for the job and continues to give this talk around the world. You also have to put careful thought into where to go direct to BW or where universes in SAP BusinessObjects meet the needs of your business.
The fact that SAP BOBJ-types and SAP BW-types are each passionate about their own semantic layers is not going to go away. But we can certainly learn to work better together as these projects arise to understand the core requirement, the capability that each tool provides the user, and ensure that IT aligns to fulfill the requirement within that realm of capability of the selected tool.
In more recent news, Josh Fletcher and Ethan got together to start a new series over on DSLayer.net to get down and dirty with SAP BW for SAP BusinessObjects customers.
Over to you, Ethan.
I just love keeping this conversation alive:
To me this goes way beyond the presentation tools or semantic layers. There are some real choices to be made at the foundation in terms of data management and storage. Having worked on EIM projects using NW BW and the traditional (Direct to Database) EIM via an ETL tool using the Kimball method, I feel like I’m in a good position to evaluate both. Both tools are capable but I can attest that the BW based solutions only work well when SAP ECC or other SAP Applications are the primary source. There are several built in transports and extractors that make delta loads easy (For example) among other advantages. However, even though it works better with SAP Apps, it is still a slow and bloated technology (Assuming SAP HANA is not the underlying RDMS). I have also heard the argument that using BW is easier because most of the EIM content is pre-delivered. This might be the case for some implementing BW but almost every BW project I have been involved with has required a mountain of customization to support the requirements of the business users. The customizations can range from re-modeling the DSO to Re-designing the InfoCubes. In some cases, ABAP code was required to radically transform the data. In my opinion, this is due to the nature of Business Intelligence and data de-normalization. Very little is cookie cutter from one company to the next. So to say using a technology other than BW is “re-inventing the wheal” is a very naive statement. To me either option has its technical coding challenges.
On the other hand, if an organization has little or no source SAP Applications it would be crazy to try and implement BW as the primary EDW. (Even if you use DataServices to load the data into BW). Most of the features of BW were designed to work with SAP Application data. If you are loading data from non-SAP systems you will find that a lot of ABAP coding will be required to properly manage that data in BW. I think sometimes people (you know who you are) look at an organization and just assume that they are using SAP ECC or other applications to run their business. This is a very narrow view of the world. Most organizations (Large or Small) have multiple platforms running their business. In my opinion, based on real world experience, the best EIM option for 3rd party data (data not native to a SAP Application) is to use DataServices (or other ETL tools) to load your RDMS of choice. I think that SAP HANA is the best RDMS platform available as a target for this style of EDW solution today. However Sybase, MS SQL, Oracle, DB2 and many others are usable solutions as well. DataServices has all the transforms, Data Quality and other options to manage data. It is not difficult, as some want you to think, to manage data using this tool or method. In the case of predominantly 3rd party data, it is several times easier and faster to implement compared to using BW.
For those organization in the middle, where 50\%(+-) of their sources are SAP and the other 50\%(+-) is something else, they have some choices to make. They can go completely BW or the traditional EIM route. The choice needs to be weighed carefully based on many factors and the requirements of the business users. If BI is truly a business user centric processes, then their opinion should really matter in terms of the solution that is chosen. Either way it won’t be easily or quickly implemented. You will likely need to have a team that is very experienced or knowledgeable in the chosen solution. There is no “Easy Button” with EIM for the typical organization.
Eric – Great post as usual, looking forward to the dialog. I really expect then next wave to include “HANA-type” as that semantic layer evolves, but two’s company…for now.
Now where’s my popcorn!
Thanks Jonathan! I’ll look forward to hearing you and Ethan go at it on that one 🙂
What I like about Jonathan’s comments – BI is not simply a case of plug-and-play. Build it and they will come does not work in analytics. To me the biggest challenges with getting adoption has almost always more to do with non-application topics – exec support, real biz involvement, quality specs from right folks – that represent the business, and are signed off. scope mgmt. Extensive prototyping….quality project mgmt…
The easier tasks then become mapping the right technology, to the specs…
All when I said “I just love keeping this conversation alive:” the comment was one part sarcasm and another part a desire to help both sides see that “both sides” are sometimes correct given the right circumstances and / or requirements of the business users. I personally prefer EIM solutions with direct access and control to the target database (IE DataServices with Sybase and HANA and BOBJ) but that does not mean that SAP BW is a bad solution. Both solutions have their own unique challenges and benefits.
Since Jonathan is quoting some of my tweets, I’d just like to briefly state that the point of view he seems to be attributing to me do not reflect me actual positions 🙂 For non-SAP data, there is very little pre-delivered semantic content for BW, so of course I would never hold that modeling was easy or “cookie cutter”. Data modeling is a hard problem that requires an expert, regardless of the platform you use.
But I do think that BW can help with some of the really tough data and systems management tasks that are hard to do with the RDBMS + ETL approach because these tools don’t directly address the tasks. To list just a couple of examples:
– How do you deal with data consistency during load operations? Are you going to do all your loads within a transaction (and does this result in a performance hit?), or are you going to have momentary data inconsistencies?
– Do you create a near-line storage approach from scratch? If your near-line storage approach is moving data from one DBMS to another, how do you handle generalized queries across both data stores? How to you maintain data consistency when near-lining a chunk of data?
– Do you have a governance process that is strong enough to ensure that you can confidently make programatic changes to all of your existing database structures to take advantage of new database features? How about managing complete platform migrations?
BW has time-tested methods of handling these and other issues, in use across 100s (and in many cases 1000s) of customers. I’m not sure of the official designation of these types of problems, but they seem to be to stray from the realm of traditional data modeling where “Kimball” or “Inmon” are reasonable answers, into the realm of data management and systems design. Despite being fairly experienced dealing with these types of problems (though not nearly as experienced as Jonathan, I’m sure), they are problems I don’t want to deal with directly if I don’t have to. These are just a few of the things that motivated my comment on “reinventing the wheel”, which Jonathan quotes. Sometimes rebuilding a wheel is the right thing to do (just ask a professional cyclist), but a lot of the time it’s not for reasons that involve cost and time.
Indeed, BW’s methods of dealing with these problems are probably not the right methods for many organizations. I think that Jonathan really brings the right approach of carefully weighing the many factors and requirements that go into these decisions, and I think in some circumstances BW could conceivably significantly reduce the cost of maintaining a datawarehouse, even for predominantly non-SAP data.
There are a lot of other reasons for not using BW (or using it!) and many of these are the fault of SAP or deep design problems with BW itself. But I do hope that this conversation will get people a little more familiar with some of the advantages that BW can offer and some of the difficulties in building data warehouses that BW can help address.
Cheers,
Ethan
Sorry Ethan. I was not trying to pick on you with the comments or quotes. That was not the first time that someone used the term “re-inventing the wheal” to describe the traditional ETL tool EIM path. My quotes were not specific to anyone nor did I intend them to be. After reviewing your response, most of your concerns are legitimate reasons to use a software tool to managing an EDW or DataMart. From my perspective Data Service and other ETL tools are very capable of managing all aspects of EIM and are very time tested and easy to use. Simply review the number of ”out-of-the box” GUI based transforms in Data Services. Most of the transforms were developed to address the questions you listed. Tens of thousands of Organizations use them to manage their data. However, not every ETL tool on the market is created equally. For example, Data Service and Informatica are very different tools compared to Microsoft’s SSIS and Open Source Talend. The top of the line tools make EIM easy. Sometimes the free tools require a lot more coding. I agree that BW 7.3 mixed with NLS is a very seamless way of managing data storage at different tiers. This is being utilized on my current project and it works well. However, I could mimic something very similar either using my ETL tool or using SAP Data Federator (BOE 4.0 Multi-Source Universes). Arguably it would not be as easy to implement as NLS, but it would work.
Thanks for the followup Jonathan. I think we’re on pretty much the same wavelength after all 🙂 Let’s keep up the conversation. I’m looking forward to learning more about your approach to handling these issues. Cheers, Ethan.
Hi,
this is an excellent discussion. But I like to give it a different angle in order to then come back to the BOBJ-BW relationship. First of all, I distinguish between DW (well, EIM) and BI. BI is not necessarily tied to a DW; BI should work on single sources too, e.g. for operational BI, let’s say on an ERP or CRM system.
Now the latter typically and already have some (reporting) semantics defined; at the bare minimum security is managed by those applications (and not via DB-based security). For that, it would be ideal if a BI infrastructure could leverage semantics provided by the underlying applications. The approach (a) to start with zero semantics on the table level (the “least common denominator”) and (b) to require rebuilding semantics that already exists in an application then leads to unnecessary effort, redundancy and non-synced semantics (e.g. security managed twice: once by the app, once by the semantic layer).
Now moving back to the case of BW: here, there is a fairly rich semantic layer and ignoring the latter is more eye-catching than in the case described above. But fundamentally it is the same thing. So my point is that the [BOBJ] approach of starting with no semantics on a pure table level has the advantage that it obviously works everywhere and by default (“I know how to use a hammer and I can use it everywhere”). However and in some cases (like BW but also ERP, CRM, …), it creates huge redundancies. Now, the BOBJ semantic layer can leverage BW semantics but in the context of the discussion here I sense that this is considered by many as a stepchild rather than an asset. I think that the smart way is to allow a semantic layer to incorporate existing semantics rather than forcing to build from scratch.
Good points Thomas. It is very true that the agnostic EIM approach mixed with Data Services to manage the data and a database to calculate and store the data is a very “works everywhere” type approach. On the other hand you really need to know how to swing the hammer to make such a solution successful. My prior comments were mostly comparing BW’s data management to Data Services’ data management capabilities in an EIM scenario. From a semantic layer and presentation layer standpoint the biggest issues with the SAP BW and BOBJ merger is the fact that both companies (when merged together) have a lot of technology with overlapping parts but functionally different components. Take for example the current need to create an InfoCube in BW, develop a BEx report and then develop a Web Intelligence report in order to use BOBJ. The fact that a user has to create two reports is a bit redundant. The way I see it, Web Intelligence was a tools developed for SQL based ROLAP analysis. BW presents its data using MDX and OLAP. As you can see, it has taken and will continue to take SAP a lot of time to seamlessly merge Web Intelligence with an OLAP backend. I have faith that SAP will find a solution but it is going to require patience and changes to both Web Intelligence and BW. One thing I really hope to see happen is that ability to connect Web Intelligence directly to an InfoCubes, Multi-providers and / or DSO’s in the future without the need to develop a BEx query report or Universe. This will really go a long way in making the two technologies seamless (in my opinion). My only fear with this merger is that the BOBJ focus will shift from being an agnostic (works everywhere) tool to a tool designed to only support SAP’s existing stack of application. However, I don’t see this happening today.