I’m intrigued by some things I’m seeing in the community about the new Common Semantic Layer (CSL) vs. the BICS Connector in SAP BusinesssObjects BI4. Disclaimer: I am not writing to call out anyone, indicate inaccuracy in statements/tweets, etc. I just get a vibe that the CSL is not getting the love that it should. I feel compelled to warn that this is a pretty opinionated post. I generally like to stick to technical blogs, but do believe there is some value in pondering both sides of the fence in building content in SAP BusinessObjects BI4.
Classic SAP customers should be excited about the CSL. Trying to report on SAP BW prior to BI4 meant a single universe per SAP BW query. This was a challenge for user adoption and usability if what you wanted was in two different queries. I am aware of circumstances where this was compelling enough to make organizations rethink integrating SAP BusinessObjects for reporting on SAP BW.
BI4 and the CSL have “changed the game” (no kittens were harmed during the publication of this blog). We now have means to not only integrate multiple cubes into a single data source, but we can now also integrate this data with non-SAP data sources such as Oracle, SQL Server, etc.
But then there is this other temptation to use the BICS connector for a direct connection, and arguably, a little more functionality when working with SAP BW. While quicker to implement by just pointing a Webi report directly to the BICS connector, we lose something, and that is the interoperability with other data. Don’t get me wrong. I think the BICS connector is an outstanding approach to quick-hit, tactical reporting, perhaps most appropriate for the rapid response requirements from the business, or by power users that really know your BW data inside and out.
I’ll Keep My Merged Dimensions….Thanks
I think every developer wants to should do right by their users in creating the best user experience possible. The great news is we have a ton of mechanisms to create that good user experience. Merged Dimensions, ETL, dumping-data-to-excel-to-import-to-access approach…you name it. But one thing holds true, and that is that we’ve not had an integrated solution like the common semantic layer in the past that created a good user experience alternative for combining disparate data, let alone one that did so in an abstract way.
Semantic Layer Strategy
The coming years with the new semantic layer make it more and more important to have a clearly defined semantic layer strategy. In years gone by, we may have only made that strategy as narrow as a data warehouse or an operational data store could extend. Why? It goes back to that single data source connection per universe.
But now, there are more factors than ever to consider. Do we use ETL to consolidate mission critical data? Is SAP HANA or Sybase IQ going to be the home of all the enterprise data our users will need? Will the common semantic layer be built in such a way to support queries to disparate systems to pull data together.
All these questions, supplemented with a subject area/data domain map in your environment, will paint the big picture of where data acquisition vs. a common semantic layer will make sense. I think it’s too early to call using the BICS connector as a defacto standard for getting back to classic SAP data because it may be perceived to be better today. It’s the forward thinking strategy that promotes sustainability of reporting that will have big implications for IT supporting business intelligence/analytics. I feel like the common semantic layer is the winner there in the long run.
When Do We Go?
If you are not already on BI4 and are already carefully considering my thoughts here so far (of course you are), the remaining question this blog poses is: Does the BI4 platform make a compelling enough business case for you to make the leap today?
That question has two answers:
- Are you a classic SAP BW customer? Then yes, do it now. Take the chance to improve the user experience over the single source universe with a well-thought semantic layer strategy. Enough said in my book.
- Are you just a classic BOBJ customer? Well, not JUST a classic BOBJ customer, my brethren. My gut tells me to go. I do not think this is a major release to consider skipping. While there are valid maturity concerns to be had at this early stage of the game, the merits of BI4 and the common semantic layer do justify the effort. That is not the only selling point but it is a really good one in book. This is a capability I know I have lusted after in my 13 years working in this technology.
I wonder if, once the Business Suite sits on HANA, will there be a need for BW at all? Or will we just build a universe directly on top of the in-memory transactional system (and of course tie that to data from other sources)?
Off topic 🙂 At the risk of serious flaming…I think that BW is so embedded, we are looking at years (if not many years), before HANA in the place of BW is pervasive enough that this will stand out. But back on topic, you can’t really expect HANA to be the only database an enterprise maintains. The common semantic layer should stand out as the hook if you can’t or don’t want to build the ETL hooks.
HI Eric,
you are making the recommendation for SAP BW based customers to leverage the Universe on top of BW, which clearly is not what SAP recommends and not what is common practice for very good technical reasons.
The Universe on top of SAP BW is providing the option to use a multi-source layer – yes – but it is a relational approach, where as the BICS connectivity is a multi-dimensional approach.
So clearly the recommendation for SAP BW based customers is to leverage always the BICS connectivity first and only in the case of a multi-source aspect to leverage the DSL.
regards
Ingo Hilgefort
Hi Eric,
I second Ingo as BICS can’t be replaced with Universes simply because of the OLAP features that BEx queries offer such as hierarchies, calculated and restricted key figures, authorizations based on user exit variables… etc.
Ingo/Ali, I very much appreciate your inputs but I think you missed my core message in this blog post. I did and continue to acknowledge BICS has its place. However, the theme of this post is in the need to pull in disparate data from many sources, not just BW. So yes, in your point, relational in thought. Your argument is not against the common semantic layer but pro-BICS. My message is there is indeed room for both and tries to underscore where the right use cases exist.
Hi Eric,
I did understand that portion but the blog also has technically speaking wrong information:
you mention that the Universe is connecting to the BW Query:
“We now have means to not only integrate multiple SAP BW queries into a single data source, but we can now also integrate this data with non-SAP data sources such as Oracle, SQL Server, etc”
The Universe is not connecting to the BEx query, the Universe is connecting to the InfoProvider level and as a result out of that there are several technical consequences.
For BW based customers the direct BICS connection is clearly the first choice and even in the case of a Multi-Source requirement it becomes a question on how often such a multi-source requirement is needed.
if customers need it only once and “now” – sure there are several options. if the requirement is a regular needed request, then such a scenario should be part of the underlying datawarehouse – in this case BW.
overall I understood the intention here, but you making a recommendation towards data connectivity without outlining the technical differences and in the last part there is no clear outline of when a customer should use what and why.
regards
Ingo Hilgefort
So, semantics in technology terms aside (I’d assert that calling this misleading is a bit strong when we are speaking in general technology terms here), the practicality is that “best case” is not always the hand one dealt. The cases mentioned above in which customers tried to use the XI 3.1 universe to this end, and ultimately hated it because they couldn’t integrate, is what drives the message here. I’d love to hear your take on where the common semantic layer fits in SAP’s strategy then.
I have worked with data federator (The CSL multi source engine) for about four years and I can attest that it is not a scalable solution for combining big data sets from multiple sources unless the data is first highly aggregated by at the source DB and then combined in in small quantity by Federator. Federator might be a great choice for small data projects but there are several push down limitations when connecting to BW base tables or other databases that result in poor performance. The best option is to get the 3rd data into BW using Data Services then use BICS via Webi 4.0 . If you don’t use BW then you can use Data Dervices to create a solid data warehouse or mart to fuel the CSL with a single connection. I am glad SAP included Federator in the CSL but you have to be realistic about its scalability and support for various databases.
I love the discussion! I do not see it as cut and dry as that. ETL, any day of the week, is going to build the most effective mart or warehouse from a pure data perspective. Flipping it the other way around, trying to develop that solution that delivers a “warehouse feeling” reporting experience that you find in relational reporting for users does strike a (flat) chord. Spot on – BICS is native and faster. It taps capabilities OLAP in nature. However, you are still left a very common problem in which we are asking them to use merged dimensions, as an example, to walk across two queries. Is that bad design on the warehouse front? Maybe. Do BW developers scoff at redesigning their cubes just so you can improve the reporting layer? Not uncommon. Does it happen often? My perception = yes. While the CSL is adding yet another middle layer to combining data, it strikes me as a preferred solution over dealing with education and performance in merging dimensions on large micro-cubes. But I’d really prefer to carefully craft that experience on the universe and be able to enforce application controls than make them keep trying to figure it out on their own in the reports. We’re just perpetuating the experience that BOBJ is limited in what it can do and driving them back to sucking it all into Access to do it on their own if BICS is considered to be the only option.
I must be a bit confussed, so back to BI School for me.
Through reading all of these blogs and the same responses time and time again, it is becoming apparent that my understanding of BI and its components is a bit off the mark. Perhaps someone can humour me with some responses to these couple of Questions:
Is a Cube a Data Warehouse? Is a Data Warehouse a Cube? Are the two mutually exclusive?
Can a Data Warehouse be Multi Dimensional? Does a Data Warehouse have to be Multi Dimensional? Is having BW akin to having MSAS?
If you only want Multi Dimensional should you use only BW/MSAS or the like? If you have BW/MSAS or the like, are you only interested in Multi Dimensional? Is there only 1 way?
Eagerly anticipating a response.
Thanks
Corey
Hey Corey, I’ll be the brave one and dive in here. A few swipes.
Is a Cube a Data Warehouse? Is a Data Warehouse a Cube? Are the two mutually exclusive?
I think the better question is, can customers leverage SAP BW as a Warehouse? Many do and live solely on that with no other relational sources they call a warehouse. They recognize all of the benefits of multi-dimensional analysis. However, customers do still want to be able to leverage that very same data in a relational way without reinventing the wheel. There is not a “perfect” recommendation for them at this time.
Can a Data Warehouse be Multi Dimensional? Does a Data Warehouse have to be Multi Dimensional? Is having BW akin to having MSAS?
Given my first answer in that customers are achieving this in practice, with limitations, yes. The pains are happening when given a business problem to solve, if data doesn’t exist within a single source set of data, users must get fancy with merged dimensions in a report, join it manually outside of SAP BusinessObjects, or use a UNX.
If you only want Multi Dimensional should you use only BW/MSAS or the like? If you have BW/MSAS or the like, are you only interested in Multi Dimensional? Is there only 1 way?
I’d defer this to long time BW guys like Ethan Jewett but as a source system yes. Consumption of that may vary, as you know, in more ways than you care to count.
I’m sure that hurt as much as it helped 🙂