A case for rethinking the data model - reporting from SAP TechEd

Yet another TechEd is over, again definitely worth the time and effort.

Mike and Stacey did amazing work yet again, my good blogger and SAP Mentor friends were a treat as always, and the management of SAP displays an amazing openness towards us unruly bloggers. Sometimes I even suspect they're having a good time despite the barrage of ad-hoc and prying questions! Are we such a nice bunch? But of course... ;)

New this year was an "NDA" note in regards some of the themes, and not being a trained journalist I had to watch my mouth and think hard in follow up discussions. Too bad, understandable of course, but it probably allowed us to dig deeper and get a better understanding, even if some of the facts had to be kept in-memory only.

Fellow traveller Dennis Howlett did an excellent report from our meeting with Marge Breya, EVP & GM Intelligence Platform and NetWeaver, aptly titled "Is BI ready to meet the real world?". That meeting I found very interesting as a distinct data-oriented culture had descended onto SAP. Like inviting an army of sleuths to roam around your business trying to make sense of all your data, beyond what your normal reports could show.

My answer to Dennis' question would be a resounding "well...". For reasons that could be summed up as "BI is trying to band-aid an imperfect Enterprise Software reality, BI does not fix any root causes for the imperfectness" making it useful, but it's still a stopgap.

Marge started with an example: Diving into their own last results she found a positive abnormality in a certain product group/area, and with a childlike "wow, how did they do that?" and "let's find out so we can replicate!" she sent off mails to the producers of the sales-spike to get better explanations.

Obviously trying to find out who did what and how; what was said and offered by whom, what sequence did things happen - all the ach so important data that they call "unstructured". In reality she was trying to reconstruct the process that delivered the good results but could not find real "process-data" only "process-results-data". The solution was email and ask for process-data, if still present in their personal in-memory DB.

Unfortunately most see the "process-results-data" as process-data, but it's clearly not so. "Sales" says nothing about the sequence of activities, instructions given, changes to process path or anything else but the results.

Whatever BI does, no amount of analysis and time consuming queries person to person, nor any type of query algorithms will ever give the exact historical picture as most of the pertinent and most important process-data resides in-human-memory. Who called first, what was said, how was that handled and by who, what then and so forth.

Hence any bettering of the process and replicating successful process will be hard, or rather impossible.

(I probably do not have to remind anybody that it's in bettering the processes where the big gains and values lies.)

A bit later we again had the pleasure of meeting with Pascal Brosset, Chief Strategy Officer, who with his daft educational talent managed to explain what the new direction they're taking with in-memory database - the man would have been a huge success as a teacher - and the bloggers gave him a good round of applause (a first according to Mike).

According to Pascal, in essence, this will allow SAP to keep the raw data before they're manipulated and stored, giving the Business Objects data-sleuths something much more valuable to work with. As such I cannot but welcome them to the club as this is one of the core concepts of Thingamy; raw data rules, manipulated date is... well... manipulated!

As a part of that a new product is underway, Constellation, that hopefully will be able to extract "process-data" (actually rather in this case "pseudo-process-data") from the mess of unstructured "process-results-data".

Using the same argument as above I sincerely doubt the practical results of that, another at-best-approximation for reality as the real process-data (not the results of a process) are simply not present.

My understanding is that such extracted pseudo-process-data could then be converted to BPM processes, in itself not a bad idea but for the fact that BPM do not do real Barely Repeatable Process (BRP) which of course are the major part of what a business engages in.

But a data model does not stand alone, it's an intrinsic part of the Enterprise system.

There is only one logical next step, turn the situation upside down and start with the process. Include the BRPs in a real system that delivers workorders and runs the processes (like Thingamy of course). There is no other way to capture real, relevant and correct process data.

The processes are where the effectiveness of the business lies, merely capturing the results and not the process itself makes little sense. In addition running the processes would eliminate the data (and the work!) created by the unstructuredness of the process itself (reports, updates, mails, search, meetings, budgets, deadlines etc).

Talking about "next step", when meeting with Jim Hagemann Snabe, Executive Board member, I asked if they have a dedicated R&D group with the single task of "making SAP's  products irrelevant"? History and nature tells us that it will happen, always, thus better to do it yourself. He agreed with the premises of the question but could not reveal the existence of any such single-minded little department. :)

[Disclaimer: SAP paid for my trip and hotel which was much appreciated of course.]

Comments

[twitter.com/_L]

When you're referring to 'in memory databases' are you talking about basically stuff inside one persons head only?

Not sure if I'm way off on my understanding or not!

Thanks,
Lee

[sig]

Lee, ooops, that was confusing, sorry about that!

They can keep the data (RAM) "in-memory" - actually it was first launched as a proper add-on product to accelerate the system. Doing a query of data in RAM makes much sense speed-wise.
In addition, and it looks like the BO acquisition elevated the importance of that, is that the data kept in-memory is raw and could have much higher value for the data-sleuths!

So that's what I'm referring to when Pascal talked about it.

Then of course I muddled it up by making a pun out of it referring to (personal) in-memory when looking for process-data, i.e. the little but so important data about who did what, in what sequence and what was said. As that data is often non-existing, after all the current systems only capture the 'results' of the process, not what actually happens, then finding out how the process played out must rely on what people can remember - i.e. personal-in-memory. :)