Michael's BI Space

(ISBN: 978-0-470-48432-6, Paperback: 648 pages)
Congrats to Roland and Jos for completing and releasing the much awaited tome for Pentaho Solutions !
It's had some good reviews and as it is with tech specific subjects expects some level of understanding re DWH/BI (DWH Toolkit vol. 2)

Pentaho's Beekeeper blurb also does a good job of tackling the open source model from their perspective.

It's not always a case of just throwing more CPU and hardware when intending to scale up for increased processing activities - you need to understand the components that wil be impacted prior to scaling up either horizontally or vertically.

Scheduling specific:

Primary throttle point:      

• Web Intelligence Job Server – scheduling request manager – 1 CPU can supposedly support 5 jobs – PROD currently has 2 job servers.

Secondary throttle point:  

• Web Intelligence Report Server – creates the report instances – will expand to use as many CPU’s as it requires and are available – user address memory cap of 2 GB per report server – Prod currently has 2 report servers.
  Note: Physical Address Extension (PAE) switch changes the addressing mod to enable the OS to access more than 4 GB RAM i.e. the Report Server 2 GB cap limit still applies, but you can effectively run multiple heavy processes.

In summary:

2 more CPU’s could double the current scheduling capacity but I would suggest adding more RAM and looking into PAE.
We need:

• To understand and adjust some parameters for XI 3’s built-in memory management.
  I.e. it is possible to recapture memory based on the “Enable Memory Analysis” options – I think the defaults are still in place.
• Additional info on the unmanaged disk PDF generation overheads.
• Average anticipated report run times / and sizes in case we have to adjust the -RequestTimeout parameter and the cache settings on the Report server.
  I.e.it’s not enough to just know anticipated disk space requirements as one also needs to scale parameters for anticipated load increases.
• Review Tomcat heap settings and/or Tomcat 6 upgrade.
• Review the CMS database for optimal use.   

Generic:

Web Application Server (Tomcat):

• 400 sessions
• 100 simultaneous requests per processor – non report requests i.e. folder clicks etc
• 50 simultaneous report viewing requests

CMS Database:

• 600 concurrent active users

Additional:

• If Prod is a quad / 8 GB – and you're running on a 2 CPU license, it has yet to be determined if & how the CPU license cap throttles back on a server that is scoped beyond the cap.
• Always collate detailed statistics on the existing installation prior to expanding – suggest this as priority 1.
  E.g. Use Perfmon or a similar monitoring tool over some consecutive batch and heavy user windows for OS / memory / processor / application stats.
• A review of ALL XI component configuration parameters - apart from the built in memory management mentioned previously, there are also Web Intelligence Report Server cache clean / dirty memory settings still use the defaults.
• The vendor has no recommendations for the Connection Server – possible throttle point – need to determine some thresholds on this component asap.
• Tomcat 6.0.2 supposedly has improved Memory usage optimizations / Advanced IO capabilities - the default install is 5.5.
  i.e. always look if it is possible to gain from any new releases e.g. XI 3.1 SP2 running on Tomcat 6 - benchmark this first!
• Crystal / Olap Voyager
  A total rethink will be required if intending to adopt / use these components along with scaling up for increased batch activity!
• Etc.

Most do not understand the delineations between the various offerings and it's even more disheartening to hear this from the mouths of account managers or CIO's - much to the detriment of some serious contenders for their BI space.
James Dixon's Blog takes a novel approach to rectifying this.

On the subject - Micro Strategy now offers the Report module with 100 free seats and Pentaho has some more stuff from Lucidera!!

Humanity prides itself on the Freedom of Choice - but with BI one cannot be emotive or overly logical about the selection of a technological direction.
New entrants probably have it easier as there is no legacy or emotive baggage re a preferred vendor and tool set, but that does open Pandora's Cube with respect to the variety and interoperability of today's many offerings.
Ramping up from an existing set of solutions oft see's brand loyalty get blindsided by the shiny new toys on offer from competitors all a-twitter with the very latest buzzwords promising rainbows of information amongst the 'BI Clouds'.
But where is brand loyalty with all the mergers - or does vendor consolidation and RDBMS tool set tie ins dictate more than they used to in the sense that the one stop SAP, Microsoft, IBM and Oracle shops make more sense these days
Sure - there are many benefits to be had from specific applications like Qlikview or from the open source application or enterprise offerings like Mondrian and Pentaho - not to mention BI Application Servers from the likes of IBM/Cognos etc, so the field is pretty complex. 
And then there are escalating costs....... ! Always a strong motivator for corporations busy reviewing what they have and what they need and what they might like against What They Will Pay!

BI for some is mission critical and you're sure to find them amongst the TDWI events and actively monitoring their investments on the Gartner Quadrants.
Others will be more open to changing technologies when there current outdated installations warrant a major upgrade or migration.

And then we're back to choice - you can buy a pre-created tool comparison from Ciny Howson or search online for a list of many offering the same e.g, BI Toolbox.
If you don't want to buy, then create your own comparison matrix: Example

The fact is that each site has specific requirements with a number of overlapping factors that influence the best fit of technological choices catering for current and future needs.
Local support and resources, compliance with the organisations overall IT strategy, scalability and scope for future requirements are but a few factors to take into consideration that have not much to do with the delivery promises of the vendor and his magic bag of tricks.

It ALL boils down to which product(s) best suite your organization in the most multi-faceted way!

And exactly how does Muay Thai fit into all this?
Well, lets just say that if you want an effective no frills systems that works for a stand up fight then Muay Thai is the way to go.
To expand the fight down to the ground as in UFC you will need some serious grappling skills - Jiu Jitsu.
I've trained in the standard styles which were great for learning the basics and discipline, but to be quite frank, I've rather switched to a Muay Thai regime for the sheer beauty of it's effectiveness.
There is a place for all styles - normally matched by your inner resolve and body type as to why you want this training.
The Chinese and Indian styles are for slim practitioners that rely on circular movements to offset their lack of bulk and power.
The Japanese and koreans are brutally leg and body strong so have opted for linear aggressive attack lines.
The Brazilians like flair, are very mobile and embrace beauty and balance with their Capoeira routines.
Not all training is available anywhere, and there have been many offshoots and break away schools - Bruce Lee was the initiator of taking the old and adapting it to something new and effective, whilst the Gracie family in Brazil took wrestling and judo to task with their ground and pound Jui-Jitsu.
Steven Segal promoted Aikido - interesting it's mechanism of using an opponents force against himself, whilst Jean Claude Van Damme trained in Shotokan and even did Ballet training for stretching and flexibility.
The russians keep appearing - hard men with Spetsnaz training.

But pound for pound and minimalistic effective approach, my money would be on Muay Thai ... maybe it's the music they play at the matches lol but the same could be said for many BI tool sets and all the marketing and hype doing the rounds.

The most obvious question one can ask about all things forecasting is: Where were you at the eco crunch?
I had just finished reading The Black Swan by http://www.fooledbyrandomness.com/ when the economy slid into the ocean of debt and was quite gobsmacked at the timing of it all.
Like anything, Analytics and Forecasting are useless unless seen and understood in the greater context and by the RIGHT people.
There were many voices trying to scream a warning about the financial state that turned nasty but they were drowned out by man's greed and the fact that people simply don't like bad news - or more to the point, will not make a judgement on the side of caution.
We're positive gambler's by nature, always looking for the big score or to increase what we have so it's those that wave the red flag that get nailed to the ridiculous cross either in front of the building as a warning, or way out of sight to stop a possible infection.
The latest tactic is to just ignore these prophets of doom and wait for the daily deluge of facts and white noise to wash it all away.

Be that as it may, with the right approach and champion, one can slowly educate a dept, corporation, industry or even a planet, as long as the basics are adhered to:
Forecasting Methodology Tree
9 Common Forecasting Errors