Home » Is business intelligence (BI) primarily a technical project or a business project? [closed]

Is business intelligence (BI) primarily a technical project or a business project? [closed]

Solutons:


Is B.I. a business or technical project?

There are too many variables to answer this categorically; I’m tempted to VTC the question as it doesn’t really have a single correct answer. However, on second thoughts I can say a few reasonably meaningful words on the subject.

Customer Intimacy

Business Intelligence (or more prosaically reporting) is very requirements heavy in comparison to the amount of implementation work involved in the delivery. Getting the requirements right requires nuanced analysis but may not require technically sophisticated solutions – in some cases a spreadsheet is sufficient.

I.T. departments may not have a sufficiently close working relationship with individual consumers within the business to effectively gather the requirements for a B.I. solution, or may simply not be responsive enough. In many cases desktop tools such as Excel and Access are sufficient for the job, so reporting teams are often run directly within the individual business units.

I have seen a number of sites where B.I., reporting or analysis teams were run within the business precisely because of the issues with responsiveness and working relationship. In many cases they even had their own server hardware, ETL processes and development staff.

Strong business sponsorship vs. responsibility without authority

On a larger scale, data warehouse or business intelligence projects are middle men. They do not produce data, but aggregate and present it in a form that is easy to report on. This means they are integration projects and have to interact with many little empires within the business – sometimes called ‘gatekeepers’ in the literature.

This interaction – and the attendant real or perceived shifts in power – tend to politicise data warehouse projects and make them heavily dependent on the cooperation of many third parties. These third parties will have responsibilities and agendas of their own and no responsibility for the project success, so they can seriously affect the progress of a project and even derail it without necessarily being held responsible.

Building a non-trivial data warehouse system involves putting fingers in a lot of pies, including ones where they may not be particularly welcome. Without a business sponsor who has the political will and the political clout to bang heads together on behalf of the project, this generates a significant risk of failure or limitations on delivery that can’t be worked around. Weak business sponsorship is the #1 cause of B.I. project failure.

Getting it right

The flip side of this is that a larger scale project is much more challenging technically and from a ‘software engineering’ perspective – keeping the code base well organised. This requires technical skills that may or may not be present in a B.A.U. reporting team. Most reports or other dependent feeds from a data warehouse will be tightly coupled to the data model unless care is exercised in interface and data mart design – and even then a database has a limited range of options for supporting abstractions efficiently. This means that getting the model wrong can be very expensive or even infeasible to fix.

Failure to build the system properly can also allow a large class of leaky abstraction problems where the underlying data exposed through the system is too dirty or inconsistent to use through an ad-hoc reporting tool. A project where the brief has aspirations to a self-service facility must be built so that the data presented is clean, consistent and presented in a form that works well with the ad-hoc reporting tools.

Leaky abstractions place non-obvious elephant traps in the data and make it easy to report incorrect data from the system. Even if the problem could (in theory) be corrected with user training this can significantly erode end-user confidence in the system. Getting conformed data (where it may behave significantly differently across multiple sources) is a critical success factor for any self-service reporting project and may well be a non-trivial undertaking.

In many cases technology is a non-issue, although I have seen projects switch platforms due to technical limitations of the original choice. In some cases data volumes may require the use of certain technologies; in others a ROLAP tool may lack sophistication necessitating the use of dedicated OLAP software. A server consolidation environment may not be the best choice of platform for a TB range data warehouse system (don’t laugh – I’ve seen this attempted on more than one occasion).

Platform choice can be politicised and can also have a significant bearing on project success or implementation costs or time scales.

Conclusion

A large B.I. project will need strong sponsorship, priority treatment from the business and good technical leadership to work effectively. In most cases the actual technology used is less of an issue, but an unsuitable platform can impede the delivery of an effective system.

  • Weak sponsorship allows gatekeepers to impede access to necessary data or domain knowledge, or the project can be constrained from delivering an effective solution by petty office politics.

  • Lack of business priority also impedes access to necessary subject matter expertise, or allows the delivery to be dumbed down to sub-optimal as-is practices.

  • Weak technical leadership can result in a platform that lacks flexibility and becomes too expensive to change.

  • Ineffectively cleaned and conformed data generates leaky abstractions in the reporting layer that can erode user conficence in the system.

  • In some cases the project can be crippled by poor choice of technology.

Stating that a B.I. project is a ‘Business’ initiative is something of a truism. Technically it is correct in that a project of this sort is likely to fail without effective business sponsorship. However, there are also a variety of technical critical success factors whose absence can seriously unseat a project.

In practice, I suggest answering queries about a business intelligence project with something like:

We can, but the price will be your firstborn child and the head of your CFO1 mounted on a plaque on the wall of the project office.

If the business sponsor just briefly considers that – even for a moment – then they have about the right level of motivation and commitment to the project’s objectives. If they have the clout to take out a contract on the CFO then they have about the right level of political influence to back a project of this sort to success.2 Anything less than that means your project is at significant risk from weak sponsorship.

1 Obviously if the CFO is the sponsor than you will have to nominate another sacrifical victim of equivalent status.

2 Also, we don’t necessarily recommend you hold your sponsor to this price. In practice, it is normally sufficient to demonstrate an appropriate level of commitment to the project.

Like all IT projects it is both. You cannot succeed without a strong business need and support nor can you succeed without people with the techincial skills to put the project together.

I would consider BI a business project, because the outputs are metrics that business is required to use. This is despite the fact that about 80% of the input is technical.

In my experience, if the focus of the project is technical, then alot of time will be spent analyzing the implementation technology, yet if the focus is business them more time will be spent deciding which metrics are to be delivered, how often, how fast etc, and the technology will be selected based on the constraints. Infact most BI project start with database based reporting before outgrowing to full blown BI.

Bottom line, for success focus on business, and bring in tech to deliver “goods” within budget and time constraints.

Related Solutions

Building a multi-level menu for umbraco

First off, no need pass the a parent parameter around. The context will transport this information. Here is the XSL stylesheet that should solve your problem: <!-- update this variable on how deep your menu should be --> <xsl:variable...

How to generate a random string?

My favorite way to do it is by using /dev/urandom together with tr to delete unwanted characters. For instance, to get only digits and letters: tr -dc A-Za-z0-9 </dev/urandom | head -c 13 ; echo '' Alternatively, to include more characters from the OWASP...

How to copy a file from a remote server to a local machine?

The syntax for scp is: If you are on the computer from which you want to send file to a remote computer: scp /file/to/send username@remote:/where/to/put Here the remote can be a FQDN or an IP address. On the other hand if you are on the computer wanting to...

What is the difference between curl and wget?

The main differences are: wget's major strong side compared to curl is its ability to download recursively. wget is command line only. There's no lib or anything, but curl's features are powered by libcurl. curl supports FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP,...

Using ‘sed’ to find and replace [duplicate]

sed is the stream editor, in that you can use | (pipe) to send standard streams (STDIN and STDOUT specifically) through sed and alter them programmatically on the fly, making it a handy tool in the Unix philosophy tradition; but can edit files directly, too,...

How do I loop through only directories in bash?

You can specify a slash at the end to match only directories: for d in */ ; do echo "$d" done If you want to exclude symlinks, use a test to continue the loop if the current entry is a link. You need to remove the trailing slash from the name in order for -L to...

How to clear journalctl

The self maintenance method is to vacuum the logs by size or time. Retain only the past two days: journalctl --vacuum-time=2d Retain only the past 500 MB: journalctl --vacuum-size=500M man journalctl for more information. You don't typically clear the journal...

How can I run a command which will survive terminal close?

One of the following 2 should work: $ nohup redshift & or $ redshift & $ disown See the following for a bit more information on how this works: man nohup help disown Difference between nohup, disown and & (be sure to read the comments too) If your...

Get exit status of process that’s piped to another

bash and zsh have an array variable that holds the exit status of each element (command) of the last pipeline executed by the shell. If you are using bash, the array is called PIPESTATUS (case matters!) and the array indicies start at zero: $ false | true $...

Execute vs Read bit. How do directory permissions in Linux work?

When applying permissions to directories on Linux, the permission bits have different meanings than on regular files. The read bit (r) allows the affected user to list the files within the directory The write bit (w) allows the affected user to create, rename,...

What are the pros and cons of Vim and Emacs? [closed]

I use both, although if I had to choose one, I know which one I would pick. Still, I'll try to make an objective comparison on a few issues. Available everywhere? If you're a professional system administrator who works with Unix systems, or a power user on...

How do I use pushd and popd commands?

pushd, popd, and dirs are shell builtins which allow you manipulate the directory stack. This can be used to change directories but return to the directory from which you came. For example start up with the following directories: $ pwd /home/saml/somedir $ ls...

How to forward X over SSH to run graphics applications remotely?

X11 forwarding needs to be enabled on both the client side and the server side. On the client side, the -X (capital X) option to ssh enables X11 forwarding, and you can make this the default (for all connections or for a specific connection) with ForwardX11 yes...

What does “LC_ALL=C” do?

LC_ALL is the environment variable that overrides all the other localisation settings (except $LANGUAGE under some circumstances). Different aspects of localisations (like the thousand separator or decimal point character, character set, sorting order, month,...

What is a bind mount?

What is a bind mount? A bind mount is an alternate view of a directory tree. Classically, mounting creates a view of a storage device as a directory tree. A bind mount instead takes an existing directory tree and replicates it under a different point. The...

Turn off buffering in pipe

Another way to skin this cat is to use the stdbuf program, which is part of the GNU Coreutils (FreeBSD also has its own one). stdbuf -i0 -o0 -e0 command This turns off buffering completely for input, output and error. For some applications, line buffering may...

Can less retain colored output?

Use: git diff --color=always | less -r --color=always is there to tell git to output color codes even if the output is a pipe (not a tty). And -r is there to tell less to interpret those color codes and other escape sequences. Use -R for ANSI color codes only....

How do I copy a folder keeping owners and permissions intact?

sudo cp -rp /home/my_home /media/backup/my_home From cp manpage: -p same as --preserve=mode,ownership,timestamps --preserve[=ATTR_LIST] preserve the specified attributes (default: mode,ownership,timestamps), if possible additional attributes: context, links,...

Can I zip an entire folder using gzip?

No. Unlike zip, gzip functions as a compression algorithm only. Because of various reasons some of which hearken back to the era of tape drives, Unix uses a program named tar to archive data, which can then be compressed with a compression program like gzip,...