Archive for May, 2012

Catching up – Starting with the common framework

Trying very hard to keep a balance between business school learning and timely tech updates. There is a lot of catching up to do ranging from MS SQL Server 2012 BI visualization capabilities like power view to the detail of using column store indexes. I did get an opportunity to work with Integration Services and certainly is a game changer in the area of ETL tools out there. I find it encouraging to see large organizations pursuing the use of SQL Server Integration Services not only for data warehousing related workloads, but also for day to day operational activities (tasks\processes).

Technical managers and decision makers should start thinking of a common platform for data movement across the enterprise; not just for governance reasons but more for the ability to trace and remove process redundancies from the organization. Need a place to start? Well, start by making an inventory of your processes. If you already have a project\process portfolio management tool use it to your advantage. Have a parallel development effort started on designing and implementing a common framework for maintaining these processes. This should have the ability to audit events, capture exceptions and errors, log process updates etc. More like a monitoring tool for all processes. Start simple. What do you want to know – “When did my process start\end?”, “Did it fail\why\Where do i see the error?”, “How many rows\files were transferred in the process”, “How do i get notified in the event of a failure?”. Let these types of questions drive your requirement. I would recommend to keep the implementation of the process framework as technology agnostic as possible as i have made this mistake by creating something close to the technology (script tasks for logging, auditing and event handling) and ‘when’ the technology upgrades, you are screwed. Just more work to redefine the framework.

Let me try to get a conceptual example with code as examples to use. Going back to the drawing board on trying to get something in place as a ‘common’ process hub, will be back soon. I hope …. with Managerial Accounting looming closer and sending shivers down my spine.

Categories: Uncategorized