My Journey from Business Intelligence to Data Science

One day I was at one of our clients working on a financial reporting project and realised that what I was doing each day was feeling a little too routine, a tad monotonous and in the main, predictable. Sure there were challenges, especially when trying to learn a new application, or wrangle a visualisation from a reporting tool that didn’t provide it as standard. But I was feeling that essentially I was being asked to achieve the same sort of things day in day out. The clients just wanted to see their data in an easy-to-understand format, which mostly meant bar or line charts in an online dashboard or report pack. There was usually little creativity called for, as deliverables were in-the-main, specified pretty rigidly in the business requirements.

I had been coming off a decade and a half of delivering application-based reporting projects in technologies such as OBIEE, QlikView and Tableau, the sort of applications that are the bread and butter of business intelligence consultants.

Admittedly, in the beginning of my career I didn’t see the need for being a generalist. I wanted to become a specialist, an expert in, of all things Oracle Business Intelligence reporting. Had I been allowed to follow that road, I probably wouldn’t have a job today – at least not in IT. The opportunities in the OBIEE space in Australia, are today few and far between.

Fortunately, I had been working for mainly low to middle tier IT consulting companies that required a little more out of their consultants. They expected their people to get their hands dirty in all aspects of a project both technical and administrative.

So I frequently found myself working in data transformation and data modelling engagements, the odd strategic advisory role, and even in a couple of project management gigs. Exposure to these roles over time has given me the chance to develop a range of both technical and soft skills, such as statistical programming, analytical thinking, business communication as well as the ability to think on my feet. It has been a slow but steady journey, and I forget when exactly I had that eureka moment when I discovered that I had most of the skills necessary to transition into a new data science career. That day was some time in 2014, and when it happened, it was probably because at the time one of my fellow consultants, and regular coffee partner, decided that he wanted to become a data scientist.

He had a similar background to me (but was mathematically brilliant) and had started doing one of those online courses in data science. I remember wondering why anyone would want to inflict the pain of further study on themselves once they already had a good job, but within six months he had left our company and was poached as a data scientist at a large company.

I knew that if I didn’t do something similar I would forever be stuck in my current role. So I began studying the online Data Science Specialisation at Coursera. Despite my fear of a lack of time after hours and not wanting to study again, I was soon actually enjoying myself, and looked forward to my planned study sessions during the week. I rapidly got caught up in the world of statistics and machine learning, finding that I was mesmerised by the idea of discovering the new and unexpected in readily available data.

When I finally completed the course a year later, I felt I still had a long way to go, but thought that if I could only get onto a data science project, I would learn the rest on the job. After all, my experiences in the last decade and a half made me accustomed to constantly learning on the job.

I have to say, I feel really lucky. My current employer has been amazing, and decided to allow me to transition into the data science team, where I have now worked on a number of interesting and challenging projects. Although the stress levels are a little higher than they were a year ago, I can say with gratitude, that I feel my career has been invigorated, and that the challenges have been well worth it!

Career change is not for everyone, and without family and employer support the average person would find it a challenge. Luckily I’ve had both.