I work as a technical lead in the Business Intelligence and Analytics area, specifically ETL. Ab Initio is the corporate ETL tool but we also use other tools and supporting technologies such as SQL and UNIX.
The majority of my time is spent on designing and building new applications for the corporate Data Warehouse. These applications may involve brand new data sources or the creation of a particular subject area Data Mart. These new applications are usually large enough that I oversee a number of developers as part of the effort. I am also heavily involved in early stages in the project lifecycle, i.e. estimation, requirements gathering, data modeling, business rule definitions, etc.
Secondary aspects to my role include mentoring other members of the team and providing second-line support for production issues.
This varies; it can be if project work is well planned with realistic deadlines. On other occasions I may be investigating data issues that have critical impacts that require me to work outside normal hours.
It was a relatively natural progression for me. I studied Computer Science at university and started as graduate IT professional at a very large Computer Services organization. There I worked on many different projects for different clients across different industries. Eventually my roles became more data focused and I found this aspect more interesting and challenging to the more front-end technologies.
Generally speaking I think there are two factors that anyone needs for any job, aptitude and attitude. This may be a bit clichéd but you need a good balance of the two to succeed.
The aptitude aspect means you have a deep understanding of the technology you work in and a wider understanding of computing and corporate environments. This allows you to consider how your designs and solutions may impact surrounding systems and means you are able to discuss and investigate integration points between your area of expertise and the points where it interfaces with others.
The attitude aspect is just as important, if not more so. Simply put this is taking professional pride in your work. You are accountable for your deliverables, be they designs, analysis, software code or anything else. This same attitude should help you create a good working relationship with colleagues and customers.
The biggest challenges are typically data volumes and requirements. Dealing with large data volumes brings a set of problems that don't exist when the volumes are small. Very careful consideration needs to be taken so that processing times are realistic, especially if large historical data volumes are involved. Larger and older data is likely to include more issues as it has probably been produced by a system that has undergone change and may have included bugs. Finding and dealing with the incorrect data is crucial to creating a smooth running application.
Capturing requirements is a problem area that affects most IT applications. The agile method has tried to address this problem but it does not necessarily lend itself to data warehouse applications as reloading years' worth of data to support a new requirement can be a very large undertaking compared to changing a screen layout that has no associated data.
The rewarding parts of my work are delivering elegant applications that run with minimal issues and are simple to maintain. If you return to an application to incorporate an enhancement and it is straightforward you know you got the design correct initially. The mentoring aspect is also rewarding when you witness junior team members progress such that they are able to work independently, confident that if they are responsible for something you are no longer required to oversee it in any great detail.
Recently my spare time has been mostly taken up by extensive house renovations that are hopefully soon to be completed! I did manage to squeeze in some training and I recently ran the Belfast half-marathon. This was very satisfying. I think perhaps the full marathon is beyond me but I would definitely like to do the half-marathon again, along with the relay which started my interest earlier this year.
I spent part of my formative years living in Fiji and I do occasionally get mistaken for being an Aussie or a Kiwi on a regular basis, much to my wife's amusement.
I don't think I could pick out a single individual but I generally admire those who achieve without fuss and self-publicity and who are open-minded enough to listen to new ideas, change their mind without fear of appearing weak and consider others views.
Hot topics at the moment are ever increasing data sizes and security. With the proliferation of so much data it is getting easier for this to be accessed by others. Novel ways of protecting this could be the next big thing!