For data contractors, AI is still in its infancy but its decade is here

At a macro-level, end-users are still looking for the same data candidates they were 10-plus years ago, so DBAs, Data Warehouse Developers and Report Writers.

But at a micro-level, the specific skills and technologies within these contracts or roles have morphed and changed significantly since 2009-2010.

The rise of both cloud technology and start-up vendors have forced ‘the usual suspects’ such as Microsoft, SAP and IBM to change their business models (-- Power BI is now patched and updated on a weekly basis). And with that, they have entered new markets as they emerged.

This means there are more skills to learn and simply more tech for contractors to master than ever before, write Harvey Nash Recruitment Solutions’ Michael Tyrell, a Data & BI Specialist, Joshua Hyde, a Senior Consultant and Colin Morley, Professional Services Director.

Can you tell a story to the C-Suite?

One area in which organisations are increasingly looking for Data Skills is the Data Visualisation and Dashboard Development space. Thanks to Power BI, QlikView and Tableau (the leading vendors), forward-thinking organisations are looking to use data to tell a story to the ‘C-Suite,’ empowering people with ‘business knowledge’ to use Analytics to spot trends, opportunities and threats before their competitors can react.

For an organisation to operate efficiently and successfully, senior members of staff must have evidence-based information to ensure the decisions they are making are the correct ones. So ‘gut-feel’ CEO’s will quickly lose out if their rivals are deciding where to place their next factory using 1,000 different variables from 200 different sources.  

But what about those feeding the C-Suite? Well, as well as mastering new tools and systems, freelance techies are increasingly having to be more commercially-minded and are expected to interact with a wide range of professionals. The days of developers in the basement with their headphones on are long gone!

Under the microscope:The Data Scientist

Data Scientists are a prime example of this. The role is incredibly varied, with Data Scientists needing business-facing ‘soft skills,’ commercial acumen as well as technical coding and mathematical/statistical skills. This blend is becoming equally as important as it is rare, as companies deploy Data Science projects to help them understand customers, processes and a whole host of other agendas.

Generally-speaking, there is actually an adequate supply of contractors to meet the current demand, mostly because Data Scientists tend to leave permanent work for a lucrative, buoyant contract career early on compared to most other Data-related roles. In fact, right now, we’ve got people with anything from two-six years’ experience leaving the permanent market behind in search of anything from £400 to £800 a day as a contractor.

But this in itself creates a wider problem in the market. As demand for permanent Data Scientists far outstrips supply, and the contractor base grows year-on-year, Data Science and its related disciplines such as Machine Learning (ML) and Applied Artificial Intelligence become costly and lengthy projects. Often companies are paying big money for a highly-qualified Data Scientist, only for them to spend the majority of their days on basic tasks such as time cleaning and managing data, as opposed to adding true value.  

Buzzwords Vs Reality

It is absolutely vital that a company leaning towards advanced Data Science, ML or AI has the appropriate Data Governance, Architecture, Platforms – and sheer quality -- for any projects to be successful. Hence the emergence of the Data Management, Big Data and Data Engineering spaces. These buzzwords of the last decade continue to be by far the most competitive for end-users trying to secure both permanent and contract practitioners.

The proof? Well, Big Data & Analytics have been top of the skills shortage list as reported by IT leaders we polled for the past four years, with 48% recently admitting to difficulty sourcing talent in this space. Therefore, rates for engineers have sky-rocketed and a ‘war on talent’ is still the reality. Organisations are often advised to be innovative in their approach to the shortage; creating, automating processes or running hackathon style events, for example.

Despite these often-sounded alerts, the reality on the ground is a reliance on supplemental resource, given that more than 85% of IT leaders use contractors to plug skills gaps in their teams (also according to our 2018 findings). 

Artificial Intelligence: where it's at for contractors in mid-2018

So if companies are still grappling with their data management, where does that leave 2018’s favourite tech buzzword, ‘AI’? It is worth noting that there is some crossover between Data Engineering, Science and AI. And there’s a lot of confusion as to who is actually doing ‘true’ AI.

Artificial Intelligence is typically categorised in two parts, General AI and Applied AI. General AI is dominated by the ‘big tech’ companies and also some cutting-edge start-ups. Applied AI is more likely to be deployed in UK organisations as a specialised platform to create a more customised user-experience (think Amazon); to improve fraud detection in Financial Services or perhaps, to improve trading decisions. It has a specific purpose, and that is where the growth will be in the UK -- for both AI and ML Engineers -- for the next decade.

The reality is that large swathes of the UK, particularly outside of London and outside of FS, are behind-the-curve, by creating small, specialised teams testing out Proof of Concepts, specific applications or simply not really investing anything significant yet. AI didn’t even make the top 10 issues that the IT leaders we polled are pursuing in 2018.

The majority of businesses are still at the point of becoming ‘data-driven’ as opposed to ‘AI-driven’ and as such the demand for thousands of AI engineers has yet to be realised in the same way that Data Engineering and Data Science has.

That is not to say there isn’t a skills shortage, because there is. But the volume of opportunities and therefore the volume of contractors required is far below that of other Data and Software engineering-orientated disciplines at this point. When will that change? Perhaps only Artificial Intelligence knows the answer because, for now, it’s beyond us mere mortals to know.