In every generation, there are technologies that are highly innovative and disruptive. Consider, for example, the impact of digital cameras on film photography, how the advent of music and video streaming services upended the entertainment industry, and how smartphones mean we no longer need to carry around separate devices for communicating, navigating, recording memories and entertainment. In each case, new technologies changed entire industries, displaced existing solutions, and fundamentally changed how people and businesses operate.

We appear to be witnessing the dawn of another such disruptor, namely artificial intelligence (AI). Many companies are now exploring the use of AI to enhance their operations and services. One high profile example is the recent explosion of Large Language Models (LLMs) such as ChatGPT into the public domain, changing the way both organisations and individuals interact with data. AI is fast becoming ubiquitous with a vast range of uses. Whether detecting and recognising information in images, analysing and recommending courses of action, or even generating entirely new content, AI is having a transformative impact across sectors and society.

AI could drive significant innovation in the energy sector, improving safety and efficiency, for instance helping to optimise fuel usage and load balancing, limiting downtime through predictive maintenance, and assisting safety specialists to carry out hazard risk assessments. The energy sector also has a large part to play in enabling AI innovations by powering the data centres and computational demands in a sustainable manner.

The Energy Institute’s Human and Organisational Factors Committee (HOFCOM) aims to help the energy and allied industries by commissioning studies, providing information and sharing knowledge. Its work focuses on the impact of human and organisational factors on the management of major accident hazards. Most of its literature is freely available via the Energy Institute’s publications website.

Part of this body of work is the Guidance on human and organisational factors aspects of implementing new technologies, originally published in 2017. This was developed with the introduction of drone technology and the spread of mobile technology in mind. Eight years later, it has been revised to address AI and automation.

So, what are human and organisational factors and why should we care about them? They are best described as the ways in which people, systems and organisations interact to influence safety, efficiency and performance in the workplace. Considering human and organisational factors is critical in high-risk industries such as energy, healthcare, aviation and transportation, where human error and systemic failures can have serious consequences. The appropriate application of the disciplines can help ease the introduction of new technology into an organisation or workplace. Human, organisational and wider safety impacts are central to efforts to govern and regulate AI whilst promoting responsible innovation. 

The new guidance was developed from a literature review of AI and automation. It identifies 10 challenges and suggests appropriate mitigations.

Safety. Safety challenges can arise from human error and from operating under conditions where automation has failed. Highly automated or AI-enabled systems can put the human in the position of ‘monitoring’ rather than ‘doing’. In such circumstances, it is often difficult for the human operator to identify if the system has made a poor decision or to intervene in a timely and effective manner when manual take-over is required.

Human factors integration. When developing automated/AI-based systems, early consideration needs to be made of how to optimally allocate functions between people and automation, combining their strengths to maximise overall system performance and safety. The design of the human-machine interface can play an important role in helping the human understand how well the system decision making is behaving. The human-machine interface (HMI) should communicate in a clear and intuitive way, allowing for transparency of how the system is working, and giving users the ability to query AI model outputs.

Teaming. Integrating advanced AI capabilities into workflows can result in AI effectively becoming a member of a team, leading to challenges related to team composition (who is doing what). The resulting human-machine team needs to be able to perform safely and effectively. This leads to challenges to do with where authority, control and responsibility lies within the system. It needs to be explicitly clear what the AI is for, what it is allowed/forbidden to exercise control over, and who holds ultimate oversight and responsibility for the consequences of actions carried out by the system.

Ethics. AI algorithms can lead to biased outcomes. For example, an AI algorithm in the US used financial costs as a proxy for patient healthcare demands, prioritising those with higher costs. Due to historically greater healthcare spending on white patients, the algorithm’s data was skewed, leading to racial bias that favoured white patients over black patients (see also article in Hopkins Bloomberg Public Health magazine). The performance of AI is often dependent on the quality of its training data. Care must be taken on the source and type of information that is used to ‘train’ AI to ensure that similar or other biases are not created in the use of AI in the energy industry.

Skills. Any new technology brings with it challenges to do with human training and skills retention. New automation and AI technology requires comprehensive human training to promote safe and appropriate usage, and to encourage development of accurate mental models. Furthermore, automating operator tasks carry risks of de-skilling which need to be addressed with a regularised training regime.

Shared situation awareness. AI systems should be designed so as to provide transparency, explainability and predictability in their decision making. Some AI systems are ‘black boxes’, meaning the way they work is difficult to interpret. Equally, their behaviour can be difficult to predict because of the way they use training data to learn and make decisions. An example is the Tesla car Autopilot system. When engaged, the system can sometimes make unexpected decisions, such as sudden braking (phantom braking), failing to recognise lane markings correctly, or hesitating at junctions (see also Reuters article). This unpredictability can leave drivers asking ‘What is it doing now?’ as they struggle to understand the car’s behaviour and whether they need to intervene. It is therefore important to address these challenges during design and through the training of users to ensure it is clear how an AI system has arrived at a particular conclusion and that AI decisions can be adequately checked.

Reliability. People can develop an over-reliance on automation or conversely learn to wrongly distrust a trustworthy system. Trust and confidence challenges can develop rapidly if the system does not deliver on its promises. However, where systems are generally reliable, over-reliance can also be problematic if people become over-trusting and complacent, and therefore unwilling or unable to be critical and intervene when required to do so.

Workload. Automation is generally considered a classic way to reduce human workload. However, the reality is often that the opposite can occur. Machines can outstrip humans in pace and speed, potentially swamping operators with data and information. The design of the system should allow users to understand how the system is operating and check the accuracy of system outputs. However, the system should be designed so that the pace of system outputs does not outpace the ability of human operators to monitor it. Equally, the system should not underload the human. In other words, workload needs to be managed.

Organisational transition. Challenges during the transition to a new automated or AI-enabled system can affect organisational requirements for existing human roles and responsibilities. The result may lead to job redesign. New roles may also be required for people skilled in operating or maintaining AI and automated systems. Such change requires careful planning to avoid reduced productivity, while the need for organisational adaptation may also require time and effort to ease user acceptance and concerns.

Workforce concerns. Job and responsibility changes are often not welcomed by the workforce, and concerns around job losses are likely whenever automation is mentioned (this relates to the general subject of employee well-being). Other concerns can coalesce around fears of reduced job satisfaction. Such staff resistance can occur if there is inadequate user engagement before and during the system design process. To help, change management techniques may be utilised that inform and involve the workforce in the development and introduction of the new system in order to influence the solution and help alleviate their concerns.

AI and automation present both immense opportunities and significant challenges for industries worldwide. The Energy Institute’s initiative to update its guidance on human and organisational factors reflects a proactive approach to ensuring that AI-driven advancements enhance safety, efficiency and performance, rather than introduce new risks.

By addressing key concerns such as trust, skills, workload and workforce adaptation, the updated framework, which has now been published, will provide invaluable support for energy businesses navigating this evolving landscape.

Going forward, organisations must remain vigilant, integrating human factors considerations to ensure that AI serves as a tool for progress rather than disruption.

• Further reading: ‘How to deal with hazards in process operations more effectively’. Despite the widespread use of risk assessments and hazard identification, accidents continue to occur in the energy industry and elsewhere. Perhaps more attention should be given to human factors in process safety, and particularly the conditions that raise the likelihood of errors. The co-author of new Energy Institute guidelines on the subject, Dr Marcin Nazaruk, founder and CEO of Psychology Applied, explains.
• As the energy industry undergoes rapid transformations on its journey to net zero, the importance of robust process safety measures cannot be overstated. Gaynor Woodford, TÜV Rheinland Sustainability Lead, and Laszlo Komaromi AMEI, Energy Institute (EI) Senior Technical Officer, share insights from the November 2023 TÜV Rheinland Process Safety Management Workshop and Conference, in which the EI actively participated.