Recommendations – AI in healthcare

a) Canvas national healthcare settings to ensure that the various needs (i.e., staff, individual patient, whānau, and community) are understood. This could:

i) Highlight local, regional, and national needs to identify and prioritise the appropriate deployment of AI healthcare interventions

ii) Be utilised to inform research and development efforts

b) Ensure ongoing horizon scanning to maintain an awareness of emerging technologies in AI and healthcare and the extent to which needs in clinical settings might be addressed

i) Highlight enablers and barriers for the deployment of AI in healthcare settings (both public and private)

ii) Identify policy/legislation for review

b) Develop an understanding of various capabilities of AI technologies and develop a robust framework to support appropriate regulation. This could:

i) Distinguish AI technologies according to type and output (for example, operational efficiency improvements compared to self-learning AI and diagnostic support) and establish the extent to which regulations are required for distinct applications

ii) Ensure independent testing requirements for the evaluation of impact and safety

i) Provide clarity around research and development outputs from New Zealand that have the potential to be implemented in our healthcare industry

ii) Provide short-to-medium term clarity around future research needs for New Zealand and our research partners

iii) Provide clarity on tertiary AI courses available across institutions

iv) Support the establishment of aspirational mid-to-long term goals for healthcare delivery in New Zealand and related research and development

b) Undertake regular horizon scanning to establish an understanding of future potential areas for research & development

c) Understand enablers and barriers experienced by technology developers in the AI healthcare sector. This should:

i) Be used to inform the ongoing development of suitable legislative settings

ii) Inform discussion around support tools/services that might help to reduce complexities

i) Canvas a diverse range of voices within the community

ii) Inform governance bodies and decision makers of the healthcare desires and levels of comfort within their respective communities distinguished by application. For example, patients may be fine with an AI scheduling system but might prefer to know if AI has been used in image diagnosis

iii) Identify the factors that contribute to comfort levels

iv) Identify at what stage of receiving healthcare that patients desire to know that AI has been used

b) Understand experiences of AI technology developers around the development and deployment of AI for healthcare in New Zealand. This work should:

i) Canvas a diverse range of technology applications

ii) Inform governance bodies and decision makers of developers experiences and the extent to which New Zealand is a desirable market to partner with

c) Understand the ongoing interactions between clinicians and AI and healthcare delivery

i) Canvas a diverse range of voices within the community

ii) Inform governance bodies and decision makers of the healthcare desires and levels of comfort within their respective communities distinguished by application. For example, patients may be fine with an AI scheduling system but might prefer to know if AI has been used in image diagnosis

iii) Identify the factors that contribute to comfort levels

iv) Identify at what stage of receiving healthcare that patients desire to know that AI has been used

b) Understand experiences of AI technology developers around the development and deployment of AI for healthcare in New Zealand. This work should:

i) Canvas a diverse range of technology applications

ii) Inform governance bodies and decision makers of developers experiences and the extent to which New Zealand is a desirable market to partner with

c) Understand the ongoing interactions between clinicians and AI and healthcare delivery

b) Ensure that healthcare workforce are adequately informed to understand newly adopted guiding principles for AI in healthcare settings

c) Identify resources required for implementation of best AI practice across the health system

d) Establish and/or adopt formal evaluation processes for pre- and post-implementation of new AI health technology. Evaluation processes should:

i) Take into consideration best evaluation practice for the technology in question (if best practice has been established)

ii) Take into consideration system resourcing and the extent to which AI technologies are compatible with existing resources (for example if AI tools are more efficient at screening for breast cancers, is the system adequately resourced to cope with increased detection)

iii) Where best practice for evaluation has not been established, the technology should be limited in its application with sufficient mechanisms to prevent use on an experimental basis outside of authorised clinical settings

iv) Evaluation results can be communicated to the public (R10) to help facilitate public trust

e) Ensure regular review (annually or as needed) of principles and practices for application of AI in healthcare settings

f) Establish clear frameworks for liability and responsibility of AI when deployed in the healthcare system. This should:

i) Distinguish by application/output

ii) Distinguish by level of supervision

iii) Distinguish by level of associated risk

iv) Establish clear criteria for insurance coverage

a) Complete a gap analysis of research and development capabilities within New Zealand. This could inform the development of funding models that require and/or reward developments for supporting positive healthcare
outcomes in New Zealand (considered in conjunction with the outcomes of R3:a)

b) Consider establishing a suitable funding model to facilitate the deployment of AI healthcare research

i) Present and future potential for improved healthcare outcomes

ii) Clear communication around benefits and limitations of AI

iii) Associated risks of members of the public using AI as an alternative and/or replacement to consulting with a healthcare professional

iv) Inevitability of errors (including types of errors, rate of errors, and comparison of error rates in settings where AI is not in use)

v) National and international use
cases

b) Ensure that targeted information and training is available to AI in healthcare governance and decision-making bodies at all levels

c) Ensure transparency around evaluation and implementation processes/frameworks to provide confidence in decision-making processes

b) Understand future resourcing and capability requirements and establish pathways to build relevant skill sets

c) Monitor AI companies that indicate potential capability for AI technology to provide training of healthcare staff and/or health students

d) Consult with training providers (including universities, accreditation bodies etc.) to develop evaluation mechanisms and criteria where adoption of AI tools for training of clinical staff and/or students would be acceptable and appropriate

e) Develop an understanding of future AI training needs for health students and healthcare practitioners

b) Ensure access to technical resource for government agencies responsible for ensuring data privacy

c) Determine appropriate frameworks for establishing dynamic informed consent

i) The tool’s effectiveness across various population group

ii) The burden of disease the tool is designed to address across different population groups

b) Require an equity impact and bias assessment before launching any AI tool in the public healthcare system

c) Develop a framework for ongoing systematic evaluation of AI tools to understand the impact on health inequity (including annual reporting)and bias. This should:

i) Be flexible to assess various typesof AI

ii) Inform decision-making bodies, funding bodies, research institutions and the technology development sector

d) Develop frameworks and/or principles for AI development that highlight the need to address inequity and bias in healthcare delivery from the starting point of the development process

b) Develop a strategy to build Māori capacity including investment into workforce training, data access, data-sharing with appropriate Māori health providers, etc

b) Establish engagement forums that enable robust discussions around practical applications of the principles
of Māori data sovereignty. Discussions might include:

i) Empowering relevant iwi, hapū, whānau and Māori organisations to determine metrics of health, wellbeing and hauora for their own communities

ii) Ensuring Māori control over Māori data and considerations of potential outcomes

iii) Establishing appropriate tikanga for collecting, classifying, storing, accessing and using Māori data

iv) Appropriate mechanisms of co-design as partners to Te Tiriti

b) Develop a strategy to build Māori workforce capacity including investment into workforce training, data access, data-sharing with appropriate Māori health providers, etc

b) Identify computing requirements to enable on-shore data storage, model hosting, and technology development

c) Expand the healthcare data strategy to consider factors relevant to data collection and data use for AI. This could include:

i) The potential for individuals to opt in or opt out

ii) The mechanisms for consent and the impact of individual consent on people groups (e.g., whānau, communities)

d) Ensure robust data collection mechanisms and understand  implications of AI tools being used for populations that are underrepresented in current data sets

e) Explore mechanisms for data linking across data sets outside healthcare, being mindful of data sovereignty

i) Consider principles of Māori data sovereignty (see R15)

ii) Include guidelines for testing of AI tools using national data sets

b) Establish international research and development capabilities and develop strategic relationships

c) Specific research strategy should be defined based on (1) need within the healthcare system, (2) capacity and capability within domestic research capabilities (or in existing research partnerships), (3) likely impact of research outcomes (4) likely time to deployment and (5) ease of deployment/implementation

b) Understand from existing AI companies the various enabling technologies that facilitate enhanced AI development

c) Generate targeted information that provides advice to start-up companies attempting to deploy AI healthcare technology in New Zealand

d) Generate advice for AI companies to navigate the legislative environment

e) Generate advice for AI companies to
navigate commercialisation processes

i) Span various stakeholder groups (e.g., occupation, iwi, ethnicity, locality, research, industry, government etc)

ii) Highlight factors that are at the forefront of the public conversation, immediate concerns to be addressed and clear opportunities to capitalise on

b) Establish annual expo (or something similar). An expo should:

i) Allow those from the research and development sector to showcase current and future potential

ii) Be used to inform the healthcare profession of available emerging AI technologies

iii) Enhance public visibility of emerging technologies

c) Establish support roles and/or networks for AI businesses. Support should:

i) Provide advice to businesses about deployment of technology in the New Zealand healthcare environment

ii) Provide mechanisms to support SMEs with regulatory costs

d) Establish links with key players in the global AI ecosystem e.g., Microsoft, Amazon, etc