On November 25, 2024, the Illinois State Legislature introduced House Bill 5918 IL HB5918, the Artificial Intelligence Systems Use in Health Insurance Act (“AI Act”). It provides the Illinois Department of Insurance (the “Department”) regulatory oversight of insurers using artificial intelligence for determinations that affect consumers. The proposed bill grants the Department the ability to adopt rules, including emergency rules per the Illinois Administrative Procedure Act, to implement and administer the AI Act.
The AI Act requires regulatory oversight of the use of AI systems and disclosure of AI utilization. The focus of the regulatory oversight of AI systems surrounds the use of AI in making or supporting adverse determinations that could affect consumers.
The AI Act enables the Department to monitor the conduct of all insurers who operate in Illinois, including insurers’ development, implementation, and use of AI-driven models in coverage determinations. Under the bill, such conduct is subject to investigation or market conduct action. The bill expands the Department’s investigative tools as they may (1) inquire further on relevant questions to specific models, AI systems, or applications of a model or AI system; and (2) also make requests for information and documentation of AI systems governance, risk management, AI use protocols, information and documentation relating to the insurer's preacquisition and pre-utilization diligence, monitoring, auditing of data or AI systems developed by a third party, and information and documentation relating to implementation and compliance with the insurer's AI systems program.
For insurers operating in Illinois, the AI Act prevents insurers from issuing an adverse consumer outcome on the denial, reduction, or termination of benefits or insurance plans where the insurers solely use an AI system or predictive model. Insurers must meaningfully review such decisions when the use of an AT system or predictive model is involved.
To enhance transparency and promote consumer trust, the Department has the ability to adopt rules that may require the insurer to disclose its AI use. While the AI Act itself does not go into further detail on such rules, the Department has the authority to set rules that require full and fair disclosure on the manner and content of the insurers’ AI use.
Insurers authorized to conduct business in Illinois must comply with the AI Act. Any decision that impacts the consumer that involves advanced analytical and computational technologies (including AI systems and machine learning), requires the insurers to comply with applicable federal and State laws, regulations, and rules, including both insurance laws and laws addressing unfair trade practices and unfair discrimination.
While the AI Act may be a proposed bill, the Centers for Medicare & Medicaid Services (“CMS”) took a stance on AI review of denial, reduction or termination of insurance plans when it issued a Final Rule in April 2023 that became effective in January 2024. CMS’ Final Rule made clear that Medicare Advantage plans must make medical necessity determinations based on the enrollee’s specific circumstances instead of relying on algorithms. Furthermore, such determinations must be reviewed by a physician or appropriate healthcare professional.
As more laws and regulations enter the insurance AI space, insurers will need to take a closer look at their compliance programs and policies and adjust their operations. There are numerous risks when using AI and its algorithms, and the growing use of AI in the healthcare and insurance industries has raised significant concerns about the opacity of the AI-driven decisions. Moreover, the increased reliance on AI models for claims adjudication, coverage determinations, and pricing can lead to situations where a consumer is denied coverage or benefits based on factors that are unclear or difficult to understand.
As public scrutiny of health insurers is on the rise, the likelihood of AI-focused litigation will also increase. In an ever-evolving and dynamic legal landscape of AI in insurance regulation, it is important for insurers to have their legal teams assist with the review, maintenance, and upkeep of their AI policies. For more information on how your organization can improve its AI compliance or questions regarding the AI Act, please contact a member of the Sheppard Mullin Healthcare Team for assistance.