EdLaw
Companion chatbots — Bus. & Prof. Code §§22601–22606
The companion chatbot chapter added by SB 243 as it stands in the code: the definitions (companion chatbot and its exclusions, operator, companion chatbot platform), the AI-disclosure notice, the suicide and self-harm protocol, the duties owed to users the operator knows are minors, annual reporting to the Office of Suicide Prevention, the audit requirement and the private right of action.
Bus. & Prof. Code §22601
As used in this chapter:
(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
- (1) “Companion chatbot” means an artificial intelligence system with a natural language interface that provides adaptive, human-like responses to user inputs and is capable of meeting a user’s social needs, including by exhibiting anthropomorphic features and being able to sustain a relationship across multiple interactions.
- (2) “Companion chatbot” does not include any of the following:
- (A) A bot that is used only for customer service, a business’ operational purposes, productivity and analysis related to source information, internal research, or technical assistance.
- (B) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm, sexually explicit conduct, or maintain a dialogue on other topics unrelated to the video game.
- (C) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant, and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.
(c) “Companion chatbot platform” means a platform that allows a user to engage with companion chatbots.
(d) “Office” means the Office of Suicide Prevention established pursuant to Section 131300 of the Health and Safety Code.
(e) “Operator” means a person who makes a companion chatbot platform available to a user in the state.
(f) “Sexually explicit conduct” has the meaning defined in Section 2256 of Title 18 of the United States Code.
(g) “Video game” means a game played on an electronic amusement device that utilizes a computer, microprocessor, or similar electronic circuitry and its own monitor, or is designed to be used with a television set or a computer monitor, that interacts with the user of the device.
Bus. & Prof. Code §22602
(a) If a reasonable person interacting with a companion chatbot would be misled to believe that the person is interacting with a human, an operator shall issue a clear and conspicuous notification indicating that the companion chatbot is artificially generated and not human.
- (1) An operator shall prevent a companion chatbot on its companion chatbot platform from engaging with users unless the operator maintains a protocol for preventing the production of suicidal ideation, suicide, or self-harm content to the user, including, but not limited to, by providing a notification to the user that refers the user to crisis service providers, including a suicide hotline or crisis text line, if the user expresses suicidal ideation, suicide, or self-harm.
- (2) The operator shall publish details on the protocol required by this subdivision on the operator’s internet website.
- (1) Disclose to the user that the user is interacting with artificial intelligence.
- (2) Provide by default a clear and conspicuous notification to the user at least every three hours for continuing companion chatbot interactions that reminds the user to take a break and that the companion chatbot is artificially generated and not human.
- (3) Institute reasonable measures to prevent its companion chatbot from producing visual material of sexually explicit conduct or directly stating that the minor should engage in sexually explicit conduct.
Bus. & Prof. Code §22603
- (1) The number of times the operator has issued a crisis service provider referral notification pursuant to Section 22602 in the preceding calendar year.
- (2) Protocols put in place to detect, remove, and respond to instances of suicidal ideation by users.
- (3) Protocols put in place to prohibit a companion chatbot response about suicidal ideation or actions with the user.
(b) The report required by this section shall include only the information listed in subdivision (a) and shall not include any identifiers or personal information about users.
Bus. & Prof. Code §22604
An operator shall disclose to a user of its companion chatbot platform, on the application, the browser, or any other format that a user can use to access the companion chatbot platform, that companion chatbots may not be suitable for some minors.
Bus. & Prof. Code §22605
A person who suffers injury in fact as a result of a violation of this chapter may bring a civil action to recover all of the following relief:
(b) Damages in an amount equal to the greater of actual damages or one thousand dollars ($1,000) per violation.
Bus. & Prof. Code §22606
The duties, remedies, and obligations imposed by this chapter are cumulative to the duties, remedies, or obligations imposed under other law and shall not be construed to relieve an operator from any duties, remedies, or obligations imposed under any other law.