- Zhang, T., Schoene, A.M., Ji, S. et al. Natural language processing applied to mental illness detection: a narrative review. npj Digit. Med. 5, 46 (2022). https://doi.org/10.1038/s41746-022-00589-7
A new artificial intelligence model uses natural language processing and advanced neural networks to analyse an oncologist's notes during a patient's first visit. Although these visits typically focus on a patient's medical history and treatment options, artificial intelligence picks up subtle clues hidden in medical language that suggest a patient might benefit from future or immediate psychological interventions.
According to results, recently published in Communications Medicine, artificial intelligence can predict with more than 70% accuracy whether a patient will see a psychologist or psychiatrist within a year. This is the first time artificial intelligence has been used to predict psychological health outcomes from medical records that are not strictly psychological/psychiatric in nature.
"Fighting cancer can be a harrowing experience, affecting not only our bodies, but also our minds and emotions," said lead researcher Dr John-Jose Nunez, psychiatrist and clinical researcher at the UBC Mood Disorders Centre and BC Cancer. "These results demonstrate the enormous potential of AI in essentially acting as a personal assistant to oncologists, improving patient care, helping them identify mental health needs more quickly and ensuring that more patients receive the support they need," he added.
Mental health has been shown to have a profound impact on treatment outcomes and quality of life for people with cancer. On average, patients with significant depression and anxiety have a worse survival. Experts believe this is because these patients have more difficulty following treatment recommendations and tolerating side effects.
The Canadian Association of Psychosocial Oncology estimates that about 15% of cancer patients need psychiatric services, while another 45% would benefit from meeting with a psychologist. However, barriers such as social stigma, lack of knowledge of services, and difficulties in diagnosing mental health conditions may prevent access to psychosocial care.
As a next step, Dr Nunez hopes to collaborate with oncologists and patients to implement an artificial intelligence system to improve early access to these important services.
The AI was developed by an interdisciplinary team with expertise in computer science, medical oncology and psychiatry. The researchers trained and tested the model using data from 59,800 patients from all six BC Cancer sites located in geographically diverse regions of British Columbia.
To protect privacy, patient data remained securely stored at BC Cancer and were submitted anonymously. Unlike medical record reviews by human research assistants, the new AI approach has the advantage of maintaining complete confidentiality of patient records.
The researchers also made significant progress in interpreting how the AI works, developing a new technique that allowed them to study the model from the inside to see how it made predictions. Common themes emerged, with certain key words and topics suggesting a greater need for psychological services: family history of cancer, alcohol or substance use, certain types of aggressive cancer, particular treatment strategies.
The team plans to extend the use of the system beyond the field of oncology to other areas of medicine where psychosocial factors have a significant impact on patients' health. This could pave the way for a broader application of AI in healthcare, targeting early intervention in various medical disciplines.