Software used to screen social media photos for depr...

“The images you put up on Instagram could be used to diagnose if you're depressed,” the Mail Online reports.

Researchers attempted to see if computer-driven image recognition could and diagnose depression based on the form and content of people’s posts on Instagram, a social media photo sharing site.

They looked at more than 43,000 images from 166 people, who also completed a survey about their mood. The researchers found people who reported having a history of depression were more likely to post images that were bluer, darker and less vibrant.

The computer programme was able to correctly identify 70% of the participants with depression, getting it wrong 24% of the time. These results were compared with a separate independent study, which estimated that GPs only correctly diagnose 42% of cases.

This is a proof of concept study into what is often referred to as “machine learning”. Machine learning involves using sophisticated algorithms that assess massive amounts of data to see if they can begin to spot patterns in the data that humans can’t.

The researchers suggest social media could become a useful screening tool. But aside from whether the science stacks up, there are ethical and legal implications that would need to be considered before this could happen.

If you’ve been feeling persistently down and hopeless in the last few weeks and no longer take pleasure in things you used to enjoy, you may be depressed. Contact your GP for advice.

Where did the story come from?

The study was carried out by researchers from Harvard University and the University of Vermont, and was funded by the National Science Foundation and the Sackler Scholars Programme in Psychobiology.

It was published in the peer-reviewed journal EPJ Data Science.

There was wide coverage of the story in the media, which was generally accurate – but none highlighted any of the study’s limitations.

The media also failed to point out that although the researchers say their 70% detection rate is better than GPs, the GP detection rate was taken from a study that looked at GPs making a depression diagnosis without using any standard assessments. This means we’re unable to verify the accuracy of this figure.

What kind of research was this?

This case-control study compared the Instagram posts of people who reported a history of depression with the posts of those who did not.

Although this is an interesting concept, this type of study isn’t able to prove cause and effect. For example, we don’t know whether the individual preferences for colour, mood or genre had changed over time in either group – more people in the depression group may have happened to always prefer the colour blue, for example.

What did the research involve?

The researchers recruited 166 adults aged between 19 and 55 using Amazon’s Mechanical Turk (MTurk) crowdwork platform. This is an online service where participants receive small rewards for taking part in regular surveys or similar tasks.

They completed an online survey about any history of depression and agreed to let researchers have access to their Instagram posts for computer analysis. A total of 43,950 photos were compared for 71 people with a history of depression and 95 healthy controls.

The researchers chose to measure differences in the following features of Instagram posts:

hue – colour on the spectrum from red (lower hue) to blue/violet (higher hue)
brightness – darker or lighter
vividness – low saturation appears faded, while high saturation is more intense or rich
use of filters to change the colour and tint
presence and number of human faces in each post
number of comments and likes
frequency of posts

They then compared these features between the two groups and ran various computer programmes to see if they could predict who had depression based on 100 of their Instagram posts.

They compared their predictions with those made by GPs using data from a previous independent meta-analysis, which found that without using any validated questionnaires or measurements, GPs are able to correctly diagnose 42% of people with depression.

The Center for Epidemiologic Studies Depression Scale (CES-D) questionnaire was used as a screening tool for depression. This uses a scale of 0-60 – it’s generally considered that a score of 16 or more indicates a likely diagnosis of depression. People with a score of 22 or more were excluded from this study.

To see if humans are able to identify factors that computers cannot, the researchers also asked a sample of online users to each rate 20 randomly selected photographs on a scale of 0-5 on the following measurements:

happiness
sadness
interest
likeability

In all, 13,184 images were rated, with each image being rated by at least three people.

What were the basic results?

The computer programme identified 70% of the people with depression. It incorrectly identified 24% of people as having depression who did not. The results were much less accurate for predicting depression before it had been diagnosed.

According to the computer-generated results, people in the depressed group were more likely to post:

photos that were bluer, darker and less vibrant
photos that generated more comments but fewer likes
more photos
photos with faces
photos without using filters

If they did use filters, they were more likely to use “inkwell”, which converts photos to black and white, whereas the healthy controls were more likely to use “valencia”, which brightens images.

The human responses to the photos found people who were in the depression group were more likely to post sadder and less happy images. Whether the images were likeable or interesting didn’t differ between the groups.

How did the researchers interpret the results?

The researchers concluded: “These findings support the notion that major changes in individual psychology are transmitted in social media use, and can be identified via computational methods.”

They say this early analysis could inform “mental health screening in an increasingly digitalised society”. They acknowledge that further work on the ethical and data privacy aspects would be required.

Conclusion

This study suggests that a computer algorithm could be used to help screen for depression more accurately than GPs using Instagram images.

But there are several limitations that need to be considered when analysing the results:

As only people with a CES-D score of between 16 and 22 (on a scale of 0-60) were included, this is likely to have ruled out those with moderate to severe depression.
There were a small number of participants.
Selection bias will have skewed the results – it only includes people who like to use Instagram and are willing to allow researchers access to all of their posts. Many potential participants refused to take further part in the research once they realised they would have to share their posts.
It relied on self-reporting of depression rather than formal diagnoses.
The data is all from US participants, so may not be generalisable to the UK.
The 100 posts from people with depression were analysed if they were within a year of the diagnosis. As we don’t know how long people may have had symptoms for before diagnosis and whether their symptoms had improved, it’s difficult to make any accurate conclusions.
We don’t know their lifelong preferences for colours or genre when posting images.
And, most importantly, the figure quoted that GP diagnostic accuracy was only at 42% was based on meta-analysis of studies where GPs were asked to diagnose depression without using questionnaires, scales or other measurement tools. This doesn’t give a very realistic representation of depression diagnosis in normal clinical practice. As such, it can’t be assumed that this model would be an improvement over standard methods for depression screening or diagnosis.

Though the results of this study are interesting, it’s unclear what benefits or risks may be attached to any future use of screening tools for depression using Instagram or other social media.

If you’re concerned that you’re depressed, it’s best to contact your GP – there are a variety of effective treatments available.

Read more about seeking advice about low mood and depression.