Unveiling the Neely Ethics & Technology Indices

We are excited to share the results from the first wave of the Neely Social Media Index and the Neely Artificial Intelligence Index, as well as preview the forthcoming Neely Mixed Reality Index.

Three powerful technologies – social media platforms, artificial intelligence tools, and mixed reality tech (AR/VR) – are influencing present-day psychology and shaping the future of our society. There are many competing ideas about how to best design, use, and regulate these technologies, but unless proposals are informed by reliable, publicly shared data generated from the very users who are affected by the technologies in question, leaders run the risk of making the wrong decisions.

To improve societal understanding and decision making about social media, AI, and mixed reality, we are excited to introduce a brand new set of public longitudinal, nationally representative panels of user experiences in America: the Neely Ethics & Technology Indices. There are three separate indices which consist of monthly surveys following a panel of U.S. adults, measuring their positive and negative experiences with social media platforms, AI, and mixed reality. Each month, we plan to release the results to the public and share the data with independent researchers. While today’s post focuses primarily on the Neely Social Media Index, we will also share initial findings from the first wave of the Neely Artificial Intelligence Index and preview the soon-to-be-launched Neely Mixed Reality Index. These tools are aimed at informing and improving the design, use, and policy surrounding these emerging technologies.

This ambitious project is an outflow of our ongoing work at the USC Marshall’s Neely Center for Ethical Leadership and Decision Making and the Psychology of Technology Institute. We are grateful for funding support from USC Marshall as well as the John S. and James L. Knight Foundation. We (Nate Fast and Juliana Schroeder, the faculty co-directors) have an outstanding team at the Neely Center that is helping to make this happen, including Ravi Iyer, Managing Director, Parama Sigurdsen, Research Program Manager, Matt Motyl, Senior Advisor and Lead Analyst, and Joo-Wha Hong, Postdoctoral Scholar.

Our purpose in tracking users’ experiences across influential technologies is four-fold:

1. By disseminating key insights to leaders, policymakers, and regulators we will inform and improve policy decision making.

2. By publicly posting the monthly results and comparing them across various products and platforms, we will help keep companies accountable for the experiences that they create.

3. By sharing our findings with the public, we will help consumers regulate their own use. 

4. By providing researchers and technologists with high-quality data they can use for their own analyses, we will advance the broader study of technology’s impact on society.

For an overview of our methodology, and – most exciting – hot-off-the-press results from our first wave of data collection from the Neely Social Media Index, please read on!

---

Introducing the Neely Social Media Index

Much has happened recently in the social media landscape, including the acquisition of Twitter by business magnate Elon Musk, the unprecedented prohibition of TikTok by Montana's Governor Greg Gianforte, the introduction of an AI bot on Snapchat, and a public advisory by the Surgeon General. These types of events bring into focus pressing questions: How can stakeholders, including users, parents, educators, and policymakers make smart decisions about social media? How can we strike a balance between encouraging meaningful social media interactions and mitigating potential harms? What might the future hold, as several new platforms compete with incumbents? 

To answer these questions, we require access to longitudinal data across multiple platforms. Toward that end, we decided to introduce a comprehensive index designed to monitor and disseminate data on U.S. social media usage trends over time. Our initiative will foster an informed understanding of our increasingly interdependent relationship with these platforms, thereby improving decision making.

Methodology: How Did We Build the Index?

The index consists of a nationally representative longitudinal panel study asking users about their experiences across social media platforms (e.g., Twitter, Instagram, Facebook, Reddit) as well as other online services (e.g., emailing, text-messaging, WhatsApp), allowing us to contextualize our findings. This allows us to compare and contrast the relationship between usage, experiences, and psychological impacts. We ask respondents about their positive and negative experiences across these online communication services, allowing us to begin to characterize similarities and differences between them. Over time, we will also be able to identify trends that could highlight emerging risks (e.g., threats to individual well-being, trust in societal institutions, free speech) that companies may want to address. 

Survey Panel. We surveyed 1,965 U.S. adults from the Understanding America Study internet panel about their experiences over the past 28 days. The researchers at USC’s Center for Economic and Social research maintain a household address based panel of roughly 12,000 respondents, from which we subsampled to get a diverse U.S. sample of respondents, with some over-sampling of younger demographics who may be more likely to use technology. This sample was then weighted to be nationally representative. You can read more about the UAS methodology here.  

Survey Questions. Our primary questions revolve around experience and usage of social platforms, for which there is no publicly available regular tracking. Previous research (e.g., New_Public’s Civic Signals work) has found that learning new things and connecting with others are primary use cases for social platforms, so we developed questions to measure those positive experiences. Negative experiences with social platforms often revolve around content that is personally upsetting and content that is perceived to be bad for the world, so we developed questions to measure those experiences. After extensive cognitive and pilot testing, we created the questions to measure these constructs that are in the accompanying graphs. We also created questions relating to how people use social media and validated surveys of the effects of social media on psychological health. Those results will be explored in follow-up posts.

Timing. The results here are from our first wave. Beginning in July, we will start collecting data daily from our sample about their experiences in the past 4 weeks, and we will launch a website where interested stakeholders can track results from the trailing 14 days. In addition to tracking trends over time, this will allow us to capture immediate reactions to daily events.

Validity. The team behind the Understanding America Panel does extensive work to ensure the quality of respondent data, with each respondent being paid for their time.  The panel and the methodologies have been used in dozens of peer reviewed studies as well as in previous presidential polling, with results that have historically matched other research and election outcomes.  

Results: What Does the First Wave of Data Collection Tell Us?

Usage of Online Services

The most widely used online services among adults in the United States are not specific to any given platform, with email usage at 82.7% and text messaging at 79.4%. Following those, the most widely used social media platforms are YouTube (66.5%) and Facebook (66.1%), which retain their large market share advantage over their peers. A tier below them are Instagram (37.5%), the iOS-specific video-chatting Facetime (30.4%), and TikTok (26.7%). TikTok is a newcomer, which hasn’t been included in most prior publicly-available surveys estimating social media habits. Additionally, we almost certainly underestimate TikTok use in our survey because we only surveyed adults, and some estimates suggest that as many as a quarter of TikTok users are under the age of 18. Another newcomer in our list is Mastodon, but fewer than 1% of our sample reported using that app. 

Regarding the frequency of use, we see that the services most likely to be used multiple times per day are again the ones that are not tied to a particular platform (e.g., 80% of U.S. adults who report using text messaging do so multiple times per day, and 67% of U.S. adults who use email report doing so multiple times per day). However, among social media companies, the frequency of use tells a different story than the percentage adoption. For example, even though YouTube was the most-adopted social media platform in our sample (with 66.5% of participants saying they used it), it is used with relatively lower frequency than many other social media platforms (with 41% of users saying they use it less often than once per day). In contrast, the majority of users on Facebook, TikTok, and Instagram (all > 50%) report using those services multiple times a day and Snapchat follows closely behind with 47%. Platforms like Twitter and Reddit are less frequently used than Facebook, TikTok, and Instagram. The least frequently used platforms (those for which the majority use is weekly, not daily) include Pinterest, LinkedIn, and Nextdoor. These discrepant frequencies of use suggest that people may be using the platforms for different purposes. 

Reported Negative Experiences

We asked respondents whether they had personally experienced something that affected them negatively on the online services that they reported using. The majority of people do not report personally experiencing something that affected them negatively. However, there are still substantial negative experiences reported across platforms. The online service with the highest percentage of its users reporting a negative personal experience is Nextdoor (26%), followed by Twitter (21%) and Facebook (19%). On the other side of the distribution, we see that LinkedIn (1.6%), Facetime (2%), and Pinterest (2.7%) have the lowest rates of users reporting experiences that negatively affected them personally. 

To help contextualize the uniqueness of these services, we can compare them to the rates of negative personal experiences on the most widely used, general online services – email (10.5%) and text messaging (8%). U.S. adults using Nextdoor, Twitter, and Facebook report between 2 and 3 times more negative personal experiences than U.S. adults using email and text messaging. By contrast, U.S. adults using LinkedIn (1.6%) report far fewer negative personal experiences than U.S. adults using email and text messaging.

Given the wide diversity in the primary audiences and typical uses of these platforms, we note that the negative personal experiences may look very different on each of these platforms. To address this possibility, we asked participants to tell us directly about their experiences. For example, to highlight the types of differences that emerged across platforms, we can report that a large percentage of the negative experiences on Nextdoor were related to crime (58%) whereas a majority of the people reporting negative experiences on Twitter cited political content (74%). We plan to write a dedicated future blog post digging into these experiences in greater detail.

We also asked respondents whether they had witnessed or experienced content that they would consider bad for the world on each of these online services. Again, the majority of U.S. adult users, regardless of service, do not report witnessing or experiencing content that is bad for the world. However, U.S. adults who use Twitter (31.3%) and Facebook (27.7%) report seeing or experiencing content that is bad for the world at a higher rate than U.S. adults using any of the other services we surveyed. This is at least 7 times greater than the rate of witnessing or experiencing content that is bad for the world in text messages, and nearly 3 times greater than the rate of witnessing or experiencing content that is bad for the world in emails. On the other side of the distribution, we see that Facetime (1.8%), LinkedIn (2%), and Pinterest (2.5%) have the lowest rates of users reporting experiences that they would consider bad for the world.

Across both the personal negative experience and the bad for the world experience questions, Facetime, LinkedIn, and Pinterest users report seeing or experiencing the least negative content relative to the other online services. On the other end of the distribution across both types of negative experiences, Facebook and Twitter users experience the most negative content. Moreover, for both of our questions about negative experiences, politics was the topic most often cited across platforms.

Reported Positive Experiences

Of course, people also have positive experiences while using online services. We examined two types of positive experiences: social connection and learning.

First, we asked people whether they had experienced a meaningful connection with others on the services that they reported using. The majority of U.S. adults did not report experiencing a meaningful connection across any of the services we asked about that they used, provoking an interesting question about whether these services are really doing what they’ve ostensibly been designed to do (i.e., to connect people to each other). The services for which the highest percentage of users reported having a meaningful connection with others were the direct messaging services – Text messaging (47.5%), Facetime (45.4%), and WhatsApp (42.9%). 

Among other services, Facebook (33.1%), Discord (32.9%), and Snapchat (23.5%) stand out as platforms where people report connecting with others in meaningful ways at higher rates than email (22%). On the other end of the distribution, we see Pinterest (4.1%) and YouTube (8.7%). This makes intuitive sense because Pinterest is a service where people design pin boards to save pictures and products to return to later, and YouTube is a service for creating and viewing videos; the focus of these applications is more about creating and enjoying content and less about directly communicating with and connecting to others. 

Twitter stands out given that it is a social media platform where people spend a lot of time engaging in back-and-forth conversations with each other, yet only 9.8% of adult users in the U.S. report having experienced a meaningful connection with others on the service. Similarly, Reddit, Nextdoor, TikTok are all services in which direct messaging and back-and-forth communication is possible if not required, yet all of these platforms showed low levels of meaningful connection (ranging from 10.1% on Reddit to 15.4% on TikTok).

Secondly, we sought to determine whether users had gained meaningful knowledge or insights from the online platforms they frequented. We asked if they had learned something useful or had their understanding of something important enhanced through these services.

YouTube topped the list with 47.9% of U.S. adults reporting having learned something significant or gaining important understanding from the platform, an indication of the high educational value people derive from this video-based service. Following was Pinterest, with 34.7% of users reporting beneficial learning experiences.

TikTok was next with 33.2% of users stating they had gained valuable knowledge or understanding from the platform. Reddit, the discussion and community-driven site, also proved to be a significant source of learning, with 30.3% of users reporting meaningful knowledge acquisition.

Email was reported by 29.1% of users as a source of important knowledge or understanding. On the lower end of the spectrum were Snapchat and Nextdoor. Despite their popularity, only 3.5% and 11.1% of users, respectively, reported learning something useful or gaining important understanding from these platforms.

Among other things, the patterns that emerged from our two positively-focused questions highlight an interesting distinction between the platforms' ability to foster social connection and their perceived educational value; many are higher in one and lower in the other. We will dig into the qualitative experiences of both sets of experiences in further posts.

What’s Next: Implications, Conclusions, Next Steps, and How to Get Involved

This is the first installment of many on the Neely Social Media Index. Stay tuned for further posts on how these experiences affect individuals, and how that might differ by identity. Please feel free to dig into the findings and develop your own insights. As an example, one pattern we found interesting was the apparent professional value of LinkedIn, which had the rare combination of providing positive experiences (23.4% reported learning valuable things on the platform, and 17.3% reported forming meaningful connections) with minimal negative experiences (only 1.6% reported being affected negatively and 2% reported viewing content that was bad for the world). 

By tracking these experiences over time as well as digging into the qualitative findings, we hope to incentivize companies to lean into their strengths for providing value to society while reducing harmful experiences. Additionally, we hope the findings will provide valuable guidance for leaders and users throughout society who seek to make smarter decisions about the use and governance of these powerful tools.

For interested readers who want to get involved further in any aspect of this project (e.g., access to data, edits to the survey, media coverage), please contact our Research Program Manager, Parama Sigurdsen, at psigurds@marshall.usc.edu.

And now, on to the initial findings from our AI index!

---

Introducing the Neely Artificial Intelligence Index

The development of generative AI marks a transformative moment in the timeline of technological advancement, introducing rapid and potentially dramatic changes to multiple domains. We are in the early stages of generative AI, but adoption is moving quickly. OpenAI’s ChatGPT had perhaps the most successful consumer application launch in history, reaching 100 million active users in two months (by comparison, TikTok took nine months and Instagram 2.5 years).

As AI continues to advance, public concerns are growing, ranging from present-day harms to existential risk. We believe that understanding the two-way relationship between AI and human psychology is paramount as we navigate this new landscape, as is learning from our experiences with social media. Humanity’s interactions with this groundbreaking technology will influence both the present and future of human experience.

These dynamic shifts raise numerous questions: How can stakeholders, such as business leaders, workers, customers, and policymakers make informed decisions about AI? How can we strike a balance between driving business innovation with AI and mitigating potential socio-economic disruptions? How can we set wise policies and avoid irreversible decisions that produce ongoing harm, suffering and inequality?

To answer these questions adequately, we will need access to longitudinal data on people’s everyday experiences with, and attitudes about, AI. In response, the Neely Center is launching an index intended to monitor and disseminate data on U.S. AI trends in society and business over time. This initiative will foster a greater understanding of our rapidly evolving relationship with these transformative technologies, informing decision-making in our journey towards a future infused with AI.

Methodology: How Did We Build the Index?

We are using the same methods as those described above for the social media index. In this case, a nationally representative longitudinal panel study will track users’ attitudes about AI, as well as their positive and negative experiences using various AI-based tools. This will allow us to better understand the relationship between usage, experiences, and downstream psychological and societal impacts. Over time, we will also be able to identify trends that could highlight emerging risks (e.g., threats to individual well-being, trust in societal institutions, free speech) that companies and policymakers will want to address.

Survey Questions. We are still developing the full AI Index but, for our initial wave of data collection, we developed questions to measure usage statistics as well as excitement and concern about the potential consequences of AI related to various domains (the military, healthcare, job opportunities for people, people’s psychological well-being, minorities and vulnerable groups, education of our youth, and people’s relationships with family and friends). This allows us to track the experiences people are having with AI as well as the implications they are most concerned and excited about.

Results: What Does the First Wave of Data Collection Tell Us?

Usage of AI Services 

We found that 16% of American adults used at least one of the AI services listed in the survey over the prior 28-day period. AI text generators led the way, with 10.1% of adults saying they had used such a product. Other uses of AI included learning (5%), image generation (4.8%), and advice (4.6%). We expect these numbers to grow over time, as generative AI becomes more central to daily tasks, such as internet search, writing emails, and creating work- or school-related documents. It is important to note that we are also interested in how people’s attitudes about AI will shift over time, apart from their reported use of these and related tools.

Next, we look at questions assessing how concerned and excited people are about the increased use of AI in daily life. Interestingly, a minority of the population reported being excited about AI, with only 30% indicating they are somewhat/very excited, while 59% indicated they were not very/not at all excited.

In contrast, a majority of people are concerned about the prospect of increased use of AI in daily life. A higher percentage reported being somewhat/very concerned (64%) relative to those who are not very/not at all concerned (30%).

Taken together, these findings indicate that the U.S. population appears to be more concerned than excited about AI. It will be interesting to monitor how these indicators change over time.

Attitudes about AI differed across the different domains we measured. Interestingly, people were most concerned (vs. excited) about psychological well-being (52% somewhat/very concerned vs. 9% somewhat/very excited), relationships with family and friends (51% somewhat/very concerned vs. 8% somewhat/very excited), and the military (50% somewhat/very concerned vs. 11% somewhat/very excited). 

People were most optimistic about the role of AI in healthcare, although they were still more concerned than excited (39% somewhat/very concerned vs. 24% somewhat/very excited).

Next, we examined excitement and concern among those who have vs. haven’t used an AI tool. We found an interesting pattern across these questions. Whereas people who have used AI were more excited about it (59% somewhat/very excited) than those who have not used AI (25% somewhat/very excited), there was no difference between these two groups related to concern. People who have used AI were equally concerned (62% somewhat/very concerned) relative to those who have not used AI (64% somewhat very concerned). 

A longitudinal approach will be helpful in better understanding the relationship between using AI and attitudes, as we can examine potential changes among people who use AI for the first time.

People who have used AI tools differed from those who have not in their concern/excitement across all seven domains we asked about, with the exception of the military. For example, with regard to healthcare, AI users were more excited (43% somewhat/very excited) than non-users (20%), and less concerned (33% somewhat/very concerned vs. 41%).

We also examined demographic differences in attitudes about AI. We will write more about these differences in future blog posts. However, one result we think is worth previewing is that concern about artificial intelligence appears to be a bi-partisan issue. On the one hand, Democrats (17%) were more likely to have used an AI system than Republicans (10%), and Democrats reported being more excited (36% somewhat/very excited) than Republicans (19%). However, concern about AI was strong among both groups, and much stronger than excitement. Democrats (69% somewhat/very concerned) and Republicans (65%) both voiced concern about the potential consequences of AI. These and many other experiences and differences across groups will be interesting to track over time.

---

Introducing the Neely Mixed Reality Index

Last but not least, the third index included in the Neely Ethics & Technology Indices is the Neely Mixed Reality Index. This index is so new that it is still being developed, and we do not have any early data to share on it yet. However, when the index is complete, we hope it will be an important way to track Americans’ attitudes about and experiences with mixed reality (augmented reality and virtual reality) technologies. The release of Apple’s Vision Pro headset in 2024 makes this a good time to begin tracking experiences in this domain. Stay tuned for updates on this and our other indices.

If you’ve made it to the end of this blog post, thank you for reading! We truly hope that the Neely Indices will provide useful data for understanding, and ultimately improving, the human-technology relationship. As always, please let us know if you have any suggestions, advice, or interest in collaborating on these projects. And if you know of a friend or collaborator who would be interested in our work, please feel free to forward this post!