8 min read

Why and how I built the world’s largest study of beauty in science

Why and how I built the world’s largest study of beauty in science

I’ve received a few questions about why and how I set out to study beauty in science. So here’s the story and a behind-the-scenes look into what went into this project.

The seeds for this study were planted in the late 1990s. When I was in 11th grade in Dubai, I had a physics teacher who was widely considered an eccentric genius. He had shocking white shoulder-length hair and was obsessed with physics. He had a knack for turning anything into a problem set. Visit an amusement park and he’d make you calculate velocities of projectiles thrown from Ferris wheels. We watched the blockbuster Speed, and he made us calculate various scenarios in which the hero could defuse the bomb on the terrorist-rigged bus in time. Our teacher found in physics and mathematics the kind of beauty that the guys in my class found in, say, Mariah Carey. I didn’t share his passion for science, but I wished I could have been moved by science in the same way, even to taste what it might be like to understand how everything worked, and to find something you considered fascinating and beautiful everywhere you looked.

Years later, I became a sociologist to understand how people across cultures found meaning in their work. In one of my projects, I studied scientists in eight countries, trying to understand their views on religion and ethics and family. In several interviews, we heard scientists talk about the sacrifices they made for the sake of their work: long hours in the lab, forsaking careers that were more lucrative, even sacrificing their families. When I asked them what about science made these sacrifices worthwhile, I was struck by the answer many of them gave me: “Because it is beautiful.”

As one Italian astronomer told us:

I mean, this is the way I always lived my life, trying to understand things... Again, you know especially in my field, we don’t do this job for money. You can imagine there is not much money involved... One of the reasons is because people find it beautiful, and especially me.

We then began asking scientists a single question at the end of the interview, on where they encountered beauty in their work. Many of their answers reminded me of my old physics teacher: their eyes lit up as they shared detailed accounts of what made science attractive to them and how it inspired awe and wonder. Once we completed the study, the data we amassed in responses to this question clearly merited further inquiry.

Templeton Religion Trust found the topic compelling enough to fund an exploration of whether it was possible to empirically measure scientists’ opinions and experiences of beauty. Many prominent scientists such as Newton, Einstein, Feynman, and others had waxed eloquent about the role of beauty in science. But nobody had empirically measured the extent to which such encounters with beauty were prevalent in the day-to-day work of ordinary scientists.

To keep the study manageable, I decided to limit it to four countries from the first study: India, Italy, UK, and USA, where I had established research networks. Similarly, I decided to limit the disciplines examined to physics and biology, the two fields that we previously examined, which are two core scientific disciplines with distinct aesthetic sensibilities.

I assembled an advisory board from a variety of fields: physics, biology, sociology, psychology, and science education to help us think through relevant issues. We conducted 65 pilot interviews with scientists and philosophers of science. We read a lot of literature on the topic of beauty in science, and iterated several versions of a survey questionnaire on the topic, based on the literature and interviews. I also organized an international symposium which further fleshed out some of the key project themes, captured in the infographic below:

Infographic designed by Grey Matter Group

Having established that the study was indeed feasible to conduct, I applied for funding to conduct the full study, a three-year effort that would include nationally representative surveys with physicists and biologists in the four countries, as well as follow-up interviews. Luckily the funder and reviewers found this worthwhile, and we began the next phase of work.

Surveying scientists

We hired the global survey firm Abt Associates to conduct the survey. Executing the survey proved to be an extraordinary challenge. We wanted nationally representative surveys so that we could make claims such as “X% of American physicists find beauty in symmetry” or “Y% of UK biologists think that it is not important for scientists to encounter beauty in their work.” Simply having a large sample wouldn’t do, since we wouldn’t know how the scientists who responded to our survey would differ from those who didn’t.

To get such a sample, the first task was to develop a sampling frame, or a list of the population to which we wanted to generalize, and from which we could extract a scientifically representative sample. Since there’s no such list readily available, we had to construct one. We restricted ourselves to PhD-granting institutions and research institutes in the four countries, since these are the places where professional scientists complete their training, and are also most likely to consistently contain information on their websites from which we could generate our email lists. Since the list of such institutions in the US is especially large, Abt ended up segmenting institutions by size and taking a stratified sample of institutions rather than sampling from all institutions. We then needed to identify “Physics” and “Biology” departments, which proved to be another challenge. In some cases, the departments can be clearly identified (e.g., Physical Sciences, Biological Sciences, etc.), but in other cases (e.g., interdisciplinary departments such as Biophysics, Earth Sciences, Natural Sciences, etc.) it became more complicated. Eventually, however, we managed to define a reasonable scope of inquiry with clear criteria for inclusion and exclusion, and developed a sampling frame of 22,840 scientists at 233 universities and research institutes. Abt developed sampling weights which allow us to then appropriately weight survey participants with respect to sampling frame. For more details on our methodology, see here.

Another factor that was especially important was branding. We needed to ensure that we didn’t bias the study towards only those scientists who cared about beauty or aesthetics. Hence, we decided to call it a study of “Work and Well-Being in Science.” This reflected our genuine interest in understanding the factors that shape scientific work and the well-being of scientists; while the main such factors we were interested in were aesthetic ones, we also wanted to ensure we avoided bias and measured other relevant factors that shaped these outcomes.

Problems problems problems

Nobody anticipated that the start of the project would coincide with a global pandemic. That certainly complicated things: lockdowns, quarantines, bans on international travel, and the mental health toll of the pandemic would be serious issues to contend with. The pandemic delayed the start of our study, and made it impossible for us to conduct research interviews in person. It also unevenly affected the different countries in our study. By the time we launched our full survey (May 2021), Italy, which had been devastated by the pandemic a year earlier, had almost returned to normal, whereas India was being seriously ravaged by the pandemic. We can see in our data the significant country-level differences in scientists’ well-being, which are certainly affected to some degree by the severity of the pandemic at the time.

Differences between countries in well-being scores (reduced version of Harvard Flourishing Index). Source: Work and Well-Being in Science (2021)

Getting scientists to take the survey was no easy task. I don’t know about you, but I personally don’t like taking surveys myself, even though I rely on them for a living. In the context of the pandemic, people were also inundated by more online work than ever before, so it’s understandable that they wouldn’t want to spend 20-25 minutes on some stranger’s survey.

Legitimacy was another serious challenge: scientists and academics generally get a lot of scam invitations to publish articles in fake journals, present at conferences for a hefty fee, and so on. I delete several such emails every week. So we had a challenge convincing scientists that ours was a legitimate study. Some who took the survey said they initially had misgivings. They had never heard of “Catholic University of America,” my home institution; they had never heard of the survey firm (“Abt Associates” sounds fake and scammy); our website (designed by a professional marketing firm) didn’t look convincing enough, etc. I have a Harvard affiliation that I had tried using on my title early on in the study, but since Harvard’s Institutional Review Board was not the one reviewing the study, they would not grant me permission to use the title. So I think we were consistently hampered by a legitimacy problem. In some cases, we learned that institutions blocked our survey entirely, thinking that it was a security threat.

We tried several strategies to overcome these problems and to improve participation:

1.    Survey invitations included an endorsement from our advisory board. We evenutally expanded our advisory board significantly to 27 members, including many prominent scientists from the different countries

2.    In addition to an electronic gift card and a summary of results, we offered participants a personalized report of the study results

3.    We created an animated video invitation to the project with a message that I recorded, so they could connect a face and voice to the invitation

4.    We placed two ads for our study in the journal Nature, along with a Letter to the Editor co-signed by several board members emphasizing the importance of the study for understanding well-being in science

5.    We emailed department chairs in all the institutions in our sample, requesting them to encourage their members to participate in the study

6.    We raffled away many iPads

7.    We also asked for suggestions from those who took the survey for how we might improve the response. We followed many of their ideas such as changing the wording of subject lines and some of the text (e.g., changing “the survey takes an average of 20 minutes to complete” to “some scientists have completed the survey in as little as 10 minutes”)

Eventually, after all these efforts, we achieved 3442 completed surveys (a response rate of 15%). It’s a good number of responses, but I’m not gonna lie; that response rate was disappointing. Still, while the rate was a lot lower than what I had expected, it’s important to keep in mind that we routinely rely on information from studies that have response rates lower than 10% (e.g., Pew Research Center polls). We also have the benefit of survey weights that allow corrections for nonresponse. I’m not sure what else we could have done to improve participation, even in hindsight; if you have ideas, by the way, please let me know in the comments below.

Beyond the survey

Following the survey, we conducted interviews with a subset of survey participants who indicated willingness to be interviewed. We also included additional participants who were not part of the survey (just to avoid any bias created from having taken the survey), as well as a group of interviewees who had left academic science careers. The 215 interviews we conducted, thankfully, were easier to implement than the survey. No angry emails, for instance. And even though we didn’t have the benefit of visiting scientists in person, the world had gained considerable facility in using Zoom thanks to the pandemic, so it was fairly straightforward to interview scientists online.

Getting the right team in place was crucial for the success of the interviews. We were fortunate to have 14 excellent researchers who conducted interviews for us in the different countries.

At this point, we’ve finally completed data collection and are in the thick of analysis. And we’ve got many events planned to share the results of our work, in places like Washington DC, New York City, Los Angeles, Cambridge, Oxford, Milan, Rimini, and Bangalore.

All of this would have been impossible without the help of an exceptional team of project managers, research assistants, collaborators, and advisors, and of course, our sponsor that funded the study. Visit our project website to learn more about the study and our results.


I'd love to hear from you about anything that you found interesting in this post, or if there are topics and questions you'd like me to address. If you found this post valuable, please share it. Also please consider supporting this project as a paid subscriber to support the costs associated with this work. You'll receive early access to content and exclusive members-only posts.

Comments

Sign in or become a Beauty at Work member to read and leave comments.
Just enter your email below to get a log in link.

Comments

Sign in or become a Beauty at Work member to read and leave comments.
Just enter your email below to get a log in link.