Christina's LIS Rant

Lunch was fabulous. Did I mention that? Basically salad-based so not too heavy, either. The day was long and coffee wasn’t allowed in the auditorium.

The first invited speaker for the afternoon got stuck in his room isolating after testing positive for COVID-19. They explicitly required in-person attendance, but I would have appreciated if he could have Zoomed in if he felt like it. (maybe he didn’t feel up to it? or maybe they couldn’t – but there was good wifi so… probably a policy thing). Disappointed because I’ve always been interested in his research. ¯\_(ツ)_/¯

Sadamor Kojaku – Mapping Science Foraging

Compared frog vs bird metaphor: work in a well, in one place vs. foraging more widely. How can we operationalize the concept of foraging in order to study it?

Trying a bag of topics approach, we can show a changing topics over time for authors. Relationships between scientists. But this approach pretends topics are discrete. Need a continuous representation.

He moved to node2vec and 64 dimensional space. It predicts PACs classifications.

Most jumps in topics are small but long jumps are not rare. Scientists don’t go back to previous topics. Longest jump can happen any time in career.

Sam Zhang – Labor Advantages Drive Greater Productivity at Elite Institutions

His web page. Relevant pre-print.

Similar scientists have different productivity based on their institution. Funded student and postdoc labor is a driver to productivity. Used the literature to classify fields by collaboration norms. 262 PhD degree-granting institutions, NSF survey counts of funded/unfunded. If you have the same address as a faculty member but aren’t on the list of faculty > group member.

More prestigious institutions have more funded students across disciplines. In fields with collaboration norms, productivity goes up with more labor.

Government Funding Panel

Daniel Correa, CEO FAS (former OSTP and other places)- Endless Frontier Act is lauded, but it’s authorization, not appropriation. The time is now for engagement. You don’t need ambitious policy from Congress to get things done. Agencies have leeway and are looking for good ideas. The challenge – we don’t know enough about who. The costs of getting funded are huge. After you complete work, need to follow up with a proposal to fix the problem. Find empowered public servants who are the connective tissues between science and government. Bring them action plans. FAS Impetus Institute – improve efficacy of federal R&D investment

Arthur Lupia – UMich, formerly gov’t. The average federal grant is 2 times the US median household income. The grant is instead of funding housing, roads, or whatever other things government has to take care of. It must return a substantial distinct public value. When you go in to meeting with government, be ready to answer the question: whey should we fund you instead of children’s cancer research? Whose quality of life will be improved. What scientific opportunities are empowered?

Dewey Murdick – CSET at Georgetown, formerly gov’t

Federal funding no longer dominant. US Federal Funding not dominant in global science. ( I wrote this down but not sure the context: https://www.sciencedirect.com/journal/technological-forecasting-and-social-change/vol/146/suppl/C#article-56)

Tara Schwetz – NIH

They have a lot of different programs in place to assess if/how NIH is achieving (? I can’t think of the word) their mission.

RCR – relative citation ratio

search analytics – word2vec cluster visualization

person disambiguation

Status of ARPA-H: transformative high risk/high reward research. Speed health breakthroughs. Data collection and assessment. Embracing evidence-based diction. Active program management. Continuous process improvement. Transparent, collaborative, and diverse. Funded by Omibus 3/22. New Deputy Director named: Adam Russell (formerly of ARLIS, IARPA, DARPA)

Clara Boothby – Turnover Replacement and Career Age Distribution

Related pre-print. This generation has a postdoc apocalypse. Attrition rates from academic settings are high. Why are universities still recruiting doctoral students when the career market is so bad? They looked at publications 1980-2019 for authors with 80% of their publications coming from a US address. Biology has a 2.8% per year growth rate. New researchers are 10% of the workforce each year and researchers leaving about the same. Turnover is higher in early career positions. Chemistry turnover may be to industry.

Kevin Kniffin Walking the Talk of Disciplinarily

His profile page. Are people more interdisciplinary? How interdisciplinary work is doesn’t depend on gender, citizenship, race. Interdisciplinary researchers get paid less and are more likely to postdoc but it does depend on field Computer Science not penalized

Xiang Zheng Parenthood contributes to gender gaps in academia

Relevant pre-print. Combined a survey of 7,764 researchers with information on their publications from WoS they found that the established gender gap in academia is primarily a parenthood gap. In their survey, women were less likely to have children and likely to have fewer. Decisions to have children are career related (60%), Women were more likely to report negative impact of children on career.

Katie Spoon Explaining gendered retention patterns in academia

Conflicting evidence of gendered differences in retention. Existing papers are about small groups or institutions or fields. She used data from Academic Analytics Research Center – 279,419.

Women were at higher risk of leaving throughout their career. Survey of people from longitudinal study: workplace culture, harassment reasons to leave. Switch institutions due to pull, leave academia due to push

Timothy Errington – Replicability of preclinical cancer biology

Article: https://elifesciences.org/articles/71601 . Reports on repeating experiments from papers. About 46% they were able to replicate. Generally smaller effect sizes than the original report.

Christian Chacua – Migration and Research Interest

profile page: https://cchacua.github.io/christianchacua/ . Studied Colombian researchers who worked overseas for some part of their publishing career. Even ones who had topics tied to geography (volcano) or ecology, changed subjects when they moved. (this was a really interesting talk but too short and also, toward the end of a really long day)

Hao Peng Gender Gap in Scholarly Self-Promotion on Social Media

pre-print Looked at 7M tweets about articles by 1.3M authors from 2018. Women less likely to self-promote. Magnitude of the gender gap has more to do with journal prestige than other factors they looked at. When women do tweet, their papers get more attention.

Louis Shekhtman Mapping Philanthropic Grants to Science

This was pretty cool and different from everyone else’s. Looked at a ton of tax returns from philanthropic organizations

Emma Zajaela In person vs Virtual Conferences

Conferences promote collaborations. Small group interactions more so. Virtual conferences still promote interactions, but usually set events and not informal mingling. pre-print here

https://twitter.com/ICSSI2022/status/1534643208440905728

In this post I’ll summarize my notes from the morning of the first day of this conference, June 7, 2022. In the last post, I provided an overview of the conference. Caveat: these are from handwritten notes with a little verification from tweets, web searches, etc. I am positive I am getting things wrong – if you notice, don’t fume. Just drop a comment or e-mail or tweet or something.

Dashun Wang opened the conference with an introduction. The purpose of bringing together and creating a useful dialog between the producers and users of science of science & innovation research. Standing on the shoulders of (white male) giants of Kuhn, Garfield, Merton (actually may have provided Zuckerman, too so not entirely male). Using a diversity of approaches.

Panel: Innovation and Growth

Moderator Heidi Williams

Rush Holt : Choosing measures of success in evaluation is important and shouldn’t be solely on the scientists’ terms nor should it be entirely on economics terms. Rather should incorporate returns to society and should incorporate society’s view. He edited a Princeton Univ Press re-release of Vannevar Bush’s Science: The Endless Frontier (in libraries, original version– freely available). Bush’s view of the bargain of science. Scientists get the funding, do evaluation, and have autonomy. Society gets the outputs and new products. Science education to fill the pipeline [gosh my handwriting is bad – missed the next point]. A cadre of well-trained scientists is no substitute for a well-informed public [I think the part my notes missed is that this autonomy with limited oversight may not direct research to the problems most important to society. The just trust us view, turns out to not be adequate in this age of mis/disinformation if the public is not well-informed] Scientists shouldn’t be telling the public they can’t handle it but instead do a better job enabling the public to learn and ask for evidence.

Benjamin Jones : 2.8% of US GDP for R&D, both public and private. Private is going up as a percentage and government funding is going down (since the 1960s). Funding of basic science is less attractive for private funders. Research shows $1 in R&D funding returns $5 in social benefits but the group making the investment isn’t the group getting the benefit. The US could go to 5x or 6x this amount and it would still pay for itself. We have seen a productivity slow down with the decrease in R&D funding. How to fund science, though? Should it go to NSF, NIH? Which is not important, they each provide returns. There is a problem, however, with herding. It is not efficient to be like children’s soccer to have everyone working on the same problem. The other problem with funding increases like this is the pipeline [aside – my blogging colleague Drug Monkey has lots to say about employment in his and related fields]. You have to have enough people who can perform the research when you increase the funding quickly.

Marcia McNutt : Innovation and Climate Change. [presented issue of climate change and discussed the relative importance of adaptation and mitigation. Teach science as exploration, not facts at all levels.

Bhaven Sampat: Crises and funding of science. What were the long run effects of the WW2 shock to funding [for one thing, MPOW wouldn’t exist, lol]. What do we know about innovation policy during crises. What are the lessons we can take for normal times. In 6 months prior to the formation of OSRD, 2k contracts with universities and funding levels that were way higher. Bottom-up priority setting with military to set goals. Coordination so there were no holes and no herding. The research was explicitly applied – basic research done before the crisis. Speed. [the authors are also summarizing this research here]. Turning to vaccine research – the mRNA research, sequencing, etc., was all well known but the Manhattan Project – like Warp Speed funding brought this to production very quickly. BUT the aerosol research and the safety of schools trailed. There was a TON of herding (for ex, how much money and time was wasted in studying therapeutics that were universally found not to be useful?) – there was a need for air traffic control. In non-crisis times, there needs to be better support for disruptive or breakthrough efforts

Brian Uzzi: Conditions favoring good ideas in science

See a list of his research here. [Extra caveat – I was so into his talk and listening super closely that I didn’t actually write much and I’m not seeing this in his pre-prints or papers to check details 🙁 ]

There is robust evidence that diverse team composition leads to more innovative and impactful work.

AI accurately identifies dominant [unreadable maybe patents or tech?] prior to the market not on actual capability but on network of dependencies [oh, i think this was patents?].

Using the author gender estimator (Santamaria et al 2018) – which is of course not inclusive of all genders but does have some face validity when compared to self-report. … Big teams dominate, gender diversity increases, and in medical subfields the same…

Using data and doc2vec over granted vs denied patents from US, UK, Canada. free text to neural networks. separate analysis for enumerated fields such as cpc, authors, organizations. Bag of words model works better than for enumerated. Ground truth they used the results of the appeals board. Compare if machine would grant patent vs board and original decision. Machine would have correctly granted 47% of originally rejected (good), but also 31% of correctly rejected (not as good – but he says to err on the grant side.. i’m not so sure).

Audience questions were quite good. Q1) Matilda effect – do women gain the benefits of the impact of gender diversity A) they looked at team leadership – first or last author – did not have effect. Q2) gender diverse teams and gender diverse fields. A)didn’t find a strong signal but the smaller the field the smaller the effects

Karim Lakhani: Studying Science While Doing Science

Dr Lakhani is from Harvard Business School coming from studying open innovation and open source software. They are working to do field work and experiments to understand the science of science and innovation. This is to get around the myopia from looking only at outputs or citations of outputs or tech transfer of IP.

How do collaborations form? They did an experiment to change search costs for finding collaboration partners. They had an event for which attendance was mandatory to get funding. Randomized breakout rooms increased collaboration by 75% (more on this in their paper). Speed dating for scientists as powerful as co-location – accelerates water cooler time.

Bias in evaluation. Inter-rater reliability in VC, SME judgement, junior SME judgement isn’t great – closer the researcher is to the proposal, the more they will punish it (more picky). Novelty gets punished.

RCTs are required to do science (in some fields) but not to manage science. Portfolio of curiosity based science vs novel/incremental science. How do we train scientists to be managers in the scientific enterprise (not at all or not well?). Corporate R&D needs to be studied as well as academic.