Analects

PaperQA2, the first AI agent that conducts entire scientific literature reviews on its own.

Shadow Prompting—how your AI prompts are altered:

When you type a prompt into an AI image model, you might expect that what you type has a meaningful impact on what you get back … OpenAI has acknowledged that what you prompt is only taken as a suggestion: your words are altered before they reach the model, with opaque editorial decisions employed to filter out problematic requests and obscure the model’s inherent biases.

Chat GPT as cultural criticism:

It throws in our faces: why do we have so many jobs and school assignments that can be done by a non-thinking probability machine? Why do our students (even the ones paying a jillion dollars!) want to skip their lessons?

More in thread.

On the ‘targeted’ community of positive psychosis

Journalists often depict the TI community as a postmodern tragedy – a byproduct of unregulated social media. Here are thousands of very sick people, we’re told, who are just reinforcing each other’s delusions and making each other sicker because they refuse to see psychiatrists
…
What if the TI community is an inevitable reaction to the shortcomings of medical psychiatry itself? Put differently, what if medical psychiatry is inadvertently pushing people like Luca deeper into the TI community?

It’s easy to hack airplane wifi. Obviously illegal, but interesting how weak the protections are on these things. Makes one wonder how secure my own wifi is.

Retraction Watch is getting more fun. 3 MDMA papers retracted; junior researchers cited more if supervisor is well known; UK has launched a meta science unit. Among other fun things.

Politicising on twitter makes you a less credible scientist. A “monotonic” penalty! Link is a PDF.

AI is slower, but much much cheaper than people:

several public models (Claude 3.5 Sonnet and GPT-4o) complete a proportion of tasks similar to what humans can do in ~30 minutes … On average, when agents can do a task, they do so at ~1/30th of the cost of the median hourly wage of a US bachelor’s degree holder. One example: our Claude 3.5 Sonnet agent fixed bugs in an ORM library at a cost of <$2, while the human baseline took >2 hours.

On the ‘empathy economy’ as jobs are automated. Many good points. Here’s one:

“in the Feeling Economy, [that emerges during the increasing automation of jobs] many previously disadvantaged groups or individuals may have a better chance to develop their talents and to be included in the labor market.” They like to believe that this shift will simultaneously raise the floor by legitimating less-recognized jobs like caregiving and open up the ceiling by causing higher-income jobs to deprioritize “hard” technical skills—thus making it more accessible to both those without an expensive formal education, and those mistakenly perceived as less technically adept. One chapter of Rust and Huang’s book is even titled the “Era of Women” in giddy anticipation of the AI revolution’s democratizing effect.

Unfortunately, this analysis fails to consider the ways in which bias also subtly creeps into our views of who is capable of empathy and care … As the media theorist Wendy Hui Kyong Chun has remarked, the category of the human subject has largely been constructed through exclusion—“through the jettisoning of the Asian/Asian American other as robotic, as machine-like” and the “African American other as primitive, as too human.” In this paradigm, only a narrow sliver of (white) people are deemed truly human, possessing the fullest range of emotive faculties … As currently “low-status” jobs like caregiving become more established, it’s easy to imagine how the women of color who have long served as the backbone of the profession might be excluded from its glorious future, losing ground to white counterparts flocking to a newly lucrative field. (Look, for instance, to the whitewashing gentrification effect of cultural legitimization in the cannabis industry.)

The emerging virtual-preaching economy in Kenya:

The preachers say these online ministries have brought religion and fellowship to people who might not have otherwise found them. But the field is unregulated and less standardized than in-person churches. Many virtual preachers have no formal training in theology. And critics question whether they can really bring the communion and bonds that brick-and-mortar churches have offered for centuries.

AI language models are almost as good as each other now, and other interesting news.

Giant rat penis redux - AI-generated diagram leads to journal article retraction. There is a hand in this leg. Still made it into the Journal Medicine. See also the giant rat penis in another AI-generated figure that made it to publication.

On Turing’s 1952 ChatGPT

The Secret Of Minecraft:

“A generative, networked system laced throughout with secrets.”

Are we in a simulation (pdf)? A head-pounding philosophy paper:

as far as I can tell, the basic thrust of the simulation argument has real philosophical force and interest—especially when interpreted in the Type 2 manner I’ve argued for here (that is, as not resting on the likelihood of any particular set of empirical claims). Perhaps it does not, ultimately, work—but I don’t think its failures are at all obvious

And whether we buy simulation arguments or not, they are a reminder that the world we see and take for granted is only a part of the world; and that in principle, our overall existential situation could in fact be many different ways, not all of which we are accustomed to considering

More evidence social media isn’t so influential. See also Stuart Ritchie on this. See also Peter Gray on this.

Thinking about God increases acceptance of artificial intelligence in decision-making. I’ll just copy the abstract. Can’t tell if this is for or against my laissez-faire attitude about the dangers of AI. Depends how religious we become I guess.

Thinking about God promotes greater acceptance of Artificial intelligence (AI)-based recommendations. Eight preregistered experiments (n = 2,462) reveal that when God is salient, people are more willing to consider AI-based recommendations than when God is not salient. Studies 1 and 2a to 2d demonstrate across a wide variety of contexts, from choosing entertainment and food to mutual funds and dental procedures, that God salience reduces reliance on human recommenders and heightens willingness to consider AI recommendations. Studies 3 and 4 demonstrate that the reduced reliance on humans is driven by a heightened feeling of smallness when God is salient, followed by a recognition of human fallibility. Study 5 addresses the similarity in mysteriousness between God and AI as an alternative, but unsupported, explanation. Finally, study 6 (n = 53,563) corroborates the experimental results with data from 21 countries on the usage of robo-advisors in financial decision-making.

LLM persuasiveness is capped. I told you.

model persuasiveness is characterized by sharply diminishing returns, such that current frontier models are barely more persuasive than models smaller in size by an order of magnitude or more. Second, mere task completion (coherence, staying on topic) appears to account for larger models’ persuasive advantage. These findings suggest that further scaling model size will not much increase the persuasiveness of static LLM-generated messages.

A data scientist’s reflections on AI:

I started working as a data scientist in 2019, and by 2021 I had realized that while the field was large, it was also largely fraudulent. Most of the leaders that I was working with clearly had not gotten as far as reading about it for thirty minutes despite insisting that things like, I dunno, the next five years of a ten thousand person non-tech organization should be entirely AI focused

Social media for AI:

While the interface looks like Instagram, the app’s main twist is that, when signing up, you create an AI character, or Butterfly, that starts generating photos and interacting with other accounts on its own. There is no limit to the number of Butterflies you can create, and they are designed to coexist with human accounts that can also post to the feed and comment.

Mostly useful for the high level ideas on how to prompt AI better.

Retrospective on AI by Jack Clark (of Anthropic fame). Obviously bullish, but interesting nonetheless.

Notebook LM from Google. Seems, on the surface, like quite an improvement on ChatGPT. Let’s see how long it remains free.

A Wikipedia page on science in 2023.

What’s making kids not alright? And some on how to make them alright. Good notes on social media and it’s value, not just harm. Also coping:

There’s coping by expressing what we’re feeling, and there’s coping by taming or bringing back under control our emotions … if we start on the expressing category, there’s talking about what we’re feeling and seeking social support … listen to music … make things … art … And then there’s the taming category. whether it’s going for a walk or taking a bath or finding a food that we love and enjoying it or getting with a TV show that we know we’re going to leave the end of the episode feeling better than we did when we started. And I think, if we can bring coping forward as the thing to focus on — the distress, that is a done deal.

See also social media might not be making us miserable.

News stripped of the crap by AI.

The Quest To Quantify Our Senses:

our new sensing machines more accurately capture and analyze the microtime and microspace of our breath, heartbeat, brainwaves, muscle tension, or reaction times. But they do this for another reason. Our sensing machines now conceive and create techniques that aim to fulfill that long sought-after dream of those forgotten 19th-century researchers like Fechner and Marey: to become one with what Fechner called the animated substance of the technological world itself.

On early Sydney, the Bing AI. Very odd.

Sydney absolutely blew my mind because of her personality; search was an irritant…This tech does not feel like a better search. It feels like something entirely new. And I’m not sure if we are ready for it.

Your DNA Can Now Be Pulled From Thin Air.

The GrubHub Of Human Affliction: a depressing satire of journalism and the gig economy.

Profile of a computer-virus maker.

Why some accidents are unavoidable. Paper on man-made technological disasters.

Most AI Fear Is Future Fear

The problem of news from nowhere. See also my article:

politically induced mental and physical symptoms appear to be more pronounced among not just the young, but specifically those who are politically engaged and left-leaning … In the United States, the combination of being young, engaged, and liberal has become associated with anxiety, unhappiness, and even despair

Why progressives? The article suggests that conservatives: “care less about politics” and “conservatives tend to be a minority. So they have little choice but to acclimate themselves to a liberal environment and learn to interact with those who are different from them”. But one wonders if it’s simply that the solutions to conservative problems seem more tractable on the surface: a rejection of change, versus the welcoming of it.

Smarter entities are less coherent. The idea behind the AI collapse is that AI will use its inevitable intelligence advantage to eliminate humans in service of some goal. The paperclip maximiser will use all the resources to make paperclips, wiping us out in the process. But the smarter the entity, the less coherent its goal states are. Humans are much more of a hot mess of competing desires and intentions than, say, honeybees. It seems like AI will follow this principle. The more complex the world something operates in, the more complex its cognition must be. Anyway, here’s an article on the idea.

On the alien characteristics of LLMs: the Waluigi effect.

Short version:

After you train an LLM to satisfy a desirable property P, then it’s easier to elicit the chatbot into satisfying the exact opposite of property P

Why?

When you spend many bits-of-optimisation locating a character, it only takes a few extra bits to specify their antipode.

Is social media making us miserable? Stuart Ritchie (of Science Fictions fame) thinks that, if so, it’s not that deep:

when the authors of the “Facebook arrival” study raised their standards in this way, running a correction for multiple comparisons, all the results they found for well-being were no longer statistically significant. That is, a somewhat more conservative way of looking at the data indicated that every result they found was statistically indistinguishable from a scenario where Facebook had no effect on well-being whatsoever.
Now let’s turn to the second study, which was a randomised controlled trial where 1,637 adults were randomly assigned to shut down their Facebook account for four weeks, or go on using it as normal. Let’s call it the “deactivating Facebook” study. This “famous” study has been described as “the most impressive by far” in this area, and was the only study cited in the Financial Times as an example of the “growing body of research showing that reducing time on social media improves mental health”.
The bottom-line result was that leaving Facebook for a month led to higher well-being, as measured on a questionnaire at the end of the month. But again, looking in a bit more detail raises some important questions. First, the deactivation happened in the weeks leading up to the 2018 US midterm elections. This was quite deliberate, because the researchers also wanted to look at how Facebook affected people’s political polarisation. But it does mean that the results they found might not apply to deactivating Facebook at other, less fractious times – maybe it’s particularly good to be away from Facebook during an election, when you can avoid hearing other people’s daft political opinions.
Second, just like the other Facebook study, the researchers tested a lot of hypotheses – and again, they used a correction to reduce false-positives. This time, the results weren’t wiped out entirely – but almost. Of the four questionnaire items that showed statistically-significant results before the correction, only one – “how lonely are you?” – remained significant after correction.
It’s debatable whether even this result would survive the researchers corrected for all the other statistical tests they ran. Not only that, but they also ran a second model, controlling for the overall amount of time people used Facebook, and this found even fewer results than the first one. Third, as well as the well-being questionnaire at the end of the study, the participants got daily text messages asking them how happy they were, among other questions. Oddly, these showed absolutely no effect of being off Facebook—and not even the slightest hint of a trend in that direction.

For those of you asking me good GPT prompts, here’s a good example:

I want to learn about <insert topic>. In a moment, I’m going to ask you a series of questions about it. But before we get into it, I’d appreciate it if you answered as though you were a no nonsense teacher with an ambitious, self-directed student. That is:

• Err in the direction of thinking that I’m relatively knowledgable and technical.
• Don’t overexplain things. I’ll ask for more information if I need it.
• Assume that I’m already skeptical and that you don’t need to qualify, hedge, or otherwise add to or manage my skepticism.
• Don’t apologize for misunderstanding or getting an answer wrong.
• It’s fine to be a bit abrupt and even “mean”. Value directness and frankness; assume I’m relatively insensitive.
• Where reasonable suggest things for me to try independently. It’s fine to tell me to install packages or run go out and do things or whatever, if you think it will help me learn quickly. (Only do this were reasonable; otherwise abstract explanations are fine.)
• Give at most one example per response.

The Moral Economy Of High-Tech Modernism.

Continuing on our hydraulic theme, comments on the intersection between algorithms and politics. In fact they’re also building on James Scott.

Algorithms extend both the logic of hierarchy and the logic of competition. They are machines for making categories and applying them, much like traditional bureaucracy. And they are self-adjusting allocative machines, much like canonical markets … Both bureaucracy and computation enable an important form of social power: the power to classify. Bureaucracy deploys filing cabinets and memorandums to organize the world and make it “legible,” in Scott’s terminology. Legibility is, in the first instance, a matter of classification … The bureaucratic capacity to categorize, organize, and exploit this information revolutionized the state’s ability to get things done. It also led the state to reorder society in ways that reflected its categorizations and acted them out. Social, political, and even physical geographies were simplified to make them legible to public officials. Surnames were imposed to tax individuals; the streets of Paris were redesigned to facilitate control … Markets, too, were standardized, as concrete goods like grain, lumber, and meat were converted into abstract qualities to be traded at scale. The power to categorize made and shaped markets … Businesses created their own bureaucracies to order the world, deciding who could participate in markets and how goods ought to be categorized.

Computational algorithms—especially machine learning algorithms—perform similar functions to the bureaucratic technologies that Scott describes … The workings of algorithms are much less visible, even though they penetrate deeper into the social fabric than the workings of bureaucracies. The development of smart environments and the Internet of Things has made the collection and processing of information about people too comprehensive, minutely geared, inescapable, and fast-growing for considered consent and resistance … Traditional high modernism did not just rely on standard issue bureaucrats. It empowered a wide variety of experts to make decisions in the area of their particular specialist knowledge and authority. Now, many of these experts are embattled, as their authority is nibbled away by algorithms whose advocates claim are more accurate, more reliable, and less partial than their human predecessors.

And then some nice comparisons between the pathologies of the bureaucratic modernism and this new computational modernism:

The problem [with bureaucratic modernism] was not that the public did not notice the failures, but that their views were largely ignored … The political and social mechanisms through which people previously responded, actively and knowingly, to their categorization—by affirming, disagreeing with, or subverting it—have been replaced by closed loops in which algorithms assign people unwittingly to categories, assess their responses to cues, and continually update and reclassify them.

Nice read.

What happens, then, when large and powerful states, along with the transnational institutions and corporations they promote and protect, are all driving towards the same goal: the universalisation of an American-style “global economy” and its associated culture? … The expansion of this system has created problems — ecological degradation, social unrest, cultural fragmentation, economic interdependence, systemic fragility, institutional breakdown. The system has responded with more expansion and more control, growing bigger, more complex and more controlling … Modernity can best be seen as a system of enclosure, fuelled by the destruction of self-sufficient lifeways, and their replacement with a system of economic exploitation, guided by states and exercised by corporations. The disempowering of people everywhere, and the deepening of technological control

This seems a little alarmist, but the increasingly hydraulic nature of our modern way of being is superficially quite obvious. I was more impressed by the author’s idea to adopt James C. Scott’s ‘shatter zones’ to ameliorate it:

In his 2009 book The Art of Not Being Governed — subtitled, “an anarchist history of upland Southeast Asia” — the historian James C. Scott … The “hill tribes” and “barbarians” living outside civilisation’s walls, he says, are neither “left behind” by “progress”, nor the “remnants” of earlier “backwards” cultures; they are in fact escapees. “Hill peoples are best understood as runaway, fugitive, maroon communities who have, over the course of two millennia, been fleeing the oppression of state-making projects in the valleys — slavery, conscription, taxes, corvée labour, epidemics and warfare.”

Scott’s thesis is that throughout history, escaping from the reach of oppressive states has been a popular aim, and that in response, some cultures have developed sophisticated ways of living in hard-to-govern “shatter zones”, which allow them to avoid being assimilated. Standard-issue historical accounts of “development”, he says, are really the history of state-making, written from the state’s point of view: they pay no attention to “the history of deliberate and reactive statelessness”. Yet that history — whether of hill tribes, runaway slaves, gypsies, maroons, sea peoples or Marsh Arabs — is global and ongoing. Taking it into account, says Scott, would “reverse much received wisdom about ‘primitivism’”. Instead, we would read a history of “self-barbarisation”: a process of reactive resistance, of becoming awkward, of making a community into a shape that it is hard for the state to absorb, or even to quite comprehend … localised, potentially dispersed cultures can be tough to conquer.

Then some ideas about how to go about it, with the obvious focus on the internet as a convenient place to create ‘shatter zones’. I must be honest though—the internet corresponds to an alarming rise in loneliness, so whatever the internet is theoretically capable of in terms of connecting people, the practice leaves much to be desired. This constant recourse to it as a solution needs to become a bit more sophisticated.

Wokeism is winding down. See also is performative populism over.

On the media as a good thing:

Hate certain parts of the media, including specific articles, false narratives, and even, if you must, individual journalists who represent the worst of their profession. But if you care about having a functional society in which forming accurate perceptions of at least some portions of reality is possible, please temper your criticism.

Seems also worth noting that media have predictable filters. Non-media entities are subject to the same filters—perhaps more so.

Books are not Information Dense. An argument for substacks as a more information dense source of information. Though, see also is the internet information overload.

The infrastructure behind ATMs. The surprisingly complicated business of making your money available to you.

Is the internet information overload? Interesting reflections on the benefits and drawbacks of the information age. Highlights:

If you look at a site like Buzzfeed, it has reports about the death of Kim Jong Il right next to viral videos about cats. It’s jarring – and seems a little amoral … [this is] pointing to the benefits of having a very small aperture for news. That aperture was controlled by full-time professional editors, but … what comes through the news hole now is anything anybody is interested in enough to post … when you have so few apertures for news and they’re controlled by such a similar set of people, you get a certain limited set of stories. We at least now have the opportunity to create filters that let in more than the traditional room of middle-aged white men. If we’re not reading the stuff that matters, it’s our fault.

Ask anybody who is in any of the traditional knowledge fields, and she or he will very likely tell you that the Internet has made them smarter. They couldn’t do their work without it; they’re doing it better than ever before, they know more; they can find more; they can run down dead ends faster than ever before. In the sciences and humanities, it’s hard to find somebody who claims the Internet is making him or her stupid, even among those who claim the Internet is making us stupid.

Reminder that TikTok is spyware. Contra this post. Is there another chance for a ‘good’ social media?

Platforms are not ecosystems:

tech platforms and proprietary software environments are not ecosystems, so don’t call them that. Call them built environments, i.e. designed, rules-based systems that explicitly structure interests to secure specific intended outcomes. It does no good – for journalists in particular – to transmit the suggestion that a walled garden is the same as a living forest. That an app market-place is the same kind of thing as an open protocol. We don’t just serve the interests of system-owners when we repeat the pretty lie. We shut down an essential way to imagine alternatives. So what if, every time we read ‘ecosystem’, we instead say ‘plantation’? A plantation is a hierarchical, exploitative monoculture … Google’s interlinked extractive systems are plantations whose single crop is data for ads. They’re designed environments; their parent company, Alphabet, a conurbation of control.

Human intelligence is converging:

most recent studies report mainly positive Flynn effects in economically less developed countries, but trivial and frequently negative Flynn effects in the economically most advanced countries … these trends, observed in adolescents today, will reduce cognitive gaps between the working-age populations of countries and world regions during coming decades.

The honesty of pornography. The last paragraph:

All of this is to say that pornography is remarkably honest, and not simply because, as anti-pornography feminists allege, it documents patriarchy’s debasement of women. Rather, it is honest because it showcases the hard, often confusing work of reconciling private desire with public life, of admitting that sex with others can be unethical, of distinguishing between fantasy and reality. Antique pornography makes these contradictions obvious, circulating knowledge that we think, today, is at odds with eroticism. But perhaps it isn’t – perhaps there’s a utility to pornography’s mixed messages. Perhaps it was designed to confuse us, the better to underscore the clarity with which we should enter into the messy endeavour of sex with other people.

The “je ne sais quoi” of TikTok:

It’s an unambiguously positive change in social media, on pretty much every front. To try to get it down to a bulleted list:

• Organic audience acquisition without need for self promotion.
• Types of content that can flourish is much broader.
• Incredible collaboration tools, leading to mixing and remixing art on the platform. The only other example of this I can think of this on other social platforms is textual. Quoting someone’s tweet and commenting on it and the like.
• Manages to maintain a platform-level “zeitgeist” of sorts, similar to Twitter, while also giving users highly customized experiences. It does this without the need for trending topics or curated hashtags, it’s all in the algorithm.
• Fosters empathy instead of sowing division. Much less emphasis on “culture war” and politics.

How to care less about work. Might be paywalled so use archive.ph. Some highlights:

So what work is actually valuable? It’s incredibly unclear. Many knowledge workers, ourselves included, find themselves insecure in some capacity about the work they’re doing: how much they do, whom they do it for, its value, their value, how their work is rewarded and by whom. We respond to this confusion in pretty confusing ways. Some become deeply disillusioned or radicalized against the extractive, capitalist system that makes all of this so muddled. And others throw themselves into work, centering it as the defining element of their self-worth. In response to the existential crisis of personal value, they jump on the productivity treadmill, praying that in the process of constant work they might eventually stumble across purpose, dignity, and security.

and

Once you figure out what [things you once took pleasure in], see if you can recall its contours. Were you in charge? Were there achievable goals or no goals at all? Did you do it alone or with others? Was it something that really felt as if it was yours, not your siblings’? Did it mean regular time spent with someone you liked? Did it involve organizing, creating, practicing, following patterns, or collaborating? See if you can describe, out loud or in writing, what you did and why you loved it. Now see if there’s anything at all that resembles that experience in your life today.

Motivating creativity:

the … optimal reward scheme is maximally uncertain—the agent receives transfers for success, but their distribution has an extreme variance

It makes you try lots of things. Is this surprising? It doesn’t feel surprising, but as the author notes, does:

shed light on the non-transparent incentives used by online platforms, such as YouTube

Matt Levine’s excellent history of crypto. Off-beat financier with possibly my favourite column. This is the most informed on the topic I’ve ever been.

How AI will change everything on the internet. Very thought provoking, but short:

Less than two years from now, maybe I will speak into my computer, outline my topics of interest, and somebody’s version of AI will spit back to me a kind of Twitter remix, in a readable format and tailored to my needs.

Seems like a good time to re-consider your approach to information extraction now.

Royal Netherlands Army commences armed robot trials in first among Western militaries.

“Why I think strong general AI is coming soon”. Very interesting.

The Personal and Private Nature of Smartphones:

because smartphones are considerably more personal and private than PCs, using them activates intimate self-knowledge and increases private self-focus, shifting attention toward individuating personal preferences, feelings, and inner states

On the possibilities for secure digital personhood.