Positive Intelligence pt.II

You might want to skip the intro, if you’ve already read the other installments

In a previous article, I wondered why we overengineer calming down so much:

I’m often struck by just how much of the pop-psych/neuroscience advice one sees for the average working person boils down to little more than “just cool the fuck out, and you’ll be better at stuff”. I guess, more to the point, I’m often left wondering why we feel the need to over-engineer this kind of thing so egregiously, particularly when most of these theories seem to produce as much bad advice as good advice.

It’s a wide-ranging article, that explores a bunch of pop-psych theories which bother me. In the process I mentioned one that was introduced to me by a start-up founder friend of mine: Positive Intelligence. They’d come across it during their executive coaching sessions, and were keen enough to walk me through it. In the course of investigating, I thought it was a good example of needlessly complicating simple advice, and spent a short paragraph teasing it.

This was a mistake.

It invited a number of emails from defenders of the company. I cannot emphasise enough how unlikely it is that your email is going to convince me to take some consumer-facing faux-brain science seriously. One day, I will play a different tune. One day, I will have my own consumer-facing brand of faux-brain science. But until then, no.

However, one of these emailers offered to pay me to evaluate it, and this admittedly does have an attention-grabbing quality. I offered a discount to publish the results as an article, and so here we are. Positive Intelligence, the review.

As usual, I overestimate how much content fits in a weekly blog, so I’m going to split this into more than one. The first article wasn’t about the content of Positive Intelligence. It was about the context.

We looked at this first because, actually, this context is the same as many of the pop-neuroscience things you’ll find out in the wild, so it’s quite nice to have it as a standalone.

Also, because, frankly, after reading it, you might not even feel inclined to read about the content.

But you should, because the content is amusing. Let’s start with the core claims in this article. Then, because it’s my bag, we’ll do another one just on the “neuroscience”.

¶What they claim

After skimming through the website, I reckon we should set the scene with their most recent claims. The book was published in 2011, but they have two science whitepapers dated 2022, so maybe they’ve spent the intervening decade sharpening things up. They sure seem to think so. Just look at their neuroscience whitepaper:

This whitepaper both updates and expands on the original neuroscience foundation of Positive Intelligence

And from one about their methods:

Positive Intelligence is based on the latest research in positive psychology, cognitive psychology, neuroscience, and performance science.

Alright. So, it’s like we thought. The book has a basis in the latest research, but because that stuff is ten years old—positively geriatric in the brain sciences—then the whitepapers will make sure we’ve cleaned up any loose ends the book left hanging.

And the absence of anything in the intervening time is quite telling. It’s a functional retraction—it tells us that they don’t think that anything from the ten years between book and white papers are worth looking at.

So, let’s start with Chamine’s book claims, then make sure we fill in any gaps with their whitepaper claims—first their ‘research foundations’, and we’ll finish off with the neuroscience whitepaper.

¶The PQ, from the book

We have to start with the PQ. This is the core of the whole thing. It’s a riff off IQ and EQ. Some kind of quantification of your “mental fitness” in the form of a ‘Positive Intelligence Quotient’. In the book, Chamine describes the PQ as:

the percentage of time your mind is acting as your friend rather than as your enemy; or, in other words, it is the percentage of time your mind is serving you versus sabotaging you. For example, a PQ of 75 means that your mind is serving you about 75 percent of the time and is sabotaging you about 25 percent of the time. We don’t count the periods of time when your mind is in neutral territory.

It’s determined with a 2-minute, 20-item online quiz that essentially asks you how often negative emotions happen. It’s not precisely clear how this is converted into the percentage, but since it’s out of 20, we might guess each item gives you 5% one way or the other.¹

Self-report is a notoriously troublesome measure—the first thing a first year psychology student learns to mistrust—but frankly, is a reasonable metric for this kind of thing. Our diagnostic manuals use individual distress to help them diagnose wellbeing issues, and you can’t really get at this without a subjective measure of wellbeing. So let’s subjectively measure our negative emotions too, why not.

Now, if your next question was to ask something like “how did they work this quiz out?” or “where did this ratio come from?”, then you’re in luck, because Chamine spends a whole section lightly detailing research that ostensibly shows that a higher PQ is better for your mental health. For example an article they claim analysed:

more than two hundred different scientific studies, which collectively tested more than 275,000 people, concluded that higher PQ leads to higher salary and greater success in the arenas of work, marriage, health, sociability, friendship, and creativity

Sounds very convincing! Until you look up the article, at which point you’d find out it has nothing in particular to do with PQ. It’s actually titled “The Benefits of Frequent Positive Affect: Does Happiness Lead to Success?”. Positive affect is psychology slang for “good mood/nice emotions”, and is only relevant to the PQ in the sense that PQ claims to measure something about positive and negative affect.

Now, Chamine does acknowledge something like this, but I think what he’s doing is using an awful lot of words to say “these studies document completely different things from the PQ”:

different researchers have used different methods to track positivity and calculate positive-to-negative ratios … For consistency and simplicity, I have translated various researchers’ findings into their PQ-equivalent interpretations

And indeed, all the studies I bothered checking mentioned nothing whatsoever about some kind of positive-to-negative ratio.

So… I suppose so far, when you cut through the noise, this has just been a chapter about how being sad is less good than being happy. Which is true! But it’s not really clear how PQ is relevant to all this yet, outside of the fact the PQ seems to be measuring someone’s (Chamine’s?) idea of happiness, somehow.

It does leave us wondering where did this whole ratio idea come from. Luckily Chamine tells us. If we skip ahead a bit in the book, we find that he makes the specific claim that a PQ of 75 is an important ‘tipping point’, based on the work of this 1999 mathematical modelling study.²

Now, this study is absolutely not about the PQ score, nor is it in any way “PQ equivalent” as Chamine claims. I guess he chose it because the authors of the study have a “Positivity vs Negativity” dimension.

The dimension in question is actually John Gottman’s³ positivity-coding from his divorce-prediction work. Gottman coded positivity in the interactions between spouses to work out whether they’d split up. And the study the PQ-ratio is based on is also about analysing conversation patterns.

Neither have anything to do with analysing mood. And, not only is the whole thing some kind of odd category error, it’s also just flat out empirically incorrect as operationalised by the study PQ is using to justify itself. From this review article:⁴

we demonstrate that the purported application of these equations contains numerous fundamental conceptual and mathematical errors

So, it’s a very wobbly kind of true to say, as Chamine says:

different methods to track positivity and calculate positive-to-negative ratios… [have relationships to positive outcomes].

But if one were cynical, one might almost feel like he was deliberately trying to trick us into thinking that the PQ has anything to do with this. If someone who wasn’t trying to justify the price-tag of their methodology was summarising this research, they might have just said something like “everyone agrees that being happy is better for you than being sad”. But Chamine doesn’t just say that. He says that while stuffing ‘PQ’ in between every other word.

What would be a less wobbly kind of true thing to say about PQ is that, so far, it seems to correspond to nothing in particular, outside of the fact that it ostensibly says something about how sad you are. But I suggest that you could design a 20-item quiz that did this, and everything they’ve said about PQ thus far would equally apply to your quiz.

This is actually very nice, because it means we don’t really have to spend our time trying to verify the validity of the other claims about PQ, like “80% of teams score below 75 PQ”, nor do we need to make sense of how we might improve our PQ. None of it really means anything yet.

I’m sure this is a simple misunderstanding that they’ll clear up in the whitepapers though. They’re an institute now, after all! So for now, let’s move on!

¶The saboteurs and the sages, from the book

Chamine goes on to talk about ten ‘saboteurs’—different flavours of inner critic. He’s ambivalent about these little critters:

the initial role of all Saboteurs: helping us survive … though I no longer needed them for survival as an adult, they continued to exert a great deal of negative influence in my mind.

I’m less ambivalent. Even if I wasn’t already thoroughly underwhelmed by the PQ thing, I’d be going into this extremely skeptical.

You see, long-time readers will recognise the classic play on any stress-related process—they’re posed as maladapted to the modern world:

The formation of the Saboteurs begins to make clear sense once you realize that the primary objective of the first fifteen to twenty years of life is to survive long enough to pass on your genes. In that sense, we aren’t much different from sea turtle hatchlings shuffling their way toward the safety of the ocean as soon as they break out of their shells.

Then, it seems a bit like he’s doing what all modern stress-avoident gurus do these days, and badly cribbing Sapolsky’s Why Zebra’s Don’t Get Ulcers:

For the human child, however, survival has a component beyond physical survival. We also need to survive emotionally. The human brain is wired to pay close attention to our environment in our early years and adjust accordingly so we can bear the emotional strains we all encounter and make it into reproductive adulthood.

As I write elsewhere, Sapolsky wrote about how social primates have to deal with social stressors, which are a bit different to the stressors faced by less social animals.

This appears to be what Chamine is trying to do, because otherwise I’d be at a loss to explain what Chamine reckons emotions are if not physical, and what drives other animals to behave, if not emotion. Maybe they are robots.

The childhood thing is a bit more unique to the parts-work milieu PQ was borne of. I’m not sure precisely why they all care so much about maladaptive childhood patterns, but it dates all the way back to Transactional Analysis from the 60’s. If I were to guess, it’s because Freud’s Structural Model (with the ego and the superego and the id) is perhaps the most primordial form of modern parts-work, and Freud was famously and lewdly obsessed with the developing mind of the child.

Chamine then wraps up by pointing out:

If you don’t think you have them, you’re especially at risk: your Saboteurs are hiding well.

Altogether, he’s telling you that not only did you evolve badly, your childhood messed you up, and especially if you think you’re ok, you’re actually not.

In reality, your stress response is usually very well adapted to the modern world and, as I cover in more detail in the next article on PQ, only about 10% of people will have a mood disorder of some kind at any given moment, and not all of those are caused by childhood stuff.⁵ So, if you think you’re ok, you probably are.

And I’m not just saying that statistically. Chamine makes it clear himself, even if he appears unaware of it. The main saboteur is the ‘Judge’, who is self-critical, which is roughly what you’d expect from an inner-critic. And I would absolutely agree that if you share Chamine’s outlook, then you are absolutely not ok:

as a child I felt I was not getting much caring and attention. My mind could make sense of this in one of two ways. I could have interpreted the situation accurately and admitted that I was being raised by flawed parents who didn’t know how to give me the attention I needed and deserved. But this would have [been scary because we depend on our parents] … Instead, the Judge came to the rescue. The Judge’s solution was that I was deeply flawed and unworthy of my perfect parents’ time: Why should they show any more affection for someone so undeserving?

I would not agree that Chamine’s Judge “rescued” him though. I would say instead that Chamine’s Judge really messed the boy up.

Most people’s inner-critics usually work very well to improve them. I’ve spoken before about how important it is to critique your personal coherence, you might have heard of Carol Dweck’s Growth Mindset, the fact you can distinguish between guilt and shame⁶ and anything you’ve ever read about metacognition should all be enough to convince you that self-criticism is usually quite helpful, without delving into the therapeutic literature on the subject.

Oh! Oops. I’m describing Chamine’s Sage! If only I’d read the next chapter before I began my skeptical hedging, I would have learned that there is another inner critic that is a good inner critic! And, in fact:

With the Sage perspective, you are more likely to be creative because your PQ Brain regions are activated. These regions are far more creative than the Survivor Brain, where the Judge lives

So it’s not only good, but it lives in a different part of the brain!

Since, unlike the PQ thing, Chamine has completely neglected to link any of this saboteur and sage stuff to any literature, I suppose we will have to wait until the brain science article to work out this particular distinction.

So let’s put a pin in this for now. I’m not yet convinced that you’d find your good inner-criticisms and your bad inner-criticisms coming from different parts of the brain, but I certainly think that if you have inner-critics like Chamine’s Judge, you need to go hunting for a better one!

In the meantime, let’s look a Chamine’s ten saboteurs in more detail, because we do have some literature on that.

¶Saboteurs and PQ, from the whitepaper

In the book Chamine lists ten saboteurs, but as I mention, suddenly stops linking his claims to vaguely relevant literature. Instead, he waits 11 years, then puts out a whitepaper that claims to provide:

updated research data regarding both the Saboteur Assessment and PQ Assessment. It includes an exploratory and confirmatory factor analysis to confirm the Saboteur Assessments’ reliability and its underlying factors. It also includes analysis of factor validity and test score reliability for the PQ Assessment

Factor analysis is a way of figuring out the latent variables that give rise to patterns in people’s responses. Put another way, it’ll help you understand if your questions actually map onto anything. So, if people answer a group of questions in a similar way a lot of the time, you might suspect those questions ‘cluster’ around something. Factor analysis is a way of working out what those clusters are.

This actually seems like a great way to work out whether it’s worth exploring Chamine’s terrible ten. They start with an exploratory factor analysis—the process of finding the clusters. The first line of the second paragraph reads:

A scree test revealed a large break in the proportion of variance accounted for after the first and second factors, with no clear breaks between subsequent factors.

This means that the first two ‘clusters’ explained most of the data. Convention would have you interpret this to mean that our survey was getting at two things, not ten. But convention be damned, because:

Ultimately, a total of 13 factors were retained, which accounted for over 99% of the variance.

They do this because they want to “err on the side of over-factoring”—something the original factor analysis statistician (Cattell) recommended to allow small but potentially important clusters to emerge that might otherwise be missed in the noise.

But I suppose they didn’t really like the fact that they had to go all the way to 13 to get that nice 99% variance explained, because they then ‘consolidated into 10 factors’.

Why? Why not. This is exploratory. That’s the point. You’re already pulling out 11 more factors than your scree test tells you are clearly important, so why not merge them into ten if you have an intuition (or a book) that tells you there are ten things and not thirteen.

All you need to do to justify this kind of thing is follow up with a confirmatory factor analysis to make sure that your intuitions actually play out. Confirmatory factor analysis tests the structure you’ve rigged together in your exploration to see how well it actually fits the data.

You will not be surprised to find out that, although they claim to have done a confirmatory factor analysis, it does not appear in the whitepaper.

Instead, they slide right past this critical step, and compute Cronbach’s α—a measure of how inter-related the questions in each cluster are—for the ten jury-rigged factors.

All ten meet the conventional threshold for reliability (i.e. inter-relatedness), which would seem to support their jury-rigging. At least, until you remember that the scree test reckoned we should have stopped at two factors, and so of course all the questions are going to be inter-related! If you shred a two-factor solution into a bunch of tiny pseudo-factors, you’d have to be pretty unlucky to get an unrelated cluster of items.

I mean, just think about it. Imagine a blue piece of paper and a red piece of paper. Shred them into bits. Mix up the pieces. Now make ten piles. You’ll have piles that are more blue or more red, and rarely will you have a pile that is a nice even split between red and blue.

And then you might start trying to label your piles to make them look different. In this case, they want you to believe that we have a Stickler and an Achiever saboteur, for example. But you could explain both with a ‘Worry-About-Perfection’ saboteur. Even if you mix up the ‘Worry’ questions with some questions about something else, you still have more ‘Worry’ questions in there. You see? Two small piles made up of the same red paper, a little bit of blue by accident, and the statistics wouldn’t be able to help you realise your mistake.

And that’s without getting into the more obvious critiques about sampling bias and whatnot. Their core method is flawed.

And it’s hard not to see it as intentional, because they didn’t “err on the side of over-factoring” when they did exploratory factor analysis on their PQ assessment. When the scree test told them that their PQ assessment probably mapped onto two factors, funnily enough, they stopped pulling out factors. It’s probably just a coincidence that this assessment is supposed to test you for two things—your positivity and your negativity.

¶Outro

So what evidence for Positive Intelligence are we left with so far? I don’t think it would be unfair to say none in particular.

The idea of PQ itself rests on little outside of “it’s good to be happier, than sadder”. There’s no ‘critical threshold’ for too much sadness. And most importantly, if you designed a 20-item quiz to determine how good vs bad you felt, everything Chamine has written would equally apply to your quiz. You could even probably replicate his factor analysis here—you’d have to have a particularly convoluted mind to design a quiz about happiness and sadness that ended up with more than two factors.

The idea of saboteurs is more obviously unsupported. Chamine doesn’t even pretend they’re related to the literature in the book, and his attempt to science-wash it in the whitepaper is clumsy. If you wanted to and it works for you, you could use his ‘saboteurs’ to try to start a dialogue with the parts of your mind you’re unhappy with. Parts-work is popular for a reason—people like it. But there’s no particular reason to choose Chamine’s parts over any of the other models.

All that’s left to do now, is evaluate the brain-science—the claim that the PQ brain, where the Sage lives, is different to the Survivor brain, where the saboteurs live. Whether there’s a distinction between the good inner-critic and the bad. Once we’ve done that, we might be able to think a bit more about what Chamine reckons we should do to be less saboteur-ey, and more sag-ey, and by extension what ‘raising your PQ’ might actually mean.

But that’ll have to wait for next time.