Most of what we know about politics on Facebook comes from the big Western democracies. The smaller and newer ones are in the data too, and nobody much writes about them. So I took six of them and asked a plain question: does the state of a country’s democracy show up in how people engage with its political parties?

My hunch going in was that it might. Countries with more polarised politics, or more hostile rhetoric from parties, might see more reaction and more sharing. The answer, with the caveat that six countries is a small sample, is that I cannot find it. What shows up instead is something much more mundane: who owns the audience.

What I did

I used posts from 35 official party pages in Finland, Lithuania, Portugal, Romania, Slovenia and Switzerland, exported from CrowdTangle in October 2020 for my doctoral research. I counted interactions: reactions, comments and shares. Then I joined them to yearly indicators from the Varieties of Democracy project, known as V-Dem: how democratic elections are, how polarised society is, and how much hate speech political parties use.

The paper asks two questions. Do countries with different levels of democracy differ in engagement? And, within one country over time, does engagement move when the democracy indicators move?

Engagement is about audiences, not regimes

The biggest difference between the countries is not their politics. It is how concentrated their Facebook audiences are. In Romania, one party page held 72% of all interactions I have. In Switzerland, one page held 51%. In the other four countries the biggest page held between 23% and 39%.

That matters because Romania looks like an outlier on every engagement measure. A typical Romanian party post drew a median of 259 interactions, against 40 to 65 elsewhere. But a few very large pages explain much more of that than anything about how the country is governed.

Across the six countries, no pattern I can trust

Romania scores lowest on electoral democracy in my sample and has the highest engagement, which looks like a story. It is not one I would tell. With six countries, a rank correlation has to be nearly perfect to count as unlikely by chance. None came close, and the smallest p-value was 0.14. The same went for polarisation and hate speech, and for engagement per follower, the angry share of reactions and the share of interactions that are shares.

With six countries, the design can only rule out very large associations. Absence of evidence is not evidence of absence.

Within countries over time, still nothing firm

A better test uses changes inside each country. I compared each page with itself, year by year, and asked whether years with more polarisation or more party hate speech had more or less engagement. Because I only have six countries to cluster on, I used a wild bootstrap, which is the honest way to get p-values with so few clusters.

None of the 18 main estimates has a p-value below 0.05. The nearest is polarisation. In years when society was more polarised than usual, median interactions per post were about 10% higher for a one standard deviation rise, with p = 0.09, and about 5% higher per follower, with p = 0.06.

But that estimate depends on 2020, the Covid year, when engagement jumped everywhere. Drop 2020 and the per-post estimate falls to about 4% (p = 0.62) and the per-follower one to about 1% (p = 0.94). Control for followers and it is 4% (p = 0.30). I ran 66 robustness tests in all, and two came in under 0.05, which is roughly what you would expect from chance. I do not treat them as findings.

Followers explain a lot of it

One more simple thing. Pages with more followers get more interactions per post, which is no surprise. For the years where I have follower counts, a 1% rise in followers goes with a rise of about 1.1% in median interactions. If anything moves engagement, it is audience size first.

The caveats

Six countries is a very small sample, so this can only rule out large effects. The democracy indicators are expert ratings and change slowly. Latvia only has politicians’ pages and Estonia is not in my data, so the Baltic states are thinly covered. I left out one Portuguese news portal and a file of Portuguese politician pages. Follower counts at posting only go back to about 2015. And the data end in October 2020.

My conclusion is a cautious one: engagement on party pages is not a usable measure of how democratic a country is. If you want to read political health off Facebook numbers, first check who owns the audience.

The views in this post and the paper are my own.