Saltelli, A., Nardo, M., Saisana, M., & Tarantola, S. (2005). Composite Indicators — The Controversy and the Way Forward. OECD Statistics Working Papers. doi:10.1787/882502627488
Relevance to Cittopia: The methodological backbone of how we aggregate 9 dimensions into The Pulse. We use geometric mean within dimensions per their recommendation for penalising imbalance. The Listening Score's bias-disclosure follows their sensitivity-analysis framework.
aggregationmethodologyfoundational
Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., & Giovannini, E. (2008). Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD / JRC. doi:10.1787/9789264043466-en
Relevance to Cittopia: The single most-cited reference work on composite indicator construction. Every methodology decision in The Pulse v0.1 maps to a step in this Handbook (normalisation, weighting, aggregation, robustness). v0.2 will explicitly cite which step each decision implements.
handbookcanonicalJRC
OECD & JRC. (2024). Handbook on Constructing Composite Indicators (2nd edition). OECD Publishing. www.oecd.org
Relevance: Updated edition that adds machine-learning-era guidance. Particularly relevant: their treatment of AI-derived weighting (which we deliberately avoid in v0.1 — too opaque) and updated sensitivity analysis protocols.
handbookupdated
Becker, W., Saisana, M., Paruolo, P., & Vandecasteele, I. (2017). Weights and Importance in Composite Indicators: Closing the Gap. Ecological Indicators, 80, 12–22. doi:10.1016/j.ecolind.2017.03.056
Relevance: Shows that weights in composite indicators don't behave the way naive readers assume — a low weight doesn't necessarily mean low importance once correlations are accounted for. Informs our decision to publish dimension correlations alongside Pulse scores in v0.2.
weightingcorrelations
Munda, G. (2012). Choosing Aggregation Rules for Composite Indicators. Social Indicators Research, 109, 337–354. doi:10.1007/s11205-011-9911-9
Relevance: The systematic comparison of arithmetic vs geometric vs other aggregation methods. We chose geometric mean within dimensions partly on Munda's analysis of substitutability — geometric mean correctly refuses to let a strong dimension fully compensate for a catastrophically weak one.
aggregationgeometric mean
JRC (Joint Research Centre). (ongoing). Competence Centre on Composite Indicators and Scoreboards. European Commission. composite-indicators.jrc.ec.europa.eu
Relevance: The reference institution for European composite indicators. The JRC Statistical Audit is the standard we're targeting for The Pulse v0.2 — formal external review of our normalisation, weighting, and aggregation choices.
auditEU institution
Greco, S., Ishizaka, A., Tasiou, M., & Torrisi, G. (2019). On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness. Social Indicators Research, 141(1), 61–94. doi:10.1007/s11205-017-1832-9
Relevance: Recent (post-Handbook) systematic review of composite-index methodology debates. Important for understanding why no composite indicator is "neutral" — every methodological choice encodes a political choice. We treat this as a constraint on transparency, not a reason to avoid composites entirely.
reviewmethodology debate
ISO. (2018). ISO 37120:2018 — Sustainable cities and communities: Indicators for city services and quality of life. International Organization for Standardization. www.iso.org/standard/68498.html
Relevance: The international standard for city-level indicators. The Pulse's Services and Mobility dimensions draw heavily from ISO 37120's themes. We deviate by emphasising open-data confidence as a separate dimension, which ISO treats only as a data-availability footnote.
ISOcity standards
ISO. (2019). ISO 37122:2019 — Sustainable cities and communities: Indicators for smart cities. International Organization for Standardization. www.iso.org/standard/69050.html
Relevance: Smart-city-specific indicator standard. Influences our Governance dimension (which includes digital service availability) and Data Confidence dimension. We're more skeptical than ISO 37122 about claiming "smartness" without measuring back-end accountability.
ISOsmart cities
UN-Habitat. (2016). SDG Indicator 11.3.1 Metadata — Ratio of Land Consumption Rate to Population Growth Rate. United Nations. unhabitat.org
Relevance: UN-Habitat's methodology for SDG 11 (Sustainable Cities). Our Environment and Climate dimensions include several SDG 11 sub-indicators verbatim. UN-Habitat's choice to NOT publish a single composite SDG-11 score (preferring sub-indicator dashboards) is a respected counter-position we considered and rejected for citizen-readability reasons.
SDG 11UN-Habitat
OECD. (2020). Cities in the World: A New Perspective on Urbanisation. OECD Urban Studies, OECD Publishing. doi:10.1787/d0efcbda-en
Relevance: The OECD's Functional Urban Area (FUA) definition that we use as the geographic granularity for cross-country comparison. FUAs solve the "is Berlin its city limits or its commuting catchment?" problem in a way administrative boundaries can't.
FUAOECD
Eurostat. (ongoing). Urban Audit — Cities and Greater Cities. European Commission. ec.europa.eu/eurostat/web/cities
Relevance: The primary harmonised data source for European cities. The Pulse pulls roughly half its sub-indicators from the Urban Audit. When Urban Audit lags (some indicators on 2-3 year cycle), we supplement with city open-data portals and flag the lag explicitly in our data confidence score.
primary data sourceEurostat
ESPON. (2019). European Territorial Reference Framework. European Observation Network for Territorial Development and Cohesion. www.espon.eu
Relevance: ESPON's typologies for European territories (metropolitan, intermediate, rural) inform how we categorise cities for like-with-like comparison. Comparing Warszawa to Bürgenstock isn't useful; ESPON typologies tell us what "comparable" actually means.
ESPONterritorial typology
SDSN & Bertelsmann Stiftung. (2023). European Cities SDG Index 2023. Sustainable Development Solutions Network. www.sdgindex.org
Relevance: The most direct comparator to The Pulse. Their academic rigour and ours' citizen-readability are complementary, not competing. We use several SDSN indicators as triangulation inputs. The systematic comparison is at
/insights/compare-sdsn.
peer indexacademic
Fung, A. (2006). Varieties of Participation in Complex Governance. Public Administration Review, 66, 66–75. doi:10.1111/j.1540-6210.2006.00667.x
Relevance: Fung's "Democracy Cube" framework — who participates, how they communicate, how decisions get made — directly informs the Listening Score's components. Our Reach component maps to Fung's "who participates" axis; Loop-closure maps to "how decisions follow from communication."
foundationalframework
Arnstein, S. R. (1969). A Ladder of Citizen Participation. Journal of the American Institute of Planners, 35(4), 216–224. doi:10.1080/01944366908977225
Relevance: The canonical typology of participation depth (from manipulation through tokenism to citizen control). Cittopia explicitly distinguishes performative participation (low Listening Score) from substantive participation (high Listening Score) using Arnstein's ladder as the underlying conceptual frame.
canonicalhistorical
IAP2 (International Association for Public Participation). (2018). Spectrum of Public Participation. IAP2 Federation. www.iap2.org/page/pillars
Relevance: The five-level practitioner framework (Inform → Consult → Involve → Collaborate → Empower) widely used by city governments. Our Loop-closure component grades cities on whether the actual practice matches the level the city claims to operate at.
practitioner framework
Gastil, J., & Levine, P. (2005). The Deliberative Democracy Handbook: Strategies for Effective Civic Engagement in the 21st Century. Jossey-Bass. Wiley reference
Relevance: Methodological depth on what "good participation process" actually looks like beyond the technology. Influences our Diversity component (representative sampling) and Response Time component (timely feedback as a deliberative requirement).
handbook
Smith, G. (2009). Democratic Innovations: Designing Institutions for Citizen Participation. Cambridge University Press. doi:10.1017/CBO9780511609848
Relevance: Smith's comparative analysis of participation institutions (citizens' assemblies, participatory budgeting, e-democracy platforms). The Pulse's Civic Participation dimension uses Smith's institutional taxonomy when classifying what kind of participation a city actually has.
comparativeinstitutions
Habermas, J. (1996). Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. MIT Press. MIT Press
Relevance: The philosophical underpinning of deliberative democracy. Cited because the Listening Score's Decision-trace component (whether decisions cite citizen reasoning) is directly downstream of Habermas's discourse-ethics framework — public deliberation only legitimises decisions if the deliberation actually informs them.
philosophyfoundational
OIDP (Observatorio Internacional de la Democracia Participativa). (ongoing). Best Practice Database on Citizen Participation. UCLG (United Cities and Local Governments). oidp.net
Relevance: Real case studies of municipal participation from across the world. We use OIDP-documented practices (Helsinki Kerro Kantasi, Madrid Decide Madrid 2015–2019) as scoring anchors for Listening Score band 80+ in our methodology.
case studiesUCLG
OECD. (2020). Innovative Citizen Participation and New Democratic Institutions: Catching the Deliberative Wave. OECD Publishing. doi:10.1787/339306da-en
Relevance: The most comprehensive comparative study of deliberative-democracy implementations to date (~300 cases). Validates our choice to weight Loop-closure heavily — the OECD finds that loop-closure is the single strongest predictor of citizens' continued willingness to participate.
OECDcomparative
Fishkin, J. S. (2009). When the People Speak: Deliberative Democracy and Public Consultation. Oxford University Press. Oxford UP
Relevance: Fishkin's experimental work on Deliberative Polling. Methodologically rigorous about the difference between "input volume" and "informed input" — informs our Diversity component (which penalises consultations dominated by a single demographic).
experimental
Convention Citoyenne pour le Climat. (2020). Les 149 propositions de la Convention Citoyenne pour le Climat. Government of France. conventioncitoyennepourleclimat.fr
Relevance: A landmark European citizens' assembly. We cite the partial loop-closure outcome (some proposals adopted, most modified or rejected without clear reasoning) as a real-world case showing why our Loop-closure component matters — a high-profile assembly with weak follow-through scores lower than its reputation suggests.
case studyFrance
European Commission. (2019). Directive (EU) 2019/1024 of the European Parliament and of the Council on open data and the re-use of public sector information. Official Journal of the European Union. EUR-Lex
Relevance: The "Open Data Directive" — legal foundation for every Cittopia data feed. The Directive's High-Value Datasets (HVDs) framework directly informs which datasets we trust as primary sources versus city-portal supplementaries.
EU lawopen data
European Commission. (2024). High-Value Datasets — Implementing Regulation (EU) 2023/138. DG CNECT. digital-strategy.ec.europa.eu
Relevance: Names the six HVD categories all EU public bodies must publish openly (geospatial, environmental, meteorological, statistical, company, mobility). Our Data Confidence dimension's top scoring band requires a city to be HVD-compliant.
HVDDG CNECT
European Commission, DG REGIO. (2021). Cohesion Policy 2021–2027: Investing in Europe. European Commission. ec.europa.eu/regional_policy
Relevance: The €392 billion programme that flows through regional managing authorities (marshal offices in Poland, regional governments elsewhere). Frames our EU Funds Pipeline article's emphasis on regional-level engagement as the highest-leverage path for civic-tech in EU countries.
DG REGIOcohesion policy
European Commission. (2024). Regulation (EU) 2024/1689 — The AI Act. Official Journal of the European Union. EUR-Lex
Relevance: The EU AI Act classifies AI systems used by public authorities as high-risk. Directly informs our "Why don't more European cities use AI for citizen consultations?" framing (Quora answer 3) and constrains how Cittopia integrates LLMs in the Agora product.
AI ActEU law
Eurocities. (ongoing). Pulse Survey of European Cities. Eurocities Network. eurocities.eu
Relevance: Largest peer network of European cities (220+ members). The Eurocities Pulse Survey (no relation to our Pulse indicator) is a triangulation source for governance trends. Their reports cite real positions European city governments take, which informs our scoring bands.
Eurocitiespeer network
Living-in.EU. (ongoing). Urban Data Platform Plus. European Commission, Eurocities, Open & Agile Smart Cities. urban.jrc.ec.europa.eu
Relevance: The EU's harmonised urban data infrastructure (130+ cities). Cittopia pulls from Urban Data Platform where it has coverage; otherwise from city portals directly. The systematic comparison is at
/insights/compare-numbeo.
data infrastructure
Greenfield, A. (2013). Against the Smart City. Do Projects. Verso (related work)
Relevance: The foundational critique that "smart city" branding obscures political choices behind technological framing. Cittopia explicitly positions itself as civic-tech rather than smart-city tech because of this lineage — we measure outcomes for citizens, not deployed sensors.
criticalfoundational
Mattern, S. (2017). A City Is Not a Computer. Places Journal. placesjournal.org
Relevance: Mattern's critique of urbanism-as-computation directly informs our refusal to claim algorithmic objectivity for The Pulse. We publish methodological choices openly precisely because we agree with Mattern that there's no neutral measurement of a city.
criticalrequired reading
Kitchin, R. (2014). The Real-Time City? Big Data and Smart Urbanism. GeoJournal, 79(1), 1–14. doi:10.1007/s10708-013-9516-8
Relevance: Kitchin's analysis of how "smart" framings can entrench inequality. Informs our deliberate choice to include an Equity dimension as a first-class component of The Pulse — equity isn't a footnote to smart-city performance, it's a dimension of it.
equitycritical
Townsend, A. (2013). Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia. W.W. Norton. W.W. Norton
Relevance: Townsend's middle-ground position: smart-city technology is neither salvation nor scam, but a tool that civic-tech (small-c) can wield against vendor capture. Informs our positioning as anti-vendor-lock civic infrastructure.
middle-ground
Margetts, H., John, P., Hale, S., & Yasseri, T. (2015). Political Turbulence: How Social Media Shape Collective Action. Princeton University Press. Princeton UP
Relevance: Empirical study of how digital platforms reshape who participates in collective action. Directly informs the Listening Score's Diversity component — without explicit demographic adjustment, digital channels systematically over-represent the already-loud.
empiricalplatforms
Coleman, S., & Blumler, J. G. (2009). The Internet and Democratic Citizenship: Theory, Practice and Policy. Cambridge University Press. doi:10.1017/CBO9780511626683
Relevance: The systematic case for why digital citizen engagement requires institutional design, not just platform deployment. Informs our positioning of the Agora as institutional infrastructure rather than a "social network for cities."
institutional design
OECD. (2024). OECD AI Principles — 2024 Update. OECD. oecd.ai
Relevance: The OECD's five-principle framework for trustworthy AI (inclusive growth, human-centered values, transparency, robustness, accountability). The Agora's AI features are built against these principles, particularly transparency and accountability.
OECDAI principles
Pomerantsev, P., & Helmus, T. C. (Eds.). (2024). Defending Democracy in the Age of AI. Carnegie Endowment for International Peace / various. Carnegie Endowment
Relevance: Contemporary analysis of AI's effects on democratic processes. Informs our explicit position that AI in citizen consultation is augmentation, never automation — a position we make publicly in
the Listening Score deep-dive.
contemporary
OECD. (2024). OECD Survey on Drivers of Trust in Public Institutions — 2024 Results. OECD Publishing. doi:10.1787/9a20554b-en
Relevance: The most rigorous comparative survey of institutional trust in Europe. Identifies responsiveness as one of the top three drivers of trust — direct empirical support for the Listening Score's policy relevance.
empiricalOECD
Bouckaert, G., & Van de Walle, S. (2003). Comparing Measures of Citizen Trust and User Satisfaction as Indicators of "Good Governance". International Review of Administrative Sciences, 69(3), 329–343. doi:10.1177/0020852303693003
Relevance: Critical methodological paper on the gap between citizen satisfaction surveys and actual governance quality. Directly informs our distinction between Numbeo-style perception (what people feel) and Listening-Score-style measurement (what cities do).
methodologymeasurement
Putnam, R. D. (2000). Bowling Alone: The Collapse and Revival of American Community. Simon & Schuster. Simon & Schuster
Relevance: The foundational text on social capital decline. The Pulse's Civic Participation dimension (which contains the Listening Score) is directly informed by Putnam's framework — civic associations, voter turnout, and participation infrastructure are all sub-indicators we track.
social capitalfoundational
Levi, M. (1998). A State of Trust. In Braithwaite, V. & Levi, M. (Eds.), Trust and Governance. Russell Sage Foundation. Russell Sage
Relevance: Levi's framework distinguishes "trustworthy government" from "trusting citizens" — they're related but not identical, and you need different measurements for each. Directly informs why Cittopia tracks both perception (via OECD/Eurostat trust survey integration in v0.2) and measurable behaviour (Listening Score now).
framework
Stiglitz, J., Sen, A., & Fitoussi, J.-P. (2009). Report by the Commission on the Measurement of Economic Performance and Social Progress. CMEPSP / French Government. Eurostat archive
Relevance: The foundational report arguing that GDP alone is an inadequate measure of social progress. Directly informs The Pulse's multi-dimensional approach — economic indicators are one of nine dimensions, not the dominant axis.
foundationalwell-being
OECD. (2020). How's Life? 2020: Measuring Well-being. OECD Publishing. doi:10.1787/9870c393-en
Relevance: OECD's framework for measuring well-being across 11 dimensions, refined over a decade. The Pulse's selection of 9 dimensions overlaps significantly with OECD's framework — we differ mainly by elevating Civic Participation to a peer dimension rather than a sub-component of governance.
OECDwell-being
United Nations. (2015). Transforming Our World: The 2030 Agenda for Sustainable Development. UN General Assembly. sdgs.un.org
Relevance: The 17 Sustainable Development Goals framework. Cittopia explicitly maps each Pulse dimension to a primary SDG (visible on our
SDGs page). We use SDG localisation as our cross-country comparability anchor.
UNSDGs
Sachs, J., Kroll, C., Lafortune, G., Fuller, G., & Woelm, F. (2024). Sustainable Development Report 2024. Cambridge University Press / SDSN. dashboards.sdgindex.org
Relevance: The most rigorous annual ranking of SDG progress by country. SDSN's methodology for the cities-level index draws from this report's country-level methodology. We use it as a sanity check for The Pulse's country-aggregated scores.
SDSNannual
Stevens, S. S. (1946). On the Theory of Scales of Measurement. Science, 103(2684), 677–680. doi:10.1126/science.103.2684.677
Relevance: The foundational paper distinguishing nominal, ordinal, interval, and ratio scales. Critical for understanding what kinds of mathematical operations are and aren't valid on composite indicators. The Pulse uses interval-scaled normalisation; we discuss the limits of this in our methodology.
foundationalscales
Krantz, D., Luce, R., Suppes, P., & Tversky, A. (1971). Foundations of Measurement, Volume I: Additive and Polynomial Representations. Academic Press. Dover reprint
Relevance: The deep mathematical foundations of measurement. Cited to indicate that we know our composite indicator stands on contested mathematical ground. Anyone arguing The Pulse is "objective" hasn't read Krantz et al. — and we don't claim objectivity.
foundationaladvanced
Porter, T. M. (1995). Trust in Numbers: The Pursuit of Objectivity in Science and Public Life. Princeton University Press. Princeton UP
Relevance: Historical and sociological study of how numerical objectivity became a substitute for political judgment. Cited because Cittopia is honest that our scores embed political choices — we just make those choices visible rather than hiding them behind a "data-driven" claim.
epistemologysociology of science
Espeland, W. N., & Stevens, M. L. (2008). A Sociology of Quantification. European Journal of Sociology, 49(3), 401–436. doi:10.1017/S0003975609000150
Relevance: Contemporary analysis of how quantification reshapes the things it measures (reactivity). We acknowledge in our methodology that publishing The Pulse will, over time, change the things it measures — and we welcome this as a feature, not a bug, provided the underlying methodology stays publicly defensible.
reactivitycontemporary