A closer look into the resolutions of the World Health Assembly

Using NLP techniques, network analysis and other data science tools, uncover patterns behind the resolutions adopted by the sessions of the World Health Assembly.
Data science
Health

Introduction

In general, WHA is held once every year in May, with the first session being held in 1948 and the 75th session in 2022. Each WHA resolutions is identified by a unique ID of the form WHA{X}.{Y} where X is the session number and Y the resolution number in that session. For example, WHA75.3 corresponds to the 3rd resolution from the 75th session. Each resolution concerns a specific topic, and contains many “clauses” that recognize and acknowledge identified problems, issues and concerns, and suggest, recommend, direct various countries, organization and bodies both within and outside the World Health Organization (WHO) to work towards solutions. For example, these are the first few clauses of WHA73.1 from 2020’s session, titled “COVID-19 response”.

Example resolution: WHA73.1 COVID-19 response

In this project, I sought to

  1. create a dataset of WHA resolutions over the years,
  2. identify topics covered by WHA resolutions, and track their time evolution,
  3. uncover relationships between WHA resolutions and visualize them as a network,
  4. analyze how different countries have contributed to WHO’s budget over the years, and
  5. identify new questions of interest concerning this dataset and other related issues to take up as future projects.

Pipeline

Create a dataset of WHA resolutions

To create a clean dataset of WHA resolutions, I followed these general steps:

  1. I first obtained the PDF files containing the resolutions from all the WHA sessions so far from the WHO website. I also obtained the titles of these resolutions.
  2. Using the package PyPDF2, I extracted text from these PDF files and created one text file per resolution.
  3. For the above two steps, all resolutions till 2019 have one PDF file per resolution, but 2020-2022 have one PDF file containing all the resolutions. So, to extract one text file per resolution for 2020-2022 as well, and to clean up the text files (get rid of extra spaces, unwatned special characters, headers and footers that are part of scanned PDFs from some years and have been unwantedly retained in the text files, and references of the form COVID-19 response1 from WHA73.1’s title), we run a script that cleans the text files and stores one text file per resolution.

Illustrating the steps from a scanned PDF to a clean TEXT file

Having done these steps, we end up with a list of text files (one per resolution) for each WHA session. We can read these into Python, and see all the information (resolution number of the form WHA{X}.{Y}, title of the resolution, year of the session and the clean text of the resolution) of all the resolutions.

number title year text
0 WHA1.10 Regarding the instruction in statistics in med... 1948 s WHEREAS satisfactory health statistics are d...
1 WHA1.2 Resolution 2: regarding the use of the Interna... 1948 WHEREAS it is essential that the International...
2 WHA1.3 Resolution 3: regarding the use of lists of se... 1948 WHEREAS this Conference has adopted for the Si...
3 WHA1.1 Resolution 1: regarding adoption of the Intern... 1948 WHEREAS in recognition of the general trend of...
4 WHA1.4 Resolution 4: regarding the preparation of a l... 1948 REAS the Intermediate List of 150 causes does ...
... ... ... ... ...
3157 WHA75.16 Agreement between the World Health Organizatio... 2022 Having considered the report on the proposed a...
3158 WHA75.17 Human resources for health 2022 Having considered the report by the Director-G...
3159 WHA75.18 Outcome of the SIDS Summit for Health: For a H... 2022 Having considered the consolidated report by t...
3160 WHA75.19 Well-being and health promotion 2022 Having considered the consolidated report by t...
3161 WHA75.20 The global health sector strategies on, respec... 2022 Having considered the consolidated report by t...

3162 rows × 4 columns

Note that some of the words in the first session’s resolutions have been cut off at the beginning. There are indeed a few other resolutions where such unwanted errors and typos are present. A common type of typo is a missing space between words. However, these aren’t very common, so I left them as they are.

First look at topics and their time evolution

To get a taste of the topics of WHA resolutions and their evolution over time, I chose a few topics of interest like “Sustainability”, “HIV”, “Climate Change”, “COVID-19” etc. and checked how often short phrases related to each of these topics are present in resolutions from different years.

Identification of topics and grouping by their time evolution

To uncover more topics covered by WHA resolutions, and use them to find relationships between WHA resolutions, I looked towards natural language processing and network analysis. I used TF-IDF vectorization to find keywords/topics for each resolution and built a network of WHA resolutions by comparing these identified keyword lists.

I won’t go into many details behind the natural language processing pipeline and will only briefly summarize it. Each resolution is associated with two strings - title and text. For both of these strings, I performed the usual NLP pre-processing steps like tokenization, removal of stop words, and additional text cleaning like removing special characters and numbers. Then, I also stemmized all the tokens (words) in these strings so that different words like “sustainability”, “sustainable”, “sustain”, “sustains”, “sustaining”, “sustained” were all reduced to a common form “sustain”. This collection of cleaned and stemmized documents (each resolution is a document) is called a corpus. We have two stemmized corpuses - corpus_title and corpus_text. For the following explanation, I’ll refer to corpus_text as the corpus, and I used only the corpus_text in this study.

So, the corpus is a collection of documents (resolutions), and each document is a collection of words (after having removed special characters, numbers and stop words which are commonly occuring English words like “the”, “a” etc.). For each word in the corpus, I calculated its document frequency, which is the fraction of documents containing the word. For each pair of word and document, I also calculated the TF-IDF score, a score that is a combination of term frequency and inverse document frequency. What is important to note is that TF-IDF score is intended to reflect how important a word is to a document.

First, we’ll check these two values for all the words in the corpus, and also check how the vocabulary size (no. of words) changes as we modify the minimum document frequency and maximum document frequency allowed.

Words with a very high document frequency are probably those that aren’t very relevant to the specific topic of the resolution, and are just very often used in WHA resolutions. So, using a maximum allowed document frequency is a good way to exclude those words from further analysis. One can use a maximum document frequency of either 0.05 or 0.1. This means that one excludes words that are present in more than 5% (around 150 resolutions) or than 10% (around 300 resolutions) of documents.

One can also put a minimum document frequency, which will require words to be present in at least a specific number of resolutions, for example 5, 10 or 50. This will get rid of super-specific words, typos and other such errors. Putting these restrictions on what words to consider, I then proceeded to calculate the TF-IDF score for each document-word pair. Remember, the TF-IDF score is intended to quantify a word’s importance in a document. Then, for each document, I selected the top n_w words with the highest TF-IDF scores.

I counted both single words (monograms) and two-word phrases (bigrams) - they will be called topics together. Using minimum document frequency as 50 resolutions, maximum document frequency as 5% (around 150 resolutions) and obtaining top n_w = 10 topics from each document, I obtained a total of 1136 unique topics.

Here are a few examples.

WHA2.27: Insulin
Topics: gener studi, advis, explor, manufactur, possibl, programm adopt

WHA4.48: WHO seals
Topics: supplement, ona, assembl suspens, toit, inconform, suggest, organ request, gener submit, fund health, gener implement

WHA6.56: Special benefit for retiring Director-General
Topics: staff pension, pension, save, work capit, capit fund, capit, contract, occur, budget estim, sum

WHA15.34: Malaria eradication special account
Topics: malaria, erad, special account, erad programm, cash, voluntari contribut, transit, programm request, laid, regularli

WHA23.18: Headquarters accommodation: report on financing
Topics: headquart, real estat, estat, expens, real, sum, reflect, main, share, outcom

WHA29.37: Amendments to the rules of procedures of the World Health Assembly
Topics: point, presid, deleg, rule procedur, associ member, add, appeal, aris, follow, statement

WHA29.22: Report on the world health situation
Topics: health situat, publish, content, analysi, programm work, gener programm, improv qualiti, subsequ health, report request, russian

WHA31.26: Health and medical assistance to Lebanon
Topics: lebanon, creat, assist provid, injuri, assembl mind, mainten, previou resolut, organ health, peac secur, express appreci

WHA32.22: Collaboration with the United Nations system: cooperation with newly independent and emerging States in Africa: liberation struggle in Southern Africa
Topics: assembl implement, elect, health year, independ, resolut health

WHA34.33: Collaboration with the United Nations system: cooperation with newly independent and emerging States in Africa: special programme of cooperation with the Republic of Chad
Topics: chad, special programm, spirit, inclos, technic financi, health situat, programm request, invit member, possibl, region director

WHA35.13: Transfer of the Regional Office for the Eastern Mediterranean
Topics: advantag, gener prepar, effect implement, goal health, submit health, opinion, resolut ebr, maximum, believ, implement health

WHA36.17: Real estate fund and Headquarters accommodation
Topics: real estat, estat, real, headquart, main, region offic, construct, casual incom, casual, water

WHA46.12: Assessment of the Czech Republic and the Slovak Republic
Topics: assess rate, nation scale, rate establish, latest, state concern, fix, replac, budgetari, share, scale assess

WHA47.1: Rights and privileges of South Africa
Topics: south africa, south, grant, activ health, democrat, elect, instal, anew, constitut health, fulfil

WHA49.21: Collaboration within the United Nations system and with other intergovernmental organizations: strengthening of the coordination of emergency humanitarian assistance
Topics: humanitarian, document rec, advocaci, disast, prepared, rec, draw attent, anew, assembl resolut, defin

WHA54.9: Assessment of the Federal Republic of Yugoslavia
Topics: yugoslavia, feder, scale assess, assembl establish, nation scale, latest, principl establish, confirm

WHA71.12: Salaries of staff in ungraded positions and of the Director-General
Topics: salari, net, annum, remuner, board regard, note recommend, gener establish, gener region, region director, post

WHA75.10: Status of collection of assessed contributions, including Member States in arrears in the payment of their contributions to an extent that would justify invoking Article 7 of the Constitution of the World Health Organization
Topics: constitut health, open, arrear, invok articl, justifi, invok, vote privileg, suspend, privileg, cameroon

Monograms + bigrams: topics and their time evolution

To visualize the found topics and their time evolution of occurance in WHA resolutions, I clustered the topics according to their time trajectory of counts and visualized it as a heatmap. Topics with similar trajectories are clustered together for easier visualization.

1-gram + 2-gram clustered heatmap of topics

Topic trajectories grouped into clusters

The heatmap contains a lot of topics but we see that topics can be put into groups according to the similarity in their time evolution. I performed this grouping quantitatively by employing k-means clustering. Taking a matrix of presence/absence of topics in resolutions each year, I performed k-means clustering for a range of number of clusters to find the optimal number of clusters (elbow point). Then I performed k-means clustering for the optimal number of clusters (32), and visualized the time trajectories of each of these clusters; I also mention some of the topics in each cluster.

Topics grouped according to similarity in time evolution

On the y-axis, I plotted a normalized value that reflects the relative occurance of topic words in resolutions by year. Each topic has a different normalization factor, so don’t compare across topics; the plots only reflect relative change over years for each topic.

Relationships among WHA resolutions

Next, to find relationships among WHA resolutions, I startd from the same TF-IDF based topics for each resolution. Then for each pair of resolutions, I added an “edge” if they share a sufficient number of topics. This way, I generated a network in which nodes are made up of WHA resolutions and edges depict resolution pairs sharing common topics.

The set of parameters used to generate the topics and form the edges of the network are crucial towards the characteristics of the network. For example, the parameters we discussed before - minimum document frequency, maximum document frequency, n-grams used, number of topics per resolution - all matter. Fixing the minimum document frequency to 50 documents, maximum document frequency to 10%, using monograms and bigrams, below I varied (a) the number of topics per resolution and (b) minimum number of shared topics to form an edge, to generate a network of WHA resolutions.

For each set of parameters, I used the Louvain method for community detection on the resulting network, and calculated a set of metrics to assess the quality of detected communities (partitions). For example, modularity which is the value optimized in the Louvain’s method, measures the density of links inside communities compared to links between communities, is one such metric. A high modularity reflects a good partitioning. However, a lot of nodes (resolutions) with no edges with any other nodes, i.e. resolutions that are in their own partition and aren’t associated with any other resolutions, will result in a high modularity.

To correct for this, I also calculated other metrics - number of partitions, number of non-isolated partitions (partitions with more than one resolution in them), fraction of resolutions in non-isolated partitions, and average number of resolutions per non-isolated partition. Among these, fraction of resolutions in non-isolated partitions is a metric that one would like to be high - a lot of resolutions are grouped with one other resolution into some partition this way. As one would expect, more topics per resolution increases this metric, and more common topics to form an edge decreases this metric.

Modularity and the fraction of resolutions in non-isolated partitions follow opposite trends. To balance these two forces, both of which we want to be high, I defined a goodness score of partitioning as their product of these two metrics. I found that this goodness of partition is the highest (0.76) when the number of topics per resolution is 5 and the minimum number of common topics for an edge is 2. So I used those parameters to generate the WHA network.

Text(0.5, 210.1666666666666, 'Minimum topics for edge')

Network of WHA resolutions

WHA resolution network with nodes colored by the partition they belong to, and topics of the partitions.

Time evolution of topics found via WHA resolution network

From the Louvain partitions of the WHA resolution network, I found the most frequent topics in the resolutions belonging to each partition and assigned them as the “partition topics”.

Partition topics and their time evolution

For each topic, on the y-axis, I plotted the fraction of documents per year that mention the topic.

Scale of assessments

I also extracted the scale of assessments every ten years, specifically for the years 1962, 1972, 1982, 1992, 2002, 2012 and 2022. These denote the percentage contributions to the WHO budget by each country. I visualized these on a heatmap and clustered countries that have contributed similarly over the years into groups.

There seem to 4-6 groups of countries according to their level of contributions + changes in their contributions. The USA, China, Canada, Italy, Japan, Germany, France and the UK have contributed most fractions of the budget.

Relative contributions of different countries to WHO budget over the years, countries grouped by similarity

Lighter colors indicate higher contribution

Scale of assessments

Future directions

  1. Quantify WHA resolution network characteristics over years in n-year periods: understand evolution of network architecture.
  2. Compare and supplement usage of resolution titles to such above analysis.
  3. Analyze citation network of WHA resolutions.
  4. Explore different parameter sets (min/max document frequency etc.) to find more general/specific topics, less frequent topics and so on.

Code

The code is available here on GitHub.