The paper “Zgli: A Pipeline for Clustering by Compression with Application to Patient Stratification in Spondyloarthritis”, authored by Diogo Azevedo, LASIGE MSc Student, and LASIGE’s integrated researcher André Souto has been published in Sensors, a top-ranked journal (h5 index: 145). The paper co-authors are Ana Maria Rodrigues and Helena Canhão (EpiDoC Unit, The Chronic Diseases Research Centre, and Comprehensive Health Research Center, NOVA Medical School, NOVA University of Lisbon), Alexandra M. Carvalho (Instituto de Telecomunicações and Department of Electrical and Computer Engineering, Instituto Superior Técnico, Universidade de Lisboa).
In this work, Zgli is proposed, a novel Python module that enables the user to compute the normalized compression distance (NCD) between files inside a given folder. Inspired by the CompLearn Linux command line tool, this module iterates on it by providing new text file compressors, a new compression-by-column option for tabular data, such as CSV files, and an encoder for small files made up of categorical data. Clustering methods usually use distances (e.g., Euclidean distance, Manhattan distance) to measure the similarity between objects. The NCD is yet another distance with particular characteristics that can be used to build the starting distance matrix for methods such as hierarchical clustering or K-medoids.
The results demonstrate that compression by column can yield better results than previous methods in the literature when clustering tabular data. Additionally, the categorical encoder shows that it can augment categorical data, allowing the use of the NCD for new data types. One of the advantages is that using this new feature does not require knowledge or context of the data. Furthermore, the fact that the new proposed module is written in Python, one of the most popular programming languages for machine learning, potentiates its use by developers to tackle problems with a new approach based on compression. This pipeline was tested in clinical data and proved a promising computational strategy by providing patient stratification via clusters aiding in precision medicine.
The paper is available here.