Document Type

Article

Publication Date

12-12-2018

Identifier

PMCID: PMC6292049 DOI: 10.1186/s12859-018-2556-9

Abstract

BACKGROUND: Unsupervised clustering represents one of the most widely applied methods in analysis of high-throughput 'omics data. A variety of unsupervised model-based or parametric clustering methods and non-parametric clustering methods have been proposed for RNA-seq count data, most of which perform well for large samples, e.g. N ≥ 500. A common issue when analyzing limited samples of RNA-seq count data is that the data follows an over-dispersed distribution, and thus a Negative Binomial likelihood model is often used. Thus, we have developed a Negative Binomial model-based (NBMB) clustering approach for application to RNA-seq studies.

RESULTS: We have developed a Negative Binomial Model-Based (NBMB) method to cluster samples using a stochastic version of the expectation-maximization algorithm. A simulation study involving various scenarios was completed to compare the performance of NBMB to Gaussian model-based or Gaussian mixture modeling (GMM). NBMB was also applied for the clustering of two RNA-seq studies; type 2 diabetes study (N = 96) and TCGA study of ovarian cancer (N = 295). Simulation results showed that NBMB outperforms GMM applied with different transformations in majority of scenarios with limited sample size. Additionally, we found that NBMB outperformed GMM for small clusters distance regardless of sample size. Increasing total number of genes with fixed proportion of differentially expressed genes does not change the outperformance of NBMB, but improves the overall performance of GMM. Analysis of type 2 diabetes and ovarian cancer tumor data with NBMB found good agreement with the reported disease subtypes and the gene expression patterns. This method is available in an R package on CRAN named NB.MClust.

CONCLUSION: Use of Negative Binomial model based clustering is advisable when clustering over dispersed RNA-seq count data.

Journal Title

BMC bioinformatics [electronic resource]

Volume

19

Issue

1

First Page

474

Last Page

474

MeSH Keywords

Cluster Analysis; Female; Humans; Male; Models, Statistical; Normal Distribution; Transcriptome

Keywords

Clustering; EM algorithm; Gaussian mixture model; Model-based; Negative binomial; RNA-seq

Comments

Grant support

This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Publisher's Link: https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2556-9

Download

Share

COinS