ml_lda {sparklyr} | R Documentation |
Fit a Latent Dirichlet Allocation (LDA) model to a Spark DataFrame.
ml_lda(x, features = dplyr::tbl_vars(x), k = length(features), alpha = (50/k) + 1, beta = 0.1 + 1, ml.options = ml_options(), ...)
x |
An object coercable to a Spark DataFrame (typically, a
|
features |
The name of features (terms) to use for the model fit. |
k |
The number of topics to estimate. |
alpha |
Concentration parameter for the prior placed on documents' distributions over topics. This is a singleton which is replicated to a vector of length |
beta |
Concentration parameter for the prior placed on topics' distributions over terms. For Expectation-Maximization optimizer value should be > 1.0 and by default |
ml.options |
Optional arguments, used to affect the model generated. See
|
... |
Optional arguments. The |
The topics' distributions over terms are called "beta" in the original LDA paper by Blei et al., but are called "phi" in many later papers such as Asuncion et al., 2009.
For terminology used in LDA model see Spark LDA documentation.
Original LDA paper (journal version): Blei, Ng, and Jordan. "Latent Dirichlet Allocation." JMLR, 2003.
Asuncion et al. (2009)
Other Spark ML routines: ml_als_factorization
,
ml_decision_tree
,
ml_generalized_linear_regression
,
ml_gradient_boosted_trees
,
ml_kmeans
,
ml_linear_regression
,
ml_logistic_regression
,
ml_multilayer_perceptron
,
ml_naive_bayes
,
ml_one_vs_rest
, ml_pca
,
ml_random_forest
,
ml_survival_regression