make_sampling_table {keras} | R Documentation

## Generates a word rank-based probabilistic sampling table.

### Description

Generates a word rank-based probabilistic sampling table.

### Usage

    make_sampling_table(size, sampling_factor = 1e-05)

### Arguments

`size` | Int, number of possible words to sample.
`sampling_factor` | The sampling factor in the word2vec formula.

### Details

Used for generating the `sampling_table` argument for `skipgrams()`. `sampling_table[[i]]` is the probability of sampling the word i-th most common word in a dataset (more common words should be sampled less frequently, for balance).

The sampling probabilities are generated according to the sampling distribution used in word2vec:

`p(word) = min(1, sqrt(word_frequency / sampling_factor) / (word_frequency / sampling_factor))`

We assume that the word frequencies follow Zipf's law (s=1) to derive a numerical approximation of frequency(rank):

`frequency(rank) ~ 1/(rank * (log(rank) + gamma) + 1/2 - 1/(12*rank))`

where `gamma` is the Euler-Mascheroni constant.

### Value

An array of length `size` where the ith entry is the probability that a word of rank i should be sampled.

### Note

The word2vec formula is: p(word) = min(1, sqrt(word.frequency/sampling_factor) / (word.frequency/sampling_factor))

### See Also

Other text preprocessing: `pad_sequences`, `skipgrams`, `text_hashing_trick`, `text_one_hot`, `text_to_word_sequence`

[Package keras version 2.2.4 Index]