$ ./guess_the_languageAI & ML
AI & ML · medium · #37

What IDF value does "the" get?

corpus: 4 documents
term "the" appears in all 4
idf(t) = log(N / df(t))
  1. 0
  2. 1
  3. 4
  4. 2
log(4 / 4) = log(1) = 0, so a word found in every document carries no TF-IDF weight.
Play AI & ML ▸