Doc2Vec, or Paragraph Vectors, in Deeplearning4j

The main purpose of Doc2Vec is associating arbitrary documents with labels, so labels are required. Doc2vec is an extension of word2vec that learns to correlate labels and words, rather than words with other words. Deeplearning4j’s implentation is intended to serve the Java, Scala and Clojure communities.

The first step is coming up with a vector that represents the “meaning” of a document, which can then be used as input to a supervised machine learning algorithm to associate documents with labels.

In the ParagraphVectors builder pattern, the labels() method points to the labels to train on. In the example below, you can see labels related to sentiment analysis:

  1.     .labels(Arrays.asList("negative", "neutral","positive"))

Here’s a full working example of classification with paragraph vectors:

  1.     public void testDifferentLabels() throws Exception {
  2.         ClassPathResource resource = new ClassPathResource("/labeled");
  3.         File file = resource.getFile();
  4.         LabelAwareSentenceIterator iter = LabelAwareUimaSentenceIterator.createWithPath(file.getAbsolutePath());
  6.         TokenizerFactory t = new UimaTokenizerFactory();
  8.         ParagraphVectors vec = new ParagraphVectors.Builder()
  9.                 .minWordFrequency(1).labels(Arrays.asList("negative", "neutral","positive"))
  10.                 .layerSize(100)
  11.                 .stopWords(new ArrayList<String>())
  12.                 .windowSize(5).iterate(iter).tokenizerFactory(t).build();
  14.         vec.fit();
  16.         assertNotEquals(vec.lookupTable().vector("UNK"), vec.lookupTable().vector("negative"));
  17.         assertNotEquals(vec.lookupTable().vector("UNK"),vec.lookupTable().vector("positive"));
  18.         assertNotEquals(vec.lookupTable().vector("UNK"),vec.lookupTable().vector("neutral"));}

Further Reading