NLP Model Detect/Rate Important Sentences utilizing NLP Intent Text Classification using NLP.js in Nodejs

If you are interested in detecting or rating sentences in a text without relation to the complete text continue reading. This means here we talking about one sentence and the model should detect if this one is relevant or not. We are talking about an online meeting domain and therefore the context is very biased. However, those setting are adjustable, and if you are looking for an easy multi-language solution follow this guide. Here is a lookout screenshot:

For the technical background, we are going to use nlp.js because it provides high precision and is simple to implement. However, you can of cause also use tensorflow.js for intent text classification.

Source Code

See here the index.js code for NLP.js

const { dockStart } = require("@nlpjs/basic");
var path = require('path');
let nlpIntentProcess;
async function intentTrainModel() {
  console.log("Server - NLP Intent model training with corpus file start.");
  const dock = await dockStart({use: ['Basic']})
  const nlp = dock.get('nlp');
  await nlp.addCorpus('./src/nlp/nlpjs/corpus.json')
  let modelPathFilename = path.join(__dirname, "./src/nlp/nlpjs", "model.nlp");
  //console.log(modelPathFilename);
  await nlp.train();
  nlp.save(modelPathFilename);
  console.log("Server - NLP Intent model training with corpus file end.");
}
async function intentQueryModel() {
  const response = await nlpIntentProcess.process('en', 'Lets focus on the tasks next.');
  console.log("Server - NLP Intent Model Response: ", response);
}
async function intentLoadModel() {
  const dock = await dockStart({use: ['Basic']})
  nlpIntentProcess = dock.get('nlp');
  await nlpIntentProcess.addCorpus('./src/nlp/nlpjs/corpus.json');
  let modelPathFilename = path.join(__dirname, "src/nlp/nlpjs", "model.nlp");
  nlpIntentProcess.load(modelPathFilename)
}
(async() => {
  // train model
  await intentTrainModel();
  // load trained model
  await intentLoadModel();
  // query model response
  await intentQueryModel();
})();

Getting started

Use the following commands to start:

npm install
npm run

In the first step, the example code will train a model base on the corpus file and then save the model in the src/nlp/nlpjs/ folder.

Afterward, the model is loaded and then we access the model to detect an intent to classify if the sentence is relevant. If soo the agent.relevant class is going to be returned.

Here is a screenshot of the training process after the start of the tool.

Here is a screenshot of the response of the loaded NLP model, which we just trained.

We can see that the sentence “Let's focus on the task next.” is classified to 90,578% to the agent.relevant class. This means the sentence is important.

And we could use this to identify if the context is relevant or not.

For this, we are using a corpus file, which contains specific words to identify thought rule-based reasoning and other mechanisms to identify if it is a relevant context. This file maybe required to be adjusted to fit into other domains to to provide more specific detections.

Find here a small example of the code inside of the corpus.json

{
  "name": "Corpus",
  "locale": "en-US",
  "data": [
    {
      "intent": "agent.relevant",
      "utterances": [
        "it is important",
        "..."
      ],
      "answers": [
        "create a talk"
      ]
    }
  ]
}

Useful stuff

the source code and all the required files are also available under a git repo. If you like you can have a look here:

GitHub - FranzTscharf/simple-nlp.js-example-text-classification: Simple nlp.js Example Text…

I hope the article helped and happy coding.