Installation | Overview | Tokenization | Stemming | Stop Words | 搜索 Index | Index Strategy

Js 搜索: client-side search library

Js 搜索 enables efficient client-side searches of JavaScript and JSON objects. It is ES5 compatible and does not require jQuery or any other third-party libraries.

Js 搜索 began as a lightweight implementation of Lunr JS, offering runtime performance improvements and a smaller file size. It has since expanded to include a rich feature set- supporting stemming, stop-words, and TF-IDF ranking.

Here are some JS Perf benchmarks comparing the two search libraries. (Thanks to olivernn for tweaking the Lunr side for a better comparison!)

• Initial building of search index
• Running a search

If you're looking for a simpler, web-worker optimized JS search utility check out js-worker-search.

---

If you like this project, 🎉 become a sponsor or ☕ buy me a coffee

---

Installation

You can install using either Bower or NPM like so:

npm install js-search
bower install js-search

Overview

At a high level you configure Js 搜索 by telling it which fields it should index for searching and then add the objects to be searched.

For example, a simple use of JS 搜索 would be as follows:

import * as Js搜索 from 'js-search';

var theGreatGatsby = {
  isbn: '9781597226769',
  title: 'The Great Gatsby',
  author: {
    name: 'F. Scott Fitzgerald'
  },
  tags: ['book', 'inspirational']
};
var theDaVinciCode = {
  isbn: '0307474275',
  title: 'The DaVinci Code',
  author: {
    name: 'Dan Brown'
  },
  tags: ['book', 'mystery']
};
var angelsAndDemons = {
  isbn: '074349346X',
  title: 'Angels & Demons',
  author: {
    name: 'Dan Brown',
  },
  tags: ['book', 'mystery']
};

var search = new Js搜索.搜索('isbn');
search.addIndex('title');
search.addIndex(['author', 'name']);
search.addIndex('tags')

search.addDocuments([theGreatGatsby, theDaVinciCode, angelsAndDemons]);

search.search('The');    // [theGreatGatsby, theDaVinciCode]
search.search('scott');  // [theGreatGatsby]
search.search('dan');    // [angelsAndDemons, theDaVinciCode]
search.search('mystery') // [angelsAndDemons, theDaVinciCode]

Tokenization

Tokenization is the process of breaking text (e.g. sentences) into smaller, searchable tokens (e.g. words or parts of words). Js 搜索 provides a basic tokenizer that should work well for English but you can provide your own like so:

search.tokenizer = {
  tokenize( text /* string */ ) {
    // Convert text to an Array of strings and return the Array
  }
};

Stemming

Stemming is the process of reducing search tokens to their root (or "stem") so that searches for different forms of a word will still yield results. For example "search", "searching" and "searched" can all be reduced to the stem "search".

Js 搜索 does not implement its own stemming library but it does support stemming through the use of third-party libraries.

To enable stemming, use the StemmingTokenizer like so:

var stemmer = require('porter-stemmer').stemmer;

search.tokenizer =
	new Js搜索.StemmingTokenizer(
        stemmer, // Function should accept a string param and return a string
	    new Js搜索.SimpleTokenizer());

Stop Words

Stop words are very common (e.g. a, an, and, the, of) and are often not semantically meaningful. By default Js 搜索 does not filter these words, but filtering can be enabled by using the StopWordsTokenizer like so:

search.tokenizer =
	new Js搜索.StopWordsTokenizer(
    	new Js搜索.SimpleTokenizer());

By default Js 搜索 uses a slightly modified version of the Google History stop words listed on www.ranks.nl/stopwords. You can modify this list of stop words by adding or removing values from the Js搜索.StopWordsMap object like so:

Js搜索.StopWordsMap.the = false; // Do not treat "the" as a stop word
Js搜索.StopWordsMap.bob = true;  // Treat "bob" as a stop word

Note that stop words are lower case and so using a case-sensitive sanitizer may prevent some stop words from being removed.

Configuring the search index

There are two search indices packaged with js-search.

Term frequency–inverse document frequency (or TF-IDF) is a numeric statistic intended to reflect how important a word (or words) are to a document within a corpus. The TF-IDF value increases proportionally to the number of times a word appears in the document but is offset by the frequency of the word in the corpus. This helps to adjust for the fact that some words (e.g. and, or, the) appear more frequently than others.

By default Js 搜索 supports TF-IDF ranking but this can be disabled for performance reasons if it is not required. You can specify an alternate I搜索Index implementation in order to disable TF-IDF, like so:

// default
search.searchIndex = new Js搜索.TfIdf搜索Index();

// 搜索 index capable of returning results matching a set of tokens
// but without any meaningful rank or order.
search.searchIndex = new Js搜索.Unordered搜索Index();

Configuring the index strategy

There are three index strategies packaged with js-search.

PrefixIndexStrategy indexes for prefix searches. (e.g. the term "cat" is indexed as "c", "ca", and "cat" allowing prefix search lookups).

AllSubstringsIndexStrategy indexes for all substrings. In other word "c", "ca", "cat", "a", "at", and "t" all match "cat".

ExactWordIndexStrategy indexes for exact word matches. For example "bob" will match "bob jones" (but "bo" will not).

By default Js 搜索 supports prefix indexing but this is configurable. You can specify an alternate IIndexStrategy implementation in order to disable prefix indexing, like so:

// default
search.indexStrategy = new Js搜索.PrefixIndexStrategy();

// this index strategy is built for all substrings matches.
search.indexStrategy = new Js搜索.AllSubstringsIndexStrategy();

// this index strategy is built for exact word matches.
search.indexStrategy = new Js搜索.ExactWordIndexStrategy();