You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Obviously part of something bigger but worth breaking out for reuse.
USAGE
require 'stopwords'
#List all stop words
Stopwords::STOP_WORDS
#Test to see if a token is a stop word
Stopwords.is?('and')
=>true
#Ensures a token is both a 'word' and not a stop word
Stopwords.valid?('vector')
=>true
SPECS
$ rake specs
SANITIZE
Not part of the library but you should probably sanitize tokens before using them (if your tokenize doesn’t already)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THESOFTWARE IS PROVIDED “AS IS”, WITHOUTWARRANTY OF ANYKIND, EXPRESS OR IMPLIED, INCLUDINGBUTNOTLIMITED TO THEWARRANTIES OF MERCHANTABILITY, FITNESSFOR A PARTICULARPURPOSEANDNONINFRINGEMENT. IN NO EVENTSHALLTHEAUTHORS OR COPYRIGHTHOLDERS BE LIABLEFORANYCLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISINGFROM, OUT OF OR IN CONNECTIONWITHTHESOFTWARE OR THEUSE OR OTHERDEALINGS IN THESOFTWARE.