NLP中WordNet里单词的义原集（Synsets）

---

介绍

WordNet是一个大型词汇数据库，存在于NLTK库中，许多自然语言相关的用例都包含多种语言的WordNet。NLTK库有一个称为Synset的接口，允许我们查找WordNet中的单词。动词、名词等被分组到义原集中。

WordNet和义原集

下图显示了WordNet的结构。

在WordNet中，单词之间的关系得以保持。例如，“悲伤”之类的词语含义相似，并且在相似的语境下使用。这些词语在使用过程中可以互换。这类词语被分组到义原集中。每个义原集都相互关联，并具有其自身的含义。由于其概念上的关系，这些义原集相互关联。

WordNet中可能存在的关系是上位词和下位词。

• 上位词（Hypernym）− 上位词是指在语义上更抽象的术语。例如，如果我们考察颜色及其类型（如蓝色、绿色、黄色等）之间的关系，那么“颜色”就是上位词。

• 下位词（Hyponym）− 在上述颜色的例子中，诸如黄色、绿色等单个颜色被称为下位词，它们更具体。

代码实现

import nltk
nltk.download('wordnet')
from nltk.corpus import wordnet
synset = wordnet.synsets('book')[0]
print ("Name of the synset", synset.name())
print ("Meaning of the synset : ", synset.definition())
print ("Example of the synset : ", synset.examples())
print ("Abstract terminology ", synset.hypernyms())
print ("Specific terminology :  ",synset.hypernyms()[0].hyponyms())
print ("hypernerm ( ROOT) :  ", synset.root_hypernyms())

输出

Name of the synset book.n.02
Synset meaning :  physical objects consisting of a number of pages bound together
Synset example :  ['he used a large book as a doorstop']
Abstract terminology  [Synset('publication.n.01')]
Specific terminology :   [Synset('book.n.01'), Synset('collection.n.02'), Synset('impression.n.06'), Synset('magazine.n.01'), Synset('new_edition.n.01'), Synset('periodical.n.01'), Synset('read.n.01'), Synset('reference.n.08'), Synset('reissue.n.01'), Synset('republication.n.01'), Synset('tip_sheet.n.01'), Synset('volume.n.04')]
hypernerm ( ROOT) :   [Synset('entity.n.01')]

使用Pattern库

!pip install pattern
from pattern.en import parse,singularize,pluralize
from pattern.en import pprint
pprint(parse("Jack and Jill went up the hill to fetch a bucket of water", relations=True, lemmata=True))
print("Plural of cat :", pluralize('cat'))
print("Singular of leaves  :",singularize('leaves'))

输出

WORD   TAG    CHUNK   ROLE   ID     PNP    LEMMA    
                                                              
          Jack   NNP    NP      SBJ    1      -      jack     
           and   CC     NP ^    SBJ    1      -      and      
          Jill   NNP    NP ^    SBJ    1      -      jill     
          went   VBD    VP      -      1      -      go       
            up   IN     PP      -      -      PNP    up       
           the   DT     NP      SBJ    2      PNP    the      
          hill   NN     NP ^    SBJ    2      PNP    hill     
            to   TO     VP      -      2      -      to       
         fetch   VB     VP ^    -      2      -      fetch    
             a   DT     NP      OBJ    2      -      a        
        bucket   NN     NP ^    OBJ    2      -      bucket   
            of   IN     PP      -      -      PNP    of       
         water   NN     NP      -      -      PNP    water    
Plural of cat : cats
Singular of leaves  : leaf

在spaCy中使用WordNet接口

!pip install spacy-wordnet
import spacy
import nltk
nltk.download('wordnet')
from spacy_wordnet.wordnet_annotator import WordnetAnnotator 
nlp = spacy.load('en_core_web_sm')
nlp.add_pipe("spacy_wordnet", after='tagger')

spacy_token = nlp('leaves')[0]

print("Synsets : ",spacy_token._.wordnet.synsets())
print("Lemmas : ",spacy_token._.wordnet.lemmas())

print("Wordnet domains:",spacy_token._.wordnet.wordnet_domains())

输出

Synsets :  [Synset('leave.v.01'), Synset('leave.v.02'), Synset('leave.v.03'), Synset('leave.v.04'), Synset('exit.v.01'), Synset('leave.v.06'), Synset('leave.v.07'), Synset('leave.v.08'), Synset('entrust.v.02'), Synset('bequeath.v.01'), Synset('leave.v.11'), Synset('leave.v.12'), Synset('impart.v.01'), Synset('forget.v.04')]
Lemmas :  [Lemma('leaf.n.01.leaf'), Lemma('leaf.n.01.leafage'), Lemma('leaf.n.01.foliage'), Lemma('leaf.n.02.leaf'), Lemma('leaf.n.02.folio'), Lemma('leaf.n.03.leaf'), Lemma('leave.n.01.leave'), Lemma('leave.n.01.leave_of_absence'), Lemma('leave.n.02.leave'), Lemma('farewell.n.02.farewell'), Lemma('farewell.n.02.leave'), Lemma('farewell.n.02.leave-taking'), Lemma('farewell.n.02.parting'), Lemma('leave.v.01.leave'), Lemma('leave.v.01.go_forth'), Lemma('leave.v.01.go_away'), Lemma('leave.v.02.leave'), Lemma('leave.v.03.leave'), Lemma('leave.v.04.leave'), Lemma('leave.v.04.leave_alone'), Lemma('leave.v.04.leave_behind'), 
Wordnet domains: ['diplomacy', 'book_keeping', 'administration', 'factotum', 'agriculture', 'electrotechnology', 'person', 'telephony', 'mechanics']

NLTK Wordnet词形还原器

from nltk.stem import WordNetLemmatizer
nltk_lammetizer = WordNetLemmatizer()
print("books :", nltk_lammetizer.lemmatize("books"))
print("formulae :", nltk_lammetizer.lemmatize("formulae"))
print("worse :", nltk_lammetizer.lemmatize("worse", pos ="a"))

输出

books : book
formulae : formula
worse : bad

结论

义原集是查找WordNet中单词的接口。它们提供了一种非常有用的方法来查找新词和关系，因为相似的词语与WordNet相互关联，并形成一个紧密的网络。