RTextTools中的德国词干分析器

2023年10月5日 208次阅读

我正在尝试使用RTextTools附带的德语词干分析器,但我得到的结果非常不合适.

说,我有以下向量：

v <- c("groß", "größer", "am", "größten", "ähnlicher")

运用

library(RTextTools)
wordStem(v, "german")

我明白了

[1] "groß"    "größer"  "am"      "größten" "ähnlich"

我错过了什么？

最佳答案 Snowball中的算法

/*
    Extra rule for -nisse ending added 11 Dec 2009
*/

routines (
           prelude postlude
           mark_regions
           R1 R2
           standard_suffix
)

externals ( stem )

integers ( p1 p2 x )

groupings ( v s_ending st_ending )

stringescapes {}

/* special characters (in ISO Latin I) */

stringdef a"   hex 'E4'
stringdef o"   hex 'F6'
stringdef u"   hex 'FC'
stringdef ss   hex 'DF'
......

看起来它被翻译回’DF’“ß”

通过以下方式表示变音符号
德语字母ä,ö和ü分别由ae,oe和ue代表.这里的干扰器是德国主要干扰器的变体,将其考虑在内.

主要的德国词干开始于规则,

First, replace ß by ss, and put u and y between vowels into upper case.

这被规则所取代,

Put u and y between vowels into upper case, and then do the following mappings,

    (a) replace ß with ss, **"MAYBE WRONG ORDER"**
    (a) replace ae with ä,
    (a) replace oe with ö,
    (a) replace ue with ü unless preceded by q. 



So in quelle, ue is not mapped to ü because it follows q, and in feuer it is not mapped because the first part of the rule changes it to feUer, so the u is not found.
点赞