A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations

In a previous post, I wondered how Latent Semantic Analysis (LSA) and Latent Relational Analysis (LRA) could be unified. LSA can be used to recognize synonyms and LRA can be used to recognize analogies. The two algorithms are quite similar, so it seems it should be possible to design a more general algorithm that subsumes both of them.

My first idea was to use a three-mode tensor, in which relational similarity (analogy) can be calculated by comparing fibers (columns, rows, and tubes) and attributional similarity (synonymy) can be calculated by comparing slices (two-mode cross-sections). This worked well for attributional similarity, but I had less success with relational similarity.

My next idea was to treat attributional similarity as a semantic relation, like any other semantic relation. This approach successfully unifies relational similarity and attributional similarity, and also handles antonyms, associations, and other semantic relations.

Consider the following SAT analogy question:

Stem: mason:stone
Choices:
(a) teacher:chalk
(b) carpenter:wood
(c) soldier:gun
(d) photograph:camera
(e) book:word
Solution: (b) carpenter:wood

The solution to this multiple-choice analogy question is the choice word pair with semantic relations that are most similar to the semantic relations of the stem word pair. Mason is to stone as carpenter is to wood. A mason works with stone, a mason cuts stone, a mason builds with stone. A carpenter works with wood, a carpenter cuts wood, a carpenter builds with wood.

Consider the following TOEFL synonym question:

Stem: levied
Choices:
(a) imposed
(b) believed
(c) requested
(d) correlated
Solution: (a) imposed

The solution is the choice word with semantic attributes (properties, characteristics) that are most similar to the semantic attributes of the stem word. To levy is to do something to somebody without requiring consent. To impose is to do something to somebody without requiring consent.

Consider the following ESL synonym versus antonym questions:

1. dissimilarity:resemblance — synonym or antonym? (antonym)
2. naive:callow — synonym or antonym? (synonym)
3. commend:denounce — synonym or antonym? (antonym)

Synonyms have very similar attributes (properties, characteristics). A naive person is a person who lacks experience. A callow person is a person who lacks experience. On the other hand, antonyms have very similar attributes, except for one attribute. To commend is to make a positive judgment about something. To denounce is to make a negative judgment about something.

Consider the following similar versus associated versus both questions:

1. table:bed — similar, associated, or both? (similar)
2. cradle:baby — similar, associated, or both? (associated)
3. music:art — similar, associated, or both? (similar)
4. flea:ant — similar, associated, or both? (similar)
5. ale:beer — similar, associated, or both? (both)
6. mug:beer — similar, associated, or both? (associated)
7. uncle:aunt — similar, associated, or both? (both)

Similar words involve the same type of thing. Tables and beds are both types of furniture. Fleas and ants are both types of insects. Associated things merely belong to the same domain. Mugs and beer belong to the domain of drinking. Cradles and babies belong to the domain of raising children.

It seems that we have five different phenomena here:

1. analogy(A:B, C:D) = A:B has same semantic relations as C:D — analogy(mason:stone, carpenter:wood)
2. synonym(A, B) = A has same properties as B — synonym(levied, imposed)
3. antonym(A, B) = A has same properties as B, except for one — antonym(commend, denounce)
4. similar(A, B) = A and B are the same type — similar(table, bed)
5. associated(A, B) = A and B belong to the same domain — associated(mug, beer)

However, we may unify all five of these phenomena by viewing them all as analogies:

1. analogy(A:B, C:D) = A:B has same semantic relations as C:D — analogy(mason:stone, carpenter:wood)
2. synonym(A, B) = analogy(A:B, levied:imposed)
3. antonym(A, B) = analogy(A:B, commend:denounce)
4. similar(A, B) = analogy(A:B, table:bed)
5. associated(A, B) = analogy(A:B, mug:beer)

We can handle all five by treating them as supervised learning problems, in which the task is to assign classes to word pairs:

1. class(mason:stone) = class(carpenter:wood) = artisan:material
2. class(levied:imposed) = synonym
3. class(commend:denounce) = antonym
4. class(table:bed) = similar
5. class(mug:beer) = associated

Thus a whole range of semantic phenomena are subsumed under analogies, which are handled by supervised word pair classification.