Copyright © 2014 Glenn R. Hofford

Coreference Resolution Based on Deep Symbolic Representation Using the ROSS

Method

Glenn R. Hofford glennhofford(at)gmail.com

Abstract:

A high-level overview of a breakthrough method for natural language understanding and

inference is described. The method performs disambiguation, and in particular

coreference resolution, for a class of sentences as typified by a sentence from the

“Winograd Schema Challenge” that involves a trophy that does not fit in a suitcase. The

method uses a representational scheme called “ROSS” in conjunction with an inference

process that performs the disambiguation. The Winograd Schema (WS) Challenge is a set

of benchmark tests for assessing whether or not an automated natural language

understanding system performs “true” comprehension. The WS challenge includes a

variety of schemas: a schema consists of a pair of descriptive sentences and an associated

pair of questions that tests whether or not the system has understood the sentence and its

alternate. The NLP task often involves performing coreference resolution for an

ambiguous, or difficult, pronoun that exists in the original sentence. The purpose of the

WS Challenge is not to test for simple disambiguation: rather it is to use this task as a test

of underlying intelligent capabilities. A particular schema – the one that involves

sentences and test questions about a trophy that does not fit in a suitcase - was chosen to

demonstrate the capabilities of the ROSS method and of the inference process that takes

place within a working NLU system (code-named “Comprehendor”).

1. Introduction

The ROSS method (Hofford 2014) is a new approach in the area of representation that has

broad implications for artificial intelligence and natural language understanding. (ROSS stands for

“Representation”, “Ontology”, “Structure’”, “Star” language). ROSS is a physical symbol-based

representational scheme1. ROSS provides a complex model for the representation of physical

structure and for the representation of processes and causality. Furthermore, the ROSS method is

capable of application to the representation of abstract things: these include the entities that are

involved in intelligent agent representation, cognition and communication.

A ROSS-based solution for the Winograd schema challenge “trophy in a suitcase” schema is

presented here. This method is based on a model of two distinct aspects that are involved when a

sentence like “The trophy didn’t fit in the suitcase because it was too big” is conveyed, either in

written form or in spoken form. The first of these includes the process of communication on the

part of an intelligent agent, and it includes abstract cognition entities that exist in the mind/brain of

the intelligent agent. The second of these involves a representation of the actual, or external

situation that the intelligent agent describes – for this schema it is modeled as a physical process of

attempting to fit an object into another object.

1 Newell and Simon (1976) explain the physical symbol system hypothesis.

2

In addition to its capabilities for static, or declarative representation, ROSS provides a

foundation for inference: the method presented here involves a breakthrough inference process that

is used to disambiguate the ambiguous word (the difficult pronoun) of the example Winograd

schema challenge sentences. This inference process involves a form of deduction: the mechanism

and details of this reasoning process are described at the overview level.

With regard to how it models the actual/external process, the present method does not exploit

deep structural aspects of the objects of a situation; rather, it uses a model that is based on

modeling the complexities of the behaviors of the relevant objects.

This method may appear to share a surface similarity with approaches that are based on the

use of scripts and narrative chains2, but it is not limited to the modeling of connected behaviors;

rather, the Star language is capable of modeling behaviors of classes of objects based on states

(expressed using attribute type, attribute value pairs). The trophy and suitcase example sentences

do not associate one behavior with another behavior, but they associate a behavior (described by

the verb phrase “does not fit”) with an attributive state that describes a causal feature (“bigness” or

“smallness”).

The scope of this effort was purposefully limited to a single schema: no other known method

or solution that solves this or similar disambiguation challenges uses a deep comprehension

approach; therefore the research and development effort behind this paper focused primarily on

achieving a breakthrough in the internal representation and reasoning areas.

1.1. Background

Davis (2011a) describes the Winograd Schema Challenge as follows:

A Winograd schema is a pair of sentences that differ in only one or two words and

that contain an ambiguity that is resolved in opposite ways in the two sentences

and requires the use of world knowledge and reasoning for its resolution. The

schema takes its name from a well-known example by Terry Winograd (1972)

The city councilmen refused the demonstrators a permit because they

[feared/advocated] violence.

If the word is ``feared'', then ``they'' presumably refers to the city council; if it is

``advocated'' then ``they'' presumably refers to the demonstrators.

The schema challenge sentences and test questions for the trophy and suitcase example is

described in Levesque et al (2012) as follows:

The trophy doesn’t fit in the brown suitcase because it’s too big. What is too big?

2 Rahman and Ng (2012)

3

Answer 0: the trophy

Answer 1: the suitcase

The obvious answer to a human is that it is the trophy that is too big. This answer is obvious to a

person at least partly because humans are able to form a picture (a conceptualization) of the

situation as it involves physical objects, processes, and causality. A human is also able to do

reasoning about the processes and the causality – i.e. the causal relationships – that are involved in

such a situation.

Levesque et al (2012) describes a further aspect of the WS schema challenge for this particular

schema:

…

4. There is a word (called the special word) that appears in the sentence and

possibly the question. When it is replaced by another word (called the alternate

word), everything still makes perfect sense, but the answer changes.

…

This is where the fourth requirement comes in. In the first example, the special

word is “big” and its alternate is “small;” and in the second example, the special

word is “given” and its alternate is “received.” These alternate words only show up

in alternate versions of the two questions:

• The trophy doesn’t fit in the brown suitcase because it’s too small. What is too

small?

Answer 0: the trophy

Answer 1: the suitcase

Levesque et al (2012) shed light on why this challenge is an appropriate one for purposes of testing

whether or not a system that purports to do intelligent thinking and natural language

comprehension is actually doing such thinking:

The claim is that doing better than guessing requires subjects to figure out what is

going on: for example, a failure to fit is caused by one of the objects being too big

and the other being too small, and they determine which is which.

Addressing this topic with another example involving the aforementioned city councilmen and

demonstrators as originally conceived by Terry Winograd, they further state:

This was the whole point of Winograd’s example! You need to have background

knowledge that is not expressed in the words of the sentence to be able to sort out

what is going on and decide that it is one group that might be fearful and the other

4

group that might be violent. And it is precisely bringing this background

knowledge to bear that we informally call thinking.

Davis (2011b) describes the trophy and suitcase example. In his conclusion he states:

“We have been studying automated commonsense reasoning for AI for many years.

McCarthy published “Programs with Common Sense” [reference] more than fifty

years ago; work on CYC [reference] was begun 25 years ago. As far as I can tell,

we are still nowhere near being able to implement the kind of reasoning needed …”

The working solution for this particular schema that is presented here unravels difficult and

seemingly intractable aspects that have blocked other approaches.

1.2. Overview: Main Components of the Solution

The solution uses the ROSS Star language for the specification of class definitions in a

knowledge base, which is a repository for conceptual, or world knowledge. The solution also relies

on an internal XML instance model, which is a proprietary representational approach of ROSS that

is used for representing factual information about particular situations (i.e. past or present

situations). The solution is implemented within a working NLU system code-named

“Comprehendor”. The main components of the solution are as follows:

A complex and internally-consistent approach to the modeling of intelligent agents,

cognitive entities, information/representation entities, and communication entities.

Loose coupling/decoupling of inference from the static/declarative representations

that the inference processes use as a platform. This is a radical departure from the

tight coupling and integration of “facts” and “rules” in traditional logic.The inference

process involves logic that is utilizes representations of the model of the cognition and

communication abstractions. The inference process also involves an “assimilation” of

the behaviors of the external/actual situation; in other words, this is a sort of “meta-

reasoning” wherein the main inference process performs reasoning about rules

(behaviors) that themselves are capable of being used in separate reasoning processes.

The main set of inferences thus is fully extensible as it can assimilate and utilize any

behaviors that are specified with ROSS Star language statements.

The knowledge base/conceptual model includes both complex object and behavior

classes that are encoded in the human-readable form of the Star language.

An internal ROSS instance model is generated from the main sentence that is inputted

(e.g. “The trophy didn’t fit in the brown suitcase because it’s too big.”). The

external/actual process or situation that is represented by the first clause of this

sentence is stored in an instance model that itself is a 4D representation of the

situation.

5

2. ROSS Overview

The ROSS method includes the Star language, which is a language for knowledge base

(ontology) creation, and it includes a formal syntax for XML-based specifications of instances, or

specific situations (instance model). These two main functional components of ROSS are

implemented as follows:

Knowledge Base: Externally, this consists of a set of text files that contain Star language

code. The Star language code consists of a set of definitions for objects and behaviors such

as “enclosable object”, “common object”, “two-sided enclosure object”, “trophy”,

“suitcase” “person”, “fitting”, “intelligent agent”, “cognitive explanation abstract entity”,

“communicating”, “communicated information”, etc. (There are also supporting definitions

such as attribute value set definitions and attribute type definitions). An internal (in-

memory) knowledge base is constructed using the external Star code definitions: these are

read in and processed by an NLU system (i.e. Comprehendor). The knowledge base is

referred to as an “Infopedia”.

Instance Model: An instance model models physical structure, processes and causality for

a particular situation. For instance, a situation may involve a particular intelligent agent

communicating something: e.g. that a particular trophy did not fit in a particular suitcase.

The instance model also models the communicated explanation of the causality behind

such an event, where this is present in the input sentence.

3. Main Components of the Implemented Solution

The following types of knowledge, NLP capability and reasoning ability were determined to

be necessary for addressing the selected challenge schema. (Where relevant, the Star language class

name is also shown).

Internal knowledge base object class definitions for each of the following:

o higher-level classes:

a special class for a structural parent object that is used to construct a 4D

frame of reference for a situation (Star:

EverydayObjectStructuralParentClass)

a class of objects that are capable of being instantiated in an instance of a

EverydayObjectStructuralParentClass: (Star: EverydayObjectFrameClass)

a class of objects that can fit into a container (Star:

EnclosableObjectFrameClass)

a class of container objects (Star: EnclosureTwoSidedObjectFrameClass)

a class of common objects that provides other attribute types such as color

(Star: CommonObjectFrameClass)

o a trophy class that inherits properties from the EverydayObjectFrameClass, the

EnclosableObjectFrameClass, and the CommonObjectFrameClass.

6

o a suitcase class that inherits properties from the EverydayObjectFrameClass, the

EnclosureTwoSidedObjectFrameClass, and the CommonObjectFrameClass.

o human intelligent agent class: it models an agent that performs cognition, and

communication – i.e. this is the person that communicates (spoken or written) the

sentences of the schema. (perception is not needed for this example) (Star:

IntelligentAgentObjectFrameClass)

o A mental conceptual representation (referred to as an “image”, although it is not

necessarily pictorial) of a static or process-wise situation, e.g. an image of the

process of a person attempting to fit a trophy into a suitcase (Star:

CognitiveImageForSituationObjectFrameClass)

o A mental conceptual representation of the intelligent agent’s cognitive

representation of the causal explanation for the situation (Star:

CognitiveExplanationObjectFrameClass)

o A higher-level more generic “information” class from which the cognitive causal

explanation class gets most of its properties: (Star:

RepresentationOfCausalExplanationObjectFrameClass)

o the information items: the spoken or written forms of the sentence and its

constituent parts (Star: CommunicationUnitSentenceObjectFrameClass,

CommunicationFragmentMeaningUnitObjectFrameClass,

CommunicationFragmentWordObjectFrameClass)

Internal knowledge base behavior class definitions. The behavior class definitions include

specifications of causality. Definitions are needed for each of the following:

o the “fitting” process or behavior (Star: EnclosableFittingIntoEnclosureBehavior)

o the “not fitting” behavior (Star: NOT_EnclosableFittingIntoEnclosureBehavior)

NLP syntactic analysis (the primary task is parsing the NL text) – the test sentence

contains two clauses, referred to as “meaning units”3. Each meaning unit has some

adverbial elements (adverbs such as “not” and adverbial phrases). This task generates a list

of syntax trees corresponding to the test sentence. This same processing subsystem also

parses the follow-up question (e.g. “What was too big?”).

Semantic processing that is performed by a semantic engine: this engine reads a list of

syntax trees and generates a semantic/conceptual instance model. The semantic engine also

performs inference that is a form of diagnosis about the overall instance model in order to

perform the needed disambiguation/coreference resolution. The semantic engine also

processes the follow-up question and generates an answer based on information contained

in the instance model, which information includes the results of the coreference resolution

inference.

The instance model represents a situation using a spatial/temporal framework supplied by

the ROSS method. The instance model is quite complex since it actually consists of a

main/overall model that contains an embedded model:

3 The term “meaning unit” is used to denote a clause that contains a subject and predicate.

7

o The overall/main instance model represents a human intelligent agent that

communicates something. The overall instance model contains an embedded

instance model that represents the “fitting” action/event. The main instance model

also involves representation of a cognitive process involving reasoning about the

causal aspects of the embedded instance model.

o The embedded instance model involves an instance of the behavior “to fit” as a

process that occurs along a timeline. The embedded instance model involves the

objects – trophy, suitcase, person, and it involves the “fitting” process.

The approach is specific with respect to how it models the cognitive aspects and the

communicative aspects of the intelligent agent, as this provides a template for a very broad

category of processes that involve communication on the part of intelligent agents. The approach is

entirely reusable and generic with respect to the embedded behavior that is described (the “not

fitting” process) – any behavior that can be modeled with ROSS is capable of being handled in the

same way.

4. Comprehendor (NLU System) Overview and Architecture

The high-level dataflow diagram for the Comprehendor NLU system is as follows:

internal

Knowledge

Base

Parser,

Semantic

Engine

External

Instance

Model

Schema

Sentence(s)

Question

Answer

Star

language

definitions

Internal

Instance

Model

Figure 1: High-level Architecture of Comprehendor

8

The Comprehendor Parser subsystem includes a number of subsystems that include lexical

analysis, sentence segmentation, morphological analysis, part of speech tagging, a (currently-

English-only) constituent parser, and semantic role labeling. The Parser subsystem generates an

internal list of syntactic trees that are augmented with semantic role information. The Semantic

Engine subsystem processes the list of syntactic trees in order to create an internal instance model.

The Engine also performs the various inference tasks that are described in this paper.

5. Analysis: Transformation of sentence to Past Tense

The following analysis addresses the first form of the schema, i.e. “The trophy doesn’t fit in

the brown suitcase because it’s too big.” For purposes of simplifying the scenario, an initial

transformation is applied to the sentence.

The following transformation is applied to the statement in order to put it in to past declarative

form. This is considered to be a nearly equivalent paraphrase of the original statement. The original

version of the sentence involves considerations of potential and the possibility of an outcome that

is based on a mental computational process on the part of a human observer. The transformed

version of the sentence is simpler and has been chosen in order to constrain the representational

challenge.

Original:

The trophy doesn’t fit in the brown suitcase because it’s too big.

Transformed to:

-> The trophy didn’t fit in the brown suitcase because it was too big. (use this)

It will be seen that even the transformed version of the schema involves human agent mental and

communicative processes and thus provides sufficient material for a solution that can be extended

to handle the original version of the sentence in present tense.

6. Use of Internal ROSS Instance Models

The Comprehendor engine makes extensive use of the ROSS instance model in order to

perform the inference that accomplishes the coreference resolution. The overall instance model

represents the situation as it involves an intelligent agent communicating something (about a trophy

and a suitcase). An embedded instance model (not shown) represents the actual situation – the

process that occurred in the past that involved a trophy not fitting into a suitcase.

9

6.1. Representation of Cognition and Communication

The overall situation involves a process wherein an intelligent agent communicates two

clauses within a single sentence. (The first clause is “The trophy did not fit in the suitcase”; the

second clause is “because it was too big”). Figure 2 shows the cognitive and communicative

aspects of the situation:

time

3 10

Cause: x was

too big

(1) CognitiveImageForSituation-01

(2) CognitiveExplanation-01

(3) CommunicationUnitSentence-01

“The trophy did not fit

in the suitcase because

it was too big.”

(A)

(B)

{ IntelligentAgent-01}

sizes: - known to speaker agent

- undetermined by listener

beliefs of the agent

Figure 2: Visualization of overall situation, for main instance model

10

The main/overall instance model corresponds to the diagram of figure 2. It is generated by the

semantic engine, and then it is used by the engine as the basis for the coreference resolution task.

The main instance model uses the following object frame class instances: PastSituation (not shown:

this involves a trophy, a person/agent (optional), and a suitcase), (1) CognitiveImageForSituation-

01 – this represents the intelligent agent’s cognitive representation of the actual past situation, (2)

CognitiveExplanation-01 (the cognitive explanation class inherits most of the necessary attribute

types from a higher-level class called RepresentationOfCausalExplanationObjectFrameClass); the

actual instance values for these attributes are filled in by the engine, IntelligentAgent-01 (the man

in green), (3) CommunicationUnitSentence-01, consisting of

CommunicationFragmentMeaningUnit-01(not labeled – “The trophy did not fit in the suitcase”)

and CommunicationFragmentMeaningUnit-02 (not labeled – “because it was too big”). The main

instance model also uses a behavior class called “IntelligentAgentCommunicatesBehavior”.

The timeline has two important time points. (Note that these particular timeline numbers have

been selected only for illustration purposes and are simply intended to represent a typical scenario.

The Comprehendor prototype uses an enumerated type consisting of timeline values such as “T01”,

“T02”, etc.). At t=3 seconds, IntelligentAgent-01 forms the mental image as shown, and also at t=3

the intelligent agent forms a cognitive explanation of the past situation. At t = 10 the agent

communicates by speaking “The trophy did not fit in the suitcase because it was too big”.

7. Inference Process as Performed by Semantic Engine

The inference process has two main components that are described here. (The second part of

the inference process is the “critical component”, and as such is only described at a summary level).

7.1. First Set of Inferences: Class Selection for Common Nouns and Verbs

The first set of inferences involve selecting appropriate classes from the internal knowledge

base that correspond to each of the common nouns and to the main verb phrase (representing “to

not fit”). The class selection task may involve disambiguation: for instance, the term “trophy” in

the input sentence may represent a trophy of “ordinary” size as is used to reward contestants in

sporting events; “trophy” may in other cases represent an object of unusual size, shape and

dimensions, as would be fit for some special event or occasion. “Trophy” may even represent

something in a metaphorical way (as in “trophy wife”). A ROSS knowledge base may contain

classes that map the word “trophy” to any of a number of classes. However for the purpose of

simplifying this example, the following mappings from common words to knowledge base classes

have been used:

“trophy” – TrophyObjectFrameClass (inherits from EverydayObjectFrameClass,

CommonObjectFrameClass and EnclosableObjectFrameClass) { an ordinary object

with varying size, shape, color, composition, etc. }

11

“suitcase” – SuitCaseObjectFrameClass (inherits from EverydayObjectFrameClass,

CommonObjectFrameClass and from EnclosureTwoSidedObjectFrameClass) { an

ordinary object with varying size, shape, color, composition, etc. }

“not fit” – NOT_EnclosableFittingIntoEnclosureBehavior { a behavior that is

associated with both EnclosableObjectFrameClass and

EnclosureTwoSidedObjectFrameClass}

(The class selection inferences do not currently use statistics-based methods but this is a

possibility for future implementations).

7.2. Second: Main Inference: Determine the Coreferent Entity

The second main inference task involves determining what the word “it” within the second

communicated clause refers to. This can be restated as follows: given a process, P1, where some

referents of the words of the communicated clauses are known (trophy, suitcase) (based on the

results of the class selection inferences, above), what can be inferred about the hidden entities: in

particular, what is the content of IntelligentAgent-01’s CognitiveExplanation -01 with respect to x

(the word “it”) (highlighted in figure 2)?

It must be noted that there are not one, but two unknowns that must be resolved:

How big/small, relative to the functional purpose of “fitting”, is at least one of the

items: the trophy, the suitcase? (and which one?). In other words, which item is

associated with the (causal) feature that has a causative effect on the “not fitting”

behavior?

What does the pronoun “it” within the second clause refer to?

o With respect to the (external) actual past situation

o With respect to the speaker’s cognitive image of the past situation

o With respect to the speaker’s cognitive explanation of the causality of the

situation

o Within the natural language (NL) text: i.e. within the first (descriptive) clause

about the past situation (“it” refers back to either “trophy” or “suitcase”)

The output of this inference process involves a determination of the unknown facts, and is

contained in an instance model for the overall (communication) situation that involves detail that is

the resolution of the “it” referent.

The following is a form of pseudo-code that uses first-order logic (FOL) to show the

specifications of the rules that are used to support the inference. The solution within Comprehendor

does not use FOL, and this section’s use of FOL is intended only for expository purposes. Since the

logic within the antecedents for each of the three rules is similar, for the last two rules only the

logic of the consequent is shown.

Each of the three rules contains several main groups of logical expressions in the antecedent:

12

Expressions that specify the actual/external situation (e.g. the situation involving an

instance of a trophy not fitting into a suitcase).

Expressions that specify the natural language text itself (the main organizing predicate is

“CommunicationUnitSentence”).

Expressions that specify shared, or generally-known commonsense cognitive knowledge

about containers and things that can fit into containers (these expressions can be

understood as representing two ROSS/Star language behavior classes – one for “not fitting

due to enclosable being too big”, and another for “not fitting due to container being too

small”).

Expressions that specify an instance of a CognitiveExplanationObjectFrameClass.

Rule 1: This rule uses an abstraction: it centers around an instance of a

CognitiveExplanationObjectFrameClass. (The CognitiveExplanationObjectFrameClass inherits its

properties from a RepresentationOfCausalExplanationObjectFrameClass class, which is actually

used). This rule resolves the referent for a representational entity (the unknown entity) that exists

within an instance of RepresentationOfCausalExplanationObjectFrameClass, referred to using the

variable name “unknown-entity”. In other words, this rule describes the logic that is used to figure

out what the intelligent agent was thinking when he/she said “because it was too big”. (Note that

this rule example uses “ContainerClass” as an equivalent for the Infopedia class called

“EnclosureTwoSidedObjectFrameClass”). (Note that the example here is only intended to illustrate

the logic for the “too big” case). 4

Rule 1: for “x was too big”

∀ unknown-entity: // (Antecedent) // Variables that represent specific attribute values: Ǝattval1: CausalFeatureAttributeValue(attval1) Ʌ StringValue(attval1, “TooBig”) Ǝattval2: CausalFeatureAttributeValue(attval2) Ʌ StringValue(attval2, “TooSmall”) // The actual past situation. E.g. this models the failed “fitting attempt” instance, where something was too big: Ǝsit: Situation(sit) Ʌ // (@ t = n-2) Ǝentity1: IsRepresentedByClassName(entity1, entity-class-name1) Ʌ PartOf(entity1,sit) Ʌ // e.g. TrophyClass Ǝentity2: IsRepresentedByClassName (entity2, entity-class-name2) Ʌ PartOf(entity2,sit) Ʌ // e.g. SuitcaseClass Ǝaction1: AttemptToFitEnclosureIntoContainer(action1) Ʌ PartOf(action1,sit) Ʌ NotFittedInsideEnclosure(entity1) Ʌ // Att-type = PassiveIsFittedInsideEnclosureState NotIsFittedInto(entity2) Ʌ // Att-type = PassiveIsFittedIntoState ( CausalFeature(entity1,atttype-name, attval1) V // disjunction CausalFeature(entity2,atttype-name, attval1) ) Ʌ

4 Complete detail for this rule is not shown as it is under review with respect to intellectual property aspects.

13

// The input natural language text (the sentence and its constituent parts) Ǝs: CommunicationUnitSentence (s) Ʌ // the input sentence // (@ t = n) Ǝm1,m2: // the two clauses of the sentence MeaningUnit (m1) Ʌ PartOf(m1,s) Ʌ // clause that is a description of a past situation Ǝsubj: ReferentPhraseSubject(subj,entity1) Ʌ PartOf(subj,m1) Ʌ // e.g. enclosable object Ǝadv: VerbModifierWord(adv) Ʌ PartOf(adv,m1) Ʌ // e.g. negation Ǝverb: WordBehavior(verb) Ʌ PartOf(verb,m1) Ʌ // e.g. “fitting” behavior Ǝdobj: ReferentPhraseDirObject(dobj,entity2) Ʌ PartOf(dobj,m1) Ʌ // e.g. container object MeaningUnit (m2) Ʌ PartOf(m2,s) Ʌ // clause that is a causal explanation for the situation Ǝadv2: CauseExplanationIntroducerWord(adv2) Ʌ PartOf(adv2,m2) Ʌ // e.g. “because” Ǝpron: PronounWord(pron) Ʌ PartOf(pron,m2) Ʌ // the unresolved “variable” Ǝverbtobe: AuxiliaryVerbToBeWord(verbtobe) Ʌ PartOf(verbtobe,m2) Ʌ // e.g. “was” Ǝ:caufeat: ReferentPhraseCausalFeatureAttValue(caufeat, attval1) Ʌ PartOf(caufeat,m2) // “too big” Ʌ // (Meta) Common/shared cognitive knowledge about behaviors (these exist in the Infopedia) // Higher-level Entity classes: Ǝenclosable: EnclosableObjectFrameClass(enclosable) Ʌ Ǝcontainer: ContainerObjectFrameClass(container) Ʌ // Inherited Entity classes: Ǝtrophy: InheritsPropertiesFrom(enclosable) Ʌ Ǝsuitcase: InheritsPropertiesFrom (container) Ʌ // Behavior classes: these are the targets of the “assimilation” process // (1) Behavior class for “an enclosable does not fit in a container if the enclosable is too big” // Ǝb1: CognitiveRepresentationOfBehaviorClass(b1) Ʌ // (@ t = any) Ǝb1a: CogReprAntecedent(b1a) Ʌ PartOf(b1a, b1) Ʌ Ǝreprentity1: Represents(repentity1, enclosable) Ʌ PartOf(repentity1,b1a) Ʌ CausalFeature(reprentity1,atttype-name, attval1) Ǝreprentity2: Represents(repentity2, container) Ʌ PartOf(repentity2,b1a) Ʌ Ǝrepaction: CogReprAction(repaction, action) Ʌ PartOf(repaction,b1) Ʌ Ǝb1c: CogReprConsequent Ʌ PartOf(b1c, b1) Ʌ // (the following is shorthand for the Consequent result states) NotFittedInsideEnclosure(repentity1) Ʌ // Att-type = PassiveIsFittedInsideEnclosureState NotIsFittedInto(repentity2) Ʌ // Att-type = PassiveIsFittedIntoState Ʌ // (2) Behavior class for “an enclosable does not fit in a container if the container is too small” // Ǝb2: CognitiveRepresentationOfBehaviorClass(b2) Ʌ // (@ t = any) Ǝb2a: CogReprAntecedent(b2a) Ʌ PartOf(b2a,b2) Ʌ Ǝreprentity1: Represents(repentity1, enclosable) Ʌ PartOf(repentity1,b2a) Ʌ Ǝreprentity2: Represents(repentity2, container) Ʌ PartOf(repentity2,b2a) Ʌ CausalFeature(reprentity2,atttype-name, attval2) Ǝrepaction: CogReprAction(repaction, action) Ʌ PartOf(repaction,b2) Ʌ Ǝb2c: CogReprConsequent Ʌ PartOf(b2c, b2) Ʌ // (the following is shorthand for the Consequent result states) NotFittedInsideEnclosure(repentity1) Ʌ // Att-type = PassiveIsFittedInsideEnclosureState NotIsFittedInto(repentity2) Ʌ // Att-type = PassiveIsFittedIntoState Ʌ

14

// (Meta) The agent’s cognitive explanation of the causal aspects of this situation: Ǝce: RepresentationOfCausalExplanationObjectFrameClass(ce) Ʌ // (@ t = n-1) Ǝce-cause: Repr-CauseEntity(cexplcause) Ʌ PartOf(ce-cause,ce) Ʌ Ǝunknown-entity: Repr-AntecedentCausalAgent Ʌ PartOf(unknown-entity, ce-cause) Ʌ ( RepresentedClassName(unknown-entity, entity-class-name1) Ʌ // e.g. “TrophyClass” V // disjunction RepresentedClassName(unknown-entity, entity-class-name2)) Ʌ // e.g. “SuitcaseClass” CausalFeatureAttributeTypeName(unknown-entity,atttype-name) Ʌ // e.g. “FunctionalSize” CausalFeatureAttributeValue(unknown-entity, attval1) Ʌ // e.g. “TooBig” → // (Consequent) // (@ t = n) RepresentationalRelationship(unknown-entity, entity-class-name1)

The consequent expresses that the unknown entity (within the mind of the cognitive agent) has a

representational relationship with entity-class-name-1, which is an instance of the trophy class.

What is not shown here (due to the complexities of specifying it with FOL), is the way in which the

substitution takes place: the Comprehendor algorithm uses the behavior class of the embedded

situation (the “not fitting” behavior) in order to derive the higher class of the object (the enclosable

object or the container object) that can affect the behavior result, which in this case is the

enclosable object. It uses this class to determine which actual object is referred to based on the

inheritance tree for the object.

Rule 2: This rule associates the unresolved NL text pronoun (e.g. “it”) with an entity in the

represented, or external world. Given the unresolved pronoun within the explanatory (“because

clause”), it resolves which actual entity in the external situation is referred to (e.g. the trophy object

instance or the suitcase object instance). (The Comprehendor engine implements this rule in code

subsequent to the resolution of Rule 1; it sets the appropriate attribute for the entity of the actual

situation (FunctionalSize = “TooBig”, or FunctionalSize = “TooSmall”). (note: “pron” designates

the pronoun).

∀ pron: // (antecedent not shown) → // (Consequent) // (@ t = n) // relationship of pronoun to actual situation entity: RepresentationalRelationshipWordToActualSituationEntity (pron, entity)

Rule 3: This rule centers around the spoken or written natural language text. Given the unresolved

pronoun within the explanatory (“because clause”), it resolves which word in the main clause the

pronoun refers to (e.g. “trophy” or “suitcase”).

15

∀ pron: // (antecedent not shown) → // (Consequent) // (@ t = n) // relationship of pronoun to CommunicationFragmentWord: RepresentationalRelationshipCoreferent (pron, subj)

Note that in the actual system Rule 3 is not implemented, since the question answering system of

the semantic engine is capable of searching the actual instance model that corresponds to the

communicated sentence.

8. Conclusions

The solution described here was developed as a proof of concept and it has achieved the goal

of showing how deep representation using ROSS can accomplish the objectives of a previously-

unsolved problem. The method has been fully implemented in a working system that processes

sentences, creates instance models, and then answers relevant questions based on its internal

knowledge5. The method of this solution uses a “deeper” approach to language understanding than

other known methods. An obvious follow-up task involves applying the deep representation

approach and the inference method more broadly – e.g. to other Winograd schemas. The solution is

generally-useful and provides a foundation for many other NLU tasks. In the first place, the

solution has application to a broad class of coreference resolution tasks:

Those that involve objects and containers and a variety of “fitting” behaviors.

The much broader category of NLU tasks that involve descriptions of objects and

processes where the causal aspects of the relevant behaviors can be represented as states.

Secondly, the solution is a breakthrough in the modeling of human cognitive entities. This has

broad implications for NLU, especially by way of providing a basis for intelligent assistants that

can be intelligent on a deeper level based on having a sufficiently-complex model of the user, a

model of “self”, and a model of the conversation with the user. It is not enough to simply have such

a model; the ROSS method provides capabilities within the Star language that richly model users,

cognition and communication.

Finally, a symbol-based ROSS approach provides the following advantage over non-symbolic

approaches (machine learning/statistics-based, including ANNs and deep learning methods) – the

internal representations of ROSS are inspectable. The representations and reasoning that are

involved in the present method more closely resemble human cognitive representation (as

perceived conceptually) and human reasoning than other known methods.

5 Information about a demo for the working system is available on the author’s web site (Hofford (2014b)).

16

References

Glenn Hofford. 2014. (a) Introduction to ROSS: A New Representational Scheme, Retrieved from

https://www.academia.edu/7145283/Introduction_to_ROSS_A_New_Representational_Scheme,

Last accessed July, 2014. (also available here:

http://www.softwareengineeringconcepts.com/IntroductionToROSS-24May2014.pdf)

Glenn Hofford. 2014. (b) Online resource at http://www.softwareengineeringconcepts.com.

Ernest Davis. 2011. (a) Online resource at: http://www.cs.nyu.edu/davise/papers/WS.html. (Created

9/8/2011 by Ernest Davis. Last update: 8/19/2011).

Ernest Davis. 2011. (b) Qualitative Spatial Reasoning in Interpreting Text and Narrative, Retrieved

from http://www.cs.nyu.edu/davise/papers/cosit.pdf, Last accessed June, 2014.

Hector J Levesque, Ernest Davis and Leora Morgenstern. 2012. "The Winograd Schema

Challenge." KR-2012. n.d. Retrieved from http://www.cs.nyu.edu/davise/papers/WSKR2012.pdf.

Allen Newell and Herbert A. Simon. 1976. Computer science as empirical inquiry: Symbols and

search, Communications of the ACM, 19(3):113-126.

Altaf Rahman and Vincent Ng. 2012. Resolving Complex Cases of Definite Pronouns: The

Winograd Schema Challenge”, in Proceedings of the 2012 Joint Conference on Empirical Methods

in Natural Language Processing and Computational Natural Language Learning, 777–789, Jeju

Island, Korea, 12–14 July 2012.