Information Retrieval and Quantum Theory

MyselfQT in IR

A formalization of Logical Imaging using QTMy work-plan in KMi

Bibliography

Information Retrieval and Quantum Theory

Guido Zuccon

July 16th, 2008

[email protected]://www.dcs.gla.ac.uk/∼guido

Information Retrieval GroupUniversity of Glasgow

Guido Zuccon Information Retrieval and Quantum Theory

MyselfQT in IR


Bibliography

Myself

I PhD student at University of Glasgow, Department ofComputing Science

I working at the EPSRC Project “Renaissance”: formalizing IR interms of Quantum Theory (QT)

I My interests are applying QT to IR, context modeling usingQT, evolutionary operators, High Dimensions Vector Spaces,Voronoi tessellation.


MyselfQT in IR


Bibliography

Myself

QT in IRWhy QT?Examples of QT in IR

A formalization of Logical Imaging using QTWhat is Logical Imaging?Our formalization of Logical Imaging in QTThe Kinematics OperatorConsiderations and Future Directions

My work-plan in KMi

Bibliography


MyselfQT in IR


Bibliography

Why QT?Examples of QT in IR

In this section:

Quantum Theory and Information RetrievalI Why QT?I Examples of QT in IR


MyselfQT in IR


Bibliography


How could QT be useful to IR?

I In the last years there have been some attempts to modelclassical systems using QT:

I Cognitive processes such as concepts formation andcombination [1]

I Modeling of semantics [4, 5, 6]I Modeling IR processes and techniques [19, 3, 11]

I QT trait d’union between logics, probability and geometriesI QT naturally models the contextual behaviour of complex

systems [13]I Inspires the invention of new models and techniques


MyselfQT in IR


Bibliography


The geometry of IR by van Rijsbergen [19] (1/2)

I Maps Logical Model, Vector Space Model and ProbabilisticModel into a Hilbert Space

I Correspondences:I Information space → Hilbert space HI Document d → vector |d〉I Relevance → density operator, or observable (hermitian:

R∗ = R and tr(R) = 1);

I Gleason’s Theorem as tool for computing probabilities


MyselfQT in IR


Bibliography


The geometry of IR by van Rijsbergen (2/2)

I The value of an observable is given by the eigenvalues of theoperator. Thus, they are real even if the matrix’s entry arecomplex numbers

I To each vector corresponds a state of the Quantum system;we can associate a density operator to each state, e.g.ρ|x〉 = |x〉〈x |

I Probability of relevance of document d given query q: tr(ρPd )where ρ = |q〉〈q| density operator associated with qand Pd projector of document d


MyselfQT in IR


Bibliography


The Nature of Search by Arafat [3]

I Introduce a conceptual frameworks which allows to look at IRas QT does a physical process

I Modeling of user, data and relevance, focusing on InteractiveIR.

I The framework simplifies evaluation & comparison of retrievalsystems


MyselfQT in IR


Bibliography


Characterising Through Erasing by Huertas [11]

I Introduce a theoretical framework for representing documentsusing QT

I Uses the following metaphor:I Documents ≡ states of a physical systemI Terms ≡ physical observables to be measuredI Lexical measurements (i.e. frequency of co-occurence) ≡

physical measurement on quantum states

I Thus, counting is viewed as a projective operation on textdocuments using transformations on documents


MyselfQT in IR


Bibliography


IR in Context using QT by Melucci [16] (1/2)

I Depicts how a basis of a vector space represents context:I Every basis vector refers to a contextual property (e.g. time,

location...)I An information object (vector) is generated by the context

(basis)I Similarly, information need is generated by a basisI Every vector can be generated by a different basis


MyselfQT in IR


Bibliography


IR in Context using QT by Melucci [16] (2/2)

I Probability of context: probability that a contextual factor hasaffected the preparation of an object

I Probability of context is function of the distance between thesubspace spanned by a basis and an information object

I Problem: the basis is assumed to be orthonormal (thus, itdescribes semantically independent contextual factors). Moreresearch is needed to model context for which factors aresemantically dependent

I Moreover, the method as has been implemented for testevaluation is ad-hoc: it performs SVD on the top 5,10 termsextracted from the top 1,2,5 documents judged relevant.


MyselfQT in IR


Bibliography


How IR can be translated in QT

Correspondences between concepts in IR and QT:

I Information space → Hilbert space HI Document d → vector |d〉I Relevance → an observable, that is a Hermitian operator (R)

The value of an observable is given by the eigenvalues of theoperator (always real).A state of the Quantum system corresponds to a vector; we canassociate a density operator to each state, e.g. ρ|x〉 = |x〉〈x |


MyselfQT in IR


Bibliography

What is Logical Imaging?Our formalization of Logical Imaging in QTThe Kinematics OperatorConsiderations and Future Directions

In this section:

A formalization of Logical Imaging using QTI What is Logical Imaging?I Our formalization of Logical Imaging in QTI The Kinematics OperatorI Considerations and Future directions


MyselfQT in IR


Bibliography


Motivation for using Logical Imaging in IR

I How do we estimate the probability of relevance of adocument given a query? P(R|q, d) ∼ P(d → q)

I LUP by van Rijsbergen:”Given any two sentences x and y; a measure of theuncertainty of y → x relative to a given data set, isdetermined by the MINIMAL EXTENT to which wehave to add information to the data set, to establishthe truth of y → x.”[18]

I Logical Imaging (LI) addresses such minimal belief revision


MyselfQT in IR


Bibliography


Logical Imaging: a technique for belief revision

Then, what is LI?I LI is a technique for belief revision, based on Possible Worlds

Semantics (PWS)I Introduced by Stalnaker [10], extended and refined by Lewis

[14, 15], investigated by Gärdenfors [9], applied to IR byCrestani [7, 8] and Amati [2]


MyselfQT in IR


Bibliography


The intuition behind LI

I A probability is assigned to each term,I There is a measure of similarity between terms (e.g. EMIM,

cosine, NGD, ...)I Probability mass is moved between similar terms: this

generates a kinematics of probabilities


MyselfQT in IR


Bibliography


A first LI model

The LI model for IR by Crestani [7, 8]:I Each term has a probabilityI For every document in the collection, we evaluate P(d → q)

by imaging on the documentI Uses EMIM as similarity measure

The kinematics:I The probabilities associated to a term NOT IN the document

are transfered to the most similar term IN the document


MyselfQT in IR


Bibliography


Formal definition of the model (1/2)

A formal definition of Crestani’s model:I |T | terms ti , probability P(ti ) associated,

∑ti∈T P(ti ) = 1

I P(d → q) =∑

T P(ti )τ(td , q)

I td is the most similar term to ti for which d is true; τ is atruth function


MyselfQT in IR


Bibliography


Formal definition of the model (2/2)

bat baseball

night

cricket

ballhit

d1


MyselfQT in IR


Bibliography


Variations on the theme

The variations affect the kinematics of probabilities:I Transfer of probabilities towards more than a single term in the

document:I proportionalI opinionated

I The transfer leaves a small probability mass associated to theterm not in the document: Jeffrey’s conditionalization


MyselfQT in IR


Bibliography


Logical Imaging in QT

I Represent a term ti with a vector of document

e.g. |t i 〉 = [2 0 1]T

I P(t) be a probability distribution over the set T of terms(thus,

∑ki1 P(ti ) = 1 and 0 ≤ P(ti ) ≤ 1)

Note: We use the Dirac notation, where the following correspondences hold:

column vector x → |x〉 row vector x∗ → 〈x |inner product y · x → 〈y |x〉 outer product y × x → |y〉〈x |


MyselfQT in IR


Bibliography


Density operator and its evolution

I We can associate a density operator to document d :ρd =

∑ti∈d αi (

∑nj=1 λ

2i ,j |e j〉〈e j |)

I Thus, the density operator associated to d after imaging is:ρ′d =

∑ti∈Wd

α′i (∑n

j=1 λi ,j2 ∣∣e j

⟩ ⟨e j∣∣),

Note: To simplify the notation, we assume P(ti ) = αi


MyselfQT in IR


Bibliography


Computing P(d → q)

We can now compute the probability P(d → q) after Imaging, or,more precisely, the probability induced by JPd → PqK.In order to do so we make use of the Gleason’s Theorem:

“Let H be a Hilbert space with dimension n (finite or countablyinfinite) larger than 2. Let P be a probability measure defined uponthe closed subspaces of H that is additive for a finite or infinitenumber of mutually orthogonal subspaces. Then there exists atrace–class positive operator ρ with unit trace such thatP(L) = tr(ρPL), PL being the projector upon a closed subspaceL."


MyselfQT in IR


Bibliography


Computing P(d → q)

Thanks to Gleason’s Theorem, P(d → q) = tr(ρ′dPR), where PR isthe projector associated to JPd → PqK

Note, in the case that Pd and Pq are not compatible (they do notcommute, PdPq 6= PqPd ):

PR = I− limn→∞((I− Pd )(I− limn→∞ PdPqPd )n(I− Pd ))n


MyselfQT in IR


Bibliography


The Kinematics Operator (1/2)

How do we move the probabilities?I Let us assume that the probability associated to each term is

stored in an entry of a diagonal matrix, eg:0.3 0 0 00 0.2 0 00 0 0.1 00 0 0 0.4

and t1 → t3

I Applying transformation A′ = KTAK with K =

0 0 1 00 1 0 00 0 1 00 0 0 1

...


MyselfQT in IR


Bibliography


The Kinematics Operator (2/2)

I ...we obtain 0 0 0 00 0.2 0 00 0 0.4 00 0 0 0.4

remember t1 → t3

I The Kinematics operator corresponds to the evolution operator U ofthe Schrödinger Picture.

I The evolution of the density operator D from time t1 to time t2 isgoverned by the equation:

D(t2) = U†(t2)D(t1)U(t2)


MyselfQT in IR


Bibliography


Recap

I LI moves probabilities from terms not in the document tosimilar terms into the document

I QT is capable to model LI and its kinematics, by means of aparticular operator, the Kinematics Operator

I The work illustrated in this presentation will be published inthe proceedings of TIR-08, a DEXA Workshop [20]


MyselfQT in IR


Bibliography


Considerations and Future Directions (1/2)

I This formulation of LI is not effective: MAP and P@R arelower than baselines values obtained with simple weightings asTF–IDF or IDF on standard TREC collections (e.g. WSJ, AP).

I The current formulation of LI is more effective than simplebaselines on small collections, e.g. CACM, NPL.

I The LI theory is soundness, but the current formulation is poor(higher rank to documents that contains not discriminativeinformation)


MyselfQT in IR


Bibliography


Considerations and Future Directions (2/2)

I We are investigating a reformulation of LI considering anintra-document kinematics

I We also want to take into account the context where a termappears: Context Based Logical Imaging (CBLI) using VoronoiTessellation in the QT framework


MyselfQT in IR


Bibliography


A Voronoi Tessellation of a 2D space, andits use to guide the Kinematics

bat

night

cricketball

baseball

hit

Voronoi Tessellation generated by d = {night, bat, . . .}


MyselfQT in IR


Bibliography



bat

night

cricketball

baseball

hit

Voronoi Tessellation generated by d = {bat, hit, . . .}


MyselfQT in IR


Bibliography



bat

night

cricketball

baseball

hit

bat

night

cricketball

baseball

hit

The difference in the Kinematics produced by two different Voronoi Tessellation


MyselfQT in IR


Bibliography

Research visit at KMi (1/4)

During my visit:I How do we compare different HAL spaces geometrically?

I There is no previous work that investigates thisI Geometric comparison: local distance (local cosine similarity)

VS subspace distanceI Is there any correlation between HAL distance (e.g.

d(HALqrel ,HALpseudorel)) and MAP?


MyselfQT in IR


Bibliography


In the future:I What is the effect of SVD and RP on a Semantic space?

I We need to reduce the dimensionality of the space forcomputational performances

I Several researchers claim that dimensionality reductionimproves retrieval effectiveness


MyselfQT in IR


Bibliography


In the future:I Is there any common geometric pattern in the reduced space?

I In [12] Karlgren et al. claim that Semantic spaces have aparticular structure (after dimensionality reduction), inparticular as filaments

I We want to investigate this idea, comparing the geometricalstructures obtained by SVD and RP

Image from [12]


MyselfQT in IR


Bibliography


In the future:I HAL in QT

I formulate in QT terms the notion introduced in [17] (i.einformation flow, information inference, abduction, entropy,concept combination and concept learning, ...)

I It would be useful to LI as well: the CBLI is based on atessellation of the Semantic space


MyselfQT in IR


Bibliography

Bibliography I

[1] D. Aerts and L. Gabora.A Theory of Concepts and their Combinations I.Kybernetes, 34:151–175, 2005.

[2] G. Amati and S. Kerpedjev.An Information Retrieval Model: Implementation and Experiments.Technical Report 5B04892, Fondazione Ugo Bordoni, Roma, IT, 1992.

[3] S. Arafat and C. J. van Rijsbergen.Quantum Theory and the Nature of Search.In Proc. of the 1st AAAI Quantum Interaction Symposium, pages 114–121,Stanford, CA, USA, 2007.

[4] P. Bruza, K. Kitto, D. Nelson, and K. McEvoy.Entangling Words and Meaning.In Proc. of the 2nd AAAI Quantum Interaction Symposium, pages 118–124,Oxford, UK, 2008.


MyselfQT in IR


Bibliography

Bibliography II

[5] P. Bruza and J. Woods.Quantum Collapse in Semantic Space: Interpreting Natural LanguageArgumentation.In Proc. of the 2nd AAAI Quantum Interaction Symposium, pages 141–147,Oxford, UK, 2008.

[6] S. Clark, B. Coecke, and M. Sadrzadeh.A Compositional Distributional Model of Meaning.In Proc. of the 2nd AAAI Quantum Interaction Symposium, pages 133–140,Oxford, UK, 2008.

[7] F. Crestani and C. J. van Rijsbergen.Information Retrieval by Logical Imaging.Journal of Documentation, 51(1):3–17, Mar 1995.

[8] F. Crestani and C. J. van Rijsbergen.Probability Kinematics in Information Retrieval.Proc. ACM SIGIR ’95, pages 291–299, Jan 1995.


MyselfQT in IR


Bibliography

Bibliography III

[9] P. Gärdenfors.Knowledge in Flux. Modeling the Dynamics of Epistemic States.The MIT Press, Cambridge, MA, USA, 1988.

[10] W. L. Harper, R. Stalnaker, and G. Pearce, editors.Ifs. Conditionals, Belief, Decision, Chance and Time.Reidel, Dordrecht, NL, 1981.

[11] A. Huertas-Rosero, L. A. Azzopardi, and C.J. van Rijsbergen.Characterising Through Erasing – A Theoretical Framework for RepresentingDocuments Inspired by Quantum Theory.In Proc. of the 2nd AAAI Quantum Interaction Symposium, pages 160–163,Oxford, UK, 2008.

[12] J. Karlgren, A. Holst, and M. Sahlgren.Filaments of Meaning in Word Space.In Proceedings of ECIR 2008, pages 531–538, Glasgow, UK, 2008.


MyselfQT in IR


Bibliography

Bibliography IV

[13] K. Kitto.Why Quantum Theory?In Proc. of the 2nd AAAI Quantum Interaction Symposium, pages 11–18,Oxford, UK, 2008.

[14] D. K. Lewis.Counterfactuals.Basil Blackwell, Oxford, UK, 1973.

[15] D. K. Lewis.Probabilities of Conditionals and Conditional Probabilities.The Philosophical Review, 85:297–315, 1976.

[16] M. Melucci.A Basis for Information Retrieval in Context.ACM Trans. Inf. Syst., 26(3):1–41, 2008.


MyselfQT in IR


Bibliography

Bibliography V

[17] D. Song, P. Bruza, and R. Cole.Concept Learning and Information Inferencing on a High Dimensional SemanticSpace.ACM SIGIR 2004 Workshop on Mathematical/Formal Methods in IR, 2004.

[18] C. J. van Rijsbergen.Towards an Information Logic.In Proc. ACM SIGIR ’89, pages 77–86, Cambridge, MA, USA, 1989.

[19] C. J. van Rijsbergen.The Geometry of Information Retrieval.CUP, 2004.

[20] G. Zuccon, L. A. Azzopardi, and C. J. van Rijsbergen.A Formalization of Logical Imaging for Information Retrieval using QuantumTheory.In IEEE Proc. 5th Int. Workshop on Text-based IR, 2008.


Information Retrieval and Quantum Theory

Documents

Transcript of Information Retrieval and Quantum Theory