Göran Sonesson:
1
Pictorial semiotics, Gestalttheory, and the ecology ofperception *
In at least one important respect, pictorial semiotics differs from most other fields and
sub-domains included in the study of signification: it is not only a new discipline; it does
not merely establish different limits and an altered focus for age-old preoccupations; it
inhabits a field which has almost never been laboured before. For, if we take pictorial
semiotics to be involved with the study of pictorial signs per se, of some general property,
more peculiar to pictures than iconicity, which may be termed pictorality, or picturehood,
and if we suppose it to apply empirical methods to this study, then it is certainly a novel
endeavour, far more so than linguistics and literary theory, more, in fact, than film
semiotics and the semiotics of architecture. For the only other domain properly devoted to
pictures, art history, always has been, and mostly continues to be, fascinated by the
singularity of the individual work of art.
For the purpose of the present essay, pictorial semiotics may be described as that
part of the science of signification which is particularly concerned to understand the nature
and specificity of such meanings (or vehicles of meaning) which are colloquially identified
by the term ’picture’. Thus, the purview of such a speciality must involve, at the very
least, a demonstration of the semiotic character of pictures, a study of the peculiarities
which differentiate pictorial meanings from other kinds of signification (particularly from
other visual meanings, and/or other meanings based on iconicity, or intrinsic motivation),
and a assessment of the ways (from some or other point of view) in which the several
species of pictorial meaning may differ without ceasing to inhere in the category of
picture.
To insist that pictorial semiotics, in its particular enterprise, has no precedents, is
not to deny that the knowledge compiled by other scientific endeavours could still be
capable of enriching our domain. Pictorial semiotics may indeed have something to learn
from, for instance, perceptual psychology in general, and Gestalt theory in particular. But,
in assessing the claim, made in Fernande Saint-Martin's (1990) latest book, that Gestalt
psychology should constitute at least part of the foundational layer of contemporary
pictorial semiotics, we have to undertake the arduous task of determining the relevance of
* Review of Saint-Martin, Fernande, La théorie de la Gestalt et l'art visual. Québec:
Presse de l'Université du Québec 1990.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
2
this branch of learning to the study of pictorial meaning. This, in turn, requires us to
investigate, not only the bearings which Gestalt psychology may have upon the peculiar
domain of problems that has evolved within the contemporary discipline of pictorial
semiotics , but also the position occupied by Gestalt theory inside present-day perceptual
psychology.
Whatever we may think about the present-day value of Gestalt psychology, there
remains a much more fundamental claim, which is at the basis of Saint-Martin's book, and
so must be addressed first: the assumption that pictorial meanings are, in some way of
other, of an intrinsically perceptual nature. This is indeed a second way in which pictorial
semiotics may turn out to be peculiar: for, no matter what may be the particular character
of pictorial meanings, there is no denying the fact that they are conveyed to us by means
of visual perception. If this is taken to mean that we must rely for our information on the
knowledge base assembled by perceptual psychology, then we shall have to abandon the
autonomy postulate, so dear to many of the exponents of present-day semiotics. As a
consequence, however, we are also faced with the task of picking out one among several
available perceptual psychologies.
Rather than offering a regular review of Saint-Martin's most recent book, what we
would like to bring here are some reflections on pictorial semiotics generally, highlighting
some sectors of the domain as they present themselves posterior to the publication of
Saint-Martin's work, until, and including, her Gestalt theory book. In the process, we will
note the challenges posed by Saint-Martin's contribution to earlier versions of pictorial
semiotics, while pinpointing such questions of our domain, which, in the light of these
confrontations, stand out as being the central issues destined to dominate all further
research. As matter of fact, we will, on one hand, agree with Saint-Martin as far as the
necessity of breaking out of semiotic autonomy is concerned, as well as on the
fundamental importance of perception to pictorial signs, yet we will, on the other hand,
propose an appreciably different view of the relationship between semiotics and the other
sciences, and we will sketch a much more comprehensive approach to perceptual
psychology.
After determining the place occupied by Saint-Martin's work within the
contemporary domain of pictorial semiotics, we will turn first to the issue opposing
semiotic autonomy to mere interdisciplinary. This will permit us to broach the more
general problems pertaining to the specific character of semiotics as a science. Then we
will review a set of questions brought into view by pictorial semiotics, which turn out to
be better elucidated by schools of perceptual psychology other than Gestalt theory; and
finally, we will relate the discoveries of Gestalt psychology to some further intrinsic
concerns of pictorial semiotics.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
3
Elements of a history of pictorial semioticsWhereas linguistics derives from the ancient practice of grammar and philology, and while
literary theory figured, though less prominently, as an ingredient of the study of literature,
well before semiotics came to its own, the inquiry into the nature of pictorial meanings has
had few real precedents, before it was initiated in the sixties as part of the structuralist
project. Even though the inception of art history and art criticism dates back to Antiquity,
or at least to the Renaissance, this enduring tradition has contributed surprisingly little, if
anything, to the investigation of pictorial signification. Moreover, in the pioneering works
of early semiotics, from Leibniz and Degerando to Peirce and Saussure, and from Russian
formalism to the Prague school, pictures are rarely discussed on their own terms.
In the early sixties, the art historian E.H. Gombrich declared the creation of a
’linguistics of the visual image’, paralleling art history, to be an urgent task; and yet, in the
late seventies, the psychologist J.J. Gibson was still complaining over the fact that nothing
even approximating a ’science of depiction’, comparable to the science of language, had
been developed. In the meantime, it is true, Gibson and his disciples and colleagues had
been making important contributions towards a psychosemiotics of picture perception.
More explicitly semiotic work on pictorial meaning had been accomplished, with some
influence from Gombrich but without any connection to Gibson, by Barthes and the
French Structuralists. Goodman, Eco, and a few others had discussed the semiotic nature
of the pictorial sign. The Greimas school, the Belgian Groupe µ, and Fernande Saint-
Martin were just beginning their work.
Since then, we have perhaps not resolved any of the issues. But we have begun to
see which might be the questions.
Structuralist beginnings and the pseudo-linguistic turnA few members of the Prague school, most notably Mukar &ovsky ! and Veltrusky !, early on
concerned themselves with the sign character of the pictorial work of art, but to them, on
the main, the latter only served to exemplify the same properties that they had so
abundantly discussed elsewhere in the case of poetry, literature, and drama. The real origin
of pictorial semiotics is to be found in a small article by Roland Barthes, written in 1964,
in which an analysis of a publicity picture boosting the delights of Panzani spaghetti is
attempted, using a few ill-understood linguistic terms taken over from Saussure and
Hjelmslev (cf. Sonesson 1989a,II.1). Beginning with a few general observations, the article
then turns into a regular text analysis concerned with one particular photograph, defined
both as to its means/ends category (publicity) and, somewhat more loosely, its channel
division (magazine picture). In spite of the confusion to which Barthes testifies in his
employment of Saussurean and Hjelmslevian terms, and although the usage to which he
puts these terms is in itself incoherent, the article marks a real breakthrough, both
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
4
intrinsically, because of its attempt to use a simple model permitting to fix the recurring
elements of pictorial signification, and, in particular, because of the influence it was to
exercise on almost all later analyses.
In the Panzani article, as well as in an earlier text (1961), Barthes proclaims his
famous paradox, according to which the picture is a message deprived of a code. The term
’image’ in fact alternates in the same paragraph of the article with the more particular term
’photographie’, as if this were the same thing, but later on the photograph is opposed in
this respect to the drawing. Yet many followers of Barthes retain the wider interpretation,
using it to defend the inanalyzability, or ineffability, of paintings and other works of art.
Actually, neither Barthes, nor his followers make any real effort to analyze the picture:
they are really discoursing all the time on the referent, that is, on the depicted scene.
Another parti pris of Barthes's, which becomes clear already in the Panzani article,
and which has had devastating consequences for pictorial semiotics, is the idea that no
picture is capable of conveying information by itself or, alternatively, that it contains so
much contradictory information that a verbal message is needed to fix its meaning. Pictorial
meaning must, in either case, be supposed to depend on linguistic meaning. No doubt
pictures offers us much less linguistic information than verbal texts, except in those cases
in which the picture itself contains the reproduction of written messages (as is in part the
case in the Panzani picture); but the picture much better conveys another kind of
information that resembles the one present in the perceptual world. It is clear, however,
that to Barthes and to many of his followers, information itself is erroneously conceived
to be something that is verbal in nature (cf. Sonesson 1989a,II.1. and 1989d).
Barthes's Panzani article stands at the origin of two diverging developments inside
semiotics, one of which is the semiotics of publicity, which, in addition to the pictorial
aspects, attends to verbal and other components of advertisements, but which has in fact
been to an appreciable degree concerned with pictures (for an excellent critical survey, cf.
Pérez Tornero 1982). Until recently, this speciality has continued to be largely derivative
on Barthes's achievement, continuing to thrive on his fairly fragile theoretical contribution.
This constitutes a problem, for what is confused in Barthes's works tends to become even
more so in that of his followers, who, moreover, inherit his exclusive attention to the
content side of the pictorial sign, or more exactly, to the extra-signic referent and its
ideological implications in the real world, even to the point of ignoring the way in which
the latter are modulated in the picture sign.
The other current to which Barthes's work has given rise is more properly at part of
pictorial semiotics, mainly preoccupied with the study of art works, and notably
paintings. Its pioneers were, among others, Louis Marin, Hubert Damisch, Jean-Louis
Schefer, etc. However, in actual fact, the Panzani model, as applied to paintings,
immediately tends to become indistinguishable from an ordinary art historical approach.
Damisch (1979) also rapidly improvised a critique (which Saint-Martin 1987a:xi approves
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
5
of) of semiotics as applied to the study of pictorial art, simply identifying the former with
the linguistic model. Such a limited view of semiotics may have been partly justified at the
time; yet Damisch is subject to a much more serious confusion in comparing the merely
intuitive, pre-theoretical notion of the picture with the concept of language as
reconstructed in linguistic theory (just as Metz did in the case of the notion of film; cf.
Sonesson 1989a,I.1.2.).
Although Umberto Eco was not, on the whole, a contributor to the study of
pictures, he must certainly be mentioned here, because of the extensive discussions of
iconical signs (which he tends to identify with pictures), occupying an appreciable portion
of his central works during the sixties and seventies. Just like Bierman and Goodman
before him, Eco was intent on showing that there could be no iconical signs (signs
motivated by some kind of similarity between expression and content), since all signs
must be conventional. There is now every reason to think that Eco, as well as Bierman and
Goodman, was basically wrong in his conclusions; but his work remains of seminal
importance.
Less influential than Barthes, but certainly at least as important for the development
of pictorial semiotics, is René Lindekens (1971, 1973, 1976, 1979), whose early death, a
few years ago, was an irreparable loss for pictorial semiotics. Although his first book
(1971) is explicitly concerned with photography, whereas the second one (1976) claims to
treat of visual semiotics generally, both really discuss questions pertaining to the basic
structure of the pictorial sign per se (e.g., conventionality and double articulation), and
both use photography as their privileged example. It is in order to demonstrate the
conventionality of pictures, and to show the way in which they are structured into binary
features, that Lindekens (1971; 1973) suggests, on the basis of experimental facts (and
common sense experience), the existence of a primary photographic opposition between
the shaded-off and the contrasted; but the same publication (1971) also turns to
experiments involving geometric drawings which have the function of brand marks, in
order to discover the different plastic meanings (which Lindekens calls ’intra-iconic’) of
elementary shapes. In fact, Lindekens would seem to argue for the same conventionalist
and structuralist thesis as applied to pictures as the early Eco (1968), but while the latter
tends to ignore the photograph as the most embarrassing apparent counter-example,
Lindekens attacks its frontally from the beginning.
In at least two respects Lindekens is exemplary. He has employed an array of
different methods: system analysis and experimental tests, which enter a fruitful
symbiosis in his two books, but also text analysis (which will be defined below). In the
second place, he theoretical baggage is complex: Hjelmslevian semiotics, of which he has a
much more solid knowledge than Barthes, with just an inkling of the Greimas school
approach, in spite of the fact that he wrote his thesis for Greimas; phenomenology, which
unfortunately affected him in the subjectivist misinterpretation due to Sartre and the
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
6
existentialists generally; and experimental psychology of perception, most notably on the
side of the Gestalt school. Yet, the different theoretical elements of Lindekens' approach
remain badly integrated, much knowledge present in these perspectives is insufficiently
exploited, and most of Lindekens' basic tenets may well turn out to be unjustified (cf.
Sonesson 1989d).
Three contemporary approachesIn the late seventies and in the eighties, pictorial semiotics has made something of a new
start, or, rather, it has offered us three fairly different, new beginnings. Although these
three approaches are largely incompatible, there can be no doubt that, in future, any
serious theory of pictorial semiotics must take account of them all, starting out from a
synthetic and critical analysis of their divergent contributions.
The first of these approaches is that of the Greimas school, as it has been applied to
pictures, in particular in the works of Jean-Marie Floch and Felix Thürlemann. These
scholars and their followers accept the basic tenets of the Greimas school, and make use of
its abundant paraphernalia, albeit with unusual restraint. Thus, like all contributions from
the Greimasian camp, their articles employ an array of terms taken over from the linguistic
theories of Saussure, Hjelmslev, Chomsky, and others, but given quite different meanings.
The real problem resulting from this approach, therefore, it not, as it is often claimed (by,
among others, Saint-Martin), that it deforms pictures and other types of non-linguistic
meanings by treating them as being on a par with language, but that, in attributing quite
different significations to terms having their origin in linguistic theory, it renders any
serious comparison between linguistic and nonlinguistic meanings impossible. Greimasians
will obviously not accept this piece of criticism; they claim to be studying meaning located
so deep down, that it is all of a kind. This arbitrary postulate never receives any
justification (cf. Sonesson 1989a).
It should be obvious, then, that, from the point of view of the Greimas school, the
specificity of pictorial meaning is of no account. This is also what Floch (1986b) affirms
(though Thürlemann 1990 would seem to take a somewhat different stand, as we will see
shortly). Yet, the interest of this approach derives not only from the fact that, unlike
those of earlier scholars mentioned above, it involves the application of a model having
fairly well-defined terms, which, at least to some extent, recur in an array of text analyses,
but also is due to the capacity of this model to account for at least some of the
peculiarities of pictorial discourse. Thus, for example, Floch and Thürlemann have noted
the presence of a double layer of signification in the picture, termed its iconic and plastic
levels. On the iconic level the picture is supposed to stand for some object recognizable
from the ordinary perceptual Lifeworld; while concurrently, on the plastic level, simple
qualities of the pictorial expression serve to convey abstract concepts. Floch, it is true, has
tried to generalize these notions to other domains, most notably to literature, but they are
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
7
clearly much better adapted to pictorial discourse (cf. the critical observations of Sonesson
1989e).
The second approach to pictorial semiotics that is of seminal importance is that of
the Groupe µ, or Liege group, the most constant members of which are the linguists Jean-
Marie Klinkenberg and Jacques Dubois, the chemist Francis Edeline and the aesthetician
Philippe Minguet. Starting in the late sixties, this Belgian group produced a book of
’general’ rhetoric (1970), in which they analyzed in a novel way the ’figures’ appearing in
the elaborate taxonomies of classical rhetoric, using linguistic feature analysis inspired in
the work of Hjelmslev, as well as the mathematical theory of amounts. A rhetorical figure
only exists to the extent that there is a deviation from a norm. The latter is understood as
redundancy, and thus identified with the Greimasian concept of isotopy, which henceforth
becomes one of the essential building-blocks of the theory (cf. Groupe µ 1977). At this
stage, Groupe µ seems heavily dependent on a set of Hjelmslevian concepts (which they
may not interpret quite correctly; cf. Sonesson 1988,II.1.3.7., and 1989a,II.3-4.), as well
as on the notion of isotopy as conceived by Greimas (which in itself may be incoherent,
cf. Sonesson 1988,II.1.3.5).
In spite of being general in import, the theory is to begin with mostly concerned with
figures of rhetoric as they appear in verbal language. In a short study of a coffee pot
disguised as a cat, Groupe µ (1976) tries to implement its theory also in the pictorial
domain. Over the years, the theory has been continuously remodelled, so as to account
better for the peculiarities of pictorial meaning. Recently, Groupe µ rhetoric appears to
leave behind at least part of the linguistic strait-jacket inherited from Hjelmslev, in order to
incorporate ’a certain amount of cognitivism’ (Klinkenberg, personal communication). Yet,
the theory still seems far from integrating the perceptual and sociocultural conditions that
constitute the foundations of all rhetorical modulations.
The third conception of importance brought forward in the domain of pictorial
semiotics is the one propounded by Fernande Saint-Martin. Her early collection of essays
(1968) on the structure of pictorial space already attends to the perceptual process as
applied to the work of art (understood à la manière de Rorschach), and contains a forceful
plea for the creation of an experimental branch of aesthetics. On the other hand, no explicit
concern with semiotics is visible here. In a number of later publications (1985, 1987a),
however, Saint-Martin starts elaborating a theory of visual semiotics which is based on the
conviction that a picture, before being anything else, is an object offered to the sense of
visual perception. Visual meaning, according to this conception, is analyzable into six
variables, equivalent to a set of dimensions on which every surface point must evince a
value: colour/tonality, texture, dimension/quantity, implantation into the plane,
orientation/vectorality, and frontiers/contours generating shapes. The surface points,
specified for all these values, combine with each other, according to certain principles,
notably those of topology, and those of Gestalt theory. The nature and use of topology,
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
8
and its employment in children's drawings and art, were studied by Saint-Martin (1980) in
an earlier work. Her recent book, on the other hand, is given over to an inquiry into the
nature and employment of Gestalt relations.
The importance of Saint-Martin's contribution consists in her radical rejection of the
linguistic model, and her insistence on perception's being the basis on which pictorial
signification must repose. Members of the Prague school already pointed to the essentially
perceptual nature of the sign, and insisted on the necessity of its being transformed into a
perceptual object, requiring the active collaboration of the addressee involved in the
signifying act, but the consequences of these insights for the character of the picture sign
were never spelled out. As compared to the binary opposition, which is the regulatory
principle of the Greimas school approach, as well as to the norm and its deviations, which
determines the conceptual economy of Groupe µ rhetoric, Saint-Martin's theory offers a
much richer tool-kit of conceptual paraphernalia, more obviously adapted to the analysis
of visual phenomena. Yet this very richness may also, in the end, turn out to constitute
the basic defect of the theory, as we shall suggest below.
In our own extensive, critical review of pictorial semiotics (Sonesson 1989a),
devoted to an analysis of the linguistic heritage preserved by this science, as it appears in
the conceptions of, most notably, Barthes, Floch, Thürlemann, and Groupe µ, we have
emphasized, just as did Saint-Martin, but quite independently of her work, the basically
perceptual nature of the picture sign, and we have tried to expound some of the
consequences of this observation, invoking the testimony of contemporary perceptual
psychology, as represented by, among others, James Gibson, Julian Hochberg, John
Kennedy, and Margaret Hagen, as well as that of a few philosophical theories of
perception, exemplified by the work of, for instance, Edmund Husserl, Ludwig
Wittgenstein, Nelson Goodman, and Richard Wollheim. It remains for us, then, to compare
our conception of perception, and of its conditions and consequences, to the one defended
by Saint-Martin, and to determine, in the process, what may be learnt from this
confrontation.
The relation of pictorial semiotics to other semiotic and non-semioticsciencesFernande Saint-Martin quite rightly rejects (mostly by implication) the postulate of
autonomy so popular with semioticians, notably those of the Greimas school. And yet,
when it is at last explicitly articulated, in the Gestalt theory book, this rejection
apparently has the effect of placing semiotics in a position quite impossible to defend
crowded in by the other sciences occupying the same field. Although she employs a few
metaphorical terms like ’visual language’, ’visual syntax’, and so on, Saint-Martin most of
the time marks her distance to the so-called linguistic model, perhaps more so than would
be justified. Moreover, when it comes to the question of what sort of science pictorial
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
9
semiotics is, she passes over the issue all too rapidly. Finally, some queries concerning the
relation of pictorial to visual and iconic semiotics stand in urgent need of being ventilated.
The peculiar character of the semiotic approachThe first question that must be elucidated is whether pictorial semiotics is a particular
discipline, having its own goals, objects of study, models and methods, or if it is simply
just another, perhaps better, way of accomplishing the goals, and studying the objects of,
art history. In particular, we must examine whether that which is studied by pictorial
semiotics is a series of unique works of arts, or the class of all pictures, that is, pictures in
general, or pictorality.
Fernande Saint-Martin certainly appears to take the latter position, although she is
never very explicit about it. Her thorough theoretical studies of the notions of topology
and of Gestalt relations (Saint-Martin 1980; 1990) would seem to suggest a generic
approach, as would the abstract character of her book on visual language (Saint-Martin
1987a). In fact, the very idea of studying ’the elements of visual language’, of searching for
its grammar and its ultimate constituents, formulated in the first chapter of the latter book,
as well as the references made in this context to Saussure and Chomsky, indicate that the
object of study envisioned must be pictorality per se. On the other hand, most of Saint-
Martin's books contain elaborate analyses of works of art (never of any other kinds of
pictures, with the exception of some children's drawings in the topology book), the status
of which is never discussed. Thus, for instance, it never becomes clear whether these
analyses are to be taken as the ultimate result of the theoretical elucidation, or merely as a
partial illustration of some of the main points.
In contrast, Floch directly tackles this problem, but his stance seems contradictory,
and, on at least one interpretation, is seriously flawed. On one hand, he sees it as his task
to account for the minute particularities of a given photograph (1986b:11); and on the
other hand, he argues that semiotics should define categories which cross-cut those which
are socially accepted, like ’picture’, ’art’, and ’photography’ (1984a:11; 1986b:12f). Both
these opposed alternatives to a study of photography and other socially accepted pictorial
kinds are explicitly present in Floch's concrete analyses, but it is not at all clear how they
could be reconciled. And while the first interpretation seems to make nonsense of
semiotics as a science, the second appears to opt for a positivistic approach of the most
gratuitous kind.
We will take care of the first part of this conundrum here, keeping the remainder for
the next sub-section. According to Floch (1984a:11), semiotics cannot tell us anything
about sociocultural categories such as ’photography’. Instead it should attend to the
particular properties of a given photograph. This is a legitimate claim if it is construed as
an argument tending to favour a text analytic approach over other conceivable methods.
But if it implies that a particular picture must constitute, not only the objected studied,
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
10
but the object of study, of a semiotic investigation, it would seem to deprive the semiotic
approach of its peculiarity, making it just another method which may be used within art
history, communication studies, and so on. That is, if the object of study characteristic of
semiotics is not specificity (of pictorality, of pictorial kinds, or whatever), then semiotics
itself will lose its specificity. We may still argue for semiotics on the grounds that it builds
models, that it uses certain constellation of methods, etc., but in any case, its originality
certainly comes out diminished.
Our own definition of pictorial semiotics derives from the view we have taken
elsewhere on general semiotics, because we believe that the former should share the goals,
models and procedures which the latter applies to the wider domain of signification, which
includes, among other things, the objects studied by pictorial semiotics (cf. Sonesson
1989a,I.1.). It is impossible to establish a consensus among all semioticians on what
semiotics is all about; and many semioticians will not even care to define their discipline.
However, if we attend less to definitions than to real research practice, and if we leave out
those would-be semioticians who simply do not seem to be doing anything very new
(those who merely go in doing art history, literary history, philosophy, logic, or
whatever), it seems possible to isolate the smallest common denominators of the
discipline.
In the following, then, semiotics will be taken to be a science, the point of view of
which may be applied to any phenomenon produced by the human race. This point of
view consists, in Saussurean terms, in an investigation of the point of view itself, which is
equivalent, in Peircean terms, to the study of mediation (cf Parmentier 1985). In other
words, semiotics is concerned with the different forms and conformations given to the
means through which humankind believe itself to have access to ’the world‘.
The very term ’point of view’ is, as Saint-Martin (1988:202) notes, a visual
metaphor. Yet the point, which is a standpoint, matters more than the sense modality.
For, in studying these phenomena, semiotics should occupy the standpoint of humankind
itself (and of its different fractions). Indeed, as Saussure argues, semiotic objects only exist
as those points of view which are adopted on other, ’material’ objects, which is why these
points of view cannot be altered without the result being the disappareance of the semiotic
objects as such.
Taking the point of view of the users, and trying the explain their particular use, we
cannot, like the philosopher Nelson Goodman, reject the folk notion of picture because of
its incoherence, but must discover its peculiar systematicity. But it does not follow, as
Prieto would claim, that we must restrict our study to the knowledge shared by all users
of the system, for it is necessary to descend at least one level of analysis below the
ultimate level of which the user is aware, in order to take account of the presuppostions
underlying the use of the system. Semiotics must go beyond the standpoint of the user, to
explain the workings of such operative, albeit tacit, knowledge which underlies the
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
11
behaviour constitutive of any system of signification (cf. Sonesson 1989a,I.1.4).
Moreover, semiotics is devoted to these phenomena considered in their qualitative
aspects rather than the quantitative ones, and it is geared to rules and regularities, instead
of unique objects. It is not restricted to any single method, and it is certainly not
dependant on a model taken over from linguistics, but it is a peculiarity of the approach
that it tends construct models which are then applied to the objects analyzed.
Pictorial semiotics, in turn, is that part of the science of signification which is
particularly concerned to understand the nature and specificity of such meanings (or
vehicles of meaning) which are colloquially identified by the term ’picture’. Thus, the
purview of such a speciality must involve, at the very least, a demonstration of the
semiotic character of pictures, as well as a study of the peculiarities which differentiate
pictorial meanings from other kinds of signification, and a assessment of the ways (from
some or other point of view) in which pictorial meanings are apt to differ from each other
while still remaining pictorial.
In the present context, we will insist on the fact that pictorial semiotics, like all
semiotic sciences, including linguistics, is a nomothetic science, a science which is
concerned with generalities, not an idiographic science, comparable to art history and most
other human sciences, which take as their object an array of singular phenomena, the
common nature and connectedness of which they take for granted. Just like linguistics, but
contrary to the natural sciences and the social sciences (according to most conceptions),
pictorial semiotics is concerned with qualities, rather than quantities — that is, it is
concerned with categories more than numbers. Being nomothetic and qualitative, pictorial
semiotics has as its principal theme a category that may be termed pictorality, or
picturehood — which is not, as we shall see, simply the same thing as iconicity.
Unlike those other semioticians which we have mentioned above, who do not start
out from art history, or at least do not seem to carry with them any essential pieces of its
intellectual heritage, Saint-Martin often merely seems to summarize the inspirations and
preoccupations which have beset this discipline during the last century. Moreover, she
does not, as we noted above, present us we any real definition of semiotics as a science.
Yet we must accept her claims to be doing semiotics, for she does certainly not form part
of those ’who simply do not seem to be doing anything very new’. One of her merits
undoubtedly is to have made the multifarious theoretical heritage of art history available to
semiotics. It is, however, because of the nomothetic format in which it is presented in
Saint-Martin's work that this heritage now appears transfigured into a tool-kit for
semiotics.
Thus, what Saint-Martin tells us about Gestalt forms and topological relations, and
about pictorial units and visual syntax, must be taken as claims about the nature of
pictorality, and should be evaluated as such.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
12
The domain of pictorial semioticsAccording to Ferdinand de Saussure, semiotics (or semiology as he called it) was to study
’the life of signs in society’; and the second mythical founding-father, Charles Sanders
Peirce, as well as his forerunner John Locke, conceived of semiotics as being the ’doctrine
of signs’. Later in life, however, Peirce came to prefer the wider term ’mediation’ as a
description of the subject matter of semiotics (cf. Parmentier 1985). And Saussure not
only argued that in the semiotic sciences, there is no object to be studied except for the
point of view that we adopt on other objects (see Sonesson 1989a,I.1.4. and above), but
also claimed the sign to the ’a relatively superficial phenomena’ resulting from the
interplay of values. More recently, Greimas has rejected the notion of sign, and his
followers Floch (1984a) and Thürlemann (1982: 1990) have argued the case in the domain
of pictorial semiotics. In a similar fashion, Umberto Eco (1976), at the end of his tortuous
critique of iconicity, substituted the notion of sign process for the traditional sign concept.
Whatever we may think of this critique, the sign certainly does not appear to be
comprehensive enough to delimit the field of semiotics: rather, the domain of semiotics is
meaning (or ’mediation’), in some wider, yet to be specified sense. Given a suitable
definition of the sign (to which we will turn below), we may ask if the picture, which
certainly conveys meaning, is also, more in particular, a sign. If so, we may continue
inquiring into the possibility that the picture is made up of units which are meanings
themselves, but not signs (which is not simply the old issue of double articulation; cf.
Sonesson 1989a,III.4.2. and 1989c, 1990f). Furthermore, we may want to ask the same
questions about perception per se, which is undoubtedly endowed with meaning, but
which does perhaps not, as we will suggest, take on the character of a sign. On these
issues, Saint-Martin takes a stand that appears unacceptable to us, which is why we will
return to this discussion below.
Since semiotics is not merely concerned with signs, but more generally with
meanings, pictorial semiotics, as a subdivision of the former, will be involved with
pictorial signs, but also with all other kinds of pictorial meanings, to the extent that there
are any such things.
Not only should pictorial semiotics study all kinds of pictorial meaning, but it
should also be concerned to determine which are, so to speak, the pictorial kinds existing
in a particular society, being part of that study of ’the life of signs in society’, advocated
by Saussure. And this brings us back to the second reading of what Floch says on the
nature of semiotics, discussed in the last section.
According to this view, semiotics is indeed devoted to the study of general facts, but
these facts or not of the kind designated by terms like ’picture’, ’photography’, ’painting’,
’publicity’, ’art’, etc. These latter terms serve to label categorizations of semiotic
resources which are ’merely’ sociocultural, that is, historical and relative (’le découpage
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
13
socio-culturel donc relatif et historique des moyens d'expression’; 1986b:13); but if we are
going to understand how, in a particular picture, meaning comes into being, Floch affirms,
we must instead develop a general theory of discourse, which includes all kinds of
discourses, architectural ones, linguistic ones, and so on, apart from the pictorial ones
(ibid.).
It is difficult to understand why such ’merely’ sociocultural division blocks should
be despised, for in all their historical relativity, they are probably the only ones we have.
Indeed, as we pointed out (in Sonesson 1989a,I.4. and above) following Prieto, who
himself quoted Saussure, semiotic objects only exist for their users, that is, they have only
the kind of existence that they are accorded by their use in a given social group; and thus,
once we pretend to go beyond sociality, there is nothing left to study. It is true that
Floch's master Greimas, following his master Hjelmslev, has argued that a semiotic theory
should be arbitrary — but also adequate; and we have already (ibid.) noted the paradoxes
of this pronouncement. We seem to be faced, once again, with that kind of gratuitous
thinking that makes Goodman substitute a picture concept of his own making for the one
commonly employed – when the real task is to account for the hidden systematicity of the
common Lifeworld notion of a picture (cf. Sonesson 1989a,III.3.). But, as we shall see,
there may actually be a somewhat more interesting sense to Floch's argument.
Floch (1986b:12) actually rejects the doctrine of signs in favour of another study,
that of ’les formes signifiantes, les systèmes de relations qui font d'une photographie,
comme de toute image ou de tout texte, un objet de sens. La sémiotique structurale qui est
nôtre ne vise pas à élaborer une classification des signes, ni selon les conditions de leur
production, ni selon les rapports qu'ils entretiennent avec la ”réalité”.’ We will not enter
once again into the quarrel over the nature of signs, but a few points must be made before
we go on. In the first place, it is curious that Floch should claim the old epithet ’structural’
for his (and Greimas') conception of semiotics, for nothing could really be more contrary
to the spirit of structuralism (that of Saussure and Hjelmslev, notably), than the idea,
presupposed here and vindicated by Greimas elsewhere, that content and expression may
be freely combined, so that, in the present case, the same sense may be produced, not only
in different pictorial genres, but in a picture and in a piece of literature. Indeed, in a
structure, (as we noted in Sonesson 1989a,I.3.4.) the parts interact and so mutually
modify each other, or even constitute each other, as is the case of the phonological
oppositions. Thus, if the sign function is structural, as it certainly was to Hjelmslev,
content and expression cannot be freely exchanged and recombined, as the Greimasians
think.
In the second place, there may nevertheless be a place in semiotics for a study of
’significant forms’ which cut across the divisions of photography and painting, and even
of visual and other discourses. The case for the latter, however, is not very convincing.
Thus, for instance, if we define iconicity as ’un effet de sense de ”réalité” ’ (Floch
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
14
1984a:12), we may be able to find instances of it even in verbal discourses; but by
admitting this definition, we have already given up the specificity of iconicity, and we
have deprived ourselves of the possibility of discovering that which is peculiar to
pictorality, within the limits set by iconicity generally (Cf. Sonesson 1989a,III. ).
Furthermore, although it remains uncertain whether there are properties shared by a set of
verbal and a set of pictorial discourses, which do not also belong to all verbal and pictorial
discourses, there certainly are categories of pictures, which have no name in our language,
as suggested by that fact that some pictures are informatively analyzed by Floch's model,
others, which do not conform to this model, by that of Groupe µ, while a third group,
which do not lend themselves to either kind of analysis, may be elucidated using Saint-
Martin's approach (for example, cf. Sonesson 1989g, 1990d; 1992a,IV).
However, even though there may actually exist other pictorial categories than those
which are explicitly recognized in our culture, it is not to be understood why we should
study these at the expense of the former categories, which are certainly the primary ones
on a social scale. The exceptional existence of such similarities as obtain between
photographs and certain hand-made pictures does in no way diminish the importance of
characterizing the socially received categories.
In this respect, the standpoint of Felix Thürlemann (1990:9) seems much less radical:
he makes a plea for a ’semiotic science of art’, conceived as a praxis of artistic
interpretation taking Greimasian semiotics as its fundamental methodological basis; and
this certainly must imply an acceptance of ’art’ as a category. Unlike Floch, he does not
apply his analytical tools indifferently to art works, advertisement pictures, and comic
strips. Yet he does not explicitly discuss the nature of pictorial art (although there is some
suggestions about it in his discussion of synaesthesia and semi-symbolicity, as in Floch's
corresponding passages).
Just like Thürlemann, Saint-Martin is only concerned with pictorial works of art, to
the exclusion of all other kinds of pictures (with the exception of a few analyses of
children's drawings, in Saint-Martin 1980). This is curious, for she does not discuss this
preference: indeed, since she conceives of pictorial semiotics as being a part of visual
semiotics, comprising, in addition, the semiotics of sculpture and of architecture, the
neglect of all ’non-artistic’ pictures appears to be particularly unjustified. Perhaps she
would argue that artistic pictures exemplify pictorial visuality in better and/or more
interesting ways than other pictures, but there is no reason to accept such a claim a priori.
Most advertisement pictures are in fact visually much more complex than, for instance,
the art works produced by minimalists. Of course, knowing the latter to be works of art,
we will probably be much more prone to project our personal feelings and experiences
onto them than to the former. What this shows, however, is that the notion of artistic
picture is basically a sociosemiotical category (as was already the case of the aestheticfunction according to Mukar &ovsky ! ), and thus artistry is a point of view which may in
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
15
principle by adopted on any picture.
Alternatively, Saint-Martin may be taken to conceive of pictorial semiotics as being
merely an ancillary science, that is, a servant of art history, as Thürlemann clearly does.
That would be a pity. In present-day information society, large quantities of the
information which reaches us is in fact pictorially conveyed, and it is becoming more
important every day to investigate all the different modes of pictorial meaning employed
in the process. Not only must pictorial semiotics, like the one of Floch and of Groupe µ,
attend to examples of the several pictorial kinds, but it should also be concerned to
determine their specificity. It should determine the different rules of construction
separating, for instance, paintings from photographs (on the latter, cf. Vanlier 1983;
Dubois 1983; Schaeffer 1987; Sonesson 1989d); it should recognize the different categories
separated by the effects they are socially expected to produce, such as advertisement
pictures, pornography, or caricature (cf. Sonesson 1988, 1990a); and it should differentiate
pictures on the basis of the channels through which they circulate, such as the poster, the
wall painting, the post card, and so on.
Beyond autonomy and interdisciplinarityIn semiotics, those who followed Saussure and Hjelmslev (and some of those who
followed Peirce) have transferred the Saussurean postulate of ’autonomy’ or ’purity’ from
linguistics to the wider of domain of signification studies. Such a conception is today
particularly characteristic of the Greimas school approach, as exemplified in the work of
Floch and Thürlemann (for a defence, see ’Hors du texte, point de salut’, in Floch
1990:3ff). In linguistics, the study of ’purely linguistic’ phenomena was undoubtedly
productive for a time, but have long since been played out. Today, linguistics is a very
’impure’ science indeed, tending to fuse with cognitive psychology, philosophy, and
computer science. To retain such a restriction in semiotics, which has to span many more
and much more complex phenomena, certainly seems absurd.
Two opposite, but complementary, requirements could therefore be imposed on a
semiotic approach. First, when analyzing an object which has been studied by other
disciplines, it should take full account of what has been learnt so far. But in the second
place, in so doing, it should retain its proper point of view, and not simply merge with one
or another of the other disciplines studying the object in question. It is a great merit of
Saint-Martin's work generally, and of the Gestalt theory book in particular (1990:3ff), that
it speaks out clearly against the autonomy postulate, showing in addition the profit which
is to be gained from an interplay with other sciences, notably in the case of Gestalt theory
and the genetic psychology of Piaget. Yet, we can only regret that, on one hand, Saint-
Martin neglects a lot of information on perception which is readily at hand and which
should have important consequences for a theory of picture perception, and that, on the
other hand, her sense of the peculiarity of the semiotic approach seems to get lost on the
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
16
way.
If there is one thing which is particularly characteristic of semiotics, then it is the
determination to take the point of view of the sign user himself, that is, to describe the
expression merely to the extent that it conveys a content, an vice-versa. Prieto, repeatedly
quoted by Saint-Martin, has often insisted on this point. But if Saint-Martin had taken
this requirement seriously, we could have been spared the long and tortuous discussion
(23 pages in Saint-Martin 1987a) of what kind of colour analysis is relevant to semiotics
(and we could have had a little more an all the other variables instead, now treated together
on another 20 pages). More importantly, she should not have come up with a list of visual
variables, in the definition of which perceptual, physical and physiological criteria are used
indiscriminately. It is true that Jakobson has often been accused of an equivalent confusion
in his famous definition of phonological features, but (to put it in the terms of a classical
doctrine of art) even the faults of giants are faults, and should not be imitated.1
As for the other requirement, Saint-Martin turns out to be remarkably unfamiliar
with at least some aspects of recent psychology which would seem to be of importance to
her theme. She fails to mention the important inquiries into the psychology of picture
perception accomplished recently by Gibson, Hochberg, Hagen, Kennedy, and others; for
although the names of the first two do appear in the Gestalt theory book (1990), they are,
as we shall see later, treated in an altogether inadequate fashion. She never attends to
recent advances in cognitive psychology, some of which tend to put the work of Piaget, on
which she relies heavily, into critical perspective. Nor does she stoop to consider the
critique directed to Gestalt theory, by, among others, Gibson and Hochberg.
The consequences of this omission are not always serious: thus, for instance, in
suggesting that colours must be conceived as ’chromatic poles’, Saint-Martin (1987a)
seems to rediscover for herself the important concept of the prototype, introduced by
Eleanor Rosch (1975; 1978), which could be described as the use, for the determination of
category membership, of approximations to the best instances, taking the place of
sufficient and necessary criteria (cf. also Sonesson 1989a,I.3.1.). It is unfortunate, in any
case, that lacking a clear notion of prototype, she fails to make any real use of it in the
sequel, in particular in the Gestalt theory book. Indeed, it is regrettable that Saint-Martin
should ignore Rosch’s (1973) demonstration that ’good forms’ are kinds of prototypes
Rosch's interpretation has actually the advantage of saving some Gestaltist intuitions from
the antiquated epistemology of the Gestalt school. The prototype concept also serves to
elucidate some of the properties of ’good forms’ mentioned in Saint-Martin's (1990)
recent book, such as their function as reference points, epitomized in the tendency to
1 This it not far from Gibson's (for instance 1982:67) denial of the ecological relevance of
such measures as wavelength, intensity, and purity of wavelength, supposedly
corresponding to hue, brightness, and saturation.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
17
render good forms better (’la boniformisation’; pp. 74ff), as well as the deviations from
the good form (pp. 104ff).
Much more damaging is her neglect of recent psychology of perception. At one
point this neglect even becomes somewhat embarrassing: Bishop Berkeley’s old
conception, according to which depth is not seen, but somehow cognitively reconstructed
using clues derived from touch, is quoted with approval from the art historians Riegl,
Wölfflin, and Worringer, who still propounded this theory around the turn of the century.
However, if the idea of depth being a merely tactile sensation has by now been
unacceptable in psychology for almost a century, even the more general notion that is has
to be constructed, instead of being directly perceived, at present seems incompatible with
the facts. The late James Gibson rediscovered the old insight of Husserlean
phenomenology, according to which it is the thing itself, not some two-dimensional
perceptual field which is directly seen. Yet the opposite, erroneous, conception turns out
to be fundamental to much of Saint-Martin’s (1987a) reasoning, in particular in her
discussion of perspective and in the analysis which she performs of the potential cube
supposedly defining sculpture; as well as, at numerous points in the Gestalt theory book
(1990), where it serves to justify her recourse to Gestalt psychology
More will be said about these issues as we go on to ponder the way in which to
choose a perceptual theory which is apt to constitute a basis for pictorial semiotics. It
remains for us to consider, in the present context, the problematical character which Saint-
Martin's rejection of the semiotic autonomy postulate acquires in the Gestalt theory book
(1990:3ff).
She starts out very well, rightly taking a number of earlier semioticians to task for
not paying any attention to relevant psychological knowledge, notably that of Gestalt
psychology, and for refusing all serious discussion of this knowledge, even when it is
invoked in passing, as in the work of Thürlemann. But, in opposing a requirement of
universal interdisciplinarity to the Greimasian autonomy postulate, she is naturally
frightened by the immensity of the task set before the semioticians, who would have to
have a perfect command of numerous other sciences, and in particular of their most recent
developments. In order to reduce the requirement of interdisciplinarity to manageable
proportions, Saint-Martin imposes a double limitation, pertaining to time and to the
nature of competence called for. First of all, a set of propositions stemming form other
sciences is claimed to go into the constitution of any new science, but as soon as the
problem domain of this particular science has been clearly delineated, no further recourse
to these sciences is supposedly necessary. Once we think we know what pictorial
semiotics is all about, we are free to close it off again to all influences from psychology
and the other sciences. The autonomy lost at the beginning is rapidly regained as we
proceed.
In the second place, the semiotician is not required to accomplish for himself the
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
18
operations of verification which are presumed to ascertain the hypotheses on which he
builds his theory. Nor should he intervene in the debates opposing the representatives of
the sciences from which he takes over his fundamental hypotheses. He is simply
supposed to pick out a set of propositions established by these sciences, on the basis of
which he goes on to develop more specific assumptions pertaining to his particular
domain.
Saint-Martin's problem is a real one, and her reaction to it is quite understandable,
but this should not keep us from noting the inadequacy of her solution, or the
inappropriate consequences it is likely to have. The idea of letting interdisciplinary have
its way in semiotics only until the proper problem domain of the latter has been
constituted seems ill-advised. In the first place, it is not at all obvious that we could agree
on when the constitution of the domain has been accomplished. In fact, Floch could argue
that the relevant domain has already been delimited, so that, on her own terms, Saint-
Martin's work is superfluous. Secondly, it seems to us that a scientific endeavour will
only remain vital, as long as its interchange of facts and theories with other sciences
continues. It might in fact be argued that linguistic structuralism at its beginning
incorporated a set of hypotheses taken over from psychology and sociology, but was then
closed off to all further influences, just as Saint-Martin advocates. When later
structuralism appeared to be inadequate, it was therefore rejected cavalierly, and primitive
reactions, such as post-structuralism and post-modernism, followed. Had structuralism
remained open to the interdisciplinary dialogue, it would have been much more easy to
move on to a position, from which the knowledge which it produced could be retained, and
in the context of which its accomplishments could still be given their (limited) due (cf.
Sonesson 1989a,I.).
It is, however, the restriction imposed on the semiotician's competence in the
interdisciplinary domain which is the most disquieting aspect of Saint-Martin's argument.
Although she does mention the problem of choosing the set of propositions needed from
those propounded by other sciences, Saint-Martin seems unaware of the difficulty of this
task. In the case of perceptual psychology, for example, which is the domain with which
Saint-Martin is particularly concerned, there are at present at least three completely
incompatible theories purporting to explain more or less the same facts (and we will
consider some of their differences below). Interestingly, Saint-Martin herself turns out to
be unable to follow the imposition, not to intervene in the internal quarrel of the other
sciences, even to the point of polemizing violently with James Gibson in numerous
passages of her book.
To the extent that we are willing to learn from perceptual psychology, therefore, we
will be in need of criteria for choosing the ’best’ theory. To do this, we may even have to
accomplish some operations of verification ourselves. If this is taken to imply the use of
the experimental method, then we can point to the fact (to which we will return in the next
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
19
subsection) that is has already been employed by a number of semioticians, and that, in
the future, as the particular problem domain of pictorial semiotics begins to emerge, we
will anyway be forced to formulate our own experiments, in order to obtain the results
adequate to the questions we stand in need of asking. But this is not all. If semiotics is a
science, it must be in possession of its own operations of verification, and thus it should
be possible to apply semiotic methods to the results of psychology and the other
sciences, and so to verify them, from the point of view of semiotics.
The prize of interdisciplinary, to Saint-Martin, is passivity. Just as, when it comes
to the chosen domain of semiotics, Saint-Martin tends to think of it as a mere servant of
art history, so, in the interdisciplinary interplay of sciences, she would seem to reduce it
to playing a purely receptive part. In contradistinction to this conception, we think, as we
indicated above, that semiotics should stake out its own domain, much larger than that of
art history, and that when, starting out from this independent position, it goes
interdisciplinary, it should have something to teach, as well as something to learn.
The dialectics of semiotic methodologySemiotics is often thought to be a method that may be employed in a number of sciences.
Yet this conception is untenable, because there is an array of methods which have been
used in semiotics so far, and no single one of them is properly speaking peculiar to
semiotics. There may be a more general methodological strain, of course, which
characterizes semiotics, a particular way of viewing things, perhaps the point of view
geared to meaning, as indicated above. In the more strict sense of a series of operations
applied to a set of phenomena, yielding a number of general conclusions, there are three or
four methods in semiotics: text analysis, system analysis, experiment, and text
classification (cf. Fig.1.).
Textual analysis consists in treating any meaningful phenomenon occurring in a
culture, e.g. a sentence, a work of art, a piece of behaviour, and so on, as being the ’text’ of
a given ’system’, i.e. as being exhaustively (at least from a certain point of view) reducible
to a series of repeatable elements and the rules for their combination. The issue here is not
whether pictures are made up of fixed, detachable units (as denied, in one or other form,
by Benveniste, Goodman, Eco, and Thürlemann) or whether the rules are really concerned
with transformation and/or construction rather than combination (as Eco may be taken to
say, and as Sonesson 1989a certainly argues). We may put the basic assumption of textual
analysis more succinctly by stating that behind all phenomena there is a categorical
framework the instantiations of which are seen (from one or other point of view) to recur
all through a series, which, as such, is characterized by a constellation of categories. It
does not follow, of course, that that which is repeated is in any way comparable to a
phoneme, a word, or even a rule of syntax. Indeed, Saint-Martin will argue that what
recurs in the pictorial text is something else which she terms a set of coloremes (as well as
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
20
their several constellations).
D o m a i n o f a n a l y s i s
O p e r a t i o n a p p l i e d
O p e r a t o r ( s ) i n v o l v e d
E x p o n e n t s i n p i c t o r i a l s e m i o t i c s
T e x t a n a l y s i s s e v e r a l t e x t s
m o d e l i n g , g e n e r a l i z a - t i o n
t h e r e s e a r c h e r
f r a g m e n t a r y a t t e m p t s
a s i n g l e t e x t
m o d e l i n g , h e u r i s t i c c o n s t r u c t i o n o f m o d e l s
t h e r e s e a r c h e r
m o s t c o m m o n e . g . , S a i n t - M a r t i n
S y s t e m a n a l y s i s i n t u i t i o n , s o c i a l l y d e r i v e d k n o w l e d g e
v a r i a t i o n , c o m b i n a t i o n
t h e r e s e a r c h e r
P e i r c e , E c o , c o m m o n i n s e m i o t i c s o f p h o t o g r a p h y
E x p e r i m e n t a r t i f i c i a l c o n s t r u c t i o n o f t e x t s
e v a l u a t i o n o r s u p p l e - m e n t a r y c o n s t r u c t i o n o f t e x t s
t h e r e s e a r c h e r + e x p e r i m e n t a l s u b j e c t s
L i n d e k e n s , T a r d y , E s p e , K r a m p e n , e t c .
T e x t c l a s s i f i c a t i o n
i n t u i t i o n a n d t e x t s
i n s e r t i o n i n t o c o n c e p t u a l f r a m e s
t h e r e s e a r c h e r
G r o u p e µ , D e l e d a l l e , m y o w n m o d e l o f i n d e x i c a l i t y
Fig.1. Semiotic methods and their application to pictorial semiotics
In this sense, textual analysis is certainly not peculiar to the semiotic approach; it
has been employed in many other disciplines, such as sociology (’natural history
approach’) and cognitive psychology (’protocol analysis’). Structuralism, in the primary,
linguistic sense, requires, in addition, that the ’system’ should be studied, and the
categories derived, by means of the application of operations to a set of ’texts’; and that
the identity of the elements in the ’texts’ should be determined through recourse to the
relations which the elements contract with each other in the ’system’. This supposes a
complex dialectic between the ’system’ and its ’texts’ which is not explicitly present in
other textual approaches.
Given these assumptions, it becomes necessary, not only to study a sizable set of
texts pertaining to the same system, but to adjust all earlier analysis of a phenomenon each
time a new analysis of other phenomena of the same general type is made, for instance as
far as the limits between units, or the identification of variants for an invariant, are
concerned. But neither Barthes nor any other exponent of structuralism have ever
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
21
analyzed more than one picture ’of the same type’ (of course, the notion of ’type’ here is
problematical); and if we take all pictures to be in some sense of type-identical, it is easy
to ascertain that no researcher has ever cared to revise earlier analyses in the light of later
ones.
Most of the work accomplished in pictorial semiotics relies on the textual method.
This is true of the work of Barthes, Marin, Schefer, Gauthier, Floch, Thürlemann, etc., as
well as of many of the analyses performed by Lindekens and Tardy. Most of these
investigators identify themselves with the structuralist movement, or once did so; but it
seems clear from what has been said above that, if text analysis retains any heuristic value,
this must be for reasons foreign to structuralism.
Saint-Martin's text-analytical method is not structural in this sense; yet it is, as we
shall see, based on a set of potentially recurrent elements or variables. Most of her
analyses are only sketched out, without there being any rigourous application of the model
to the text. This is understandable, for the list of variables introduced by her model is too
vast for the analytical operations to be exhausted in a reasonable amount of time. In fact,
only two full analyses would seem to have been carried through by Saint-Martin (1986;
1990) so far: one pertaining to Pellan's ’Mascarade’, and another one concerned with
Leduc's ’Masses’. They are reasonably similar, although the emphases are, as is natural,
differently placed. In addition, there are a series of, published and unpublished, analyses
executed by her disciples (for instance, Carani 1986a).
Groupe µ denies all value whatsoever to textual analysis (Francis Edeline, personal
communication). In the opinion of these scholars, text analysis tends to get lost in
idiosyncratic detail. Contrary to text classification which is, as we will see, the method
favoured by Groupe µ, it requires an exhaustive account, from one or other point of view,
of the entire text. It seems to us that such a requirement of exhaustiveness imposes a
constraint on the analysis, which may at least have some heuristic value. To the extent
that semiotics is really a nomothetic science, however, we should be wary of taking
individual text analyses to constitute its end results, contrary to what is suggested by the
placement of a text analyses on the final pages of Saint-Martin's Gestalt theory book.
For our purpose, the experimental method may be described as an approach in which
a fragmentary ’text’ is constructed by the investigator and completed (or, alternatively,
evaluated) under strictly limited and controlled conditions by the experimental subjects,
the ’system’ being deduced on the basis of majority reactions. In this way, Tardy with
collaborations studied the procedures of picture perception. Lindekens investigated what
different values where attributed to shapes such as triangles and circles, as well as the way
in which the relative degree of contrast and nuance of a photograph modified the manner in
which its referent was interpreted. Espe, apparently independently, approached the same
problem, and later went on to study the effects of photographic angle on interpretation.
The latter theme was independently addressed by Bengtsson, Bondesson, & Sonesson, in
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
22
the particular case in which the subject matter of the photographs were facial expressions
of emotion. The linguistic commutation test, in which one expression feature is exchanged
for another in order to study the effect on the content plane (or the reverse), was
transferred to pictorial semiotics, and transformed into a psychological procedure by
Porcher and Gauthier. Krampen, Espe, and other collaborators obtained experimental
information on the degree to which different signs where judged to be iconical.
Both the experimental and the semiotical sophistication of these studies is extremely
varied. Experiments of this kind, whether performed by psychologists or semioticians, are
important for our understanding of the pictorial sign. They have, however, the
disadvantages found in all experimental studies: either they are based on artificial, instead
of actual ’texts’; or, when they involve real texts, they expose them to experimental
subjects in situations which are in some or other way artificial. In pictorial semiotics, very
few problems have been treated experimentally (as listed above). Instead, questions
bearing on the semiotic nature of pictures or concerning the existence of iconical signs, the
possibility of dividing up the picture into units having or not having independent meaning,
the layers into which the picture sign may be dissolved, the presence of indexicality in
pictures, the question as to what makes up the specificity of particular picture types, and
the paradox connected with pictures lacking depiction, have all been analyzed (to the
degree that they have not simply been the object of pronouncements) by means of
system analysis.
In a weak sense, we shall take system analysis to mean an attempt to account for,
and systematize, the researcher’s own intuitions pertaining to a semiotic system, of which
he is also a user in his everyday life. But we shall also introduce a stronger sense of the
term system analysis, in which it means a systematic variation of features contained in the
system, to detect the limits of their variability, and their possibilities of combination. Both
Husserlean ’ideation’ and Hjelmslevean ’commutation’ are varieties of this general kind, in
spite of their differences, as is the Chomskyan judgment of grammaticality and
synonymity. In its ideal form, then, system analysis gives rise to a table having (at least)
double entries, and permitting the cross-classification of a number of categories according
to the possibility of putting certain features together. The Peircean apparatus for
classifying signs, which, in its most simple version, involves three times three categories,
is a good illustration of this type of analysis. Eco's work could be ranked with system
analysis in the weaker sense. What we find most of the time in Saint-Martin's books is
also a kind of system analysis, largely informed by psychological studies.
There is, however, yet another method of pictorial semiotics (cf Sonesson
1989b,d,g), which is a kind of hybrid between system analysis and text analysis, and
which might well be the second most common approach to our domain. Text classification,
as it will be called in the following, is similar to system analysis in its developed form in
that it is based on a intersection of two or more conceptual series, the compatibilities of
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
23
which are tried out in the analysis. But these compatibilities are not tested on a purely
conceptual basis, but by means of spotting actual examples of pictures, answering to that
particular constellation of features defining each single case. The result of a text
classification is thus a series of analyses of pictorial texts, but unlike what occurs in real
text analysis, there is no attempt to account completely for the given picture; indeed, it is
characterized only to the extent that it realises that particular constellation of features
which is contained in the conceptual series defining the cross-classification. It is clear that,
should we be able to inscribe a given text in a sufficient number of such categorical
frameworks, we will have ended up with a text analysis, but this prospect is not only
utopian; it also goes beyond the intentions of those who have recourse to this method.
The most significant contribution of this kind inside pictorial semiotics is now doubt
the one given us by Groupe µ (1979; 1980; 1989b,c); but also Sonesson (1989a,I.2.5.)
constructed a skeleton for the analysis of indexicality, in which an array of pictorial
examples are similarly inscribed. In these cases, each picture is treated as the
representative of a class, but there is no claim that the category classified is the dominant
one in the definitional hierarchy of the picture. Normally, we would also expect one
picture to appear in just one square of the table, and this seems true of all the
implementations mentioned above. However, when the present author tried out the most
recent version of the Groupe µ (1989b,c) model on new examples, he managed to place a
single picture in a number of squares (see Sonesson 1989g). This, too, would of course be
a way of approaching text analysis.
System analysis claims to have direct access to the system, whereas the latter is
attained indirectly by the experimental method making a detour over artificial texts. In the
case of text analysis, on the other hand, the object studied is the text(s), while the object of
study, in which we are interested, remains the system. Since it is impossible to start from
zero, the researcher will have to construct a model before he embarks on the analysis, and
since this model cannot be completely arbitrary (in spite of Hjelmslev and Greimas), he
must rely on his intuitions as a user of pictures and, ideally, on his system analysis of
these intuitions; he will then modify the model as he goes along. If the analysis is to be a
test of the model, we must retain the requirement of exhaustivity (as far as the parameters
defined by the model are concerned).
As we noted above, most studies of pictorial semiotics are of the general type of text
analysis, and many of them have, in one or other form, the additional presumption of
being structural. There is, however, a certain dialectical relationship between the three
approaches: we need to have some notion of the system, before we can set the procedures
of text analysis to work; system analysis need to be elucidated using real texts, and the
artificial texts of the experimental approach must be constructed by comparison with texts
which can really be observed ’out there’ in culture. It is a curious fact that, so far, even
those exponents of pictorial semiotics, who have employed all three methods, as for
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
24
instance Lindekens, or at least two of them (as Tardy, in the case of experiment and text
analysis)2, have never brought the results of one of these methods to bear on the other
ones. Thus, for instance, Lindekens did not have recourse to the results of his experimental
analysis of triangles and rectangles in order to build a model for realising a text analysis of
a more complex picture (but using Lindekens’ results, Sonesson 1988 and 1989d did just
that).
This sets the scene for Saint-Martin's attempt: denying herself the possibility of
having recourse to experimental evidence, she would have to show, using the system
analytical or the text analytical method, that Gestalt theory is somehow better equipped
to explain the nature of pictorial meaning than any of the other available doctrines
pertaining to the nature of perceptual processes.
Ontological and epistemological pan-linguisticismLike all semiotical sciences, including linguistics, pictorial semiotics is a nomothetic
science, which, just like linguistics, but contrary to the natural sciences and the social
sciences, is concerned with qualities, rather than quantities. To admit this parallel to
linguistics is not the same thing as embracing the so-called linguistic model, which consists
in transposing concepts and terms derived form the (structural) study of language to the
analysis of pictures. In fact, for the last 15-20 years, numerous exponents of pictorial
semiotics have marked their distance to the linguistic model, but this has often meant a
return to a prestructuralist (paradoxically called poststructuralist), and even pretheoretical,
stage of reflexion, as is the case of the late Barthes, and in part of the work of Damisch,
Marin, Schefer, and Lyotard. Fortunately, it can also result in a more relevant critique of
the linguistic model, and an attempt to establish new kinds of models, as in the case of
Saint-Martin's work (cf. also Carani 1986b, 1988).
Indeed, it is seldom appreciated that the outright rejection of the linguistic model
must be at least as naive, and as epistemologically unsound, as its unqualified acceptance;
for, the use of one science as a metaphor for another involves such a long series of choices
and comparisons, on different levels of abstraction and analysis, that there can be no
rational way of undoing them all at one stroke (cf. Sonesson 1989a,I.1.2. and 1989c). This
is not, however, the kind of criticism which could be addressed to Saint-Martin's work.
Although she often seems to deny all relevance to linguistic comparisons, she actually
retains some of the more current metaphors ("syntax", and so on), without ever inquiring
into the value of such parallels. Thus, she does accept some of the comparisons, but she
fails to justify her particular choice.
We may well doubt that, in a deeper sense, there has even been a linguistic model in
pictorial semiotics. Barthes, Marin, and many of their followers in different countries did
2 In its weak sense, system analysis is of course always involved in the other two.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
25
certainly have recourse, in their attempts to analyze pictures, to a number of terms taken
over from Saussure, Hjelmslev, and Jakobson. When closely scrutinized, these analyses
generally turns out to be concerned with very abstract notions like connectedness (in the
guise of syntagms, syntax, and metonymies) and categorical identity (epitomized as
paradigms and metaphors). Even connotational language, as it is misinterpreted by
Barthes, is introduced as a means of establishing complex networks of meaning (see
Sonesson 1989a,II.1.). The notion of sign is never highlighted, although the terms
’expression’ and ’content’ appear abundantly. Structure is certainly essential to Lévi-
Strauss' analysis of, among other things, the Northwest Coast masks, but it is actually
easy to show (as we did in Sonesson 1989a,I.1.3/5.; 1990d, 1991b, 1992a) that
methodologically, Lévi-Strauss is really putting the structure concept of linguistic
structuralism on its head. In fact, it is only when the ultimate constituents of pictures are
compared to those of verbal language that linguisticism looms large, and this happens, in
particular, in the approach of Umberto Eco (cf. Sonesson 1989a,III.2. and 1989c).
Of course, Barthes, Marin, and others, did think they were making use of the
linguistic model. They failed, however, both because they did not manage to construct
anything sufficiently explicit to qualify as a model, and because their linguisticism had
essentially the derived form characteristic of literary critics and other non-specialists. The
latter remains basically true about the current approach of the Greimas school. No matter
which may be the deformations which the linguistic model imposes on pictorial
significations, they stem less from the linguistic terms as such, than from the distortions
which the latter have suffered in the hands of non-linguistics. With few exceptions,
linguists cannot legitimately be accused of having imposed their model on other brands of
meaning. On the contrary, they should be held responsible for having treated the analysis
of all non-linguistic significations as something spurious, either denying the interest of
their study altogether, or citing these meaning types only in the guise of simplistic
examples at the beginning of introductory courses to linguistics.
As we have noted elsewhere (in Sonesson 1989a,II.1.1), the pan-linguisticism
characteristic of French structuralism seems to be of at least two kinds. While the Greimas
school would seem to adopt, to some extent, the linguistic model, because all meaning is
considered to be similar to the linguistic kind, or to admit of the same treatment, that is,
for ontological reasons, the justifications Barthes appears to have for the same choice are
rather epistemological, and basically opposed to those of the Greimas school. Actually,
Barthes seems to think that semiotical systems other than verbal language are inaccessible
to analysis, and thus can only be attained indirectly, through the way language refers to
them and describes them. To reject ontological pan-linguisticism, we will have to show
that pictures or other meanings are, in some essential respects, fundamentally different
from verbal language. To reject epistemological pan-linguisticism, on the other hand, it is
necessary to demonstrate that there are meanings which are accessible to us independently
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
26
of verbal language, for instance before it is even acquired.
Curiously, Saint-Martin at numerous places seems to attribute such an
epistemological panlinguisticism to representatives of the Greimas school, notably to
Floch. She takes it to be responsible for the tendency to study merely what the picture
represents, without caring for other meanings which less easily lend themselves to
linguistic expression. Actually, even Floch (1978) attacks this tendency in the work of
Barthes, for the very same reason (which is not to say that he may not also be guilty of
the same error on other occasions). But even Saint-Martin (1987a,b; 1990) accepts, to
some extent, epistemological pan-linguisticism: the iconic function of the picture is
repeatedly treated not only as being conventional, along the lines suggested by Eco's
argument, but as deriving essentially from a verbal act of nomination. Yet there is really no
reason, as we shall see when turning to perceptual theory, to accept this brand of
epistemological pan-linguisticism.
Models in pictorial semioticsAs pointed out above, pictorial analyses must, in order to be semiotical, make explicit use
of a model. A model will be understood to be a scheme made up of interconnected,
recurrent categories which are sufficiently explicit to permit its coherent application to a
series of ’texts’ (that is, in our case, pictures), thereby engendering a description of the
’texts’, in the process of which the model is further enriched.
Barthes seems to be using a kind of model in his Panzani analysis, at least when
positing the existence of pictorial denotations and connotations. Unfortunately, no clear
meaning can be attached to these terms, since Barthes only rarely have recourse to
Hjelmslev’s understanding of this terminological couple, and since very different
phenomena are subsumed under the term connotation (cf. Sonesson 1989a,II.1.). What is
transposed from Barthes’ famous article into a disturbing number of analyses conducted in
several languages is therefore not a model, but a family of expressions.
In the following, we will distinguish three models: the Greimas school model, the
Groupe µ model, and Saint-Martin’s model. Henceforth, they will be called the G model,
the µ model, and the SM model, respectively.
According to the earlier version of the µ model, the picture contains a number of
layers, each having its particular norm: roughly, the matter, substance, and form of both
expression and content and of both the plastic and the iconic level, (cf Groupe µ
1979:178ff)3. On the iconic level, the picture stands for some object recognizable in the
3 These terms are taken over from Hjelmslev, though “graduality” has been substituted for
“form” in order to insist on the less systematic character. of the organization For a full
critique of this model, including some doubts on the Hjelmslevean orthodoxy, cf. Sonesson
1988.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
27
ordinary perceptual world; whereas, on the plastic level, expression appears to be
conveyed by simple qualities of the picture thing itself, which tend to have abstract
concepts as their content. In the case of plastic discourse , the norm is said to be
isotopicality, or more precisely, isomateriality of the expression matter, and allotopicality,
which is to say, allograduality of the expression graduality — more simply put, the norm
requires there to be a single type of material employed in the composition of the picture,
while at the same time distinctions are expressed between the intervening signs so as to
make it possible to hold them apart (cf. Groupe µ 1978:17ff). The purpose of the analysis
is to establish a repertory of different rhetorical figures, or deviations from the norms, and
there are two standards from which the figures may diverge: the so-called generic and local
degree zero. While the first is defined already in the system, the second is separately
produced in each particular ’text’ (cf. Groupe µ 1980:252).
In its more recent version, the µ model apparently dispenses with all the different
layers, retaining only the distinction between iconic and plastic language, as well as the
two kinds of norms. Rhetorical figures may now be purely iconic, or purely plastic; or
they may be iconico-plastic, in which case a divergence from what is expected in one layer
of the sign is reduced to normality with the aid of information contained in the other layer.
Both plastic and iconic figures are divided according as they are in absentia or in
praesentia, and conjoint or disjoint; whereas they iconico-plastic figures can be disjoint or
conjoint, and either plastic with iconic redundance, or the reverse. The possibility of
distinguishing separately such units as have present or absent, as well as conjoint or
disjoint, elements, results from the multidimensionality of pictures not found in verbal
language: two entities are susceptible of appearing together, without occupying the same
place.4 Here, the norm is supposed to prescribe the coincidence of plastic units with
iconic ones, a well as the concurrence of the three kinds of elements pertaining to shape,
colour, and texture (see Groupe µ 1989b; for a systematization and examples, also cf.
Sonesson 1989g).
In both its variants, the µ model is a text classification. Although it equally starts out
from a separation of the iconic and plastic layers, the G model has been designed to guide
text analysis. The second operation of this model thus involves the partition of the entire
4 It is difficult to explain these notions without entering very complex discussions, or
recurring to elaborate pictorial examples. One may wonder if this cross-classification of
rhetorical devices into those the elements of which are present or absent, as well as
disjoint or conjoint, is really adequate, for only the four resulting categories really seems
intelligible. Actually, Groupe µ here touches questions which we have treated under the
general heading of indexicality, in Sonesson 1989a,I.2.5., which is not to say that our
analysis is preferable, and it would in any case have to be integrated with wider rhetorical
issues.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
28
pictorial field into two parts, according to some or other criteria (In some cases, an
immediate separation into more than two fields is later reduced to the twofold division).
One of the parts, or both of them, will then be further divided into smaller division blocks,
and the procedure may sometimes continue on still further levels. All these segmentations
are then justified by means of listing bundles of binary oppositions, the relata of which are
manifested in different fields resulting from the earlier division. At some point of the
analysis, a proportionality of the kind familiar from Lévi-Strauss’ myth studies tends to
appear, and it is shown that, inside the plastic layer, or between the iconic and plastic
layers, or elsewhere, there is some A which relate to some B, in the same way as some C
is related to some D (for further discussion, cf. Sonesson 1989a,II.3.).
This description corresponds mainly to the Greimas model as used by Floch. The
fundamental difference between Floch and Thürlemann is really one of method: although
both the approaches differ from one article to another, Floch generally tends to begin from
global configurations and work his way down to features and feature oppositions, whereas
Thürlemann, in particular in the Klee analyses, takes his point of departure in supposedly
minimal elements, using them to build up larger configurations. Moreover, binary divisions
appear to play a much less preponderant part in Thürlemann's work.
The SM model, too, would seem to be intended to function text analytically, but its
coverage is much more extensive. From the very start, Saint-Martin imposes on the
picture a grid containing five times five division blocks, to which is added, inside each of
the resulting squares, a fivefold segmentation separating the four sides and the central part.
Each one of these 125 portions of the picture, called coloremes, must be assigned a value
on six different dimensions: colour/tonality, texture, dimension/quantity, implantation into
the plane, orientation/vectorality, and frontiers/contours generating shapes. Moreover, the
coloremes are connected with each other by means of three kinds of rules determining
visual syntax: the interaction of colours; topological relations, such as proximity,
separation, encasing, envelopment, and orders of succession; and finally Gestaltian
relations, such as figure/ground, contiguity, similarity, closure, good forms, vectorality and
familiarity.
The visual variables, when appearing in a picture, are further constrained by the
energetic potentialities of the basic plane , which is supposed to contain four formative
angles, two diagonals, which give rise to two pairs of partly overlapping triangles, a
cruciform, and a lozenge. In addition, the model attends to the variable denominated
implantation into the plane, distinguishing around two dozen different perspective and
distance effects. This description concerns the syntactic part of the model; a projected
book on visual semantics is still to be published (for a full critical review, cf. Sonesson
1992a)
It should be clear from our description that Saint-Martin’s model offers a much more
exhaustive analysis than the models suggested by Floch, Thürlemann, and Groupe µ: it
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
29
actually scrutinizes every single point of the pictorial surface. From one point of view,
this is certainly an advantage of the model: such a procedure may constitute a good check
on many an overly adventurous hypothesis Yet, it also constitutes a problem: the model
seems to engage us in a procedure which has no natural stopping point, and it does very
little to guide us in the selection of relevant traits, presupposed not only be every picture
analysis, but even by the plain perception of a picture. What is lacking, then, a a principle
of pertinence, determining what to attend to in the picture under analysis (No doubt Saint-
Martin may be reserving this surprise for us in her forthcoming semantics book). Such
principles of pertinence may well turn out to be too restricted, and too ambiguous, as we
have shown to be the case of the binary divisions favoured by Floch (Cf. Sonesson 1990d;
1992a). Or they may be geared merely to react to exceptional pictures, and only to
isolated properties of the picture, as could be true of the norm and deviation model
propounded by Groupe µ. Yet, a model which leaves us such a freedom as the SM
variety is very close to being pointless.
Although the idea is nowhere clearly stated, some kind of principle of pertinence
may well be embodied in Saint-Martin's description of the pictorial base plane. Moreover,
we may take the very fact that she has cared to write an entire book about topological
relations, and another one about Gestalt theory, to mean that she would consider topology
and Gestalt relations to occupy a particularly important position among the configurations
of visual meaning. These are possibilities which we will have to explore in the following.
The semiotics of iconicity and visualitySo far, we have been supposing that Saint-Martin's work is concerned with pictorial
semiotics; and it is, in actual fact, mostly devoted to the study of pictures. Yet, as the
titles of some of her books suggest, Saint-Martin is actually intent on studying a much
wider domain, that of visual semiotics, which, in her view, should comprise architecture
and sculpture, in addition to pictures. In one of her books (1987a:98), she claims that they
are all characterized by the same visual variables, which are then differently constrained,
according as they are inserted into the basic planes of pictures, the virtual cube of
sculpture, or the environmental cube singularizing architecture.
Unfortunately, Saint-Martin does not further discuss architecture. She has, however,
the merit of having studied almost for the first time the essential coordinates determining
sculptural meaning. Even earlier, Dora Vallier compared sculpture, painting, and
architecture in a series of articles, but her approach is based on badly understood linguistic
analogies, in addition to being entirely aprioristic in character. In contrast, Saint-Martin's
approach seems much closer to the subject matter. We must however address a much more
fundamental issue: is there, or could there be, such a domain as visual semiotics?
Even Preziosi (1983) conceives of architecture as being a kind of visual semiosis,
which he then opposes to linguistic meanings, identified with auditive semiosis. Roman
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
30
Jakobson has treated of the differences between visual and auditive signs, and Thomas
Sebeok has divided up semiotics according to the sense modalities. On the other hand,
from the point of view of Hjelmslevean semiotics, we would normally not expect
visuality, being a mere ’substance’ or even ’matter’, to determine any relevant
categorisations of semiotic means.
Although this type of argument is often made, it is based on a confusion of the terms
’substance’ and ’matière’, as employed by Hjelmslev, and in their ordinary usage. Thus,
the term ’matière’, to Hjelmslev, is simply that which is unknowable, and, as a
consequence, not susceptible of being analyzed; that is, it is the residue of the analysis;
and ’substance’, which, in the earlier texts, is the term used for ’matière’ in the above-
mentioned sense, stands, in the later works, for the combination of ’matière’ and ’form’.
Thus, ’substance’, in the early works, and ’matter’ later, simply means ’that which is not
pertinent relative to the other plane of the sign’ (see discussion in Sonesson 1989a,II.4.
and 1988); it does not necessarily stand for matter in the sense of ordinary language, the
material of which something is made, the sense modality (as Groupe µ, for instance, must
supposes when making ’allomateriality’ into one of the possible characterizing traits of
the collage). If the material or the sense modality turns out to be relevant in relation to the
other plane of signification, it becomes form.
More importantly, the psychology of perception certainly seems to suggest the
existence of some common organization which puts all or most visually conveyed
meanings on the same level.. If, as we have argued, all signs must also be objects of
perception, there is every reason to believe that the modality according to which they are
perceived determine at least part of their nature. There may, of course, be other, perhaps
more fundamental division blocks of semiosis, of which pictures form a part (that of
iconicity, for instance). If we accept the legitimacy of the domain of visual semiotics,
another problem is brought into view: why should such a domain merely comprehend
pictures, buildings, and sculptures? No doubt, in suggesting such a list, Saint-Martin is
influenced by the traditional divisions of fine art. From a semiotical standpoint, visual
semiotics should have to comprise much more. Some significations are only partly visual,
as those of theatre communication. Others might be considered not to have an intrinsically
visual organization, such as writing, the conformation of which depends in part on spoken
language. But all kinds of gestures and bodily postures, objects, dummies, logotypes,
clothing, and many other phenomena must be counted as visual signs and significations. In
fact, even visual perception per se supposes a pick-up of meaning of sorts. We therefore
have to face the arduous task of determining the ways in which the various kinds of visual
semiosis differ.
Even though pictures may form part of the division of visual signs, they are no
doubt also members of the category of iconic signs — and yet they are not the only kind
of iconic signs there is. Iconicity is often wrongly taken to be that which is peculiar to
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
31
pictures. Indeed, Eco’s plaidoyer against the existence of iconical signs most of the time
reads like an argument against the specificity of pictures. To Peirce, an icon is a sign which
is based on similarity; or, more strictly, a sign consisting of an expression which stands for
a content because of properties which each of them possess intrinsically.5 ; This means
that, not only do iconic signs abound in sense modalities other than vision, but there may
also be visual, iconic signs which are not pictures.
Peirce only distinguished three subtypes: the image, the diagram, and the metaphor.
Without any claim to exhaustivity, we have opposed the picture to a set of other
iconically motived signs, including the metaphor, the dummy, other self-identifications and
exemplifications; and the symbol, in the traditional European sense, that is, as preserved in
the name for the artistic movement ’symbolism’ (see Sonesson 1989a,II.2.2. and III.6.).
A primary requirement which should be imposed on pictorial semiotics is to
determine the categories of which pictorial signs are subcategories, and to show that the
latter are in fact so related to the former. In the second place, we need to specify the
differences between pictorial signs and other members of the same superior categories.
Thus, although it may be evident that pictures are indeed visual signs, we need to show
that they are intrinsically visual, that is, that visuality is part and parcel of their
Hjelmslevian form, that which could not be exchanged without the sign becoming another
sign having a different meaning. And we have to determine in which way pictures differ
from other, intrinsically visual signs, not only (if that is really another, intrinsically visual,
sign) from sculpture. Also, we have to show (against Eco, Goodman, and others), that
pictures are, in one or another sense, iconical signs, and that pictorality is a peculiar
modification of iconicity (see, in particular, Sonesson 1989a,III.3.).
From psychologies of perception to theories of signs and iconicityFernande Saint-Martin is right, we believe, in claiming that pictorial signs are inherently
perceptual, that is, visual, in nature. The model of the sign as an object of perception
deployed by the Prague school seems much more obviously relevant in the analysis of
pictures than in that of literature, for which it was essentially developed. Not only must
such a claim be justified from a discussion of perceptual theory, but some criteria must
also be proposed for choosing as a foundation one among the several conflicting theories
occupying the contemporary scene of perceptual psychology. In selecting this perceptual
theory, we will not be able to share entirely the convictions of Saint-Martin. Moreover, if
5 We must dispense here with Peirce’s idiosyncratic terminology. For reasons explained in
Sonesson 1989a,III.1., we will take “representamen” to be roughly equivalent to
“expression”, whereas “object” corresponds to both “content” and “referent”. The
Peircean “interpretant”, on the other hand, appears to be a determination of the relation
between the former two.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
32
all perception turns out to carry meaning, we are faced with the further task of determining
in what way visual signs, such as pictures, differ from mere meanings conveyed be visual
means. In working out such a distinction, we will again have to part company with Saint-
Martin.
On the choice of a perceptual theoryEven granted that the pictorial sign is an object of perception, it remains to be determined
whether it is a Gibsonian object of perception, a Gestaltist one, a constructivist one, or
perhaps even something else. The choice of a perceptual theory which is to serve as a
foundation to pictorial semiotics is by no means as easy to accomplish as Saint-Martin
would like to think.
There are, in present-day psychology, basically three ways of conceiving the
relationship between that which is perceived and the cause of the perception: and the three
corresponding theories are those of constructivism, Gestalt psychology, and direct
registration theory, or Gibsonianism (Hagen 1980:4ff; 1979; Winner 1982:84ff; Sonesson
1989a,III.3.3.). It is the contention of the latter theory, that all information needed is
available directly in the light coming from the environment, and is determined by this light,
although only if we take into account all the higher-order variables of the environment and
their invariants over time. According to Hagen, constructivists like Gregory and Gombrich
claim that reality lacks all intrinsic organization, and so must be set in order by a
hypothesis on the part of the perceiving subject; but the resulting arrangement is only
given with a certain degree of probability, and may have to be further revised. Again
according to Hagen, Gestaltists such as Arnheim and Hochberg (sic!) would agree with the
constructivists in affirming that reality is fundamentally ambiguous, and so must be
supplemented by the beholder's share, but, in their view, the perceived organization
results deterministically from the Gestalt laws, built into the human mind. Also, while the
Gestalt laws, or at least the simplicity principle on which they are based, are supposedly
innate, constructivists rather tend to suppose that the hypotheses employed in perception
are either explicitly posited as conventions, or derive in a more tacit fashion from earlier
experience of the world (cf. Winner 1982:108). Surprisingly, Saint-Martin (1990:86 )
affirms, on unclear evidence, that no such innateness is required by Gestalt theory.
Hagen maintains that all three theories are descriptively inadequate: constructivism
because no criteria have been proposed for when a hypothesis is confirmed; Gestalt
psychology, because its laws are mysterious; and Gibsonianism, because no list of the
invariants picked up from the environment can at present be given (p.21ff). In spite of
these observations, however, Hagen herself clearly remains within the bounds of direct
registration theory. This is precisely the theory which Winner (1982:98ff) declares to be
descriptively inadequate. On the other hand, she argues that there are cases in pictorial
perception, in which simplicity may be shown to override familiarity, thus favouring
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
33
Gestalt psychology, as well as other cases in which familiarity gains the upper hand,
which is a result favouring constructionism. Contrary to Hagen, Winner thus concludes
that reality is ambiguous, but may be supplemented in various ways.
It is natural that, in her book on Gestalt theory, Saint-Martin should neglect
constructionism and direct registrations theory in favour of the Gestalt school. However,
it is as pity that these movements are never ever mentioned as such, and thus are never
presented as the alternative conceptions which in fact they are. Gibson is even introduced
as a continuator of Gestalt theory (1990:58), only to be later attacked as the apostate he
must thus appear to be. Having never tired, in his numerous publications, of criticizing
Gestalt psychology, Gibson would have been surprised and shocked by this suggestion.
On the other hand, it is true that he has always recognized in Koffka one of the most
important influences on his thinking.
In fact, there seems to be no real Gestalt psychologists left, except for those who are
rather to be counted among the students of pictorial art, as, for instance, Arnheim. There is
undoubtedly an array of phenomena, discovered by the Gestalt psychologists, which are
still with us, but which now are in need of new explanations. The theoretical stance taken
by Hochberg, who Hagen treats as a Gestaltist, is, in actual fact, that of constructivism, as
he himself affirms. He has, on the other hand, undertaken a critical appraisal of the Gestalt
tradition. Sometimes, he claims, the most natural three-dimensional interpretation of a
picture is not the simplest one, as Gestalt theory would make us expect (Hochberg
1972:59f). Gestalt phenomena are really peculiar cases of Helmholtz' law, according to
which we perceive that which is most probable, given the pattern of stimulation
(Hochberg 1980:58f; cf. 1974;196ff). The ’minimum principle’ cannot be due to a built-in
perceptual mechanism that makes us perceive always the simplest object fitting the overall
stimulus pattern, Hochberg (1978a) claims, and goes on to suggest that, instead, it may be
an effect of putting together fragmentary sensory data, in a manner corresponding to the
most likely object, or that it may result from the arrangement that has the best chance of
being seen and remembered from one momentary glance to another.
Indeed, Hochberg (1972:60) even claims that the very fact that perceptual objects
must be grasped in a long series of momentary glances imposes limitations on the validity
of Gestalt organization, since different parts of the whole will fall on the fovea at different
moments. Saint-Martin (1990:28f), who notes this last points, takes Hochberg to task for
neglecting the coherence which Gestalt psychology takes to persist from one glance to
another, and for supposing only that portion which is reflected in the fovea to be actually
perceived. In other writings of his, not quoted by Saint-Martin, Hochberg certainly gives
due attention to peripheral seeing. Yet it may be true, as we have argued in discussing
pictures of impossible objects (in Sonesson 1989a,III.3.4.), that Hochberg exaggerates the
importance of foveal perception. The same could be said, however, about the notion of
coloreme, defined by Saint-Martin (to which we will turn below).
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
34
In perceptual psychology, the really interesting discussion nowadays takes place
between constructionism and direct registration theory. But why should Gregory,
Hochberg, and others think that ’inferences’ are necessary to explain what is actually
perceived, when Gibson, Kennedy, and Hagen feel they can dispense with them
altogether? Among the facts to be explained by perceptual psychology, figure
prominently, in Gregory's view (1966:1974), such things as the pick-up of non-optical
properties, gaps in the stimuli, visual illusions, ambiguities, illusory contours, and the
perception of logically impossible objects. To Gibson, on the other hand, most of these
phenomena are simply curiosities, of very little weight to everyday perception, and
therefore to perceptual psychology. Thus, one of the differences between the theories lies
in the choice of facts which they consider worth-while explaining. Yet, it is perhaps not
beside the point to argue about which facts we should care to explain.
One of the most celebrated pioneers of constructionism, Ulric Neisser, has lately
recognized the necessity of accounting for the fact that ordinary perception usually proves
right. Just like Gibson claims, information is picked up from light, Neisser (1976: 16, 20ff)
grants, but this pick-up only serves to start a perceptual cycle taking place in time:
anticipatory schemes generate generic, rather than specific, hypothesis which are modified
by the information available, engendering subsequently more detailed schemes, which
guide the further exploration of the optic array. In his latest publications, Neisser (1987)
seems even more convinced of the fact that, as Gibson affirms, information for that which
is perceived is present in the array of light available to the eye, as soon as we attend to
higher-order variables, and their modification over time. Even categorisation is now said to
be ecologically grounded, though somewhat ’less direct’. A theory which has made a
convert, a rare occurrence in any scientific community, is not so easily refuted as Saint-
Martin (1990:58ff) appears to believe.
There thus seems to be a fourth alternative in perceptual psychology, not recognized
neither by Hagen nor by Winner, which amounts to a blend of direct registration theory
with some facets of constructionism. This is Hochberg's position, as it was that of Neisser
at least as late as in 1976. Although Saint-Martin often quotes Piaget to help buttressing
the common bias of constructivism and Gestalt theory, the latter's general conception,
according to which both assimilation and accommodation are involved in our ordinary
experience of reality, certainly seems to suggest that he, too, would favour a mixed
approach. In the next subsection will will suggest some reasons for taking this alternative
seriously.
Thus far, it will be noted, we have been mainly concerned with the perception of the
world, and with the relation between the assumed physical cause and the resulting
percept, not with picture perception. No doubt, the whole issue appears to be roughly
analogous to that of pictorial iconicity, the relation between the picture and ordinary
perceptual reality reproducing, in that order, that between the ordinary percept and its
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
35
physical cause. We should therefore expect constructionists and Gestaltists to favour a
version of a conventionalist theory of picture perception, and Gibson to defend a
similarity theory, but the opposite turns out to be closer to the truth.
Not only do Gestaltists and constructivists (with the exception of Hochberg) treat
pictures and reality as being of a kind, but most of their reasoning is based on pictorial
examples, although their conclusions concern the perception of the real, three-dimensional
world. Gibson actually argues that their theories are artefacts of their having studied
pictures rather than reality. And he goes on to claim that pictures are not at all based on
similarity. Yet he certainly does not want to maintain that they are conventional, in the
way semioticians would use that term: instead, because of being so different from the
perceptual environment, they must render the invariants of perception, and convey them
to us, in a very different way from that in which they become manifest in the real world. It
is thus misleadingly that Saint-Martin (1990:15) quotes Gibson as saying, like Piaget, that
depicted objects are not perceived: they are indirectly perceived, as he continues the
phrase elsewhere.
No doubt, Saint-Martin is not interested in the analogy between physical cause and
percept, on one hand, and picture and reality, on the other: her concern with perceptual
psychology has nothing to do with pictorial iconicity. We will later discuss Gestalt theory
from the point of view which preoccupies Saint-Martin. Yet, it is important to pursue the
discussion also according to the facets neglected in her argument.
From the Gibsonian environment to the Husserlean LifeworldOne peculiar thing about the so-called ecological psychology devised by James Gibson is
that it is, on so many counts, remarkably similar to the phenomenological philosophy
conceived around the turn of the last century by Edmund Husserl. Gibson, is has been
reported, was very interested in philosophy; he was even accused of trying to resolve
philosophical puzzles empirically (cf. Lombardo 1987; Reed 1988:45). Apparently,
Gibson had some knowledge of Husserl's work. Yet, from reading Lombardo and Reed,
one gets the impression that neither they nor Gibson have any idea of the degree to which
their conceptions coincide. No doubt, apart form being a good psychologist, Gibson was
also an excellent phenomenologist.
Scientists organize experiments and execute analyses, which establish the limits of
what may be reasonably thought about a particular subject, but no amount of facts will
ever engender a theory on its own. Facts are like indices of reality; but they have to be
bound together by abductions, in Peirce's sense: general rules and regularities which are
taken for granted and which link one singular fact with another. The Ancient Greeks and
the Chinese may have observed at least in part the same stars; yet each group put the
starts together according to the fashion of their particular sociocultural Lifeworld, the first
forging 48 constellations, and the second 283. In the same way, given the same facts, there
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
36
may be Greek abductions, and there may be Chinese abductions.
Peirce wondered how it was possible that so many abductions prove right,
postulating a natural instinct as an explanation. Actually, there is an infinite number of
ways to relate starts, and facts, but most of them would seem to be humanly
inconceivable. It is difficult to say if they are therefore right, in an absolute sense. The
limited number of alternative abductions being really proposed may be due, not to a
natural instinct, but to the commonality of the most general organizational framework of
the Lifeworld, in Husserl's sense, that is, of the ’world taken for granted’, as Husserl's
discipline Alfred Schütz also called it. And psychologists, like philosophers, inhabit the
common Lifeworld; sometimes, they ar even more or less aware of it.
There are differences between the constructivists, the Gestaltists, and the
Gibsonians, which have to do with which experiments they consider relevant, and to
which properties of the experimental results they attribute most importance, but this in
turn must be due to the way they, as laymen, inhabit our common Lifeworld, and how
consciously they relate to it. This does not necessarily mean that the differences between
these psychologists are mere disparities of taste and personal predispositions; for they
could as well be explained by their different aptness for the difficult task of doing
phenomenology.
This is not the place to describe in details the organizational framework of the
Lifeworld (cf. Sonesson 1989a,I.2.1.), or the similarities between Husserl's Lifeworld and
the Gibsonian environment (cf. op.cit.,III.3.2.). Suffice-it to point to a single, importance
coincidence. When Gibson (1978:228) observes that, when we are confronted with the-
cat-from-one-side, the-cat-from-above, the-cat-from-the-front, etc., what we see is all the
time the same invariant cat, he actually recovers the central theme of Husserlean
phenomenology, according to which the object is entirely, and directly, given in each one
of its noemata (see Husserl 1939, etc.). Husserl's cube and Gibson's cat exemplify the
same phenomenal fact — for it remains a phenomenal fact, and not an experimental one,
also in Gibson's work.
Whereas Husserl called into question the conception of his contemporary
Helmholtz, according to which consciousness is like a box, in which the world is
represented by signs and pictures, from the fragmentary pieces of which we must
construct our percepts, Gibson's strawmen are the followers of this same Helmholz, who
claim that hypotheses are needed to build up perceptions from the scattered pieces offered
us by the sensations (cf. Sonesson 1989a,III.3.). The two arguments appear to proceed
along different lines, but converge at the end. Husserl rejects what has been termed ’the
picture metaphor of consciousness’, showing Brentano and Helmholz to have an
erroneous conception of the very pictures and other signs to which they compare
consciousness, when they ignore the transparency of their expression to the content.
Gibson (1971; 1978), on the other hand, emphasizes the dissimilarity between the picture
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
37
and the real-world scene, to show that conclusions dealing with the real world which are
based on experiments with picture perception are seriously misguided. Yet, to both
Husserl and Gibson, normal perception gives direct access to reality, and while Gibson
thinks pictures represent a kind of indirect perception, Husserl tells us that they are
’perceptually imagined’. Indeed, this is the sense in which Gibson's phrase, quoted by
Saint-Martin, according to which depicted objects are not perceived, should be taken.
It is precisely this phenomenal observation, to the effect that perceptual objects,
rather than piecemeal perceptions, are that which is perceived, which Saint-Martin
(1990:58ff) finds unacceptable. Not surprisingly, she finds the same faults (p. 27) with a
disciple of Husserl, Merleau-Ponty, who also claims that perception is of the object, not
of its appearances. In a way, she is right in affirming that this proposition cannot be
verified in a laboratory; but it is verified by each and every instance of human perception.
Children's drawings at first render ’things’ (and abstract properties) instead of their
appearances, because the capacity to see appearances must be laboriously acquired (cf.
Sonesson 1989a,III.3.2.). In the ’naive attitude’, Gibson (1971:31f) affirms, we look
through a series of perspectives in movement to the invariant features of the object, while
in the ’perspectival attitude’, we fix a single perspectival view in order to consider it in its
own right. Roughly the same opposition exists between the ’natural attitude’ and
’phenomenological reflection’ in Husserl's work.6 Moreover, Husserl would argue, just
like Gibson (quoted by Lombardo 1987: 350), that what is ’seen-now’ and ’seen-from-
here’ specify the self rather than the environment.
It is an misunderstanding to believe, as Saint-Martin (1990:11) does, that Gibson
supposes there to by any kind of ’pre-established similarity’ between human knowledge
and the objects of this knowledge. This description is based the idea, defended by
constructivism and Gestalt theory alike, that the communication between the world and
the mind is somehow interrupted. Constructivists and Gestaltists assumed that something
must be added to the information given, because it seemed to them that only impoverished
information could be available. It is the merit of Gibson (and here he goes well beyond
phenomenology) to have shown that all the information needed is actually there to be
picked up, once we realize that the perceptual system is able to attend to higher-order
properties of the array of light, in particular as they change over time. In terms more
familiar to semioticians, it is a question of determining what kinds of units form the
pertinent input to the perceptual system.
After some hundred odd years of discussion about what must be add to the stimuli,
6 Saint-Martin (1990:60ff) criticizes an earlier variant of this distinction, in terms of "the
visual world" and "the visual field", because it seems to suppose that "sensations" persist
to no purpose whatsoever in the human organism, but she fails to note that Gibson
himself rejected the distinction in this form in his later work.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
38
in order for perception to occur, the claim that reality is perceived directly may appear
much too Gordian a solution. Also, Husserl's position is, on the face of it, more
sophisticated, since what is directly perceived in his view is some kind of object internal
to consciousness (although the difference is ’reduced’ away), in fact a Lifeworld object,
not a physical one. Indeed, Gibson (1982:106) observes that he is concerned with
properties noticed by phenomenologists, but that he assumes them also to be real. On the
other hand, although he was certainly committed to some kind of psychophysical
parallelism in the earlier versions of his theory, he later (1982:217) argued that ’ecological
physics’ must be distinct from the ordinary one, and that its invariants were of a quite
different order. Indeed, the kind of ’implicitly known regularities’ prevailing in the world
of Gibsonian ecology are not very different from the ’customary ways things have of
behaving’ in Husserl's Lifeworld.
The similarities between Husserlean phenomenology and Gibson's ecological
psychology are not merely of anecdotal interest. For, whatever we may think of its
ultimate philosophical postulates, phenomenology constitute a exceptionally careful
description of reality as it appears to us, when closely scrutinized. As a consequence, the
coincidence between this description, and that offered by Gibson, guaranties that
ecological psychology is really concerned to explain perceptual reality as it is, and not
some artifact of the historically evolved theories of perception.
The real problem with Gibson's and Husserl's conceptions, is that they do not take
the argument far enough. Not only do we not see sensations, but real objects, but we do
not perceive geometrical volume as such, but a cultural-laden object, not a cube but a dice,
not the tea cup formula but the tea cup itself, not the cat as a geometrical shape in
movement, but that peculiar domestic animal of the Occidental Lifeworld (cf. Sonesson
1989a,I.2.2. and III.3.2.). In an interesting discussion of the changing meaning given by
Gibson to the notion of direct perception through the years, Costall (1989:10ff) makes a
similar observation, concluding that no example of human perception could ever count as
direct on Gibson's terms. Yet, the only world we could ever directly perceive is the world
of our own culture. Just as some disciples of Husserl, as for instance Schütz, discovered
the sociocultural character of the Lifeworld, Costall thus points to the cultural overlay of
the Gibsonian environment. In a way, therefore, constructions and unconscious inferences
are really there: they are only much more deeply embedded.
Plastic and iconic layers of the picture.There is presently a kind of consensus for distinguishing the plastic and iconic layers of
the picture. Earlier made in other terms by Lindekens, the distinction is now incorporated
into two of the leading models of pictorial semiotics, that of the Greimas school and of
Groupe µ. Only Saint-Martin would ignores the distinction. It is indeed problematical, in
a number of ways, only one of which will be discussed here, and yet it should no doubt be
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
39
made, in one way or another.7.
According to this conception, roughly, the picture stands, on the iconic level, for
some object recognizable from the ordinary perceptual Lifeworld; whereas, on the plastic
level, the expression is conveyed by simple qualities of the picture thing itself, which tend
to correspond to increasingly abstract concepts. As used in semiotics, on the other hand,
iconicity is unavoidably connected, in some way of other, with Peirce’s concept of icon,
even when, as in the Greimas school approach, is has been redefined to mean something
like ’the illusion of reality’, or to correspond to ’verisimilitude’, as it is also found in
literature. But the iconicity of the iconic layer is not the same at that of the general sign
theory formulated by Peirce: notably, plastic features, in the sense of the µ and G models,
may well be iconic in Peirce’s sense!
In order to avoid some of the problems posed by Peirce’s definition, it will be
suggested here that two items may share an iconic ground, being thus apt to enter, in the
capacity of being its expression and content, into a semiotic function forming an iconic
sign, to the extent that there are some or other set of properties which they possess
independently of each other, which are identical or similar when considered from a
particular point of view, or which may be perceived or, more broadly, experienced as being
identical or similar, where similarity is taken to be an identity perceived on the background
of fundamental difference (cf. Sonesson 1989a,III.1-3. and below).
There are many varieties of iconic grounds, however. If a particular iconic sign gives
us an illusion of literally seeing in the twodimensional surface of the expression plane the
projection of a scene extracted from real world of threedimensional existence (with or
without a suggestion of lineal perspective), then it is more particularly a pictorial sign. The
symbol, in the sense in which this term is ordinarily used, not by Peirce, but in the
European tradition, is also a kind of iconic sign, having in addition certain indexical traits: it
reposes on the isolation of an abstract, not necessarily perceivable, property, connected
with a generalization from the object serving as an expression, and a particularization from
the object serving as a content (a dove standing for peace, scales signifying justice, etc.).
There are other types of iconical signs as well (cf. Sonesson 1989a,III.6.).
A pictorial sign is a sign the primary allofunctional relation of which is pictorial, in
the sense defined above. Pictures also tend to manifest a secondary function, which,
following Floch and Groupe µ, we will call plastic, in the case of which meanings are
derived from the properties which the expression plane of the picture really possesses,
when considered as made up of mere twodimensional shapes on a surface. In this sense,
however, the plastic layer may well function iconically. Thus, for instance, if the circle is
seen to convey softness, and the rectangle signifies hardness, to pick up some of the
7 For a discussion of another problematical facet, connected with the use of the
Hjelmslevean term “matter”, cf. Sonesson 1988,1.3.7.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
40
results obtained by Lindekens (1971) in one of his experiments, then there must be some
properties mediating synaesthetically between the visual and tactile sense modalities.
When the circle is declared to be feminine, on the other hand, and when the triangle is said
to be calculating, and the rectangle mathematical, much more conventional elements would
seem to enter the semiotic function.
This certainly suggests that the meanings conveyed by pictures are much more
varied and complex than what present-day models admit: for the moment, however, we
would be well-advised to distinguish, not the plastic and the iconic layers, but perhaps the
plastic and pictorial ones, both of which may have an iconic function (for the full
argument, cf. Sonesson 1990e). In this sense, the problem of iconicity, as customarily
discussed in semiotics, is mainly concerned with the pictorial function. There could be a
problem of iconicity also in the case of the plastic function, yet other issues may have to
be resolved before it comes into view. In the rest of this section, we will attend to some
problems of pictorial iconicity, and in the next we will turn to the questions involved with
plastic language. The latter are central to Saint-Martin's concern, whereas the former
appears to be largely neglected.
The critique of pictorial iconicitySince we have discussed pictorial iconicity extensively in other contexts (in particular in
Sonesson 1989a,III; 1990e,f, 1992a), we will only summarize relevant aspects of the
argument here. Saint-Martin (1987a: xvi; 1990: 2ff) always passes over the question
rapidly, apparently subscribing to some version of Eco's conception, but insisting that
pictorial iconicity is linguistically determined (cf. Saint-Martin 1987b).8 She is thus
subject, like Barthes, to some variant of epistemological linguisticism, although she is eager
to demonstrate that what we (following the Greimasians and Groupe µ) have termed
plastic meanings, are differently conveyed. If it can be shown that Eco's conventionalist
theory is mistaken, Saint-Martin's linguisticism will also appear to be dispensable.
Bierman, Goodman, and Eco, have all argued against using similarity as a criterion in
the definition of iconical signs and/or pictures; and even Burks and Greenlee have
introduced some qualifications on Peirce’s view which serve to emphasize
conventionality. To an important extent, these arguments are erroneous, among other
things because they are based on an identification of the common sense notion of
similarity with the equivalence relation of logic. Differently put, they are inadequate
because they suppose man to live in the world of the natural sciences when in fact he
always inhabits a particular sociocultural Lifeworld. The most immediate consequence of
this fact is that many of the conventionalities attributed to pictures turn out to be inherent
8 In fact, Saint-Martin has written an article in the title of which she declares her intention
of formulating Eco's model anew, but I have unfortunately been unable to see it.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
41
in the particular Lifeworld. This means that, whenever some peculiarities of an individual
or a thing, some traits of the woman or the zebra, are locally given importance, they also
make up the features given primary importance in a picture (cf, Sonesson 1989,III.2.2/6.).
But some more general principles will also follow. Pictures, being a kind of visual
thinking, are required to follow the phenomenological rule of all thinking, according to
which an object can only be seized each time from a particular point of view, and not in its
entirety, which means a choice has to be made among the proper parts, the perceptual
parts, and the attributes of the object. Moreover, much thinking, also that which occurs in
pictures, occurs in terms of prototypes, that is to say, by means of construing an object as
an approximation to a more characteristic instance of the same class; and even abductions
and simple structures often intervene in the constitution of pictorial sings (ibid.,III.2.2.
and III.2.7.).
Similarity, then, is really asymmetric and irreflexive. Indeed, this fact is not only
intuitively obvious, but has now been experimentally demonstrated (notably by Rosch &
Tversky; cf. also Sonesson 1989a,III.2.1. and III.6.2.). It should not be confused with
identity: indeed, between two pictures there is identity, according to a principle of
pertinence, and on the basis of this property a picture, just as any other object, may be
used as a self-identification or an exemplification (as, for instance, in an art exhibition, or in
front of the artist’s workshop; see ibid.,III.2.3.). There is similarity, on the other hand,
only on the basis of a fundamental dissimilarity. It is certainly not in their ’important’
properties, if that means the attributes defining them as ’selves’, that the picture and its
referent (or content) are similar. In fact, the hierarchically dominant categories of the
picture and its referent must be different; for a picture which is just a picture of the
picture-of-X, is indistinguishable from a picture of X.
Although the sign relation is thus not needed in order to render similarity asymmetric
and irreflexive, it is required in order to distinguish similarities which are signs from those
which are not. At this stage, then, it would seem that the picture could be defined by the
sign relation, together with similarity; but Eco rightly observes that, on closer inspection.,
there is really no similarity between the painted nose, and the nose of a real person.
Yet we must account in some way for the impression of similarity, which is
immediately given, and which persists as long as we do no choose to scrutinize the details
of the composition. If similarity is a perceptual effect, then the impression of similarity
simply is similarity. But similarity then appears to be a result of the sign relation, instead
of its motivation (cf. ibid.,III.1.4.). This is possible only if there is some other property
held in common by all pictures, and by no other objects, which somehow precedes the
sign relation. Gibson (1971:33), who also rejects the similarity theory of pictures,
apparently thinks there is some kind of identity between the picture and the real-world
scene, rightly insisting that identity is not the maximum of similarity. To Gibson, this
identity relies on higher-order properties, recurring in different fashions in the picture and
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
42
in the world. Similar conceptions are present in the work of other psychologists, such as
Kennedy and Hochberg (cf. Sonesson 1989a,III.3.2.). Such a conception must suppose our
perceptual systems to be capable of picking up isolated features of the environment (not
necessarily identical to linguistic features; cf. Sonesson 1989a,III.4.1.). Contrary to Eco’s
contention, therefore, the pictorial sign (as well as the real-world scene) must be, not
conventional and unanalyzable into features, but essentially feature-based and
motivated.
Iconicity and the ecology of semiosisThe impression of similarity found in pictures is not in doubt, as long as we do no choose
to scrutinize the details. Similarity appears to be, not the motivation for the sign relation,
but a result of it, or perhaps rather of some other property equally preceding the sign-
relation. We have therefore yet to discover a property which is common to all pictures,
and which characterizes no other objects. For not only is there a coherent Lifeworld notion
of pictures, but there is no other way of explaining that pictures have meaning.
Goodman’s and Greenlee’s contention that the referent of each picture is appointed
individually (if that is indeed what they want to suggest), and Eco’s proposal that the
relations of the picture are so correlated with those of the referent, are utterly
unconvincing, and besides, incompatible with what psychology tells us about the child’s
capacity for interpreting pictures when first confronted with them at 19 months of age (as
demonstrated in a famous experiment by Hochberg). But it does not follow that this
common property must be similarity.
Goodman may be taken to suggest that this property is ’analogy’ or perhaps
’syntactic and semantic density’. Density here means roughly that, no matter how close a
division we have made of a picture into units, it is always possible to proceed, introducing
a third unit between each of the earlier pairs, and so on indefinitely. Unfortunately, there
are many problems with this proposal. To begin with, it is strange that the difference
between verbal language and pictures is supposed to reside in the ways in which types
relate to tokens, and not in the relations between expression, content, and referent.
Another problem is that ’analogy’, in the common and required sense, does not seem to
follow from double density, as Goodman supposes. In any case, there are reasons to
doubt that pictures are dense, in the strict sense, for while Goodman’s definition excludes
pictograms, any partition of actual pictures and pictograms is bound to be arbitrary. On
the other hand, the definition includes, among the objects which it qualifies as pictures,
diagrams and thermometers, and no doubt many other signs which are not ordinary
thought to be such. Repleteness, which is Goodman’s term for density resulting from
divisions made from many different points of view, cannot, contrary to Goodman’s
opinion, make the difference between pictures and diagrams, for it can actually be shown
to exist in some instances of the latter (for details, see Sonesson 1989a,III.2.3-5.).
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
43
Goodman is right in claiming that the sign function and similarity are not jointly
sufficient to define the pictorial sign; but substituting analogy for similarity, or adding
them together is not enough either. However, we can make sense of Goodman’s counter-
example if we require similarity, or the impression of similarity, to be at least a partial
reason for the sign function. This requires there to be a kind of taken-for-granted hierarchy
of prominence between the things in the Lifeworld. Some ’things’ are more apt to serve as
expressions of a sign relation than others, in fact, those which are relatively less
prominent. Interestingly, the only verified case in which a so-called primitive tribe failed
to recognize pictures as such, concerned a group which had never seen paper, and was
therefore led to emphasize the material per se. When pictures where instead printed on
cloth, they immediately recognized their function (se Sonesson 1989a,III.3.1.).
In the case of a droodle, similarity is only discovered once we have been informed
about the precise sign function, or when we have guessed at it; but in an ordinary picture
the impression of similarity precedes the sign function. On the other hand, the probability
of there being a sign function, would seem to be a prerequisite for our hitting upon the
similarity. For something to be a sign of something else, it must, as we just observed, be
relatively low-ranked on the scale of prototypicality applying to the ’things’ of the
Lifeworld. No doubt signs can also be made out of high-ranked Lifeworld ’things’, but
then the sign function must be introduced explicitly as a convention or be expected in the
situation. In fact, the painting at the art exhibition, the tin can in the shop window, and the
objects exposed in the museum are all signs of themselves, some of their properties, or the
class of which they are members; but the sign function only emerges in given situations.
In a recent collection of essays, Neisser (1987) considerably broadens the notion of
ecological psychology, incorporating Rosch's theory of prototypicality, as well as
Gibson's work on perception, in order to account for the conceptual negotiations going on
in the everyday environment. In his contribution to Neisser's anthology, George Lakoff
(1987) argues for the reconstruction of Roschian prototypicality using different kinds of
cognitive models. Elsewhere, in their study of the basic metaphors which underlie both
poetry and ordinary language, Lakoff & Turner (1989:160ff) describe a ’cultural model’
which they call ’The great chain of being’. This model, which ’places beings and their
properties on a vertical scale with 'higher' beings and properties above 'lower' beings and
properties’ (p.167), has been studied by historians of ideas since the time of Lovejoy, but
Lakoff & Turner shows it to be still current and active in a lot of everyday thinking, as for
instance in ordinary adages. This ’commonplace theory about the nature of things’ (p.170)
would only stand in need of being slightly amended to also account for the naturalness
with which surfaces stand for scenes, rather than the reverse.
Such regularities of the Lifeworld, together with the laws of environmental physics,
and other commonplace theories of the world, stand at the origin of an even broader
domain of study, which we could call the ecology of semiosis. This discipline should,
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
44
among other things, lay the groundwork for all future conceptions of cultural semiotics.
But it will also be needed to explain the varieties of iconicity. It will begin by teaching us
that some meanings are deeper than signs.
The semiotic function according to PiagetTwo facets should be distinguished, according to Saint-Martin (1985:6ff; 1990:9ff), within
the perceptual act: the objective aspect, linked to external stimulation, and the subjective
aspect, which derives from the perceptual instrumentation of man, that is, the
psychological and psychological structures at work in perception. In Saint-Martin's
opinion, this is what Saussure was thinking of when he compared the signifier and the
signified of the linguistic sign to the two faces of a piece of paper. Just as the linguistic
sign has its sensorial signifier and its conceptual signified, the ordinary percept, Saint-
Martin claims, possesses its dual face.
No doubt this is a serious misrepresentation of Saussure's intention: for he never
tires of repeating that both the signifier and the signified are mental entities, and thus
pertain to the conceptual, not the external, aspect, in Saint-Martin's sense. If he had been
interested in making a distinction of the kind which Saint-Martin suggests here, he could
perhaps have made it in terms of ’form’ and ’substance’ — which really amounts to
saying that he would have put Saint-Martin's external stimulation in his version of the
pragmatic waste-basket, la parole. More importantly, however, Saussure certainly
conceives of the sign function as being something very different from the act of
perception, although it may stand in need of a perceptual act to become manifest. It might
be argued, of course, following Saint-Martin (1990:10), that the signified must be inferred
from the signifier, just as the object must be discovered in the percept: but if the signified
is hidden behind the signifier, the signifier is in itself an object hidden behind its
appearances.
In discussing the perceptual act, Saint-Martin (1988; 1990:9ff) relies abundantly on
the work of Piaget. Yet, curiously, she ignores Piaget's important attempt to define the
semiotic function (which, in the early writings, was less adequately termed the symbolic
function). In his model of the sign, Saussure made a set of important conceptual
distinctions (which we have discussed elsewhere), yet he supposed us (and himself) to
understand the basic meaning of such terms as ’signifier’ and ’signified’. The discussion of
Piaget's semiotic function will help us to discover that which is taken for granted in this
model.
The semiotic function is a capacity acquired by the child at around 18 to 24 months
of age, which enables him to imitate something outside the direct presence of the model, to
use language, make drawings, play ’symbolically’, and have access to mental imagery and
memory..The common factor underlying all these phenomena, according to Piaget, is the
ability to represent reality by means of a signifier which is distinct from the signified.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
45
Indeed, Piaget argues that the child’s experience of meaning antedates the semiotic
function, but that is does not then suppose a differentiation of signifier and signified in the
sign (see Piaget 1945; 1967; 1970. Also cf. Bentele 1984. and Sonesson 1990f).
In the numerous passages in which he introduces this notion of semiotic function,
Piaget goes on to point out that ’indices’ and ’signals’ obviously are possible long before
the age of 18 months, but then they do not really suppose any differentiation between
expression and content. The signifier of the index is, Piaget says, ’an objective aspect of
the signified’; thus, for instance, the visible butt of an almost entirely hidden object is the
signifier of the object for the baby; and the tracks in the snow stand for the prey to the
hunter, just as any effect stands for its cause. But when the child uses a pebble to signify
candy, he is well aware of the difference between them, which implies, as Piaget tells us, ’a
differentiation, from the subject’s own point of view, between the signifier and the
signified’.
Piaget is, I believe, quite right in distinguishing the manifestation of the semiotic
function from other ways of ’connecting significations’, to employ his own terms.
Nevertheless, it is important to note that while the signifier of the index is said to be an
objective aspect of the signifier, we are told that in the sign and the symbol (i.e. in Piaget’s
terminology, the conventional and the motivated variant of the semiotic function,
respectively) expression and content are differentiated form the point of view of the subject.
We could actually imagine this same child that in Piaget’s example uses a pebble to stand
for a piece of candy have recourse instead to a feather in order to represent a bird, without
therefore confusing the feather and the bird: then the child would be using the feature,
which is objectively a part of the bird, while differentiating the former form the latter from
his point of view. Only then would he be using an index, in the sense in which this term is
employed (our should be employed) in semiotics (that is, in Peirce's sense). And
obviously the hunter, who has recourse to the tracks to identify the animal, and to find out
which direction is has followed, and who does this in order to catch the animal, does not,
in his construal of the sign, confuse the tracks with the animal itself, in which case he
would be satisfied with the former.
Both the child in our example and the hunter are using indices, or indexical signs. On
the other hand, the child and the adult will fail to differentiate the perceptual adumbration
in which he has access to the object from the object itself; indeed, they will identify them,
as least until they decide to change their perspective and approach the object from another
vantage point. And at least the adult will consider a branch jutting out behind a wall as
something which is non-differentiated from the tree, to use Piaget’s example, in the rather
different sense of being a proper part of it.9 In the Peircean sense an index is a sign, the
9 About proper parts, perceptual perspectives, and attributes as different ways of
dividing an object and thus different indexicalities, cf. Sonesson 1989a,I.2.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
46
relata of which are connected, independently of the sign function, by contiguity or by that
kind of relation which obtains between a part and the whole (henceforth termed
factorality). But of course contiguity and factorality are present everywhere in the
perceptual world without as yet forming signs: we will say, in that case, that they are
mere indexicalities. Perception is profused with indexicality.
The concept of appresentation in phenomenologyEach time we perceive two objects together in space, there is contiguity; and each time
something is seen to be a part of something else, or to be a whole made up of many parts,
there is factorality. Not all instances of these are signs, however. In the case of an actual
perceptual context, two items must be present together in consciousness, whereas in a
sign, one item is actually present while the other only appears indirectly through the first.
Yet the latter is also true of what may have termed an abductive context, which is the way
in which the side of the dice at which we are not looking at this moment is present to
consciousness, and the way into which we retain the preceding moment in time, or
anticipate the one to follow (retention, protention).10
The phenomenological tradition stemming form Edmund Husserl and later amply
developed by Alfred Schütz and Thomas Luckman has contributed some useful
distinctions here. According to these thinkers, two or more items may enter into different
kinds of ’pairings’, from the ’paired association’ of two co-present items, over the
’appresentative pairing’ with one item present and the other indirectly given through the
first, to the real sign relation, where again one item is directly present and the other only
indirectly so, but where the indirectly presented member of the pair is the theme, i.e. the
centre of attention for consciousness. This will be enough to distinguish the abductive
context from the sign, if we suppose the former to always carry the theme in the directly
presented part or to have it span the whole context. But this is by no means certain, and
there seems to be many intermediate cases between a perfect sign and an abductive context
(the poetic function, ostensive definitions, proto-indices, etc.; cf. Fig. 2.).
This is where Piaget’s idea of the semiotic function supposing a differentiation turns
out to be useful. Whereas the items forming the sign are conceived to be clearly
differentiated entities and indeed as pertaining to different ’realms’ of reality, the ’mental’
and the ’physical’ in terms of naive consciousness, the items of the context continuously
flow into each other, and are not felt to be different in nature. Before we go on to illustrate
this, two things should be noted: First, both content and expression of the sign are actually
’mental’ or, perhaps better, ’intersubjective’, as most linguists would insist; but we are
interested in the respect in which the sign user conceive them to be different. In the second
10 For a more thorough analysis, concerned with the problems addressed in this and the
following paragraphs, cf., in particular, Sonesson 1989a,I.2.5., I.4.2, and 1990b.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
47
place, Piaget’s notion of differentiation is vague, and in fact multiply ambiguous, but, on
the basis of his examples, I have introduced two interpretations for it: first, the sign user's
idea of the items pertaining to different basic categories of the common sense Lifeworld;
and, in the second place, the impossibility of one of them going over into the other,
following the flow of time or an extension in space.
d i r e c t l y - t h e m a t i c d i f f e r e n t i a t e d p r e s e n t c o n t i n u o u s s a m e n a t u r e
p a i r e d a s s o c i a - t i o n ( p e r c e p - t u a l c o n t e x t )
b o t h i t e m s b o t h i t e m s y e s y e s
a p p r e s e n t e d p a i r i n g
o n e i t e m d i r e c t l y p r e s e n t e d i t e m o r b o t h
y e s y e s
p r o t o t y p i c a l s i g n
o n e i t e m i n d i r e c t l y p r e s e n t e d i t e m
n o n o
p r o t o i n d e x o n e i t e m m o s t l y d i r e c t l y p r e s e n t e d i t e m
p r o v i s i o n a l - l y d i s c o n - t i n u o u s
y e s
p i c t u r e d p r o t o - i n d e x
o n e i t e m m o s t l y d i r e c t l y p r e s e n t e d i t e m ( o f i n d e x , n o t o f p i c t u r e )
i n t h e r e f e r e n t , y e s ; i n s i g n , n o
i n i n d e x r e l a t a , y e s
a e s t h e t i c f u n c t i o n / c o n n o t a t i o n
o n e i t e m m o s t l y d i r e c t l y p r e s e n t e d i t e m ( o f f i r s t s i g n )
n o n o
o s t e n s i v e d e f i n i t i o n
b o t h i t e m s , o n e a l s o i n d i r e c t l y
o n e o f t h e r e l a t a
y e s , a s c o n t e x t
a c c o r d i n g t o c i r c u m s t a n c e s
Fig. 2. The prototypical sign and other meanings
Suppose that, turning around a corner of the forest path, we suddenly catch a
glimpse of the wood-cutter lifting his axe other his shoulder and head. This experience
perfectly illustrates the flow of indexicalities which do not stop to become signs: it is
sufficient to observe the wood-cutter in one phase of his action to know what has gone
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
48
before and what is to come: that he has just raised his tool from some base level, and that
at the next moment, he is going to hit the trunk of the tree. If we take a snap-shot of one of
the phases of the wood-cutter's work, we could use it, like the well-known traffic sign
meaning ’roadworks ahead’, as a part for the whole or, more oddly perhaps, as a phase
signifying contiguous phases. There has been a radical change from the flow of
indexicalities occurring in reality, for not only is there now a separation of expression and
content ’from the point of view of the subject’, but this separation has been objectified in
the picture. Not even a series of pictures will reconstitute the perceptual continuum, but a
film may of course do so. However, when we ask the wood-cutter to stand still for a
moment (like in a ’tableau vivant’), his position as such, before it is transformed into the
motive of a picture, is already a sign for the whole of the action, although the directly
presented position does not seem to be non-thematized, continuity is only provisionally
interrupted, and expression and content are felt to be of the same nature. We are
somewhere in between the abductive context and the sign: this may be termed a proto-
index.
Figure 3. Quadrangular face
The picture is undoubtedly a sign, in the sense of it having a signifier which is doubly
differentiated from its signified, and which is non-thematic and directly given, while the
signified is thematic and only indirectly present. Yet none of these properties applies as
unequivocally to the picture as to, for instance, the verbal sign. As noted by philosophers
from Husserl to Wittgenstein and Wollheim, we seem to ’see’ the content of the pictorial
sign directly ’into’ its expression. This is true is a quite concrete sense. For instance,
although no real faces are quadrangular, we have no trouble identifying figure 3 as a face;
and, more to the point, we can even indicate the precise place of the expression plane
where the ears are lacking. This certainly has something to do with that peculiar property
of iconic signs, observed by Peirce, and called exhibitive import by Greenlee, which makes
it possible for icons to convey more information than goes into their construction (cf.
Sonesson 1989a,III.3.6. and III.5.1.)
In spite of his intention to distinguish signs from other ways of connecting meanings,
Piaget in fact confounds meanings of very different kinds, and is therefore unable to
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
49
discover the stages which have to be reached in order to attain the semiotic function. Yet it
is precisely because we have taken the Piagetian notion of semiotic function seriously that
we have been able to isolate the sign as a peculiar brand of meaning. And this will permit
us to evaluate the unique contribution to the study of meaning made by Gestalt
psychology.
The varieties of pictorial meaning — Configurations and other holisticpropertiesDeveloping some ideas of Piaget's and Husserl's, we have imposed a set of constraints on
meanings which are signs. We have thus also rejected the parallel between the strata of the
sign, expression and content, and the perceptual act, suggested by Saint-Martin. In the
following, we will offer some general remarks on meanings which are not signs. In the
process, we will discover, with Saint-Martin and Gestalt psychology, that all perception
is imbued with meaning. Yet is remain for us to investigate what kind of meaning this is, if
it does not answer to the requirements imposed on the semiotic function. Perhaps
perceptual meanings, which are not signs, but appear in pictures, can be constrained in
some other way, permitting us to characterize, if not all kinds of meaning manifested in
pictures, then at least those types which tend to loom large on the pictorial surface. It is
conceivable, for instance, that topological and/or Gestalt relations may dominate pictorial
meaning in this way. Indeed, Saint-Martin could perhaps by taken to make precisely this
claim.
Signs among other meaningsSome employments of the term ’sign’ and similar expressions are not congruent with the
semiotic function, but may yet correspond to some kind of meaning. In cognitive science,
terms like ’sign’, ’symbol’, and ’representation’ are used in a vastly more comprehensive
sense than the one favoured here. The contents of consciousness are said to be ’symbols’,
and so on, of things in the ’real’ world (see Johnson-Laird 1988). Interestingly, that is an
employment of the term found also in John Locke, one of the first explicit semioticians, at
the beginning of the 18th century. Even before that, however, Pedro Fonseca, in his
treatise on signs from 1564, distinguished two types of signs: formal signs, by means of
which we know the outside world, and instrumental signs, which lead to the cognition of
something else, like the track of an animal, smoke, a statue, and the like (cf. Deely 1982).
However, as recognized in philosophical phenomenology, and more recently in the
ecological psychology of James Gibson, we do not ordinarily perceive signs of the world,
but the world itself; and thus, if indeed meaning is involved, its relata cannot be
differentiated, and there can be no semiotic function.
Not all of Piaget's examples of the semiotic function may really be of that kind, even
applying his own criteria. Günter Bentele (1984), who quotes Piaget's definition according
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
50
to which the content of the sign must be differentiated from its expression, rightly
observes that imitation does not manifest the semiotic function in this sense, but is a
prerequisite for it: indeed, it will function as a sign only on rare occasions when the act of
imitation is taken to refer back to the imitated act, instead of just being another instance of
the same kind (Moreover, Trevarthen & Logotheti 1989 shows the child's ability to
imitate something outside the presence of the corresponding model to antedate the
acquisition of the semiotic function). The same remark should apply to ’symbolic’ play,
and is in fact made by Bentele in another context: the toy is a sign, only to the extent that
the child takes it to represent the real thing, which cannot be true, for instance, in the case
of a toy lion if the child has no experience of the real animal11. Nor is is clear that imagery
and memory (not discussed by Bentele) requires any real differentiation of expression and
content (see Sonesson 1990f and 1989a,III.3.5-6.). On the other hand, the semiotic
function is not only embodied in verbal language and drawing ability, together with some
instances of play and imitation, but also appears, for instance, in gesture, music, etc.
Trevarthen’s critique is in fact more general. He argues that the child is attuned to
meaning from the first, that is, not only from birth, but in fact already at the end of the
fetal stage: co-operation, and the capacity to pick-up other people's meanings, is somehow
built into the organism from the start. Yet, as far as I have been able to determine, the term
’meaning’ is here employed in a more general sense than the one characteristic of the
semiotic function, as we have tried to develop this notion taking our hints from Piaget and
Husserl: it includes perception, particularly of an interpersonal kind.12
Ever since the time of Gestalt psychology, there has been another meaning of
meaning around, quite distinct from the sign: the whole which is perceived to be something
more than the elements out of which it is observed to be constituted. Although no
psychologist would nowadays accept the Gestalt psychological explanation for the
emergence of the whole, the phenomena are still there to be accounted for, and are
described in contemporary cognitive psychology as being some kind of perceptual
prototypes. Meaning according to Gestalt psychology amounts to something being more
than its parts; but the sign, according to Roman Jakobson’s formula, which we tried to
explicate earlier, supposes there to be something standing for something quite distinct
11 Cf. the discussion of the asymmetric nature of similarity, relying on experimental
evidence from cognitive psychology, conducted in Sonesson 1989a,III.6.2.12 Gardner & Wolf (1984) also criticise the Piagetian semiotic function, yet their criticism
does not seem to attain the features which interest us here, for there seems to be no
contradiction between the existence an unitary semiotic function, in the sense of
differentiation between expression and content, and the existence of unequal rates of
development within different semiotic domains. For a complete discussion, see Sonesson
1991d. Also cf. Tamm 1990.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
51
from itself.
There is certainly a wider sense of meaning, which may be related, as Lévi-Strauss
once put it, to order, that is, organization, relatedness, indexicality. What is important here
is the connecting of things together, and the selection of elements to connect from a wider
field of possibilities. It is interesting to observe that it is not the sign function but the
paradigm, the feature, and the phoneme, as metaphors for selection, and the syntagm and
the index, as metaphors for connection, which have had an important role to play in the
adoption of the linguistic model in semiotics, notably in the work of Barthes, Greimas,
Lévi-Strauss, and the Peirceans. When Lévi-Strauss presents the myth as a sign function,
this interpretation is contradicted by his own detailed description, which really manifests
a second-order texture. And when Greimas claims that even the phoneme carries meaning,
this can only be understood in the sense of its forming a whole, a category having its own
limits.13
As we have already observed, the picture is certainly a sign, in the sense of it having
a signifier which is doubly differentiated from its signified, and which is non-thematic and
directly given, while the signified is thematic and only indirectly present. On the other
hand, the picture is made up of, and presupposes, a number of meanings which are more
elementary than signs. We shall see the importance of this observation later. For the time
being, let us simply note that these meanings are of two kinds, those which, in Piagetian
terms, pertains to operativity (logic, classification, etc.), and those which have to do with
figurativity (which, in Piaget, is a residue concept). According to serious brain research
(which should not be confused with the dubious lore pertaining to brain hemispheres
nowadays found in the weeklies), operativity and figurativity may be very roughly
distributed into the left and the right half, respectively, of the brain. Adding together the
information contained in the writings of Gardner (1977; 1982; 1984) and Coffman (1980),
we can construct a table in which the importance of both operativity and figurativity to
drawing ability is made clear (Fig. 4): contours and global properties are on the side of
figurativity, while details, inner elements, and richness of details are on the side of
operativity. Put more generally, operativity seems to account for structure, in which the
whole works on the parts to make them stand out more prominently, whereas figurativity
explains the configuration, in which many elements are fused into a new whole (cf.
Sonesson 1989a,I.3.4. and I.4.3.).
13 For another interesting argument concerned with the difference between direct
perception of the world, and the indirect apprehension typical of pictures and other signs,
see Pérez Carreño 1988:63ff.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
52
Figure 4. Figurativity and operativity (in Piaget’s sense) and the brainhemispheres (from Sonesson 1989a)
All wholes are imbued with meaning. But not all meanings are wholes, nor are all
wholes configurations. These are some insights stemming from the holistic psychologies
being active in Germany during the first half of this century, but not perpetuated by that
most famous of its branches, Gestalt psychology, and therefore lost for Saint-Martin.
Gestalt theory and the other holistic psychologiesAt least three different versions of holistic psychology developed in the German speaking
world around the turn of the century: the Graz school, the Berlin school, and the Leipizig
school. Christian von Ehrenfels is reputed to have been the first to discuss, in his 1890
article (reprinted in Weinhandl 1960:11ff), the nature of ’Gestaltqualitäten’, in particular
as manifested in music. A melody may by transposed, he observes, without any of its
elements remaining constant. To von Ehrenfels and his immediate followers in the Graz
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
53
school, such as Meinong, Bernussi, and Witasek, these configurational qualities are
supposed to be added to elementary sensations, which serve as their foundation (see
Weinhandl 1960; Gurwitsch 1957:54ff). Husserl's discussion of ’figural moments’ (treated
by Saint-Martin 1990:21ff) would also seem to tend to this conclusion. About a century
earlier, however, some of the ideologues, close precursors of semiotics, observed that there
was one kind of whole which must be apprehended prior to the perception of its parts (cf.
Sonesson 1989a,I.3.4.). Moreover, on a closer reading, von Ehrenfels actually seems to
maintain that there are two kinds of wholes, those, like the melody and the square, which
are directly perceived, and those which depend on our initiative for their existence, as the
similarities which might be discovered between two notes.
According to Koffka, Köhler, Wertheimer, and other members of the Berlin school,
and later to Arnheim, Gurwitsch, and Merleau-Ponty, the configuration is immediately
given, whereas the putative elementary sensations must be abstracted out of the whole.
The criteria by which a phenomena is recognized as being a configuration are, according to
Köhler, demarcation, closure, and over-summativity (cf. Weinhandl 1960:334ff, 384ff). A
whole is said to be a sum if, when one part is detached from the whole, no modification
occurs in the part, nor in the remaining whole; otherwise, it is over-summative, that is, aGestalt. Writing in 1947, Mukar &ovsky ! (1974:7ff, 20ff) protested that all wholes were not
like this: while structural wholes result from the mutual relations between its components,
including negative ones, a holistic whole, that is, the Gestalt, is primarily a demarcation
made in the field, from which an inner differentiation may later ensue. Piaget (1972a: 137f;
1972b:47) distinguishes two kinds of wholes, schemes and Gestalts, along similar lines.
As early as 1906, Krueger (quoted in Weinhandl 1960:385) criticizes the all too
general use of the term ’Gestalt’ to designate all kinds of wholes, proposing a distinction
between wholes distinctly moulded to a particular shape, and wholes in a more general
sense. According to Krueger, emotions and the experience of small children generally are
non-configurational wholes. Such properties, he maintains, cannot be transposed, unlike
von Ehrenfels' melodies. Other criteria are proposed by Volkelt (in Sander & Volkelt
1962.43ff, passim), another member of the Leipzig school, according to whom a typical
configuration stands out from a background and is internally articulated, whereas other
holistic properties may be externally and internally diffuse. In his studies of children's
drawings, Volkelt encountered holistic properties, such as the closure and angularity of the
cube, transposed, though undoubtedly modified, from the child's perception (see the cube
drawing in the left column of Fig.4.). Volkelt and Sander later recognized many degrees of
demarcation and articulation of wholes, bridging the distance between the typical
configuration and the merely holistic properties (see further discussion in Sonesson 1989a,
I.3.4.).
Saint-Martin's (1990) book is exclusively concerned with the Berlin school, more
commonly known as Gestalt psychology, no doubt because, in comparison to the other
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
54
schools, it showed a greater predilection for the Gestalts. Refuting popular belief, Saint-
Martin (1990:65ff) sets out to demonstrate that the opposition between figure and ground
was less central to the Berlin school than is usually claimed, and was understood in a much
more dynamical way. In a similar vein, she argues that Gestalt psychology did not reduce
all phenomena to simple configurations, as the circle and the square, but diagnoses a
tendency in perception to make any shape as similar as possible to a regular Gestalt, or at
least to conceive it as a deformation of such a configuration (p.,8, p.17, p.73ff, p.78). As
against a common misunderstanding, she insists that Gestalt theory, instead of assimilating
all perception to a small set of static shapes, looked upon it as a dynamic process in which
contrary forces could, in some cases, give rise to perfect configurations (p.40ff, p.71). In
this view, perception is a field of tensions, of forces and counter-forces, striving to
establish an equilibrium of forms, imposing the configurational properties on
comparatively ’bad’ forms, and accepting, in the last analysis, the ’badness’ of certain
shapes. Although a picture containing no trace of configurations rapidly appears
uninteresting, this is also true of a composition made up a completely regular Gestalts
(p.107). Some amount of ’badness’ is needed in order to retain our interest in the picture,
requiring ever greater deformations, as we grow accustomed to the extravagances
perpetuated by the different waves of modernism.
Some of the other affirmations which Saint-Martin makes in defense of Gestalt
psychology are more surprising. For instance, she believes it is still possible to entertain
the theory according to which there is an isomorphism between the tensions intrinsic to
the perceptual field and some kind of electromagnetic field in the brain (p.34ff). This
conception may not have been refuted; indeed it may not be susceptible of being refuted; it
simply is not the kind of theory any psychologist would take seriously today. This is not
to deny that something very similar to a interplay of tensions takes place in perception,
only that it will translate directly into magnetic brain fields (some of those effects could
even be reinterpreted in terms of Gibsonian ’affordances’). As far is I understand, Saint-
Martin also wants to claim (p.86) that the Gestalt theorists did not necessarily believe the
Gestalts, or the forces responsible for their emergence, to be innate. Such a controversial
claim, to say the least, stands in need of a proof which is never produced.
Instead, Saint-Martin (p.85) quotes Chomsky as defending such an innateness
hypothesis. She seems much more favourable, however, to Ehrensweig's idea, according to
which ’good forms’ should result from some kind of censure accomplished by forces
emerging from the unconscious. Whatever its source, it might be argued, however, that
there is a general tendency to organize different shapes, as well as other phenomena,
around a central case which is the best realization of the category (cf. Rosch 1973). Such a
tendency to prototypicality could be innate, or may result from some very early
experience in the common human Lifeworld. The particular ’good forms’ would then be
superficial phenomena derived from this general tendency. Indeed, prototypicality itself
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
55
could be a relatively superficial effect, which is based on different kinds of modelling (cf.
Lakoff 1987; Rosch 1987).
Saint-Martin clearly considers iconicity to be an phenomenon which is comparable,
in this respect, to that of the good form (p.83ff,107f). Like Eco and many other
semioticians, she takes iconicity to imply the recognizability of real-world objects on the
pictorial surface (which is a much to limited meaning, as we have seen). Curiously, this is
quite the opposite of Gibson's and Kennedy's conception, according to which Gestalt
effects simply result from the recognition of real-world objects on the pictorial surface (cf.
Gibson 1982: 57f; and Sonesson 1989a,III3.). Yet, Saint-Martin (p.83ff) may possibly
furnish refutations of both hypotheses, as she quotes the observations of Boas and Leroi-
Gourhan, to the effect that there is an opposition between real-world objects, and the
simple, regular, symmetric and rhythmically repeated, shapes appearing in the pictures
created by ’primitive’ tribes. Even if Gestalt effects exclusively turn up on the pictorial
surface, and even if they are derivative effects of pictorial representation, as Gibson
claims, a prototypicality principle is needed in order to explain this tendency to discover
’good forms’.
Interestingly, Saint-Martin considers ’good forms’ to be a comparatively late
development in perception, being preceded by aggregates organized according to simple
topological relations, which may be observed in children's drawings (p.89). Had she been
aware of the non-configurational, holistic, properties discovered by the Leipizig school, it
seems to us that she could have gone much further.
The features of perception and languageWhen pictorial semiotics was first initiated, some kind of minimal unit of pictorial meaning
was commonly believed to exist, for instance by Eco, Koch, Floch, Gauthier, Thürlemann,
Lindekens, Groupe µ, Gubern, Tardy, Vilchez, Paromio, etc., though most of these
authors only assumed the existence of such features only on the pictorial level. According
to Eco, conic codes (which should really be called pictorial codes) possess their figurae,
signs, and statements, just like verbal language. Although Eco was later to reject, ever more
emphatically, the existence of pictorial features, he retains, at least as late as in 1976, his
idea to the effect that films are organized into three articulations, which supposes the
double articulation model for pictures, according to which pictures are built up of
distinctive traits having no independent meaning and forming together autonomous signs
(see critique in Sonesson 1989a,III.4.2. and 1989c).
Eco’s contention that the cinema possesses three articulations, which is based on the
double articulation of static pictures, clearly derives from an at least threefold confusion
about the import of the notion of articulation in linguistics. Most importantly, Eco con-
founds levels of configuration, where a whole contains meaning going beyond that of its
parts, and levels of appresentation, where there is a passage to a quite different realm of
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
56
reality (that is, there is more than one layer of sign functions).. In fact, although linguistics
have never offered us any clear definition of articulation, it is apparent from linguistic
practice that it is supposed to involve a concurrent shift in the levels of appresentation
and configuration. Thus, there cannot be any triple articulation in the cinema (cf. Sonesson
1989a,III.4.2. and 1989c).
Nor can Eco’s three stratum model of pictorial meaning be sustained, for, as soon as
we attend to the definitions given, the figurae level merges with the sign level, and the
sign level with the semata level. Although pictorial features really lack meaning in
isolation, each one of them acquires a specific signification as a part of a signifying
configuration, and thus there is nothing comparable to the second linguistic articulation. If
there is something like the first linguistic articulation in pictures, then it appears on
different configurational and extensional levels of the picture, in different kinds of picture.
A nose is a nose is a nose — but it may appear as a feature or a sign, depending on the
pictorial style selected (cf. Sonesson 1989a,III.4.3.).
In spite of the arguments of Barthes, Metz, and the second Eco, there is every
reason to accept, along with such psychologists of perception as Gibson, Kennedy, and
Hochberg, the existence of pictorial features. But these features differ in important
respects from those of linguistics. Like all features that pertain to signs, pictorial features
must be allo-functionally defined, that is, they derive their identity from the relation
which they contract with the other plane of the sign; and although there is probably not
just a small number of them, they could scarcely be infinite. On the other hand, pictorial
features are not deprived of meaning, at least not in the way phonemes are. Indeed,
according to Gibson and Kennedy, they are not simple, physiological correlates of vision,
but ’higher-order properties’. These may, or may not, be identical to what we above,
following the Leipzig school, termed holistic, non-configurational properties. In fact, both
terms really stand in needing of being better defined.
While pictorial features do seem to be categorical in themselves, their relation to the
other plane of the sign is merely probabilistic. Indeed, the pictorial sign contains many
redundant expressions for one content, but also a cumulation of contents conveyed by a
single expression. In this way, they are similar to the features present in the perceptual
Lifeworld, but, for the same reason, they allow for rhetorical modifications of our
Lifeworld experience (cf. Sonesson 1989a,III,4.1.).
Just like the later Eco, Goodman believes pictures to be similar to verbal language in
being conventional, while differing from it in allowing for no division into units. His idea of
inanalyzability, or ’density’, supposes, as we noted above, that no matter how close a
division has been made of a picture, it always remains possible to continue the division,
introducing a third unit between each of the earlier pairs, and so on indefinitely. This
theory is untenable, at least in the case of pictorial (Goodman's referential) meaning, for it
is unable to account for the fact that real-world objects can be recognized in pictures. If the
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
57
referents of pictures were simply appointed in each particular case, as Goodman suggests,
the theory might possibly be right, but we have seen, when discussing iconicity, that such
as conception is entirely unfeasible (cf. Sonesson 1989a,III.2.4-5. and III.6.1.).
According to Saint-Martin (1987a), the students of visual language have failed to
discover so far an elementary unit equivalent to the phoneme, because they have expected
to find too great a similarity between the units of the diverging systems. In the case of
visual language, this unit is the ’coloreme’, which may perhaps roughly be described as a
minimally perceptible surface point, or as that amount of information which is picked up
by the viewer in one glance as it is organized around a single point of fixation. Having
introduced this intriguing notion, our author proceeds to explore the nature of six visual
variables, which turn out to be dimensions on each one of which every surface point must
evince a value: these are termed colour/tonality, texture, dimension/quantity, implantation
into the plane, orientation/vectorality, and frontiers/contours generating shapes. Clearly, to
the extent that the coloreme is made up of visual variables, it will encounter at least some
of the same problems facing earlier feature analysis. Besides, according to its definition,
the coloreme is a segment, not of the object perceived, but of the very process of
perception, and this, as we shall see below, in not quite the same thing.
Perceptual features, as recognized by psychologists of perception, are in no sense
meaningless, contrary to the linguistic ones. For instance, Kennedy would show, using a
familiar landscape scene, that certain constellations of lines meeting at particular angles,
stand for bounds, edges, surfaces, corners and cracks of the three-dimensional perceptual
world. Although it is the constellation which triggers the meaning, its parts are then
distributed onto the constituent parts of the constellation. Features like those discussed
by Kennedy have been implemented in computer programs, and works out quite nicely.
It does not follow, however, that this is really the essence of human perception.
Gregory, who emphasizes the constructive character of perception, points out that the
presence of a simple tangent line may transform what looks like a tree into the semblance
of a woman’s head profile having a cigarette in her mouth. And Hochberg observes that
there are, in addition to the spatial layout features, so-called canonical features, which
account for the minimal opposition between Hitler and Chaplin in some caricatures. I
would suggest, however, that canonical features are much less marginal than Kennedy and
Hochberg imply. Indeed, even in Kennedy’s favourite landscape picture, the sea and the
clouds are not seen because of any layout features, but are clearly conveyed simply by
means of prototypical forms. In fact, even the drawing of the house, with the same spatial
configuration, could represent any number of other cubic objects, if it were not for its
prototypical house features.
It may be suggested, then, that canonical features are really pervasive in pictorial
perception. Just as the meaning of a sentence may be grasped directly, independently of
the details of syntax, there may be a direct perception of gist in the picture, in some cases
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
58
eventually supplemented by the registration of spatial layout. Actually, there is (in spite
of Hochberg’s attempt) no way of accounting for the phenomenon of impossible pictures,
like the devil’s turning fork, or many of Escher’s and Reutersvärd’s compositions, without
recognizing an opposition between what may, rather metaphorically, be termed the
’syntax’ and the ’semantics’ of pictures (see Sonesson 1989a,III.3.4.).
In spite of the existence of pictorial features, Goodman’s observations on density are
not entirely off the mark. Indeed, once we have decided between the category of a tree or a
woman’s profile, the drawing will tell us a lot about the particular conformation of the
crown, or the nose, the hair-cut, and so on. But not indefinitely: only up to a point set by
the principle of pertinence embodied in the pictorial medium.
Thus far, our discussion has been mainly concerned with pictorial features (the
’iconic’ features of most semioticians). The problem posed by plastic features, which are
what interests Saint-Martin, could well turn out to be quite different. Before inquiring
further into this issue, however, it will be necessary to scrutinize the notion of the
coloreme.
The coloreme as sensation and noemaThe coloreme is characterized by Saint-Martin (1987a, 1988) as an aggregate of visual
variables perceived in the visual world owing to a centration or fixation of the eye, which
gives rise to a macular, and then a peripheral, field surrounding the foveal area of clear
perception. In its primary sense, the coloreme is thus a segment, not of the object
perceived, but of the very process of perception, and in that respect, as well as in its being
structured as an array of adumbrational variants organized around a thematic centre, it is
clearly comparable to what is known in Husserlean phenomenology as the perceptual
noema, and to what the psychologist of perception Julian Hochberg terms, more
colloquially, a ’glance’.
There can be no one-to-one mapping of the units of the perceptual process onto the
units of the object, however. Joining together the kindred traditions of Gestalt psychology
and phenomenology, Aron Gurwitsch (1957), in a classical work (quoted in other contexts
by Saint-Martin 1990) made a particularly thorough analysis of perception showing that
the object is entirely contained, differently adumbrated, in each one of the corresponding
noemata; his result still seems compatible with what we know from perceptual
psychology, in particular, as we noted above, with the ecological conception of Gibson
and his followers. Indeed, that Saint-Martin should ignore this fundamental distinction
between the object and the different adumbrations in which it appears to us is less
surprising after reading her Gestalt theory book, in which she explicitly rejects the parallel
distinction made by Gibson, and considers ’metaphysical’ the idea that the object is
present through its appearances.
There is nothing ’metaphysical’ about such a description. What it amounts to is,
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
59
primarily, that we have the impression of seeing the whole object in each one of its
appearances. This conception is not incompatible with Gurwitsch analysis according to
which the object is nothing but an ’internoematic system’, but such a system must be
made up of an infinite number of different adumbrations, which overlap one another more
or less, permitting the constitution of Gestalt-coherence (on which Saint-Martin insists in
her polemics with Hochberg referred to above). Any selection of elements from the
internoematic system, corresponding to the coloremes of Saint-Martin, must therefore be
arbitrary, and is thus unfit to serve as the basis for an analysis of the object, that is, in this
case, the picture.
In the case of picture perception, however, investigations, ever since the time of
Boswell, have tended to demonstrate the existence, if not of a order of reading, then at
least of certain points of fixation where the glances tend to cluster. In this sense, the
noemata would seem to offer a kind of indirect access to the thematic centres of the
picture, to the extent that those points attracting recurrent glances could be taken to define
a small set of privileged noemata extracted from the internoematic system. But since
noemata have no clear limits, but tend to shade into each other, and encompass one
another, they could not, even in this case, delimit the real building blocks of the picture.
Furthermore, the topological relations of the noemata are not at all those of the parts of
the object contained in them. In any case, Saint-Martin denies every relevance to these
results, without even attending to the difference between the hierarchies defined by the
order of glances, and by their recurrence.
Actually, the coloreme, on closer inspection, appears to be a much less sophisticated
notion than the noema. Its division into the foveal, macular, and peripheral, areas already
suggests that it is to be interpreted as a portion of the retinal image. Like all contemporary
psychologists, indeed, like Descartes, Saint-Martin observes that the retinal image is
actually never seen as such, but like Descartes, and like most present-day psychologists,
she keeps forgetting all the time the irrelevance of the static retinal pattern, clearly
recognized by Gibson (cf. Lombardo 1987). Indeed, at least in the Gestalt theory book,
the coloreme appears to be simply a sensation, in the sense of classical psychology.
Moreover, in Saint-Martin’s (1987a) earlier book, the coloreme also figures in the
guise of a unit defined ad hoc by the analytical scheme resulting from her analytical
model. In order to analyze a picture, an arbitrarily divided grid is imposed on it. Such a
grid, according to Saint-Martin, may well possess 15, 20, 25, or any other number of
squares, although she arbitrarily decides to employ the latter number, resulting from five
horizontal, and five vertical, divisions. Moreover, each square, designated by a letter of the
alphabet, may be further divided by the numbers 1-5, corresponding to each of the sides of
the square, in addition to its central point. And here comes the surprise: these 125 points
are identical to the coloremes! Thus, the points of fixation defining the coloremes are
simply imposed ad hoc by the model!
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
60
Curiously, Saint-Martin then proceeds to suggest that each of the squares are
comparable to the basic plane as a whole, in containing a differential distribution of
energies corresponding to the cruciform structure. But this is an absurd suggestion, if we
remember that the number of squares of the grid, and thus the localization of the limits
between the squares, was arbitrarily determined. Saint-Martin herself here seems to forget
the conventional character of the framework imposed by her model.
One cannot but feel sympathetic to Saint-Martin's suggestion (largely derived from
Piaget, cf. 1988; 1990:14ff) that perception is an activity, and that the pictorial work of
art, in order to be adequately perceived, requires a long series of perceptual acts. It is a
pity, however, that she should identify these acts with retinal fixations, for the latter are
modified all the time, and never become conscious as such (cf. Gibson 1982:96, 180ff). Is
is also curious that she should oppose her own view of perception as an active search to
the static character of phenomenology, when Husserl actually insisted on the explorative
nature of perception ("Ich kann immer weiter"), and that she should associate this
conception with Gestalt psychology, when it is much better represented by Gibson's
ecological psychology ("the permanent possibilities of perception"). Indeed, according to
the common bias of constructivism and Gestalt theory, reality is not there for us to see, no
matter how much we try, which means that no series of perceptual acts, however
extensive, will ever be able to grasp the object itself. Only if we admit that there are ways
of having access to reality, is it possible to claim that we can close in on it, while also
allowing for the fact that no actual search is fully adequate, because it will never exhaust
its object. Yet, all perceptual acts contribute, as much as possible, to the construction of
the picture as an object, to the picture thing, which is given in its entirety, but
incompletely known, in each one of its adumbrations.
Plastic language reconstructed from the feature hierarchyIn our further discussion of pictorial perception, it will be possible to dispense with the
notion of coloreme, shown to be contradictory and confused, and thus to concentrate on
the visual variables, or features, of which it is made up. In the following, we will focus on
the features of the plastic level, that is, those properties really possessed by the picture
considered as a thing in itself, made up of mere twodimensional shapes on a surface (see
Groupe µ 1978; 1979; 1985; Floch 1985: 15; 1986a, passim; 1986b: 126ff, and passim;
and critique in Sonesson 1991b). In Gibson's terms, then, we will not consider the objects
referred to by the markings on the surface taken to be a picture, but those meanings which
are, in some other, further to be elucidated way, conveyed by the markings considered in
themselves.
It is not easy to see how we are to determine the plastic meanings of a picture.
Twodimensional shapes on a surface simply are too ambiguous in themselves. Groupe µ
so far has proposed nothing to answer this query; Jean-Marie Floch, however, even tries
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
61
to discover the meaning of ’abstract art’, that is, pictures having only plastic language. In
order to make such pictures accessible to semiotic analysis, Floch attempted the
interpretation of one of Kandinsky’s non-figurative paintings, ’Composition IV’, by
means of a comparison with other, more nearly figurative works by the same painter. In
spite of Floch’s perspicacity, this procedure is unsatisfactory, because it is only possible
if we assume there to be a redundance of plastic language in relation to the plastic one
(Floch's ’iconic’ language): that is to say, if it is always true that plastic language only
repeats (though perhaps on a more abstract, and thus partial, level) that which is already
contained in pictorial language. But this is certainly not a presupposition that can be made
a priori (cf. Sonesson 1987; 1989a,II.3.2.). Apart from rendering the distinction between
the two layers trivial, this assumption is nowhere justified.
If, instead, we suppose there to be an autonomous plastic language, then we are con-
fronted with two problems. First, we need to discover what kinds of meanings could be
contained in the configurations, shapes and colours themselves. But in the second place,
since any visual configuration has a potentially infinite number of properties, we need to
know which of these properties are most likely to be relevant to our experience. Sonesson
(1989a,I.4.3-7. and II.3.6.; 1990g, 1991a) speculates on the possibility of there being a
small number of topological, bodily anchored properties, which predominates in such
plastic interpretations. Saint-Martin (1980; 1987a) broaches the problem of topology in a
more comprehensive fashion. Relying on Piaget's studies of children's drawings, she lists
such topological properties as proximity, separation, inclusion or interiority/exteriority
(that is, encasing and envelopment), succession, and continuity. In fact, different variants
of inclusion also intervene, without any reference to topology, in numerous text analyses
by Floch (cf. Sonesson 1992a). But there is nothing to justify the particular choice of
properties in these works either.
One way of approaching the intrinsic meanings of visual elements could be to
establish a feature hierarchy, similar to the one found by Jakobson (1942), according to
which there is a parallelism between the stages of phonetic development in child language,
the stages of phonetic reduction in the aphasic, and the relative complexity of the world’s
languages, as far as the phoneme repertory is concerned. Indeed, one exponent of the
Leipzig school, Lotte Hoffmann (1943) asked children between 2,2 and 9,7 years of age to
imitate a set of simple geometrical configurations using ready-made material, like sticks,
plates, and rings. All the configurations lacked all directly iconic content. By combining
the ready-made implements, it was possible to reproduce all the geometrical configurations
faithfully. Instead, however, children between 3-4 years would use any object whatsoever
to stand for all of the different configurations; but more often than not, a perfectly round,
compact object would be preferred. Older children would pick up only one, global,
property present in the configuration and imitate it with a single implement, for instance,
such properties as being closed, angular, pointed, having holes, and so on. Later several
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
62
pieces would be used, corresponding more often to the number of parts of the imitated
object than its shape (more details in Sonesson 1989a,II.3.6.).
Fig.5. Different limits between prototype categories in children’s
drawings. (adapted from Hoffman’s tex t in Sonesson 1989a)
It becomes clear from these example that a prototypical shape is used by the child to
represent a whole class of geometrical configurations. A round, compact object seems to
function like a prototype, a ’best form’, to which more deviant cases are assimilated. This
is seen most clearly is cases in which different children impose differently placed limits
between the classes subsumed by the prototypes (cf. Fig. 5.). Given these facts, we are
able to set up the rudiments of a tentative hierarchy: it will start out from the circle, the
only shape the centre and periphery of which are perceptually salient; and it will continue
with an elementary division into circles and straight lines, the latter being the first element
in which extension becomes salient (see Fig.6.).
The properties common to the child's drawing and the object imitated, in these
examples, are often non-configurational holistic properties; often enough, these properties
are represented in the drawing by the corresponding perfect configurations or "good
forms". Many of the properties transposed from the object to the drawing are clearly
topological. The task set before the child is admittedly to produce as close a resemblance
as possible of the object. Yet, some other instance is clearly looms large in the child's mind
making him predominantly pick up, among all the numerous properties of the object,
those which are holistic and/or topological in character. There is thus a second principle of
pertinence holding sway, and commanding that of pictorial representation. Topological
and holistic properties may indeed be basic meanings of plastic language.
The logic of qualities and the bodily modesComparing the two ’great structuralists’ Piaget and Lévi-Strauss, the cognitive
psychologist Howard Gardner (1973) argues that, whereas the first is a pure formalist, the
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
63
second is really more concerned with the peculiar qualities connected by the relations. In
the conception of Lévi-Strauss, it is as important that the mind tends to focus on specific
qualities such as the raw and the cooked as it is that it will conceive of them in binary
opposition to each other. The negation as such is not important, but what is negating
what. Piaget, on the other hand, merely reduces all reasonings of his experimental subjects
to a group of four operations and 16 binary propositions or some approximation to these,
and it is therefore of no avail to know whether this mathematical form was manifested in a
particular case as a reasoning about chemical substances, billiard.balls, or a pendulum.
prototype:something (present)
prototype:
? discontinuous continuous
straight round
compact contoured
Fig 6.. A hierarchy of shapes, derived form children’s drawings (adapted
from Hoffman’s text in Sonesson 1989a)
This interpretation does not seem to be entirely justified. In fact, according to Lévi-
Strauss’ own interpretation of his ’logic of qualities’, it is not the qualities, but the logic
which matters. Although there is a small set of qualities which tend to occur over and over
again in his analyses, Lévi-Strauss is most explicit about their arbitrary nature. Like the
phonemes, they have only positional or differential value, he claims. Just as the phonemes
in the word ’sun’ are meaningless separately and may be used in other words with quite
different meanings, so, according to Lévi-Strauss, the content ’sun’ in its turn is
meaningless relative to mythology, outside a particular ’mytheme’. It would be easy to
adduce numerous instance in which Lévi-Strauss rejects the idea of qualities having a
meaning of their own (see examples and discussion in Sonesson 1989a,I.4.); he even
censors Jung and Levy-Bruhl for embracing that idea.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
64
Yet Gardner’s interpretation of Lévi-Strauss' work is not without all foundation.
There are passages which go in Gardner’s sense. But, more to the point, Lévi-Strauss' own
analyses certainly suppose that qualities have meaning, even if the myths then determine
these meanings in peculiar ways. Lévi-Strauss (1983) tells us that things as different as the
sheep’s horn, the eagle’s claw, and certain parts of different kinds of mollusc, may occupy
the same place in a myth, because they have in common the property of being ’des
organes qu’on retranche de l’animal avant de le comsommer, ou dont on retranche une
partie avant de les consommer’ (p.185). There is separation, not just of a part from a
whole, but of a part which is in some sense supplementary to the core, an appendage, a
protuberant part. What the myth treats as being the same is either the (edible) core
separated from the (inedible) appendage, or the (edible) appendage cut lose from the
(inedible) core. It least in this case, Lévi-Strauss has discovered a common denominator
which can only be justified as a holistic, non-configurational properties, and which
appears to have some topological facets, pertaining to inclusion, as well as mere proximity
and separation.
Although there are certain recurring terms in Lévi-Strauss' work, he does not propose
any list of fundamental qualities, nor does he try to show why certain qualities should be
more important than others. In an experimental study, however, Gardner (1970) found
that children were particularly sensitive to what he calls modal/vectorial qualities, such as
the opening or closing of the hand or mouth, direction, balance, rhythm., penetration, and
so on. These properties may manifest themselves in different sensory modalities, and even
in the motoric realm. There are reasons to suppose that they continue to be registered by
the adult, and that he attends to them, in particular, when experiencing works of art. These
modal/vectorial properties, Gardner suggests, elaborating on an idea of Erik Eriksson,
receive their particular import from first being experienced in the relationship between
one’s own body and the field of objects outside the body, sometimes in relation to the
keeping of portions of the environment inside the body, and sometimes in relation to the
release of what was once part of the body.
Eriksson and Gradner refer these properties to different erotic zones, in the wide
Freudian sense of the term (As far as I understand, also Saint-Martin is pondering some
Freudian interpretation of pictorial meaning, which she will however present to us in some
forthcoming book). This is, to my mind, the problematical aspect of the scheme. It
certainly seems probable that the little girl will learn inception from enclosing a cherry in
her hand, and that the little boy will stick his hand into a hole in the tree, well before any
of them get the opportunity of having sexual intercourse. Should they have to await the
sexual experience, the boy would only acquire the notion of intrusion, and the girl could
only know of inception. However, a more general interpretation seems possible.
According to Piaget, schemes of thinking are abstracted from actions through the
several stages of intellectual development. What is, from the beginning, taking place as an
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
65
exchange between one’s own body and the objects outside, is projected onto the object
pole, and transformed into an object in its own right. In a parallel, but inverse fashion,
modal properties seem to evolve from the body’s confrontation with the world, but to be
projected back to the subject pole. Thus, they are experienced as bodily, and perhaps
more generally, spatial, relations. Indeed, in a study of the bodily continuum underlying
gestures,we discovered long ago, and quite independently of Gardner’s scheme, that
gestural meaning is largely a question of creating and dissolving bodily spaces, and of
entering into and deserting bodily constructed enclosures. (Cf. Sonesson 1981a,b,c; 1990g;
1991a).14
Figure 7. Topological-dynamical interpretation of Gardner’s modal-vectorial
properties (from Sonesson 1989a)
The dynamics of topology and semiotic integrationThese modal properties are readily reduced into a set of notions taken over from
mathematical topology (Cf. fig. 7. Two more properties, which may be similarly reduced,
were added in Sonesson 1989a,I.4.3-5.: the refusal to let something else in, and the
resistance to be enveloped). This reduction should be interesting for a number of reasons.
14 This analysis, which involved the analysis of a number of documentary films produced
by French television, was accomplished by a working group under the direction of the
present author, and was realized in collaboration between the Groupe de recherches
sémio-linguistiques, and the Institut National de l'Audiovisuel, in Paris, from 1979 to 1981.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
66
Piaget and his collaborators have demonstrated that the geometry of the child’s first
experiential space is topological, i.e. it contains the kind of relations that would be
preserved in a figure drawn on a piece of rubber when the rubber is distended or
compressed. Topology has also been invoked numerous times in semiotics: in the
morphology of René Thom, in Pierre Boudon’s semiotics of architecture, and, of course, in
Fernande Saint-Martin’s visual semiotics. But beyond the fact that topological space,
according to Piaget, is the first space of child experience, the importance of topology has
never been justified, nor its categories clearly determined.
Strictly speaking, this analysis introduces two elements which are not found in
topology, in the mathematical sense: movement, that is, dynamics; and emphasis, that is,
the locus of the ego. But we know from other of Piaget’s investigations that primordial
space is heavily egocentric, also in the literal sense; and actions are at the origin of all
structures. Thus, also when considering plastic features, we are forced to return to the
world taken for granted, this time on the side of the subjective pole.
The reduction scheme merely involves a single species of topological qualities,
inclusion. Since there are a few other notions in topology, and since the body partakes of
other elementary spatial relationships, we should expect there to be other modal
properties. If movement is taken away , most of the relations would reduce to contiguity
and factorality, which are basic to indexicality. Thus, there is not only a need to extend the
reduction scheme, but pursue the reduction further, in order to discover such properties as
relate topology, indexicality, Gestalt relations, and so on. These observations should bring
us to observe another shortcoming of Saint-Martin's approach: her neglect to clarify the
relationship between the topological notions and the Gestaltian ones, which constitute her
second class of syntactical relations. Some of these notions are clearly related, as is the
case of proximity and contiguity. Furthermore, it would be important to elucidate the
relationship of these latter notions to indexicality and to the syntagmatic axis, both of
which involve contiguity. Conceivably, we may be able to derive all the basic notions of
visual semiosis, starting out from the elementary idea of indexicality, which (as we hinted
above) is the basic meaning relation in perception.
Perhaps we could take the elementary notion, in this context, to be the topological
relation of proximity. Two units in proximity which remain separate already constitute an
indexicality; in terms of Husserlean phenomenology, they form a pairing; in Gestalt
psychology, the same phenomena give rise to a contiguity; in the rhetorics of Groupe µ,
they could, under certain circumstances, appear as a disjoint figure in praesentia, and in
ordinary rhetorics as a metonymy. If the two units are visibly different in some relevant
respect, they are said, in structural linguistics, to form a contrast. If, however, one of the
separate units is not present in perception, but only in the mind of the perceiver, the units
together create an opposition, more exactly, a binary opposition, in terms of structural
linguistics. Many oppositions together will form a paradigm and, eventually, a structure.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
67
In the rhetorics of Groupe µ, the relation is one of disjunction in absentia. Depending on
the temporal modification of the absent member, the pair will constitute a retention or a
protention, from the point of view of phenomenology. Under certain circumstances, the
member which is directly given to perception becomes the expression, and the absent
member the content, of an appresentation, semiotic function, or sign. If the constituents of
such a sign relation remain in contiguity, or if the are retained or protained as entering such
a relation, they form, more in particular, an indexical sign, or index.
Gestalts, or perfect configurations, repose on some kind of inclusion. They are thus
related to the semiotic concept of factorality, the classical rhetoric notion of synecdoche,
and the the Groupe µ notion of conjunction in praesentia. Inclusion is also involved in
figure/ground, centre/periphery, body/appendages, and similar phenomena. Adding
movement, all the modal/vectorial properties may equally be derived. In a similar way, we
can derive syntagms and syntax from the topological property of order, the notion of
isotopy, as well as Goodman's density, from the notion of continuity, and the
corresponding notion of allotopy from discontinuity. And many other parallels remain to
spelled out.
It would be wrong to think of such a series of parallels as a reduction without
residue. There is no point in merely identifying, for instance, as Jakobson did, metaphor
and paradigm, as well as metonymy and syntagm. Instead, one of these terms should be
reconstructed on the basis of the other (if possible). Although structural linguistics,
Husserlean phenomenology, Piagetian topology, Peircean semiotics, Groupe µ rhetorics,
and the psychology of perception all start out, for example, from a kind of two term
relation, they use it to establish different constructs, phenomenology adding time
modification and degree of emphasis, linguistics introducing the idea of distinctness, and so
on. Nor is is certain that all these concepts really merit a place in an integrated framework
of perceptual semiotics.
As always, however, when we conceive of something in terms of reduction, the
ultimate terms must be based on something which is itself more firmly established. And
that is where we are brought back again to the ecology of semiosis.
Conclusion: Towards a semiotic ecologySaint-Martin's Gestalt theory book must be seen as part of a project, including a number
of earlier and forthcoming books, which are intended to lay a firm, perceptual basis for
pictorial semiotics. Like Saint-Martin's earlier book (and no doubt her forthcoming ones),
it makes for illuminating and, in particular, stimulating, reading. Two points, forcefully
made by Saint-Martin in this book, as well as in other contexts, have been retained here:
the idea that pictorial semiotics (and visual semiotics generally) must be based on the fact
that the pictorial sign, before being anything else, is an object of visual perception; and the
requirement, following upon that observation, that pictorial semiotics should assimilate
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
68
the knowledge on perception assembled by Gestalt psychology and other approaches to
visual perception, thereby abandoning the autonomy postulate so characteristic of many
schools of present-day semiotics.
We have tried to adduce further reasons for giving up the autonomy postulate, but,
at the same time, we have pointed to the weak position of mere passivity and receptivity
in which Saint-Martin's proposal seems to leave semiotics. As against this conception of
semiotics as a mere ancillary, or second hand, science, we have argued for the necessity of
going beyond mere interdisciplinarity, as well as total autonomy, of giving semiotics
access to its proper procedures of verification, including the setting up of properly
semiotic experiments. Indeed, we have suggested that semiotics (in due time) should have
something of its own to offer to the other sciences, not merely take over the knowledge
base contributed by other approaches. Moreover, in order to steer free of interdisciplinary
triviality, we must insert this knowledge base into the problem space developed within
semiotics, as a consequence of the continuing concerns of own research tradition.
While admitting that the picture sign is, first and foremost, an object present in
visual perception, and therefore is dependant on the organization of the visual world, we
have pointed out the difficulty of choosing one among the several extant perceptual
psychologies as a basis for semiotic research, and we have emphasized the importance of
making this selection using criteria intrinsic to the semiotic domain, as well as such which
can be derived from the discussions of the warring factions of psychology. If, however,
semiotics, as we have suggested, is concerned to account for the implicit knowledge
underlying the performance of the users of different signification systems, Gibsonian
ecological psychology stands a much better chance than Gestalt theory of offering an
adequate ground for a theory of pictorial meaning. Inside the domain of perceptual
psychology, moreover, direct registration theory clearly holds the domineering position.
In order to serve the goals of pictorial semiotics, Gibsonian ecology probably needs
to be amended, notably to take account of cultural objects. Psychologists like Neisser,
Rosch, and Costall, and linguistics like Lakoff, have already begun working on this task.
But semioticians can also make a contribution. We will need a semiotic ecology, serving as
intermediary between perceptual psychology and pictorial semiotics. Furthermore, there
is really no reason to let pictorial semiotics be dissolved into the psychology of picture
perception. Just like there is a science of linguistics, which is not identical to
psycholinguistics, we will need a science of pictorial meanings, different from perceptual
psychology. Indeed, there is work to do. We have hinted at some of the great issues here:
the difference between signs and other (perceptual) meanings, pictorality as a sub-category
of iconicity, the nature of plastic meanings. The knowledge base of perceptual psychology
simply serves to bring these questions into view. It is the task of semiotics to resolve
them.
Semiotic ecology could no doubt have a much broader use than the one required in
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
69
pictorial semiotics. It would certainly be basic to the study of sculpture, architecture, and
many other domains not included by Saint-Martin in her visual semiotics. But over and
beyond that, it would lay the foundations of cultural semiotics.
References
Barthes, Roland (1961). Le message photographique, Communications, 1. Also in Barthes,
Roland, L'obvie et l'obtus., 9-24. Paris: Seuil 1982.
– (1964). Rhétorique de l'image, Communications, 4, 40-51. Also in Barthes, Roland,
L'obvie et l'obtus., 25-42. Paris: Seuil 1982.
Bengtsson, H., Bondesson, L., & Sonesson, G. (1985). Facial expression from different
viewing positions. A study in experimental semiotics. Unpublished experimental study.
Bentele, Günter (1984) Zeichen und Entwicklung. Vorüberlegungen zu einer genetischen
Semiotik. Tübingen: Narr.
Bierman, Arthur K. (1963). That there are no iconic signs, Philosophy and
phenomenological research, XXIII:2: 243-249.
Carani, Marie (1986a). Le corpus Pellan: une relecture sémiotique, Protée 14, no 3, 15-26.
– (1986b). Le logos greimasien et l´œuvre visuelle, in Recherches sémiotiques/Semiotic
inquiry , 6:3, 335-344.
– (1988). Re/naissance d'une sémiologie visuelle. Compte rendu de Fernande Saint-Martin,
Sémiologie du langage visuel, Protée, 16, no 1-2, 239-241.
Coffman, Hugh (1980). Pictorial perception: hemispheric specialization and
developmental regression in the neurologically impaired. In The perception of pictures,
Volume II: Dürer's devices, Margaret Hagen, (ed.), 227-261. New York: Academic
Press.
Costall, Alan (1989). A closer look at direct perception. In Cognition and social worlds,
Angus Gellatly, Don Rogers, & John Sloboda, (eds.), 10-21..Oxford: Clarendon Press.
Damisch, Hubert (1979). Sur la sémiologie de la peinture. In A semiotic
landscape/Panorama sémiotique. Actes du premier congrés de l'Association
internationale de sémiotique, Milan 1974., S. Chatman, U. Eco, U., & J.M.
Klinkenberg, (eds.), 128-136. The Hague, Paris, & New York: Mouton.
Deely, John (1982). Introducing semiotic. Its history and doctrine. Bloomington: Indiana
University Press.
Dubois, Philippe (1983). L'acte photographique. Paris & Bruxelles: Nathan/Labor.
Eco, Umberto (1968). La struttura assente. Milan: Bompiani. Swedish translation: Den
frånvarande strukturen. Lund: Cavefors 1971.
– (1976). A theory of semiotics. Bloomington: Indiana University Press.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
70
Floch, Jean-Marie (1978). Quelques positions pour une sémiotique visuelle/Roland
Barthes: Sémiotique de l'image, Bulletin du Groupe de recherches sémio-linguistiques,
4/5, 1-16, 27-32.
– (1985). Petites mythologies de l'œil et l'esprit. Paris: Hadès.
– (1986a). /entries in/ Sémiotique. Dictionnaire raisonné de la théorie du langage. Tome II.
A.J. Greimas, & J. Courtès, J., (eds.). Paris: Hachette.
– (1986b). Les formes de l'empreinte. Périgueux: Pierre Fanlac.
– (1990). Sémiotique, marketing et communication. Sous les signes, les stratégies. Paris:
PUF.
Gardner, Howard (1970). From mode to symbol, British Journal of Aesthetics, 10:3, 359-
375.
– (1973) The arts and human development. New York: Wiley & Sons.
– (1977). Senses, symbols, operations. In The arts and cognition, D. Perkins, & B.
Leondar, (eds.), 88-117.. Baltimore & London: John Hopkins University Press;
– (1982). Art, mind, and brain. New York: Basic books.
– (1984). Frames of mind. The theory of multiple intelligences. London: Heinemann.
Gardner, Howard & Wolf, Dennie (1983). Waves and streams of symbolization: Notes on
the development of symbolic capacities in young children. In The Acquisition of
Symbolic Skills, John Sloboda, & Don Rogers, (eds.), 19-42.
Gibson, James (1971). The information available in pictures, Leonardo, 4:1, 27-35.
– (1978). The ecological approach to visual perception of pictures, Leonardo, 4:2: 227-
235.
– (1980). A prefatory essay on the perception of surfaces versus the perception of
markings on a surface. In The perception of pictures, Volume I: Alberti's window.,
Margaret Hagen, (ed.), xi-xvii. New York: Academic Press.
– (1982). Reasons for realism. Selected essays of James J. Gibson, Edward Reed, &
Rebecca Jones, (eds.). Hillsdale, New Jersey, & London: Lawrence Erlbaum Ass.
Gombrich, E.H. (1960). Art and Illusion. London: Phaidon Press. Second edition 1962.
Goodman, Nelson (1968). Languages of art. London: Oxford University Press.
Greenlee, Douglas (1973). Peirce's concept of sign. The Hague & Paris: Mouton.
Gregory, R.L. (1966). Visual illusions. In New Horizons in pychology, 1, Brian Foss, (ed.),
68-96. Harmondsworth, Penguin. Second edition 1973;
Groupe µ (Dubois, J., Dubois, Ph., Edeline, Fr., Klinkenberg, J.M., Minguet, Ph, Pire, Fr.,
& Trinon, H.) (1970). Rhétorique générale. Paris: Larousse.
– (1976). La chafetière est sur la table, Communications et langages, 29, 36-49.
– (1977). Rhétorique de la poésie. Bruxelles: Editions Complexe.
– (1978). Douce bribes pour décoller en 40 000 signes, Revue d'ésthétique, 3-4, 11-41.
– (1979) Iconique et plastique: sur un fondement de la rhétorique visuelle, Revue
d'ésthétique, 1-2, 173-192.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
71
– (1980). Plan d'un rhétorique de l'image, Kodikas/Code, 3, 249-268
– (1985). Structure et rhétorique du signe iconique. In Exigences et perspective de la
sémiotique. Recueil d'hommages pour A.J. Greimas, H. Parret, & H.G. Ruprecht,
(eds.), Volume I, 449-462. Amsterdam: Benjamins Publishing Co.
– (1989a). Sémiotique et rhétorique du cadre, La part de l'œil, 5, 114-131
– (1989b). Rhétorique visuelle fondamentale, to be published in Text,
– (1989c). Das System der plastischen Form, Semiotische Berichte, 2,3, 215-233.
Gurwitsch, Aron (1957). Théorie du champs de la conscience. Bruges: Desclée de
Brouver.
Hagen, Margaret (1979). A new theory of the psychology of representation art. In
Perception and pictorial representation, C. Nodine, & D. Fisher, (eds.), 196-212. New
York: Praeger Publishers.
– (1980). Generative theory: a perceptual theory of pictorial representation. In The
perception of pictures, Volume II: Dürer's devices, Margaret Hagen, (ed.), 3-46. New
York: Academic Press.
Hjelmslev, Louis (1959). Essais linguistiques. Copenhagen. Second edition: Paris, Minuit
1971.
Hochberg, Julian (1972). The representation of things and people. In Art, perception, and
reality, E.H. Gombrich et al., (eds.), 47-94. Baltimore and London: John Hopkins
University Press.
– (1974). Organization and the Gestalt tradition. In Handbook of perception, Volume I, E.
Carterette, & M. Friedman, (eds)., 180-210. New York & London, Academic Press.
– (1978). Perception. Second, revised edition. Englewood Cliffs, Prentice Hall, Inc.
– (1980). Pictorial functions and perceptual structures. In The perception of pictures,
Volume II: Dürer's devices, Margaret Hagen, (ed.), 47-93. New York: Academic
Press.
Hoffmann, Lotte (1943). Vom schöpferischen Primitivganzen zur Gestalt. München: C.H.
Beck. New edition 1961.
Husserl, Edmund (1939). Erfahrung und Urteil. Prag: Academia Verlagsbuchhandlung.
Jakobson, Roman (1942). Kindersprache, Aphasie, und allgemeine Lautgesetze, Uppsala:
Uppsala University. New edition: Frankfurt/M.: Suhrkamp 1968.
Johnson-Laird, P.N. (1988). The computer and the mind. An introduction to cognitive
science. Cambridge, Massachusetts: Harvard University Press.
Kennedy, John M. (1974). A psychology of picture perception. San Francisco: Jossey-Bas,
Inc.
Krampen, Martin (ed.) (1983). Visuelle Kommunikation und/oder verbale
Kommunikation. Hildesheim & Berlin: Olms Verlag/Hochschule der Künste.
Lakoff, George (1987). Cognitive models and prototype theory. In Concepts and
conceptual development: ecological and intellectual factors in categorization, Ulric
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
72
Neisser, (ed.), 63-100. Cambridge & London: Cambridge University Press;
Lakoff, George & Turner, Mark (1989). More than cool reason. A field guide to poetic
metaphor. Chicago & London: University of Chicago Press.
Lévi-Strauss, Claude (1983). Le regard éloigné, Plon, Paris.
Lindekens, René (1971). Eléments pour une sémiotique de la photographie. Paris &
Bruxelles: Didier/Aimav.
– (1973). Eléménts pour une analyse du code de l'image photographique. In Recherches
sur les systèmes significants. Symposium de Varsovie, J. Rey-Debove, (ed.), 505-534.
The Hague & Paris: Mouton.
– (1976). Eléments de sémiotique visuelle. Paris: Klincksieck.
– (1979). Sémiotique de la photographie. In A semiotic landscape/Panorama sémiotique.
Actes du premier congrés de l'Association internationale de sémiotique, Milan 1974., S.
Chatman, U. Eco, U., & J.M. Klinkenberg, (eds.), 139-146. The Hague, Paris, & New
York: Mouton.
Lombardo, Thomas (1987). The reciprocity of perceiver and environment. Hilldale, New
Jersy, & London: Lawrence Erlbaum Ass.
Luckman, Thomas (1980). Lebenswelt und Geschichte. Paderborn: Schöning,Mukar &ovsky !, Jan (1974). Studien zur strukturalistishcne Ästhetik und Poetik. München:
Hanser Verlag.
Neisser, Ulric (1967). Cognitive psychology. New York: Appleton-Century-Crofts.
– (1976). Cognition and reality. San Fransisco, Freeman & Co.
– (1987). From direct perception to conceptual structure. In Concepts and conceptual
development: ecological and intellectual factors in categorization, Ulric Neisser, (ed.),
11-24. Cambridge & London: Cambridge University Press.
Parmentier, Richard J. (1985). Signs' place in Medias Res: Peirce's concept of semiotic
mediation. In Semiotic mediation. Sociocultural and psychological perspectives, R.
Mertz, & R. Parmentier, (eds.), 23-48. New York & London: Academic Press, Inc.
Pérez Carreño, Francisca (1988). Los placeres del parecido. Icono y representación.
Madrid: Visor.
Pérez Tornero, J.M. (1982). La semiótica de la publicidad, Barcelona: Editorial Mitre.
Piaget, Jean (1945). La formation du symbole chez l'enfant. Neuchatel: Delachaux &
Niestlé. Third edition 1967.
– (1967). La psychologie de l’intelligence. Paris: Armand Colin.
– (1970). Epistémologie des sciences de l'homme. Paris: Gallimard.
– (1972a). Problèmes de psychologie génétique. Paris: Gallimard.
– (1972b). The concept of structure, Scientific thought. 35-56. The Hague & Paris:
Mouton/Unesco;
Preziosi, Donald (1983). Advantages and limitations of visual communication. In Visuelle
Kommunikation und/oder verbale Kommunikation, Martin Krampen, (ed.), 1-34.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
73
Berlin: Olms Verlag, Hildesheim/Hochschule der Künste.
Reed, Edward S. (1988). James J. Gibson and the psychology of perception. New Haven &
London, Yale University Press.
Rosch, Eleanor (1973). On the internal structure of perceptual and semantic categories. In
Cognitive development and the acquisition of language, Thomas E. Moore, (ed.), 111-
144. New York & London: Academic Press.
– (1975). Cognitive reference points, Cognitive psychology, 7,4: 532-547.
– (1978). Principles of categorization. In Cognition and categorization., E. Rosch, & B.
Lloyd, B., (eds.)., 27-48. Hillsdale: Lawrende Erlbaum Ass.
– (1987). Wittgenstein and categorization research in cognitive psychology. In Meaning
and the growth of understanding., M. Chapman, & R. Dixon, (eds.), 151-166. Berlin,
Heidelberg, & New York, Springer Verlag.
Saint-Martin, Fernande (1968). Structures de l'espace pictural.. Montréal: Hurtubise. New
edition 1989.
– (1980). Les fondements topologique de la peinture. Montréal: Hurtubise. New edition
1989.
– (1985). Introduction to a semiology of visual language. Toronto: Toronto Semiotic
Circle.
– (1986). Sémiologie et syntaxe visuelle. Une analyse de ’Mascade’ de Pellan, Protée 14,
no 3, 27-40.
– (1987a). Sémiologie du langage visuel. Québec: Presse de l'Université du Québec.
– (1987b). L'insertion du verbal dans le discours de Pellan, Canadian Literature, no 112-
114, 28-45.
– (1988). De la fonction perceptive dans la constitution du champ textuel, Protée 16, no 1-
2, 202-213.
– (1990). La théorie de la Gestalt et l'art visual. Québec: Presse de l'Université du
Québec.
Sander, Friedrich & Volkelt, Hans (1962). Granzheitspsychologie. München: Verlag C.H.
Beck.
Schaeffer, Jean-Marie (1987). L'image précaire, Paris: Seuil.
Sonesson, Göran (1981a). Esquisse d´une taxonomie de la spatialité gestuelle. Paris: École
des Hautes Études en Sciences Sociales (mimeographed).
– (1981b). Du corps propre à la grande route, Bulletin du Groupe de recherches sémio-
linguistiques , 18, 31-42.
– (1981c). Variations autour d´un bras. Paris: École des Hautes Études en Sciences
Sociales (mimeographed).
– (1987). Notes sur la macchia de Kandinsky; le problème du langage plastique, Actes
Sémiotiques: le Bulletin , X, 44, décembre, 23-28.
– (1988). Methods and models in pictorial semiotics. Report 3 from the Semiotics project.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
74
Lund: Institute of Art History (mimeographed).
– (1989a). Pictorial concepts. Inquiries into the semiotic heritage and its relevance for the
analysis of the visual world. Lund: ARIS/Lund University Press.
– (1989b). Bildanalysens plats i semiotiken,. Konstvetenskaplig bulletin, 35, Autumn
1989.
– (1989c). Le mythe de la triple articulation. Modèles linguistiques et perceptifs dans la
sémiotique des images. Contributions to the Fourth congress of the International
Association for Semiotic Studies, Barcelona/Perpignan, Mars-April 1989, to be
published in the acts of that congress.
– (1989d). Semiotics of photography. On tracing the index. Report 4 from the Semiotics
project. Lund: Institute of Art History (mimeographed).
– (1989e). A bibliography of pictorial and other kinds of visual semiotics. Report 1# from
the Semiotics project. Lund: Institute of Art History (mimeographed). Also in EIDOS.
Bulletin international de sémiotique de l'image, 2, 1989, 3-29; and 2, 1990, 5-42.
– (1989g). Tjugofem års soppa på Panzanis pasta. Postbarthesianska betraktelser över
bildsemiotiken. Conference given at the invitation of the Norwegian Association for
Semiotic Studies, November 1989, to be published in the review of that organization,
Livstegn, 1, 1990.
– (1990a). Rudimentos de una retórica de la caricatura. In Investigaciones semióticas III.
Actas del III simposio internacional de la Asociación española de semiótica, Madrid 5-
7 de diciembre de 1988; Volumen II, 389-400. Madrid: UNED.
– (1990b). Vägen bortom bilden. Datorbilden inför bildsemiotiken, Konstvetenskaplig
bulletin, 36: 14-29.
– (1990c). The challenge of visual semiotics. Review of Saint-Martin, Fernande,
Sémiologie du langage visuel, The semiotic review of books, 2 , 6-8.
– (1990d). Cuadraturas del círculo hermenéutico. El caso de la imagen. To be published in
the acts from Jornadas de semiótica visual, Bilbao, octubre de 1990.
– (1990e). Iconicité de l’image — Imaginaire de l’iconicité. De la ressemblance à la
vraisemblance. To be published in the acts of Premier Congrès de l’Association
internationale de sémiologie de l’image, Blois, novembre de 1990.
– (1990f). The semiotic function and the genesis of pictorial meaning. To be published in
the acts of the 3rd Annual Meeting and Congress of The International Semiotics
Insitute, ’Center/Periphery in representations and institutions’, Imatra, Finland, July
16-21, 1990.
– (1990g). Bodily semiotics and the extensions of man. To be published in the acts of the
3rd Annual Meeting and Congress of The International Semiotics Insitute,
’Center/Periphery in representations and institutions’, Imatra, Finland, July 16-21,
1990.
– (1991a). Kroppens byggnad och bruk. Några semiotiska modeller. In Att tala utan ord.
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
75
Människan icke-verbala uttrycksformer, Göran Hermerén, (ed.), 83-100. Stockholm:
Almqvist & Wiksell International.
– (1991b). Comment le sens vient aux images. Un autre discours de la méthode.
Conference given at the Université Laval, Québéc, Spring 1991, to be published in Les
Nouveaux Cahiers du CÉLAT, , Québéc: Les éditions du Septentrion/CÉLAT.
– (1992a) Bildbetydelser. Inledning till bildsemiotiken som vetenskap. Lund:
Studentlitteratur.
– (1992b). The semiotics of picturehood. The state of the art at the beginning of the
nineties. To be published in The semiotic web 1992, Thomas Sebeok, & Jean Umiker-
Sebeok, (eds.). Bloomington, Indiana: Indiana University Press.
Tamm, Maare (1990). The semiotic function. Studies in children’s representations. Umeå:
University of Umeå.
Tardy, Michel (1964a). Les zones privilégiés de perception de l’image, Terre d’images, 2:
197-199.
– (1964b). Le trosiéme signifiant, Terre d’images, 3: 313-322.
Thürlemann, Félix (1982). Paul Klee. Analyse sémiotique de trois peintures. Lausanne:
L’Age d'Hommes.
– (1990).Vom Bild sum Raum. Beiträge zu einer semiotischen Kunstwissenschaft. Köln.
DuMont.
Trevarthen, Colwyn & Logotheti, Katerina (1989). Child and culture: Genesis of co-
operative knowing. In Cognition and social worlds, Angus Gellatly, Don Rogers, &
John Sloboda, (eds.), 37-56. Oxford: Clarendon Press.
Tversky, Amos (1977). Features of similarity, Psychological Review, 84,4: 327-352.
Vallier, Dora (1977). Pour une définition de l'espace dans l'art, Critique, 366: 1044-1058
- (1981). Minimal units in architecture. In Image and code., Wendy Steiner, (ed.), 139-
146. Ann Arbor: University of Michigan.
Vanlier, Henri (1983). Philosophie de la photographie, Laplume: Les cahiers de la
photographie.
Weinhandl, Ferdinand (ed.) (1960). Gestalhaftes Sehen. Zum hundertjährigen Gebutstag
von Christian von Ehrenfels. Darmstadt: Wissenschaftliches Buchgesellschaft.
Winner, Ellen (1982). Invented worlds. The psychlogy of art. Cambridge, Mass., Harvard
University Press.
Göran Sonesson obtained his doctoral degree in linguistics from Lund University,
Sweden, in 1978, and was awarded in the same year an equivalent degree in semiotics by
the Ecoles des Hautes Etudes en Sciences Sociales in Paris. From 1974 to 1981, he
conducted research in general semiotics and the semiotics of gesture in Paris, and was later
involved with general and Mayan linguistics in Mexico. Since 1983, he has been in charge
Sonesson, Pictorial semiotics, Gestalt theory, and the ecology of perception
76
of the Semiotics Project at Lund University, some results of which have been published in
his book Pictorial concepts (1989). President of the Swedish Association for Semiotic
Studies, and a Swedish representative in the executive commission of the IASS, as well as
in the Nordic Association for Semiotic Studies, he is also one of the founding-members,
and vice-president, of the International Association for Visual Semiotics, created at Tours
in 1989.