• A new 2023/24 Arts Council research-based project proposal investigating sound-color and time-space synesthesia, algorithms, and anthropomorphism.


  • ** Please note that this is a working research page and in its very early stages of development. The following tests were undertaken in order to showcase them to a cross-disciplinary team for the purposes of obtaining funding for its further research and development, and to provide context to potential research sources  that I contacted.

    The text is partly in note form and constitutes a series of themes and an investigation into their commonality **

    Page last updated: 15th Dec 2023

    Please note: If you are viewing from a smart-phone, audio-visual tests can be found scrolling toward the lower section of this page.


  • I am currently establishing my aims for the project and it is at an initial stage early stage so many aspects are still to be developed and consolidated.

    In summary they would constitute an investigation into generating simple visual and audio algorithms. These algorithms would then be being applied to various software to generate a series of experimental films and constitute the beginning of an interactive public art event and education resource for schools and SEN pupils.

    The ‘data’ that would inform the audio and visual elements would be a playful experiment into collating responses from questionnaires, assessing if the participant was happy, sad, excited and other basic human emotions. The audio and visual components from this data would be animated and offer a fun digital reflection of the participant.

    I would hope that by including research into synaesthesia that it would help develop a simple visual audio and visual vocabulary for the algorithms.

    As part of the project I am aiming to undertake research with the following professionals:

    • – Interdisciplinary creative practitioner
    • – Special educational needs (SEN) teacher
    • – Algorithm engineer /  algorithm developer
    • – Musicians and composers
    • – Sound engineer

 

 

 

 

 


A previous completed funded project for the Arts Council, E8 UFO, involving 10 London schools, can be viewed HERE

A related previous completed project / short film screening, can be viewed HERE

I’ve also been working on a public art project for a new bandstand design in Bedfordshire titled, ‘Sound and Vision’. During a collaborative outreach stage of the project I worked with a group of SEN pupils, (Chiltern School, Bedfordshire) recording random audio and passing it through a digital oscilloscope to explore a visual representation of sound. This project can be viewed HERE


 

SYNESTHESIA

Sound-color synesthesia: Sound-color synesthesia is where you see specific colors when you hear certain sounds. It tends to be specific to certain sounds or music. Musicians and artists often describe having this form.

Time-space synesthesia: This is a form of synesthesia where you visualize things in a very specific way. People who have this form of synesthesia often “see” sequences with specific patterns or forms. An example of this is visualizing a calendar or a string of numbers in a certain way. Some people can mentally “map” these out in vivid or detailed ways.

Notes: 2023-11-21

  • Combined 2 or more senses, affecting 4% of the wold’s population.
  • Sound-colour synaesthesia can affect opacity, thickness and texture.
  • ‘Normal’ brain development separates the the mind’s senses at 4 months from birth.
  • Grapheme–colour synaesthesia or coloured grapheme synaesthesia is a form of synesthesia in which an individual’s perception of numerals and letters is associated with the experience of colours.
  • Ordinal linguistic personification is a form of synesthesia. Those who have OLP automatically associate personalities, genders, and even interpersonal relationships to numbers, letters, months, days of the week, or other list items.
  • Wesley Trisnadi describes Marcia Smilack, a synesthete, whom is able to sense the perfect moment in which to take a photograph.
  • At first glance, synesthesia and autism are two completely unrelated things: synesthesia is a blending of the senses, while autism is characterized by challenges with social skills, repetitive behaviours, speech, and nonverbal communication.
  • At first glance, synesthesia and autism are two completely unrelated things: synesthesia is a blending of the senses, while autism is characterized by challenges with social skills, repetitive behaviors, speech, and nonverbal communication. However, studies have shown that the prevalence of synesthesia is almost three times higher in people with ASD (18.9%) compared to that of the general population (7.2%) (Baron-Cohen et al., 2013), which suggests some type of connection between the two conditions. Further research has shown that both conditions can be attributed to excessive neuro-connection and activities (Belmonte, 2004).

The genetic basis and the mechanism of both conditions are still unclear. However, in terms of brain connectivity, a predominance of local over global connectivity is a possible pattern in both autism and synaesthesia. A recent study by van Leeuwen.et.al (2020) pointed out that more studies involving synaesthetes are needed to confirm this hypothesis with direct comparisons between autism and synaesthesia. With regard to sensory processing, both groups show altered sensory sensitivity; and in perceptual tasks, evidence for an autistic-like, detail-oriented style in synaesthesia is accumulating. 

From The Interesting Connection Between Autism and Synesthesia, written by: Amber Wu

Question:

Can a robot have Synesthesia? Would it be an equvalent to a malfunctioning or incorrect algorithm?

  • Annie Dickinson describes seeing different colours for individuals voices, and this colour changing depending if they are overstressed, or about to become ill. With this ability being she was able to tell her father, for example, that he was about to come down with a bad cold when the colour of his voice changed to a particular colour tone.
  • Imogen Malpas discussed time-space synaesthesia and how cultural influences can determine the visualiseation of time. For example, in the west we read from left to right so our perception of cultural understanding of time can be visualised or illustrated with a time line running from left to right. This changes with Arabic for example which is written from right to left. In addiotn certain communites in Papa New Guniea describe the future, instead of in front of them and the past behind them, as being at the peak of a mountain and at the souse of their local river, and the past being down river. Different cultural influences dictate the axis line upon which we understand time. Interestly she discusses the our understanding of time is so abstract and difficult to properly understand that we have the need to collate to our perception of space, (which is easier to comprehend).

ALGORITHMS

An algorithm is a set of commands that must be followed for a computer to perform calculations or other problem-solving operations.According to its formal definition, an algorithm is a finite set of instructions carried out in a specific order to perform a particular task.

ISAAC ASIMOV’S DEFINITION OF ROBOTS ‘We think of robots as performing tasks more rapidly and more efficiently than man, therefore is any machine a robot? For example, a sowing machine can sow faster than a human and a television can detect and organise radio waves as we can not and so on….therefore, we must apply the term robot that is more specialised than an ordinary devise – a robot is a computerised machine.’


ANTHROPOMORPHISM

The showing or treating of animals, gods, and objects as if they are human in appearance, character, or behaviour: The books “Alice in Wonderland,” “Peter Rabbit,” and “Winnie-the-Pooh” are classic examples of anthropomorphism.

Question:

Does giving mechanical devises sounds such as beeps, flashing lights and graphic visual displays endear them to us, beyond dependence and toward an artificial love?

Is our relationship toward modern sophisticated electronic household devices becoming increasingly all encompassing, that the boundary that separates and distinguishes both entities is becoming diluted to the point that our combined synergy is what maintains and calibrates our existence?

GOLEM NOTES: in the Hebrew, god makes a golem

Golem is a creature-type describing artificial life-forms created by magic, typically with power stones providing them with energy.

Robots and automatons are sometimes described as golems, or machines made in the form of a human being. In Hebrew, golem means “shapeless mass” or “embryo.”

golem is an animated, anthropomorphic being in Jewish folklore, which is entirely created from inanimate matter, usually clay or mud.

 


SYNERGY

The combined power of a group of things when they are working together that is greater than the total power achieved by each working separately