focus agent

jiabao li fa1.png

Focus Agent (FA) is an exploration into how an agent can improve its host’s focus and performance by changing their virtual environment. The FA learns how to visually and audibly augment or diminish the host’s world view for optimal performance through a neuro and bio feedback loop. As part of our research we built an FA prototype that is designed to provide optimal environmental states to optimize the host’s focus on personal tasks. In particular, our demonstration sought to improve meditation, creativity and logic tasks. To that end we built an AR system using a commercially-available EEG headset for input, an interface created in Unity along with a Microsoft Hololens for output. Through our experience building the FA we believe that this type of system can improve focus and task performance. There is, however, work to be done to improve the system. The user experience can be more intuitive and the agent interventions can be more subliminal. The hardware implementation must integrate the sensors with the display hardware. Finally, improved signal processing along with the implementation of a machine learning component are necessary to enable the agent to learn the needs of the hosts through an iterative feedback loop.

IMG_4886.JPG

 

Motivation and Idea: Overwhelming digital intake

The balance between work and home is harder to manage than ever before. We are often so overcommitted and overbooked that the hectic pace of modern life takes a toll. This is also called cognitive overload and occurs when task difficulty exceeds resources available by a person – in this case task performance starts to decline.

fa5.jpg

 

If you’ve ever been trapped in a no-focus infinity loop, you know how hard it can be to get out. The field of cognitive science has also shown us that there are ways to get our focus back as quickly and efficiently as possible. Selective attention is a cognitive process in which a person attends to one or a few sensory inputs while ignoring the other ones. Selective attention can be likened to the manner by which a bottleneck restricts the flow rate of a fluid. The bottleneck doesn’t allow the fluid to enter into the body of the bottle all at once; rather, it lets the fluid to enter in certain amounts depending on the flow rate, until all of it has entered the bottle’s body. Selective attention is necessary for us to attend consciously to sensory stimuli in such a way that we will not experience sensory overload. To implement the focus agent we leverage how the brain chooses what to focus on.

 

 

Neuroscientists call this ‘selective attention’, and it comes in 2 different forms:

1. Top-Down: This is the holy grail of focus. Top-down focus is goal oriented. It is responsible for seeing the bigger picture and uses your past experiences to figure things out.

Happens when: You’re studying for an exam or trying to solve a difficult problem.

2. Bottom-Up

When a thought creeps up on you or something around you grabs your attention (like a ping, bing, or notification) you’re suffering from Bottom-Up focus. You can’t help but pay attention to what’s happening.

Happens when: You hear a loud noise, someone pops out of the bushes or your phone buzzes.

fa6.jpg

We perceive the world indirectly by processing and interpreting the raw data from our senses and our thoughts and behavior are frequently biased by our senses. Our perception of people and our attitude towards tasks are influenced by stimulus that our environment present us like temperature, colors and sounds. Human- and computer-based interactions are ubiquitous in our everyday lives—everything from our smartphone, to a variety of computer systems creates serious cognitive demands when switching from one to the next.

We explored subliminal techniques to improve human-computer interaction. The main idea is to “inject” information into the human mind below active awareness, thus transferring supplementary information in a subliminal style without adding load on the cognitive channel. The main benefit of this approach would be the reception of additional, essential information even in the case where (almost) no capacity is left for information transmission in a traditional way.

fa2.jpg
fa3.jpg

How can we understand the user’s cognitive state and how can we intervene?

Detecting cognitive load from EEG signals

jiabao li fa3.png

EEG signal is a non-stationary signals which have different frequency elements at different time intervals. In this study, EEG signals was analyzed using MATLAB Toolbox for EEG ANALYSIS AND CLASSIFICATION (EEGAAC). Matlab’s toolbox  is suitable method for multi-resolution time frequency analysis. In this work, EEGAAC decomposed the EEG into four sub-frequency bands; Delta (1-4Hz), Theta (4-8 Hz), Alpha (8-12.5 Hz) and Beta (12.5-30 Hz). The wavelet decomposition level was set to 5-levels and one final approximation since we are interested in the frequency range of 0-30 Hz only. Table 1 gives a summary of the frequency distribution with wavelet decomposition levels.

Another method of measuring cognitive load is the NASA-TLX test. This test describes cognitive load in terms of subjective responses to a post-experiment survey. This methods is not ideal for the system architecture that we would like to explore but can be used to cross corroborate our EEG setup.  

jiabao li fa4.png

Research on this topic shows that an increase of the rhythm for Beta waves are correlated to analytical type tasks, suggesting that we can use this band to understand the user state during this type of tasks. Decrease in alpha rhythm power from level one to level two and failed to drop in level three of mental stress.

Neurofeedback Loop

Neurofeedback also called neurotherapy or neurobiofeedback, is a type of biofeedback that uses real-time displays of brain activity—most commonly electroencephalography (EEG), to teach self-regulation of brain function.  Research shows neurofeedback may be a potentially useful intervention for a range of brain-related conditions. It has been used for pain, addiction, aggression, anxiety, autism, depression, Schizophrenia, epilepsy, headaches, insomnia, Tourette syndrome, and brain damage from stroke, trauma, and other causes.

 Diminished Reality Test, OpenCV

Diminished Reality Test, OpenCV

We envision the relation of the focus agent and the host to be mutually dependent.

  1. Focus Agent – Host Relationship is a system where the Agent learns from its user in a mutual symbiotic relationship
  2. The Focus Agent can subliminally change the environment visually and audibly (augment, diminish).
  3. The Focus Agent can provide assistance to complete tasks based on user gestures.
  4. The Focus Agent can communicate to other agents within the user’s network.
 

Advisor:  Pattie Maes

Partners: Guillermo Bernal, John Stillman