NAT Logo

Bermuda

Bermuda is a custom solution for real-time integrated recording and analysis of EEG, eye-tracking, and body-tracking. Each of these three elements is being developed by a subgroup of the Bermuda team and are all based upon relatively inexpensive and widely accessible components. As a result of taking full advantage of consumer hardware, Bermuda will offer a complete system that is capable of real-time analysis for only a fraction of the cost of comparable systems that have been developed in the past. Additionally, the communications and integration of these 3 data streams are being built from the ground up.

What is Bermuda?
Objectives

The primary goal for Bermuda in the near future is to create a system that researchers can use to investigate novel research questions in areas such as prosthetics and movement disorder research. Ease of adoption and replication are central to the ethos of Bermuda. For this reason, the team has a heavy focus on communicating ideas and enabling the whole team to benefit from every individual’s work. This allows for cross-team idea-sharing and easy onboarding of new members.

Rapid Development

Bermuda is currently in the early prototyping stage. For body-tracking, the team has recently achieved real-time tracking of body positions by using the PoseNet deep learning model running locally on a Raspberry Pi. Specifically, two simultaneous images are captured from a stereo camera module on the Raspberry Pi, then PoseNet is used to find the 2D locations of body parts individually in both of these images. The two estimated 2D poses are then combined into a 3D skeleton using OpenCV which will be sent to a central computer. One of the important next steps for this group is to perform an analysis of tracking accuracy by comparing predicted values to real-world measurements. Additionally, the first version of this system does not meet the performance standards required and will need to be improved and optimized to achieve smooth tracking of rapid movements. This could include using external processing modules (e.g. Coral Edge TPU), cloud processing, smaller/lighter machine learning models, or a combination of these.

The EEG system of Bermuda is being rapidly developed and prototyped currently. This system has a few major parts, outlined in the following paragraph: The first part of the system has to do with the power and battery charging which will aid us in having the EEG part of Bermuda be completely wireless. Next, there is the Analog to Digital Converter, a device that will allow us to take the raw EEG data and turn it into signals which can be read by a microcontroller or a computer over an SPI interface. Next, we have a temperature probe and accelerometer which will give more useful information for researchers conducting EEG experiments. Lastly, we have a BlueTooth module that both acts as the on-board microcontroller and is how this EEG system is able to wirelessly stream data to a computer or even a phone.

As of June 2020, the system has a schematic which outlines all electrical connections for each part of the board as well as a layout of the prototype PCB. We are now working to revise this layout to ensure that everything will work correctly and there are no faults.

User Interaction

At the center of Bermuda, there will be a Python-based graphical user interface (GUI) that streams data wirelessly from all 3 of the components by utilizing parallel processing. This GUI will display eye-gaze, body position, and EEG data in real-time as it is received. For eye-tracking, the team is making use of the Pupil Labs open source eye-tracking platform, although there will be significant alterations to the design of the device and communications protocols. Last, but certainly not least, the EEG team has been working on interfacing an ADS1299 development board with a Raspberry Pi which will form the core of the EEG device. This is far from a trivial task and requires solving many low-level electrical engineering problems through a joint effort of many talented team members.

Bermuda Project Team
Eden Redman Landon Fuhr James Davis Madeleine Ridgway Sebastian Aravena Pacheco
Team Lead - BA, Fine Arts and Design Body Tracking Lead - BSc, Physiology PCB Designer - BSc, Electrical Engineering UX/UI Programmer - BSc, Neuroscience Communications - BSc, Electrical Engineering
Responsible for coordinating communication and collaboration between software and hardware developers as well as between the sub-teams working on each of the three modalities that make up Bermuda. Eden also sets project goals and oversees the progression towards these milestones, beyond this he works hands-on where he is needed in order to keep the project moving forward. Responsible for conceptualizing and developing the body tracking system for Bermuda. Landon is using lightweight deep learning models to provide real-time pose estimation. He is building the system centered around a Raspberry Pi to keep costs down along with a stereo camera module to allow for identification of 3D locations of parts. Responsible for aiding all teams in hardware development, with an emphasis on the EEG team. James is responsible for developing the initial in-house EEG prototype that allows for future scalability, through which the project is poised to have a significant impact in the research community, and beyond. Responsible for designing and programming the user interface for Bermuda. Madeleine is working on integrating the data streams from the project's three elements into a single interface, which will allow easy access to the information and simplify future applications of the system. Responsible for SPI communication between the Raspberry Pi and the ADS1299 analog to digital converter used in the collection of EEG data. One of Bermuda’s main design constraints will be tracking free moving bodies therefore Sebastian works to eliminate wires that could impede movement through the use of a QN9080SIP, an Ultra-low-power Bluetooth 5 SIP, to transfer EEG data to an on-board laptop BLE.