Bermuda
Bermuda is a custom solution for real-time integrated recording and analysis of EEG, eye-tracking, and body-tracking. Each of these three elements is being developed by a subgroup of the Bermuda team and are all based upon relatively inexpensive and widely accessible components. As a result of taking full advantage of consumer hardware, Bermuda will offer a complete system that is capable of real-time analysis for only a fraction of the cost of comparable systems that have been developed in the past. Additionally, the communications and integration of these 3 data streams are being built from the ground up.
The primary goal for Bermuda in the near future is to create a
system that researchers can use to investigate novel research
questions in areas such as prosthetics and movement disorder
research. Ease of adoption and replication are central to the
ethos of Bermuda. For this reason, the team has a heavy focus on
communicating ideas and enabling the whole team to benefit from
every individual's work. This allows for cross-team idea-sharing
and easy onboarding of new members.
Bermuda is currently in the early prototyping stage. For
body-tracking, the team has recently achieved real-time tracking
of body positions by using the PoseNet deep learning model
running locally on a Raspberry Pi. Specifically, two
simultaneous images are captured from a stereo camera module on
the Raspberry Pi, then PoseNet is used to find the 2D locations
of body parts individually in both of these images. The two
estimated 2D poses are then combined into a 3D skeleton using
OpenCV which will be sent to a central computer. One of the
important next steps for this group is to perform an analysis of
tracking accuracy by comparing predicted values to real-world
measurements. Additionally, the first version of this system
does not meet the performance standards required and will need
to be improved and optimized to achieve smooth tracking of rapid
movements. This could include using external processing modules
(e.g. Coral Edge TPU), cloud processing, smaller/lighter machine
learning models, or a combination of these.
The EEG system of Bermuda is being rapidly developed and prototyped
currently. This system has a few major parts, outlined in the following
paragraph: The first part of the system has to do with the power
and battery charging which will aid us in having the EEG part of
Bermuda be completely wireless. Next, there is the Analog to Digital
Converter, a device that will allow us to take the raw EEG data and
turn it into signals which can be read by a microcontroller or a
computer over an SPI interface. Next, we have a temperature probe
and accelerometer which will give more useful information for researchers
conducting EEG experiments. Lastly, we have a BlueTooth module that
both acts as the on-board microcontroller and is how this EEG system
is able to wirelessly stream data to a computer or even a phone.
As of June 2020, the system has a schematic which outlines all electrical
connections for each part of the board as well as a layout of the
prototype PCB. We are now working to revise this layout to ensure
that everything will work correctly and there are no faults.
At the center of Bermuda, there will be a Python-based graphical user interface (GUI) that streams data wirelessly from all 3 of the components by utilizing parallel processing. This GUI will display eye-gaze, body position, and EEG data in real-time as it is received. For eye-tracking, the team is making use of the Pupil Labs open source eye-tracking platform, although there will be significant alterations to the design of the device and communications protocols. Last, but certainly not least, the EEG team has been working on interfacing an ADS1299 development board with a Raspberry Pi which will form the core of the EEG device. This is far from a trivial task and requires solving many low-level electrical engineering problems through a joint effort of many talented team members.
Eden Redman | Landon Fuhr | James Davis | Madeleine Ridgway | Sebastian Aravena Pacheco |
---|---|---|---|---|
Team Lead - BA, Fine Arts and Design | Body Tracking Lead - BSc, Physiology | PCB Designer - BSc, Electrical Engineering | UX/UI Programmer - BSc, Neuroscience | Communications - BSc, Electrical Engineering |
Responsible for coordinating communication and collaboration between software and hardware developers as well as between the sub-teams working on each of the three modalities that make up Bermuda. Eden also sets project goals and oversees the progression towards these milestones, beyond this he works hands-on where he is needed in order to keep the project moving forward. | Responsible for conceptualizing and developing the body tracking system for Bermuda. Landon is using lightweight deep learning models to provide real-time pose estimation. He is building the system centered around a Raspberry Pi to keep costs down along with a stereo camera module to allow for identification of 3D locations of parts. | Responsible for aiding all teams in hardware development, with an emphasis on the EEG team. James is responsible for developing the initial in-house EEG prototype that allows for future scalability, through which the project is poised to have a significant impact in the research community, and beyond. | Responsible for designing and programming the user interface for Bermuda. Madeleine is working on integrating the data streams from the project's three elements into a single interface, which will allow easy access to the information and simplify future applications of the system. | Responsible for SPI communication between the Raspberry Pi and the ADS1299 analog to digital converter used in the collection of EEG data. One of Bermuda's main design constraints will be tracking free moving bodies therefore Sebastian works to eliminate wires that could impede movement through the use of a QN9080SIP, an Ultra-low-power Bluetooth 5 SIP, to transfer EEG data to an on-board laptop BLE. |