Physiological earthquakes: Predicting cardiac events combining seismic experience with AI in the hospital
It is rare for a mineral engineering professor to receive funding from the Canadian Institutes of Health Research (CIHR), as it usually falls to biomedical engineering research and others directly involved with health care.
Professor Sebastian Goodfellow, from the Lassonde Mineral Engineering Program, with some CivMin graduate students and a team of predominantly medical doctors at The Hospital for Sick Children (SickKids), affiliated with the University of Toronto, have achieved exactly this. They’ve been awarded a $657,900 grant from CIHR to fund their research into deploying artificial intelligence (AI) in the intensive care unit (ICU) at SickKids to detect and diagnose heart arrhythmias.
Up to 29 per cent of critically ill children experience arrhythmias that cause deterioration and even death. These arrhythmic complications affect roughly 700 critically ill children at SickKids each year. Preventing these complications requires timely detection and accurate heart rhythm diagnosis, which is currently done by continuous clinician surveillance of single-lead electrocardiograms (ECG) displayed on patient monitors. Differing clinician expertise and experience in this task lead to errors and delays in detection and diagnosis associated with preventable patient harm.
Prof. Goodfellow, along with Dr. Mjaye Mazwi and the team at Laussen Labs, a multidisciplinary research group at SickKids, are working to develop and deploy a highly accurate AI capable of expert classification heart rhythm to prototype what they believe will be a generalizable solution to addressing the translation gap in AI for health care.
We met (virtually) with Prof. Goodfellow to talk about the plans for deploying this AI system and how it’s not just as simple as training a model.
You were recently told you have been awarded funding through the Canadian Institutes of Health Research (CIHR).
Yes, along with co-PI [principal investigator] Mjaye Mazwi and the team at Laussen Labs, we have applied many times – perhaps this is the fourth or fifth time – and we’ve been close. In 2020 we were ranked 7th, but funding went to the top six. Again, in 2021 we were ranked 11th, when funding went to the top 10. This time we were successful and it’s very gratifying.
Usually someone in the medical field would receive this kind of funding. It’s unusual for the CIHR to award something engineering related, right?
Well, yes, it is strange. Always raises a few eyebrows. What does mineral engineering have to do with health care? On the surface, not much.
I joined Laussen Labs in 2017 to bring my signal-processing expertise to the group. My PhD research, which I conducted in our Department, focused on applied seismology, which is the study of seismic waves generated by engineering processes such as mining. At the time, Laussen Labs had just started acquiring physiologic waveform data, such as ECGs, which are the electrical signal of the heart. The analysis and modelling of high-frequency time series data require a skillset called digital signal processing. When analyzing earthquake seismograms during my PhD, and afterwards in the private sector, I acquired this skill set, which is how I first got into the health-care field.
However, these multidisciplinary teams are more common than you may think and the reason is the important problems of today and tomorrow spill across borders, cultural divides and fields of knowledge. For example, Laussen Labs developed a bespoke time-series database for the storage of physiologic waveform data at SickKids. The lead database architect was a hydrologist by training whose previous experience was developing a database for the storage of drone photography for a flood plane mapping application. There are, of course, many doctors in Laussen Labs but also computer scientists, a seismologist, a cognitive psychologist and yes, a hydrologist.
Over time, the gap between AI in mineral engineering and AI in health care has become smaller and smaller for me. Beyond publishing proof-of-concept studies in academic journals, deploying AI models in the real world is very hard and the challenges span mineral engineering, health care, and beyond.
What is it you’ve proposed? And what will you do to fulfill this grant?
We are building and deploying a model that detects and diagnoses common pediatric heart arrhythmias using continuous ECG data, which is generally a task staff physicians in the ICU can do very well. The challenge is there are only two staff physicians on duty at any given time to service 42 ICU beds, and detecting and diagnosing heart arrhythmias is just a small part of their job. As a result, these arrhythmias often go undiagnosed for a period and the longer the delay, the worse the outcome for the patient. The idea is to use our expert clinicians to train an AI, which can match their performance and monitor all ICU beds 24 hours a days, seven days a week, looking for arrhythmias.
The model is actually quite vanilla. AI experts from the Vector Institute would be rather underwhelmed – it’s just a WaveNet model performing multiclass classification. Big deal, right? But, employing the golden rule of engineering, Keep It Simple Stupid (KISS), was deliberate. If you search the keywords AI/ Machine Learning/ Deep Learning + Health care in Google Scholar, you’ll find hundreds of thousands of academic papers and the growth is exponential. It’s a hot topic to say the least. However, if you dig a little deeper to see how many of those AI models actually made it to clinical deployment, it’s less than 0.1 per cent. We call this the “translation gap” and we made the decision to keep our model simple, so we could focus on translation.
The translation gap is a result of multiple factors. These include difficulties creating computational infrastructure that can reliably ingest data for real-time classification, the requirement of a production-grade Machine Learning Operations [MLOps] platform for serving, monitoring, and re-training AI models, regulatory challenges integrating AI models into clinical domains, and concerns about responsible validation and bias, sometimes described as “algorithmic fairness”. The team that can close this gap must include a wide range of expertise including bio-ethics, MLOps, law, cloud, software development, human factors, cognitive psychology, digital signal processing and machine learning.
VIDEO: This animation shows an ECG signal transitioning from a normal rhythm to an arrhythmia. In the top right corner is the model score for a particular pediatric arrhythmia called Junctional Ectopic Tachycardia (JET). When the signal transitions, you can see the model score increase.
You have a background in AI, correct?
Before joining U of T, I was the AI Lead at a startup in the mining industry called KORE Geosystems. We developed an AI product that automated various parts of geotechnical and geological core logging workflows. For example, rock type classification and fracture counting. In this role I had to deploy AI models that geoscientists relied on to do their work. This is where I learned just how hard deploying AI models in the real world is. I was able to bring this experience to Laussen Labs where they were running up against similar challenges.
When you’re building products, you’re forced to start from the business requirements and work backwards to the technical solution. Because products are built for users, it’s no surprise why this is the preferred approach. In research, it’s more common to start from a dataset, build a model, publish a paper, and then start thinking about the application, which is no wonder less than 0.1 per cent of AI studies make it to deployment. The product development mindset I developed in the private sector has been invaluable in successfully translating AI models into complex clinical environments.
I’ve sometimes heard people say they don’t trust AI. Is this going to be a challenge?
Trust is always a challenge when introducing any new technology into an established workflow, and AI is no exception. In health care and mining, AI adoption has been seriously impeded because of early and very public projects that started with building a model and only involved the end user, such as a doctor or a geologist, at the very end if at all. This model-centric approach gets people’s walls up quickly and we’re still trying to overcome it even to this day. Therefore, it is imperative to think about your AI model as a product from the very start, which will involve those end users in documenting requirements and ultimately build trust.
There is also a sci-fi perception of AI perhaps resembling Skynet from the Terminator movies. Whenever talking about AI with doctors or geologists, I always try to use the most boring descriptions I can think of. My favourite at the moment comes from the Head of Decision Intelligence at Google, Cassie Cozirkov, who describes Machine Learning, which is a subdomain of AI, as a “thing labeller”. The “thing” could be a photo of an intersection and the “label” could be the number of pedestrians in the intersection. For our arrhythmia model, the “thing” is a five-second segment of ECG data and the “label” is whether an arrhythmia is present. What’s all the hype about, right?
Lastly, how we present the performance of a model to the end user is important and, in my opinion, is the best way to promote trust. We need to use metrics that map to clinical key performance indicators, and we need to present those metrics in a transparent manner over long periods. Most people have no clue how a plane achieves flight or how a jet engine works but they feel safe flying. The reason is there is a one in 20 million chance of dying in a commercial airline plane crash. So, an arrhythmia model that is consistently performing at the level of a board-certified cardiologist will build trust.
By Phill Snel
Team members:
Mazwi, Mjaye, SickKids (co-PI)
Goodfellow, Sebastian (co-PI)
Assadi, Azadeh, SickKids
Bulic, Anica, SickKids
Ehrmann, Daniel, SickKids
Eytan, Danny, SickKids
Goldenberg, Anna, SickKids
Goodwin, Andrew, SickKids
Greer, Robert, SickKids
McCradden, Melissa, SickKids
Gallant, Sara, SickKids
Gnatenko, Vladislav (CivMin, MASc candidate)
Shubin, Dmitrii (CivE MASc 2T1)
July 21, 2022 | Yahoo News
July 21, 2022 | CBC News
A University of Toronto student team took top honours for architecture at an international competition. Congratulations to design and architecture lead Bo Zhao (Year 3 CivE) from the Department of Civil & Mineral Engineering, currently on PEY, and architecture lead Adela Hua a third-year student from the Daniels Faculty of Architecture, who won the Best Architecture Award at the 2022 Seismic Design Competition at Salt Lake City, Utah. The event is held by the Earthquake Engineering Research Institute (EERI).
The team designed a building titled “Tomorrow’s Tower” for the competition. View the score sheet here.
“They placed first out of 32 international teams and this is the best achievement in our team’s history, proving U of T has the best,” says U of T Seismic Design Team captain Eliza Van Weerdhuizen (EngSci 2T1).
The team managed to present their project at the in-person event, with three members attending, but without any model to show. “Tragically, our model actually got lost in the airline and never made it to the competition on time. It is still somewhere in transit/storage with no news on when it will return home,” says Hua. The team persevered Hua recounts,”We were, however, still able to participate in all other parts of the competition, including the poster and power-point presentation where all components of the design were shown, including the model’s digital simulation, build process, and cost estimation.”
July 11 | Global News
CivMin faculty promotions
Faculty members in the Department of Civil & Mineral Engineering have received promotions effective July 1:
- Oya Mercan – promoted to full professor
- Shoshanna Saxe – promoted to associate professor
- Marianne Touchie – promoted to associate professor
A 366-foot-tall sphere is gaining attention as it quickly becomes a noticeable part of the Las Vegas skyline. The construction of the largest sphere on Earth involves CivMin alumni, who played an integral role in the design and construction of the exosphere for this enormous and complex undertaking.
The co-founders of Cast Connex Corporation, President and CEO Carlos de Oliveira (CivE MASc 0T6) and Executive Vice-President Michael Gray (CivE BASc 0T5, PhD 1T2), sat down virtually to chat about the project, sharing perspective on its scope and objectives.
The MSG Sphere at The Venetian, a Madison Square Garden Entertainment Corp. project, is scheduled to open late 2023. “It has developed an entirely new way of sharing and creating media,” offers de Oliveira. “The Sphere being constructed in Las Vegas is the first of what they suggest will be many to be built worldwide.”
It is to be the most technologically advanced entertainment venue ever built; with surrounding screens completely enveloping both exterior and interior surfaces. Covering about 580,000 square feet of LED panels, it will be the largest LED screen in existence. An astoundingly detail-rich 14K uninterrupted interior screen will give visitors a sensory-laden experience.
“It’s a complete wrap-around 14K ultra-high resolution LED screen. Everywhere you look is an immersive surface – everyone will be completely enveloped in the show,” expresses de Oliveira. “Additionally, they will have audio technologies that will enable incredible control of audio perception of everyone in this venue.”
The ability to divide audio feeds to various audience areas, via its 164,000 speakers, will allow for the tailoring of specific content. Astoundingly, different groups could be seated quite close to each other but not experience interference from the other’s custom audio feed. Explains de Oliveira, “In other words, they could have a block of seats over here with someone speaking in English and another block of seats, just a few seats over, hearing a different language.”
Getting to the technical aspects of constructing the sphere, Gray says, “The outside of the Sphere is also covered with an LED array, so they can project anything onto it. If you’re going to have the world’s biggest curved screen on the inside and the outside, then it needs to be dimensionally controlled. All of the LED panels have to match exactly, and the only way the constructors could figure out how to reliably pull this off was to come to Cast Connex.”
“The build is created with unique cast nodes at every location, so the constructor can just put straight members between, locking in the proper geometry,” continues Gray. In all, some 368 cast steel nodes, fitting like X-intersections in the design, are being used in the exosphere.
Giving scope to the immense project, Toronto’s Ontario Place Cinesphere, a 35 m (115 ft) wide dome with seating for up to 614, would fit many, many times over inside the MSG Sphere. For some perspective, the new venue will have 17,500 permanent seats, with an additional 2,500 retractable seats, bringing maximum seating capacity to 20,000.
U of T origin story
A chance meeting, while both Gray and de Oliveira were on a work term, led to them both attending U of T, working together, then founding their company. While student work colleagues, Gray persuaded de Oliveira to not pursue graduate studies at a U.S. institution but choose U of T instead. He raved about the professors and CivMin’s resources, especially the Structures Lab.
“With my research we were looking at the use of steel castings to simplify the fabrication and improve the performance of seismic resistant connections for steel structures; particularly bracing connections,” recounts de Oliveira. While he completed his MASc in 2006, it was suggested to him his research continue with, ironically, someone who was already a friend, “We found a perfect student to continue your work. His name is Michael Gray and he’s really intelligent.”
Gray was persuaded to move on from his MASc thesis and go directly to a PhD with this work. “Michael’s research continued to establish and broaden our understanding of casting manufacturing and to improve the performance of connections and structures that have to resist earthquakes,” says de Oliveira. The work proved so demonstrably effective, their thesis supervisors suggested the methods were patentable. With this encouragement, provisional patents were filed and a fledgling new niche industry was created.
The pair had not originally thought to strike out on their own, thinking their work might be something woven into roles at larger companies.
“A key part of our success was the founding of Cast Connex to be the vehicle to commercialize our technologies,” de Oliveira recounts, “and that action was linked to my having been selected for a Heffernan Commercialization Fellowship at U of T. It provides a sum to cover costs over two years to help a grad student develop a business plan. If not for this grant, I don’t think we would have really taken that big step; this was a big part of getting the momentum to create the business and start growing it.”
Cast Connex, spawned from collaboration at U of T, continues to forge strong bonds with CivMin research. Ongoing relationships with past Professors Constantin Christopoulos, Oh-Sung Kwon and Jeffrey Packer have led to long-term collaborations in research too. Current PhD candidate Pedram Mortazavi has continued the work on cast steel replaceable links with eccentrically braced frames (EBF).
The company continues to employ many U of T Engineering alumni, with five in the Toronto office of ten employees.
The importance of being bilingual
Asked if the Sphere project is in metric, as most Canadian and international designs might be, the reply from de Oliveira brings a humorous, but diplomatic response, “This project is entirely designed in U.S. Standard, even inches, but it’s okay … we’re bilingual.”
By Phill Snel
——
BY THE NUMBERS:
- US$1.8-billion project
- 14K resolution video
- 23 luxury suites
- 20,000 seating capacity
- 164,000 speakers
- 580,000 square feet of LED panels
- 366 feet tall
- 516 feet wide
- 368 Cast Connex cast steel nodes in the exosphere
- 734 Cast Connex cast steel nodes in the structural backup to the immersive surface
- 18,000 lbs – weight of the heaviest exosphere cast steel nodes
- 775 tons of steel
- 1,400 workers on site
July 7, 2022 | Discover
June 29, 2022 | TVO
Are panels made from mushrooms and hemp a realistic solution to cut greenhouse gas emissions in the construction industry?
The construction industry has a pollution problem, but University of Toronto alumnus John Kabanda (CivE MASc Structural and Geotechnical Engineering 1T3) has developed a new building material that promises to cut costs for contractors and reduce a project’s carbon footprint.
“With the environment, we’re almost at a point of no return, and we can’t just keep building things the way we used to. As a structural engineer, I care about the environment we live in, so I thought maybe I can make a difference here.”
Kabanda is vice-president, engineering with okom wrks labs, overseeing the development and testing of a structural mycelium composite that would replace plywood panels typically used to cover the exterior walls of building before any siding is installed.
This mycelium biocomposite known as zero-frm is made from hemp and fungi to create a durable and eco-friendly building material, which Kabanda contends to be 33% stronger than competitors’ offerings.
He hopes it will be the first mycelium composite to be cleared for structural use soon and one day, it may be suitable for roofs and subfloors too.
“The process is not too complicated and not too energy-consuming. I was really taken with this idea of making something very simple that can have a huge impact.”
The production of this mycelium composite can reduce carbon emissions in several ways.
All the raw materials and resulting products are lighter and therefore cheaper to transport.
Hemp used in manufacturing is recycled by farmers who simply grow it and then throw it away to keep their soil nutrient-rich between crop cycles.
And because all the ingredients are naturally occurring materials, the panels can be sent to the landfill in good conscience.
They’re also entirely edible, although Kabanda would never advise anyone to start chewing the scenery.
“I think every company right now must embrace change and if we work together, we can solve this,” says Kabanda. “There’s a lot of pressure on engineers to look for innovative methods to reduce the emissions we’re producing.”
The future is green, says Kabanda, who believes that young engineers can learn a lot from University of Toronto researchers, just like he did.
“Many U of T professors are putting a lot of effort into sustainability, which tells you that’s where the industry’s headed. And if we can all work together, that’s how we can solve this climate crisis.”
By David Goldberg