December 2019

Letter from the Editor

Is AI Set to Transform ECE Departments?

By Badri Roysam, University of Houston


Dear ECEDHA family and friends,

This is the week of NIPS 2019 – the 33rd Neural Information Processing Systems Conference – a hotbed of excitement and new ideas.

A natural question on our minds is the following: How has Artificial Intelligence, especially its resurgent incarnation in the form of Deep Neural Networks (DNNs), made an impact in our department’s research and/or teaching portfolios? Does it have the potential to transform the very structure of ECE and related departments, as it appears to be doing at MIT? Or will this subject plateau out, or worse?

DNNs are fascinating, despite their seeming conceptual bluntness. My own laboratory has been utterly transformed by them over the past few years, since they have been able to solve long-standing image processing problems. They have also become essential for competitive grantsmanship – our recently funded NIH R01 grant was enabled by DNNs. This is not entirely surprising, since my research involves the application of computer vision methods to biological images, and the resurgence of DNNs was driven by their success on classical computer vision problems. But, what about other ECE areas?

At this very special, and reflective, time of the year, we wish you a restful holiday season.  We look forward to our annual meetings. Bring the family along - the next Annual Meeting and ECExpo will be held march 18 – 21, 2020 at the Renaissance Orlando at SeaWorld.

What is Driving the Resurgence of Neural Networks?

Artificial Intelligence was all the rage in the mid 80’s when I was an undergraduate student, and represented a refreshing gusher of new ways of thinking about computers and problem solving. I was also selected to attend the Connectionist Models Summer School at Carnegie Mellon, and all of the thought leaders including Geoffrey Hinton, Yoshua Bengio, and others were there. It was exciting. Unfortunately, the field lost its charm during the mid 90’s and it became painfully difficult to get any funding for neural network research. Most of my colleagues and myself moved on to other things. However, some of the pioneers carried on.

Around 2014, a singular event happened – neural-network based computer vision algorithms just started to outperform humans, and the computer vision world was rudely woken up to DNNs. This is a big deal because roughly two-thirds of our brains are devoted to vision and associated faculties. There has been no looking back since this singular event. As I mentioned above, my own laboratory has been totally transformed by them over the past few years, since they have been able to solve long-standing biological image processing problems that we were struggling with. They have also become essential for competitive grantsmanship – our recently funded NIH R01 grant was, in fact, enabled by DNNs. As I serve on NIH proposal review panels (study sections), I have witnessed DNNs emerge as a key competitive differentiator, and increasingly, a necessity for tackling hard problems. Importantly, there is no longer the long-standing prejudice/discomfort against DNNs, especially the fact that they are not supported by elegant math and algorithmic constructs like we are accustomed to, and still involve a high degree of “ad hockery." 

How is AI Impacting ECE Departments?

While the impact of DNNs on computer vision is understandable, as Chair, one naturally wonders how other research groups in ECE departments are doing. Out of curiosity, I informally polled some of my colleagues, and was surprised by the sheer breadth and depth of DNNs impact. DNNs are now being used in imaginative new ways, in a surprisingly diverse array of fields that had not seemed like suitable candidates not long ago, including array signal processing, information theory, bioinformatics, systems biology, mobile computing, wireless communications, quantum mechanical modeling, electrochemistry, nanotechnology, protein folding and binding, imaging systems, oil well logging, computational electromagnetics, graphic arts, electronic music, retail analytics, brain-computer interfaces, and financial businesses. Interestingly, this subject is still on the rising edge of the Gartner Hype Cycle, suggesting that DNNs will continue to grow in terms of impact over the next decade.  DNNs ability to capture variability using high-dimensional models has been transformational.

Advances in computer architecture, especially GPU’s have been at the heart of the DNN revolution, and with the recent advent of wafer-scale integrated DNN processors and 7 – 5nm chips, we seem to be on the verge of a “Cambrian Explosion” in computer architecture that David Patterson described, and a lot of it is driven by DNNs. In my own laboratory, my previously beefy but non-GPU servers are now utterly obsolete, and we have been forced to rethink our computing infrastructure completely.

On a somewhat sobering note, there is a growing chorus of voices, that are pointing out the many limitations of DNNs, for example, the massive computation and data needs that greatly exceed what humans need to learn, and importantly, the lack of a well-developed body of theory.  Also, DNNs do not constitute the broader notion of “General-purpose AI” that is ultimately needed, although they are thought to serve as important building blocks.

What does all this mean for Future of ECE departments?

First, DNNs present both opportunities and challenges for the faculty. This field has a fast-paced open source culture that rapidly (on a time scale of hours and days) disseminates insights and code via non-peer-reviewed forums like Arxiv, rather than traditional journals. This is a free-for-all global competition for ideas that is being driven by academic as well as industry players. Young faculty members in this area need special mentoring in order to succeed in their peer community while also succeeding in our classical tenure system.

DNNs have provided new opportunities for students and faculty to invent new algorithms, and solve new problems. They have also called for a newfound focus on developing a better understanding of biological intelligence. They have also noted a modest down-tick in venture investments going towards AI products.  I think we should see all this differently. There is much left to do, and we now have a new toolkit to tackle long-standing problems. Isn’t what we all live for?

While we have witnessed many changes to our discipline over the years, and lived through many hype cycles, I am increasingly convinced that this revolution is a durable one. It appears on course to impact every nook and cranny of ECE departments, globally. It is increasingly plausible that we all may need re-imagine our discipline, and the structure of our departments. No kidding.

Holiday Wishes

As we head into the Holiday season, we at ECEDHA wish you and your family, and your extended (academic) family of students and postdocs, a restful holiday after all the hard work that we have all put in.

 

Badri Roysam, D.Sc., Fellow IEEE, Fellow AIMBE
Hugh Roy and Lillie Cranz Cullen University Professor
Chair, Electrical & Computer Engineering
University of Houston
Houston, Texas 77204-4005
Phone: 713-743-1773
Email: broysam@uh.edu