October 2019
Letter from the Editor
Artificial Intelligence (AI) Poised to Surge Forward with $124M Investment
By Badri Roysam, University of Houston
Dear ECEDHA family and friends,
The recent $124M investment announced by the NSF’s National Artificial Intelligence (AI) Research Institutes call for proposals from the NSF in six different thematic areas (Trustworthy AI, Foundations of Machine Learning, AI-Driven Innovation in Agriculture and the Food System, AI-Augmented Learning, AI for Accelerating Molecular Synthesis and Manufacturing, and AI for Discovery in Physics), in collaboration with multiple other federal agencies is the much awaited stimulus to the ECE and EECS research and scholarship enterprise.
Many questions arise: What is the best course of action for ECE Chairs? Does my work even relate to AI? Can AI have an impact on my field? What is it to begin with? How can I engage productively with this freshly reinvigorated field?
We start by recognizing that ECE faculty, together with our colleagues in Computer Science, are exceptionally well positioned to participate in this major initiative, but we have to choose to do so. This initiative can touch diverse parts of our departments, ranging from our machine learning and signal processing groups, to our computer engineering and materials science groups), and teaching. Importantly, this initiative provides a broad range of opportunities for institutions of all types and sizes to participate in this ongoing revolution, under the umbrella of the updated 2019 National Artificial Intelligence Research and Development Strategic Plan. We can choose to “drive our departmental cars” by aiming for an AI Institute (an endeavor that is comparable to a $4M/year NSF ERC, suitable for larger well-established departments), ride shotgun (by teaming with another larger institution), or ride in the back seat (by submitting a $500K planning grant for an AI Institute), depending upon where our department is positioned in this newly resurgent field of AI. There is room for everyone.
As a quick memory jogger, the field of AI was founded as a discipline in 1956, and since then, has experienced many cycles of breakthroughs creating excitement and bubble-like expectations that drove investments, all followed by failures, disappointments, and a flight of investments and talent that were akin to deep winters. Through all these cycles, the field quietly inched forward, without fanfare, mostly driven by academics in CS and ECE departments. This period ended abruptly in 2014 - 2015 when the error rate for computer vision systems dipped below the error rate for humans. This singular event transformed the field of computer vision completely. When I attended the recent IEEE Computer Vision and Pattern Recognition Conference (CVPR), the vast majority of discussion was about deep neural networks, whereas just a decade ago, there were diverse custom algorithms being talked about. Similar revolutions happened to the fields of speech and handwriting recognition. Indeed, these topics are now no longer considered to be AI. Traditionally, the field of AI was all about developing artificial human-like capabilities. Going forward, the field of AI is best thought of as "AI is whatever hasn't been done yet."
In the Gartner hype cycle as of this writing, various aspects of AI are tagged on both sides of the peak of expectations, including Generative Adversarial Networks, Augmented Intelligence, Explainable AI, Edge AI, Autonomous Driving Level 5, and Graph Analytics.
The current surge in interest is primarily driven by four basic factors.
- The success of deep neural networks1 at tasks that earlier generations of AI systems did not have enough success with to adopt into mainstream applications, for example, speech recognition, natural language understanding, handwriting recognition, and general computer vision.
- The maturation of “traditional AI” – the methodology that is based on extensive use of logic and knowledge representations2.
- The emergence of vast collections of annotated images, speech, and video that is freely available on the internet (cloud), has created datasets at scales that were unattainable conventionally (big data). Smart mobile phones were primarily responsible for much of the “Cambrian explosion” in terms of data availability on the internet. These data provide the all-important grist for the current revolution – training and validation datasets. Another perversely, and definitely non-automated revolution has been the emergence of low-cost crowdsourcing of manual annotation tasks via the Amazon Cloud.
- Finally, the emergence of computing accelerators, mainly GPU’s that are even built into our mobile phones, that provide the computational horsepower needed to achieve reasonable performance for machine learning inference tasks.
On a personal note, while much of the excitement has a sense of déjà vu, I am also acutely aware of the fact that a key aspect of AI – deep neural networks, completely changed my own research in ways that I could not have foreseen just a few years ago.
After building a productive career over many decades developing computer vision algorithms for biological images (cells and tissue), my students and I started to find that these newfangled neural networks were capable of beating our previous algorithms handily when they worked. We now understand why – they are much better at coping with biological image variability – the primary bane of my discipline, so we can no longer ignore them. This is clearly the time to embrace them. Happily, there is good funding available for doing this type of work, and my group was recently blessed with a NIH R01 grant to pursue these newfangled but promising deep neural networks for brain tissue studies.
On the other hand, we find that current neural networks are not (yet) the definitive answer. The primary problem is the difficulty of successfully training a well-performing deep neural network. The training process is slow even on our campus supercomputer. It is unpredictable in that there is no guarantee that after many hours of training you will get a good network. The process is opaque, so it is difficult to diagnose. Even after a network is trained successfully, they often lack essential properties that one would expect in an engineered system, especially interpretability, reliable measures of confidence/ uncertainty, robustness to adversarial/unseen examples, bounds on bias and errors, ability to adapt to new inputs, scalability, self-diagnosis ability, ability to cope with small and/or noisy training data points, demonstrable generalization ability, and importantly, the ability to interface effectively with traditional AI systems. Importantly, the training requires insight, skill, and extensive lots of trial-and-error hacking.
In a perverse way, the limitations of current deep networks is reassuring, since it is very clear that there is still much work left to do. We need to develop powerful and scalable theories, modeling methods, algorithms, and architectures that will transform this “wild west” field into a rigorous and well-understood discipline. In this regard, the NSF AI Institute programs will go down in history as an important watershed event that triggered comprehensive democratization of Artificial Intelligence, while at the same time, helping to fill in the important intellectual gaps. After all, that’s what ECE departments are very good at!
--
1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436-44.
2. Stuart Russell, Artificial Intelligence: A modern Approach, Pearson Education, 2015
Badri Roysam, D.Sc., Fellow IEEE, Fellow AIMBE
Hugh Roy and Lillie Cranz Cullen University Professor
Chair, Electrical & Computer Engineering
University of Houston
Houston, Texas 77204-4005
Phone: 713-743-1773
Email: broysam@uh.edu