Who is Jakob Uszkoreit?

Jakob Uszkoreit is an ML and NLP researcher and computer scientist with a particular focus on language translation. Uszkoreit is also one of the co-authors of the seminal paper on transformer architecture entitled Attention Is All You Need.

Table of Contents

Education and early career

Uszkoreit earned his Master’s degree in computer science and mathematics from Technische Universität Berlin in 2007. To support himself as a student, he worked as a freelance software developer for a production company.

Uszkoreit interned at Google Research in 2006 and 2007. According to his LinkedIn profile, he was tasked with distributing “clustering algorithms for applications in machine translation, language modeling and other natural language understanding systems.”

Post-university, he worked at Acrolinx as a software engineer for around twelve months. Acrolinx is a provider of enterprise content governance for human and AI-generated content.

Google Brain

Uszkoreit joined Google Brain’s Berlin facility in March 2008 and worked on early versions of Google Translate.

He was motivated to abort his Ph.D. and return to Google because the company was one of the few places where one could work on an ML project that was intellectually stimulating while also being challenging and exciting.

Later, he managed the team responsible for the natural language query understanding system that underpins Google Assistant and several other products.

In subsequent years, Uszkoreit led a team in Google Machine Intelligence and worked on large-scale deep learning for natural language understanding. This work also had implications for Google Assistant and so forth.

Uszkoreit spent over 13 years at Google Brain and co-authored several influential papers on self-attention and transformers for music and image recognition, among many other topics.

When he left the company in July 2021, he was part of the so-called “brain drain” of AI talent who had resigned from big tech firms to pursue their own interests.

Inceptive

That same month, Uszkoreit co-founded the biotech company Inceptive with computational biochemist and Stanford associate professor Rhiju Das.

In essence, Inceptive incorporates AI into RNA biology to develop software that can execute complex functions in a biological system. This software can be used to develop novel and broadly accessible therapeutics as well as biotechnologies once believed impossible.

On his reasons for starting the company, Uszkoreit explained in an interview with Andreesen Horowitz that “For quite a while now, biology struck me as such a problem where it doesn’t seem inconceivable that there are bounds to how far we can go in terms of, say, drug development and direct design with traditional biology as the backbone…”

Uszkoreit then mentioned that deep learning at scale, in particular, was appropriate in this context. This would enable the company to observe life at a sufficient scale with enough fidelity but also much less specificity whilst focusing on the problem that needs to be solved.

Key takeaways

Jakob Uszkoreit is a deep learning researcher and computer scientist with a particular focus on language translation. Uszkoreit is also one of the co-authors of the seminal paper on the transformer architecture entitled Attention Is All You Need.
Uszkoreit joined Google Brain in March 2008 and worked on early versions of Google Translate. He was motivated to abort his Ph.D. and return to Google because the company was one of the few places where one could work on exciting but challenging machine learning projects.
Uszkoreit worked at Google Brain for 13 years before founding Inceptive with computational biochemist Rhiju Das. Inceptive develops software that can execute complex functions in a biological system and facilitate the creation of new medicines.

Timeline

Jakob Uszkoreit’s Background: Jakob Uszkoreit is a computer scientist and machine learning (ML) researcher with a specialization in natural language processing (NLP) and language translation. He co-authored the influential paper on transformer architecture titled “Attention Is All You Need.”
Education and Early Career: Uszkoreit earned his Master’s degree in computer science and mathematics from Technische Universität Berlin in 2007. He interned at Google Research and worked as a software engineer at Acrolinx.
Google Brain: In March 2008, Uszkoreit joined Google Brain’s Berlin facility, where he contributed to early versions of Google Translate. Over 13 years at Google Brain, he worked on natural language query understanding systems for Google Assistant and large-scale deep learning for various NLP applications.
Brain Drain and Departure from Google: In July 2021, Uszkoreit was part of the “brain drain” of AI talent leaving big tech firms to pursue their interests. He left Google Brain to explore new opportunities.
Co-Founder of Inceptive: In the same month, Uszkoreit co-founded Inceptive, a biotech company that incorporates AI into RNA biology. The company aims to develop software that can execute complex functions in biological systems, leading to novel therapeutics and biotechnologies.
Vision for Inceptive: Uszkoreit’s goal with Inceptive is to use deep learning at scale to address complex biological problems, such as drug development and direct design. The company’s approach focuses on solving critical problems in the field of biology using AI technology.
Contribution to AI Research: Throughout his career, Uszkoreit has contributed to several influential papers on self-attention and transformers, advancing the fields of NLP and deep learning.