State of AI Report tracks transformers in critical infrastructure

人工智能状态报告跟踪关键基础设施中的变压器

Artificial intelligence and machine learning pioneers are rapidly expanding on techniques that were originally designed for natural language processing and translation to other domains, including critical infrastructure and the genetic language of life. This was reported in the 2021 edition of the State of AI Report by investors Nathan Benaich of Air Street Capital and Ian Hogarth, an angel investor.

人工智能和机器学习的先驱们正在迅速扩展最初设计用于自然语言处理和翻译到其他领域的技术，包括关键基础设施和生命的遗传语言。 Air Street Capital 的投资者 Nathan Benaich 和天使投资人 Ian Hogarth 在 2021 年版的 AI 状况报告中报告了这一点。

Started in 2018, their report aims to be a comprehensive survey of trends in research, talent, industry, and politics, with predictions mixed in. The authors are tracking “182 active AI unicorns totaling $1.3 trillion of combined enterprise value” and estimate that exits by AI companies have created $2.3 trillion in enterprise value since 2010.

他们的报告始于 2018 年，旨在对研究、人才、行业和政治的趋势进行全面调查，并加入预测。作者正在追踪“182 家活跃的人工智能独角兽，总企业价值总计 1.3 万亿美元”，并估计退出 自 2010 年以来，人工智能公司创造了 2.3 万亿美元的企业价值。

One of their 2020 predictions was that we would see the attention-based transformers architecture for machine learning models branch out from natural language processing to computer vision applications. Google made that come true with its vision transformer, known as ViT. The approach has also shown success with audio and 3D point cloud models and shows potential to grow as a general-purpose modeling tool. Transformers have also demonstrated superior performance at predicting chemical reactions, for example — the UK’s National Grid utility significantly halved the error in its forecasts for electricity demand using a type of transformer.

他们对 2020 年的预测之一是，我们将看到用于机器学习模型的基于注意力的转换器架构从自然语言处理扩展到计算机视觉应用。 谷歌通过其名为 ViT 的视觉转换器实现了这一目标。 该方法还在音频和 3D 点云模型方面取得了成功，并显示出作为通用建模工具的潜力。 例如，变压器在预测化学反应方面也表现出卓越的性能——英国国家电网公用事业公司使用一种变压器将其对电力需求的预测误差显着降低了一半。

Introduced in a 2017 paper, “Attention Is All You Need,” Transformers take an “attention-based” approach to limit the computing power required for analysis, for example by focusing attention on one word at a time in a sentence rather than letting the model grow exponentially in complexity with each additional word. The Perceivers’ architecture from DeepMind, the deep learning business unit of Google’s parent company Alphabet, is another variation on the attention concept that has shown strong results with inputs and outputs of various sizes, according to the report.

在 2017 年的一篇论文“Attention Is All You Need”中介绍，Transformers 采用“基于注意力”的方法来限制分析所需的计算能力，例如将注意力一次集中在句子中的一个词上，而不是让 每增加一个单词，模型的复杂性就会呈指数级增长。 报告称，来自谷歌母公司 Alphabet 的深度学习业务部门 DeepMind 的 Perceivers 架构是注意力概念的另一种变体，它在各种规模的输入和输出中显示出强大的结果。

Amping up linguistic analysis

加强语言分析

Making sense of human language is one of the toughest problems in AI, but lessons learned from linguistic analysis turn out to pay off in other realms such as computational biology and drug discovery.

理解人类语言是人工智能中最棘手的问题之一，但从语言分析中汲取的经验教训在计算生物学和药物发现等其他领域得到了回报。

As one example, researchers are “learning the language of COVID-19” for a grammatical understanding of its genetics, showing the potential to identify future possible mutations that could produce the next threat akin to the Delta variant. This raises the possibility that future vaccines and treatments could be prepared to address those variants before they emerge, the authors suggest.

例如，研究人员正在“学习 COVID-19 的语言”，以从语法上理解其遗传学，从而显示出识别未来可能的突变的潜力，这些突变可能会产生类似于 Delta 变体的下一个威胁。 作者建议，这增加了未来疫苗和治疗方法可以在这些变异出现之前准备好解决这些变异的可能性。

Investor dollars are following for AI-first biotech and drug discovery firms, most notably with the October IPO of Britain’s Exscientia at a valuation of over $3 billion. Recursion Pharmaceuticals of Utah raised $436 million in an April IPO.

投资者资金紧随人工智能优先的生物技术和药物发现公司，最引人注目的是英国 Exscientia 的 10 月首次公开募股，估值超过 30 亿美元。 犹他州 Recursion Pharmaceuticals 在 4 月份的 IPO 中筹集了 4.36 亿美元。

Yet, despite the promising outlook for AI in medicine, the report’s authors also note that “despite a loud call to arms and many willing participants, the ML community has had surprisingly little positive impact against COVID-19. One of the most popular problems – diagnosing coronavirus pathology from chest X-ray or chest computed tomography images using computer vision – has been a universal clinical failure.” They also caution against overstated claims about the applications of AI to domains such as radiology, noting that one study found 94% of AI systems designed to improve breast cancer screening are less accurate than the original radiologist.

然而，尽管人工智能在医学领域的前景充满希望，但该报告的作者还指出，“尽管大声呼吁武装和许多愿意的参与者，但机器学习社区对 COVID-19 的积极影响却出人意料地小。 最流行的问题之一——使用计算机视觉从胸部 X 光或胸部计算机断层扫描图像诊断冠状病毒病理学——一直是普遍的临床失败。” 他们还告诫不要夸大人工智能在放射学等领域的应用，并指出一项研究发现，94% 的旨在改善乳腺癌筛查的人工智能系统不如最初的放射科医生准确。

Global rush for large language models and critical infrastructure

大型语言模型和关键基础设施的全球热潮

Large language models (LMMs) are proving so important that they “have become ‘nationalized,’ where every country wants their own LMM,” according to the report. These are models that attempt to understand all the words in a given language, and the largest to date is the Chinese model, Wudao, with 1.75 trillion parameters. In general, China has emerged as the world leader in academic AI research – at the same time that U.S. universities are suffering a significant “brain drain,” according to the report.

报告称，大型语言模型 (LMM) 被证明如此重要，以至于它们“已经‘国有化’，每个国家都想要自己的 LMM”。 这些是试图理解给定语言中所有单词的模型，迄今为止最大的是中文模型，五道，有 1.75 万亿个参数。 报告称，总体而言，中国已成为学术人工智能研究的世界领先者——与此同时，美国大学正在遭受严重的“人才流失”。

In addition to being one of the most important fronts in AI research, linguistic understanding is one of the most fraught. The machine understanding that emerges often turns out to reveal racist and sexist biases that might reflect an accurate understanding of human nature – but not one we want to promote. One of the recent scandals in the field was Google’s firing of Timnit Gebru, an AI researcher who says she was cut loose after raising ethical objections to the way Google was using LMMs. Alphabet/Google also quashed an effort by DeepMind to be spun off as a nonprofit research group, according to the report.

除了作为人工智能研究中最重要的前沿之一之外，语言理解也是最令人担忧的领域之一。 出现的机器理解往往会揭示种族主义和性别歧视偏见，这些偏见可能反映了对人性的准确理解——但不是我们想要推广的。 该领域最近的丑闻之一是谷歌解雇了人工智能研究员 Timnit Gebru，她说她在对谷歌使用 LMM 的方式提出道德异议后被解雇。 报告称，Alphabet/Google 还取消了 DeepMind 将其作为一个非营利性研究小组剥离出来的努力。

The report highlights these incidents in the context of a broader discussion of AI safety – the challenge of ensuring that AI progress is kept in alignment with human wellbeing – including worries over military applications like autonomous war-fighting machines.

该报告在更广泛地讨论人工智能安全的背景下强调了这些事件——确保人工智能进步与人类福祉保持一致的挑战——包括对自主作战机器等军事应用的担忧。

These are just some of the many highlights included in the report, which was published as a 168-screen Google Slides deck. Among their predictions for the coming year are that this sector will likely see a wave of consolidation among AI semiconductor companies and the emergence of a new research company focused on artificial general intelligence (the most ambitious branch of AI) with a focus on a vertical like life sciences, critical infrastructure or developer tools.

这些只是报告中包含的众多亮点中的一部分，该报告以 168 屏幕的 Google 幻灯片组的形式发布。 他们对来年的预测包括，该行业可能会出现一波人工智能半导体公司之间的整合浪潮，以及一家专注于人工智能（人工智能最雄心勃勃的分支）的新研究公司的出现，专注于垂直领域 生命科学、关键基础设施或开发工具。