Xinhua News Agency, Tokyo, August 24th-China’s ambassador to Japan, Wu Jianghao, lodged a solemn protest with Japan’s Deputy Foreign Affairs Minister Masayoshi Okano on the 24th, regarding Japan’s launching of the Fukushima nuclear polluted water discharge into the sea.

　　Wu Jianghao said that today, Japan ignored strong doubts and opposition at home and abroad, shifted the risk of nuclear pollution to neighboring countries including China and the international community, put its own self-interest above the long-term well-being of people in the region and around the world, put the global marine environment and the health of all mankind as a dangerous bet, and insisted on officially launching the Fukushima nuclear pollution water discharge. China resolutely opposed this and once again lodged a solemn protest with Japan, demanding that Japan immediately stop this extremely selfish and irresponsible behavior.

　　Wu Jianghao pointed out that the Fukushima nuclear accident is one of the most serious nuclear accidents in the world so far, causing a large amount of radioactive materials to leak. There is no precedent in the world to deal with it by discharging the sea, and there is still a lack of technology that has been tested by science and practice and proved to be "safe and harmless". Without fully demonstrating the long-term safety and reliability of the existing sea-discharging technology, gaining extensive understanding from all stakeholders, and establishing a complete international supervision and compensation mechanism, Japan insists on forcibly starting sea-discharging, which contains huge risks and unpredictable hazards.

　　Wu Jianghao emphasized that the ocean is the blue home on which all mankind depends, and all countries have the obligation of international law to protect and preserve the marine environment. Japan has no right to decide to dump nuclear polluted water into the sea without authorization, and must bear full responsibility for the consequences caused by it. China seriously urges Japan to immediately stop discharging the sea, communicate honestly with its neighboring countries in a responsible manner, accept strict supervision by the international community, and effectively dispose of nuclear polluted water in a scientific, safe and transparent way. If Japan insists on doing it, it will bear long-term moral and legal responsibilities.

　　Wu Jianghao pointed out that’s exclusion from the sea has aroused strong concerns of Japanese consumers about food safety. China government has always adhered to the people first, and must resolutely respond to people’s concerns and safeguard the marine environment, food safety and public health. It is natural and necessary for the China government to announce a total suspension of the import of Japanese aquatic products. The responsibility for this situation lies entirely with the Japanese side, which should reflect on itself.

Original attention to frontier science and technology qubit

Mengchen Bowen is from Aofei Temple.

Quantum bit report | WeChat official account QbitAI

There was once a master student of Peking University who won eight papers in ACL of the International Summit at one time during his school days, two of which were one work, and also boarded a hot search in Zhihu.

After that hot search, this "big paper writer" seems to have gradually disappeared.

Today, she came back with AliceMind, a deep language model system of Ali Dharma Institute.

Luo Fuli, who is called "AI Loli" by the outside world, is responsible for the open source of seven models in AliceMind.

Her experience is a bit legendary.

I didn’t have much contact with computers before I went to college, but I stumbled into the computer major of Beijing Normal University by mistake.

When I first entered school, I jumped to the top one or two by hard work because I didn’t have the bottom of basic grades.

In my junior year, I entered the Language Computing Laboratory of Peking University as an intern, chose NLP as my research direction, taught myself Python within three months and submitted a top-level paper (not a work).

Bao Yan entered Peking University, and his master’s degree published more than 20 papers at international summit in two years.

However, unexpectedly, she did not choose to continue her doctoral studies, but joined the Ali Dharma Institute after graduation in 2020, hoping to do some practical research.

In the two years since she entered the industry, the number of papers she sent has obviously decreased.

When reading, the surrounding evaluation mechanism is very concerned about the number of your papers. But when it comes to industry, I don’t pursue quantity now, but mainly pursue whether this work is really valuable and influential in this field.

She led the development of cross-language pre-training model VECO in Dharma Institute, and became one of the eight models of AliceMind. This time, AliceMind opened the source collectively, and she provoked the beam.

Simple is the best.

Luo Fuli worked in the industry this year, compared with when she was in academia, her mentality has changed a lot:

When I was in school, I always tried to put forward a very complicated model, which everyone couldn’t understand and the paper reviewers couldn’t understand, but when I got to the industry, I would find that the model that was understandable and effective at a glance was the best.

This is also the thinking of the deep language model team of Dharma Institute where she works. The eight AliceMind models they created have successively reached the top of the six NLP authoritative lists of GLUE, CLUE, XTREME, VQA Challenge, DocVQA and MS MARCO.

The meaning of Alice in AliceMind is actually very simple, that is, Alibaba’s collection of encoder-decoders.

The models are as simple as this name, all of which are based on the actual business needs and are innovated and improved on the basis of Encoder-decoder.

StructBERT, a general language model, adds two new objective functions at word level and sentence level on the basis of BERT, which is equivalent to letting AI master the ability of "reading Chinese characters in sequence without affecting".

This is because the team found in Ali’s business that users often have incorrect grammar and word order when using scenes easily such as e-commerce and entertainment products.

This requires the language model to accurately understand and give correct expressions and responses when faced with words with disordered word order and irregular grammar.

AliceMind has just topped the multimodal authority list VQA Challenge 2021 again.

The competition task of VQA Challenge is similar to picture-looking question and answer. Given an image and natural language questions about the image, AI needs to provide accurate natural language answers.

In this regard, AliceMind’s multi-modal model StructVBERT, based on the general model StructBERT, introduces both text and image modes.

Using more efficient visual features and innovative cross-attention mechanism, joint modeling is carried out in a unified multimodal semantic space.

In addition to cross-modal, VECO, a cross-language model led by Luo Fuli, was also hired by ACL2021.

The cross-attention mechanism is also introduced into VECO, which changes the instability of automatic modeling of cross-language information in the hidden layer in the past, but completes it explicitly.

Another innovation of VECO is to fully learn the tasks for language understanding (NLU) and generation (NLG) during the pre-training, and let them learn from each other and improve each other.

Now Luo Fuli reviews the work of VECO, and there are also some regrets:

If I was in school two years ago, I would think it was so simple. I could add a lot of fancy skills. However, in order to consider the universality of the architecture in different business scenarios, the industry has to sacrifice some complicated and interesting model designs.

PALM, the generative language model in AliceMind, changes the pre-training goal from reconstructing the input text to predicting the subsequent text.

Such a change makes the model understand the input text more deeply, and achieves better results in questions and answers generation, text retelling, reply generation, text summarization and other tasks.

There are also StructuralLM, machine reading comprehension model UED and knowledge-driven language model LatticeBERT, all of which have made obvious advantages in their respective fields.

In addition to the seven models of this open source, AliceMind also includes a very large-scale unified model PLUG for Chinese understanding and generation.

The models in AliceMind seem to have a common feature, that is, they are good at "cross-border".

From cross-language and cross-modal to the unification of language understanding and generation, different inputs are modeled in a larger coding space based on Transformer architecture.

Luo Fuli added:

AliceMind’s solution of using Transformer as a unified model architecture is mature, but to achieve better "cross-border", the next direction of efforts is to solve the problem of deep integration and matching of different types or granularity inputs.

Expanding models with diverse abilities from the basic model and combining them in practical business make AliceMind the most comprehensive deep language model system in the industry.

So where did AliceMind go?

Landing is a systematic project.

AliceMind has been launched on the NLP platform inside Ali, which can be used by different departments.

Demo is also provided on the official website, such as this language generation module based on PLUG model.

Enter excerpts from Dream of Red Mansions:

You can generate a continuation:

And there are dozens of Demo like this for everyone to try out.

However, these services on official website are not all supported by AliceMind, and many of them are just small models inspired by this system.

So where are the cores of this open source currently working?

The most widely used is e-commerce.

In particular, Alibaba International Business Unit (ICBU) or a department with cross-border e-commerce business like AliExpress is the direct beneficiary of the multilingual model VECO.

VECO is one of the eight models in AliceMind system, which is used for multi-language understanding and cross-language text embedding and classification, and has mastered more than 100 languages.

Ali’s internal translation platform based on AliceMind is called about 1 billion times a day, creating hundreds of millions of dollars of international cross-border trade and other international business business value.

As Huang Songfang, the head of the deep language model team of Dharma Institute, said, "Language model landing is a systematic project":

The language model is completed on the platform from training, fine-tuning to distillation and compression, to the whole deployment and online. After online, it is connected with the business system and can be directly embedded in their business logic and business system.

AliceMind’s contribution is also found in the more familiar Taobao photo-taking and Tmall Elf smart speakers.

At present, AliceMind has landed in dozens of core businesses in Ali, with an average of 5 billion calls per day and more than 200 active scenes.

In addition to Ali, AliceMind has also contributed a lot to the medical field, especially cancer treatment.

As a deep learning language model system with autonomous learning ability, AliceMind will have a rearrangement mechanism when it is applied to search engines.

Aiming at a specific kind of medical literature, AliceMind will make a series of relevant texts after rough arrangement, and then combine the information such as article type and citation map again to rearrange them constantly.

At the same time, the extracted information is fused with the known structured knowledge to build a knowledge system, and finally the highest quality clinical literature is obtained.

In the recent international evaluation of precision medicine attended by 16 world-renowned teams, with this precision medicine search engine, Ali team won the first place in two clinical evidence quality evaluations:

Such a high-precision professional medical search engine can provide high-quality clinical decision-making assistance for clinicians when treating diseases.

AliceMind also appears in the legal field.

The Higher People’s Court of Zhejiang Province cooperated with Dharma Institute to realize the intelligent trial system of the whole process from filing a case to the generation of judgment documents.

In this pilot unit, AI’s sharing of the workload of judges has increased the rate of sentencing in court to 90%, and the time for closing the case has also been shortened from an average of 40 days to 50 minutes.

At present, the daily cumulative call volume of AliNLP platform based on AliceMind exceeds trillions, and it is used by more than 1,000 business parties every day.

E-commerce, education, medical care, energy, communication, law, content search, urban brain … More and more fields have become more convenient and smarter with the participation of AliceMind.

What to do after open source?

At present, the pre-training language model is very popular in NLP field and the whole learning field, and the model of super-large-scale parameters has become a trend.

In this regard, Huang Songfang, the head of the deep language model team of Dharma Institute and the general manager of AliceMind, said:

In fact, our side will not blindly pursue greatness, but will place great emphasis on its landing.

From research and development to practical application, a language model can’t be done by an enterprise.

It is also necessary for the developers of the whole community to participate, so that it is possible to apply the formula algorithm in academic papers to everyone’s convenience.

Dharma Institute hopes that through open source, it can lower the threshold of research and innovative application in the industry and make the language AI enter the era of big industry.

Next, AliceMind intends to strengthen cooperation with interdisciplinary units such as linguistics and neuroscience, and expand the language AI into larger applications.

Open source address:

https://github.com/alibaba/AliceMind

AliceMind official website:

https://nlp.aliyun.com/portal#/alice

Related papers:

General pre-training model StructBERT;

https://arxiv.org/abs/1908.04577

Multilingual pre-training model VECO;

https://arxiv.org/abs/2010.16046

Production pre-training model PALM;

https://arxiv.org/abs/2004.07159

Multimodal pre-training model E2E-VLP;

https://arxiv.org/abs/2106.01804

Structured pre-training model StructuralLM;

https://arxiv.org/abs/2105.11210

Reading comprehension model:

https://ojs.aaai.org/index.php/AAAI/article/view/16584

Lattice-BERT, a pre-training model integrating knowledge;

https://arxiv.org/abs/2104.07204

Reference link:

[1][2]— End—

This article is the original content of the signing account [Quantum Bit] of Netease News Netease Special Content Incentive Plan, and it is forbidden to reprint it at will without the authorization of the account.

Original title: "AI Loli" came back after Peking University was born in 1995. She won eight top papers at a time, and now she is open to seven NLP models in Dharma Institute. "

Read the original text