Click on the card below to follow the " CVer " public account
AI/CV heavy-duty information, delivered as soon as possible
Click to enter -> [Target Detection and Transformer] communication group
Reprinted from: Xinzhiyuan | Editor: Taozi
[Introduction] The results of paper acceptance for the annual NeurIPS conference have been announced. This year’s acceptance rate is as high as 26.1%, and many excellent studies have been accepted.
NeurIPS 2023 admission results announced!
According to the official email, there were 12,343 submissions this year, with an acceptance rate of 26.1%. Compared with 2022 (25.6%), there is still an increase.
NeurIPS is one of the most prestigious AI academic conferences in the world. It is now the 37th conference and will be held in New Orleans, Louisiana, from December 10th to 16th.
Selected papers
Let’s take a look at which papers of famous people have been accepted.
Ma Yi team
The latest research results released by Professor Ma Yi's team in June were accepted by NeurIPS.
This study designed a white-box Transformer model CRATE that is fully interpretable by mathematics, and achieved performance close to ViT on the real-world data set ImageNet-1K.
Through extensive experiments, the researchers demonstrate that when adopting the white-box Transformer model CRATE, which is designed to explicitly model and pursue low-dimensional structure in the data distribution, whole- and part-level segmentation properties have emerged with minimally supervised training recipes.
Hierarchical fine-grained analysis shows that emergent properties strongly confirm the design mathematical capabilities of white-box networks. Our results suggest a path to designing white-box base models that are simultaneously high-performance and mathematically fully interpretable.
Professor Ma Yi also said that deep learning research will gradually shift from empirical design to theoretical guidance.
Paper address: https://arxiv.org/pdf/2306.01129.pdf
Currently, this project has 500+ stars on GitHub.
Tian Yuandong’s team: 3 papers accepted
In this year's NeurIPS 2023, three papers from Tian Yuandong's team were accepted.
Our Scan&Snap paper was accepted by NeurIPS 2023!
The paper analyzes the training dynamics of a single-layer Transformer (self-attention + linear decoder) and shows that it can be proven that attention only focuses on discriminative key tokens that appear multiple times with the query, thus becoming sparse over time. .
Follow-up research on multi-layer Transformer + non-linear MLP activation is in progress, let's see how the situation is different.
Paper address: https://arxiv.org/pdf/2305.16380.pdf
Another LANCER paper was also accepted by NeurIPS2023.
Many studies have attempted to learn a solution g to predict the solution to the objective function f, which can be a difficult combinatorial optimization problem to solve.
However, what if we jointly learn alternative cost functions for "f consists of g"? This avoids multiple calls to the solver and greatly speeds up the learning process.
This is LANCER, which achieves excellent results on multiple combinatorial problems, including real-world problems such as portfolio optimization.
Paper address: https://arxiv.org/pdf/2305.16380.pdf
The third article is the H2O paper accepted by NeurIPS 2023!
H2O shows that we can remove 80% of the tokens in the KV cache, but the perplexity of the next token prediction remains the same! This will significantly reduce your reasoning costs.
Paper address: https://arxiv.org/pdf/2306.14048.pdf
Oral: Mathematical theorem prover LeanDojo
More importantly, scholars from California Institute of Technology, NVIDIA, MIT and other institutions have built a theorem prover based on open source LLM and received it as NeurIPS Oral.
Here, researchers propose an open source platform, LeanDojo, which provides toolkits, benchmarks, and models to create an interactive environment for theorem proving for LLM.
Yang Kaiyu, the first author of the paper, once said that the formula proof is a computer program and its correctness can be verified.
Most importantly, this study opens a new way to address the shortcomings of LLM, both in factuality and illusion.
Because theorem proving is a form of code generation with strict evaluation, there is no room for illusions in the model.
Project address: https://leandojo.org/
The research team of Zhejiang University and Microsoft Research Asia released a large model collaboration system HuggingGPT and was accepted.
HuggingGPT uses ChatGPT as a controller to connect various AI models in the HuggingFace community to complete multi-modal complex tasks.
This means that you will have a kind of super magic. Through HuggingGPT, you can have multi-modal capabilities, including pictures, videos, and voices.
Paper address: https://arxiv.org/pdf/2303.17580.pdf
The SuTI proposed by the Google team was accepted.
This is a topic-driven text-image generator that replaces topic-specific fine-tuning with i-context learning.
Given a few demonstrations of a new theme, SUTI can instantly generate new renditions of that theme in different scenarios without the need for any theme-specific optimization.
SuTI is powered by “apprentice learning,” in which an apprentice model learns from data generated by a large number of subject-specific expert models.
Paper address: https://arxiv.org/pdf/2304.00186.pdf
A paper "Thought Cloning" published by former OpenAI scientist Jeff Clune this year was also accepted by NeurIPS 2023.
The study proposed that artificial intelligence can learn to "think" and "act" like humans by imitating humans.
Paper address: https://arxiv.org/pdf/2306.00323.pdf
The "Minecraft" special model STEVE-1 won a Spotlight in this year's NeurIPS.
This is a team from the University of Toronto and the Vector Artificial Intelligence Research Institute, which proposed the intelligent agent STEVE-1 in "Minecraft".
The unCLIP method already used in DALL·E 2 proved to be equally effective for creating sequential decision-making agents that follow instructions.
Paper address: https://sites.google.com/view/steve-1
Did you win an award?
Click to enter -> [Target Detection and Transformer] communication group
ICCV/CVPR 2023 paper and code download
Backstage reply: CVPR2023, you can download the collection of CVPR 2023 papers and code open source papers
后台回复：ICCV2023，即可下载ICCV 2023论文和代码开源的论文合集目标检测和Transformer交流群成立
扫描下方二维码，或者添加微信：CVer333，即可添加CVer小助手微信，便可申请加入CVer-目标检测或者Transformer 微信交流群。另外其他垂直方向已涵盖：目标检测、图像分割、目标跟踪、人脸检测&识别、OCR、姿态估计、超分辨率、SLAM、医疗影像、Re-ID、GAN、NAS、深度估计、自动驾驶、强化学习、车道线检测、模型剪枝&压缩、去噪、去雾、去雨、风格迁移、遥感图像、行为识别、视频理解、图像融合、图像检索、论文投稿&交流、PyTorch、TensorFlow和Transformer、NeRF等。
一定要备注：研究方向+地点+学校/公司+昵称（如目标检测或者Transformer+上海+上交+卡卡），根据格式备注，可更快被通过且邀请进群
▲扫码或加微信号: CVer333，进交流群
CVer计算机视觉（知识星球）来了！想要了解最新最快最好的CV/DL/AI论文速递、优质实战项目、AI行业前沿、从入门到精通学习教程等资料，欢迎扫描下方二维码，加入CVer计算机视觉，已汇集数千人！
▲扫码进星球
▲点击上方卡片，关注CVer公众号整理不易，请点赞和在看