Click the card below to follow the " CVer " official account
AI/CV heavy dry goods, delivered in the first time
Click to enter —> [Computer Vision and Paper Submission] Exchange Group
Reprinted from: Heart of the Machine | Editors: Du Wei, Xiao Zhou
Compared with the 15 outstanding papers selected last year , the number of award-winning papers in ICML 2023 was greatly reduced, only 6.
The full name of ICML is International Conference on Machine Learning, which is organized by the International Machine Learning Society (IMLS) and is the top conference in the field of computer artificial intelligence. This year's ICML conference is the 40th and will be held at the Hawaii Convention Center from July 23 to 29, 2023.
This year, ICML received a total of 6538 submissions, of which 1827 were accepted, an acceptance rate of approximately 27.9%. Compared with 2022, the number of submitted and accepted papers and the acceptance rate have all increased (5,630 submissions, 1,117 short papers, 118 long papers, and an acceptance rate of 21.9%).
ICML officials state that each submission is reviewed by an Area Chair and a Senior Area Chair to ensure that each submission is properly evaluated.
Today, ICML officially released the winning papers of the Outstanding Paper Award.
Six Outstanding Paper Awards
A total of 6 outstanding papers were judged at this session, and the research covered no learning rate, watermarking for LLM, unseen domain generalization, near-optimal strategies for incomplete information zero-sum games, Bayesian learning of MCMC and frequency order Sri Lanka design principles and other topics.
论文 1：Learning-Rate-Free Learning by D-Adaptation
Organization: Meta AI, Inria Sierra
By Aaron Defazio, Konstantin Mishchenko (currently Research Scientists at Samsung AI Center)
Paper address: https://openreview.net/forum?id=GXZ6cT5cvY
This research aims to obtain the optimal bound without learning rate for non-smooth stochastic convex optimization. The proposed method overcomes the limitation of traditional learning rate selection when optimizing such problems, and makes a valuable and practical contribution to the field of optimization.
The study also proposes SGD and Adam variants of the new method, which will be used for large-scale CV and NLP problems.
论文 2：A Watermark for Large Language Models
Institution: University of Maryland
Author: John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein
Paper address: https://openreview.net/forum?id=aX8ig9X2a7
Paper brief: This paper presents a method for watermarking the output of large language models -- embedding signals into the generated text that are invisible to humans but detectable by algorithms. Generate watermarks without retraining language models, and detect watermarks without accessing APIs or parameters.
For watermark detection, the paper also proposes a statistical test method with interpretable p-values, and an information-theoretic framework for analyzing watermark sensitivity. The method proposed in this study is simple and novel, and provides thorough theoretical analysis and solid experiments. Given the serious challenges of detecting and generating text from large language models (LLMs), this research could have a significant impact on the machine learning community.
论文 3：Generalization on the Unseen, Logic Reasoning and Degree Curriculum
Agencies: EPFL, Apple
作者：Emmanuel Abbe、Samy Bengio、Aryo Lotfi、Kevin Rizk
Paper address: https://openreview.net/forum?id=3dqwXb1te4
Abstract: This paper makes important progress in the learning of Boolean functions, especially Generalization on the Unseen (GOTU), a challenging out-of-distribution generalization problem. The paper delves into this topic and proposes a well-structured approach backed by ample theoretical analysis and extensive experiments. In addition, the paper outlines a key research direction in the field of deep neural networks.
Specifically, the researchers explore the problem of function learning with holdout, where partial distribution support is rarely or never seen in training, and use Boolean objective functions to capture various inference tasks (such as arithmetic, decision trees, and logic circuits) The discrete and combined properties of .
Finally, the researchers gave their own explanation for the length generalization problem and proposed a curriculum-based learning algorithm called "Degree-Curriculum", which learns monomials more efficiently by adding support. The algorithm looks like this:
论文 4：Adapting to game trees in zero-sum imperfect information games
Institutions: CREST, ENS Lyon, Omron Sinic X, Deepmind, etc.
作地：Côme Fiegel、Pierre MENARD、Tadashi Kozuno、Remi Munos、Vianney Perchet、Michal Valko
Paper address: https://openreview.net/forum?id=O1j4uFuSVW
Abstract: This paper introduces near-optimal strategies for zero-sum games with incomplete information. The researchers established a novel lower bound and proposed two algorithms—balanced FTRL and adaptive FTRL. These contributions have greatly advanced the field of optimization for games with incomplete information. A number of experiments in the paper confirmed these statements and provided sufficient support for the research results.
论文 5：Self-Repellent Random Walks on General Graphs - Achieving Minimal Sampling Variance via Nonlinear Markov Chains
Organizations: IQVIA Inc, North Carolina State University
Authors: Vishwaraj Doshi, Jie Hu, Do Young Eun
Paper address: https://openreview.net/forum?id=450iImFM4U
Abstract: This paper addresses a challenging set of open problems proposing Markov Chain Monte Carlo (MCMC) with self-exclusive random walks. Given any Markov chain corresponding to the target probability distribution, this self-exclusive random walk (SRRW) is less likely to transition to nodes that were highly visited in the past, and more likely to transition to rarely visited nodes.
This method goes beyond traditional backtracking-free methods and paves the way for new research directions in MCMC sampling. Researchers have made original and significant contributions to MCMC research, and it is worth mentioning that the process can be rigorously analyzed and proved. The results are also very comprehensive and convincing.
One of the paper's authors, Jie Hu, is a doctoral student at NC State, and he earned his undergraduate degree at Wuhan University of Technology and his master's degree at Northwestern University.
论文 6：Bayesian Design Principles for Frequentist Sequential Learning
Institution: Columbia University
Author: Yunbei Xu, Assaf Zeevi
Paper address: https://openreview.net/forum?id=tRhQsHnoFw
About the paper: This paper examines the very general problem of designing bandit and other sequential decision strategies. The paper proposes a way to constrain the regret of any policy using a new quantity called the algorithmic information ratio, and derives a method to optimize this constraint. This constraint is stricter than earlier similar information-theoretic quantities, and these methods perform well in both stochastic and adversarial bandit settings, achieving global optima.
Of particular interest is that this paper may open the door to entirely new exploration-exploitation strategies besides the well-known Thompson Sampling and UCB for bandit. In fact, this principle is very promising if extended to the field of reinforcement learning. The paper received unanimous and strong support from expert reviewers.
The first author of the thesis, Yunbei Xu, Ph.D., Columbia Business School, is currently a postdoctoral researcher at MIT, and will start working as an assistant professor at NUS in fall 2024. He graduated from the Department of Mathematics, Peking University.
Reference link: https://icml.cc/Conferences/2023/Awards
*Cover image source: https://twitter.com/icmlconf/status/1683917404689305600?s=20
Click to enter —> [Computer Vision and Paper Submission] Exchange Group
ICCV/CVPR 2023 Paper and Code Download
Background reply: CVPR2023, you can download the collection of CVPR 2023 papers and code open source papers
后台回复：ICCV2023，即可下载ICCV 2023论文和代码开源的论文合集目标检测和Transformer交流群成立
扫描下方二维码，或者添加微信：CVer333，即可添加CVer小助手微信，便可申请加入CVer-目标检测或者Transformer 微信交流群。另外其他垂直方向已涵盖：目标检测、图像分割、目标跟踪、人脸检测&识别、OCR、姿态估计、超分辨率、SLAM、医疗影像、Re-ID、GAN、NAS、深度估计、自动驾驶、强化学习、车道线检测、模型剪枝&压缩、去噪、去雾、去雨、风格迁移、遥感图像、行为识别、视频理解、图像融合、图像检索、论文投稿&交流、PyTorch、TensorFlow和Transformer等。
一定要备注：研究方向+地点+学校/公司+昵称（如目标检测或者Transformer+上海+上交+卡卡），根据格式备注，可更快被通过且邀请进群
▲扫码或加微信号: CVer333，进交流群
CVer计算机视觉（知识星球）来了！想要了解最新最快最好的CV/DL/AI论文速递、优质实战项目、AI行业前沿、从入门到精通学习教程等资料，欢迎扫描下方二维码，加入CVer计算机视觉，已汇集数千人！
▲扫码进星球
▲点击上方卡片，关注CVer公众号It's not easy to organize, please like and watch