The Hidden Challenges of Short Papers in Academic Conferences

The Hidden Challenges of Short Papers in Academic Conferences

In the fast-evolving landscape of academic publishing, short papers—typically capped at four pages—have emerged as a vital conduit for concise, innovative research in fields like machine learning and natural language processing. Yet, a growing chorus of researchers is questioning whether these condensed submissions are being fairly assessed. A recent Reddit thread on r/MachineLearning, initiated by user Efficient_Ad_6772, sparked a wide-ranging discussion about the inconsistent treatment of short papers at major venues such as the Association for Computational Linguistics (ACL) and the International Conference on Learning Representations (ICLR). The thread revealed not only frustration over misaligned reviewer expectations but also a broader concern: are short papers being judged by the same standards as their longer counterparts, undermining their intended purpose?

According to the ACL’s official CFP guidelines, short papers are explicitly designed to highlight "early-stage work, negative results, or focused contributions" that may not warrant a full-length submission. Reviewers are instructed to evaluate them on clarity, originality, and impact relative to their length—not on the depth or breadth expected of 8-10 page long papers. Yet, multiple contributors to the Reddit thread recounted experiences where reviewers demanded extensive experiments, theoretical proofs, or comparative analyses typically reserved for full papers. One researcher noted, "I was told my paper lacked "sufficient empirical validation," despite the submission guidelines explicitly stating that short papers are not required to include large-scale evaluations."

The issue is not isolated to ACL. ICLR, once a pioneer in offering a dedicated short paper track, discontinued it after the 2020 conference. While the conference organizers have not issued a formal statement, internal communications cited by several anonymous program committee members indicate that the decision stemmed from persistent reviewer confusion and a perceived imbalance in acceptance rates. "Reviewers didn’t understand the assignment," one former ICLR PC member told this outlet. "They kept applying long-paper criteria to short papers, leading to unfair rejections and declining submission quality."

Lexical definitions from authoritative sources like Cambridge Dictionary and Merriam-Webster reinforce the conceptual foundation of "short" as relating to "small in length, distance, or height." In academic contexts, this linguistic precision should translate into evaluation criteria: a short paper is not an incomplete paper—it is a distinct genre. Yet, as the Reddit thread underscores, this distinction is often lost in practice. Reviewers, often overburdened and unfamiliar with genre-specific rubrics, default to familiar benchmarks, penalizing brevity as a deficiency rather than a design feature.

The consequences are tangible. Early-career researchers and institutions with limited resources are disproportionately affected. Short papers were meant to democratize access to top-tier venues, offering a lower-barrier entry for novel ideas that don’t require months of additional experimentation. When those papers are held to long-paper standards, the opportunity for diversity in thought and method diminishes. Moreover, the removal of ICLR’s short paper track may signal a broader industry trend toward homogenization, where only exhaustive, resource-intensive studies are deemed worthy of publication.

Experts in scholarly communication argue that the solution lies in better reviewer training and clearer, enforceable guidelines. Some suggest adopting dual-reviewer systems—one focused on technical rigor, another on genre appropriateness—or mandatory orientation modules for reviewers assigned to short paper tracks. Without structural reforms, the promise of short papers as a vehicle for innovation risks becoming a rhetorical flourish rather than a functional pillar of academic discourse.

As the AI research community continues to expand, the integrity of its publication ecosystem depends on honoring the diversity of scholarly expression—not just in topic, but in form. Short papers are not lesser; they are different. And if the field is to remain inclusive and intellectually vibrant, it must learn to judge them accordingly.