Research

I am broadly interested in Natural Language Processing (especially Text Generation) and Machine Learning Security & Privacy.

Google Scholar, Semantic Scholar

Selected Papers

* denotes equal contributions / lead authors.

• Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
  Satyapriya Krishna, Kalpesh Krishna*, Anhad Mohananey*, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, Manaal Faruqui
  arXiv // dataset
  arXiv 2024

• Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation
  Tu Vu*, Kalpesh Krishna*, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, Yun-Hsuan Sung
  EMNLP 2024
  arXiv // tweet

• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
  Gemini Team, Google (Kalpesh Krishna is a Core Contributor)
  technical report // tweet
  arXiv 2024

• Gemini: a family of highly capable multimodal models
  Gemini Team, Google (Kalpesh Krishna is a Core Contributor for Bard/Gemini Advanced)
  technical report, see Section 6 onwards for Gemini Advanced work // blogpost
  arXiv 2024

• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
  Sewon Min*, Kalpesh Krishna*, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi
  EMNLP 2023
  PIP package // code // tweet

• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
  Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, Mohit Iyyer
  NeurIPS 2023
  model + data + code // tweet

• LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
  Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, Kyle Lo
  EACL 2023 (Outstanding Paper Award)
  data + code // tweet // video // external article

• RankGen: Improving Text Generation with Large Ranking Models
  Kalpesh Krishna, Yapei Chang, John Wieting, Mohit Iyyer
  EMNLP 2022
  code // tweet // video // external summary

• Hurdles to Progress in Long-form Question Answering
  Kalpesh Krishna, Aurko Roy, Mohit Iyyer
  NAACL 2021
  blog // code // project page // tweet

• Reformulating Unsupervised Style Transfer as Paraphrase Generation
  Kalpesh Krishna, John Wieting, Mohit Iyyer
  EMNLP 2020
  project page (with demo) // external video

• Thieves on Sesame Street! Model Extraction of BERT-based APIs
  Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer
  ICLR 2020
  blog // project page

Other Papers

• PostMark: A Robust Blackbox Watermark for Large Language Models
  Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Wieting, Mohit Iyyer
  EMNLP 2024
  arXiv // tweet

• GEE! Grammar Error Explanation with Large Language Models
  Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Kevin Gimpel, and Mohit Iyyer
  NAACL 2024 (Findings)

• On the Risks of Stealing the Decoding Algorithms of Language Models
  Ali Naseh, Kalpesh Krishna, Mohit Iyyer, Amir Houmansadr
  CCS 2023 (Distinguished Paper Award)

• ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution
  Ankita Gupta, Marzena Karpinska, Wenlong Zhao, Kalpesh Krishna, Jack Merullo, Luke Yeh, Mohit Iyyer, Brendan O’Connor
  EACL 2023 (Findings)
  code

• NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
  Kaustubh D. Dhole and others
  NEJLT 2023
  code

• SLING: Sino Linguistic Evaluation of Large Language Models
  Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Mohit Iyyer
  EMNLP 2022
  code

• Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature
  Katherine Thai*, Marzena Karpinska*, Kalpesh Krishna, Bill Ray, Moira Inghilleri, John Wieting and Mohit Iyyer
  EMNLP 2022
  code + data

• Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
  Kalpesh Krishna, Deepak Nathani, Xavier Garcia, Bidisha Samanta, Partha Talukdar
  ACL 2022
  data // project page

• RELiC: Retrieving Evidence for Literary Claims
  Katherine Thai, Yapei Chang, Kalpesh Krishna, Mohit Iyyer
  ACL 2022
  code + data // project page

• User and Technical Perspectives of Controllable Code Generation
  Stephanie Houde, Vignesh Radhakrishna, Praneeth Reddy, Juie Darwade, Haoran Hu, Kalpesh Krishna, Mayank Agarwal, Kartik Talamadupula, Justin D. Weisz
  NeurIPS HCAI workshop 2022 (short paper)

• Do Long-Range Language Models Actually Use Long-Range Context?
  Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke and Mohit Iyyer
  EMNLP 2021

• Weakly-Supervised Open-Retrieval Conversational Question Answering
  Chen Qu, Liu Yang, Cen Chen, W. Bruce Croft, Kalpesh Krishna and Mohit Iyyer
  ECIR 2021

• Long Document Summarization in a Low Resource Setting using Pretrained Language Models
  Ahsaas Bajaj*, Pavitra Dangati*, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das and Andrew McCallum
  ACL Student Research Workshop (SRW) 2021

• An Analysis of Frame-skipping in Reinforcement Learning
  Shivaram Kalyanakrishnan, Siddharth Aravindan, Vishwajeet Bagdawat, Varun Bhatt, Harshith Goka, Archit Gupta, Kalpesh Krishna, Vihari Piratla
  arXiv 2021

• SunPy: A Python package for Solar Physics
  Stuart J. Mumford and others
  JOSS 2020

• Generating Question-Answer Hierarchies
  Kalpesh Krishna, Mohit Iyyer
  ACL 2019
  project page (demo + code + data) // poster // external blog

• Syntactically Supervised Transformers for Faster Neural Machine Translation
  Nader Akoury, Kalpesh Krishna, Mohit Iyyer
  ACL 2019
  code // poster

• Trick or TReAT: Thematic Reinforcement for Artistic Typography
  Purva Tendulkar, Kalpesh Krishna, Ramprasaath R. Selvaraju, Devi Parikh
  ICCC 2019 (oral presentation, Best Presentation Award)
  code // slides // video // demo

• Revisiting the Importance of Encoding Logic Rules in Sentiment Classification
  Kalpesh Krishna, Preethi Jyothi, Mohit Iyyer
  EMNLP 2018 (oral presentation, short paper)
  code + data // slides // video

• Hierarchical Multitask Learning for CTC-based Speech Recognition
  Kalpesh Krishna, Shubham Toshniwal, Karen Livescu
  [external video]
  arXiv 2018

• A Study of All-Convolutional Encoders for Connectionist Temporal Classification
  Kalpesh Krishna, Liang Lu, Kevin Gimpel, Karen Livescu
  ICASSP 2018 (Awarded SPS Travel Grant)
  [poster]

Thesis

Undergraduate Thesis - Constraint Driven Learning
(under guidance of Prof. Preethi Jyothi)
IIT Bombay (2017-2018)