Md Masudur Rahman

I am a Ph.D. Candidate in the department of Computer Science at the Purdue University. My advisor is Professor Yexiang Xue.

I am interested in Artificial Intelligence, Reinforcement Learning, and Robotics. My work focuses on designing and developing intelligent learning agents capable of making interpretable, critical decisions under uncertain conditions. Currently, my efforts are concentrated on the issue of generalization in reinforcement learning. I aim to develop algorithms that remain robust against the effects of confounders, with the ultimate goal of using these algorithms to solve real-world tasks.
I am exploring the use of foundational models as a source of prior knowledge in reinforcement learning algorithms. I am investigating how this can enable RL in various practical applications, such as surgical robotics and medical imaging, including the automation of burn injury treatment.

I completed my M.S. from the department of Computer Science at the University of Virginia in 2018. Before joining the University of Virginia, I worked as a Lecturer in the Computer Science and Engineering department at the BRAC University. I completed my B.Sc. in Computer Science and Engineering from Bangladesh University of Engineering and Technology (BUET) in 2013.

Email: rahman64@purdue.edu
[Home] [Publications] [Talks] [Teaching] [Mentoring] [Service] [CV] [Google Scholar]

Publications

[W3] Accelerating Policy Gradient by Estimating Value Function from Prior Computation in Deep Reinforcement Learning. [pdf]

Md Masudur Rahman and Yexiang Xue.

Workshop on Reincarnating Reinforcement Learning (RRL) at ICLR 2023. workshop

[C14] Bootstrap Advantage Estimation for Policy Optimization in Reinforcement Learning. [arXiv] [pdf] [video]

Md Masudur Rahman, Yexiang Xue.

ICMLA 2022, In Proceedings of the IEEE International Conference on Machine Learning and Applications (ICMLA), 2022. conference

[C13] Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning. [arXiv] [pdf][video]

Md Masudur Rahman, Yexiang Xue.

ECML-PKDD 2022, In Proceedings of the 2022 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2022. conference

[C12] ASAP: A Semi-Autonomous Precise robotic framework for remote surgery under delays.

Glebys Gonzalez, Mythra Balakuntala, Mridul Agarwal, Md Masudur Rahman, Thomas Low, Vaneet Aggarwal, Yexiang Xue, Richard M. Voyles, Juan Wachs.

MHSRS 2022 , In Military Health System Research Symposium, 2022. Abstract Paper (Oral Presentation)

[C11] Sequential Prediction with Logic Constraints for Surgical Robotic Activity Recognition. [pdf] [code] [video]

Md Masudur Rahman, Richard Voyles, Juan Wachs, Yexiang Xue.

RO-MAN 2021, In Proceedings of the 30th IEEE International Conference on Robot & Human Interactive Communication - 2021, 8 pages. conference

[C10] Dexterous Skill Transfer between Surgical Procedures for Teleoperated Robotic Surgery. [pdf]

Mridul Agarwal, Glebys Gonzalez, Mythra V. Balakuntala, Md Masudur Rahman, Vaneet Aggarwal, Richard M. Voyles Yexiang Xue, Juan Wachs.

RO-MAN 2021, In Proceedings of the 30th IEEE International Conference on Robot & Human Interactive Communication - 2021, 7 pages. conference

[C9] A Semi-autonomous Robotic Framework for Remote Surgery under Delays.

Glebys Gonzalez, Mythra Varun Balakuntala Srinivasa Murthy, Mridul Agarwal, Md Masudur Rahman, Richard M. Voyles, Vaneet Aggarwal, Yexiang Xue, Juan Wachs.

MHSRS 2021 , In Military Health System Research Symposium, 2021. Abstract Paper (Oral Presentation)

[C8] DESERTS: Delay-Tolerant Semi-Autonomous Robot Teleoperation for Surgery. [pdf]

Glebys Gon-zalez, Mridul Agarwal, Mythra Varun Balakuntala Srinivasa Mur, Md Masudur Rahman, Upinder Kaur, Richard Voyles, Vaneet Aggarwal, Yexiang Xue, Juan Wachs.

ICRA 2021, In Proceedings of the 2021 IEEE International Conference on Robotics and Automation - 2021, 8 pages. conference

[J3] SARTRES: A Semi-Autonomous Robot TeleopeRation Environment for Surgery. [pdf]

Md Masudur Rahman*, Mythra Varun Balakuntala Srinivasa Mur*, Mridul Agarwal, Upinder Kaur, Vishnunandan Lakshmi Venkatesh, Glebys Gonzalez, Natalia Sanchez Tamayo, Yexiang Xue, Richard Voyles, Vaneet Aggarwal, Juan Wachs. [* equal authorship]

AECAI, TCIV 2020, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization Journal - 2020, AE-CAI | CARE | OR 2.0 Joint MICCAI Workshop - 2020, 14 pages. journal

[J2] From the DESK (Dexterous Surgical Skill) to the Battlefield - A Robotics Exploratory Study

Glebys T. Gonzalez*, Upinder Kaur*, Md Masudur Rahman*, Vishnunandan Venkatesh, Natalia Sanchez, Gregory Hager, Yexiang Xue, Richard Voyles, Juan Wachs. [* equal authorship]

MHSRS Journal (Military Medicine) 2020, In Military Health System Research Symposium (Military Medicine), 2020 23 pages. journal

[C7] ASTRO: A Semi-Autonomous Telemedicine Robot for Operative Surgery.

Glebys Gonzalez, Md Masudur Rahman, Mridul Agarwal, Mythra Balakuntala, Vishnu Venkatesh, Vaneet Aggarwal, Yexiang Xue, Richard Voyles, Gregory Hager, MAJ Andrew W Kirkpatrick, MAJ Steve Overholser, Juan Wachs.

MHSRS 2020, In Military Health System Research Symposium, 2020. 3 pages. Abstract Paper

[C6] Transferring Dexterous Surgical Skill Knowledge between Robots for Semi-autonomous Teleoperation. [pdf]

Md Masudur Rahman*, Natalia Sanchez-Tamayo*, Glebys Gonzalez, Mridul Agarwal, Vaneet Aggarwal, Richard M. Voyles, Yexiang Xue, and Juan Wachs. [* equal authorship]

Ro-Man 2019, In Proceedings of the 28th IEEE International Conference on Robot and Human Interactive Communication, 2019, 6 pages. conference

[W2] Morality in Decision-Making: A Causal Approach. [video]

Md Masudur Rahman.

MoDeM 2019, RLDM Workshop on Moral Decision Making [Link] , 2019. talk workshop

[C5] DESK: A Robotic Activity Dataset for Dexterous Surgical Skills Transfer to Medical Robots. [pdf] [arXiv]

Naveen Madapana*, Md Masudur Rahman*, Natalia Sanchez-Tamayo*, Mythra V. Balakuntala, Glebys Gonzalez, Jyothsna Padmakumar Bindu, L. N. Vishnunandan Venkatesh, Xingguang Zhang, Juan Barragan Noguera, Thomas Low, Richard Voyles, Yexiang Xue, Juan Wachs. [* equal authorship]

IROS 2019, In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, 8 pages. conference

[C4] Toward Optimal Selection of Information Retrieval Models for Software Engineering Tasks. [pdf]

Md Masudur Rahman, Saikat Chakraborty, Gail Kaiser, and Baishakhi Ray

SCAM 2019, In Proceedings of the 19th IEEE International Working Conference on Source Code Analysis and Manipulation, 2019, 12 pages. conference

[J1] Recommending GitHub Projects for Developer Onboarding. [pdf] [link]

Chao Liu, Dan Yang, Xiaohong Zhang, Baishakhi Ray, Md Masudur Rahman.

IEEE Access 2018, 13 pages. journal

[C3] Evaluating How Developers Use General-Purpose Web-Search for Code Retrieval. [pdf] [arXiv] [slides] [code]

Md Masudur Rahman, Jed Barson, Sydney Paul, Joshua Kayan, Federico Andres Lois, Sebastian Fernandez Quezada, Christopher Parnin, Kathryn T. Stolee, Baishakhi Ray.

MSR 2018, In Proceedings of the 15th International Conference on Mining Software Repositories, 11 pages. conference

[C2] Which Similarity Metric to Use for Software Documents? A study on Information Retrieval based Software Engineering Tasks. [pdf] [poster]

Md Masudur Rahman, Saikat Chakraborty, Baishakhi Ray.

ICSE 2018 Companion, In Proceedings of 40th International Conference on Software Engineering Companion, 2018, 2 pages. conference

[W1] Finding Similar Projects in GitHub using Word2Vec and WMD. [slides]

Md Masudur Rahman.

NL+SE @FSE 2016 talk workshop

[C1] Topic Model based Privacy Protection in Personalized Web Search. [pdf] [code]

Wasi Ahmad, Md Masudur Rahman, Hongning Wang.

SIGIR 2016, In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2016, 4 pages. conference

Technical Report:

[T1] A Case Study on the Impact of Similarity Measure on Information Retrieval based Software Engineering Tasks. [pdf][arXiv]

Md Masudur Rahman, Saikat Chakraborty, Gail Kaiser, Baishakhi Ray.

Technical Report 2018, 22 pages. technical report

Talks

Talk at AE-CAI | CARE | OR 2.0 JOINT MICCAI WORKSHOP 2020, Link , [Full presentation], [Short presentation] 2020.
Talk at MoDeM 2019, RLDM Workshop on Moral Decision Making [Link] , [Presentation], 2019.

Teaching

Teaching Assistant

Statistical Machine Learning (Graduate), Spring 2021, Purdue University
Web Information Search And Management (Undergraduate), Fall 2020, Purdue University
Introduction to Information Retrieval (Graduate), Fall 2018, University of Virginia
Data Science for Software Engineering (Graduate), Spring 2016, University of Virginia
Theory of Computation (Undergraduate), Fall 2015, University of Virginia
Computer Architecture (Undergraduate), Fall 2015, Spring 2016, Fall 2018, University of Virginia

Instructor

Data Structure (Undergraduate), Fall 2014, Spring 2015, Summer 2015, BRAC University, Bangladesh
Digital Logic Design (Undergraduate), Summer 2015, BRAC University, Bangladesh
Introduction to Computer (Undergraduate), Fall 2014, Spring 2015, BRAC University, Bangladesh

Mentoring

I am fortunate to work with following students in various capacities.

Zachery Peter Berg (Undergrad - Spring 2021, Grad, Purdue University, 2021-2022)
Topic: Reinforcement Learning
Brian Yifei Sun (Undergrad, Purdue University, 2021)
Topic: Reinforcement Learning
Chao Liu (Ph.D Student, Chongqing University, China, 2018)
Topic: Recommending GitHub Project for Developer Onboarding.
Paper: IEEE Access 2018
Jed Barson (Undergrad, University of Virginia, 2018)
Topic: Code Search
Paper: MSR 2018
First appointment after graduation: Software Engineer at Cisco.
Eliza Yixuan Nie (Undergrad, University of Virginia, 2017)
Topic: GitHub Project Search
First appointment after graduation: Software Engineer at Facebook.

Service

Conference Reviewer

ICLR: 2024
NeurIPS: 2023, 2022
ICML: 2023, 2022 (Outstanding reviewer, Top 10%)
AAMAS: 2023, 2022
AISTATS: 2023
UAI: 2023
ECML-PKDD: 2023, 2022, 2021
ICMLA: 2021
ICRA: 2021
RO-MAN: 2021

Journal Reviewer

IEEE Robotics and Automation Letters (RA-L) 2023
Military Medicine 2022
IEEE Access 2018

Sub-reviewer

ICLR: 2023
NeurIPS: 2019, 2020, 2021
ICML: 2019, 2021
IJCAI: 2019, 2021
UAI: 2020, 2021
AISTATS: 2020
AAMAS: 2021
KDD-DMAIC (workshop): 2019
VLDB Journal 2019