Microsoft uses AI to expedite pull requests – the feature that developers can use to notify others of changes to code – by reminding authors to deal with their overdue requests. This comes from a new whitepaper released this week that describes Nudge, an end-to-end service that uses models based on effort estimation to predict the completion time for a given pull request. According to Microsoft, Nudge has been deployed to 147 internal repositories since 2019 and “significantly” reduced the completion time for the 8,500 pull requests for which notifications were sent by 60%.
With the advent of platforms like GitHub and GitLab, pull requests have become the standard mechanism for distributed code reviews. They allow one or more developers and even bots to review changes. Once the reviewers have logged out, the changes can be merged with the main branch and posted. While pull requests optimize the review process, they can also slow down the software development process. For example, if reviewers become overloaded and lose track of the requirement, it may not be reviewed in a timely manner.
Looking for a solution, researchers from Microsoft and Delft University of Technology developed Nudge, which uses effort estimation and activity detection algorithms to predict the completion time for a given pull request. Nudge takes it one step further, analyzing the request status to identify the author or reviewers of the pull request holding the request from completion and notifying them of comments in the pull request environment.
To create nudge, the researchers performed a correlation analysis to analyze various factors related to pull requests, pull request authors, pull request reviewers, and even timing factors that affect the request completion time. They then built the cost estimation model to predict the completion time of the pull request at the time the request was created, using metrics such as code churn, reviewer information, and ownership information. Next, they implemented activity detection to monitor updates to the pull request (such as new commits and review comments) and adjust notifications accordingly. To make the nudge notifications actionable, the researchers used what is known as a dependency determination that identifies the author of the pull request or a specific reviewer.
After serving nudge to repositories at Microsoft for 18 months, the researchers report that 73% of pull request notifications were “positively acknowledged” by users. “Pull requests are an important part of the collaborative software development process,” they write. “Only 1,570 pull requests out of 8,500 pull requests (18.47%) took more than a week to close. 81.53% of pull requests are closed within a week. One important note is that 2,300 pull requests (27.05%) of the pull requests that Nudge sent notification on were completed in one day. This distribution shows that most pull requests that Nudge sends notifications on complete relatively quickly. “
Nudge builds on Microsoft’s previous efforts to infuse the DevOps process with AI and machine learning. For example, in April the company detailed a system that claims it correctly differentiates between security and non-security software bugs 99% of the time, and that accurately identifies high priority critical security bug an average of 97% of the time . Microsoft uses AI to troubleshoot cloud service outages.