LLVM Code of Conduct Transparency report July 15, 2023 - July 15, 2024 (original) (raw)
The Code of Conduct (CoC) Committee is a group of volunteers including some LLVM Foundation board members who volunteer to handle Code of Conduct reports for the LLVM Project. For each report received, a sub-committee of members was selected to investigate. This is the transparency report for resolved incidents from the period July 15, 2023 - July 15, 2024.
Overview
Reports:
The LLVM Code of Conduct Committee resolved 5 incident reports during this time period. If a report has been received but not resolved, it will not be reported here until resolved.
Report #1:
The first occurrence reported was received on September 25th 2023. There was no direct violation of the CoC.
Report #2:
The report was received on November 11th 2023. The reported person did not respond to requests from the committee to discuss the reported behavior, resulting in the account being blocked and the committee not reaching a conclusion on whether the CoC was violated.
Report #3:
The report was received on April 11th 2024. There was no direct violation of the CoC.
Report #4:
The report was received on 3rd of May 2024. The people involved worked out a resolution between themselves before the committee had the chance to do an in-depth analysis.
Report #5:
The report was received on 11th of June 2024. There was no direct violation of the CoC.
Resolutions:
Report #1:
The committee reported to the community about the presence of a banned individual attempting to circumvent a ban and clarified our policy in the LLVM Developer policy.
Report #2:
The committee attempted to reach out to the user to come to a resolution that may not have ended in a ban, however they did not follow up. As a result, they have been banned from the LLVM github organization. This may change in the future if the user responds to the committee and rectifies their behavior.
Report #3:
The committee proposed a number of rules for the person creating a huge number of low-quality PRs to follow. This aims to reduce the workload on maintainers while still enabling, even enhancing, the person’s ability to contribute to LLVM.
Report #4:
The reporter and reportees worked out a resolution between themselves. Inappropriate comments on Discourse were edited to no longer be problematic.
Report #5:
The committee shared a few pieces of advice to help de-escalate miscommunication on the reported review.
Report Details
Report 1: Unknowingly helping a banned individual to circumvent a ban.
The CoC committee has received several reports that a banned individual has been reaching out to different people in the community to circumvent their ban. The banned person has been asking them to create code reviews, submit code, or post review comments on their behalf.
Banning an individual means that the individual has lost access to all project infrastructure and cannot attend LLVM events (in-person or virtual). As a policy, the code of conduct committee does not publish personal identifiable information, such as names, of banned individuals publicly.
We do realize that this means that most members of the community will not be aware of who may be banned. We may revisit this policy in the future.
The committee published these events on Discourse at
Submitting code or review comments on someone else’s behalf
The CoC committee proposed an addition to the developer policy to clarify the intent of bans, and worked with community members to refine the proposal.
The developer policy was update in commit [DevPolicy] Add guidance on bans (#69701) · llvm/llvm-project@608d602 · GitHub to state the following about bans:
Bans
----
The goal of a ban is to protect people in the community from having to interact
with people who are consistently not respecting the
:ref:`LLVM Community Code of Conduct` in LLVM project spaces. Contributions of
any variety (pull requests, issue reports, forum posts, etc.) require
interacting with the community. Therefore, we do not accept any form of direct
contribution from a banned individual.
Indirect contributions are permissible only by someone taking full ownership of
such a contribution and they are responsible for all related interactions with
the community regarding that contribution.
When in doubt how to act in a specific instance, please reach out to
conduct@llvm.org for advice.
Report 2: A user keeps opening nonsensical or insulting issues on Github
A user repeatedly opened nonsensical or insulting issues on LLVM’s github issue tracker.
The committee attempted to reach out to the user to come to a resolution that may not have ended in a ban, however they did not follow up. As a result, they have been banned from the LLVM github organization and are not able to file issues anymore. This may change in the future if the user responds to conduct@llvm.org and rectifies their behavior.
Report 3: A user is submitting a very large number of low quality pull requests.
The reporter states that the user is a drain on their review capacity and reviewer mental health.
The CoC committee requested feedback from multiple reviewers to obtain the most balanced perspective possible.
After analyzing this feedback, the CoC committee concludes that no code of conduct violation has happened; but at the same time proposes a resolution for the reportee consisting of a set of rules to follow.
Repeated behaviors that reviewers observed that caused some frustration from the reviewers’ side included:
- Sometimes, PRs are opened that don’t have a rationale.
- Often PRs don’t have tests.
- Review feedback sometimes gets marked as “resolved”, while the reviewers think it wasn’t resolved. This leads to PRs getting merged, without all review feedback getting properly addressed.
- The reportee has created a number of back-port cherry-picks to release branches on issues for which they were not the author, reviewer or approver. In at least one instance, the issue the patch was fixing wasn’t present on the release branch.
The reviewers observe no malicious intent, but, with the above bullet points in mind, it seems these PRs and back-port requests are sometimes over-enthusiastic and somewhat reckless. The reportee has been asked on a number of occasions to change their behavior, but it’s unclear if that feedback really has been taken on board. As a result, the reviewers wonder if the reportee has a deep enough understanding of the underlying issue. The reviewers do not always observe them doing the necessary work to understand the history of the underlying issue.
These behaviors, when combined, result in a few negative outcomes:
- It puts a high workload and also some emotional stress on reviewers, which is a scarce resource. The result is less reviewer bandwidth for other contributors. It seems that many PRs created by the reportee end up shifting effort that most people would expect the PR creator to do, towards the reviewers.
- Creating back-ports quickly without understanding issues in depth can introduce regressions or other issues on release branches.
- Marking feedback as resolved can lead to PRs getting merged where reviewer feedback hasn’t been appropriately addressed. (Although, note that this remains a contentious point without consensus in the community: RFC: GitHub PR “Resolve Conversation” button)
- By increasing reviewer strain, the quality of the codebase and therefore software can degrade.
The committee proposes the following resolution and rules to be followed by the reportee to reduce the negative side-effects listed above, while also enabling the reportee to further learn how to contribute to the project effectively and efficiently:
- The reportee should only have one PR open at any given time. They should focus on obtaining a deeper understanding of the one PR they work on, rather than spending the same effort on opening many different PRs.
- The reportee should not create back-port requests themselves. When they think a back-port request is needed on the one PR they work on, they should ask one of the reviewers first if they agree.
- The reportee should not resolve feedback comments themselves, but ask reviewers to resolve it on their PR.
- The reportee should not get direct commit/push access.
- The reportee should prioritize responding to reviewers’ questions or suggestions before updating a PR. When abandoning a PR, a rationale comment should be added to the PR explaining the reason for abandoning.
- The above rules can be reconsidered and dropped after the reviewers involved have observed a deeper understanding of issues from the reportee, and their concerns about over-enthusiastic and potentially reckless behaviors have disappeared.
While there are no punitive actions listed in this resolution, failure to comply with these rules would result in code of conduct violations.
Report 4: Multiple people in an RFC Discourse thread posting unfriendly and disrespectful comments.
The reporter and reportees are all part of the same organization. Ultimately, this issue was addressed within that organization before the committee acted. Reportees edited reported comments in Discourse to no longer be problematic. No further action was taken by the CoC committee.
Report 5: Miscommunication on a code review is resulting in escalated tension.
The CoC committee is thankful for the reporter reaching out for guidance with the disagreements in the reported PR. The CoC committee agreed that things were escalating and makes a few recommendations that they hope are useful, including:
- A video chat between PR participants in this particular case could be a good approach. Intent and tone can be lost in written words and it could also speed up the review process. The committee offers to help navigate the conversation on the video chat, if that would be helpful.
- The committee suggests avoiding non-constructive feedback. The committee pointed out examples of PR feedback that should be omitted in the future and possible alternatives.
- Avoid phrases rooted in opinion without much explanation and prefer providing concrete descriptions why an opinion was formed. It is even better to provide alternative suggestions.
- Avoid profanity and exaggerated language. For example, if a reviewer is frustrated by repeating feedback, respond to the PR author along the lines of “I am frustrated because I have mentioned this before. Do we have the same understanding on how to apply such feedback? If not, let’s work through examples.”
- Ask questions or for clarification before reiterating you are correct.
- Limit feedback to the direct topics related to the PR, and do not suggest behavioral or conflict resolution improvements unless they violate the code of conduct and when doing so, stick to concrete advice on observed behavior.
- Be specific in the changes you are asking for.
- When someone asks for you to stop a behavior and seems upset, do not repeat it and go into further discussion. While there may have been a misunderstanding on intent, it’s wise to stay focused on the topic of the PR.
For anyone interested in exploring this topic further, the committee also recommends talks on the topic of effective code reviews by April Wensel, including the keynote she gave at the LLVM developers meeting in 2023 and the talk she gave at “try! Swift conference” in 2018. (video)(slides). There are more articles on this topic by many others too if you search for “compassionate code review”.