https://github.com/libuv/libuv/pull/3952 Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} libuv / libuv Public * Notifications * Fork 3.4k * Star 21.5k * Code * Issues 106 * Pull requests 47 * Discussions * Actions * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Security * Insights New issue Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community. Pick a username [ ] Email Address [ ] Password [ ] [ ] Sign up for GitHub By clicking "Sign up for GitHub", you agree to our terms of service and privacy statement. We'll occasionally send you account related emails. Already on GitHub? Sign in to your account Jump to bottom linux: introduce io_uring support #3952 Merged bnoordhuis merged 1 commit into libuv:v1.x from bnoordhuis:iou Apr 18, 2023 Merged linux: introduce io_uring support #3952 bnoordhuis merged 1 commit into libuv:v1.x from bnoordhuis:iou Apr 18, 2023 +662 -35 Conversation 52 Commits 1 Checks 36 Files changed 7 Conversation This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters bnoordhuis Copy link Member @bnoordhuis bnoordhuis commented Apr 12, 2023 * edited Add io_uring support for several asynchronous file operations: * read, write * fsync, fdatasync * stat, fstat, lstat io_uring is used when the kernel is new enough, otherwise libuv simply falls back to the thread pool. Performance looks great; an 8x increase in throughput has been observed. This work was sponsored by ISC, the Internet Systems Consortium. Sorry, something went wrong. 33 olof-nord, ericphanson, espoal, mcollina, asio1337, ronkorving, MomenNano, joy4eg, gurgunday, H4ad, and 23 more reacted with hooray emoji [?] 58 tsondergaard, puzpuzpuz, feniljain, debadree25, ozcanay, CircuitCoder, jorgebay, vdeturckheim, Theofilos-Chamalis, paambaati, and 48 more reacted with heart emoji 43 paambaati, jmillan, bricss, haoyu234, MomenNano, aapoalas, rektide, Ender-events, GaryGSC, debadree25, and 33 more reacted with rocket emoji All reactions * 33 reactions * [?] 58 reactions * 43 reactions @bnoordhuis bnoordhuis force-pushed the iou branch from 8dd8b3e to efc9bbf Compare April 12, 2023 21:48 @bnoordhuis bnoordhuis mentioned this pull request Apr 12, 2023 Use io_uring for read/write/fsync on Linux #2322 Closed 7 tasks trevnorris trevnorris reviewed Apr 13, 2023 View reviewed changes src/unix/linux.c Show resolved Hide resolved @Moelf Moelf mentioned this pull request Apr 13, 2023 Scaling IO with number of threads to achieve high throughput JuliaLang/julia#48232 Open @bnoordhuis bnoordhuis force-pushed the iou branch from 86c2f62 to ea34563 Compare April 13, 2023 12:33 @rektide Copy link rektide commented Apr 13, 2023 * edited Also potentially resolves or progresses #1947. Also: ! ! Comment from the peanut gallery here, but, personally I'd be fine with/love/prefer an io_uring that targeted more modern io_urings. My understanding is 5.15 (2021.10.31) had a bunch of pretty sizable improvements to how folks could use io_uring. That's already nearly two years old, and is going to get more and more expectable. Meanwhile folks on older kernels (5.1 is 2019.5.5) still have the very fast existing libuv thread pool implementation to fall back on. Maybe there's no sacrifice, but "a carefully chosen feature subset" leaves the impression that we're going way out of our way to support already pretty aged kernels. Grain of salt here, but being rip-raring future-forward-facing & adopting every whiz-bang shiny-new-whistle we can would be my pick at this juncture. 7 paulstuart, johnm, matthiasgoergens, hackerjones, ptrwis, chientrm, and haf reacted with thumbs up emoji 1 nalply reacted with laugh emoji All reactions * 7 reactions * 1 reaction Sorry, something went wrong. @trevnorris Copy link Member trevnorris commented Apr 13, 2023 Did some basic benchmarking, and it's giving me > 8x performance when reading small chunks out of /dev/zero. So great news on that side. valgrind isn't happy about a few tests. Mostly with the error Syscall param io_uring_enter(sigsz) contains uninitialised byte(s). Simplest repro I could make is this: #include <uv.h> void on_read(uv_fs_t *req) { } int main() { uv_fs_t read_req; uv_buf_t buffer; int fd; char bytes[64]; buffer = uv_buf_init(bytes, 64); fd = uv_fs_open(NULL, &read_req, "/dev/zero", UV_FS_O_RDONLY, 0, NULL); uv_fs_read(uv_default_loop(), &read_req, fd, &buffer, 1, 0, on_read); uv_run(uv_default_loop(), UV_RUN_DEFAULT); uv_fs_close(NULL, &read_req, fd, NULL); uv_fs_req_cleanup(&read_req); uv_loop_close(uv_default_loop()); return 0; } [?] 8 matthiasgoergens, orin220444, dvergeylen, GaryGSC, chientrm, MomenNano, xorgy, and corporatepiyush reacted with heart emoji 1 benjamingr reacted with rocket emoji All reactions * [?] 8 reactions * 1 reaction Sorry, something went wrong. @bnoordhuis bnoordhuis force-pushed the iou branch from ea34563 to a8e328d Compare April 14, 2023 18:43 @bnoordhuis Copy link Member Author bnoordhuis commented Apr 14, 2023 personally I'd be fine with/love/prefer an io_uring that targeted more modern io_urings Things may be heading that way out of necessity. IORING_SETUP_SQPOLL requires linux v5.11 if you want to use it with non-fixed files (IIUC) but libuv can't do fixed files and without sqpoll it has to call io_uring_enter(IORING_ENTER_GETEVENTS) so often that any performance benefits are negated. 3 hackerjones, rektide, and espoal reacted with thumbs up emoji All reactions * 3 reactions Sorry, something went wrong. santigimeno santigimeno reviewed Apr 14, 2023 View reviewed changes src/unix/linux.c Outdated Show resolved Hide resolved @bnoordhuis bnoordhuis force-pushed the iou branch from a8e328d to 87fd159 Compare April 15, 2023 18:57 @bnoordhuis bnoordhuis marked this pull request as ready for review April 15, 2023 18:58 @bnoordhuis Copy link Member Author bnoordhuis commented Apr 15, 2023 This is ready for review now. Linux v5.13 is the minimum required, otherwise libuv falls back to the thread pool. All reactions Sorry, something went wrong. santigimeno santigimeno reviewed Apr 15, 2023 View reviewed changes Copy link Member @santigimeno santigimeno left a comment There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Should we document this change in the fs docs? Sorry, something went wrong. All reactions src/unix/linux.c Show resolved Hide resolved test/test-threadpool-cancel.c Show resolved Hide resolved src/unix/linux.c Show resolved Hide resolved @bnoordhuis Copy link Member Author bnoordhuis commented Apr 15, 2023 * edited Should we document this change in the fs docs? Yes. I thought it didn't say anything about the implementation but it does. I'll do that tomorrow. All reactions Sorry, something went wrong. santigimeno santigimeno approved these changes Apr 16, 2023 View reviewed changes Copy link Member @santigimeno santigimeno left a comment * edited There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment LGTM (once docs are updated). This is awesome, thanks Sorry, something went wrong. All reactions @trevnorris Copy link Member trevnorris commented Apr 17, 2023 Still getting the following from valgrind: fs_file_async Syscall param io_uring_enter(sigsz) contains uninitialised byte(s) at 0x4993A3D: syscall (syscall.S:38) by 0x4F1366: uv__io_uring_enter (src/unix/linux.c:351) by 0x4F23A9: uv__iou_submit (src/unix/linux.c:644) by 0x4F24B5: uv__iou_fs_read_or_write (src/unix/linux.c:692) by 0x4DDAAC: uv_fs_write (src/unix/fs.c:2212) by 0x422F97: create_cb (test/test-fs.c:491) by 0x4DA38C: uv__fs_done (src/unix/fs.c:1779) by 0x4CDFD0: uv__work_done (src/threadpool.c:326) by 0x4D6433: uv__async_io (src/unix/async.c:176) by 0x4F2F95: uv__io_poll (src/unix/linux.c:1076) by 0x4D6B2D: uv_run (src/unix/core.c:417) by 0x42241C: run_test_fs_file_async (test/test-fs.c:915) All reactions Sorry, something went wrong. trevnorris trevnorris approved these changes Apr 17, 2023 View reviewed changes Copy link Member @trevnorris trevnorris left a comment There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment LGTM other than the valgrind error. The current code you have for incrementing events is fine enough since it basically follows what the threadpool is doing. It's a bug, but not one you should need to worry about. I'll take care of that in a future PR. Sorry, something went wrong. All reactions trevnorris trevnorris reviewed Apr 17, 2023 View reviewed changes src/unix/linux.c Outdated Show resolved Hide resolved @bnoordhuis Copy link Member Author bnoordhuis commented Apr 18, 2023 FWIW, I don't get that warning with valgrind 3.18.1. I'm 99% sure it's a valgrind bug because that sigsz it's complaining about is something that libuv doesn't use and doesn't exist in newer kernels. On kernels where it was a thing libuv simply never calls io_uring_enter(). All reactions Sorry, something went wrong. @bnoordhuis bnoordhuis force-pushed the iou branch from a1b1303 to a66dbfe Compare April 18, 2023 10:02 @bnoordhuis linux: introduce io_uring support ... 26c79a9 Add io_uring support for several asynchronous file operations: - read, write - fsync, fdatasync - stat, fstat, lstat io_uring is used when the kernel is new enough, otherwise libuv simply falls back to the thread pool. Performance looks great; an 8x increase in throughput has been observed. This work was sponsored by ISC, the Internet Systems Consortium. @bnoordhuis bnoordhuis force-pushed the iou branch from a66dbfe to 26c79a9 Compare April 18, 2023 10:07 Hide details View details @bnoordhuis bnoordhuis merged commit d2c31f4 into libuv:v1.x Apr 18, 2023 35 of 36 checks passed @bnoordhuis bnoordhuis deleted the iou branch April 18, 2023 10:32 @trevnorris Copy link Member trevnorris commented Apr 18, 2023 * edited @bnoordhuis I'm also running valgrind 3.18.1. Upon further testing, the error only appears when I compile with clang (I tried all versions 11 to 15). It doesn't happen with gcc. Think it's a bug in clang or just a strange interaction with valgrind? Side note: need to compile with -gdwarf-4 when using clang for valgrind to work. All reactions Sorry, something went wrong. @bnoordhuis Copy link Member Author bnoordhuis commented Apr 18, 2023 Oh, that's interesting. Yes, I'm getting the same warning with clang. gcc emits this code for the syscall wrapper: 0x00000000000997f0 <+0>: endbr64 0x00000000000997f4 <+4>: sub $0x10,%rsp 0x00000000000997f8 <+8>: mov %ecx,%r8d 0x00000000000997fb <+11>: xor %r9d,%r9d 0x00000000000997fe <+14>: mov %edx,%ecx 0x0000000000099800 <+16>: push $0x0 0x0000000000099802 <+18>: mov %esi,%edx 0x0000000000099804 <+20>: xor %eax,%eax 0x0000000000099806 <+22>: mov %edi,%esi 0x0000000000099808 <+24>: mov $0x1aa,%edi 0x000000000009980d <+29>: call 0xe810 <syscall@plt> 0x0000000000099812 <+34>: add $0x18,%rsp 0x0000000000099816 <+38>: ret While clang emits this: 0x00000000000a31f0 <+0>: push %rax 0x00000000000a31f1 <+1>: mov %ecx,%r8d 0x00000000000a31f4 <+4>: mov %edx,%ecx 0x00000000000a31f6 <+6>: mov %esi,%edx 0x00000000000a31f8 <+8>: mov %edi,%esi 0x00000000000a31fa <+10>: movl $0x0,(%rsp) 0x00000000000a3201 <+17>: mov $0x1aa,%edi 0x00000000000a3206 <+22>: xor %r9d,%r9d 0x00000000000a3209 <+25>: xor %eax,%eax 0x00000000000a320b <+27>: call 0xd780 <syscall@plt> 0x00000000000a3210 <+32>: pop %rcx 0x00000000000a3211 <+33>: ret I'm guessing that valgrind gets confused by that movl $0x0,(%rsp) for the sigsz argument somehow. All reactions Sorry, something went wrong. @KyleSanderson KyleSanderson mentioned this pull request Apr 18, 2023 feature: nonblocking disk IO transmission/transmission#2462 Open bnoordhuis added a commit to bnoordhuis/libuv that referenced this pull request Apr 19, 2023 @bnoordhuis linux: fix academic valgrind warning ... 571c551 Fix a valgrind warning that only manifested with clang (not gcc!) by explicitly passing 0L instead of plain 0 as the |sigsz| argument to io_uring_enter(). That is, pass a long instead of an int. On x86_64, |sigsz| is passed on the stack (the other arguments are passed in registers) but where gcc emits a `push $0` that zeroes the entire stack slot, clang emits a `movl $0,(%rsp)` that leaves the upper 32 bits untouched. It's academic though since we don't pass IORING_ENTER_EXT_ARG and the kernel therefore completely ignores the argument. Refs: libuv#3952 @bnoordhuis bnoordhuis mentioned this pull request Apr 19, 2023 linux: fix academic valgrind warning #3960 Merged @bnoordhuis Copy link Member Author bnoordhuis commented Apr 19, 2023 @trevnorris figured it out while I was doing something completely different (isn't it always like that?), please see #3960. :-) 1 trevnorris reacted with hooray emoji All reactions * 1 reaction Sorry, something went wrong. bnoordhuis added a commit that referenced this pull request Apr 19, 2023 @bnoordhuis linux: fix academic valgrind warning (#3960) ... a7ff759 Fix a valgrind warning that only manifested with clang (not gcc!) by explicitly passing 0L instead of plain 0 as the |sigsz| argument to io_uring_enter(). That is, pass a long instead of an int. On x86_64, |sigsz| is passed on the stack (the other arguments are passed in registers) but where gcc emits a `push $0` that zeroes the entire stack slot, clang emits a `movl $0,(%rsp)` that leaves the upper 32 bits untouched. It's academic though since we don't pass IORING_ENTER_EXT_ARG and the kernel therefore completely ignores the argument. Refs: #3952 @espoal espoal mentioned this pull request Apr 19, 2023 Add experimental support for io_uring nodejs/node#34388 Closed @espoal Copy link espoal commented Apr 19, 2023 * edited amazing work, I will make sure to add this PR to my awesome io_uring repo. Is there a way to test it? Maybe I could write a small tutorial / benchmark Also is this PR just for files or also for network operations? All reactions Sorry, something went wrong. espoal espoal reviewed Apr 19, 2023 View reviewed changes docs/src/fs.rst @@ -12,6 +12,12 @@ otherwise it will be performed asynchronously. All file operations are run on the threadpool. See :ref:` threadpool` for information on the threadpool size. Starting with libuv v1.45.0, some file operations on Linux are handed off to `io_uring <https://en.wikipedia.org/wiki/Io_uring>` when possible. Apart from a (sometimes significant) increase in throughput there should be no change in Copy link @espoal espoal Apr 19, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment is it true that there are no changes in observable behaviour? I can think of a couple of edge cases with NFS. I guess it's time to test :) Also is this PR just for files or also for network operations? Sorry, something went wrong. All reactions Copy link Member Author @bnoordhuis bnoordhuis Apr 20, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Just file operations for now. If you want to test without io_uring to compare, you can set UV_USE_IO_URING=0 in the environment (but note that's undocumented and may go away in the future.) Sorry, something went wrong. All reactions Copy link Contributor @clason clason May 12, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Hi! Neovim dev here. You know what's coming now... This change has introduced regressions in some file operations (hang when polling a large number of files neovim/neovim#23595 and reproducible downstream failures in CI that we're still trying to narrow down). So an official way to disable io_uring, preferably through the API, would be necessary for us. Sorry, something went wrong. All reactions Copy link Member Author @bnoordhuis bnoordhuis May 12, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment I don't plan on doing that but if it's a libuv bug, I'll fix it of course, and if you're hitting a kernel bug, I can probably add detection-and-fallback code. I'll need the kernel version, arch and faulting IORING_OP at the very least for that. Sorry, something went wrong. All reactions Copy link Contributor @clason clason May 12, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Affected kernels I know about: * Arch Linux, kernel 6.3.1-arch1-1 * ChromeOS M113's 5.15.103-17409 inside of a Crostini container * and possibly whatever GHA is currently using with ubuntu-latest I'll have to see about tracing the IORING_OP calls for that (any advice or links for that?) Sorry, something went wrong. All reactions Load more... Copy link Contributor @clason clason May 12, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment And here's a minimal reproducer (in Lua): neovim/neovim#23595 (comment) Sorry, something went wrong. All reactions Copy link Member Author @bnoordhuis bnoordhuis May 12, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Thanks, I'll take a look! Caveat emptor: weekend, kids, etc., so it might take me a few days to circle back. Sorry, something went wrong. 1 clason reacted with thumbs up emoji All reactions * 1 reaction Copy link Member @santigimeno santigimeno May 12, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment It seems to me that the cq ring might have overflowed IORING_SQ_CQ_OVERFLOW and we might need to call io_uring_enter() to recover the events that didn't make it. Sorry, something went wrong. All reactions Copy link Member Author @bnoordhuis bnoordhuis May 14, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Yes, I strongly suspect it's something along those lines. I'll open a pull request once I've managed to reproduce with just libuv. (Or if you want to send one, of course go ahead.) Sorry, something went wrong. All reactions Copy link Member @santigimeno santigimeno May 14, 2023 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment @bnoordhuis I've given this a try in #3991. Let me know what you think. Thanks! Sorry, something went wrong. All reactions @angelsanzn angelsanzn mentioned this pull request May 11, 2023 io_uring new uSockets backend uNetworking/uWebSockets#1603 Open @santigimeno santigimeno mentioned this pull request May 12, 2023 Release proposal: v1.45.0 #3887 Closed 2 tasks @aktau Copy link aktau commented May 15, 2023 After Neovim merged the fix for overflow, I could start using it and it seems to work. But I see the following in dmesg: nvim: epoll_ctl support in io_uring is deprecated and will be removed in a future Linux kernel version. Neovim does not appear to use epoll_ctl(2) itself, so this is likely libuv. All reactions Sorry, something went wrong. @santigimeno Copy link Member santigimeno commented May 15, 2023 * edited @aktau it's being taken care of upstream: https://lore.kernel.org/ io-uring/20230501185240.352642-1-info@bnoordhuis.nl/T/#u All reactions Sorry, something went wrong. @plajjan plajjan mentioned this pull request May 18, 2023 New kernel polling interface for Linux 4.18 (io_uring)? #1947 Closed @santigimeno santigimeno mentioned this pull request May 19, 2023 deps: upgrade to libuv 1.45.0 nodejs/node#48078 Closed @zhaozg zhaozg mentioned this pull request May 20, 2023 request bind v1.45.0 new features luvit/luv#646 Closed santigimeno added a commit to santigimeno/node that referenced this pull request May 20, 2023 @santigimeno deps: upgrade to libuv 1.45.0 ... 7170fc9 - linux: introduce io_uring support libuv/libuv#3952 - src: add new metrics APIs libuv/libuv#3749 - unix,win: give thread pool threads an 8 MB stack libuv/libuv#3787 - win,unix: change execution order of timers libuv/libuv#3927 - Fixes: nodejs#43931 - Fixes: nodejs#42496 @squeek502 squeek502 mentioned this pull request May 20, 2023 valgrind thinks statbuf is uninitialised during uv_fs_stat (false positive) #4008 Open @aantron aantron mentioned this pull request May 23, 2023 Upgrade libuv to 1.45.0 aantron/luv#151 Open santigimeno added a commit to santigimeno/node that referenced this pull request May 24, 2023 @santigimeno deps: upgrade to libuv 1.45.0 ... 851508b - linux: introduce io_uring support libuv/libuv#3952 - src: add new metrics APIs libuv/libuv#3749 - unix,win: give thread pool threads an 8 MB stack libuv/libuv#3787 - win,unix: change execution order of timers libuv/libuv#3927 Fixes: nodejs#43931 Fixes: nodejs#42496 Fixes: nodejs#47715 Fixes: nodejs#47259 Fixes: nodejs#47241 nodejs-github-bot pushed a commit to nodejs/node that referenced this pull request May 24, 2023 @santigimeno @nodejs-github-bot deps: upgrade to libuv 1.45.0 ... 9e68f94 - linux: introduce io_uring support libuv/libuv#3952 - src: add new metrics APIs libuv/libuv#3749 - unix,win: give thread pool threads an 8 MB stack libuv/libuv#3787 - win,unix: change execution order of timers libuv/libuv#3927 Fixes: #43931 Fixes: #42496 Fixes: #47715 Fixes: #47259 Fixes: #47241 PR-URL: #48078 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> Reviewed-By: Debadree Chatterjee <debadree333@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Michael Zasso <targos@protonmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Tobias Niessen <tniessen@tnie.de> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> @corporatepiyush corporatepiyush mentioned this pull request May 28, 2023 improve socket read throughput dart-lang/sdk#48210 Open 4 tasks Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment Reviewers @trevnorris trevnorris trevnorris approved these changes @santigimeno santigimeno santigimeno approved these changes @clason clason clason left review comments @espoal espoal espoal left review comments Assignees No one assigned Labels None yet Projects None yet Milestone No milestone Development Successfully merging this pull request may close these issues. None yet 7 participants @bnoordhuis @rektide @trevnorris @espoal @aktau @santigimeno @clason Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.