Big thanks for all the coffee! :)
This was a motivation for me to do an early upgrade of GNOME Shell to v3.38 (my Fedora 32 still uses v3.36) and make the WindowOverlay Icons extension support it.
Thanks to you, my supporters, I spent some time rewriting the extension and there is now a new version v36 of the extension that not only works on GNOME Shell v3.38 but also has prettier transitions when opening and closing the windows overview.
GitHub release: https://github.com/sustmi/gnome-shell-extension-windowoverlay-icons/releases/tag/v36
Link to extension: https://extensions.gnome.org/extension/302/windowoverlay-icons/