win-wasapi: Add support for capturing a process by jpark37 · Pull Request #5218 · obsproject/obs-studio (original) (raw)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation98 Commits6 Checks0 Files changed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
Description
Use new Windows API for capturing audio from a process.
Motivation and Context
Users want it.
How Has This Been Tested?
Not extensively tested, but seems to work. Tested input/output/process, and fixed various issues.
Types of changes
- New feature (non-breaking change which adds functionality)
Checklist:
- My code has been run through clang-format.
- I have read the contributing document.
- My code is not on the master branch.
- The code has been tested.
- All commit messages are properly formatted and commits squashed where appropriate.
- I have included updates to all appropriate documentation.
Checks are probably busted until GitHub Actions provides 20348 SDK.
I wonder if this should be a mere separate mode or a separate source. Either way, should still keep it in the same dll.
As for the process selection, #2659 would greatly simplify it, right?
So here are my thoughts on how I think this should be exposed to the user:
- This should not be a thing you can set to a global audio source (the default Desktop Audio source, for example). This is because the core of that is modified in the Audio section of Settings, which a user shouldn't need to modify live. It is accessible from the Audio Mixer using the gear + Properties, but in this particular situation that should remain device-only.
- The "Method" dropdown should have three options:
a) Device
b) OBS Capture Source
c) Process / Application - For the "OBS Capture Source" method, the list should be populated with a list of Game Capture sources, a ---- separator, then a list of Window Capture sources. This should behave identically to Process mode, but doesn't require the user to manually specify a process, it just mirrors the Window / Game capture source's selected window.
- For the Process / Application method, I agree that process ID would be bad. I'd say this method should essentially use the same window lookup/storage functions as Window Capture (same kind of dropdown). This is for when the user doesn't want to window capture the process but does want its audio (eg. a media player).
One thing for sure, I do not want duplicated code for the process/window selection.
Have not tested my PR myself yet, but from a UX standpoint is there any reason this can't/shouldn't just be a checkbox in game capture / window capture that says "Capture audio directly" or something?
Have not tested my PR myself yet, but from a UX standpoint is there any reason this can't/shouldn't just be a checkbox in game capture / window capture that says "Capture audio directly" or something?
What if you want to include some audio but don't want to show that window?
Sure you can hide the source but that's still taking unnecessary ressources.
Having it in the properties of window / game capture and an extra source you can add perhaps?
One UX improvement would be for the Game/Window capture source to have a "add audio capture source" button somewhere.
Have not tested my PR myself yet, but from a UX standpoint is there any reason this can't/shouldn't just be a checkbox in game capture / window capture that says "Capture audio directly" or something?
What if you want to include some audio but don't want to show that window?
Sure you can hide the source but that's still taking unnecessary ressources.
A dropdown then to choose between capturing video, audio, or both perhaps?
I do not think we should have two separate flows through the program for capturing a game or window depending on what the user wants to capture from it
Could work similarly to this plugin which does something similar but attaches before WASAPI since v2.0 I believe. It is only recent. GitHub
Have not tested my PR myself yet, but from a UX standpoint is there any reason this can't/shouldn't just be a checkbox in game capture / window capture that says "Capture audio directly" or something?
What if you want to include some audio but don't want to show that window?
Sure you can hide the source but that's still taking unnecessary ressources.A dropdown then to choose between capturing video, audio, or both perhaps?
I do not think we should have two separate flows through the program for capturing a game or window depending on what the user wants to capture from it
I've thought on this more and changed my mind a bit. I'm mostly in agreement with what Matt suggested.
However, I don't think this should piggyback off the Audio Output Capture source type, at least UI/UX wise. Internally it can work that way if need be, but I think this should be an additional source type in the list and leave Audio Output Capture for devices only.
- A setting in window and game capture properties to also grab audio from their specified process
- An "Application Audio" source type that is purely audio capture
This way people can get video/audio from things like games all in one place, as well as intuitively grab audio only from things like Discord or a music app.
I did a force-push, so you can select process by window now. I tested that Chrome and Spotify work. I'm a bit surprised that Chrome worked out of the box because I thought I was going to have to play shenanigans to find the audio process. I don't know how they map processes to windows/audio.
There's a lot of copy/paste from win-capture, but I didn't want to push that code into libobs until we get closer to deciding what to do. Window priority is not selectable and hard-coded to title to now.
Are we sure we want to add a new source type? I thought people wanted to consolidate them, e.g. merging window/game capture, but I might be remembering wrong.
Member
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just nitpicking on a string name change.
I feel like there's a lot of code in here that is derived or copied entirely from code that we have elsewhere in the codebase for selecting application windows and determining information about a window. Would it make sense to have these in some utils or platform code that plugins could use instead of copying them around from place to place?
Edit: I should have read the most recent comment more closely, as you've already mentioned this. Addressing in another comment.
Fwiw, I can only reproduce the scene cut issue if I use separate sources. I don't hear a cut if I add the existing source to the other scene.
Is it possible to prevent a user from picking the same process twice, or displaying a red error on the selection or anything if they do?
The audio mixer should show no audio getting through. It's similar to the video capture sources; if it doesn't work, you just get no picture in the scene preview, and we log the failure.
The mixer shows no audio, but it's not really clear to the user why unless they know they have another source trying to hook the same process.
I am aware game capture suffers the same problem, but I'm not a fan of that either :P
There is a brief cut/crackle when using a Cut transition between two scenes that both have the same audio capture.
Are you using the same source, or two separate sources pointed at the same process? Is this an issue with regular Audio Output Capture?
Same source in both scenes, as I cannot get two sources to capture the same process.
Add "ms_" prefix as makeshift namespace.
Use new process output API, and retrofit existing WASAPI abstractions.
Marked as "(BETA)" until we figure out the crackling at 60 minutes.
Lets OBS find Spotify.exe even if the window title changes.
Probably needs a follow-up PR for Yami theme.
will this support surround sound?
will this support surround sound?
If the application outputs surround sounds, then OBS should be able to pull surround sound. To the API in question, it's just data.
This was referenced
Jan 17, 2023
I'm having the Noisy after 30-ish minutes issue with this audio capture method. Are you still looking for logs?
I'm having the Noisy after 30-ish minutes issue with this audio capture method. Are you still looking for logs?
We are still collecting information on this, yes. I can be reached directly on Discord at RytoEX#5545. Specifically, we are still looking for the data outlined here.
Could you (the devs) provide a new build with this feature? I'd love to try it for testing, etc. Thank you.
Could you (the devs) provide a new build with this feature? I'd love to try it for testing, etc. Thank you.
The feature has been included in OBS Studio since OBS Studio 28.0 which was released in August 2022.
Oh terribly sorry for the necro then I'll dig deeper into the settings then. I am truly sorry and I appreciate all the work you put into the project.
Update: In case anyone is looking for this... it's a different Source. You have to add a new Source (Game Audio Capture) and that's it. Amazing feature btw.
This feature is unfortunately still lacking, compared to an available win-capture-audio plugin, which has exclusions. Please consider adding that. So that e.g. few music apps and all non-music apps can be easily configured into 2 separate sources.
Wish this had the exclusion feature from win-audio-capture.
Reviewers
