Kranky Geek WebRTC 2015 - What's next for WebRTC? (original) (raw)
1. WebRTC: Google Notes KrankyGeek, Sep 2015
2. •
3. Our 40 minutes together ❖ Vital Signs ❖ WebRTC News ❖ Public Service Announcements ❖ Recent Improvements ❖ Coming Attractions •
5. Vital Signs: WebRTC Adoption ❖ Tracking 720 companies using WebRTC in some form ➢ Up from 550 this past February ❖ Many major applications using WebRTC, confirmed by analysis on webrtchacks.com ➢ Hangouts ➢ Facebook Messenger ➢ Mayday ➢ Wire •
6. ❖ 28 WebRTC related acquisitions to date ❖ Some recent ones: ➢ Acision ➢ Speek ➢ mPortal ➢ Tropo ➢ ESNA ➢ Screenhero ➢ Jitsi Vital Signs: WebRTC Adoption •
7. And now, the news. •
8. News: An open alliance for media (1/2) Focused on developing next generation media formats, codecs and technologies in the public interest. Founding members: Amazon, Netflix, Microsoft, Intel, Cisco, Mozilla, Google http://aomedia.org
9. News: An open alliance for media (2/2) The Alliance’s initial focus is to deliver a next-generation video format that is: ❖ Interoperable and open; ❖ Optimized for the web; ❖ Scalable to any modern device at any bandwidth; ❖ Designed with a low computational footprint and optimized for hardware; ❖ Capable of consistent, highest-quality, real-time video delivery; and ❖ Flexible for both commercial and non-commercial content, including user-generated content. Among the Alliance’s goals is the creation of a new, high-quality open video format that improves core media experiences for all.
10. News: Codecs, Codecs, Codecs ❖ Many companies actively contributing to open codecs: Google (VP9), Mozilla (Daala), Cisco (Thor) ❖ Microsoft announces VP9 support in their new Edge browser ❖ Apple apparently is dropping HEVC from iPhone 6s (source: 9 to 5 Mac)
11. News: Microsoft Edge ❖ A brand new web browser, with support for getUserMedia and ORTC (coming soon). ❖ Reported to include VP9 ❖ More on ORTC later :-)
13. PSA: IP Address 'Leakage' Recap ❖ WebRTC is a P2P stack ❖ P2P means trying to find the fastest route. A fastest route may be locally on your LAN, or over your VPN ❖ Web pages may use data channels silently for CDN purposes ❖ This presents problems when users use VPN to hide their public IP addresses on the Internet
14. PSA: IP Address 'Leakage' TL;DR ❖ The behavior of the stack was by design ❖ However, the behavior of users has changed ❖ We believe we can solve this problem in a smart, safe way
16. Network Limiter Extension ❖ Step 1: Network Limiter Extension ➢ Makes WebRTC traffic follow default route - even if this isn't the 'optimal' path ➢ Explicit opt-in lets us safely understand what side effects it might have
17. Next Steps ❖ Step 2: Make Default ➢ Once issues are resolved (a few remain), turn this behavior on by default ➢ Sites with camera permission can still get previous behavior ➢ May result in subtle behavior changes for data channel apps (e.g. more relay) ➢ Try the extension with your app
18. PSA: Mandatory HTTPS Chrome 47 rolling out in December 2015 WILL NOT ALLOW getUserMedia requests from http:// •
19. PSA: HTTPS One. more. time: NO getUserMedia from HTTP starting in December trusted origins only! •
20. PSA: ECDSA ❖ Elliptic Curve Digital Signature Algorithm (ECDSA) ➢ Reduces call setup crypto overhead from 1000 -> 50ms ➢ More secure than RSA-1024 ❖ Step 1: Opt-in via API (Chrome 47) ➢ pc.generateCertificate({name: "ECDSA", namedCurve: "P- 256"}) ❖ Step 2: Default (Chrome 48 - 49) ➢ Test your system now
21. PSA: DTLS 1.2 ❖ DTLS 1.2 ➢ Rolling out in Chrome 46-47 timeframe ➢ Should be mostly seamless; falls back to 1.0 ➢ Allows more hardware offload of encryption ➢ Try it with your app: --enable-webrtc-dtls12
22. PSA: test.webrtc.org Open source tool that tests: ❖ Network conditions (bw, firewalls, etc…) ❖ If microphone is connected and working properly ❖ If camera is connected and working properly Please contribute: github.com/webrtc/testrtc
23. PSA: Report it! Simple Serge / Justin rule: "If you did not report it, it did not happen." http://www.webrtc.org/report-bug •
24. Enhancements
25. Enhancements: Delay Agnostic - AEC ❖ 99% on for Windows ❖ 99% on for Linux ❖ 99% on for ChromeOS ❖ Mac next week, after encouraging 1% experiment ❖ App control via googDAEchoCancellation constraint •
26. Enhancements: AEC rollout troubles Rise in full echo reports in June ❖ Caused by condition where interface between the render and browser processes causes audio duplication ❖ Throws old AEC off (which means Mac right now) ❖ New DA-AEC adapts after 1-2 seconds ❖ Still investigating why duplicates happen ❖ Apps can control •
27. Enhancements: Screensharing ❖ Faster screen sharing in Chrome 45 ❖ Improved handling of complex scene switches ❖ Better support for scrolling and spreadsheet borders ❖ More improvements waiting in 46 around BWE •
29. Enhancements: Mobile performance ❖ 5x faster rendering code ➢ Encoding and rendering local video from textures ❖ Capture performance improvements ➢ No more frame copies. No more conversions. ❖ Battery and CPU usage ➢ 75% reduction in GPU usage ➢ ~50% reduction power consumption •
30. Enhancements: Mobile power savings •
31. Enhancements: Mobile audio ❖ New Android audio stack ➢ 40% reduction in loopback latency on devices that support Open SL ❖ Major improvements to iOS audio ➢ Early tests show round-trip latency reduced from ~90ms to ~60ms ➢ Complexity (CPU) has also been reduced (no figures yet). ➢ Many existing crash issues have been resolved •
32. Enhancements: Mobile video ❖ VP8 + H.264 HW accel on Android ➢ For Android 5.0+ ➢ Works best on Qualcomm chipsets ❖ H.264 HW accel on iOS ➢ For iOS 8+ ❖ Works well; further optimizations coming ❖ Track it: bugs.webrtc.org, issue 4081
35. Enhancements: Chrome on desktop ❖ New JS API for selecting audio output devices ➢ HTMLMediaElement.setSinkId() ➢ Currently behind flag; track it: crbug.com/438023 ❖ Updated API for device enumeration ➢ MediaDevices.enumerateDevices() ❖ Event-driven datachannel buffering ➢ DataChannel.onbufferedamountlow ❖ IPv6
37. Around the corner: iOS API revamp ❖ v2 API underway ➢ Adjusts API to mimic JS API more closely ➢ Many cosmetic changes to conform to ObjC guidelines ➢ Callbacks will come on main event queue ➢ Release via CocoaPods ❖ v2 API will be side-by-side with current API ➢ For some finite transition period...
38. Around the corner: VP9 ❖ On (no flag) in M47, with standard packetization ❖ Initial rollout will be single layer ➢ "40% fewer bits at 15% more CPU" ❖ Working with Vidyo on SVC support ➢ Temporal + Spatial with low overhead ➢ Still need to figure out APIs to control this ➢ Initial SVC rollout using flag
39. Around the corner: H.264 in Chrome ❖ Working in limited scenarios on mobile and desktop ❖ Initial Chrome release will be software codec* ❖ Will interop with WebRTC standalone on Android/iOS ❖ M48 is a good guess (beta by EOY 2015) ❖ Track it: crbug.com/500605
40. Around the corner: MediaStreamRecorder Top request on chromium.org (over 2500 stars) ❖ Initial focus on recording local MediaStreams ❖ Goal is to have it working behind a flag in Chrome 47 ➢ Your help in testing will be crucial ➢ Would love to hear about your use cases ❖ Track it: crbug.com/262211
41. Around the corner: Misc ❖ Faster ICE, TURN setup ❖ Instant Wifi/Cell handoff ❖ More efficient radio management
42. Icebox ❖ Remote MediaStream processing in Web Audio ➢ crbug.com/121673 ❖ Unified Plan ➢ crbug.com/465349 ❖ WebRTC in Chrome for iOS ➢ crbug.com/416226
43. WebRTC/ORTC (1/2) ❖ WebRTC 1.0 targeting Last Call* by EOY2015 ➢ Incorporates many ORTC objects (RtpSender/Receiver, Ice/Dtls/SctpTransport) ❖ New objects provide more direct control: ➢ Switch cameras/tracks/codecs on the fly ➢ Set maximum bitrate for an encoding ➢ Configure simulcast (hopefully) ❖ Coming to Chrome in 2016 (Maybe a little in 2015)
44. WebRTC/ORTC (2/2) ❖ After WebRTC 1.0? WebRTC NV ❖ WebRTC NV fully converges ORTC with WebRTC 1.0 ❖ Apps can program to 1.0 (high-level) or object APIs ➢ Complete bypass of SDP, if desired
46. That’s all folks