Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Media Recording: Fix Canvas audio/controls #12316

Merged
merged 1 commit into from
Sep 15, 2022
Merged

Media Recording: Fix Canvas audio/controls #12316

merged 1 commit into from
Sep 15, 2022

Conversation

merapi
Copy link
Contributor

@merapi merapi commented Sep 14, 2022 •
edited by swissspidy
Loading

Context

Summary

Relevant Technical Choices

To-do

User-facing changes

Testing Instructions

  • This is a non-user-facing change and requires no QA

This PR can be tested by following these steps:

  1. Record a new video with background blurring
  2. Stop recording
  3. Verify there's audio
  4. Verify there's a GIF toggle and a play/pause button
  5. Insert video
  6. Verify audio is still there

Reviews

Does this PR have a security-related impact?

No

Does this PR change what data or activity we track or use?

No

Does this PR have a legal-related impact?

No

Checklist

  • This PR addresses an existing issue and I have linked this PR to it in ZenHub
  • I have tested this code to the best of my abilities
  • I have verified accessibility to the best of my abilities (docs)
  • I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
  • This code is covered by automated tests (unit, integration, and/or e2e) to verify it works as intended (docs)
  • I have added documentation where necessary
  • I have added a matching Type: XYZ label to the PR

Fixes #12315
Fixes #12314

@merapi merapi self-assigned this Sep 14, 2022
@googleforcreators-bot
Copy link
Collaborator

Plugin builds for ac06cad are ready 🛎️!

@github-actions
Copy link
Contributor

Size Change: +37 B (0%)

Total Size: 2.72 MB

ℹ️ View Unchanged
Filename Size Change
assets/css/carousel-view-rtl.css 702 B 0 B
assets/css/carousel-view.css 701 B 0 B
assets/css/web-stories-block-rtl.css 4.52 kB 0 B
assets/css/web-stories-block.css 4.56 kB 0 B
assets/css/web-stories-embed-rtl.css 318 B 0 B
assets/css/web-stories-embed.css 317 B 0 B
assets/css/web-stories-list-styles-rtl.css 2.36 kB 0 B
assets/css/web-stories-list-styles.css 2.39 kB 0 B
assets/css/web-stories-theme-style-twentyeleven-rtl.css 102 B 0 B
assets/css/web-stories-theme-style-twentyeleven.css 102 B 0 B
assets/css/web-stories-theme-style-twentyfifteen-rtl.css 251 B 0 B
assets/css/web-stories-theme-style-twentyfifteen.css 251 B 0 B
assets/css/web-stories-theme-style-twentyfourteen-rtl.css 287 B 0 B
assets/css/web-stories-theme-style-twentyfourteen.css 287 B 0 B
assets/css/web-stories-theme-style-twentyseventeen-rtl.css 288 B 0 B
assets/css/web-stories-theme-style-twentyseventeen.css 288 B 0 B
assets/css/web-stories-theme-style-twentysixteen-rtl.css 224 B 0 B
assets/css/web-stories-theme-style-twentysixteen.css 224 B 0 B
assets/css/web-stories-theme-style-twentyten-rtl.css 143 B 0 B
assets/css/web-stories-theme-style-twentyten.css 143 B 0 B
assets/css/web-stories-theme-style-twentytwelve-rtl.css 256 B 0 B
assets/css/web-stories-theme-style-twentytwelve.css 256 B 0 B
assets/css/web-stories-theme-style-twentytwenty-rtl.css 86 B 0 B
assets/css/web-stories-theme-style-twentytwenty.css 86 B 0 B
assets/css/web-stories-theme-style-twentytwentyone-rtl.css 326 B 0 B
assets/css/web-stories-theme-style-twentytwentyone.css 326 B 0 B
assets/css/web-stories-widget-rtl.css 482 B 0 B
assets/css/web-stories-widget.css 482 B 0 B
assets/css/wp-dashboard-rtl.css 657 B 0 B
assets/css/wp-dashboard.css 659 B 0 B
assets/css/wp-story-editor-rtl.css 737 B 0 B
assets/css/wp-story-editor.css 738 B 0 B
assets/js/1512.js 6.15 kB 0 B
assets/js/1814.js 7.45 kB 0 B
assets/js/2388.js 85.3 kB 0 B
assets/js/4319.js 34.9 kB 0 B
assets/js/4422.js 49.3 kB 0 B
assets/js/4484.js 84.7 kB 0 B
assets/js/5051.js 92.1 kB 0 B
assets/js/6753.js 113 kB 0 B
assets/js/9796.js 3.71 kB 0 B
assets/js/carousel-view.js 3.41 kB 0 B
assets/js/chunk-colorthief.js 2.64 kB 0 B
assets/js/chunk-ffmpeg.js 5.89 kB 0 B
assets/js/chunk-focus-visible.js 1.01 kB 0 B
assets/js/chunk-getStoryMarkup.js 12.4 kB 0 B
assets/js/chunk-html-to-image.js 4.5 kB 0 B
assets/js/chunk-opentype.js 96 B 0 B
assets/js/chunk-react-calendar.js 12.5 kB 0 B
assets/js/chunk-react-color.js 44.3 kB 0 B
assets/js/chunk-web-animations-js.js 14.6 kB 0 B
assets/js/chunk-web-stories-template-0-metaData.js 545 B 0 B
assets/js/chunk-web-stories-template-0.js 11.4 kB 0 B
assets/js/chunk-web-stories-template-1-metaData.js 539 B 0 B
assets/js/chunk-web-stories-template-1.js 9.61 kB 0 B
assets/js/chunk-web-stories-template-10-metaData.js 533 B 0 B
assets/js/chunk-web-stories-template-10.js 7.37 kB 0 B
assets/js/chunk-web-stories-template-11-metaData.js 539 B 0 B
assets/js/chunk-web-stories-template-11.js 9.09 kB 0 B
assets/js/chunk-web-stories-template-12-metaData.js 497 B 0 B
assets/js/chunk-web-stories-template-12.js 9.7 kB 0 B
assets/js/chunk-web-stories-template-13-metaData.js 525 B 0 B
assets/js/chunk-web-stories-template-13.js 7.4 kB 0 B
assets/js/chunk-web-stories-template-14-metaData.js 582 B 0 B
assets/js/chunk-web-stories-template-14.js 7.37 kB 0 B
assets/js/chunk-web-stories-template-15-metaData.js 544 B 0 B
assets/js/chunk-web-stories-template-15.js 9 kB 0 B
assets/js/chunk-web-stories-template-16-metaData.js 588 B 0 B
assets/js/chunk-web-stories-template-16.js 10.9 kB 0 B
assets/js/chunk-web-stories-template-17-metaData.js 539 B 0 B
assets/js/chunk-web-stories-template-17.js 9.2 kB 0 B
assets/js/chunk-web-stories-template-18-metaData.js 585 B 0 B
assets/js/chunk-web-stories-template-18.js 9.91 kB 0 B
assets/js/chunk-web-stories-template-19-metaData.js 501 B 0 B
assets/js/chunk-web-stories-template-19.js 10.8 kB 0 B
assets/js/chunk-web-stories-template-2-metaData.js 586 B 0 B
assets/js/chunk-web-stories-template-2.js 9.3 kB 0 B
assets/js/chunk-web-stories-template-20-metaData.js 548 B 0 B
assets/js/chunk-web-stories-template-20.js 9.01 kB 0 B
assets/js/chunk-web-stories-template-21-metaData.js 535 B 0 B
assets/js/chunk-web-stories-template-21.js 9.85 kB 0 B
assets/js/chunk-web-stories-template-22-metaData.js 525 B 0 B
assets/js/chunk-web-stories-template-22.js 7.83 kB 0 B
assets/js/chunk-web-stories-template-23-metaData.js 605 B 0 B
assets/js/chunk-web-stories-template-23.js 7.48 kB 0 B
assets/js/chunk-web-stories-template-24-metaData.js 516 B 0 B
assets/js/chunk-web-stories-template-24.js 11.7 kB 0 B
assets/js/chunk-web-stories-template-25-metaData.js 544 B 0 B
assets/js/chunk-web-stories-template-25.js 7.06 kB 0 B
assets/js/chunk-web-stories-template-26-metaData.js 601 B 0 B
assets/js/chunk-web-stories-template-26.js 7.27 kB 0 B
assets/js/chunk-web-stories-template-27-metaData.js 542 B 0 B
assets/js/chunk-web-stories-template-27.js 7.82 kB 0 B
assets/js/chunk-web-stories-template-28-metaData.js 532 B 0 B
assets/js/chunk-web-stories-template-28.js 9.07 kB 0 B
assets/js/chunk-web-stories-template-29-metaData.js 562 B 0 B
assets/js/chunk-web-stories-template-29.js 9.25 kB 0 B
assets/js/chunk-web-stories-template-3-metaData.js 539 B 0 B
assets/js/chunk-web-stories-template-3.js 8.42 kB 0 B
assets/js/chunk-web-stories-template-30-metaData.js 576 B 0 B
assets/js/chunk-web-stories-template-30.js 7.89 kB 0 B
assets/js/chunk-web-stories-template-31-metaData.js 502 B 0 B
assets/js/chunk-web-stories-template-31.js 10.3 kB 0 B
assets/js/chunk-web-stories-template-32-metaData.js 552 B 0 B
assets/js/chunk-web-stories-template-32.js 13.3 kB 0 B
assets/js/chunk-web-stories-template-33-metaData.js 492 B 0 B
assets/js/chunk-web-stories-template-33.js 9.07 kB 0 B
assets/js/chunk-web-stories-template-34-metaData.js 571 B 0 B
assets/js/chunk-web-stories-template-34.js 7.58 kB 0 B
assets/js/chunk-web-stories-template-35-metaData.js 566 B 0 B
assets/js/chunk-web-stories-template-35.js 8.91 kB 0 B
assets/js/chunk-web-stories-template-36-metaData.js 577 B 0 B
assets/js/chunk-web-stories-template-36.js 12.7 kB 0 B
assets/js/chunk-web-stories-template-37-metaData.js 528 B 0 B
assets/js/chunk-web-stories-template-37.js 6.71 kB 0 B
assets/js/chunk-web-stories-template-38-metaData.js 572 B 0 B
assets/js/chunk-web-stories-template-38.js 7.94 kB 0 B
assets/js/chunk-web-stories-template-39-metaData.js 589 B 0 B
assets/js/chunk-web-stories-template-39.js 8.08 kB 0 B
assets/js/chunk-web-stories-template-4-metaData.js 564 B 0 B
assets/js/chunk-web-stories-template-4.js 12.7 kB 0 B
assets/js/chunk-web-stories-template-40-metaData.js 557 B 0 B
assets/js/chunk-web-stories-template-40.js 10.2 kB 0 B
assets/js/chunk-web-stories-template-41-metaData.js 572 B 0 B
assets/js/chunk-web-stories-template-41.js 7.75 kB 0 B
assets/js/chunk-web-stories-template-42-metaData.js 521 B 0 B
assets/js/chunk-web-stories-template-42.js 7 kB 0 B
assets/js/chunk-web-stories-template-43-metaData.js 557 B 0 B
assets/js/chunk-web-stories-template-43.js 8.76 kB 0 B
assets/js/chunk-web-stories-template-44-metaData.js 584 B 0 B
assets/js/chunk-web-stories-template-44.js 11.1 kB 0 B
assets/js/chunk-web-stories-template-45-metaData.js 565 B 0 B
assets/js/chunk-web-stories-template-45.js 7.52 kB 0 B
assets/js/chunk-web-stories-template-46-metaData.js 531 B 0 B
assets/js/chunk-web-stories-template-46.js 5.22 kB 0 B
assets/js/chunk-web-stories-template-47-metaData.js 592 B 0 B
assets/js/chunk-web-stories-template-47.js 9.42 kB 0 B
assets/js/chunk-web-stories-template-48-metaData.js 555 B 0 B
assets/js/chunk-web-stories-template-48.js 9.09 kB 0 B
assets/js/chunk-web-stories-template-49-metaData.js 518 B 0 B
assets/js/chunk-web-stories-template-49.js 9.69 kB 0 B
assets/js/chunk-web-stories-template-5-metaData.js 556 B 0 B
assets/js/chunk-web-stories-template-5.js 9.94 kB 0 B
assets/js/chunk-web-stories-template-50-metaData.js 503 B 0 B
assets/js/chunk-web-stories-template-50.js 9.15 kB 0 B
assets/js/chunk-web-stories-template-51-metaData.js 526 B 0 B
assets/js/chunk-web-stories-template-51.js 10.4 kB 0 B
assets/js/chunk-web-stories-template-52-metaData.js 602 B 0 B
assets/js/chunk-web-stories-template-52.js 10.4 kB 0 B
assets/js/chunk-web-stories-template-53-metaData.js 553 B 0 B
assets/js/chunk-web-stories-template-53.js 5.78 kB 0 B
assets/js/chunk-web-stories-template-54-metaData.js 547 B 0 B
assets/js/chunk-web-stories-template-54.js 7.67 kB 0 B
assets/js/chunk-web-stories-template-55-metaData.js 573 B 0 B
assets/js/chunk-web-stories-template-55.js 7.13 kB 0 B
assets/js/chunk-web-stories-template-56-metaData.js 542 B 0 B
assets/js/chunk-web-stories-template-56.js 9.87 kB 0 B
assets/js/chunk-web-stories-template-57-metaData.js 527 B 0 B
assets/js/chunk-web-stories-template-57.js 14.9 kB 0 B
assets/js/chunk-web-stories-template-58-metaData.js 555 B 0 B
assets/js/chunk-web-stories-template-58.js 5.74 kB 0 B
assets/js/chunk-web-stories-template-59-metaData.js 590 B 0 B
assets/js/chunk-web-stories-template-59.js 8.96 kB 0 B
assets/js/chunk-web-stories-template-6-metaData.js 569 B 0 B
assets/js/chunk-web-stories-template-6.js 7.07 kB 0 B
assets/js/chunk-web-stories-template-60-metaData.js 510 B 0 B
assets/js/chunk-web-stories-template-60.js 9.51 kB 0 B
assets/js/chunk-web-stories-template-7-metaData.js 569 B 0 B
assets/js/chunk-web-stories-template-7.js 7.46 kB 0 B
assets/js/chunk-web-stories-template-8-metaData.js 569 B 0 B
assets/js/chunk-web-stories-template-8.js 8.93 kB 0 B
assets/js/chunk-web-stories-template-9-metaData.js 581 B 0 B
assets/js/chunk-web-stories-template-9.js 8.46 kB 0 B
assets/js/chunk-web-stories-templates.js 845 B 0 B
assets/js/chunk-web-stories-textset-0.js 5.06 kB 0 B
assets/js/chunk-web-stories-textset-1.js 6.65 kB 0 B
assets/js/chunk-web-stories-textset-2.js 7.65 kB 0 B
assets/js/chunk-web-stories-textset-3.js 15.1 kB 0 B
assets/js/chunk-web-stories-textset-4.js 4.15 kB 0 B
assets/js/chunk-web-stories-textset-5.js 5.47 kB 0 B
assets/js/chunk-web-stories-textset-6.js 5.28 kB 0 B
assets/js/chunk-web-stories-textset-7.js 10.2 kB 0 B
assets/js/generateBlurhash.worker.worker.js 1.1 kB 0 B
assets/js/imgareaselect.js 3.77 kB 0 B
assets/js/lightbox.js 550 B 0 B
assets/js/tinymce-button.js 2.85 kB 0 B
assets/js/web-stories-activation-notice.js 25.4 kB 0 B
assets/js/web-stories-block.js 18 kB 0 B
assets/js/web-stories-embed.js 20 B 0 B
assets/js/web-stories-widget.js 587 B 0 B
assets/js/wp-dashboard.js 64.1 kB 0 B
assets/js/wp-story-editor.js 1.36 MB +37 B (0%)

compressed-size-action

@merapi
Copy link
Contributor Author

merapi commented Sep 15, 2022

It happens quite often. I will re-run the tests a few more times.

[percy] Encountered an error uploading snapshot: CUJ: Text Sets (Text and Shape Combinations): Using Text Sets should allow user to drag and drop text set onto page: Text set dragged
[percy] Error: Cannot create snapshot, the build is already finalized. If this happens frequently in a parallel test suite, you may want to use a "finalize all" step in CI: https://docs.percy.io/docs/parallel-test-suites#using-finalize-all
[percy] Snapshot taken: CUJ: Text Sets (Text and Shape Combinations): Using Text Sets should allow user to drag and drop text set onto page: Text set added
[percy] Encountered an error uploading snapshot: CUJ: Text Sets (Text and Shape Combinations): Using Text Sets should allow user to drag and drop text set onto page: Text set added
[percy] Error: Cannot create snapshot, the build is already finalized. If this happens frequently in a parallel test suite, you may want to use a "finalize all" step in CI: https://docs.percy.io/docs/parallel-test-suites#using-finalize-all

<--- Last few GCs --->

[3792:0x3084003c4000]   328778 ms: Scavenge 645.6 (701.9) -> 641.4 (702.9) MB, 8.6 / 0.0 ms  (average mu = 0.985, current mu = 0.982) task; 
[3792:0x3084003c4000]   329275 ms: Scavenge 648.1 (703.7) -> 643.2 (705.4) MB, 9.8 / 0.0 ms  (average mu = 0.985, current mu = 0.982) allocation failure; 
[3792:0x3084003c4000]   329486 ms: Scavenge 650.3 (705.9) -> 644.9 (707.7) MB, 8.9 / 0.0 ms  (average mu = 0.985, current mu = 0.982) allocation failure; 


<--- JS stacktrace --->

[0915/084254.983903:ERROR:v8_initializer.cc(730)] V8 process OOM (Oilpan: Reserving memory.).
[0915/084255.189840:WARNING:sync_reader.cc(298)] SyncReader::Read timed out, audio glitch count=10
[0915/084255.199951:WARNING:sync_reader.cc(298)] SyncReader::Read timed out, audio glitch count=10
[0915/084255.338130:WARNING:sync_reader.cc(273)] ASR: No room in socket buffer.: Broken pipe (32)
[0915/084255.338682:WARNING:sync_reader.cc(273)] ASR: No room in socket buffer.: Broken pipe (32)
.....................15 09 2022 08:43:05.367:WARN [Chrome Headless 106.0.5249.0 (Linux x86_64)]: Disconnected (0 times) reconnect failed before timeout of 10000ms (transport close)

Chrome Headless 106.0.5249.0 (Linux x86_64) ERROR
  Disconnected reconnect failed before timeout of 10000ms (transport close)
Chrome Headless 106.0.5249.0 (Linux x86_64) ERROR
  Disconnected reconnect failed before timeout of 10000ms (transport close)
Chrome Headless 106.0.5249.0 (Linux x86_64): Executed 115 of 159 (skipped 5) DISCONNECTED (5 mins 35.348 secs / 5 mins 19.691 secs)
[percy] Error: Can only finalize pending builds
Error: Process completed with exit code 1.

@kkalarickal
Copy link

Status: Bug fix is confirmed

Test Post: https://stories-qa-wordpress-amp.pantheonsite.io/fix-canvas-audio-controls12316/

Scenarios:

  • used video recorder to capture web cam stream + audio, with background blurred
  • embedded captured video into a web-story as background and inside a rectangular box
  • saved draft & previewed the video embed
  • published web-story and added to the wordpress post

Permutations:

  • converted captured video into GIF (without audio)
  • toggled blurred/un-blurred background
  • tried different resolutions for webcam stream input
  • applied animation, captioning, trimming, background audio, cta link
  • verified on desktop/mobile/tablet viewports

Obervations:
@swissspidy - I noticed some instability unrelated to this bug fix. These were relate to:

  1. Trimming was taking a long time. While the video was trimmed in the end, as a user experience, it did not really work for me.
  2. When I first did a preview of the story, the three devices that were chosen by default did not show the preview immediately. I had to remove and re-add the devices to get the preview to display.

@swissspidy
Copy link
Collaborator

Ack. Thanks for testing!

Trimming was taking a long time. While the video was trimmed in the end, as a user experience, it did not really work for me.

Yes, depending on the device and length of video it can take a while.

When I first did a preview of the story, the three devices that were chosen by default did not show the preview immediately. I had to remove and re-add the devices to get the preview to display.

The story preview is handled by AMP, so if that is not working reliably, you would need to report that upstream.

@swissspidy swissspidy changed the title Fix Canvas audio/controls Media Recording: Fix Canvas audio/controls Sep 15, 2022
@swissspidy swissspidy merged commit a049d01 into main Sep 15, 2022
@swissspidy swissspidy deleted the fix/canvas-audio branch September 15, 2022 20:07
swissspidy pushed a commit that referenced this pull request Sep 19, 2022
@merapi @swissspidy
Media Recording: Fix Canvas audio/controls (#12316)
6243849
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@swissspidy swissspidy swissspidy approved these changes

Assignees

@merapi merapi

Labels
Group: Media Type: Bug Something isn't working
Projects
None yet
Milestone
No milestone
4 participants
@merapi @googleforcreators-bot @kkalarickal @swissspidy