If you need to add a new video to an existing collection, or want to remove a video from a collection, go into the "Video Manager" from the left side-panel of the collection editor.
Once in the Video Manager page, you can associate new videos with the collection by selecting them in the main view. You can also remove videos from the collection by deselecting them in the main view or clicking the 'X' below the video's thumbnail on the right side-panel.
Once you have added or removed the desired videos, click "Save" in the button right.
If you attempt to remove a video from a collection that has buttons referencing it, you will receive a warning that requires you to approve.